Original: Leveraging Inter-Layer Dependency for Post -Training Quantization
Revised: Leveraging Inter-Layer Dependency for Post -Training Quantization

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Prior works on Post-training Quantization (PTQ) typically separate a neural network into sub-nets and quantize them sequentially.
Revised: Prior works on Post-training Quantization (PTQ) typically separate a neural network into sub-nets and quantize them sequentially.

Original: This process pays little attentionto the dependency across the sub-nets, hence is less optimal.
Revised: This process pays little attentionto the dependency across the sub-nets, hence is less optimal.

Original: In this paper, wepropose a novel Network-Wise Quantization (NWQ) approach to fully leveraginginter-layer dependency.
Revised: In this paper, wepropose a novel Network-Wise Quantization (NWQ) approach to fully leveraginginter-layer dependency.

Original: NWQ faces a larger scale combinatorial optimization problem of discrete variables than in previous works, which raises two major challenges:over-fitting and discrete optimization problem.
Revised: NWQ faces a larger scale combinatorial optimization problem of discrete variables than in previous works, which raises two major challenges:over-fitting and discrete optimization problem.

Original: NWQ alleviates over fitting viaa Activation Regularization (AR) technique, which better controls the activationdistribution.
Revised: NWQ alleviates over fitting viaa Activation Regularization (AR) technique, which better controls the activationdistribution.

Original: To optimize discrete variables, NWQ introduces Annealing Softmax(ASoftmax) and Annealing Mixup (AMixup) to progressively transition quantizedweights and activations from continuity to discretization, respectively.
Revised: To optimize discrete variables, NWQ introduces Annealing Softmax(ASoftmax) and Annealing Mixup (AMixup) to progressively transition quantizedweights and activations from continuity to discretization, respectively.

Original: Extensiveexperiments demonstrates that NWQ outperforms prior state-of-the-art approachesby a large margin: 20.24% for the challenging configuration of MobileNetV2 with2 bits on ImageNet, pushing extremely low-bit PTQ from feasibility to usability.
Revised: Extensiveexperiments demonstrates that NWQ outperforms prior state-of-the-art approachesby a large margin: 20.24% for the challenging configuration of MobileNetV2 with2 bits on ImageNet, pushing extremely low-bit PTQ from feasibility to usability.

Original: In addition, NWQ is able to achieve competitive or better results with only 10%computation cost of previous works.
Revised: In addition, NWQ is able to achieve competitive or better results with only 10%computation cost of previous works.

Original:  Introduction
Revised:  Introduction

Original: Deep learning has seen a boom of architectures with ever-increasing capabilities and capacity.
Revised: Deep learning has seen a boom of architectures with ever-increasing capabilities and capacity.

Original: Hugecapacity, however, always comes with boosting parameters, resulting in large computation costand memory footprint.
Revised: Hugecapacity, however, always comes with boosting parameters, resulting in large computation costand memory footprint.

Original: Researchers have utilized strategies like neural architecture search (NAS)[4, 11, 29, 49], network pruning [12, 19] and knowledge distillation (KD)
Revised: Researchers have utilized strategies like neural architecture search (NAS)[4, 11, 29, 49], network pruning [12, 19] and knowledge distillation (KD)

Original: [18, 21] to design compactnetworks and improve their accuracy.
Revised: [18, 21] to design compactnetworks and improve their accuracy.

Original: Furthermore, neural network quantization [7, 15, 22, 25, 28]is typically applied along with the methods mentioned above to reduce the computation cost andmemory footprint.
Revised: Furthermore, neural network quantization [7, 15, 22, 25, 28]is typically applied along with the methods mentioned above to reduce the computation cost andmemory footprint.

Original: Quantization represents neural networks with lower bit precision, usually 8 bits inpractice and saves 75% of memory while offering 2 ∼ 4 × speedup[26].
Revised: Quantization represents neural networks with lower bit precision, usually 8 bits inpractice and saves 75% of memory while offering 2 ∼ 4 × speedup[26].

Original: Network quantization techniques are typically categorized into Quantization Aware Training (QAT)and Post-training Quantization(PTQ) based on the dependency on labeled data and the necessity ofweight tuning.
Revised: Network quantization techniques are typically categorized into Quantization Aware Training (QAT)and Post-training Quantization(PTQ) based on the dependency on labeled data and the necessity ofweight tuning.

Original: Generally, PTQ techniques are preferred in industry because they rely less on labeleddata and take into account privacy concerns as well as compute resources.
Revised: Generally, PTQ techniques are preferred in industry because they rely less on labeleddata and take into account privacy concerns as well as compute resources.

Original: This paper focuses on
Revised: This paper focuses on

Original: PTQ.
Revised: PTQ.

Original: PTQ usually suffers from significant accuracy degradation, especially for low-bit quantization whichoffers much more memory reduction and speedup.
Revised: PTQ usually suffers from significant accuracy degradation, especially for low-bit quantization whichoffers much more memory reduction and speedup.

Original: To improve the accuracy of PTQ, much efforthas been devoted [5, 32, 34, 35, 39].
Revised: To improve the accuracy of PTQ, much efforthas been devoted [5, 32, 34, 35, 39].

Original: AdaRound[32] has recently discovered that round-to-nearestappears to be optimal for single weight but not for the entire layer.
Revised: AdaRound[32] has recently discovered that round-to-nearestappears to be optimal for single weight but not for the entire layer.

Original: The rounding errors of weights ina single layer have the capability of compensating for each other, contributing to a smaller layer-wise layerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (a) layer-wise layerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (b) block-wise layerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (c) network-wise quantization error.
Revised: The rounding errors of weights ina single layer have the capability of compensating for each other, contributing to a smaller layer-wise layerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (a) layer-wise layerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (b) block-wise layerlayerlayer N … blockblockblock N … layerlayerlayermodel layerlayerlayer N … (c) network-wise quantization error.

Original: We term this property as intra-layer dependency .
Revised: We term this property as intra-layer dependency .

Original: Motivated by this property,
Revised: Motivated by this property,

Original: AdaRound[32] proposes to learn the round policy layer by layer to improve accuracy.
Revised: AdaRound[32] proposes to learn the round policy layer by layer to improve accuracy.

Original: Inspired by AdaRound[32], we hypothesize that the quantization errors of all layers in a network areable to compensate for each other, resulting in a reduced quantization error.
Revised: Inspired by AdaRound[32], we hypothesize that the quantization errors of all layers in a network areable to compensate for each other, resulting in a reduced quantization error.

Original: We refer to this capabilityas inter-layer dependency .
Revised: We refer to this capabilityas inter-layer dependency .

Original: However, AdaRound[32] assumes that layers are mutual-independentand derives a layer-diagonal Hessian, leading to a layer-wise solution(Figure 1a).
Revised: However, AdaRound[32] assumes that layers are mutual-independentand derives a layer-diagonal Hessian, leading to a layer-wise solution(Figure 1a).

Original: BRECQ[27]points out that the dependency should not be ignored and the assumption of AdaRound[32] is notaccurate, they instead utilize the inter-layer dependency within each block to build a block-wiseapproach(Figure 1b).
Revised: BRECQ[27]points out that the dependency should not be ignored and the assumption of AdaRound[32] is notaccurate, they instead utilize the inter-layer dependency within each block to build a block-wiseapproach(Figure 1b).

Original: However, BRECQ[27] assumes that blocks are mutual-independent.
Revised: However, BRECQ[27] assumes that blocks are mutual-independent.

Original: Weargue that neither the approximations of AdaRound nor that of BRECQ are accurate enough, theinter-layer dependency should be leveraged in a network-wise manner.
Revised: Weargue that neither the approximations of AdaRound nor that of BRECQ are accurate enough, theinter-layer dependency should be leveraged in a network-wise manner.

Original: To achieve this, we propose a
Revised: To achieve this, we propose a

Original: Network-Wise Quantization (NWQ, Figure 1c) scheme to fully leverage the inter-layer dependency.
Revised: Network-Wise Quantization (NWQ, Figure 1c) scheme to fully leverage the inter-layer dependency.

Original: AdaRound and BRECQ separate a network into sub-nets and train them sequentially, this processmakes the preceding layers unaware of succeeding layers.
Revised: AdaRound and BRECQ separate a network into sub-nets and train them sequentially, this processmakes the preceding layers unaware of succeeding layers.

Original: NWQ solves this problem via trainingthe entire network end-to-end, where layers are aware of each other and they are able to cooperatefor a more optimal solution.
Revised: NWQ solves this problem via trainingthe entire network end-to-end, where layers are aware of each other and they are able to cooperatefor a more optimal solution.

Original: However, quantizing the entire network end-to-end is a combinatorialoptimization problem where huge number of discrete parameters from all layers are combinedtogether for optimization.
Revised: However, quantizing the entire network end-to-end is a combinatorialoptimization problem where huge number of discrete parameters from all layers are combinedtogether for optimization.

Original: This optimization problem will raise two major challenges that result insignificant accuracy degradation.
Revised: This optimization problem will raise two major challenges that result insignificant accuracy degradation.

Original: The first challenge is over-fitting.
Revised: The first challenge is over-fitting.

Original: Extending the reconstruction unit from layer/block to entire networkwill increase parameters dramatically and thus raises the risk of over-fitting over a typically smallcalibration set.
Revised: Extending the reconstruction unit from layer/block to entire networkwill increase parameters dramatically and thus raises the risk of over-fitting over a typically smallcalibration set.

Original: To alleviate this issue, NWQ applies Activation Regularization (AR), a variant of
Revised: To alleviate this issue, NWQ applies Activation Regularization (AR), a variant of

Original: L2 regularization, on intermediate representation to keep the quantized activations as close to thefloating-point counterpart as possible.
Revised: L2 regularization, on intermediate representation to keep the quantized activations as close to thefloating-point counterpart as possible.

Original: The benefits of AR are two folds.
Revised: The benefits of AR are two folds.

Original: First, AR regularizesquantized activation flow to prevent over-fitting.
Revised: First, AR regularizesquantized activation flow to prevent over-fitting.

Original: Second, it works as a quantization loss function toencourage minimal intermediate quantization error.
Revised: Second, it works as a quantization loss function toencourage minimal intermediate quantization error.

Original: AR turns out to be both simple and effective.
Revised: AR turns out to be both simple and effective.

Original: Another challenge is discrete optimization.
Revised: Another challenge is discrete optimization.

Original: Based on AdaRound, NWQ learns the weight roundpolicy and activation quantization parameters, both of which involve in discrete variables.
Revised: Based on AdaRound, NWQ learns the weight roundpolicy and activation quantization parameters, both of which involve in discrete variables.

Original: However,the substantially increased depth and discrete variables make it more difficult to converge comparedto layer-wise[23, 32] and block-wise[27, 40] approach.
Revised: However,the substantially increased depth and discrete variables make it more difficult to converge comparedto layer-wise[23, 32] and block-wise[27, 40] approach.

Original: The insufficient convergence will leave thediscrete variables (weight round policy) in a state of continuity rather than discretization, leadingto great performance drop.
Revised: The insufficient convergence will leave thediscrete variables (weight round policy) in a state of continuity rather than discretization, leadingto great performance drop.

Original: To solve this problem, we propose Annealing Sofmtax (ASoftmax)and Annealing Mixup (AMixup) to better optimize discrete weights and activations.
Revised: To solve this problem, we propose Annealing Sofmtax (ASoftmax)and Annealing Mixup (AMixup) to better optimize discrete weights and activations.

Original: Specifically,
Revised: Specifically,

Original: ASoftmax provides a mechanism to guarantee the discritization of round policy learning via annealingtemperature.
Revised: ASoftmax provides a mechanism to guarantee the discritization of round policy learning via annealingtemperature.

Original: AMixup resolves the train-test inconsistency problem indtroduced by QDROP[40] viagradually decreasing the floating-point percentage in mixed activations.
Revised: AMixup resolves the train-test inconsistency problem indtroduced by QDROP[40] viagradually decreasing the floating-point percentage in mixed activations.

Original: Equipped with AR, ASoftmax and AMixup, we are able to build our NWQ framework that fullyleverages inter-layer dependency to achieve higher accuracy than existing works.
Revised: Equipped with AR, ASoftmax and AMixup, we are able to build our NWQ framework that fullyleverages inter-layer dependency to achieve higher accuracy than existing works.

Original: To summarize, this paper makes the following contributions:
Revised: To summarize, this paper makes the following contributions:

Original: We propose a novel network-wise quantization framework for PTQ to fully leverage interlayer dependency .
Revised: We propose a novel network-wise quantization framework for PTQ to fully leverage interlayer dependency .

Original: We analyze the combinatorial optimization problem introduced by network-wise quan-
Revised: We analyze the combinatorial optimization problem introduced by network-wise quan-

Original: tization and propose AR, ASoftmax and AMixup to solve the over-fitting and discreteoptimization problem.
Revised: tization and propose AR, ASoftmax and AMixup to solve the over-fitting and discreteoptimization problem.

Original: We establish a new state-of-the-art through significantly improving PTQ by a large margin:20.24% for MobileNetV2 with 2 bits and thus push the limit of extremely low-bit PTQ fromfeasibility to usability.
Revised: We establish a new state-of-the-art through significantly improving PTQ by a large margin:20.24% for MobileNetV2 with 2 bits and thus push the limit of extremely low-bit PTQ fromfeasibility to usability.

Original:  Related Work
Revised:  Related Work

Original: Network quantization is commonly divided into QAT [7, 15, 22, 25, 48] and PTQ [27, 32, 33, 47].
Revised: Network quantization is commonly divided into QAT [7, 15, 22, 25, 48] and PTQ [27, 32, 33, 47].

Original: QAT methods tend to minimize task error by training weights and quantization parameters jointlywith the entire labeled dataset and thus require a large number of computing resources, whereas PTQmethods focus on finding optimal quantization parameters for off-the-shelf well-trained networks withmuch less data and compute resources than QAT.
Revised: QAT methods tend to minimize task error by training weights and quantization parameters jointlywith the entire labeled dataset and thus require a large number of computing resources, whereas PTQmethods focus on finding optimal quantization parameters for off-the-shelf well-trained networks withmuch less data and compute resources than QAT.

Original: PTQ can further be classified into statistic-based
Revised: PTQ can further be classified into statistic-based

Original: PTQ and learning-based PTQ .
Revised: PTQ and learning-based PTQ .

Original: Statistic-based PTQ methods [33, 35, 47] first estimate the distributionof activation on calibration data, and then find the optimal quantization parameters that minimizelocal quantization errors heuristically.
Revised: Statistic-based PTQ methods [33, 35, 47] first estimate the distributionof activation on calibration data, and then find the optimal quantization parameters that minimizelocal quantization errors heuristically.

Original: Learning-based PTQ methods[23, 27, 32, 40] dedicate toslightly weight tuning and quantization parameters learning.
Revised: Learning-based PTQ methods[23, 27, 32, 40] dedicate toslightly weight tuning and quantization parameters learning.

Original: Our work follows the learning-based
Revised: Our work follows the learning-based

Original: PTQ scheme.
Revised: PTQ scheme.

Original: Learning-based PTQ : AdaRound[32] adopts the regime of learning the optimal round policy layerby layer.
Revised: Learning-based PTQ : AdaRound[32] adopts the regime of learning the optimal round policy layerby layer.

Original: BRECQ [27] succeeds in extending AdaRound [32] to a block-wise reconstruction andconcludes that block-wise quantization is superior to network-wise method.
Revised: BRECQ [27] succeeds in extending AdaRound [32] to a block-wise reconstruction andconcludes that block-wise quantization is superior to network-wise method.

Original: We argure that bothlayer-wise and block-wise approach is sub-optimal because the preceding layers/blocks are unawareof succeeding layers/block, thus the dependency they leveraged is unidirectional.
Revised: We argure that bothlayer-wise and block-wise approach is sub-optimal because the preceding layers/blocks are unawareof succeeding layers/block, thus the dependency they leveraged is unidirectional.

Original: Some works focuson the sequence of weight and activation quantization.
Revised: Some works focuson the sequence of weight and activation quantization.

Original: AdaQuant [23] proposes to tune weights andactivation quantization parameters separately for each layer to reduce over-fitting, while QDROP[40] finds that incorporating activation quantization into weight tuning can achieve higher accuracy.
Revised: AdaQuant [23] proposes to tune weights andactivation quantization parameters separately for each layer to reduce over-fitting, while QDROP[40] finds that incorporating activation quantization into weight tuning can achieve higher accuracy.

Original: Activiation Regularization : Large values in the learned dense representation can be a sign of beingoverfit.
Revised: Activiation Regularization : Large values in the learned dense representation can be a sign of beingoverfit.

Original: To encourage small activations, [31] employs L2 regularization on hidden state of recurrentneural networks (RNN), while various normalizations [2, 24, 38, 42] focus on centralize the activationdistribution to regularize large values.
Revised: To encourage small activations, [31] employs L2 regularization on hidden state of recurrentneural networks (RNN), while various normalizations [2, 24, 38, 42] focus on centralize the activationdistribution to regularize large values.

Original: Some prior works attempt to learn sparse representation.
Revised: Some prior works attempt to learn sparse representation.

Original: Dropout and its variants [1, 10, 36, 37, 41] randomly drop activations from the neural network toencourage sparsity.
Revised: Dropout and its variants [1, 10, 36, 37, 41] randomly drop activations from the neural network toencourage sparsity.

Original: ReLUs[16, 20, 30, 43] dedicate to learning sparse representation via rectifyingnegative values.
Revised: ReLUs[16, 20, 30, 43] dedicate to learning sparse representation via rectifyingnegative values.

Original: These techniques aim to regularize activations when training a neural networkfrom scratch, which inapplicable for PTQ where the networks are off-the-shelf and well-trained,thus the activation distributions are deterministic and immutable.
Revised: These techniques aim to regularize activations when training a neural networkfrom scratch, which inapplicable for PTQ where the networks are off-the-shelf and well-trained,thus the activation distributions are deterministic and immutable.

Original: Instead of encouraging small andsparse activations, NWQ apply activation regularization on intermediate representation to encouragesmall offsets between quantized activations and the floating-point counterpart.
Revised: Instead of encouraging small andsparse activations, NWQ apply activation regularization on intermediate representation to encouragesmall offsets between quantized activations and the floating-point counterpart.

Original: AdaRound[32] and
Revised: AdaRound[32] and

Original: BRECQ [27] also employ quantization losses on intermediate activations but those losses works asquantization objectives rather than regularization, while the quantization objective of NWQ is tominimize the quantization error of the final network output and the regularization on intermediateactivations is to prevent optimization process from over-fitting.
Revised: BRECQ [27] also employ quantization losses on intermediate activations but those losses works asquantization objectives rather than regularization, while the quantization objective of NWQ is tominimize the quantization error of the final network output and the regularization on intermediateactivations is to prevent optimization process from over-fitting.

Original: Discrete Optimization for Quantization : Network quantization optimizes discrete variables viagradient backpropagation.
Revised: Discrete Optimization for Quantization : Network quantization optimizes discrete variables viagradient backpropagation.

Original: However, the quantization process introduces a round function which isindifferentiable.
Revised: However, the quantization process introduces a round function which isindifferentiable.

Original: Some works on QAT focus on approximating the gradient through the round function[3, 6, 13, 14, 17, 44].
Revised: Some works on QAT focus on approximating the gradient through the round function[3, 6, 13, 14, 17, 44].

Original: For PTQ, AdaRound [32] introduces additional discrete parameters(the binaryround policy) and optimize them via Rectified Sigmoid with an explicit Regularization (RSeR) lossfunction.
Revised: For PTQ, AdaRound [32] introduces additional discrete parameters(the binaryround policy) and optimize them via Rectified Sigmoid with an explicit Regularization (RSeR) lossfunction.

Original: However, RSeR is adversarial to quantization loss and thus difficult to convergence innetwork-wise quantization.
Revised: However, RSeR is adversarial to quantization loss and thus difficult to convergence innetwork-wise quantization.

Original: QDROP [40] proposes to randomly drop quantized activations to improvethe flatness of optimization landscape, but introcues train-test inconsistency problem which results inperformance degradation if used in network-wise PTQ.
Revised: QDROP [40] proposes to randomly drop quantized activations to improvethe flatness of optimization landscape, but introcues train-test inconsistency problem which results inperformance degradation if used in network-wise PTQ.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: … 𝐴𝑅
Revised: … 𝐴𝑅

Original: [Figure]
Revised: [Figure]

Original: 𝐿 :;<=> + … 𝐴𝑅 ?
Revised: 𝐿 :;<=> + … 𝐴𝑅 ?

Original: @ 𝑙𝑜𝑠𝑠
Revised: @ 𝑙𝑜𝑠𝑠

Original: [Figure]
Revised: [Figure]

Original: 𝐿 <A
Revised: 𝐿 <A

Original: +
Revised: +

Original: [Figure]
Revised: [Figure]

Original:  Approach
Revised:  Approach

Original: [Figure]
Revised: [Figure]

Original: Network-Wise Quantization (NWQ) is a simple end-to-end post-training quantization approach.
Revised: Network-Wise Quantization (NWQ) is a simple end-to-end post-training quantization approach.

Original: The structure of NWQ is illustrated in Figure 2.
Revised: The structure of NWQ is illustrated in Figure 2.

Original: The optimization objective is composed of outputquantization loss and layer-wise AR loss.
Revised: The optimization objective is composed of outputquantization loss and layer-wise AR loss.

Original: Meanwhile, the annealing approaches (Asoftmax and
Revised: Meanwhile, the annealing approaches (Asoftmax and

Original: AMixup) are introduced in training process, which enable the quantized weights and activations togently converge from continuous state to discrete state, lowering training complexity and improvingaccuracy.
Revised: AMixup) are introduced in training process, which enable the quantized weights and activations togently converge from continuous state to discrete state, lowering training complexity and improvingaccuracy.

Original: 3.1 Activation Regularization
Revised: 3.1 Activation Regularization

Original: NWQ leverages Activation Regularization (AR) to penalize quantized intermediate activations thatare substantially away from the floating-point counterparts, thereby encouraging the activations toremain unchanged.
Revised: NWQ leverages Activation Regularization (AR) to penalize quantized intermediate activations thatare substantially away from the floating-point counterparts, thereby encouraging the activations toremain unchanged.

Original: Formally, we define AR by:
Revised: Formally, we define AR by:

Original: [Equation]
Revised: [Equation]

Original: (1) while G represents floating-point network and G is the simulated quantized counterpart.
Revised: (1) while G represents floating-point network and G is the simulated quantized counterpart.

Original: a l and a ldenote the floating-point and the simulated quantized activation of the l -th layer, respectively.
Revised: a l and a ldenote the floating-point and the simulated quantized activation of the l -th layer, respectively.

Original: Nrepresents the number of total layers and ∥·∥ F denotes the Frobenius norm.
Revised: Nrepresents the number of total layers and ∥·∥ F denotes the Frobenius norm.

Original: Our network quantizationerror L quant ( a N , a N ) is defined by:
Revised: Our network quantizationerror L quant ( a N , a N ) is defined by:

Original: [Equation]
Revised: [Equation]

Original: then the final objective can be formulated as:
Revised: then the final objective can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: Practically, L ar is not restricted to be utilized layer by layer, it can also be performed block byblock and stage by stage, as BRECQ [27] defined.
Revised: Practically, L ar is not restricted to be utilized layer by layer, it can also be performed block byblock and stage by stage, as BRECQ [27] defined.

Original: We will discuss how the granularity affects theperformance in Sec. 4.2.1.
Revised: We will discuss how the granularity affects theperformance in Sec. 4.2.1.

Original: By default, we employ layer-wise AR in our experiments.
Revised: By default, we employ layer-wise AR in our experiments.

Original: 3.2 Annealing Softmax
Revised: 3.2 Annealing Softmax

Original: For a weight vector w , AdaRound [32] proposes to learn the round policy by:
Revised: For a weight vector w , AdaRound [32] proposes to learn the round policy by:

Original: [Equation]
Revised: [Equation]

Original: where ⌊·⌋ gives as output of the integral part and clip ( · ) clamps all values between q − and q + .
Revised: where ⌊·⌋ gives as output of the integral part and clip ( · ) clamps all values between q − and q + .

Original: ˆ w idenotes the quantized weight for the i -th element of w .
Revised: ˆ w idenotes the quantized weight for the i -th element of w .

Original: We will omit i below for simplicity.
Revised: We will omit i below for simplicity.

Original: s is thequantizer step size.
Revised: s is thequantizer step size.

Original: v is a learnable continuous vector w.r.t.
Revised: v is a learnable continuous vector w.r.t.

Original: w and h ( v ) is recitified sigmoid function(see Eqn.(4)) which is regularized to converge to { 0 , 1 } .
Revised: w and h ( v ) is recitified sigmoid function(see Eqn.(4)) which is regularized to converge to { 0 , 1 } .

Original: [Equation]
Revised: [Equation]

Original: Based on Eqn.(4), AdaRound proposes Rectified Sigmoid with an explicit Regularization (RSeR)loss to discrete h ( v ) :
Revised: Based on Eqn.(4), AdaRound proposes Rectified Sigmoid with an explicit Regularization (RSeR)loss to discrete h ( v ) :

Original: [Equation]
Revised: [Equation]

Original: However, there are two limitations of RSeR. First, RSeR works only for binary round policy.
Revised: However, there are two limitations of RSeR. First, RSeR works only for binary round policy.

Original: Second,
Revised: Second,

Original: RSeR is adversarial to quantization loss.
Revised: RSeR is adversarial to quantization loss.

Original: For example, for a linear transformation network, thequantization loss is
Revised: For example, for a linear transformation network, thequantization loss is

Original: [Equation]
Revised: [Equation]

Original: where x denotes the input and we ignore the clip function in Eqn.
Revised: where x denotes the input and we ignore the clip function in Eqn.

Original: Eqn.
Revised: Eqn.

Original: (6) indicates that quantization loss will prevent L RSeR from convergence ( i.e. , L RSeR encourages h ( v ) to converge to 0 or 1 while L q prefers h ( v ) to remain w s − ⌊ w s ⌋ ).
Revised: (6) indicates that quantization loss will prevent L RSeR from convergence ( i.e. , L RSeR encourages h ( v ) to converge to 0 or 1 while L q prefers h ( v ) to remain w s − ⌊ w s ⌋ ).

Original: Besides, this adversarialbecome more severe when extending layer/block-wise methods to network-wise quantization(see Sec.4.2.

Revised: Besides, this adversarialbecome more severe when extending layer/block-wise methods to network-wise quantization(see Sec.4.2.

Original: and Table 4).
Revised: and Table 4).

Original: 3.2.1 Formulation
Revised: 3.2.1 Formulation

Original: To resolve the two problems of RSeR, we propose Annealing Softmax (ASoftmax).
Revised: To resolve the two problems of RSeR, we propose Annealing Softmax (ASoftmax).

Original: First, we extendthe discrete optimization space of h ( · ) from { 0 , 1 } to K , where K is a integer range and defined by:
Revised: First, we extendthe discrete optimization space of h ( · ) from { 0 , 1 } to K , where K is a integer range and defined by:

Original: [Equation]
Revised: [Equation]

Original: where n and m are hyper-parameters that specify the discrete optimization space.
Revised: where n and m are hyper-parameters that specify the discrete optimization space.

Original: Then, ASoftmaxcan be formulated as:
Revised: Then, ASoftmaxcan be formulated as:

Original: [Equation]
Revised: [Equation]

Original: where V is a continuous learnable matrix and represents the logits of probability over K .
Revised: where V is a continuous learnable matrix and represents the logits of probability over K .

Original: h t ( V ) isdefined by:
Revised: h t ( V ) isdefined by:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where t represents for t -th iteration.
Revised: where t represents for t -th iteration.

Original: τ t is a temperature that decays linearly to transition h t ( V )from continuity to discretization.
Revised: τ t is a temperature that decays linearly to transition h t ( V )from continuity to discretization.

Original: For example, when τ t is large, p ( k | V , τ t ) is a continuous vectorof probability and h t ( V ) is the expectation of the discretization.
Revised: For example, when τ t is large, p ( k | V , τ t ) is a continuous vectorof probability and h t ( V ) is the expectation of the discretization.

Original: When τ t turns tiny, p ( k | V , τ t )becomes a one-hot vector and h t ( V ) selects k by p ( k | V , τ t )
Revised: When τ t turns tiny, p ( k | V , τ t )becomes a one-hot vector and h t ( V ) selects k by p ( k | V , τ t )

Original: = 1 .
Revised: = 1 .

Original: τ t is simply decayed by:
Revised: τ t is simply decayed by:

Original: [Equation]
Revised: [Equation]

Original: where τ 0 and τ T are hyper-parameters of initial and final temperature.
Revised: where τ 0 and τ T are hyper-parameters of initial and final temperature.

Original: ASoftmax is a general formation of learning wider range of discrete parameters than binary.
Revised: ASoftmax is a general formation of learning wider range of discrete parameters than binary.

Original: It easilydegenerates to round policy learning by setting n = 0 and m = 1 .
Revised: It easilydegenerates to round policy learning by setting n = 0 and m = 1 .

Original: We use τ T = 0 .
Revised: We use τ T = 0 .

Original: 01 and τ 0 = 1 bydefault.
Revised: 01 and τ 0 = 1 bydefault.

Original: 3.2.2 Initialization
Revised: 3.2.2 Initialization

Original: According to our observations, initialization has significant impacts on learning round policy.
Revised: According to our observations, initialization has significant impacts on learning round policy.

Original: Giventhe weight quantizer step size s , which has been computed using statistic-based PTQ methods, we attempt to initialize V in order to make s · ˆ w 0 as close to floating-point w as possible at the start.
Revised: Giventhe weight quantizer step size s , which has been computed using statistic-based PTQ methods, we attempt to initialize V in order to make s · ˆ w 0 as close to floating-point w as possible at the start.

Original: Toachieve this goal, we first calculate the un-normalized probability by:
Revised: Toachieve this goal, we first calculate the un-normalized probability by:

Original: [Equation]
Revised: [Equation]

Original: Then, we normalize the probability:
Revised: Then, we normalize the probability:

Original: [Equation]
Revised: [Equation]

Original: Finally, the initialization for V
Revised: Finally, the initialization for V

Original: i is log ( σ ( V ) i ) .
Revised: i is log ( σ ( V ) i ) .

Original: It can be easily proved (see Appendix A) that theinitialized s · ˆ w 0 equals w as long as the integer range K is symmetric w.r.t.
Revised: It can be easily proved (see Appendix A) that theinitialized s · ˆ w 0 equals w as long as the integer range K is symmetric w.r.t.

Original: 0.5 ( i.e. n = 1 − m ).
Revised: 0.5 ( i.e. n = 1 − m ).

Original: 3.3 Annealing Mixup
Revised: 3.3 Annealing Mixup

Original: NWQ considers activation quantization as a discrete optimization problem and solves it via continuousrelaxation.
Revised: NWQ considers activation quantization as a discrete optimization problem and solves it via continuousrelaxation.

Original: Different from ASoftmax which is applied on element-wise level, AMixup works ontensor-wise level: we randomly mix up the floating-point activations with the simulated quantizedactivations.
Revised: Different from ASoftmax which is applied on element-wise level, AMixup works ontensor-wise level: we randomly mix up the floating-point activations with the simulated quantizedactivations.

Original: To encourage continuous relaxation converge to discretization, NWQ progressivelydecreases the percentage of floating-point activations, so the mixed activations will become fullyquantized activation gradually.
Revised: To encourage continuous relaxation converge to discretization, NWQ progressivelydecreases the percentage of floating-point activations, so the mixed activations will become fullyquantized activation gradually.

Original: Formally, the percentage of floating-point activations is defined by:
Revised: Formally, the percentage of floating-point activations is defined by:

Original: [Equation]
Revised: [Equation]

Original: where P ( t ) is the percentage of floating-point activation in the mixed activation for the t -th iteration.
Revised: where P ( t ) is the percentage of floating-point activation in the mixed activation for the t -th iteration.

Original: P s and P e denote the percentage at start and end respectively, and T is the number of iterations.
Revised: P s and P e denote the percentage at start and end respectively, and T is the number of iterations.

Original: Weuse P s = 0 .
Revised: Weuse P s = 0 .

Original: 5 and P e = 0 by default.
Revised: 5 and P e = 0 by default.

Original: AMixup is similar to QDROP [40] in the mixup operation.
Revised: AMixup is similar to QDROP [40] in the mixup operation.

Original: The key difference, however, is that
Revised: The key difference, however, is that

Original: AMixup introduces an annealing process.
Revised: AMixup introduces an annealing process.

Original: This is very important for NWQ.
Revised: This is very important for NWQ.

Original: QDROP trains with mixedactivations while testing with fully quantized activations, thus introducing inconsistency bias.
Revised: QDROP trains with mixedactivations while testing with fully quantized activations, thus introducing inconsistency bias.

Original: Thisbias accmulates as network depth increases and leads to great accuracy degradation in network-wisequantization.
Revised: Thisbias accmulates as network depth increases and leads to great accuracy degradation in network-wisequantization.

Original: AMixup is able to eliminate the inconsistency bias via the annealing process.
Revised: AMixup is able to eliminate the inconsistency bias via the annealing process.

Original:  Experiments
Revised:  Experiments

Original: In this section, we conduct extensive experiments with various architectures to verify our NWQ.
Revised: In this section, we conduct extensive experiments with various architectures to verify our NWQ.

Original: We first compare our method with previous state-of-the-art, and then we delve into our importantdesigns and fully explore the scalability, effectiveness, and efficiency of our solution.
Revised: We first compare our method with previous state-of-the-art, and then we delve into our importantdesigns and fully explore the scalability, effectiveness, and efficiency of our solution.

Original: We report top-1classification accuracy on the ImageNet[9] validation set.
Revised: We report top-1classification accuracy on the ImageNet[9] validation set.

Original: Implementation Details.
Revised: Implementation Details.

Original: Our code is based on open-source codes BRECQ and QDrop.
Revised: Our code is based on open-source codes BRECQ and QDrop.

Original: Weapply layer-wise AR.
Revised: Weapply layer-wise AR.

Original: For ASoftmax, we set n = 0 , m = 1 and τ is decayed from 1 to 0 .
Revised: For ASoftmax, we set n = 0 , m = 1 and τ is decayed from 1 to 0 .

Original: For
Revised: For

Original: AMixup, we set default P s as 0.5 and P e as 0.
Revised: AMixup, we set default P s as 0.5 and P e as 0.

Original: We randomly sample 1024 images from ImageNettrain set and employ Cutmix [45] and Mixup[46] as data augmentation.
Revised: We randomly sample 1024 images from ImageNettrain set and employ Cutmix [45] and Mixup[46] as data augmentation.

Original: The learning rates are0 .
Revised: The learning rates are0 .

Original: 01 for round policy and 0 . 

Revised: 01 for round policy and 0 .

Original: for activation quantizer step size.
Revised: for activation quantizer step size.

Original: We train for 20000 iterationswith a mini-batch size of 32 in 8 Tesla V100 GPUs, taking ∼ 30 minutes for ResNet18, whichis on par with BRECQ and QDROP.
Revised: We train for 20000 iterationswith a mini-batch size of 32 in 8 Tesla V100 GPUs, taking ∼ 30 minutes for ResNet18, whichis on par with BRECQ and QDROP.

Original: Our experiments are conducted on 5 architectures, including
Revised: Our experiments are conducted on 5 architectures, including

Original: ResNet18(Res18), MobileNetV2(MNV2), RegNet-600MF(Reg600MF), RegNet-3.2GF(Reg3.2GF)and MnasNet(Mnas).
Revised: ResNet18(Res18), MobileNetV2(MNV2), RegNet-600MF(Reg600MF), RegNet-3.2GF(Reg3.2GF)and MnasNet(Mnas).

Original: Other settings remain same as QDROP[40] if not specified.
Revised: Other settings remain same as QDROP[40] if not specified.

Original: 4.1 Comprehensive Comparison
Revised: 4.1 Comprehensive Comparison

Original: Table 1 compares our NWQ with several main-stream PTQ approaches.
Revised: Table 1 compares our NWQ with several main-stream PTQ approaches.

Original: Overall, NWQ outperformsprevious works across all neural network architectures and bit widths, and our performance improveseven more as the bit width decreases.
Revised: Overall, NWQ outperformsprevious works across all neural network architectures and bit widths, and our performance improveseven more as the bit width decreases.

Original: Classical Calibration Set.
Revised: Classical Calibration Set.

Original: Following previous works, we compare the performance on a smallcalibration set of 1024 images.
Revised: Following previous works, we compare the performance on a smallcalibration set of 1024 images.

Original: For start, we achieve 0.2% ∼ 0.9% improvement even compared with
Revised: For start, we achieve 0.2% ∼ 0.9% improvement even compared with

Original: [Table]
Revised: [Table]

Original: strong baselines on W4A4, which are rather close to full precision accuracy.
Revised: strong baselines on W4A4, which are rather close to full precision accuracy.

Original: For W3A3, NWQ yields0.83% ∼ 3.26% and reduces the gap between W3A3 and W4A4.
Revised: For W3A3, NWQ yields0.83% ∼ 3.26% and reduces the gap between W3A3 and W4A4.

Original: For the challenging W2A2, ourmethod surpasses the previous state-of-the-art by a large margin of 13.37% for MobileNetV2 and12.4% for MnasNet, pushing the limit of extremely low-bit PTQ step further.
Revised: For the challenging W2A2, ourmethod surpasses the previous state-of-the-art by a large margin of 13.37% for MobileNetV2 and12.4% for MnasNet, pushing the limit of extremely low-bit PTQ step further.

Original: Scaled-up Calibration Set.
Revised: Scaled-up Calibration Set.

Original: To further explore the potential of NWQ, we scale up the calibrationset by 10 × and reproduce QDROP on the scaled-up calibration set, building a very strong baseline.
Revised: To further explore the potential of NWQ, we scale up the calibrationset by 10 × and reproduce QDROP on the scaled-up calibration set, building a very strong baseline.

Original: Interestingly, NWQ outperforms QDROP even with one-tenth data.
Revised: Interestingly, NWQ outperforms QDROP even with one-tenth data.

Original: When equipping NWQ withthe scaled-up calibration set, we observe significant improvement under all settings.
Revised: When equipping NWQ withthe scaled-up calibration set, we observe significant improvement under all settings.

Original: Particularly,the accuracy gaps of W2A2 between NWQ and QDROP are enlarged even more as the calibrationset scales up.
Revised: Particularly,the accuracy gaps of W2A2 between NWQ and QDROP are enlarged even more as the calibrationset scales up.

Original: For example, the gap of MnasNet increases from 12.4% to 19.36%, and that of
Revised: For example, the gap of MnasNet increases from 12.4% to 19.36%, and that of

Original: MobileNetV2 increases from 13.37% to 20.24%, showing the excellent scalability of our method.
Revised: MobileNetV2 increases from 13.37% to 20.24%, showing the excellent scalability of our method.

Original: 4.2 Ablation Study
Revised: 4.2 Ablation Study

Original: 4.2.1 AR
Revised: 4.2.1 AR

Original: [Figure]
Revised: [Figure]

Original: The Effect of AR.
Revised: The Effect of AR.

Original: We study the effect of AR via comparing the experiments with layer-wise AR to the experimentswithout AR on W2A2.
Revised: We study the effect of AR via comparing the experiments with layer-wise AR to the experimentswithout AR on W2A2.

Original: Table 2 shows that layer-wise ARimproves the accuracy significantly.
Revised: Table 2 shows that layer-wise ARimproves the accuracy significantly.

Original: It’s worth notingthat layer-wise AR improves 5.22% for MobileNetV

Revised: It’s worth notingthat layer-wise AR improves 5.22% for MobileNetV

Original: and10.42% for MnasNet.
Revised: and10.42% for MnasNet.

Original: To understand how AR works, weanalyze the loss curves during training in Figure 3.
Revised: To understand how AR works, weanalyze the loss curves during training in Figure 3.

Original: First,our AR experiment consistently converges to lower losson both the train and test datasets, resulting in higher accuracy.
Revised: First,our AR experiment consistently converges to lower losson both the train and test datasets, resulting in higher accuracy.

Original: Second, AR accelerates convergence significantlyand thus allows NWQ to reach competitive accuracy at a
Revised: Second, AR accelerates convergence significantlyand thus allows NWQ to reach competitive accuracy at a

Original: [Table]
Revised: [Table]

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: [Table]
Revised: [Table]

Original: substantially lower computing cost.
Revised: substantially lower computing cost.

Original: Third, even though AR does not totally eliminate over-fitting, iteffectively bridges the loss gap between train and test datasets.
Revised: Third, even though AR does not totally eliminate over-fitting, iteffectively bridges the loss gap between train and test datasets.

Original: The Effect of Granularity.
Revised: The Effect of Granularity.

Original: Table 2 studies how AR granularity affects accuracy.
Revised: Table 2 studies how AR granularity affects accuracy.

Revised: AR granularityrepresents for applying AR after each layer/block/stage, while training(reconstruction) granularityrepresents for optimizing each layer/block/stage sequentially and independently.

Original: We discoverthat lightweight networks prefer coarse-grained AR, whereas heavyweight networks prefer finegrained AR.
Revised: We discover thatlightweight networks prefer coarse-grained AR, whereas heavyweight networks prefer fine-grained AR.

Original: MobileNetV2, RegNet-600MF, and MnasNet, for example, achieve the best performancewith block-wise AR, whereas ResNet18 and RegNet-3.2GF achieve the maximum accuracy withlayer-wise AR.
Revised: MobileNetV2, RegNet-600MF, and MnasNet, for example, achieve the best performance withblock-wise AR, whereas ResNet18 and RegNet-3.2GF achieve the maximum accuracy with layer-wise AR.

Original: Comparison with Data Augmentation.
Revised: Comparison with Data Augmentation.

Original: Data augmentation is a classical strategy used to relieveover-fitting due to its simplicity and effectiveness.
Revised: Data augmentation is a classical strategy used to relieveover-fitting due to its simplicity and effectiveness.

Original: We compare AR with strong augmentation ( e.g.
Revised: We compare AR with strong augmentation ( e.g.

Original: AutoAugment[8]) in Table 3.
Revised: AutoAugment[8]) in Table 3.

Original: Notably, both AR and AutoAugment significantly improve accuracy,and AR contributes more than AutoAugment, particularly for sparse networks like MobileNetV2 and
Revised: Notably, both AR and AutoAugment significantly improve accuracy,and AR contributes more than AutoAugment, particularly for sparse networks like MobileNetV2 and

Original: MnasNet.
Revised: MnasNet.

Original: Coupling these two techniques enables us to achieve better performance.
Revised: Coupling these two techniques enables us to achieve better performance.

Original: 4.2.2 ASoftmax Study
Revised: 4.2.2 ASoftmax Study

Original: The Effect of ASoftmax.
Revised: The Effect of ASoftmax.

Original: Table 4 compares ASoftmax to Rectified Sigmoid with explicit Regularization (RSeR) which is used in AdaRound.
Revised: Table 4 compares ASoftmax to Rectified Sigmoid with explicit Regularization (RSeR) which is used in AdaRound.

Original: We simply replace ASoftmax with RSeR in our NWQ andrun a grid search on regularization loss weight to explore the potential of RSeR.
Revised: We simply replace ASoftmax with RSeR in our NWQ andrun a grid search on regularization loss weight to explore the potential of RSeR.

Original: We discover that
Revised: We discover that

Original: RSeR requires a huge loss weight for NWQ, which is 10 3 ∼ 10 5 × larger than that of subnet-wiseapproaches.
Revised: RSeR requires a huge loss weight for NWQ, which is 10 3 ∼ 10 5 × larger than that of subnet-wiseapproaches.

Original: Even with exhausting hyper-parameter search, RSeR does not perform as well as in thesubnet-wise method.
Revised: Even with exhausting hyper-parameter search, RSeR does not perform as well as in thesubnet-wise method.

Original: ASoftmax, on the other hand, outperforms RSeR by 12.58% and 21.56% on
Revised: ASoftmax, on the other hand, outperforms RSeR by 12.58% and 21.56% on

Original: ResNet18 and MobileNetV2, respectively, without any additional hyper-parameter tuning.
Revised: ResNet18 and MobileNetV2, respectively, without any additional hyper-parameter tuning.

Original: The Effect of Initialization.
Revised: The Effect of Initialization.

Original: Table 5 demonstrates the effectiveness of our initialization, which keepsthe quantization model close to the float-point at the start to steady the training.
Revised: Table 5 demonstrates the effectiveness of our initialization, which keepsthe quantization model close to the float-point at the start to steady the training.

Original: It is shown that ourinitialization method significantly outperforms random initialization.
Revised: It is shown that ourinitialization method significantly outperforms random initialization.

Original: The Robustness of τ .
Revised: The Robustness of τ .

Original: We investigate the robustness of τ in Table 6.
Revised: We investigate the robustness of τ in Table 6.

Original: τ starts with a large numberto encourage continuous relaxation and ends with a small value to favor one-hot distribution.
Revised: τ starts with a large numberto encourage continuous relaxation and ends with a small value to favor one-hot distribution.

Original: Thefinal τ T determines the decaying speed of τ .
Revised: Thefinal τ T determines the decaying speed of τ .

Original: It can be observed that τ T has a robustness range of10 − 2 ∼ 10 − 4 , making it easily adaptable to a wider range of neural network architectures.
Revised: It can be observed that τ T has a robustness range of10 − 2 ∼ 10 − 4 , making it easily adaptable to a wider range of neural network architectures.

Original: The Extensiveness of ASoftmax.
Revised: The Extensiveness of ASoftmax.

Original: As described in Sec.3.2.1, ASoftmax can easily extend the discreteparameter space from { 0 , 1 } to a larger integer range, offering the opportunity for a better optimum.
Revised: As described in Sec.3.2.1, ASoftmax can easily extend the discreteparameter space from { 0 , 1 } to a larger integer range, offering the opportunity for a better optimum.

Original: However, extending discrete range benefits MobileNetV2 and RegNet-600MF while degrading
Revised: However, extending discrete range benefits MobileNetV2 and RegNet-600MF while degrading

Original: RegNet-3.2GF on a small calibration set, as shown in Table 7 upper group. We argue that the enlarged
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Revised: RegNet-3.2GF on a small calibration set, as shown in Table 7 upper group.

Original: discrete range raises the risk of over-fitting for large networks.
Revised: We argue that the enlargeddiscrete range raises the risk of over-fitting for large networks.

Original: When we scale up the calibration set,the performance no longer degrades and the performance improves even more.
Revised: When we scale up the calibration set,the performance no longer degrades and the performance improves even more.

Original: 4.2.3 AMixup Study
Revised: 4.2.3 AMixup Study

Original: The Effect of AMixup.
Revised: The Effect of AMixup.

Original: QDROP presents good performance in block-wise reconstruction, but failsin our network-wise scheme.
Revised: QDROP presents good performance in block-wise reconstruction, but failsin our network-wise scheme.

Original: As illustrated in Table 8, we achieve 1.95% improvement for ResNet18and 19.96% for MobileNetV2 by simply removing QDROP (w/o Drop row).
Revised: As illustrated in Table 8, we achieve 1.95% improvement for ResNet18and 19.96% for MobileNetV2 by simply removing QDROP (w/o Drop row).

Original: We hypothesizethat randomly mixing activations up introduces inconsistency bias between the training and testphase, which is concealed by the improvement of flatness in block-wise reconstruction.
Revised: We hypothesizethat randomly mixing activations up introduces inconsistency bias between the training and testphase, which is concealed by the improvement of flatness in block-wise reconstruction.

Original: However,as the reconstruction unit goes deeper (from block to network) and sparser, the problem becomesmore pronounced, resulting in significant degeneration.
Revised: However,as the reconstruction unit goes deeper (from block to network) and sparser, the problem becomesmore pronounced, resulting in significant degeneration.

Original: Over-parameterized ResNet18 is able tocompensate for the side effect, but MobileNetV2 suffers a great performance drop.
Revised: Over-parameterized ResNet18 is able tocompensate for the side effect, but MobileNetV2 suffers a great performance drop.

Original: Remarkably, our
Revised: Remarkably, our

Original: AMixup is able to overcome the inconsistency while taking advantage of the flatness, contributing tohigher accuracy.
Revised: AMixup is able to overcome the inconsistency while taking advantage of the flatness, contributing tohigher accuracy.

Original: The Policy of Decay.
Revised: The Policy of Decay.

Original: In Table 9, we run a grid search for P s to explores the influence of differentdecay policies.
Revised: In Table 9, we run a grid search for P s to explores the influence of differentdecay policies.

Original: It’s worth noting that MobileNetV2 and MnasNet prefer 0.5, whereas ResNet18 and
Revised: It’s worth noting that MobileNetV2 and MnasNet prefer 0.5, whereas ResNet18 and

Original: RegNet prefer 1.
Revised: RegNet prefer 1.

Original: The basic law is very similar to that of AR granularity, which states that heavyweightnetworks require more regularization, implying that AMixup and AR are both capable of regularizingaccuracy.
Revised: The basic law is very similar to that of AR granularity, which states that heavyweightnetworks require more regularization, implying that AMixup and AR are both capable of regularizingaccuracy.

Original: Efficiency 20 [-Iterations/
 A cc u r a c y / % 53.-]
Revised: [Figure] 20 [-Iterations/
 A cc u r a c y / % 53.-]Iterations/

Original: NWQ(Ours)
Revised: [Figure]

Original: QDROP
Revised: [Figure]

Original: BRECQ
Revised: [Figure]

Original: We compare the efficiency of several approacheson ResNet18.
Revised: We compare the efficiency of several approacheson ResNet18.

Original: According to Figure 4, NWQ outperforms QDROP and BRECQ under various iterations.
Revised: According to Figure 4, NWQ outperforms QDROP and BRECQ under various iterations.

Original: Specifically, NWQ achieve competitiveresults with only 10% computation cost, demonstrating the efficiency of NWQ.
Revised: Specifically, NWQ achieve competitiveresults with only 10% computation cost, demonstrating the efficiency of NWQ.

Original: As the number ofiterations increases, NWQ continues to improveaccuracy, whereas BRECQ and QDROP reachtheir saturation at ∼ 1 .
Revised: As the number ofiterations increases, NWQ continues to improveaccuracy, whereas BRECQ and QDROP reachtheir saturation at ∼ 1 .

Original: 6 × 10 4 iterations.
Revised: 6 × 10 4 iterations.

Original:  Discussion and Conclusion
Revised:  Discussion and Conclusion

Original: Simple algorithms that scale well are the core ofdeep learning.
Revised: Simple algorithms that scale well are the core ofdeep learning.

Original: Prior works tend to quantize networks in layer-wise and block-wise manner.
Revised: Prior works tend to quantize networks in layer-wise and block-wise manner.

Original: We prove that network-wise quantization is a simplerand more effective way than layer/block-wise solutions.
Revised: We prove that network-wise quantization is a simplerand more effective way than layer/block-wise solutions.

Original: To accomplish this, we analyze the problemsintroduced by the network-wise regime and propose three key designs to address them.
Revised: To accomplish this, we analyze the problemsintroduced by the network-wise regime and propose three key designs to address them.

Original: Based on that,we construct a simple and effective end-to-end network-wise PTQ framework.
Revised: Based on that,we construct a simple and effective end-to-end network-wise PTQ framework.

Original: Extensive experimentsdemonstrate that our solution is scalable, effective, and efficient.
Revised: Extensive experimentsdemonstrate that our solution is scalable, effective, and efficient.

Original: However, there is still a largeaccuracy gap between low-bit PTQ and full precision network, and we believe that there is still muchroom for further improvement for PTQ, which is a key direction for future research.
Revised: However, there is still a largeaccuracy gap between low-bit PTQ and full precision network, and we believe that there is still muchroom for further improvement for PTQ, which is a key direction for future research.

Original: Broder impacts.
Revised: Broder impacts.

Original: The proposed method uses GPU resources for training, thus resulting in a certainamount of energy consumption and carbon footprints.
Revised: The proposed method uses GPU resources for training, thus resulting in a certainamount of energy consumption and carbon footprints.

Original: For example, it takes about 4 GPU-hours forour method to quantize ResNet18, which would consume around 1 kWh and produce around 1 lbs of
Revised: For example, it takes about 4 GPU-hours forour method to quantize ResNet18, which would consume around 1 kWh and produce around 1 lbs of

Original: CO2, accelerating climate change and global warming in some degree.
Revised: CO2, accelerating climate change and global warming in some degree.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See
Revised: [Yes] See

Original: broader impacts.
Revised: broader impacts.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes](b) Did you include complete proofs of all theoretical results?
Revised: [Yes](b) Did you include complete proofs of all theoretical results?

Original: [Yes]3.
Revised: [Yes]3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes](b) Did you specify all the training details (e
Revised: [Yes](b) Did you specify all the training details (e

Original: g., data splits, hyperparameters,
Revised: g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-
Revised: g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes] We report error bars in Table 1.
Revised: [Yes] We report error bars in Table 1.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type

Original: of GPUs, internal cluster, or cloud provider)?
Revised: of GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g.
Revised: g.

Original: , code,
Revised: , code,

Original: data
Revised: data,

Original: [Table]

Original: ) or curating
Revised: models) or curating

Original: [Table]
Revised: [Table]

Original: releasing
Revised: releasing

Original: [Table]
Revised: [Table]

Original: assets...
Revised: assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [No]
Revised: [No]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [No] We use public datasets only.
Revised: [No] We use public datasets only.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A ASoftmax Initialization
Revised: A ASoftmax Initialization

Original: First of all, We define the fractional part of x as: { x } = x − ⌊ x ⌋ , and we use w 0 to represent initiallysimulated quantization of w .
Revised: First of all, We define the fractional part of x as: { x } = x − ⌊ x ⌋ , and we use w 0 to represent initiallysimulated quantization of w .

Original: According to equation (9),(10), (12) and (13), we get:
Revised: According to equation (9),(10), (12) and (13), we get:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: If we disregard the clip function, then
Revised: If we disregard the clip function, then

Original: [Equation]
Revised: [Equation]