Original: D EFENSIVE Q UANTIZATION : W HEN E FFICIENCY M EETS R OBUSTNESS
Revised: D EFENSIVE Q UANTIZATION : W HEN E FFICIENCY M EETS R OBUSTNESS

Original: Anonymous authors
Revised: Ji Lin MIT jilin@mit.edu

Original: Paper under double-blind review
Revised: Chuang Gan MIT-IBM Watson AI Lab ganchuang@csail.mit.edu Song Han MIT songhan@mit.edu

Original: A BSTRACT
Revised: A BSTRACT

Original: Neural network quantization is becoming an industry standard to efﬁciently deploy deep learning models on hardware platforms, such as CPU, GPU, TPU, and FPGAs.
Revised: Neural network quantization is becoming an industry standard to efﬁciently deploy deep learning models on hardware platforms, such as CPU, GPU, TPU, and FPGAs.

Original: However, we observe that the conventional quantization approaches are vulnerable to adversarial attacks.
Revised: However, we observe that the conventional quantization approaches are vulnerable to adversarial attacks.

Original: This paper aims to raise people’s awareness about the security of the quantized models, and we designed a novel quantization methodology to jointly optimize the efﬁciency and robustness of deep learning models.
Revised: This paper aims to raise people’s awareness about the security of the quantized models, and we designed a novel quantization methodology to jointly optimize the efﬁciency and robustness of deep learning models.

Original: We ﬁrst conduct an empirical study to show that vanilla quantization suffers more from adversarial attacks.
Revised: We ﬁrst conduct an empirical study to show that vanilla quantization suffers more from adversarial attacks.

Original: We observe that the inferior robustness comes from the error ampliﬁcation effect , where the quantization operation further enlarges the distance caused by ampliﬁed noise.
Revised: We observe that the inferior robustness comes from the error ampliﬁcation effect , where the quantization operation further enlarges the distance caused by ampliﬁed noise.

Original: Then we propose a novel Defensive
Revised: Then we propose a novel Defensive

Original: Quantization (DQ) method by controlling the Lipschitz constant of the network during quantization, such that the magnitude of the adversarial noise remains non-expansive during inference.
Revised: Quantization (DQ) method by controlling the Lipschitz constant of the network during quantization, such that the magnitude of the adversarial noise remains non-expansive during inference.

Original: Extensive experiments on CIFAR-10 and SVHN datasets demonstrate that our new quantization method can defend neural networks against adversarial examples, and even achieves superior robustness than their fullprecision counterparts, while maintaining the same hardware efﬁciency as vanilla quantization approaches.
Revised: Extensive experiments on CIFAR-10 and SVHN datasets demonstrate that our new quantization method can defend neural networks against adversarial examples, and even achieves superior robustness than their fullprecision counterparts, while maintaining the same hardware efﬁciency as vanilla quantization approaches.

Original: As a by-product, DQ can also improve the accuracy of quantized models without adversarial attack.
Revised: As a by-product, DQ can also improve the accuracy of quantized models without adversarial attack.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Neural network quantization (Han et al., 2015; Zhu et al., 2016; Jacob et al., 2017) is a widely used technique to reduce the computation and memory costs of neural networks, facilitating efﬁcient deployment.
Revised: Neural network quantization (Han et al., 2015; Zhu et al., 2016; Jacob et al., 2017) is a widely used technique to reduce the computation and memory costs of neural networks, facilitating efﬁcient deployment.

Original: It has become an industry standard for deep learning hardware.
Revised: It has become an industry standard for deep learning hardware.

Original: However, we ﬁnd that the widely used vanilla quantization approaches suffer from unexpected issues — the quantized model is more vulnerable to adversarial attacks (Figure 1).
Revised: However, we ﬁnd that the widely used vanilla quantization approaches suffer from unexpected issues — the quantized model is more vulnerable to adversarial attacks (Figure 1).

Original: Adversarial attack is consist of subtle perturbations on the input images that causes the deep learning models to give incorrect labels (Szegedy et al., 2013;
Revised: Adversarial attack is consist of subtle perturbations on the input images that causes the deep learning models to give incorrect labels (Szegedy et al., 2013;

Original: Goodfellow et al.).
Revised: Goodfellow et al.).

Original: Such perturbations are hardly detectable by human eyes but can easily fool neural networks.
Revised: Such perturbations are hardly detectable by human eyes but can easily fool neural networks.

Original: Since quantized neural networks are widely deployed in many safety-critical scenarios, e.g ., autonomous driving (Amodei et al., 2016), the potential security risks cannot be neglected.
Revised: Since quantized neural networks are widely deployed in many safety-critical scenarios, e.g ., autonomous driving (Amodei et al., 2016), the potential security risks cannot be neglected.

Original: The efﬁciency and latency in such applications are also important, so we need to jointly optimize them.
Revised: The efﬁciency and latency in such applications are also important, so we need to jointly optimize them.

Original: The fact that quantization leads to inferior adversarial robustness is counter intuitive, as small perturbations should be denoised with low-bit representations.
Revised: The fact that quantization leads to inferior adversarial robustness is counter intuitive, as small perturbations should be denoised with low-bit representations.

Original: Recent work (Xu et al., 2017) also demonstrates that quantization on input image space , i.e . color bit depth reduction, is quite effective to defend adversarial examples.
Revised: Recent work (Xu et al., 2017) also demonstrates that quantization on input image space , i.e . color bit depth reduction, is quite effective to defend adversarial examples.

Original: A natural question then rises, why the quantization operator is yet effective when applied to intermediate DNN layers?
Revised: A natural question then rises, why the quantization operator is yet effective when applied to intermediate DNN layers?

Original: We analyze that such issue is caused by the error ampliﬁcation effect of adversarial perturbation (Liao et al., 2018) — although the magnitude of perturbation on the image is small, it is ampliﬁed signiﬁcantly when passing through deep neural network (see Figure 3b).
Revised: We analyze that such issue is caused by the error ampliﬁcation effect of adversarial perturbation (Liao et al., 2018) — although the magnitude of perturbation on the image is small, it is ampliﬁed signiﬁcantly when passing through deep neural network (see Figure 3b).

Original: The deeper the layers are, the more signiﬁcant such side effect is.
Revised: The deeper the layers are, the more signiﬁcant such side effect is.

Original: Such ampliﬁcation pushes values into a different quantization bucket, which is undesirable.
Revised: Such ampliﬁcation pushes values into a different quantization bucket, which is undesirable.

Original: We conducted empirical experiments to analyze how quantization inﬂuences the activation error between clean and adversarial samples (Figure 3a): when the magnitude of the noise is small, activation quantization is capable of reducing the errors by eliminating small perturbations; However, when the magnitude of perturbation is larger than certain threshold, quantization instead amplify the errors, which causes  the quantized model to make mistakes.
Revised: We conducted empirical experiments to analyze how quantization inﬂuences the activation error between clean and adversarial samples (Figure 3a): when the magnitude of the noise is small, activation quantization is capable of reducing the errors by eliminating small perturbations; However, when the magnitude of perturbation is larger than certain threshold, quantization instead amplify the errors, which causes [Figure] the quantized model to make mistakes.

Original: We argue that this is the main reason causing the inferior robustness of the quantized models.
Revised: We argue that this is the main reason causing the inferior robustness of the quantized models.

Original: [Figure] In this paper, we propose Defensive Quantization (DQ) that not only ﬁxes the robustness issue of quantized models, but also turns activation quantization into a defense method that further boosts adversarial robustness.
Revised: In this paper, we propose Defensive Quantization (DQ) that not only ﬁxes the robustness issue of quantized models, but also turns activation quantization into a defense method that further boosts adversarial robustness.

Original: We are inspired by the success of image quantization in improving robustness.
Revised: We are inspired by the success of image quantization in improving robustness.

Original: Intuitively, it will be possible to defend the attacks with quantization operations if we can keep the magnitude of the perturbation small.
Revised: Intuitively, it will be possible to defend the attacks with quantization operations if we can keep the magnitude of the perturbation small.

Original: However, due to the error ampliﬁcation effect of gradient based adversarial samples, it is non-trivial to keep the noise at a small scale during inference.
Revised: However, due to the error ampliﬁcation effect of gradient based adversarial samples, it is non-trivial to keep the noise at a small scale during inference.

Original: Recent works (Cisse et al., 2017; Qian & Wegman, 2018) have attempted to make the network non-expansive by controlling the Lipschitz constant of the network to be smaller than 1, which has smaller variation change in its output than its input.
Revised: Recent works (Cisse et al., 2017; Qian & Wegman, 2018) have attempted to make the network non-expansive by controlling the Lipschitz constant of the network to be smaller than 1, which has smaller variation change in its output than its input.

Original: In such case, the input noise will not propagate through the intermediate layers and impact the output, but attenuated.
Revised: In such case, the input noise will not propagate through the intermediate layers and impact the output, but attenuated.

Original: Our method is built on the theory.
Revised: Our method is built on the theory.

Original: Defensive quantization not only quantizes feature maps into low-bit representations, but also controls the Lipschitz constant of the network, such that the noise is kept within a small magnitude for all hidden layers.
Revised: Defensive quantization not only quantizes feature maps into low-bit representations, but also controls the Lipschitz constant of the network, such that the noise is kept within a small magnitude for all hidden layers.

Original: In such case, we keep the noise small, as in the left zone of Figure 3a, quantization can reduce the perturbation error.
Revised: In such case, we keep the noise small, as in the left zone of Figure 3a, quantization can reduce the perturbation error.

Original: The produced model with our method enjoys better security and efﬁciency at the same time.
Revised: The produced model with our method enjoys better security and efﬁciency at the same time.

Original: Experiments show that Defensive Quantization (DQ) offers three unique advantages.
Revised: Experiments show that Defensive Quantization (DQ) offers three unique advantages.

Original: First, DQ provides an effective way to boost the robustness of deep learning models while maintains the efﬁciency.
Revised: First, DQ provides an effective way to boost the robustness of deep learning models while maintains the efﬁciency.

Original: Second, DQ is a generic building block of adversarial defense, which can be combined with other adversarial defense techniques to advance state-of-the-art robustness.
Revised: Second, DQ is a generic building block of adversarial defense, which can be combined with other adversarial defense techniques to advance state-of-the-art robustness.

Original: Third, our method makes quantization itself easier thanks to the constrained dynamic range.
Revised: Third, our method makes quantization itself easier thanks to the constrained dynamic range.

Original:  B ACKGROUND AND R ELATED W ORK 2. M ODEL Q UANTIZATION
Revised:  B ACKGROUND AND R ELATED W ORK 2. M ODEL Q UANTIZATION

Original: Neural network quantization (Han et al., 2015; Rastegari et al., 2016; Zhou et al., 2016; Courbariaux & Bengio, 2016; Zhu et al., 2016)) are widely adopted to enable efﬁcient inference.
Revised: Neural network quantization (Han et al., 2015; Rastegari et al., 2016; Zhou et al., 2016; Courbariaux & Bengio, 2016; Zhu et al., 2016)) are widely adopted to enable efﬁcient inference.

Original: By quantizing the network into low-bit representation, the inference of network requires less computation and less memory, while still achieves little accuracy degradation on clean images.
Revised: By quantizing the network into low-bit representation, the inference of network requires less computation and less memory, while still achieves little accuracy degradation on clean images.

Original: However, we ﬁnd that the quantized models suffer from severe security issues — they are more vulnerable against adversarial attacks compared to full-precision models, even when they have the same clean image accuracy.
Revised: However, we ﬁnd that the quantized models suffer from severe security issues — they are more vulnerable against adversarial attacks compared to full-precision models, even when they have the same clean image accuracy.

Original: Adversarial perturbation is applied to the input image, thus it is most related to activation quantization (Dhillon et al., 2018).
Revised: Adversarial perturbation is applied to the input image, thus it is most related to activation quantization (Dhillon et al., 2018).

Original: We carry out the rest of the paper using ReLU6 based activation quantization (Howard et al., 2017; Sandler et al., 2018), as it is computationally efﬁcient and is widely adopted by modern frameworks like TensorFlow (Abadi et al., 2016).
Revised: We carry out the rest of the paper using ReLU6 based activation quantization (Howard et al., 2017; Sandler et al., 2018), as it is computationally efﬁcient and is widely adopted by modern frameworks like TensorFlow (Abadi et al., 2016).

Original: As illustrated in Figure 2, a quantized convolutional network is composed of several quantized convolution block, each containing a serial of conv + BN + ReLU6 + linear quantize operators. 
Revised: As illustrated in Figure 2, a quantized convolutional network is composed of several quantized convolution block, each containing a serial of conv + BN + ReLU6 + linear quantize operators. As the quantization operator has [Figure] X

Original: As the quantization operator has 0 gradient almost everywhere, we followed common practice to use a STE (Bengio et al., 2013) function y
Revised: X i X ic X ibn X ir

Revised: X iq

Revised: W i

Revised: [Figure]

Revised: ||

Revised: [Figure]

Revised: Tl

Revised: [Figure]

Revised:

Revised: 0 gradient almost everywhere, we followed common practice to use a STE (Bengio et al., 2013) function y

Original: = x
Revised: = x

Original: + stop gradient [ Quantize ( x ) − x ] for gradient computation, which also eliminates the obfuscated gradient problem (Athalye et al., 2018).
Revised: + stop gradient [ Quantize ( x ) − x ] for gradient computation, which also eliminates the obfuscated gradient problem (Athalye et al., 2018).

Original: Our work bridges two domains: model quantization and adversarial defense.
Revised: Our work bridges two domains: model quantization and adversarial defense.

Original: Previous work (Galloway et al., 2017) claims binary quantized networks can improve the robustness against some attacks.
Revised: Previous work (Galloway et al., 2017) claims binary quantized networks can improve the robustness against some attacks.

Original: However, the improvement is not substantial and they used randomized quantization, which is not practical for real deployment (need extra random number generators in hardware).
Revised: However, the improvement is not substantial and they used randomized quantization, which is not practical for real deployment (need extra random number generators in hardware).

Original: It also causes[Figure]  one of the obfuscated gradient situations (Athalye et al., 2018): stochastic gradients, leading to a false sense of security.
Revised: It also causes one of the obfuscated gradient situations (Athalye et al., 2018): stochastic gradients, leading to a false sense of security.

Original: Rakin et al.
Revised: Rakin et al.

Original: (2018) tries to use quantization as an effective defense method.
Revised: (2018) tries to use quantization as an effective defense method.

Original: However, they employed Tanh-based quantization, which is not hardware friendly on ﬁxed-point units due to the large overhead accessing look-up table.
Revised: However, they employed Tanh-based quantization, which is not hardware friendly on ﬁxed-point units due to the large overhead accessing look-up table.

Original: Even worse, according to our re-implementation, their method actually leads to severe gradient masking problem (Papernot et al., 2016a) during adversarial training, due to the nature of Tanh function (see A.1 for detail).
Revised: Even worse, according to our re-implementation, their method actually leads to severe gradient masking problem (Papernot et al., 2016a) during adversarial training, due to the nature of Tanh function (see A.1 for detail).

Original: As a result, the actual robustness of this work under black-box attack has no improvement over full-precision model and is even worse.
Revised: As a result, the actual robustness of this work under black-box attack has no improvement over full-precision model and is even worse.

Original: Therefore, there is no previous work that are conducted under real inference setting to study the quantized robustness for both black-box and white-box.
Revised: Therefore, there is no previous work that are conducted under real inference setting to study the quantized robustness for both black-box and white-box.

Original: Our work aim to raise people’s awareness about the security of the actual deployed models.
Revised: Our work aim to raise people’s awareness about the security of the actual deployed models.

Original: 2. A DVERSARIAL A TTACKS & D EFENSES
Revised: 2. A DVERSARIAL A TTACKS & D EFENSES

Original: Given an image X , an adversarial attack method tries to ﬁnd a small perturbation ∆ with constraint || ∆ || ≤ (cid:15) , such that the neural network gives different outputs for X and X adv (cid:44) X + ∆ .
Revised: Given an image X , an adversarial attack method tries to ﬁnd a small perturbation ∆ with constraint || ∆ || ≤ (cid:15) , such that the neural network gives different outputs for X and X adv (cid:44) X + ∆ .

Original: Here (cid:15) is a scalar to constrain the norm of the noise ( e.g ., (cid:15) = 8 is commonly used when we represent colors from 0-255), so that the perturbation is hardly visible to human.
Revised: Here (cid:15) is a scalar to constrain the norm of the noise ( e.g ., (cid:15) = 8 is commonly used when we represent colors from 0-255), so that the perturbation is hardly visible to human.

Original: For this paper we choose to study attacks deﬁned under || · || ∞ , where each element of the image can vary at most (cid:15) to form an adversary.
Revised: For this paper we choose to study attacks deﬁned under || · || ∞ , where each element of the image can vary at most (cid:15) to form an adversary.

Original: We introduce several attack and defense methods used in our work in the following sections.
Revised: We introduce several attack and defense methods used in our work in the following sections.

Original: 2.2. A TTACK METHODS Random Perturbation (Random)
Revised: 2.2. A TTACK METHODS Random Perturbation (Random)

Original: Random perturbation attack adds a uniform sampled noise within [ − (cid:15), (cid:15) ] to the image, The method has no prior knowledge of the data and the network, thus is considered as the weakest attack method.
Revised: Random perturbation attack adds a uniform sampled noise within [ − (cid:15), (cid:15) ] to the image, The method has no prior knowledge of the data and the network, thus is considered as the weakest attack method.

Original: Fast Gradient Sign Method (FGSM) & R+FGSM Goodfellow et al. proposed a fast method to calculate the adversarial noise by following the direction of the loss gradient ∇ X L ( X , y ) , where
Revised: Fast Gradient Sign Method (FGSM) & R+FGSM Goodfellow et al. proposed a fast method to calculate the adversarial noise by following the direction of the loss gradient ∇ X L ( X , y ) , where

Original: L ( X , y ) is the loss function for training ( e.g . cross entropy loss).
Revised: L ( X , y ) is the loss function for training ( e.g . cross entropy loss).

Original: The adversarial samples are computed as:
Revised: The adversarial samples are computed as:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: As FGSM is an one-step gradient-based method, it can suffer from sharp curvature near the data points, leading a false direction of ascent.
Revised: As FGSM is an one-step gradient-based method, it can suffer from sharp curvature near the data points, leading a false direction of ascent.

Original: Therefore, Tram`er et al.
Revised: Therefore, Tram`er et al.

Original: (2017) proposes to prepend FGSM by a random step to escape the non-smooth vicinity.
Revised: (2017) proposes to prepend FGSM by a random step to escape the non-smooth vicinity.

Original: The new method is called R+FGSM , deﬁned as follows, for parameters (cid:15) and (cid:15) 1 (where (cid:15) 1 < (cid:15) ):
Revised: The new method is called R+FGSM , deﬁned as follows, for parameters (cid:15) and (cid:15) 1 (where (cid:15) 1 < (cid:15) ):

Original: [Equation]
Revised: [Equation]

Original: [Equation] X (cid:48)
Revised: ) , where X (cid:48)

Original: =
Revised: =

Original: X

Original: [Equation]

Original:  (cid:15) [Equation] sign ( N ( 0 d , I d ))[Equation]
Revised: X + (cid:15) 1 ∗ sign ( N ( 0 d , I d ))

Original: [Equation]

Original: In our paper, we set (cid:15) 

Revised: In our paper, we set (cid:15)

Original: = (cid:15)/ 2 following Tram`er et al.
Revised: = (cid:15)/ 2 following Tram`er et al.

Original: Basic Iterative Method (BIM) & Projected Gradient Descend (PGD) Kurakin et al.
Revised: Basic Iterative Method (BIM) & Projected Gradient Descend (PGD) Kurakin et al.

Original: (2016) suggests a simple yet much stronger variant of FGSM by applying it multiple times iteratively with a small step size α .
Revised: (2016) suggests a simple yet much stronger variant of FGSM by applying it multiple times iteratively with a [Table] [Table] and

Revised: [Table]

Revised: Perturbation Strength ε

Original: The method is called BIM, deﬁned as:
Revised: [Table] Desired Actual DQ

Revised: [Table]

Revised: ) Noise increases with perturbation strength.

Revised: Quan- tization makes the slope deeper.

Revised: [Table]

Revised: N

Revised: [Figure]

Revised: een

Revised: [Figure]

Revised: l ean and

Revised: [Figure]

Revised: r

Revised: [Figure]

Revised: s 0.

Revised: Layer Index

Revised: [Figure]

Revised: N o r m a li z ed

Revised: D i s t an c e be t w een

Revised: C l ean and A d v e r s a r i a l

Revised: S a m p l e s 0.

Revised: Perturbation Strength ε

Revised: [Figure]

Revised: Desired

Revised: Actual

Revised: DQ

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: small step size α .

Revised: The method is called BIM, deﬁned as:

Original: [Equation]
Revised: [Equation]

Original:  where clip (cid:15) X means clipping the result image to be within the (cid:15) -ball of X .
Revised: (3) where clip (cid:15) X means clipping the result image to be within the (cid:15) -ball of X .

Original: In (Madry et al., 2017), the BIM is prepended by a random start as in R+FGSM method.
Revised: In (Madry et al., 2017), the BIM is prepended by a random start as in R+FGSM method.

Original: The resulting attack is called PGD, which proves to be a general ﬁrst-order attack.
Revised: The resulting attack is called PGD, which proves to be a general ﬁrst-order attack.

Original: In our experiments we used PGD for comprehensive experiments as it proves to be one of the strongest attack.
Revised: In our experiments we used PGD for comprehensive experiments as it proves to be one of the strongest attack.

Original: Unlike Madry et al.
Revised: Unlike Madry et al.

Original: (2017) that uses a ﬁxed (cid:15)
Revised: (2017) that uses a ﬁxed (cid:15)

Original: and α , we follow Kurakin et al.
Revised: and α , we follow Kurakin et al.

Original: ; Song et al.
Revised: ; Song et al.

Original: (2017) to use α = 1 and number of iterations of (cid:98) min( (cid:15) + 4 , 1 . 

Revised: (2017) to use α = 1 and number of iterations of (cid:98) min( (cid:15) + 4 , 1 .

Original: (cid:15) )
Revised: (cid:15) )

Original: (cid:99) , so that we can test the model’s robustness under different strength of attacks.
Revised: (cid:99) , so that we can test the model’s robustness under different strength of attacks.

Original: 2.2. D EFENSE METHODS
Revised: 2.2. D EFENSE METHODS

Original: Current defense methods either preprocess the adversarial samples to denoise the perturbation (Xu et al., 2017; Song et al., 2017;
Revised: Current defense methods either preprocess the adversarial samples to denoise the perturbation (Xu et al., 2017; Song et al., 2017;

Original: Liao et al., 2018) or making the network itself robust (Warde-Farley &[Table]
Revised: Liao et al., 2018) or making the network itself robust (Warde-Farley &

Original: N o r m a li z ed

Original: D i s t an c e be t w een

Original: C l ean and A d v e r s a r i a l

Original: S a m p l e

Original: [Table]

Original: [Table]

Original: Desired

Original: Actual

Original: DQ

Original: [Table]

Original: Noise increases with perturbation strength

Original: [Table]

Original: tization

Original: [Table]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: N o r m a li z ed

Original: D i s t an c e be t w een

Original: C l ean and A d v e r s a r i a l

Original: S a m p l e s 0.

Original: Layer Index

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: N o r m a li z ed

Original: D i s t an c e be t w een

Original: C l ean and A d v e r s a r i a l

Original: S a m p l e s 0.

Original: [Figure]

Original: [Figure]

Original: [Figure]  [Figure] Goodfellow, 2016; Papernot et al., 2016b; Madry et al., 2017; Kurakin et al., 2016; Goodfellow et al.;
Revised: Goodfellow, 2016; Papernot et al., 2016b; Madry et al., 2017; Kurakin et al., 2016; Goodfellow et al.;

Original: Tram`er et al., 2017).
Revised: Tram ` er et al., 2017).

Original: Here we introduced several defense methods related to our experiments.
Revised: Here we introduced several defense methods related to our experiments.

Original: Feature Squeezing Xu et al.
Revised: Feature Squeezing Xu et al.

Original: (2017) proposes to detect adversarial images by squeezing the input image.
Revised: (2017) proposes to detect adversarial images by squeezing the input image.

Original: Speciﬁcally, the image is processed with color depth bit reduction (5 bits for our experiments) and smoothed by a 2 × 2 median ﬁlter.
Revised: Speciﬁcally, the image is processed with color depth bit reduction (5 bits for our experiments) and smoothed by a 2 × 2 median ﬁlter.

Original: If the low resolution image is classiﬁed differently as the original image, then this image is detected as adversarial.
Revised: If the low resolution image is classiﬁed differently as the original image, then this image is detected as adversarial.

Original: Adversarial Training Adversarial training (Madry et al., 2017; Kurakin et al., 2016; Goodfellow et al. ; Tram ` er et al., 2017) is currently the strongest method for defense.
Revised: Adversarial Training Adversarial training (Madry et al., 2017; Kurakin et al., 2016; Goodfellow et al.; Tram`er et al., 2017) is currently the strongest method for defense.

Original: By augmenting the training set with adversarial samples, the network learns to classify adversarial samples correctly.
Revised: By augmenting the training set with adversarial samples, the network learns to classify adversarial samples correctly.

Original: As adversarial
Revised: As adversarial

Original: FGSM can easily lead to gradient masking effect (Papernot et al., 2016a), we study adversarial
Revised: FGSM can easily lead to gradient masking effect (Papernot et al., 2016a), we study adversarial

Original: R+FGSM as in (Tram`er et al., 2017).
Revised: R+FGSM as in (Tram`er et al., 2017).

Original: We also experimented with PGD training (Madry et al., 2017).
Revised: We also experimented with PGD training (Madry et al., 2017).

Original: Experiments show that above defense methods can be combined with our DQ method to further improve the robustness.
Revised: Experiments show that above defense methods can be combined with our DQ method to further improve the robustness.

Original: The robustness has been tested under the aforementioned attack methods.
Revised: The robustness has been tested under the aforementioned attack methods.

Original:  C ONVENTIONAL NN Q UANTIZATION IS N OT R OBUST
Revised:  C ONVENTIONAL NN Q UANTIZATION IS N OT R OBUST

Original: Conventional neural network quantization is more vulnerable to adversarial attacks.
Revised: Conventional neural network quantization is more vulnerable to adversarial attacks.

Original: We experimented with VGG-16 (Simonyan & Zisserman, 2014) and a Wide ResNet (Zagoruyko & Komodakis, 2016) of depth 28 and width 10 on CIFAR-10 (Krizhevsky & Hinton, 2009) dataset.
Revised: We experimented with VGG-16 (Simonyan & Zisserman, 2014) and a Wide ResNet (Zagoruyko & Komodakis, 2016) of depth 28 and width 10 on CIFAR-10 (Krizhevsky & Hinton, 2009) dataset.

Original: We followed the training protocol as in (Zagoruyko & Komodakis, 2016).
Revised: We followed the training protocol as in (Zagoruyko & Komodakis, 2016).

Original: Adversarial samples are generated with a
Revised: Adversarial samples are generated with a

Original: FGSM (Goodfellow et al.)
Revised: FGSM (Goodfellow et al.)

Original: attacker ( (cid:15) = 8 ) on the entire test set.
Revised: attacker ( (cid:15) = 8 ) on the entire test set.

Original: As in Figure 1, the clean image accuracy doesn’t signiﬁcantly drop until the model is quantized to 4 bits (Figure 1a).
Revised: As in Figure 1, the clean image accuracy doesn’t signiﬁcantly drop until the model is quantized to 4 bits (Figure 1a).

Original: However, under adversarial attack, even with 5-bit quantization, the accuracy drastically decreased by 25 . 3% and 9 .
Revised: However, under adversarial attack, even with 5-bit quantization, the accuracy drastically decreased by 25 . 3% and 9 .

Original: 2% respectively.
Revised: 2% respectively.

Revised: Although the full precision model’s accuracy has dropped, the quantized model’s

Revised: [Figure]

Original: Although the full precision model’s accuracy has dropped, the quantized model’s accuracy dropped much harder, showing that the conventional quantization method is not robust.
Revised: Vanilla Quantization Defensive Quantization [Figure]  accuracy dropped much harder, showing that the conventional quantization method is not robust.

Original: Clean image accuracy used to be the sole ﬁgure of merit to evaluate a quantized model.
Revised: Clean image accuracy used to be the sole ﬁgure of merit to evaluate a quantized model.

Original: We show that even when the quantized model has no loss of performance on clean images, it can be much more easily fooled compared to full-precision ones, thus raising security concerns.
Revised: We show that even when the quantized model has no loss of performance on clean images, it can be much more easily fooled compared to full-precision ones, thus raising security concerns.

Original: Input image quantization, i.e ., color bit depth reduction is an effective defense method (Xu et al., 2017).
Revised: Input image quantization, i.e ., color bit depth reduction is an effective defense method (Xu et al., 2017).

Original: Counter intuitively, it does not work when applied to hidden layers, and even make the robustness worse.
Revised: Counter intuitively, it does not work when applied to hidden layers, and even make the robustness worse.

Original: To understand the reason, we studied the effect of quantization w.r.t.
Revised: To understand the reason, we studied the effect of quantization w.r.t.

Original: different perturbation strength.
Revised: different perturbation strength.

Original: We ﬁrst randomly sample 128 images X from the test set of CIFAR-10, and generate corresponding adversarial samples X adv .
Revised: We ﬁrst randomly sample 128 images X from the test set of CIFAR-10, and generate corresponding adversarial samples X adv .

Original: The samples are then fed to the trained Wide
Revised: The samples are then fed to the trained Wide

Original: ResNet model.
Revised: ResNet model.

Original: To mimic different strength of activation perturbation, we vary the (cid:15) from 1 to 8.
Revised: To mimic different strength of activation perturbation, we vary the (cid:15) from 1 to 8.

Original: We inspected the activation after the ﬁrst convolutional layer f 1 (Conv + BN + ReLU6), denoted as A 1 = f 1 ( X ) and A adv= f 1 ( A adv ) .
Revised: We inspected the activation after the ﬁrst convolutional layer f 1 (Conv + BN + ReLU6), denoted as A 1 = f 1 ( X ) and A adv= f 1 ( A adv ) .

Original: To measure the inﬂuence of perturbation, we deﬁne a normalized distance between clean and perturbed activation as:
Revised: To measure the inﬂuence of perturbation, we deﬁne a normalized distance between clean and perturbed activation as:

Original: [Equation]
Revised: [Equation]

Original: We compare D ( A , A adv ) and D ( Quantize ( A ) , Quantize ( A adv )) , where Quantize indicates uniform quantization with 3 bits.
Revised: We compare D ( A , A adv ) and D ( Quantize ( A ) , Quantize ( A adv )) , where Quantize indicates uniform quantization with 3 bits.

Original: The results are shown in Figure 3a.
Revised: The results are shown in Figure 3a.

Original: We can see that only when (cid:15) is small, [Figure] [Figure]
Revised: We can see that only when (cid:15) is small, quantization helps to reduce the distance by removing small magnitude perturbations.

Original: 
Revised: The distance will be enlarged when (cid:15) is larger than 3.

Original: The above experiment explains the inferior robustness of the quantized model.
Revised: The above experiment explains the inferior robustness of the quantized model.

Original: We argue that such issue arises from the error ampliﬁcation effect (Liao et al., 2018), where the relative perturbed distance will be ampliﬁed when the adversarial samples are fed through the network.
Revised: We argue that such issue arises from the error ampliﬁcation effect (Liao et al., 2018), where the relative perturbed distance will be ampliﬁed when the adversarial samples are fed through the network.

Original: As illustrated in Figure 4, the perturbation applied to the input image has very small magnitude compared to the image itself ( ± 8 versus 0 − 255 ), corresponding to the left zone of Figure 3a (desired), where quantization helps to denoise the perturbation.
Revised: As illustrated in Figure 4, the perturbation applied to the input image has very small magnitude compared to the image itself ( ± 8 versus 0 − 255 ), corresponding to the left zone of Figure 3a (desired), where quantization helps to denoise the perturbation.

Original: Nevertheless, the difference in activation is ampliﬁed as the inference carries on.
Revised: Nevertheless, the difference in activation is ampliﬁed as the inference carries on.

Original: If the perturbation after ampliﬁcation is large enough, the situation corresponds to the right zone (actual) of Figure 3a, where quantization further increases the normalized distance.
Revised: If the perturbation after ampliﬁcation is large enough, the situation corresponds to the right zone (actual) of Figure 3a, where quantization further increases the normalized distance.

Original: Such phenomenon is also observed in the quantized VGG-16.
Revised: Such phenomenon is also observed in the quantized VGG-16.

Original: We plot the normalized distance of each convolutional layer’s input in Figure 3b.
Revised: We plot the normalized distance of each convolutional layer’s input in Figure 3b.

Original: The fewer bits in the quantized model, the more severe the ampliﬁcation effect.
Revised: The fewer bits in the quantized model, the more severe the ampliﬁcation effect.

Original:  D EFENSIVE Q UANTIZATION
Revised:  D EFENSIVE Q UANTIZATION

Original: Given the robustness limitation of conventional quantization technique, we propose Defensive
Revised: Given the robustness limitation of conventional quantization technique, we propose Defensive

Original: Quantization (DQ) to defend the adversarial examples for quantized models.
Revised: Quantization (DQ) to defend the adversarial examples for quantized models.

Original: DQ suppresses the noise ampliﬁcation effect, keeping the magnitude of the noise small, so that we can arrive at the left zone (Figure 3a, desired) where quantization helps robustness instead of making it worse.
Revised: DQ suppresses the noise ampliﬁcation effect, keeping the magnitude of the noise small, so that we can arrive at the left zone (Figure 3a, desired) where quantization helps robustness instead of making it worse.

Original: We control the neural network’s Lipschitz constant (Szegedy et al., 2013; Bartlett et al., 2017; Cisse et al., 2017) to suppress network’s ampliﬁcation effect.
Revised: We control the neural network’s Lipschitz constant (Szegedy et al., 2013; Bartlett et al., 2017; Cisse et al., 2017) to suppress network’s ampliﬁcation effect.

Original: Lipschitz constant describes: when input changes, how much does the output change correspondingly.
Revised: Lipschitz constant describes: when input changes, how much does the output change correspondingly.

Original: For a function f : X → Y , if it satisﬁes
Revised: For a function f : X → Y , if it satisﬁes

Original: [Equation]
Revised: [Equation]

Original: for a real-valued k ≥ 0 and some metrics D X and D Y , then we call f Lipschitz continuous and k is the known as the Lipschitz constant of f .
Revised: for a real-valued k ≥ 0 and some metrics D X and D Y , then we call f Lipschitz continuous and k is the known as the Lipschitz constant of f .

Original: If we consider a network f with clean inputs x 1 = X and corresponding adversarial inputs x 2 = X adv , the error ampliﬁcation effect can be controlled if we  have a small Lipschitz constant k (in optimal situation we can have k ≤ 1 ).
Revised: If we consider a network f with clean inputs x 1 = X and corresponding adversarial inputs x 2 = X adv , the error ampliﬁcation effect can be controlled if we [Table] have a small Lipschitz constant k (in optimal situation we can have k ≤ 1 ).

Original: In such case, the error introduced by adversarial perturbation will not be ampliﬁed, but reduced.
Revised: In such case, the error introduced by adversarial perturbation will not be ampliﬁed [Table] but reduced.

Original: Speciﬁcally, we consider a feed-forward network composed of a serial of functions:
Revised: Speciﬁcally, we consider a feed-forward network composed of a serial of functions:

Original: [Equation]
Revised: [Equation]

Original: where φ i can be a linear layer, convolutional layer, pooling, activation functions, etc .
Revised: where φ i can be a linear layer, convolutional layer, pooling, activation functions, etc .

Original: Denote the
Revised: Denote the

Original: Lipschitz constant of a function f as Lip ( f ) , then for the above network we have
Revised: Lipschitz constant of a function f as Lip ( f ) , then for the above network we have

Original: [Equation]
Revised: [Equation]

Original: As the Lipschitz constant of the network is the product of its individual layers’ Lipschitz constants,
Revised: As the Lipschitz constant of the network is the product of its individual layers’ Lipschitz constants,

Original: Lip ( f ) can grow exponentially if Lip ( φ i ) > 1 .
Revised: Lip ( f ) can grow exponentially if Lip ( φ i ) > 1 .

Original: This is the common case for normal network training (Cisse et al., 2017), and thus the perturbation will be ampliﬁed for such a network.
Revised: This is the common case for normal network training (Cisse et al., 2017), and thus the perturbation will be ampliﬁed for such a network.

Original: Therefore, to keep the Lipschitz constant of the whole network small, we need to keep the Lipschitz constant of each layer Lip ( φ i ) ≤ 1 .
Revised: Therefore, to keep the Lipschitz constant of the whole network small, we need to keep the Lipschitz constant of each layer Lip ( φ i ) ≤ 1 .

Original: We call a network with Lip ( φ i ) ≤ 1 , ∀ i = 1 , ..., L a non-expansive network.
Revised: We call a network with Lip ( φ i ) ≤ 1 , ∀ i = 1 , ..., L a non-expansive network.

Original: We describe a regularization term to keep the Lipschitz constant small.
Revised: We describe a regularization term to keep the Lipschitz constant small.

Original: Let us ﬁrst consider linear layers with weight W ∈ R c out × c in under || · || 2 norm.
Revised: Let us ﬁrst consider linear layers with weight W ∈ R c out × c in under || · || 2 norm.

Original: The Lipschitz constant is by deﬁnition the spectral norm of W : ρ ( W ) , i.e ., the maximum singular value of W .
Revised: The Lipschitz constant is by deﬁnition the spectral norm of W : ρ ( W ) , i.e ., the maximum singular value of W .

Original: Computing the singular values of each weight matrix is not computationally feasible during training.
Revised: Computing the singular values of each weight matrix is not computationally feasible during training.

Original: Luckily, if we can keep the weight matrix row orthogonal, the singular values are by nature equal to 1, which meets[Table] our non-expansive requirements.
Revised: Luckily, if we can keep the weight matrix row orthogonal, the singular values are by nature equal to 1, which meets our non-expansive requirements.

Original: Therefore we transform the problem of keeping ρ ( W ) ≤ 1 into keeping W T W ≈ I , where I is the identity matrix.
Revised: Therefore we transform the problem of keeping ρ ( W ) ≤ 1 into keeping W T W ≈ I , where I is the identity matrix.

Original: Naturally, we introduce a regularization term || W T W −
Revised: Naturally, we introduce a regularization term || W T W −

Original: I || , where I is the identity matrix.
Revised: I || , where I is the identity matrix.

Original: Following (Cisse et al., 2017), for convolutional layers with weight W ∈ R c out × c in × k × k , we can view it as a two-dimension matrix of shape
Revised: Following (Cisse et al., 2017), for convolutional layers with weight W ∈ R c out × c in × k × k , we can view it as a two-dimension matrix of shape

Original: W ∈ R c out × ( c in kk ) and apply the same regularization.
Revised: W ∈ R c out × ( c in kk ) and apply the same regularization.

Original: The ﬁnal optimization objective is:
Revised: The ﬁnal optimization objective is:

Original: [Equation]
Revised: [Equation]

Original: where L CE is the original cross entropy loss and W denotes all the weight matrices of the neural network.
Revised: where L CE is the original cross entropy loss and W denotes all the weight matrices of the neural network.

Original: β is the weighting to adjust the relative importance.
Revised: β is the weighting to adjust the relative importance.

Original: The above discussion is based on simple feed forward networks.
Revised: The above discussion is based on simple feed forward networks.

Original: For ResNets in our experiments, we also follow Cisse et al.
Revised: For ResNets in our experiments, we also follow Cisse et al.

Original: (2017) to modify the aggregation layer as a convex combination of their inputs, where the 2 coefﬁcients are updated using speciﬁc projection
Revised: (2017) to modify the aggregation layer as a convex combination of their inputs, where the 2 coefﬁcients are updated using speciﬁc projection

Original: algorithm (see (Cisse et al., 2017) for details).
Revised: algorithm (see (Cisse et al., 2017) for details).

Original: Our Defensive Quantization is illustrated in Figure 2.
Revised: Our Defensive Quantization is illustrated in Figure 2.

Original: The key part is the regularization term, which suppress the noise ampliﬁcation effect by regularizing the Lipschitz constant.
Revised: The key part is the regularization term, which suppress the noise ampliﬁcation effect by regularizing the Lipschitz constant.

Original: As a result, the  perturbation at each layer is kept within a certain range, the adversarial noise won’t propagate.
Revised: As a result, the [Figure] perturbation at each layer is kept within a certain range, the adversarial noise won’t propagate.

Original: Our method not only ﬁxes the drop of robustness induced by quantization, but also takes quantization as a defense method to further increase the robustness.
Revised: Our method not only ﬁxes the drop of robustness induced by quantization, but also takes quantization as a defense method to further increase the robustness.

Original: Therefore it’s named Defensive Quantization.
Revised: Therefore it’s named Defensive Quantization.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Our experiments demonstrate the following advantages of Defensive Quantization.
Revised: Our experiments demonstrate the following advantages of Defensive Quantization.

Original: First, DQ can retain the robustness of a model when quantized with low-bit.
Revised: First, DQ can retain the robustness of a model when quantized with low-bit.

Original: Second, DQ is a general and effective defense method under various scenarios, thus can be combined with other defensive techniques to further advance state-of-the-art robustness.
Revised: Second, DQ is a general and effective defense method under various scenarios, thus can be combined with other defensive techniques to further advance state-of-the-art robustness.

Original: Third, as a by-product, DQ can also improve the accuracy of training quantized models on clean images without attacks, since it limits the dynamic range.
Revised: Third, as a by-product, DQ can also improve the accuracy of training quantized models on clean images without attacks, since it limits the dynamic range.

Original: 5. F IXING R OBUSTNESS D ROP
Revised: 5. F IXING R OBUSTNESS D ROP

Original: Setup: We conduct experiments with Wide ResNet (Zagoruyko & Komodakis, 2016) of 28 × 10 on the CIFAR-10 dataset (Krizhevsky & Hinton, 2009). ReLU6 based quantization is inserted for every convolutional and linear layers, with number of bits ranging from 1 to 5.
Revised: Setup: We conduct experiments with Wide ResNet (Zagoruyko & Komodakis, 2016) of 28 × 10 on the CIFAR-10 dataset (Krizhevsky & Hinton, 2009) using ReLU6 based activation quantization, with number of bits ranging from 1 to 5.

Original: All the models are trained following (Zagoruyko & Komodakis, 2016) with momentum SGD for 200 epochs.
Revised: All the models are trained following (Zagoruyko & Komodakis, 2016) with momentum SGD for 200 epochs.

Original: The adversarial samples are generated using FGSM attacker with (cid:15) = 8 .
Revised: The adversarial samples are generated using FGSM attacker with (cid:15) = 8 .

Original: Result:
Revised: Result:

Original: The results are presented in Table 1.
Revised: The results are presented in Table 1.

Original: For vanilla models, though the adversarial robustness increases with the number of bits, i.e ., the models closer to full-precision one has better robustness, the best quantized model still has inferior robustness by − 9 .
Revised: For vanilla models, though the adversarial robustness increases with the number of bits, i.e ., the models closer to full-precision one has better robustness, the best quantized model still has inferior robustness by − 9 .

Original: While with our Defensive Quantization, the quantized models have better robustness than full-precision counterparts.
Revised: While with our Defensive Quantization, the quantized models have better robustness than full-precision counterparts.

Original: The robustness is better when the number of bits are small, since it can de-noise larger adversarial perturbations.
Revised: The robustness is better when the number of bits are small, since it can de-noise larger adversarial perturbations.

Original: We also ﬁnd that the robustness is generally increasing as β gets larger, since the regularization of
Revised: We also ﬁnd that the robustness is generally increasing as β gets larger, since the regularization of

Original: Lipschitz constant itself keeps the noise smaller at later layers.
Revised: Lipschitz constant itself keeps the noise smaller at later layers.

Original: At the same time, the quantized[Figure] models consistently achieve better robustness.
Revised: At the same time, the quantized models consistently achieve better robustness.

Original: The robustness of quantized model also increases with β .
Revised: The robustness of quantized model also increases with β .

Original: We conduct a detailed analysis of the effect of β in Section
Revised: We conduct a detailed analysis of the effect of β in Section

Original: B.
Revised: B.

Original: The conclusion is: (1) conventional quantized models are less robust.
Revised: The conclusion is: (1) conventional quantized models are less robust.

Original: (2) Lipschitz regularization makes the model robust.
Revised: (2) Lipschitz regularization makes the model robust.

Original: (3) Lipschitz regularization + quantization makes model even more robust.
Revised: (3) Lipschitz regularization + quantization makes model even more robust.

Original: As shown in (Athalye et al., 2018), many of the defense methods actually lead to obfuscated gradient, providing a false sense of security.
Revised: As shown in (Athalye et al., 2018), many of the defense methods actually lead to obfuscated gradient, providing a false sense of security.

Original: Therefore it is important to check the model’s robustness under black-box attack.
Revised: Therefore it is important to check the model’s robustness under black-box attack.

Original: We separately trained a substitute VGG-16 model on the same dataset to generate adversarial samples, as it was proved to have the best transferability (Su et al., 2018).
Revised: We separately trained a substitute VGG-16 model on the same dataset to generate adversarial samples, as it was proved to have the best transferability (Su et al., 2018).

Original: The results are presented in Figure 5.
Revised: The results are presented in Figure 5.

Original: Trends of white-box and black-box attack are consistent.
Revised: Trends of white-box and black-box attack are consistent.

Original: Vanilla quantization leads to inferior black-box robustness, while with our method can further improve the models’ robustness.
Revised: Vanilla quantization leads to inferior black-box robustness, while with our method can further improve the models’ robustness.

Original: As the robustness gain is consistent for both white-box and black-box setting, our method does not suffer from gradient masking.
Revised: As the robustness gain is consistent for both white-box and black-box setting, our method does not suffer from gradient masking.

Original: 5. D EFEND WITH D EFENSIVE Q UANTIZATION
Revised: 5. D EFEND WITH D EFENSIVE Q UANTIZATION

Original: In this section, we show that we can combine Defensive Quantization with other defense techniques to achieve state-of-the-art robustness.
Revised: In this section, we show that we can combine Defensive Quantization with other defense techniques to achieve state-of [Table]

Original: Setup: We conducted experiments on the Street View House Number dataset (SVHN)
Revised: Setup: We conducted experiments on the Street View House Number dataset (SVHN)

Original: (Netzer et al., 2011) and CIFAR-10 dataset (Krizhevsky & Hinton, 2009).
Revised: (Netzer et al., 2011) and CIFAR-10 dataset (Krizhevsky & Hinton, 2009).

Original: Since adversarial training is time consuming, we only use the ofﬁcial training set for experiments.
Revised: Since adversarial training is time consuming, we only use the ofﬁcial training set for experiments.

Original: CIFAR-10 is another widely used dataset containing 50,000 training samples and 10,000 testing samples of size 32 × 32 .
Revised: CIFAR-10 is another widely used dataset containing 50,000 training samples and 10,000 testing samples of size 32 × 32 .

Original: For both datasets, we divide the pixel values by 255 as a pre-processing step.
Revised: For both datasets, we divide the pixel values by 255 as a pre-processing step.

Original: Following (Athalye et al., 2018; Cisse et al., 2017; Madry et al., 2017), we used Wide
Revised: Following (Athalye et al., 2018; Cisse et al., 2017; Madry et al., 2017), we used Wide

Original: ResNet (Zagoruyko & Komodakis, 2016) models in our experiments as it is considered as the standard model on the dataset.
Revised: ResNet (Zagoruyko & Komodakis, 2016) models in our experiments as it is considered as the standard model on the dataset.

Original: We used depth 28 and widening factor 10 for CIFAR-10, and depthand widening factor 4 for SVHN.
Revised: We used depth 28 and widening factor 10 for CIFAR-10, and depthand widening factor 4 for SVHN.

Original: We followed the training protocol in (Zagoruyko & Komodakis, 2016) that uses a SGD optimizer with momentum=0.9.
Revised: We followed the training protocol in (Zagoruyko & Komodakis, 2016) that uses a SGD optimizer with momentum=0.9.

Original: For CIFAR-10, the model is trained forepochs with initial learning rate 0.1, decayed by a factor of 0.2 at 60, 120 and 160 epochs.
Revised: For CIFAR-10, the model is trained forepochs with initial learning rate 0.1, decayed by a factor of 0.2 at 60, 120 and 160 epochs.

Original: For SVHN dataset, the model is trained for 160 epochs, with initial learning rate 0.01, decayed by 0.1 at 80 and 120 epochs.
Revised: For SVHN dataset, the model is trained for 160 epochs, with initial learning rate 0.01, decayed by 0.1 at 80 and 120 epochs.

Original: For DQ, we used bit=1 and β = 2e-3 as it offers the best robustness (see Section B).
Revised: For DQ, we used bit=1 and β = 2e-3 as it offers the best robustness (see Section B).

Original: We combine DQ with other defense methods to further boost the robustness.
Revised: We combine DQ with other defense methods to further boost the robustness.

Original: For Feature Squeezing (Xu et al., 2017), we used 5 bit for image color reduction, followed by a 2 × 2 median ﬁlter.
Revised: For Feature Squeezing (Xu et al., 2017), we used 5 bit for image color reduction, followed by a 2 × 2 median ﬁlter.

Original: As adversarial FGSM training leads to gradient masking issue (Tram`er et al., 2017)
Revised: As adversarial FGSM training leads to gradient masking issue (Tram ` er et al., 2017)

Original: (see A.2 for our experiment), we used the variant adversarial R+FGSM training.
Revised: (see A.2 for our experiment), we used the variant adversarial R+FGSM training.

Original: To avoid over-ﬁtting into certain (cid:15) , we randomly sample (cid:15) using (cid:15) (cid:48) ∼ N (0 , δ ) , (cid:15) =
Revised: To avoid over-ﬁtting into certain (cid:15) , we randomly sample (cid:15) using (cid:15) (cid:48) ∼ N (0 , δ ) , (cid:15) =

Original: clip [0 , 2 δ ] ( abs ( (cid:15) (cid:48) )) .
Revised: clip [0 , 2 δ ] ( abs ( (cid:15) (cid:48) )) .

Original: Speciﬁcally we used δ =to cover (cid:15) from 0-16.
Revised: Speciﬁcally we used δ =to cover (cid:15) from 0-16.

Original: During test time, the (cid:15) is set to a ﬁxed value (2/8/16).
Revised: During test time, the (cid:15) is set to a ﬁxed value (2/8/16).

Original: We also conducted adversarial PGD training.
Revised: We also conducted adversarial PGD training.

Original: Following (Kurakin et al., 2016; Song et al., 2017), during training we sample random (cid:15) as in R+FGSM setting, and generate adversarial samples using step size 1 and number of iterations (cid:98) min( (cid:15) + 4 , 1 . 25 (cid:15) )
Revised: Following (Kurakin et al., 2016; Song et al., 2017), during training we sample random (cid:15) as in R+FGSM setting, and generate adversarial samples using step size 1 and number of iterations (cid:98) min( (cid:15) + 4 , 1 . 25 (cid:15) )

Original: (cid:99) .
Revised: (cid:99) .

Original: Result:
Revised: Result:

Original: The results are presented in Table 2 and Table 3, where (B) indicates black-box attack with a seperately trained VGG-16 model.
Revised: The results are presented in Table 2 and Table 3, where (B) indicates black-box attack with a seperately trained VGG-16 model.

Original: The bold number indicates the best result in its column.
Revised: The bold number indicates the best result in its column.

Original: We observe that for all normal training, feature squeezing and adversarial training settings, our DQ method can further improve the model’s robustness.
Revised: We observe that for all normal training, feature squeezing and adversarial training settings, our DQ method can further improve the model’s robustness.

Original: Among all the defenses, adversarial training provides the best performance against various attacks, epecially adversarial R+FGSM training.
Revised: Among all the defenses, adversarial training provides the best performance against various attacks, epecially adversarial R+FGSM training.

Original: While white box PGD attack is generally the strongest attack in our experiments.
Revised: While white box PGD attack is generally the strongest attack in our experiments.

Original: Our DQ method also[Table] [Table] consistently improves the black-box robustness and there is no sign of gradient masking.
Revised: Our DQ method also consistently improves the black-box robustness and there is no sign of gradient masking.

Original: Thus DQ proves to be an effective defense for various white-box and black-box attacks [Table] [Table]
Revised: Thus DQ proves to be an effective defense for various white-box and black-box attacks.

Original: 5. I MPROVE THE T RAINING OF Q UANTIZED M ODELS
Revised: 5. I MPROVE THE T RAINING OF Q UANTIZED M ODELS

Original: As a by-product of our method, Defensive Quantization can even improve the accuracy of quantized models on clean images without attack, making it a beneﬁcial drop-in substitute for normal quantization procedures.
Revised: As a by-product of our method, Defensive Quantization can even improve the accuracy of quantized models on clean images without attack, making it a beneﬁcial drop-in substitute for normal quantization procedures.

Original: Due to conventional quantization method’s ampliﬁcation effect, the distribution of activation can step over the truncation boundary (0-6 for ReLU6, 0-1 for ReLU1), which makes the optimization difﬁcult.
Revised: Due to conventional quantization method’s ampliﬁcation effect, the distribution of activation can step over the truncation boundary (0-6 for ReLU6, 0-1 for ReLU1), which makes the optimization difﬁcult.

Original: DQ explicitly add a regularization to shrink the dynamic range of activation, so that it is ﬁtted within the truncation range.
Revised: DQ explicitly add a regularization to shrink the dynamic range of activation, so that it is ﬁtted within the truncation range.

Original: To demonstrate our hypothesis, we experimented with
Revised: To demonstrate our hypothesis, we experimented with

Original: ResNet and CIFAR-10.
Revised: ResNet and CIFAR-10.

Original: We quantized the activation with 4-bit (because NVIDIA recently introduced
Revised: We quantized the activation with 4-bit (because NVIDIA recently introduced

Original: INT4 in Turing architecture) using ReLU6 and ReLU1 respectively.
Revised: INT4 in Turing architecture) using ReLU6 and ReLU1 respectively.

Original: Vanilla quantization and DQ training are conducted for comparison.
Revised: Vanilla quantization and DQ training are conducted for comparison.

Original: As shown in Table 4, with vanilla quantization, ReLU1 model has around 1% worse accuracy than ReLU6 model, although they are mathematically equal if we multiply the previous BN scaling by 1/6 and next convolution weight by 6.
Revised: As shown in Table 4, with vanilla quantization, ReLU1 model has around 1% worse accuracy than ReLU6 model, although they are mathematically equal if we [Table] [Table] the previous BN scaling by 1/6 and next convolution weight by 6.

Original: It demonstrates that improper truncation function and range will lead to training difﬁculty.
Revised: It demonstrates that improper truncation function and range will lead to training difﬁculty.

Original: While with DQ training, both model has improved accuracy compared to vanilla quantization, and the gap between ReLU1 model and ReLU6 model is ﬁlled, making quantization easier regardless of truncation range.
Revised: While with DQ training, both model has improved accuracy compared to vanilla quantization, and the gap between ReLU1 model and ReLU6 model is ﬁlled, making quantization easier regardless of truncation range.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work, we aim to raise people’s awareness about the security of the quantized neural networks, which is widely deployed in GPU/TPU/FPGAs, and pave a possible direction to bridge two important areas in deep learning: efﬁciency and robustness.
Revised: In this work, we aim to raise people’s awareness about the security of the quantized neural networks, which is widely deployed in GPU/TPU/FPGAs, and pave a possible direction to bridge two important areas in deep learning: efﬁciency and robustness.

Original: We connect these two domains by designing a novel
Revised: We connect these two domains by designing a novel

Original: Defensive Quantization (DQ) module to defend adversarial attacks while maintain the efﬁciency.
Revised: Defensive Quantization (DQ) module to defend adversarial attacks while maintain the efﬁciency.

Original: Experimental results on two datasets validate that the new quantization method can make the deep learning models be safely deployed on mobile devices.
Revised: Experimental results on two datasets validate that the new quantization method can make the deep learning models be safely deployed on mobile devices.

Revised: A CKNOWLEDGMENTS

Revised: We acknowledge Google and Amazon providing us the cloud computing resources.

Original: A G RADIENT M ASKING OF O THER M ETHODS A. T ANH - BASED A CTIVATION Q UANTIZATION L EADS TO F AKE S ECURITY
Revised: A G RADIENT M ASKING OF O THER M ETHODS A. T ANH - BASED A CTIVATION Q UANTIZATION L EADS TO F AKE S ECURITY

Original: Rakin et al.
Revised: Rakin et al.

Original: (2018) tried to use activation quantization as a defense method against adversarial samples.
Revised: (2018) tried to use activation quantization as a defense method against adversarial samples.

Original: According to their experiments, simply Tanh based activation quantization can greatly improve the model’s robustness under PGD adversarial training setting.
Revised: According to their experiments simply Tanh based activation quantization can greatly improve the model’s robustness under PGD adversarial training setting.

Original: We reproduced their experiments by training a ResNet-18 model with PGD adversarial training on CIFAR-10 following Madry et al.
Revised: We reproduced their experiments by training a ResNet-18 model with PGD adversarial training on CIFAR-10 following Madry et al.

Original: To make a comparison, we also trained a full precision model and a ReLU6 quantized model following same setting.
Revised: To make a comparison we also trained a full precision model and a ReLU6 quantized model following same setting.

Original: All the quantized models use bit=2.
Revised: All the quantized models use bit=2.

Original: We tested the trained model using PGD attack by Madry et al.
Revised: We tested the trained model using PGD attack by Madry et al.

Original: (2017) under both white-box and black-box setting.
Revised: (2017) under both white-box and black-box setting.

Original: We use a VGG-16 model separately trained with PGD adversarial training as the black-box attacker.
Revised: We use a VGG-16 model separately trained with PGD adversarial training as the black-box attacker.

Original: The results are provided in Table
Revised: The results are provided in Table

Original: [Table]
Revised: [Table]

Original: Our white-box result is consistent with (Rakin et al., 2018), where Tanh based quantizaion with PGD training gives much higher white-box accuracy compared to the full precision model.
Revised: Our white-box result is consistent with (Rakin et al., 2018), where Tanh based quantizaion with PGD training gives much higher white-box accuracy compared to the full precision model.

Original: However, we can see that the black-box robustness decreases.
Revised: However, we can see that the black-box robustness decreases.

Original: Worse still, the black-box attack successful rate is even lower than white-box, which is abnormal since black-box attack is generally weaker than white-box.
Revised: Worse still, the black-box attack successful rate is even lower than white-box, which is abnormal since black-box attack is generally weaker than white-box.

Original: This phenomenon indicates severe gradient masking problem (Papernot et al., 2016a;
Revised: This phenomenon indicates severe gradient masking problem (Papernot et al., 2016a;

Original: Athalye et al., 2018), which gives a false sense of security.
Revised: Athalye et al., 2018), which gives a false sense of security.

Original: As a comparison, we also trained a ReLU quantized model.
Revised: As a comparison, we also trained a ReLU quantized model.

Original: With ReLU6 quantization, there is no sign of gradient masking, nor improvement in robustness, indicating that gradient masking problem majorly comes from Tanh activation function.
Revised: With ReLU6 quantization, there is no sign of gradient masking, nor improvement in robustness, indicating that gradient masking problem majorly comes from Tanh activation function.

Original: Actually, ReLU6 quantized model has slightly worse robustness.
Revised: Actually, ReLU6 quantized model has slightly worse robustness.

Original: Therefore we conclude that simple quantization cannot help to improve robustness.
Revised: Therefore we conclude that simple quantization cannot help to improve robustness.

Original: Instead, it leads to inferior robustness.
Revised: Instead, it leads to inferior robustness.

Original: A. A DVERSARIAL FGSM T RAINING C AUSES G RADIENT M ASKING
Revised: A. A DVERSARIAL FGSM T RAINING C AUSES G RADIENT M ASKING

Original: Here we demonstrate that FGSM adversarial training leads to signiﬁcant gradient masking problem, while R+FGSM ﬁxes such issues.
Revised: Here we demonstrate that FGSM adversarial training leads to signiﬁcant gradient masking problem, while R+FGSM ﬁxes such issues.

Original: We trained a Wide ResNet using adversarial FGSM and adversarial
Revised: We trained a Wide ResNet using adversarial FGSM and adversarial

Original: R+FGSM respectively.
Revised: R+FGSM respectively.

Original: Then the model is tested using FGSM with (cid:15) = 8 under white-box and black-box setting (VGG-16 as substitute).
Revised: Then the model is tested using FGSM with (cid:15) = 8 under white-box and black-box setting (VGG-16 as substitute).

Original: The results are shown in Table
Revised: The results are shown in Table

Original: [Table]
Revised: [Table]

Original: We can clearly see the same gradient masking effect.
Revised: We can clearly see the same gradient masking effect.

Original: For adversarial FGSM training, it gives much higher white box robustness than R+FGSM adversarial training, while the black-box robustness is much worse.
Revised: For adversarial FGSM training, it gives much higher white box robustness than R+FGSM adversarial training, while the black-box robustness is much worse.

Original: To avoid gradient masking, we thus use R+FGSM adversarial training instead.
Revised: To avoid gradient masking, we thus use R+FGSM adversarial training instead.

Original: B H YPER -P ARAMETERS : β S TUDY
Revised: B H YPER -P ARAMETERS : β S TUDY

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 
Revised:

Original: As observed in Section 5.
Revised: As observed in Section 5.

Original: [Figure]
Revised: [Figure]

Original: the adversarial robustness of the model generally increases as β gets larger.
Revised: the adversarial robustness of the model generally increases as β gets larger.

Original: Here we aim to ﬁnd what is the optimal β for our experiment.
Revised: Here we aim to ﬁnd what is the optimal β for our experiment.

Original: We took 4-bit quantized Wide
Revised: We took 4-bit quantized Wide

Original: ResNet for example.
Revised: ResNet for example.

Original: As shown in Figure
Revised: As shown in Figure

Original: [Figure]
Revised: [Figure]

Original: the adversarial accuracy ﬁrst gets larger as β increases, and slowly goes down afterwards, reaching peak performance at β = 0 .
Revised: the adversarial accuracy ﬁrst gets larger as β increases, and slowly goes down afterwards, reaching peak performance at β = 0 .

Original: The clean accuracy is more stable.
Revised: The clean accuracy is more stable.

Original: At the ﬁrst stage, as regularization gets stronger, the effect of noise amplify is suppressed more.
Revised: At the ﬁrst stage, as regularization gets stronger, the effect of noise amplify is suppressed more.

Original: While for the second stage, the training suffers from a too strong regularization.
Revised: While for the second stage, the training suffers from a too strong regularization.

Original: Therefore, we used a β = 0 .
Revised: Therefore, we used a β = 0 .

Original: 002 for our experiments unless speciﬁed.
Revised: 002 for our experiments unless speciﬁed.

Original: C V ISUALIZE S AMPLES AND P REDICTIONS
Revised: C V ISUALIZE S AMPLES AND P REDICTIONS

Original: We visualize some of the clean and adversarial samples from CIFAR-10 test set, and the corresponding prediction for full precision model ( FP ), vanilla 4-bit quantized model ( VQ ) and our 4-bit Defensive
Revised: We visualize some of the clean and adversarial samples from CIFAR-10 test set, and the corresponding prediction for full precision model ( FP ), vanilla 4-bit quantized model ( VQ ) and our 4-bit Defensive

Original: Quantization model ( DQ ), as 4-bit quantization is now now supported by Tensor Cores with NVIDIA’s
Revised: Quantization model ( DQ ), as 4-bit quantization is now now supported by Tensor Cores with NVIDIA’s

Original: Turing Architecture (NVIDIA).
Revised: Turing Architecture (NVIDIA).

Original: Predicted class and probability is provided.
Revised: Predicted class and probability is provided.

Original: Compared to FP and DQ,
Revised: Compared to FP and DQ,

Original: VQ has worse robustness by misclassifying more adversarial samples (sample 1, 2, 4, 6).
Revised: VQ has worse robustness by misclassifying more adversarial samples (sample 1, 2, 4, 6).

Original: Our DQ model enjoys better robustness than FP model in two aspects: 1.
Revised: Our DQ model enjoys better robustness than FP model in two aspects: 1.

Original: our DQ model succeeds to defend some of the attacks when FP failed (sample 5, 7); 2.
Revised: our DQ model succeeds to defend some of the attacks when FP failed (sample 5, 7); 2.

Original: our DQ model has a better conﬁdence for true label compared to FP model when both succeeds to defend (sample 1, 4, 6).
Revised: our DQ model has a better conﬁdence for true label compared to FP model when both succeeds to defend (sample 1, 4, 6).

Original: Even when all models fail to defend, our DQ model has the lowest conﬁdence for the misclassiﬁed class (sample 3, 8).
Revised: Even when all models fail to defend, our DQ model has the lowest conﬁdence for the misclassiﬁed class (sample 3, 8).

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]
Revised: [Figure]

Original: Clean
Revised: Clean

Original: Adversarial 
Revised: Adversarial