Title: DEFENSIVE QUANTIZATION: WHEN EFFICIENCY MEETS ROBUSTNESS

Abstract: Neural network quantization is becoming an industry standard to efficiently deploy deep learning models on hardware platforms, such as CPU, GPU, TPU, and FPGAs. However, we observe that the conventional quantization approaches are vulnerable to adversarial attacks. This paper aims to raise people's awareness about the security of the quantized models, and we designed a novel quantization methodology to jointly optimize the efficiency and robustness of deep learning models. We first conduct an empirical study to show that vanilla quantization suffers more from adversarial attacks. We observe that the inferior robustness comes from the error amplification effect, where the quantization operation further enlarges the distance caused by amplified noise. Then we propose a novel Defensive Quantization (DQ) method by controlling the Lipschitz constant of the network during quantization, such that the magnitude of the adversarial noise remains non-expansive during inference. Extensive experiments on CIFAR-10 and SVHN datasets demonstrate that our new quantization method can defend neural networks against adversarial examples, and even achieves superior robustness than their fullprecision counterparts, while maintaining the same hardware efficiency as vanilla quantization approaches. As a by-product, DQ can also improve the accuracy of quantized models without adversarial attack.

1. INTRODUCTION:

Neural network quantization (Han et al., 2015; Zhu et al., 2016; Jacob et al., 2017) is a widely used technique to reduce the computation and memory costs of neural networks, facilitating efficient deployment. It has become an industry standard for deep learning hardware. However, we find that the widely used vanilla quantization approaches suffer from unexpected issues -the quantized model is more vulnerable to adversarial attacks (Figure 1 ). Adversarial attack is consist of subtle perturbations on the input images that causes the deep learning models to give incorrect labels (Szegedy et al., 2013; Goodfellow et al.) . Such perturbations are hardly detectable by human eyes but can easily fool neural networks. Since quantized neural networks are widely deployed in many safety-critical scenarios, e.g., autonomous driving (Amodei et al., 2016) , the potential security risks cannot be neglected. The efficiency and latency in such applications are also important, so we need to jointly optimize them.

The fact that quantization leads to inferior adversarial robustness is counter intuitive, as small perturbations should be denoised with low-bit representations. Recent work (Xu et al., 2017 ) also demonstrates that quantization on input image space , i.e. color bit depth reduction, is quite effective to defend adversarial examples. A natural question then rises, why the quantization operator is yet effective when applied to intermediate DNN layers? We analyze that such issue is caused by the error amplification effect of adversarial perturbation (Liao et al., 2018) -although the magnitude of perturbation on the image is small, it is amplified significantly when passing through deep neural network (see Figure 3b ). The deeper the layers are, the more significant such side effect is. Such amplification pushes values into a different quantization bucket, which is undesirable. We conducted empirical experiments to analyze how quantization influences the activation error between clean and adversarial samples (Figure 3a ): when the magnitude of the noise is small, activation quantization is capable of reducing the errors by eliminating small perturbations; However, when the magnitude of perturbation is larger than certain threshold, quantization instead amplify the errors, which causes

No or little loss of accuracy at bit=5:

(a) Quantization preserves the accuracy till 4-5 bits on clean image.

Large drop of robustness at bit=5:

(b) Quantization no longer preserves the accuracy under adversarial attack (same legend as left).

Figure 1 . Quantized neural network are more vulnerable to adversarial attack. Quantized models have no loss of accuracy on clean image (≥5 bits), but have significant loss of accuracy under adversarial attack compared to full precision models. Setup: VGG-16 and Wide ResNet on the test set of CIFAR-10 with FGSM attack.

the quantized model to make mistakes. We argue that this is the main reason causing the inferior robustness of the quantized models.

In this paper, we propose Defensive Quantization (DQ) that not only fixes the robustness issue of quantized models, but also turns activation quantization into a defense method that further boosts adversarial robustness. We are inspired by the success of image quantization in improving robustness. Intuitively, it will be possible to defend the attacks with quantization operations if we can keep the magnitude of the perturbation small. However, due to the error amplification effect of gradient based adversarial samples, it is non-trivial to keep the noise at a small scale during inference. Recent works (Cisse et al., 2017; Qian & Wegman, 2018) have attempted to make the network non-expansive by controlling the Lipschitz constant of the network to be smaller than 1, which has smaller variation change in its output than its input. In such case, the input noise will not propagate through the intermediate layers and impact the output, but attenuated. Our method is built on the theory. Defensive quantization not only quantizes feature maps into low-bit representations, but also controls the Lipschitz constant of the network, such that the noise is kept within a small magnitude for all hidden layers. In such case, we keep the noise small, as in the left zone of Figure 3a , quantization can reduce the perturbation error. The produced model with our method enjoys better security and efficiency at the same time.

Experiments show that Defensive Quantization (DQ) offers three unique advantages. First, DQ provides an effective way to boost the robustness of deep learning models while maintains the efficiency. Second, DQ is a generic building block of adversarial defense, which can be combined with other adversarial defense techniques to advance state-of-the-art robustness. Third, our method makes quantization itself easier thanks to the constrained dynamic range.

2. BACKGROUND AND RELATED WORK:

2.1 MODEL QUANTIZATION Neural network quantization (Han et al., 2015; Rastegari et al., 2016; Zhou et al., 2016; Courbariaux & Bengio, 2016; Zhu et al., 2016) ) are widely adopted to enable efficient inference. By quantizing the network into low-bit representation, the inference of network requires less computation and less memory, while still achieves little accuracy degradation on clean images. However, we find that the quantized models suffer from severe security issues -they are more vulnerable against adversarial attacks compared to full-precision models, even when they have the same clean image accuracy. Adversarial perturbation is applied to the input image, thus it is most related to activation quantization (Dhillon et al., 2018) . We carry out the rest of the paper using ReLU6 based activation quantization (Howard et al., 2017; Sandler et al., 2018) , as it is computationally efficient and is widely adopted by modern frameworks like TensorFlow (Abadi et al., 2016) . As illustrated in Figure 2 , a quantized convolutional network is composed of several quantized convolution block, each containing a serial of conv + BN + ReLU6 + linear quantize operators. As the quantization operator has

Conv BN ReLU6 Quantize … … Input Image Quantized Conv i X0 Xi Xi c Xi bn Xi r Xi q Wi Xt L = L CE + 2 X Wl2W ||W T l W l I|| 2 2 Lipschitz Regularization 1 Figure 2.

Defensive quantization with Lipschitz regularization.

0 gradient almost everywhere, we followed common practice to use a STE (Bengio et al., 2013) function y = x + stop gradient[Quantize(x)x] for gradient computation, which also eliminates the obfuscated gradient problem (Athalye et al., 2018) .

Our work bridges two domains: model quantization and adversarial defense. Previous work (Galloway et al., 2017) claims binary quantized networks can improve the robustness against some attacks. However, the improvement is not substantial and they used randomized quantization, which is not practical for real deployment (need extra random number generators in hardware). It also causes one of the obfuscated gradient situations (Athalye et al., 2018) : stochastic gradients, leading to a false sense of security. Rakin et al. (2018) tries to use quantization as an effective defense method. However, they employed Tanh-based quantization, which is not hardware friendly on fixed-point units due to the large overhead accessing look-up table. Even worse, according to our re-implementation, their method actually leads to severe gradient masking problem (Papernot et al., 2016a) during adversarial training, due to the nature of Tanh function (see A.1 for detail). As a result, the actual robustness of this work under black-box attack has no improvement over full-precision model and is even worse. Therefore, there is no previous work that are conducted under real inference setting to study the quantized robustness for both black-box and white-box. Our work aim to raise people's awareness about the security of the actual deployed models.

2.2. ADVERSARIAL ATTACKS & DEFENSES:

Given an image X, an adversarial attack method tries to find a small perturbation ∆ with constraint ||∆|| ≤ , such that the neural network gives different outputs for X and X adv X + ∆. Here is a scalar to constrain the norm of the noise (e.g., = 8 is commonly used when we represent colors from 0-255), so that the perturbation is hardly visible to human. For this paper we choose to study attacks defined under || • || ∞ , where each element of the image can vary at most to form an adversary. We introduce several attack and defense methods used in our work in the following sections.

2.2.1. ATTACK METHODS:

Random Perturbation (Random) Random perturbation attack adds a uniform sampled noise within [-, ] to the image, The method has no prior knowledge of the data and the network, thus is considered as the weakest attack method.

Fast Gradient Sign Method (FGSM) & R+FGSM Goodfellow et al. proposed a fast method to calculate the adversarial noise by following the direction of the loss gradient ∇ X L(X, y), where L(X, y) is the loss function for training (e.g. cross entropy loss). The adversarial samples are computed as:

X adv = X + * sign(∇ X L(X, y)) (1)

As FGSM is an one-step gradient-based method, it can suffer from sharp curvature near the data points, leading a false direction of ascent. Therefore, Tramèr et al. (2017) proposes to prepend FGSM by a random step to escape the non-smooth vicinity. The new method is called R+FGSM, defined as follows, for parameters and 1 (where 1 < ):

EQUATION

In our paper, we set 1 = /2 following Tramèr et al. (2017) .

Basic Iterative Method (BIM) & Projected Gradient Descend (PGD) Kurakin et al. (2016) suggests a simple yet much stronger variant of FGSM by applying it multiple times iteratively with a small step size α. The method is called BIM, defined as:

EQUATION

where clip X means clipping the result image to be within the -ball of X. In (Madry et al., 2017) , the BIM is prepended by a random start as in R+FGSM method. The resulting attack is called PGD, which proves to be a general first-order attack. In our experiments we used PGD for comprehensive experiments as it proves to be one of the strongest attack. Unlike Madry et al. (2017) that uses a fixed and α, we follow Kurakin et al. (2016) ; Song et al. (2017) to use α = 1 and number of iterations of min( + 4, 1.25 ) , so that we can test the model's robustness under different strength of attacks.

2.2.2. DEFENSE METHODS:

Current defense methods either preprocess the adversarial samples to denoise the perturbation (Xu et al., 2017; Song et al., 2017; Liao et al., 2018) or making the network itself robust (Warde-Farley & Goodfellow, 2016; Papernot et al., 2016b; Madry et al., 2017; Kurakin et al., 2016; Goodfellow et al.; Tramèr et al., 2017) . Here we introduced several defense methods related to our experiments.

Feature Squeezing Xu et al. (2017) proposes to detect adversarial images by squeezing the input image. Specifically, the image is processed with color depth bit reduction (5 bits for our experiments) and smoothed by a 2 × 2 median filter. If the low resolution image is classified differently as the original image, then this image is detected as adversarial.

Adversarial Training Adversarial training (Madry et al., 2017; Kurakin et al., 2016; Goodfellow et al.; Tramèr et al., 2017) is currently the strongest method for defense. By augmenting the training set with adversarial samples, the network learns to classify adversarial samples correctly. As adversarial FGSM can easily lead to gradient masking effect (Papernot et al., 2016a) , we study adversarial R+FGSM as in (Tramèr et al., 2017) . We also experimented with PGD training (Madry et al., 2017) .

Experiments show that above defense methods can be combined with our DQ method to further improve the robustness. The robustness has been tested under the aforementioned attack methods.

3. CONVENTIONAL NN QUANTIZATION IS NOT ROBUST:

Conventional neural network quantization is more vulnerable to adversarial attacks. We experimented with VGG-16 (Simonyan & Zisserman, 2014 ) and a Wide ResNet (Zagoruyko & Komodakis, 2016) of depth 28 and width 10 on CIFAR-10 (Krizhevsky & Hinton, 2009) dataset. We followed the training protocol as in (Zagoruyko & Komodakis, 2016) . Adversarial samples are generated with a FGSM (Goodfellow et al.) attacker ( = 8) on the entire test set. As in Figure 1 , the clean image accuracy doesn't significantly drop until the model is quantized to 4 bits (Figure 1a ). However, under adversarial attack, even with 5-bit quantization, the accuracy drastically decreased by 25.3% and 9.2% respectively. Although the full precision model's accuracy has dropped, the quantized model's 

Vanilla Quantization Defensive Quantization:

Small increase of perturbed range.

The large accumulated error pushes the activation to a different quantization bucket

Quantization Bucket:

Activations stay within the same quantization bucket.

Error Amplify 1 accuracy dropped much harder, showing that the conventional quantization method is not robust.

Clean image accuracy used to be the sole figure of merit to evaluate a quantized model. We show that even when the quantized model has no loss of performance on clean images, it can be much more easily fooled compared to full-precision ones, thus raising security concerns.

Input image quantization, i.e., color bit depth reduction is an effective defense method (Xu et al., 2017) . Counter intuitively, it does not work when applied to hidden layers, and even make the robustness worse. To understand the reason, we studied the effect of quantization w.r.t. different perturbation strength. We first randomly sample 128 images X from the test set of CIFAR-10, and generate corresponding adversarial samples X adv . The samples are then fed to the trained Wide ResNet model. To mimic different strength of activation perturbation, we vary the from 1 to 8. We inspected the activation after the first convolutional layer f 1 (Conv + BN + ReLU6), denoted as

A 1 = f 1 (X) and A adv 1 = f 1 (A adv ).

To measure the influence of perturbation, we define a normalized distance between clean and perturbed activation as:

EQUATION

We compare D(A, A adv ) and D(Quantize(A), Quantize(A adv )), where Quantize indicates uniform quantization with 3 bits. The results are shown in Figure 3a . We can see that only when is small, quantization helps to reduce the distance by removing small magnitude perturbations. The distance will be enlarged when is larger than 3.

The above experiment explains the inferior robustness of the quantized model. We argue that such issue arises from the error amplification effect (Liao et al., 2018) , where the relative perturbed distance will be amplified when the adversarial samples are fed through the network. As illustrated in Figure 4 , the perturbation applied to the input image has very small magnitude compared to the image itself (±8 versus 0 -255), corresponding to the left zone of Figure 3a (desired), where quantization helps to denoise the perturbation. Nevertheless, the difference in activation is amplified as the inference carries on. If the perturbation after amplification is large enough, the situation corresponds to the right zone (actual) of Figure 3a , where quantization further increases the normalized distance. Such phenomenon is also observed in the quantized VGG-16. We plot the normalized distance of each convolutional layer's input in Figure 3b . The fewer bits in the quantized model, the more severe the amplification effect.

4. DEFENSIVE QUANTIZATION:

Given the robustness limitation of conventional quantization technique, we propose Defensive Quantization (DQ) to defend the adversarial examples for quantized models. DQ suppresses the noise amplification effect, keeping the magnitude of the noise small, so that we can arrive at the left zone (Figure 3a , desired) where quantization helps robustness instead of making it worse.

We control the neural network's Lipschitz constant (Szegedy et al., 2013; Bartlett et al., 2017; Cisse et al., 2017) to suppress network's amplification effect. Lipschitz constant describes: when input changes, how much does the output change correspondingly. For a function f :

X → Y , if it satisfies D Y (f (x 1 ), f (x 2 )) ≤ kD X (x 1 , x 2 ), ∀x 1 , x 2 ∈ X (5)

for a real-valued k ≥ 0 and some metrics D X and D Y , then we call f Lipschitz continuous and k is the known as the Lipschitz constant of f . If we consider a network f with clean inputs x 1 = X and corresponding adversarial inputs x 2 = X adv , the error amplification effect can be controlled if we have a small Lipschitz constant k (in optimal situation we can have k ≤ 1). In such case, the error introduced by adversarial perturbation will not be amplified, but reduced. Specifically, we consider a feed-forward network composed of a serial of functions:

EQUATION

where φ i can be a linear layer, convolutional layer, pooling, activation functions, etc. Denote the Lipschitz constant of a function f as Lip(f ), then for the above network we have

EQUATION

As the Lipschitz constant of the network is the product of its individual layers' Lipschitz constants, Lip(f ) can grow exponentially if Lip(φ i ) > 1. This is the common case for normal network training (Cisse et al., 2017) , and thus the perturbation will be amplified for such a network. Therefore, to keep the Lipschitz constant of the whole network small, we need to keep the Lipschitz constant of each layer Lip(φ i ) ≤ 1. We call a network with Lip(φ i ) ≤ 1, ∀i = 1, ..., L a non-expansive network.

We describe a regularization term to keep the Lipschitz constant small. Let us first consider linear layers with weight W ∈ R cout×cin under || • || 2 norm. The Lipschitz constant is by definition the spectral norm of W : ρ(W), i.e., the maximum singular value of W. Computing the singular values of each weight matrix is not computationally feasible during training. Luckily, if we can keep the weight matrix row orthogonal, the singular values are by nature equal to 1, which meets our non-expansive requirements. Therefore we transform the problem of keeping ρ(W) ≤ 1 into keeping W T W ≈ I, where I is the identity matrix. Naturally, we introduce a regularization term ||W T W -I||, where I is the identity matrix. Following (Cisse et al., 2017) , for convolutional layers with weight W ∈ R cout×cin×k×k , we can view it as a two-dimension matrix of shape W ∈ R cout×(cinkk) and apply the same regularization. The final optimization objective is:

L = L CE + β 2 W l ∈W ||W T l W l -I|| 2 (8)

where L CE is the original cross entropy loss and W denotes all the weight matrices of the neural network. β is the weighting to adjust the relative importance. The above discussion is based on simple feed forward networks. For ResNets in our experiments, we also follow Cisse et al. (2017) to modify the aggregation layer as a convex combination of their inputs, where the 2 coefficients are updated using specific projection algorithm (see (Cisse et al., 2017) for details).

Our Defensive Quantization is illustrated in Figure 2 . The key part is the regularization term, which suppress the noise amplification effect by regularizing the Lipschitz constant. As a result, the perturbation at each layer is kept within a certain range, the adversarial noise won't propagate. Our method not only fixes the drop of robustness induced by quantization, but also takes quantization as a defense method to further increase the robustness. Therefore it's named Defensive Quantization.

5. EXPERIMENTS:

Our experiments demonstrate the following advantages of Defensive Quantization. First, DQ can retain the robustness of a model when quantized with low-bit. Second, DQ is a general and effective defense method under various scenarios, thus can be combined with other defensive techniques to further advance state-of-the-art robustness. Third, as a by-product, DQ can also improve the accuracy of training quantized models on clean images without attacks, since it limits the dynamic range.

5.1. FIXING ROBUSTNESS DROP:

Setup: We conduct experiments with Wide ResNet (Zagoruyko & Komodakis, 2016) of 28 × 10 on the CIFAR-10 dataset (Krizhevsky & Hinton, 2009) using ReLU6 based activation quantization, with number of bits ranging from 1 to 5. All the models are trained following (Zagoruyko & Komodakis, 2016) with momentum SGD for 200 epochs. The adversarial samples are generated using FGSM attacker with = 8.

Result::

The results are presented in Table 1 . For vanilla models, though the adversarial robustness increases with the number of bits, i.e., the models closer to full-precision one has better robustness, the best quantized model still has inferior robustness by -9.1%. While with our Defensive Quantization, the quantized models have better robustness than full-precision counterparts. The robustness is better when the number of bits are small, since it can de-noise larger adversarial perturbations. We also find that the robustness is generally increasing as β gets larger, since the regularization of Lipschitz constant itself keeps the noise smaller at later layers. At the same time, the quantized models consistently achieve better robustness. The robustness of quantized model also increases with β. We conduct a detailed analysis of the effect of β in Section B. The conclusion is: (1) conventional quantized models are less robust.

(2) Lipschitz regularization makes the model robust.

(3) Lipschitz regularization + quantization makes model even more robust.

As shown in (Athalye et al., 2018) , many of the defense methods actually lead to obfuscated gradient, providing a false sense of security. Therefore it is important to check the model's robustness under black-box attack. We separately trained a substitute VGG-16 model on the same dataset to generate adversarial samples, as it was proved to have the best transferability (Su et al., 2018) . The results are presented in Figure 5 . Trends of white-box and black-box attack are consistent. Vanilla quantization leads to inferior black-box robustness, while with our method can further improve the models' robustness. As the robustness gain is consistent for both white-box and black-box setting, our method does not suffer from gradient masking.

5.2. DEFEND WITH DEFENSIVE QUANTIZATION:

In this section, we show that we can combine Defensive Quantization with other defense techniques to achieve state-of-the-art robustness. (Netzer et al., 2011) and CIFAR-10 dataset (Krizhevsky & Hinton, 2009) . Since adversarial training is time consuming, we only use the official training set for experiments. CIFAR-10 is another widely used dataset containing 50,000 training samples and 10,000 testing samples of size 32 × 32. For both datasets, we divide the pixel values by 255 as a pre-processing step.

Following (Athalye et al., 2018; Cisse et al., 2017; Madry et al., 2017) , we used Wide ResNet (Zagoruyko & Komodakis, 2016 ) models in our experiments as it is considered as the standard model on the dataset. We used depth 28 and widening factor 10 for CIFAR-10, and depth 16 and widening factor 4 for SVHN. We followed the training protocol in (Zagoruyko & Komodakis, 2016) that uses a SGD optimizer with momentum=0.9. For CIFAR-10, the model is trained for 200 epochs with initial learning rate 0.1, decayed by a factor of 0.2 at 60, 120 and 160 epochs. For SVHN dataset, the model is trained for 160 epochs, with initial learning rate 0.01, decayed by 0.1 at 80 and 120 epochs. For DQ, we used bit=1 and β = 2e-3 as it offers the best robustness (see Section B).

We combine DQ with other defense methods to further boost the robustness. For Feature Squeezing (Xu et al., 2017) , we used 5 bit for image color reduction, followed by a 2 × 2 median filter. As adversarial FGSM training leads to gradient masking issue (Tramèr et al., 2017 ) (see A.2 for our experiment), we used the variant adversarial R+FGSM training. To avoid over-fitting into certain , we randomly sample using ∼ N (0, δ), = clip [0,2δ] (abs( )). Specifically we used δ = 8 to cover from 0-16. During test time, the is set to a fixed value (2/8/16). We also conducted adversarial PGD training. Following (Kurakin et al., 2016; Song et al., 2017) , during training we sample random as in R+FGSM setting, and generate adversarial samples using step size 1 and number of iterations min( + 4, 1.25 ) .

The results are presented in Table 2 and Table 3 , where (B) indicates black-box attack with a seperately trained VGG-16 model. The bold number indicates the best result in its column. We observe that for all normal training, feature squeezing and adversarial training settings, our DQ method can further improve the model's robustness. Among all the defenses, adversarial training provides the best performance against various attacks, epecially adversarial R+FGSM training. While white box PGD attack is generally the strongest attack in our experiments. Our DQ method also consistently improves the black-box robustness and there is no sign of gradient masking. Thus DQ proves to be an effective defense for various white-box and black-box attacks.

5.3. IMPROVE THE TRAINING OF QUANTIZED MODELS:

As a by-product of our method, Defensive Quantization can even improve the accuracy of quantized models on clean images without attack, making it a beneficial drop-in substitute for normal quantization procedures. Due to conventional quantization method's amplification effect, the distribution of activation can step over the truncation boundary (0-6 for ReLU6, 0-1 for ReLU1), which makes the optimization difficult. DQ explicitly add a regularization to shrink the dynamic range of activation, so that it is fitted within the truncation range. To demonstrate our hypothesis, we experimented with ResNet and CIFAR-10. We quantized the activation with 4-bit (because NVIDIA recently introduced INT4 in Turing architecture) using ReLU6 and ReLU1 respectively. Vanilla quantization and DQ training are conducted for comparison. As shown in 

6. CONCLUSION:

In this work, we aim to raise people's awareness about the security of the quantized neural networks, which is widely deployed in GPU/TPU/FPGAs, and pave a possible direction to bridge two important areas in deep learning: efficiency and robustness. We connect these two domains by designing a novel Defensive Quantization (DQ) module to defend adversarial attacks while maintain the efficiency. Experimental results on two datasets validate that the new quantization method can make the deep learning models be safely deployed on mobile devices. Therefore we conclude that simple quantization cannot help to improve robustness. Instead, it leads to inferior robustness.

A GRADIENT MASKING OF OTHER METHODS:



A.2 ADVERSARIAL FGSM TRAINING CAUSES GRADIENT MASKING:

Here we demonstrate that FGSM adversarial training leads to significant gradient masking problem, while R+FGSM fixes such issues. We trained a Wide ResNet using adversarial FGSM and adversarial R+FGSM respectively. Then the model is tested using FGSM with = 8 under white-box and black-box setting (VGG-16 as substitute). The results are shown in Table As observed in Section 5.1, the adversarial robustness of the model generally increases as β gets larger. Here we aim to find what is the optimal β for our experiment. We took 4-bit quantized Wide ResNet for example. As shown in Figure 6 , the adversarial accuracy first gets larger as β increases, and slowly goes down afterwards, reaching peak performance at β = 0.002. The clean accuracy is more stable. At the first stage, as regularization gets stronger, the effect of noise amplify is suppressed more. While for the second stage, the training suffers from a too strong regularization. Therefore, we used a β = 0.002 for our experiments unless specified.

C VISUALIZE SAMPLES AND PREDICTIONS:

We 

:



Figures and Tables:

Noise increases with perturbation strength. Quantization makes the slope deeper.

With conventional quantization, noise increases with layer index (the amplification effect).

Figure 3. (a) Comparison of the noise introduced by adversarial-attack, with and without quantization. For small perturbation, quantization reduces the noise; for large perturbation, quantization magnifies the noise. (b) The noise amplification effect: the noise is amplified with layer index. Setup: conventional activation quantization for VGG-16, normalized difference of full-precision and low-precision activation.

Figure 4. The error amplification effect prevents activation quantization from defending adversarial attacks.

Figure5. The white-box and black-box robustness are consistent: vanilla quantization leads to significant robustness drop, while DQ can bridge the gap and improve the robustness, especially with lower bits (bit=1). Setup: white-box and black-box robustness of Wide ResNet with vanilla quantization and defensive quantization.

Figure 7. Visualization of adversarial samples and the corresponding predictions (label and probability). FP: full-precision model; VQ: vanilla 4-bit quantized model; DQ: our 4-bit Defensive Quantization model



Table content: <table><tr><td>0</td><td>1 0.05988395808439</td><td>1</td><td>0</td><td>1</td><td>0</td><td>1</td><td>0</td><td>1</td><td>0</td></tr><tr><td/><td>349</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>0.46</td><td>2.0000 0.46921857947162</td><td>2.0000</td><td>0</td><td>2.0000</td><td>0.332712</td><td>2.0000</td><td>0</td><td>2.0000</td><td>0.2896</td></tr><tr><td/><td>576</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>0.72</td><td>3.0000 0.75508619994314</td><td>3.0000</td><td>0.6733</td><td colspan="2">3.0000 0.6304687265237110000</td><td>3.0000</td><td>0.6050</td><td>3.0000</td><td>0.5774</td></tr><tr><td/><td>32</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>1.05</td><td>4.0000 0.90917467235393</td><td>4.0000</td><td>0.7761</td><td colspan="2">4.0000 0.69524474064087600</td><td>4.0000</td><td>0.6601</td><td>4.0000</td><td>0.6252</td></tr><tr><td/><td>6</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>1.27</td><td>5.0000 0.84354168020697</td><td>5.0000</td><td>0.7445</td><td colspan="2">5.0000 0.70679410870240300</td><td>5.0000</td><td>0.6733</td><td>5.0000</td><td>0.6748</td></tr><tr><td/><td>52</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>1.44</td><td>6.0 0.95666628541334</td><td>6.0</td><td>0.9097</td><td>6.0</td><td>0.8250</td><td>6.0</td><td>0.7665</td><td>6.0</td><td>0.7683</td></tr><tr><td/><td>33</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>1.58</td><td>7.0000 1.01211496414316</td><td>7.0000</td><td>1.005</td><td>7.0000</td><td>0.9032</td><td>7.0000</td><td>0.8307</td><td>7.0000</td><td>0.8325</td></tr><tr><td/><td>4</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>1.7</td><td>8.0000 0.98628442443606</td><td>8.0000</td><td>0.9515</td><td colspan="2">8.0000 0.86693617000095300</td><td colspan="2">8.0000 0.80045166445080300</td><td>8.0000</td><td>0.788</td></tr><tr><td/><td>38</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>1.81</td><td>9.0000 1.06819201784740</td><td>9.0000</td><td>1.0577</td><td>9.0000</td><td>0.977219</td><td colspan="2">9.0000 0.8388344297226030</td><td>9.0000</td><td>0.8151</td></tr><tr><td/><td>03</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>2.0</td><td>10.000 1.09937739124844</td><td>10.000</td><td>1.1288</td><td>10.000</td><td>1.023</td><td>10.000</td><td>0.8246</td><td>10.000</td><td>0.7588</td></tr><tr><td/><td>81</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>2.27</td><td>11.0000 1.10909185824266</td><td>11.0000</td><td>1.1015</td><td>11.0000</td><td>1.0199</td><td>11.0000</td><td>0.8027</td><td>11.0000</td><td>0.7192</td></tr><tr><td/><td>85</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td>2.78</td><td>12.00 1.20793207402883</td><td>12.00</td><td>1.1706</td><td>12.00</td><td>1.099673</td><td>12.00</td><td>0.827749</td><td>12.00</td><td>0.7486</td></tr><tr><td/><td>7</td><td/><td/><td/><td/><td/><td/><td/><td/></tr><tr><td/><td>13.00 1.28888988272241</td><td>13.00</td><td>1.1754</td><td>13.00</td><td>1.097803</td><td>13.00</td><td>0.829665</td><td>13.00</td><td>0.7676</td></tr><tr><td/><td>33</td><td/><td/><td/><td/><td/><td/><td/><td/></tr></table>

The clean and adversarial accuracy of Wide ResNet on CIFAR-10 test set. We compare the accuracy of full-precision and quantized models. With our DQ method, we not only eliminate the robustness gap between full-precision and quantized models, but also improve the robustness over full-precision ones. The accuracy gain from quantization compared to the full-precision model (Quantize Gain) has gradually been improved as β increases. Bold and underline numbers are the first and second highest accuracy at each row.

Table content: <table><tr><td>Method</td><td>Acc.</td><td>Full Prec.</td><td>1</td><td>2</td><td>Quantize Bit 3</td><td>4</td><td>5</td><td>Quantize Gain</td><td>Best Acc.</td></tr><tr><td>Vanilla</td><td>clean adv.</td><td>94.8 39.3</td><td colspan="5">42.0 84.9 92.8 93.9 94.7 9.0 8.5 14.1 19.0 30.2</td><td>-9.1</td><td>39.3</td></tr><tr><td>DQ (β =3e-4)</td><td>clean adv.</td><td>95.3 41.8</td><td colspan="5">95.2 95.3 95.1 95.1 95.1 43.2 41.1 40.1 39.8 39.3</td><td>+1.4</td><td>43.2</td></tr><tr><td>DQ (β =6e-4)</td><td>clean adv.</td><td>95.6 45.7</td><td colspan="5">95.7 95.2 95.4 95.6 95.5 48.3 47.9 43.8 43.9 44.6</td><td>+2.6</td><td>48.3</td></tr><tr><td>DQ (β =1e-3)</td><td>clean adv.</td><td>95.9 49.1</td><td colspan="5">95.8 95.6 95.6 95.8 95.8 51.3 50.4 51.3 49.8 51.8</td><td>+2.7</td><td>51.8</td></tr></table>

SVHN experiments tested with = 2/8/16. (B) indicates black-box attack. We conducted experiments on the Street View House Number dataset (SVHN)

Table content: <table><tr><td>Training Technique</td><td>Clean Random FGSM R+FGSM PGD FGSM(B) PGD(B)</td></tr><tr><td>Normal</td><td>96.8 97/97/96 74/42/26 86/55/35 77/05/00 90/62/40 92/58/30</td></tr><tr><td>Normal + DQ</td><td>96.7 96/96/96 77/45/31 87/59/40 79/10/00 90/63/42 92/60/31</td></tr><tr><td>Feature Squeezing</td><td>96.3 96/96/95 69/34/20 84/48/28 72/03/00 89/61/38 91/56/27</td></tr><tr><td>Feature Squeezing + DQ</td><td>96.2 96/96/96 75/42/28 86/56/36 77/06/00 90/63/40 92/59/30</td></tr><tr><td>Adversarial R+FGSM</td><td>96.6 97/96/96 84/53/38 91/70/57 87/30/06 93/74/54 94/83/72</td></tr><tr><td colspan="2">Adversarial R+FGSM + DQ 96.6 97/96/95 88/59/40 93/77/61 91/47/17 94/80/61 95/89/84</td></tr><tr><td>Setup:</td><td/></tr></table>

Table 4, with vanilla quantization, ReLU1 model has around 1% worse accuracy than ReLU6 model, although they are mathematically equal if we

Table content: <table/>

CIFAR-10 Experiments tested with = 2/8/16. (B) indicates black-box attack.

Table content: <table><tr><td>Training Technique</td><td>Clean Random FGSM R+FGSM PGD FGSM(B) PGD(B)</td></tr><tr><td>Normal</td><td>94.8 95/91/77 59/39/29 72/37/17 56/01/00 88/63/36 90/64/42</td></tr><tr><td>Normal + DQ</td><td>95.9 96/94/84 68/53/42 77/50/30 62/04/00 84/59/40 87/50/27</td></tr><tr><td>Feature Squeezing</td><td>94.1 94/92/81 61/35/27 76/40/21 64/02/00 87/62/30 89/70/42</td></tr><tr><td>Feature Squeezing + DQ</td><td>94.9 95/93/82 66/48/33 77/51/29 66/06/01 87/62/29 90/70/43</td></tr><tr><td>Adversarial R+FGSM</td><td>91.6 92/91/91 81/52/38 87/69/48 84/43/11 92/89/85 92/91/89</td></tr><tr><td colspan="2">Adversarial R+FGSM + DQ 94.0 94/93/93 85/63/51 90/74/58 87/50/22 93/91/87 94/92/92</td></tr><tr><td>Adversarial PGD</td><td>86.6 86/86/86 74/46/31 79/63/46 76/44/20 84/83/81 84/83/83</td></tr><tr><td>Adversarial PGD + DQ</td><td>87.5 87/87/87 79/53/36 83/69/52 81/50/22 87/86/82 87/86/86</td></tr></table>

DQ method improves the training of normal quantized models by limiting the dynamic range of activation. With conventional quantization, ReLU1 suffers from inferior performance than ReLU6. While with DQ, the gap is fixed, ReLU1 and ReLU6 quantized models achieve similar accuracy. the previous BN scaling by 1/6 and next convolution weight by 6. It demonstrates that improper truncation function and range will lead to training difficulty. While with DQ training, both model has improved accuracy compared to vanilla quantization, and the gap between ReLU1 model and ReLU6 model is filled, making quantization easier regardless of truncation range.

Table content: <table><tr><td/><td>ReLU1</td><td>ReLU6</td><td>Difference</td></tr><tr><td>Vanilla Quantization</td><td>93.49%</td><td>94.41%</td><td>-0.92%</td></tr><tr><td>Defensive Quantization</td><td>95.14%</td><td>95.09%</td><td>0.05%</td></tr></table>

.1 TANH-BASED ACTIVATION QUANTIZATION LEADS TO FAKE SECURITY Rakin et al. (2018) tried to use activation quantization as a defense method against adversarial samples. According to their experiments, simply Tanh based activation quantization can greatly improve the model's robustness under PGD adversarial training setting. We reproduced their experiments by training a ResNet-18 model with PGD adversarial training on CIFAR-10 following Madry et al.(2017). To make a comparison, we also trained a full precision model and a ReLU6 quantized model following same setting. All the quantized models use bit=2. We tested the trained model using PGD attack byMadry et al. (2017) under both white-box and black-box setting. We use a VGG-16 model separately trained with PGD adversarial training as the black-box attacker. The results are provided in Table5.

Table content: <table/>

Tanh-based quantization improves white-box robustness, while suffers from inferior black-box robustness compared to full-precision model. The white-box accuracy is even higher than black-box, suggesting gradient masking. Setup: white-box and black-box robustness of PGD adversarially trained ResNet-18 on CIFAR-10. Quantization uses 2 bits.Our white-box result is consistent with(Rakin et al., 2018), where Tanh based quantizaion with PGD training gives much higher white-box accuracy compared to the full precision model. However, we can see that the black-box robustness decreases. Worse still, the black-box attack successful rate is even lower than white-box, which is abnormal since black-box attack is generally weaker than white-box. This phenomenon indicates severe gradient masking problem(Papernot et al., 2016a;Athalye et al., 2018), which gives a false sense of security. As a comparison, we also trained a ReLU6 quantized model. With ReLU6 quantization, there is no sign of gradient masking, nor improvement in robustness, indicating that gradient masking problem majorly comes from Tanh activation function. Actually, ReLU6 quantized model has slightly worse robustness.

Table content: <table><tr><td/><td>Clean Acc.</td><td>White-Box Acc.</td><td>Black-Box Acc.</td></tr><tr><td>Full-Precision</td><td>83.47%</td><td>50.51%</td><td>65.58%</td></tr><tr><td>Tanh Quantized</td><td>81.60%</td><td>71.03%</td><td>61.57%</td></tr><tr><td>ReLU6 Quantized</td><td>78.97%</td><td>48.90%</td><td>59.44%</td></tr></table>

White-box and black-box robustness of FGSM/R+FGSM adversarially trained Wide ResNet on CIFAR-10.

Table content: <table><tr><td/><td>Clean Acc.</td><td>White-Box Acc.</td><td>Black-Box Acc.</td></tr><tr><td>Adversarial FGSM</td><td>94.55%</td><td>94.18%</td><td>51.65%</td></tr><tr><td>Adversarial R+FGSM</td><td>91.61%</td><td>51.65%</td><td>87.39%</td></tr></table>

Clean and adversarial accuracy of 4-bit quantized Wide ResNet w.r.t. different β.

Table content: <table><tr><td/><td>100%</td><td/><td/><td/><td/><td/></tr><tr><td/><td>87%</td><td/><td/><td colspan="2">Clean Accuracy</td><td/></tr><tr><td>Accuracy</td><td>61% 74%</td><td/><td/><td colspan="3">Adversarial Accuracy</td></tr><tr><td/><td>48%</td><td/><td/><td/><td/><td/></tr><tr><td/><td>35% 0E+00</td><td>1E-03</td><td>2E-03</td><td>3E-03</td><td>4E-03</td><td>5E-03</td></tr><tr><td/><td/><td/><td>β</td><td/><td/><td/></tr><tr><td>Figure 6.</td><td/><td/><td/><td/><td/><td/></tr><tr><td/><td/><td/><td/><td>Clean Accuracy</td><td/><td>Adversarial</td></tr><tr><td/><td/><td/><td/><td/><td/><td>Accuracy</td></tr><tr><td/><td/><td/><td>0</td><td>0</td><td>93.94</td><td>0.9394</td><td>0</td><td>19</td><td>0</td></tr><tr><td/><td/><td/><td>0.46</td><td>0.0003</td><td>95.09</td><td>0.9509</td><td>0.0003</td><td>40</td><td>0</td></tr><tr><td/><td/><td/><td>0.72</td><td>0.0006</td><td>95.59</td><td>0.9559</td><td>0.0006</td><td>43.8700</td><td>0.4387</td></tr><tr><td/><td/><td/><td>1.05</td><td>0.0010</td><td>95.76</td><td>0.9576</td><td>0.0010</td><td>49.7900</td><td>0.4979</td></tr><tr><td/><td/><td/><td>1.27</td><td>0.0020</td><td>95.92</td><td>0.9592</td><td>0.0020</td><td>55.8700</td><td>0.5587</td></tr><tr><td/><td/><td/><td>1.44</td><td>0.0</td><td>96.05</td><td>0.9605</td><td>0.0</td><td>53.4400</td><td>0.5344</td></tr><tr><td/><td/><td/><td>1.58</td><td>0.0040</td><td>96.21</td><td>0.9621</td><td>0.0040</td><td>53.000</td><td>0.530</td></tr><tr><td/><td/><td/><td>1.7</td><td>0.0050</td><td>96.08</td><td>0.9608</td><td>0.0050</td><td>52.1200</td><td>0.5212</td></tr></table>

Peer review (iclr2019):
 Summary of paper This paper presents an approach for quantising neural networks such that the resulting quantised model is robust to adversarial and random perturbations. The core idea of the paper is to enforce the Lipschitz constant of each linear layer of the network approximately close to 1. Since the Lipschitz constant of the neural network is bounded by the product of the Lipschitz constant of its linear layer (assuming Lipschitz 1 activation functions) the Lipschitz constant of the trained neural network is bounded by 1. This results in a model which is robust to adversarial and random noise ad all directions in the model space are non-expansive. Algorithmically, controlling the Lipschitz constant is achieved by using the orthogonal regulariser presented in the paper Cisse et.al which has the same motivation for this work but for standard neural network training but not quantising. The authors presents thorough experimental study showing why standard quantisation schemes are prone to adversarial noise and demonstrate clearly how this approach improves robustness of quantised network and sometimes even improve over the accuracy of original model. Review: The paper is well written with clear motivation and very easy to follow. The core idea of using orthogonal regulariser for improving the robustness of neural network models have been presented in Cisse et.al and the authors re-use it for improving the robustness of quantised models. The main contribution of this work is in identifying that the standard quantised models are very vulnerable to adversarial noise which is illustrated through experiments and then empirically showing that the regulariser presented in Cisse et. al improves the robustness of quantised models with rigorous experiments. The paper add value to the research community through thorough experimental study as well as in industry since quantised models are widely used and the presented model is simple and easy to use. Some suggestions and ideas: 1. It will be great if the authors could add a simple analytical explanation why the quantised networks are not robust. 2. The manifold of Orthogonal matrices does not include all 1 - Lipschitz matrices and also the Orthogonal set is not convex. I think a better strategy for this problem is to regularise the spectral norm to be 1. Regularising the spectral norm is computationally cheaper than Orthogonal regulariser when combined with SGD using power iterations. Moreover the regulariser part of the model becomes nice and convex. 3. Another strategy to control the Lipschitz constant of the network is to directly penalise the norm of the Jacobian as explained in Improved Training of Wasserstein GANs (Gulrajani et. al).