Title: STOCHASTIC QUANTIZED ACTIVATION: TO PREVENT OVERFITTING IN FAST ADVERSARIAL TRAINING

Abstract: Existing neural networks are vulnerable to "adversarial examples"-created by adding maliciously designed small perturbations in inputs to induce a misclassification by the networks. The most investigated defense strategy is adversarial training which augments training data with adversarial examples. However, applying single-step adversaries in adversarial training does not support the robustness of the networks, instead, they will even make the networks to be overfitted. In contrast to the single-step, multi-step training results in the state-of-the-art performance on MNIST and CIFAR10, yet it needs a massive amount of time. Therefore, we propose a method, Stochastic Quantized Activation (SQA) that solves overfitting problems in single-step adversarial training and fastly achieves the robustness comparable to the multi-step. SQA attenuates the adversarial effects by providing random selectivity to activation functions and allows the network to learn robustness with only single-step training. Throughout the experiment, our method demonstrates the state-of-the-art robustness against one of the strongest white-box attacks as PGD training, but with much less computational cost. Finally, we visualize the learning process of the network with SQA to handle strong adversaries, which is different from existing methods.

1. INTRODUCTION:

As Convolutional Neural Networks (CNNs) stand out as a solution to many real world computer vision tasks (LeCun et al., 2015; Angelova et al., 2015; Levine et al., 2016; Litjens et al., 2017) , achieving a certain level of robustness has become indispensable for security-sensitive systems, such as autonomous driving, robot vision, and identity authentication. However, recent studies (Szegedy et al., 2014; Goodfellow et al., 2015) have shown that the existing CNNs are vulnerable to small perturbations of the input that are intentionally or adversarially designed to fool the system. The adversarial attack is a serious problem since these maliciously designed attacks have shown effective in physical world scenarios, where inputs are obtained from signals of cameras and other sensors (Kurakin et al., 2016; Evtimov et al., 2017) . Another disconcerting feature about adversarial examples is their transferability across different models (Szegedy et al., 2014; Papernot et al., 2016; Liu et al., 2016) that enables black-box attacks. In other words, adversarial examples can be designed from a different model without having the information about the target network.

The most studied defense strategy against adversarial attacks is adversarial training (Goodfellow et al., 2015; Kurakin et al., 2017; Tramr et al., 2018; Madry et al., 2018) Recently, Madry et al. (2018) have found that adversarial training on examples created via gradient descent with random restarts, Projected Gradient Descent (PGD) training, results in a universally and partially unbreakable model on MNIST and CIFAR-10. This method shows the state-of-the-art performance on MNIST and CIFAR-10 to the best of our knowledge, but the examples are created iteratively and the time increases proportionally to the number of steps. For instance, in our CIFAR-10 training, FGSM training on ResNet18 took less than 2 hours for 30 epochs; however, PGD training took about 30 hours for the same epochs. Thus, it is essential to find the universal method that is resistant against all of the attacks, with less computational cost.

Since high dimensional representations of the neural networks give extreme complexity to the boundary of trained manifolds (Tanay & Griffin, 2016; Dube, 2018) , we start from the idea that is to reduce degrees of freedom available to the adversary. In this sense, we propose a Stochastic Quantized Activation (SQA) that provides stochastic randomness to the output of an original activation and reduces the opportunity for the attacker to make adversaries. The best advantage of SQA is that SQA with fast adversarial training, training with only FGSM examples, allows the model to have robustness comparable to PGD training with less computational cost. In particular, although SQA is one of the obfuscated gradients defined by Athalye et al. (2018) , iterative optimization-based methods does not successfully circumvent our defense. Besides, SQA can be combined with any deep learning models with a few lines of code but guarantees a certain level of robustness against adversarial attacks.

In this paper, we first explain existing methods for adversarial attacks and defenses we refer in Section 2. We separate the existing defense strategies into two categories and analyze the strengths and weaknesses. In Section 3, we introduce the procedure of SQA, with an algorithm described in 1. In Section 4, we show our experimental results on MNIST and CIFAR-10 and compare with existing defense systems. Lastly, we visualize the penultimate layer of our networks and compare how SQA with fast adversarial training, learns differently from the existing methods. Section 5 concludes the work and contributions of this paper are as follows:

• We propose a Stochastic Quantized Activation (SQA) which achieves a significant level of robustness combined with FGSM training, comparable to state-of-the-art PGD adversarial training with much less computational cost. • Due to the efficiency and the flexibility of the proposed method, it can be fastly and widely applied to any existing deep neural networks and combine with other types of defense strategies. • We analytically demonstrate how SQA makes the model robust against adversaries in highlevel and low-level by using t-SNE, and plotting activation maps.

2. RELATED WORK:

In this section, we investigate the existing methods of adversarial attacks and defenses that appear in the following subsections. First, we define the adversarial examples with the notations formally used in this paper. Let x denote input and y denote the prediction of the input from the DNN classifier f , y = f (x). Then, an adversarial example is crafted by adding a malicious noise η into the original input x, causing a different prediction from the true label, y * . The formal representation is as follows, where x is an adversarial example and is the noise level.

x = x + • η, where f (x ) = y * (1)

2.1. GENERATING ADVERSARIAL EXAMPLES:

Fast Gradient Sign Method (FGSM) is a fast single-step method to create adversarial examples proposed by Goodfellow et al. (2015) . The authors suggest the adversarial examples are crafted because of the effects of the linear summation in DNNs, and the algorithm is as follows.

EQUATION

Here J(f (x), y * ) is the loss between the output prediction f (x) and the true label y * . However, calculating the loss based on the difference between predictions and true labels makes the label leaking effect (Kurakin et al., 2017) , so one simple way to prevent it is to put the prediction y instead of y * . The intuition behind of the Equation 2 is that increasing loss J by perturbing the input x adding the gradient of loss, which makes the prediction get out of the extrema.

Projected Gradient Descent (PGD) is one of the strongest known white box attacks (Madry et al., 2018) . It is a multi-step variant of FGSM, which means that it finds the adversarial perturbation η n by using the same equation from FGSM, but iteratively. What makes this attack stronger is that it finds the adversary from starts with random -uniform perturbation clipped in the range of the normalized pixel values, [0,1].

x 0 = Clip x (x + unif orm(-, )), x n+1 = Clip x, (x n + α • sign(∇ x J(f (x n ), y * ))) (3) Carlini & Wagner Attack (C & W Attack

) is strong optimization-based iterative attack proposed by Carlini & Wagner (2017) . It uses Adam (Kingma & Ba, 2014) to optimize over the adversarial perturbation η n using an auxiliary variable ω n and solves the equation below.

minimize ||η n || p + c • f (x n + η n )

where

EQUATION

The function f (•) is defined as

EQUATION

and we can determine the confidence with which the misclassification occurs by adjusting κ.

2.2. DEFENSIVE STRATEGY:

Adversarial training increases robustness by augmenting training data in relation to adversarial examples. Previous studies (Goodfellow et al., 2015; Kurakin et al., 2017; Tramr et al., 2018) 2018) make the attacks which successfully circumvent the defense by making 0% accuracy on 6 out of 7 defenses at ICLR2018. SQA can be considered as both shattered gradients and stochastic gradients. However, we found that our method does not overfit to the adversarial examples and shows robustness against the different type of attacks including the one used to break obfuscated gradients. The next section explains the details of our method.

3. STOCHASTIC QUANTIZED ACTIVATION:

Algorithm 1 Stochastic Quantized Activation

1: function FORWARD(h i , λ) 2: g i ← (h i -min ∀j⊆J h i J ) / (max ∀j⊆J h i J -min ∀j⊆J h i J ) * λ 3: g i ← g i + Bernoulli(g i -g i ) 4: g i ← (g i / λ) * (max ∀j⊆J h i J -min ∀j⊆J h i J ) + min ∀j⊆J h i J 5:

return g i 6: function BACKWARD(∂g i /∂h i ) 7:

return ∂g i /∂h i

In this section, we introduce the concept of SQA starting from a typical low-bit representation in DNNs as prerequisites (Courbariaux et al., 2015) . Then, we show the procedure of our quantization stochasticity. The difference between typical low-bit DNNs (Hubara et al., 2016a; Courbariaux et al., 2015; Hubara et al., 2016b) and our proposed method is that we only consider the quantization of activations except weight vectors. We found that this does not significantly slow down the training with PyTorch (Paszke et al., 2017) but maintains full-precision weight representation, which enables easier convergence than BNNs without additional training strategies.

BinaryConnect constraints the weights to either +1 or -1 during propagations (Courbariaux et al., 2015) . Two types of binarization, deterministic and stochastic, are introduced. They are respectively described by the following equations.

EQUATION

)

w b = +1 with probability p = σ(w), -1 with probability 1 -p.

where

EQUATION

BNNs are originally designed to reduce the significant amount of memory consumption and costs taken by propagating in full-precision networks. Recently, however, Galloway et al. (2018) shows another benefit of low-precision neural networks, which improves robustness against some adversarial attacks.

Thus, we propose SQA, a stochastic activation function giving the quantized threshold effects into vanilla CNNs, which is described in Algorithm 1. The algorithm can be divided into three steps.

• Min-Max normalization with scaling • Stochastic Quantization • Inverse Min-Max normalization after rescaling Let h i be a latent space, the output from a ith convolutional layer after ReLU activation. We first perform min-max normalization, making h i ranging from 0 to 1. Then we scale the h i ranging from 0 to λ by multiplying a scale factor λ, which determines the level of quantization from binary to quaternary in our experiment. In the next step, we stochastically quantize the scaled g i as g i presented in the below equation.

EQUATION

This makes g i converge into the closest or second closest integers, either g i or g i + 1 with a probability of each, 1 -(g i -g i ) and g i -g i . For instance, if we let g i = 1.7, then the probability of g i = 1 is 0.3 and g i = 2 is 0.7. The final step is rescaling g i into the range of original output ReLU activation h i . To rescale the value within the original range, g i is first divided by λ, and inverse min-max normalization is applied as presented in Algorithm 1.

Since it is impossible to find exact derivatives with respect of discretized activations, an alternative is to approximate it by a straight through estimator (Bengio et al., 2013) . The concept of a straight through estimator is fixing the incoming gradients to a threshold function equal to its outgoing gradients, ignoring the derivative of the threshold function itself. This is the reason why we rescale g i to the original range of h i . In other words, we do not want to consider the scale factors multiplied in the activation function when we use a straight through estimator.

4. EXPERIMENT:



4.1. DATASET AND IMPLEMENTATION DETAILS:

In this experiment, we show the feasibility of our approach with several different settings on MNIST and CIFAR-10 using PyTorch (Paszke et al., 2017) . We use Adversarial Box (Wang & Gavin Ding, 2018) 

Model Parameters:

For MNIST, we use a baseline model as a Vanilla CNN consisting of three convolutional layers with two fully-connected layers on the top. Since there is a correlation between robustness and model capacity (Madry et al., 2018) , we use two networks with different channel sizes and increase channels by a factor of 2. This result in networks with each (16, 32, 64) and (64, 128, 256) filters and they are denoted as SMALLandLARGE in Table 1 . We apply SQA on the first and second layers with each λ = 1 and 2. We use Stochastic Gradient Descent (SGD) with learning rate of 0.1, momentum of 0.9, and weight decay of 5e-4. We adjust the learning rate decreasing by 0.1 after every 30 steps within total 100 epochs.

For CIFAR-10, we use ResNet model (He et al., 2015) as a baseline. We adpot 34 and 101 layers of ResNets denoted as RES34 and RES101 in Table 2 . An interesting property found on training CIFAR-10 dataset is that quantization with stochasticity shows much higher accuracy rather than deterministic quantization. It seems reasonable since stochasticity is able to provide higher capacity to learn complex RGB images. We apply SQA on the output from the first layer of ResNet and its bottleneck module with each λ = 1 and 2. The same hyper-parameters are applied to the MNIST training except with total epochs of 350 and decreasing learning rate by 0.1 after every 150 steps.

Attack Parameters:

Throughout the experiments, different l ∞ intensity levels are applied to the attacks. For MNIST, = 0.2 and 0.3 are used for FGSM and C&W attacks to give strong adversarial perturbations. We choose 40 steps for C&W attacks. Also, we set = 0.2, a step size of 0.01 and 40 steps for PGD attacks. For CIFAR-10, = 4, 8 are considered for the adversarial attacks. We choose 30 steps for C&W attacks. For PGD attacks we fix 7 steps and the step size as 2 with random perturbation 8. Note that the values for MNIST are in the scale of (0,1) and (0,255) for CIFAR-10. Step sizes for the attacks are chosen to be consistent with Madry et al. (2018) .

4.2. ATTACK ON MNIST:



Quantization on Different Layers:

Since quantizing the weights or activation lowers the accuracy on clean images (Courbariaux et al., 2015; Hubara et al., 2016a) , it is important to find where to put SQA modules in networks. Thus, we investigate the nth layer-wise quantization applying the deterministic quantization from the first layer of CNN to the third. The result is shown in Figure 1 . It is clear that applying quantization on the earlier steps gives higher robustness. This observation is another proof for the argument from Liao et al. (2017) that a small perturbation in an image is amplified to a large perturbation in a higher-level representation so that quantizing the activations in lower-level representation gives more robustness. We further, empirically found that giving binary quantization on the first layer and ternary quantization on the second layer provides less degradation for accuracy and a fair amount of robustness. We explore the robustness of SQA against three types of adversarial attacks and the result is shown in Table 1 . The networks are all trained with fast single-step adversaries and we could find two known, but interesting properties from the experiments. First, FGSM training the full-precision networks, denoted as SMALL f ull , LARGE f ull , makes themselves overfit to the adversaries. They show depressed accuracy on especially, PGD attacks, nearly close to 0. However, SQA models does not overfit to the adversaries. Even though SQA models show lower performance on FGSM attacks, they exhibit remarkably high accuracy on the other adversarial examples that have not seen before. The second interesting fact is that the correlation between robustness and model capacity. Madry et al. (2018) have shown that increasing model capacity helps to train the network against strong adversaries successfully. Our experiment also confirms this phenomenon. The performance of LARGE SQA is stronger than SMALL SQA against FGSM attacks and more than ten times robust against PGD attacks. This result shows that the model capacity not only increases robustness against the adversaries that have been learned but also prevent overfitting to them.

4.3. ATTACK ON CIFAR10:



SQA v.s. Full-Precision:

We performed experiments on CIFAR-10 to show the effectiveness of SQA on the RGB image dataset. We tried the same types of white-box attacks as in MNIST experiments, and the result is shown in Table 2 . Instead of training Vanilla networks, we adopt ResNet (He et al., 2015) since the Vanilla networks are hard to learn useful features on CIFAR-10. Two different ResNets are used for comparing robustness regarding the model capacity, and we found the same phenomena as in MNIST experiments. In other words, SQA module helps to get out of overfitting to the FGSM adversaries, and the larger capacity provides, the higher robustness against different types of attacks.

SQA v.s. Other Existing Methods:

We compare our module, SQA, with recently proposed defenses including the state-of-the-art, Madry et al. (2018) . We also include SAP, PixelDefend, and Thermometer (Dhillon et al., 2018; Song et al., 2018; Buckman et al., 2018) since they use stochastic gradients or shattered gradients that are one of the obfuscated gradients, where our method belongs to. Table 3 shows the performance1 comparison against PGD and C&W attacks for l ∞ ( = 8). Note that the architectures from the defenses on Table 3 are all different and it is impossible to exactly compare the robustness. We denote the architectures as RESN k W,C , where W stands for Wider ResNets, N is depth, C is the channel size of the first layer, and k is the widen factor. As Athalye et al. (2018) claimed, our method is more robust against gradient-based PGD rather than optimization-based C&W, pushing the state-of-the-art accuracy to 52% against PGD attacks. Also, it shows a fair amount of accuracy against C&W attacks comparable to Adv. Training. This result shows a dramatic impact in a sense that other methods based on obfuscated gradients almost fail to defend against these strong adversaries. 

4.5. VISUALIZING PENULTIMATE LAYERS:

In this subsection, we analyze the penultimate layers of the network trained with our method comparing with two full-precision networks: with no defense and with FGSM training. We use C&W attacks to make adversaries with the parameters described in Section 4.1. We use two different ways to visualize the penultimate layers in high level and low level by using t-SNE (van der Maaten & Hinton, 2008) and plotting activation maps with both clean images and adversarial examples. Firstly, Figure 2 shows t-SNE results from the penultimate layer of our network and a point in t-SNE is represented as an image. We select four classes to clearly show how the networks learn and what happens when adversarial noise is added. Here, we demonstrate that the full-precision network trained with FGSM does not correctly classify the classes against the adversarial attacks, as depicted in (B). However, only (C) which is our method shows that the clusters are less broken compared to the other methods. Furthermore, in light of the fact that the robust classifier requires a more complicated decision boundary (Madry et al., 2018) , our model seems to have the complicated one by learning adversarial examples.

Secondly, we closely look into the penultimate layer in a low level by plotting the each of the activations. In this time, a point of an activation map stands for the mean value of the activations across about a thousand images per classes. We found that the yellow spots which are the highest values stay in the same location under the adversarial attack, as depicted in (C), Figure 3 . In other words, our method shows stable activation frequencies against the adversarial attacks, but training full-precision models with FGSM adversaries does not help to increase robustness, as shown in (B).

5. CONCLUSION:

In this paper, we have found that SQA, a stochastic quantization in an activation function, make existing neural networks prevent overfitting to FGSM training. It provides stochastic randomness in quantization to learn a robust decision boundary against adversarial attacks with FGSM training. Our method not only shows dramatic improvements against one of the strongest white-box attacks, comparable to state-of-the-art PGD training but also significantly reduces the computational cost. Throughout visualizing the penultimate layers of our network, we demonstrate that the network learns strong adversaries without overfitting. We expect that SQA could be fastly and widely applied to other defense strategies because of its efficiency and flexibility. In the future work, we plan to experiment on large scale image datasets.

:

The performance of SAP, PixelDefend, and Thermometer is fromAthalye et al. (2018).

Figures and Tables:

, which increases robustness by augmenting training data with adversarial examples. Since adversarial training requires the model to train adversarial examples in addition to training data, the model consumes extra time to learn features of the examples via fine-tuning. Even though the model is trained on more examples, it still might be defenseless to new examples generated by different attack due to the overfitting problem.

Figure 1: FGSM attack to deterministic quantization on different layers of the baseline CNN.

Figure 2: t-SNE results from the penultimate layer of our network against clean images and C&W adversaries. (A) is a full-precision network with no defense, (B) is a full-precision network + FGSM training, and (C) is SQA + FGSM training (Ours). While (A) and (B) shows significant degradation on adversaries, (C) still finds proper decision boundaries against adversarial attacks.

Figure 3: Activation maps from the penultimate layer of our network. Top two rows are the results on MNIST, and the bottom rows are for CIFAR-10. For each dataset, top rows are activations given inputs as clean images, and bottom rows are the ones given adversarial images. (A) is a fullprecision network with no defense, (B) is a full-precision network + FGSM training, and (C) is SQA + FGSM training (Ours). C&W attacks are applied and (C) shows little difference between clean and adversaries.

Performance comparison for MNIST between full-precision and SQA

Table content: <table><tr><td>Model</td><td>Clean</td><td colspan="2">FGSM =0.2 FGSM =0.3</td><td>C&amp;W =0.2</td><td>C&amp;W =0.3</td><td>PGD</td></tr><tr><td>SMALL f ull</td><td>99.40</td><td>98.48</td><td>98.86</td><td>19.43</td><td>0.71</td><td>0.50</td></tr><tr><td colspan="5">SMALLSQA 96.61±0.28 89.30±0.19 82.67±0.21 82.39±0.17</td><td>57.7±0.40</td><td>7.43±0.26</td></tr><tr><td>LARGE f ull</td><td>99.25</td><td>98.58</td><td>98.21</td><td>64.21</td><td>19.49</td><td>0.86</td></tr><tr><td colspan="7">LARGESQA 98.64±0.34 95.30±0.06 91.14±0.24 92.75±0.34 83.62±0.33 83.90±0.32</td></tr><tr><td>Model</td><td>Clean</td><td>FGSM =4</td><td>FGSM =8</td><td>C&amp;W =4</td><td>C&amp;W =8</td><td>PGD</td></tr><tr><td>RES34 f ull</td><td>87.07</td><td>46.21</td><td>63.47</td><td>13.5</td><td>8.20</td><td>8.87</td></tr><tr><td>RES34SQA</td><td colspan="6">81.53±0.38 70.51±0.49 53.45±0.44 66.45±0.45 38.70±0.62 48.32±0.19</td></tr><tr><td>RES101 f ull</td><td>87.67</td><td>52.44</td><td>78.59</td><td>12.17</td><td>8.09</td><td>7.52</td></tr><tr><td colspan="7">RES101SQA 82.94±0.42 73.30±0.39 56.54±0.52 68.40±0.69 41.52±0.41 52.12±0.63</td></tr></table>

Performance comparison for CIFAR10 between full-precision and SQA SQA v.s. Full-Precision

Table content: <table/>

Performance comparison with other defenses against l ∞ ( = 8) adversaries for CIFAR10

Table content: <table><tr><td>Defense</td><td>Model</td><td colspan="2">PGD C&amp;W =8</td></tr><tr><td>SAP (Dhillon et al., 2018)</td><td>RES20</td><td>-</td><td>0</td></tr><tr><td>PixelDefend (Song et al., 2018)</td><td>RES62</td><td>9</td><td>-</td></tr><tr><td colspan="2">Thermometer (Buckman et al., 2018) RES30 4 W Adv. Training (Madry et al., 2018) RES 10 W</td><td>30 50</td><td>-46.8</td></tr><tr><td>SQA (Ours)</td><td>RES101</td><td>52.1</td><td>41.5</td></tr><tr><td>Defense</td><td colspan="2">Training time</td><td/></tr><tr><td>FGSM Training</td><td>0.43</td><td/><td/></tr><tr><td>SQA + FGSM Training</td><td>0.57</td><td/><td/></tr><tr><td colspan="3">PGD Training κ=10/20/100 1.23 / 2.34 / 10.17</td><td/></tr></table>

SQA + FGSM training is almost 18 times faster than PGD training where κ is 100.

Table content: <table/>

Peer review (iclr2019):
 The paper proposes a model to improve adversarial training, by introducing random perturbations in the activations of one of the hidden layers. Experiments show that robustness to attacks can be improved, but seemingly at a significant cost to accuracy on non-adversarial input. I have not spent significant time on adversarial training, and review the paper under the following understanding: It was observed that the decision regions of a class are sprinkled with "holes" that get misclassified. These holes are neither naturally occuring. Their existence allows a potential attacker to coerce a model into mis-classifying by providing specially crafted inputs, in order to attain a benefit. Therefore, those holes are called "adversarial" examples. The risk is heightened by the fact that adversarial examples are commonly not mis-classified by humans (or even detectable by the eye). To "plug" the holes, one includes adversarial examples in the training, called "adversarial training." A resulting system should now have a much improved accuracy for the "holes", while ideally not affecting classification accuracy for the natural examples, which will continue to constitute nearly 100% of the samples the system will be used on. (The "hole" metaphor may not be entirely appropriate, since the space of adversarial examples that are neither misclassified by humans nor detectable is likely much larger than the space of naturally occuring samples.) The paper proposes a way of plugging the hole by quantizing layer activations. The results show that this makes the system robust to adversarial attacks. Clarity: I spent a lot of time figuring out, as someone who has not spent a lot of time with this, what is being evaluated. It is very unclear whether the non-clean systems in Tables 1 and 2 do apply FGSM etc. also in training (in combination with SQA), or only to the test samples. Table 4, the wording in 4.2, and the wording of the Conclusion indicate that they are. But then, where do I find the accuracy on the naturally-occuring (non-manipulated) samples? The only combination of interpretations that makes sense in the end is to parse "The networks are all trained with fast single-step adversaries" as to mean "The networks are all trained with FGSM", and that the non-Clean columns in Table 1 refer to test data perturbed by the respective method, while the Clean column shows the accuracy on the natural data. This *must* be clarified in the final version, as it took way too long to understand this. I strongly suggest to do this with the naming: change small_full to small_FGSM, and small_SQA to small_SQA+FGSM. Assuming I figured this out right, the tables still lack the baseline accuracy of doing nothing (clean-clean), so one can know how much the nearly-100% use case gets affected. Results: The second concern I have is that, assuming my reading of the results as described above is correct, that the SQA method quite severely affects accuracy on the clean test data, e.g. increasing the error rate on CIFAR by 72% (from 12.33% to 17.06%). There must be a discussion on why such severe performance hit is worth it, especially since there often is an accuracy cliff below which there is a steep loss of usability of a system. For example, according to my personal experience in speech recognition, the difference between 12% and 17% is the difference between decent and unacceptable user experience (also considering that a few percent of errors are caused by ambiguities in the ground-truth annotations themselves, which should be the case for CIFAR as well). Figure 1 seems a little misleading in this regard since the areas of good accuracy are very condensed. It should be rescaled, as only the area close to the optimum performance is relevant. It does not matter whether we degrade from 99.x% to 77% or 58%, or even 95-ish. All of those hurt performance to the point of not being useful. It would be nice to discuss what an accuracy metric would be that is useful for the end user. It would have to be a combination of the expected cost of a misclassification of a natural image and the expected cost caused by attacks. A good method would improve this overall metric. A paper attempting to address adversarial attacks should at least discuss this topic briefly, in my view. Technical soundness: A technical question I have is whether the min-max normalization may be too susceptible to outliers. A single extreme activation can drastically shift the threshold for .lambda=1. How about a mean-var normalization? If there is batch or layer normalization in the system, your activations may already be scaled into a consistent range anyway, that might allow you to use a constant scaling on top of that. Another question I have is: quantization is often modeled as adding uniform noise. Why not add noise directly? And why uniform noise? For example, would compute g = h + Gaussian noise with std dev=(max-min)/lambda work equally well? What is special about quantization? And another technical question: My guess is that the notable loss of accuracy is caused by the strong quantization (two values only in the case of .lambda=1). I think the paper should show results for larger lambdas, specifically whether there is a better trade-off point between the accuracy loss from quantization vs. robustness to adversarial samples. Section 3/SQA: "This is the reason why we rescale g^i to the original range of h^i" This seems wrong. I think the main reason is that one would not want to totally change the dynamic ranges of the network, as it may affect convergence merely by scaling. You*d want to limit any impact on convergence to the quantization itself. Significance: I think the significance is limited. Given that the accuracy impact of the mitigation method is very large, I do not consider this paper as substantially solving the problem, or even bringing a practical solution much closer in reach. Pros: - tnteresting idea; - comparison against various attacks. Cons: - Hard to understand because it was left unclear what is evaluated, at least to readers who are not familiar with a possibly existing implied convention; - The method seems to harm accuracy on clean data a lot, which is the main use case of such a system. I would in the current form reject the paper. To make it acceptable, the clarity of presentation, especially of the results, must be improved, but more importantly, more work seems necessary to reduce the currently significant accuracy hit from the method, and the trade-off of quantization level vs. robustness should be addressed. Minor feedback: Please review the paper for grammar and spelling errors (e.g. "BinaryConnect constraints" or the use of "make", which is often not correct). In Algorithm 1, I suggest to not use *g*, as it may be mis-read as "gradient." Unless this is a common symbol in this context. "Thus, we propose SQA" warrants another .subsubsection{}, to indicate where .subsubsection{BinaryConnect} ends. Section 2.2*s early reference to SQA is a little confusing, since SQA has not formally been defined. I would smooth this a little, e.g. change "SQA can be considered" to "We will see that our SQA, as introduced in the next section, can be considered" "an alternative is to approximate it" probably should be "our approach is to approximate it"