Improved Gradient based Adversarial Attacks for Quantized Networks
Authors: Kartik Gupta
Thalaiyasingam Ajanthan
Venue:  (2021)
Keywords: binary neural network
gradient masking
fake robustness
temperature scaling
adversarial attack
signal propagation

Abstract: Neural network quantization has become increasingly popular due to efficient memory consumption and faster computation resulting from bitwise operations on the quantized networks. Even though they exhibit excellent generalization capabilities, their robustness properties are not well-understood. In this work, we systematically study the robustness of quantized networks against gradient based adversarial attacks and demonstrate that these quantized models suffer from gradient vanishing issues and show a fake sense of robustness. By attributing gradient vanishing to poor forward-backward signal propagation in the trained network, we introduce a simple temperature scaling approach to mitigate this issue while preserving the decision boundary. Despite being a simple modification to existing gradient based adversarial attacks, experiments on CIFAR-10/100 datasets with multiple network architectures demonstrate that our temperature scaled attacks obtain near-perfect success rate on quantized networks while outperforming original attacks on adversarially trained models as well as floating point networks.


Under review as a conference paper at ICLR 2021
IMPROVED GRADIENT BASED ADVERSARIAL ATTACKS FOR QUANTIZED NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Neural network quantization has become increasingly popular due to efficient memory consumption and faster computation resulting from bitwise operations on the quantized networks. Even though they exhibit excellent generalization capabilities, their robustness properties are not well-understood. In this work, we systematically study the robustness of quantized networks against gradient based adversarial attacks and demonstrate that these quantized models suffer from gradient vanishing issues and show fake sense of robustness. By attributing gradient vanishing to poor aawaaddbackward signal propagation in the trained network, we introduce simple temperature scaling approach to mitigate this issue while preserving the decision boundary. Despite being simple modification to existing gradient based adversarial attacks, experiments on CIFAR- 10/100 datasets with multiple network architectures demonstrate that our temperature scaled attacks obtain near perr-eerect success rate on quantized networks while outperforming original attacks on adversarially trained models as well as floating-pott -pintt networks.
INTRODUCTION
Neural Network (NN) quantization has become increasingly popular due reduced memory and time complexity enabling real-time applications and inference on shource-limitee devices. Such quantized networks often exhibit excellent generalization capabilities despite having low capacity due to reduced precision for parameters and activations. However, their robustness properties are not well- understood. In particular, while parameter quantized networks are claimed to have better robustness against gradient based adversarial attacks (Galloway et al. (2018)), activation only quantized methods are shown to be vulnerable (Lin et al. (2019)).
In this work, we consider the extreme case of Binary Neural Networks (BNNs) and systematically study the robustness properties of parameter quantized models, as well as both parameter and activation quantized models against gradient based adversarial attacks. Our analysis reveals that these quantized models suffer from gradient masking issues (Athalye et al. (2018) (especially vanishing gradients) and in turn show fake robustness. We attribute this vanishing gradients issue to poor forward-backe backward signal propagation caused by trained binary weights, and our idea is to improve signal propagation of the network without affecting the prediction of the classifier.
There is a body of work on improving signal propagation in a neural network (e.g., Glorot & Bengio (2010): Pennington et al. (2017); Lu al. (2000), however, we are facing unique challenge of improving signal propagation while preserving the decision boundary, since our ultimate objective to generate adversarial attacks. To this end, we first discuss the conditions to ensure informative
gradients and then resort to temperature scaling approach (Guo et al. (2017)) (which scales the logits before applying softmax cross-entropy) to show that, even with a single positive scalar the vanishing gradients issue in BNNS can be alleviated achieving near perfect success rate in all tested cases.
Specifically, we introduce two techniques to choose the temperature scale: 1) based on the singular values of the input-output Jacobian, 2) by maximizing the norm of the Hessian of the loss with respect to the input. The justification for the first case is that if the singular values of input-output Jacobian are concentrated around 1 (defined as dynamical isometry (Pennington et al. (2017)) then the network is said to have good signal propagation and intend to make the mean of singular

---

Under review as a conference paper at ICLR 2021
values to be 1. On the other hand, the intuition for maximizing the Hessian norm is that if the Hessian norm is large, then the gradient of the loss with respect to the imput sensitive to an infinitesimal change in the input. This is sufficient condition for the network to have good signal propagation as well as informative gradients under the assumption that the network does not have any randomized or non-differentiable components.
We evaluated our improved gradient based adversarial attacks using BNNS with weight quantized (BNN-WQ) and weight and activation quantized (BNN-WAQ), floating point networks (REF), and adversarially trained models. We employ quantized and floating point networks trained on CIFAR- 10/100 datasets using several architectures. In all tested BNNS, both versions of our temperature scaled attacks obtained near- pear-perfect success rate outperforming gradient based attacks (FGSM (Goodfellow et al. (2014)), PGD (Madry et al. (2017))) Furthermore, this temperature scaling improved gradient based attacks even on adversarially trained models (both high-precision and quantized) as well as floating point networks, showing the significance of signal propagation for adversarial attacks.
PRELIMINARIES
We first provide some background on the neural network quantization and adversarial attacks.
2.1 NEURAL NETWORK QUANTIZATION
Neural Network (NN) quantization is defined as training networks with parameters constrained to a minimal, discrete set of quantization levels. This primarily relies the hypothesis that since NNs are usually overparametrized, it is possible to obtain a quantized network with performance comparable to the floating point network. Given a dataset D {x, Y..y.le y. NN quantization can be written as:
min L(w; D) = Σ c(w: (x,,
wEQ
Here, E(.) denotes the input-output mapping composed with a standard loss function (e.g., cross- entropy loss), is the dimensional parameter vector, and Q is a predefined discrete set representing quantization levels (e.g., . Q = {-1, 1} in the binary case).
Most of the NN quantization approaches (Ajanthan et al. (2015 9a (2017) convert the above problem into an unconstrained problem by introducing auxiliary variables and optimize via (stochastic) gradient descent. To this end, the algorithms differ in the choice of quantization set (e.g., keep it discrete (Courbariaux et al. (2015)), relax it to the convex hull (Bai al. (2019) or convert the problem into a lifted probability space (Ajanthan et al. (20T9a)), )), the projection used, and how differentiation through projection is performed. In the case when the constraint
Bai et al. 2019 Hubara et al.
relaxed, gradually increasing annealing hyperparameter quantized solution (Ajanthan et al. (2019a b); Bai et al. (019)). We refer the interested reader to respective papers for more detail. this paper, we use BNN-WO obtained using MD-tanh-S (Ajanthan et al. (2019b)) and BNN-WAQ obtained using Hubara et al. (2017).
used to enforee
2.2 ADVERSARIAL ATTACKS
Adversarial examples consist of imperceptible perturbations to the data that alter the model's predic- tion with high confidence. Existing attacks can be categorized into white-box and black-box attacks where the difference lies in the knowledge of the adversaries. White-box attacks allow the adversaries access to the target model's architecture and parameters, whereas black-box attacks can only query the model. Since white-box gradient based attacks are popular, we summarize them below.
First-order gradient based attacks can be compactly written as Projected Gradient Descent (PGD) on the negative of the loss function (Madry et al. (2017)) Formally, let x" E IR. be the input image, then at iteration t, the PGD update can be written as:
++1 P P
where P RN + X is a projection, X C IRN is the constraint set that bounds the perturbations,
77 > 0 is the step size, and g' a form of gradient of the loss with respect to the input x evaluated at
x With this general form, the popular gradient based adversarial attacks can be specified:
Fast Gradient Sign Method (FGSM): This is a one step attack introduced in Goodfellow et al. (2014). Here, P is the identity mapping, 1) is the maximum allowed perturbation magnitude, and
2

---

Under review as a conference paper at ICLR 2021
ResNet-18
VGG-16
Method
Clean
Adv.(1)
Adv.(20)
Clean
Adv.(1)
Adv.(20)
REF
94.46
0,00
0.00
93.31
0.04
0.00
BNN-WQ
93.18
26.98
17.91
91.53
47.32
38.49
BNN-WAQ
87.67
8.57
1.94
89.69
78.01
59.26
Table 1: Clean and adversarial accuracy (PGD attack with Log bound) on the test set of CIFAR-10 using R8 and VGG-16. In brackets, we mention number of random restarts used perform the atlack. Note, BNNs oulperform adversarial accuracy of floating point networks consistently.
Varying Mack Interations
Varying PGD
Advessora secunscy in 9g
RBF
White Bos Acc.
BNN-NQQ
BAN-WAQ
GRN4-WAQ
Ataak Iterations
Aitack Rodis
SNUWNG
DMI-IN
ENI-WGO
ENN INLAWGO
(b)
(C)
Figure 1: Gradient marking checks in R8 on CIFAR-10 for PGD attack with Loe bound: (a)
varying iterations, (b) varying radius, and (c) black-box atlacks on ResNet-18 18 and VGG-16. While (a), (c) show signs of gradient masking, (b) does not. We atribute this discrepancy to the random initial step before PGD.
gx sign <xf(w";(x++ (x',))), where l denotes the loss function, w" is the trained weights and is the ground truth label corresponding to the image x
PGD with Loo bound: Arguably the most popular adversarial attack introduced in Madry et al. (017)) and sometimes referred to as Iterative Fast Gradient Sign Method (IFGSM). Here, the Loo norm based projection, 1) is chosen step size, and gx sign (V VVl((">> (w"; (x''((("''' yx), the sign of gradient same as FGSM.
2017 which performs the standard PGD in the Euclidean space. Here, P is the L2 norm based projection, r) is a chosen step size, and gx (=( (x", y)) is simply the gradient of the loss with respect the input.
PGD with L2 bound: This is also introduced in Madry al.
These attacks have been further strengthened by a random initial step (Tramrr et al. (2017)) In this paper, we perform this single random initialization for all experiments with FGSM/PGD attack unless otherwise mentioned.
ROBUSTNESS EVALUATION OF BINARY NEURAL NETWORKS
We start by evaluating the adversarial accuracy (i.e. accuracy on the perturbed data) of BNNs using the PGD attack with Log bound.
PGD attack details: perturbation bound of 8 pixels (assuming each pixel in the image is in [0, 255)) with respect to Too norm, step size and the total number of iterations T = 20. The attack details are the same in all evaluated settings unless stated otherwise.
We perform experiments on CIFAR-I0 dataset using ResNet-11 and VGG-16 architectures and report the clean accuracy and PGD adversarial accuracy with 1 and 20 random restarts in Table [ It can be clearly and consistently observed that binary networks have high adversarial accuracy compared to the floating point counterparts. Even with 20 random restarts, BNNs clearly outperform floating point networks in terms of adversarial accuracy. Since this result is surprising. we investigate this phenomenon further to understand whether BNNs are actually robust to adversarial perturbations or they show a fake sense of security due to some form of obfuscated gradients (Athalye et al. (018)).
3.1 IDENTIFYING ORFUSCATED GRADIENTS
Recently, it has been shown that several defense mechanisms intentionally or unintentionally break gradient descent and cause obfuscated gradients and thus exhibit a false sense of security (Athalye et al. (2018). Several gradient based adversarial attacks tend to fail to produce adversarial perturbations in scenarios where the gradients are uninformative, referred to gradient masking. Gradient masking
3

---

Under review as a conference paper at ICLR 2021
oceur due shattered gradients, stochastic gradients or exploding and vanishing gradients. We try to identify gradient masking in binary networks based on the empirical checks provided in AAthalye et al. (2018). If any of these checks fail, it indicates gradient masking issue in BNNs.
To illustrate this, we analyse the effects of varying different hyperparameters of PGD attack on BNNS trained on CIFAR-10 using ResNet-18 architecture. Even though varying PGD perturbation bound does not show any signs of gradient masking, varying attack iterations and black-box vs white-box results (on ResNet-18 18 and VGG-16) clearly indicate gradient masking issues as depicted in Fig. [ The black-box attack outperforning white-box attack for BNNs certainly indicates gradient masking issues since the black-box attack do not use the gradient information from model being attacked. Here, our black-box model to BNN is the analogous floating point network trained on the same dataset and the attack is the same PGD with Tooc bound.
These checks demonstrate that BNNS are prone to gradient masking and exhibit fake robustness. Note, shattered gradients occur due to non-differentiable components in the defense mechanism and stochastic gradients are caused by randomized gradients. Since BNNs are trainable from scratch and does not have randomized gradiente we narrow down gradient masking issue to vanishing or exploding gradients. Since, vanishing exploding gradients occur due to poor signal propagation, by introducing a single scalar, we discuss two approaches to mitigate this issue, which lead to almost 100% success rate for gradient based attacks on BNNS.
4 SIGNAL PROPAGATION OF NEURAL NETWORKS
We first describe how poor signal propagation in neural networks can cause vanishing exploding gradients. Then we discuss the idea of introducing a single scalar to improve the existing gradient based attacks without affecting the prediction (i.e., decision boundary) of the trained models.
We consider a neural network fw for an input x", having logits K = (x²). Now, since softmax cross-entropy is usually used as the loss function, we can write:
E(a .y) = -y log(p).
p softmax(a*)
(3)
where y E IRd is the one-hot encoded target label and log is applied elementwise.
For various gradient based adversarial attacks discussed in Sec. Sec. gradient of the loss é is used with respect to the input x", which can also be formulated using chain rule as,
al(a y) Ox
al(a ,y) 0a Oak
Ox (aaa yy
(4)
where e denotes the error signal and J E R N is the input-output Jacobian. Here we use the convention that /v/Ou is of the form v-size u-size.
Notice there are two components that influence the gradients, 1) the Jacobian J and 2) the error signal vb. Gradient based attacks would fail if either the Jacobian is poorly conditioned the error signal has saturating gradients, both of these will lead to vanishing gradients in öf /0xx
The effects of Jacobian on the signal propagation is studied in dynamical isometry and mean-field theory literature (Pennington et al. (017)) b0: Saxe et al. (2013) and it is known that a network is said to satisfy dynamical isometry if the singular values of J are concentrated near 1. Under this condition, error signals v/ backpropagate isometrically through the network, approximately preserving its norm and all angles between error vectors. Thus, as dynamical isometry improves the trainability of the floating point networks, similar technique can be useful for gradient based attacks as well.
In fact, almost all initialization techniques (e.g.. Glorot & Bengio (010)) approximately ensures that the Jacobian J is cellicondit for better trainability and it is hypothesized that approximate isometry is preserved even at the end of the training. But, for BNNS, the weights are constrained be
(- 3 and hence the weight distribution at end of training is completely different from the random initialization. Furthermore,
not clear that fully-quantized networks can achieve well-conditioned Jacobian, which guided some research activity in utilizing layerwise scalars (either predefined or learned) to improve BNN training [McDonnel] (2018); Rastegari et al. (2016)). We would like to point out that the focus of this paper is to improve gradient based attacks already trained BNNs. To
RNN--W have binary weights, but there non-differentiable randomized component once trained.
4

---

Under review as a conference paper at ICLR 2021
this end learning new scalar to improve signal propagation at each layer is not useful as it can alter the decision boundary of the network and thus cannot be used in practice on already trained model.
4.1 TEMPERATURE SCALING FOR BETTER SIGNAL PROPAGATION
In this paper, we propose to use single scalar per network to improve the signal propagation of the network using temperature scaling. In fact, one could replace softmax with a monotonic function such that the prediction is not altered, however, we will show in our experiments that a single scalar with softmax has enough flexibility to improve signal propagation and yields almost 100% success rate with PGD attacks. Essentially, we can use scalar, 3 > 0 without changing the decision boundary of the network by preserving the relative order of the logits. Precisely, we consider the following:
( ()) softmax(a*))
ak = -Ba
Here, we write the softmax output probabilities p as a function of 3 to emphasize that they are softmax output of temperature scaled logits. Now since in this context, the only variable is the temperature scale B, we denote the loss and the error signal as functions of only B. With this simplified notation the gradient of the temperature scaled loss with respect to the inputs can be written as:
0E(3) )E(3)) Oa Oa
K
Ox oāk Oa Ox (B)
(6)
Note that 33 affects the input-output Jacobian linearly while it nonlinearly affects the error signal vb. To this end, we hope to obtain 3 that ensures the eπror signal is useful (i.e., not all zero) as well as the Jacobian is well-conditioned to allow the error signal propagate to the input.
We acknowledge that while one can find 3 > 0 to obtain softmax output ranging from a uniform distribution (B 0) to one-hot vectors (B + 20), 33 only scales the Jacobian. Therefore, the Jacobian J has zero singular values, our approach has no effect in those dimensions. However, since most of the modern networks consist of ReLU nonlinearities (generally positive homogeneous functions), the effect of a single scalar would be equivalent (ignoring the biases) to having layerwise scalars such as McDonnell (2018). Thus, we believe a single scalar is sufficient for our purpose.
5 IMPROVED GRADIENTS FOR ADVERSARIAL ATTACKS
Now we discuss strategies to choose a scalar 3 such that the gradients with respect to input are informative. Let us first analyze the effect of / on the error signal. To this end,
JSV Mear)
llvl-
aE(3) 0p(3) = -y - -(9)))
(5) = 03(3) Oak
(7)
0.54
022
where y is the one encoded target label, and p(3) is the softmax output of scaled logits.
B
Figure 2: Error signal (x((3) and Ja- cobian of softmax (0p(3) vs. B is no need to generate adversarial examples corresponding for a random correctly classified logits. to misclassified samples. From the above formula, it is clear that when p(3) is one-hot encoding then the error signal is 0. This is one of the reason for vanishing gradient issue in BNNS. Even if this does not happen for a given image, one can increase 3 -> oo to make this error signal O. Similarly, when p(3) is the uniform distribution, the norm of the error signal is at the maximum. This can be obtained by setting 3 = 0. However, this would also make 0l(3)/xx) = 0 as the singular values of the input-output Jacobian would all be O. How error signal is affected by / is illustrated in Fig. 2
For adversarial attacks, we only consider the correctly clas- sified images (i.e,, argmax, M/ = argmax, P) (3)) as there
This analysis indicates that the optimal 3 cannot be obtained by simply maximizing the norm of the error signal and we need to balance both the Jacobian as well as the error signal. To summarize, the scalar 3 should be chosen such that the following properties are satisfied:
|| > P for some p > O. 2. The Jacobian 33 is well-conditioned, i.e., the singular values of ,3 is concentrated around
1.
5.1 NETWORK JACOBIAN SCALING (NIS)
We now discuss a straightforward, two-step approach to attain the aforementioned properties. Firstly, to ensure BJ is well-conditioned, we simply choose be the inverse of the mean of singular values
5

---

Under review as a conference paper at ICLR 2021
of J. This guarantees that the mean of singular values of 3J is 1. After this scaling, it is possible that the resulting error signal is very small. To ensure that [|b( > > O, we ensure that the
softmax output P& (3) corresponding to the ground truth class l: is at least p away from 1. We now state it as a proposition to derive / given a lowerbound on - Pk (3).
Proposition 1. Let ax IRd with d and af 2 a2 ... 2 of and at a ? such that 1 softmax(Baf p. then
For a given < (d- 1)/d, there exists a (1)///
B < - lolooo(p(C
Proof. This derived via a simple algebraic manipulation of softmax. Please refer to Appendix. n
This 3 can be used together with the computed using inverse of mean Jacobian Singular Values (ISV). We provide the pseudocode for our proposed PGD++ (NIS) attack in Appendix. Similar approach can also applied for FGSM+ Notice that, this approach is simple and adds negligible overhead to the standard PGD attacks. However, it has a hyperparameter p which is hand designed. To mitigate this, next we discuss a hyperparameter-Free approach to obtain J.
5.2 HESSIAN NORM SCALING (HNS)
We now discuss another approach to obtain informative gradients. Our idea is to maximize the Frobenius norm of the Hessian of the loss with respect to the input, where the intuition is that the Hessian norm is large, then the gradient 0E/0x is sensitive an infinitesimal change in x". This means, the infinitesimal perturbation in the input is propagated in the forward pass to the last layer and propagated back to the input layer without attenuation (i.e., the returned signal is not zero), assuming there are no randomized or non-differentiable components in the network. This clearly indicates that the network has good signal propagation as well as the error signals are not all zero. This objective can now be written as:
)C((B) 0(x0)²
OJ
[Op(8))) 1)²
(8)
8
argmax 830
argmax
3
(3)
Ixo
B
Jak
8>0
F
The derivation is provided in Appendix. Note, since J does not depend on B, J and OJ /0x are computed only once, 3 is optimized using grid search as it involves only single scalar. In fact, it is easy to see from the above equation that, when the Hessian is maximized, 9 cannot be zero. Simi- larly, (BB) cannot be zero because it is zero, then the pre- diction p(3) is one-hot encoding (Eq. (7). consequently 0p(3)/Oa = and this cannot be a maximum for the for
2.0
Hessian Norm
1:00
8.00
Hessian norm. Hence, this ensures that [|*((3)) some p > 0 and 33 is bounded according to Proposition[] Therefore, the maximum is obtained for a finite value of 3. Even though, it is not clear how exactly this approach
B
0.5 0.5
Figure 3: Hessian norm vs. 3 on ran- dom correctly classified image. The plot clearly shows concave behaviour, would affect the singular values of the input-output Jacobian (3 J), we know that they are finite and not zero. How Hessian norm is influenced by /8 is illustrated in Fig. 3
Furthermore, there are some recent works Dezosoii et (0199; Qin al. (2019)) show that adversarial training makes the loss surface locally linear around the vicinity of training samples and enforcing local linearity constraint on loss curvature can achieve better robust to adversarial attacks. On the contrary, our idea of maximizing the Hessian, i.e,, increasing the nonlinearity of f, could make the network more prone to adversarial attacks and we intend to exploit that. The psuedocode for PGD++ attack with HNS is summarized in Appendix.
6 EXPERIMENTS
We evaluate robustness accuracies of BNNs with weight quantized (BNN-WQ), weight and activation quantized (BNN floating point networks (REF), and adversarially trained networks. We evaluate our two PGD++ variants corresponding to Hessian Norm Scaling (INS) and Network Jacobian Scaling (NJS) on CIFAR-10 and CIFAR-100 datasets with multiple network architectures. Briefly, results indicate that both of our proposed attack variants yield attack success rate much higher than original PGD attacks not only on Lox bounded attack but also on L2 bounded attacks on both floating point networks and binarized networks. Our proposed PGD++ variants also reduce PGD

---

Under review as a conference paper at ICLR 2021
Adversarial Accuracy (%)
Network
FGSM
FGSM+
PGD (L...)
PGD++ (Too)
PGD (L2)
PGD++ (72)
NIS
HNS
NI8
HNS
NJS
HNS
ResNet-18
40.49
3.46
2.51
26.98
0.00
0.00
74.59
0.05
0.05
CIEAR-100 CIFAR-10
VGG-16
57.55
4.00
3,43
47.32
0.00
0.00
61.90
0.35
1.32
ResNet-50
57.62
6.44
5.35
43.14
0.00
0.00
74.75
0.11
0.08
DenseNet-12
26.80
4.67
4.24
0.00
0.00
72.99
0.06
MobileNet-V.
33.50
6.42
5.42
26.86
0.00
0.00
35.22
0.12
0.09
ResNet-18
25.22
14.08
1.80
8.23
2.45
0.00
42.67
6.79
0.26
VGG-16
19.82
7.98
1.76
17.44
0.88
0.16
19.26
3.17
0.63
ResNet-50
37.76
16.33
14.17
25.71
2.33
2.73
38.95
7.9
7.41
DenseNet- 121
28.32
12.21
10.86
8.87
1.15
1.09
43.78
4.54
4.16
MobileNet-Vi 12.09
10.18
8.79
1.44
0.57
0.66
8.97
3.39
3.01
Table 2: Adversarial accuracy the test set for BNN-WQ. Both our NIS and HNS variants consistently outperform original Loo bounded FGSM and PGD attack, and L2 bounded PGD attack.
Adversarial Accuracy (%)
Network
FGSM++
PGD++ (L.x)
PGD+ (L2)
FGSM
PGD (Too)
PGD (T2)
NJS
HNS
NJS
HNS
NJS
ResNet-18
5,55
5.35
0.00
0.00
0.00
45.18
0.09
REF
VGG-16
11.01
10.04
9.66
0.04
0.00
0.00
2.23
0.78
1.10
ResNct-50
21.64
6.08
5.70
0.69
0.00
0.00
65.56
0.07
0.09
DenseNet-122 121
11.40
7.58
7.30
0.00
0.00
0.00
38.15
0.08
0.06
BNN- WAQ
ResNet-18
40.84
19.46
19.09
8.57
0.03
0.04
67.84
2.33
2.59
VGG-16
79.92
15.96
15.39
78.01
0.01
0.02
85.62
0.49
0.62
ResNet-50
33.16
25.89
27.05
0.49
0.23
0,45
32,93
6.68
8.77
DenseNet-121
37.20
23.89
24.69
0.81
0.10
0.18
59.32
3.72
Table 3: Adversarial accuracy the test set of CIFAR-IO for REF and BNN-WAQ. Both our NIS and IINS variants consistently outperform original FGSM and PGD (Loo/ / L2 bounded) attacks.
adversarial accuracy of adversarially trained floating point and adversarially trained binarized neural networks while outperforming much stronger attacks such as DeepFool (Moosavi-Dezfooll et al. (2016) and Brendel & Bethge Attack (BBA) (Brendel al. (2019)). Among variants, even though they perform similarly in our experiments, Hessian based scaling (HNS) outperforms Jacobian based scaling (NJS) in majority of the cases and this difference is significant for one step FGSM attacks. This indicates that nonlinearity of the network indeed has some relationship to its adversarial robustness.
We use state of the art models trained for binary quantization (where all layers are quantized) for our experimental evaluations. We provide adversarial attack parameters used for FGSM/PGD in Appendix and for other attacks, we use default parameters used Foolbox (Rauber et al. (2017). For our HNS variant, we sweep 3 from range such that the hessian norm is maximized for each image, as explained Appendix. For our NJS variant, we the value of p 0.01. fact, our attacks are not very sensitive and provide the ablation study in the Appendix. The PyTorch (Paszke et al. (2017)) implementation of our algorithm will be released upon publication.
6.1 RESULTS
We first compared the original PGD (I2/L.2)) and FGSM attack with both versions (NIS and HNS) of improved PGD++ and FGSM+ attack, on CIFAR- 10/100 datasets with ResNet-18/50 VGG-16, DenseNet-122 121 and MobileNet-V V2 network architectures and the adversarial accuracies for different BNN-WQ are reported in Table Our PGD++ variants consistently outperform original PGD on all networks on both datasets. Even being gradient based attack, proposed PGD++ (Lz/Loo) variants can in fact reach adversarial accuracy close to on CIFAR-10 dataset, demystifying the fake robustness binarized networks tend to exhibit due poor signal propagation.
7

---

Under review as a conference paper at ICLR 2021
Adversarial Accuracy (%)
Network
FGSM
FGSM+
PGD
Deep
BBA
PGD++
FGSM
PGD
B = 0.1
NJS
HNS
= 0.1
Fool
NJS
HNS
REF
62.38
69.52
61.43
61.40
48.73
61.27
51.01
48.43
47.17
48.54
BC
53,91
62,46
52.90
52.27
41.29
54.24
42.65
40.14
39.35
39.34
GD-Lanh
56.13
65.06
55.54
54.81
42.77
56.78
44.78
42.94
42.14
42.30
MD-Lanh-S
55.10
63.42
54.74
53.82
41.34
54.22
43.46
40.69
40.76
40.67
Table 4: Adversarial accuracy the test of CIFAR-IO with RexNet-18 for adversarially trained REF and BNN-WQ using different quantization methods (BC, GD-tanh, MD-tanh-S) S). Our improved attacks are compared against FGSM, L bounded PGD, heuristic choice of 0.1, DeepFool and BBA. Albeit on adversarially trained networks, our methods oulperform all the comparable methods.
Adversarial Accuracy (%)
Network
FGSM
FGSM+
PGD
PGD++
FGSM
(DLR)
NJS
HNS
(DLR)
NJS
HNS
REF
7.62
19.48
5,55
5.35
0.00
0.00
0.00
0.00
BNN-WQ
40.49
19.72
3,46
2.51
26.98
0.00
0.00
0.00
BNN-WAQ
40.84
41.78
19.46
19.09
8.57
4.57
0.03
0.04
REF* 62.38
66.39
61.43
61.40
48.73
49.73
47.17
48.54
BNN-WQ"
55.10
59.14
54.74
53.82
41.34
41.42
40.76
40.67
Table 5: Adversarial accuracy for REF, BNN-WQ, and BNN-WAQ trained on CIFAR-1O using ResNet-18. Here denotes adversarially trained models. Both our NIS and HNS variants consistently outprrorrm Los bounded FGSM and PGD attack performed with Difference of Logits Ratio (DLR) loss instead of cross entropy loss. Notice, FGSM and PGD attack with DLR loss (Croce Hein 2020 perform even worse than their original form on adversarially trained models.
Similarly, for one step FGSM attack, our modified versions outperform original FGSM attacks by significant margin consistently for both datasets on various network architectures. would like to point out such improvement in the above two attacks is considerably interesting, knowing the fact that FGSM, PGD with To attacks only use the sign of the gradients so improved performance indicates, our temperature scaling indeed makes some zero elements in the gradient nonzero. We would like to point out here that one can use several random restarts to increase the success rate of original form of FGSM/PGD attack further but to keep comparisons fair for both original and modified attacks. Nevertheless, as it has been observed in Table Teven with 20 random restarts PGD adversarial accuracies for BNNS cannot reach zero, whereas our proposed PGD++ variants consistently achieve perfect success rate.
single random restart
ImageNet. For other large scale datasets such as ImageNet, BNNs are hard to train with full binarization of parameters and result in poor performance. Thus, most existing works (Yang al. (2019) on BNNs keep the first and the last layers floating point and introduce several layerwise scalars to achieve good results on ImageNet. In such experimental setups, according our experiments, trained BNNs do not exhibit gradient masking issues or poor signal propagation and thus are easier to attack using original FGSM/PGD attacks with complete success rate. In such experiments, our modified versions perform equally well compared to the original forms of these attacks.
experiment,
The adversarial accuracies of REF and BNN- BNN-WAQ trained on CIFAR-I0 using ResNet-18/50 VGG-16 and DenseNet-12 121 for our variants against original counterparts are reported in Table 3 Overall, for both REF and BNN-WAQ, our variants outperform the original counterparts consistently. Particularly interesting, PGD++ variants improve the attack success rate REF networks. This effectively expands the applicability of our PGD++ variants and encourages to consider signal propagation of any trained network to improve gradient based attacks. PGD++ with Loc variants achieve pear-eerfect success rate on all ENN-WAQS. again validating the hypotheses of fake robustness of BNNs.
To further demonstrate the efficacy, first adversarially trained the ONN-WQS (quantized using BC (Courbariaux et al. (2015)), GD-tanb/MD-tanh-s (Ajanthan et al. (2019b))) and floating point networks in similar manner in Madry al. (2017), using Tooe bounded PGD with T
8

---

Under review as a conference paper at ICLR 2021
iterations, 17 2 and 8. We report the adversarial accuracies of Lee bounded attacks and our variants on CIFAR-10 using ResNet-18 in Table [4] These results further strengthens the usefulness of our proposed PGD++ variants. Moreover, with a heuristic choice of 8 = 0.1 to scale down the logits hefore performing gradient based attacks performs even worse. Finally, even against stronger attacks (DeepFool (Moosavi-Dezfooll et al. (20T6), BBA (Brendel al. (20T9)) )) under the same Toxe perturbation bound, our variants outperform consistently on these adversarially trained models. We would like point out that our variants have negligible computational overhead over the original gradient based attacks, whereas stronger attacks are much slower in practice requiring 100-1000 iterations with an adversarial starting point (instead of random initial perturbation).
To illustrate the effectiveness of our proposed variants in improving signal propagation, we compare against gradient based attacks performed using recently proposed Difference of Logits Ratio (DLR) loss (Croce& & Hein (2020)) that aims to avoid the issue of saturating error signals. We show these experimental comparisons performed on ResNet-18 models trained on CIFAR- 10 dataset in Table The attack parameters are same as used for the other experiments. It can be clearly observed that in almost all cases our proposed variants are much better than original form of gradient based attacks performed with DLR loss. The margin of difference is significant in case of FGSM attack and adversarial trained models. Infact, it is important to note that gradient based attacks with DLR loss perform worse on adversarially trained models than the original form of gradient based attacks.
7 RELATED WORK
Adversarial examples are first observed in Szegedy et al. (2014) and subsequently efficient gradient based attacks such as FGSM (Goodfellow et al. ))) and PGD (Madry et al. (2017)) are introduced. There exist recent stronger attacks such as Moosavi- Deztooli et al. (2016); Carlini & Wagner (2017);
Yao et al. (2019); Finlay et al." (2019); Brendel et al. (2019). however, compared to PGD, they are much slower to be used for adversarial training in practice. For comprehensive survey related to adversarial attacks, we refer the reader to Chakraborty et al. (2018).
Some recent works focus on the adversarial robustness of BNNS (Bernhard al. (2019); Sen et al.) (2020): Galloway et al.] (2018); Khalil et al. (2019):[ Lin et al. (2019)) however, a strong consensus on the robustness properties of quantized networks lacking. In particular, while Galloway et al.) (2018) claims parameter quantized networks are robust to gradient based attacks based on empirical evidence. (Lin et al. (2019) shows activation quantized networks are vulnerable to such attacks and proposes a defense strategy assuming the parameters are floating-pois al. (2019) proposes a combinatorial attack hinting that activation quantized networks would have obfuscated gradients issue. [Sen et al. (2020) shows ensemble of mixed precision networks to be more robust than original floating point networks; however Tramer et al. (2020) later shows the presented defense method can be attacked with minor modification in the loss function. In short, although it has been hinted that there might be some sort of gradient masking in BNNS (especially in activation quantized networks), thorough understanding is lacking on whether BNNS are robust, if not what the reason for the inferior performance of most commonly used gradient based attacks on binary networks. We answer this question in this paper and introduce improved gradient based attacks.
Differently, Khalil
8 CONCLUSION
In this work, we have shown that both BNN-WQ and BNN-WAQ tend to show fake sense of robustness on gradient based attacks due to poor signal propagation. To tackle this issue, we introduced our two variants of PGD++ attack, namely NJS and HNS. Our proposed PGD++ variants not only possess near-complete success rate on binarized networks but also outperform standard Lxx and L2 bounded PGD attacks on floating point networks. We finally show improvement in attack success rate on adversarially trained REF and ENN-WQ against stronger attacks (DeepFool and BBA). In future, we intend focus more on improving the robustness of the BNNS with provable robustness guarantees.
REFERENCES
Thalaiyasingam Ajanthan, Puneet K Dokania, Richard Hartley, and Philip HS Torr. Proximal mean-field for neural network quantization. ICCV, 2019a.

---

Under review as a conference paper at ICLR 2021
Thalaiyasingam Ajanthan, Kartik Gupta, Philip HS Torr, Richard Hartley, and Puneet K Dokania. Mirror descent view for neural network quantization. arXiv preprint arXiv:1910.08233 2019b.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query-efficient black-box adversarial attack via random search. In European Conference on Computer Vision, pp. 484-501. Springer, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. arXiv preprint arXiv:/802.00422 2018.
Yu Bai, Yu-Xiang Wang, and Edo Liberty. Proxquant: Quantized neural networks via proximal operators. ICLR, 2019.
Rémi Bernhard, Pierre-Alain Moellic, and Jean-Mat Dutertre. Impact of low-bitwidth quantization on the adversarial robustness for embedded neural networks. In 2019 International Conference on Cyberworlds (CW), pp. 308-315, IEEE, 2019.
Wieland Brendel, Jonas Rauber, Matthias Kümmerer, Ivan Ustyuzhaninov. and Matthias Bethge. Accurate, reliable and fast robustness evaluation. In Advances in Neural Information Processing Systems, pp. 12861-1221 12871, 2019.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, 2017.
Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopad- hyay. Adversarial attacks and defences: A survey, arXiv preprint arXiv: 1810.0080000000 2018.
Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. Binaryconnect: Training deep neural networks with binary weights during propagations. NeurlPS, 2015.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fiee attacks. /CML, 2020.
Chris Finlay, Aram-Alexandre Pooladian, and Adam Oberman. The logharrier adversarial attack: making effective use of decision boundary information. In Proceedings of the IEEE International Conference on Computer Vision, pp. 4862-4870, 2019.
Angus Galloway, Graham W. Taylor, and Medhat Moussa. Attacking binarized neural networks. In International Conference on Learning Representations, 2018.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249-256, 2010.
Ian Goodfellow. Gradient masking causes clever overestimate adversarial perturbation size. arXiv preprint /8Xiv:/804.008700 2018
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 2014.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 1321-1330. JMLR. org, 2017.
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El- El-Yaniv, and Yoshua Bengio. Quantized neural networks: Training neural networks with low precision weights and activations. JMLR, 2017.
Elias B Khalil, Amrita Gupta, and Bistra Dilkina. Combinatorial attacks on binarized neural networks. In International Conference on Learning Representations, 2019.
Lin, Chuang Gan, and Song Han. Defensive quantization: When efficiency meets robustness, In International Conference Learning Representations, 2019.
10

---

Under review as a conference paper at ICLR 2021
Yao Lu, Stephen Gould, and Thalaiyasingam Ajanthan. Bidirectional self self-normalizing neural networks. arXiv preprint arXiv:2006. 12169, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 001177000000000 2017.
Mark McDonnell, Training wide residual networks for deployment using a single bit for each weight. ICLR, 2018.
Seyed- Moyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2574-2582, 2016.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Ro- bustness via curvature regularization, and vice versa. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.
2019.
Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Beat Buesser, Ambrish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, et al. Adversarial robustness toolbox v1. 0.0. arXiv preprint arXiv: 1807.0060, 2018.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary Vetioo, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in PyTorch. 2017.
Jeffrey Pennington, Samuel Schoenholz, and Surya Ganguli. Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice. In Advances in neural information processing systems, pp. 4785-4795, 2017.
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, and Pushmeet Kohli. Adversarial robustness through local linearization. In Advances in Neural Information Processing Systems, pp. 13824-13833, 2019.
Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xnor-net: Imagenet classification using binary convolutional neural networks. ECCV, 2016.
Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: A python toolbox to benchmark the robustness of machine learning models. arXiv preprint arXiv:1707.04131, 2017.
Andrew M Saxe, James L McClelland, and Surya Ganguli. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint 13Xiv:1312.6120) 2013.
Sanchari Sen, Balaraman Ravindran, and Anand Raghunathan. Empir: Ensembles of mixed precision deep networks for increased robustness against adversarial attacks. International Conference on Learning Representations, 2020.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. International Conference on Learning Representations, 2014.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.077 2017.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. arXiv preprint wwXiv:2002.08347, 2020.
Tsui-Wei Weng, Huan Zhang, Pin- Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. Eyaluating the robustness of neural networks: An extreme value theory approach. In International Conference Learning Representations, 2018.
Jiwei Yang. Xu Shen, Jun Xing, Xinmei Tian, Houqiang Li, Bing Deng, Jianqiang Huang, and Xian-sheng Hua. Quantization networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7308-7316, 2019.

---

Under review as a conference paper at ICLR 2021
Zhewei Yao, Amir Gholami, Qi Lei, Kurt Keutzer, and Michael W Mahoney. Hessian-based analysis of large batch training and robustness to adversaries. Advances Neural Information Processing Systems, pp. 4949-4959 2018.
Zhewei Yao, Amir Gholami, Peng Xu, Kurt Keutzer, and Michael W Mahoney. Trust region based adversarial attack on neural networks. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition, pp. 11350-11359, 2019.
Appendices
Here, we first provide the pseudocodes, proof of the proposition and the derivation of Hessian. Later give additional experiments, analysis and the details of our experimental setting.
A PSEUDOCODE
We provide pseudocode for PGD++ with NJS in Algorithm 1 and PGD++ with HNS in Algorithm 2
Algorithm PGD++ with NJS with Loo, T iterations, radius step size v), network fw-, input x". label k. one-hot y (0, 13d, gradient threshold p.
Require: T, P, x0 k ||x
Ensure:
Se 1: 81 dM E (J.))
> 81 computed using Network Jacobian.
2: xl [x + Uniform
& Random Initialization with Projection
3: for + I.....T do
4:
32 -1.0
5:
p' = ( (x'))
6:
if F1- - p/s then
p 0.01
7:
B2 = log(p/ 1)(1 - p))/y
y computed using Proposition 2
8:
= yy log(softmax((2))) (fw- (x!)))
9:
x [+1 Ps (x* signn (issin(((((())) (vx((x)))
Update Step with Projection
Algorithm 2 PGD++ with INS with Loo. I' iterations, radius e, step size 1), network fwi, input x". label k. one-hot y E {0, 17 gradient threshold
Require: T, T, k
Ensure: 1xT+1 x" 100 1o Se
1: x Ps (x" + Uniform( (11ff)mm-111)))
Random Initialization with Projection Grid Search
2: 8 utgmaxg20 (0)))30xxx)))
3: for +- 1,... do
4: {= -y" logg softmaxx 8 (fw- (x))))
5: x++ =p (x' +
n sign V vx((x')))
> Update Step with Projection
B DERIVATIONS
B. DERIVING 3 GIVEN A LOWERBOUND ON 1 - (3)
Proposition 2. Let a E IRd with d > and at 2 ax 2 2 at and at - ag
For a given < < (d - 1)/d, there exists a B > such that 1 - softmax(Baaf) (Ba > p. then
3 3< < log(p/// 1)(1 p))//.
12

---

Under review as a conference paper at ICLR 2021
Proof. Assuming of ad y, derive condition on such that 1 - soft > p.
softmax > p:
softmax (Ba <1 P,
d
exp(Baft)
(B))) k <1 -
A=1
1/(1 Σ exp(3(a -))))) -p.
Since, of -ax S for all A> 1.
d
1/(1+ + Σ
exp(3(a - & 11((1+
Σ A=2
exp(-37))
(10)
2=2
Therefore, to ensure 1/(1 + Ex= exp(3(a - a) < 1 - p. we consider,
d
1/(1+ + Σ
exp( exp(-BA1) <1 p. - S for all À > 1,
(11)
A=2
1/(1 + (d - exp(-33r < <1-p:
exp(-B7) > p/(d - [[(d-)](1--))))) - p)
37 log(p/ - 1)(1 - p))
exp is monotone,
B < log(p/(d (d - ([l1-))/7>
Therefore for any 3 < log(p/(d 1)(1 p))/7, the above inequality
1 (oftmax(Baf] p satisfied.
B.2 DERIVATION OF HESSIAN
We now derive the Hessian of the input mentioned in Eq. (8) of the paper. The input gradients can be written as:
0E(3)
Op(3) 0p(3) da (3)
0E(3)
3J = (B)33J.
(12)
0x"
Now by product rule of differentiation, input hessian can be written as:
(P((3)
OJ
(a)(3))
T
3
(B3
+
J
(13)
0(x0)2
Ox
Ox
OJ
(Op(3)
8
(
+
J
xb( = = - -
Ox"
Ox
OJ
@pp(3)
3
(
+
3
OxU
Oak
C ADDITIONAL EXPERIMENTS
In this section we first provide more experimental details and then some ablation studies.
C.1 EXPERIMENTAL DETAILS
13

---

Under review as a conference paper at ICLR 2021
ResNet-11
VGG-16
Method
Square Attack
PGD++
Square Attack
PGD++
APGD
APGD
NJS
HNS
NJS
HNS
REF
0.00
0.55
0.00
0.00
0.79
2.25
0.00
0.00
BNN-WQ
0.00
0.41
0.00
0.00
8.23
1.98
0.00
0.00
BNN-WAQ
6.32
21.45
0.03
0.04
0.38
16.67
0.01
0.02
Table 7: Adversarial accuracy for REF, BNN-WQ and BNN-WAQ trained on CIFAR 10 using ResNet- 18.
Both NJS and HNS variants consistently outperform Auto-PGD (APGD) (Croce & Hein (020
performed using Difference of Logits Ratio (DLR) loss and a gradient free attack namely, Square Attack (Andriushchenko et al. (2020)) under Lox. bound (8/255).
We first mention the hyperparameters used to perform FGSM and PGD attack for the experiments in the keep
Dataset
Attack
paper in T To make fair comparison, the attack parameters same for our proposed variants of FGSM++ and PGD++ attacks. For PGD++ with HNS variant, we maximize Frobenius norm Hessian with respect to the input specified in Eq. (8) of the pa- per by grid search for the optimum 3. We would like point out that since only (3) and p(3) terms are dependent on B, we do not need to do forward and backward pass of the network multiple times during the grid search. This significantly reduces the computational overhead during the grid search. We can simply use the same network outputs a and network jacobian J (as computed without using (9) for the grid search, while computing the other terms at each iteration of grid search. We apply grid search to find the optimum beta between 100 equally spaced intervals of 3 starting from B1 to 82. Here, 3₁ and 33 are computed based on Proposition in the paper where p 1e 72 and number of classes and y = K a² so that softmax (Baff < p. Also, note that we estimate the optimum 3 for each test sample only the start of the first iteration of an iterative attack and then use the same B for the next iterations.
FGSM
8
8
CIFAR-10
PGD (Loc)
8
2
PGD (L2)
120
15
20
FGSM
CIFAR-100
PGD (Loc)
4
1
10
PGD (L2)
60
15
10
Table 6: Attack parameters (e & in pixels).
(1/d) (le 2) respectively, where d is
Computational Overhead of NJS and HNS. Our Jacobian calculation takes just single backward pass through the network and thus adds negligible overhead. Our NJS approach for scaling estimates 3 as inverse of mean ISV using 100 random test samples, which is similar to 100 backward passes. For HNS, in Eq. (8) Jacobian can be computed in single backward pass. Moreover, for piecewise linear networks (eg, relu activations), aJ/Ox" = 0 almost everywhere Yaoo et al. (018)). Thus PGD++ with NJS and HNS is almost as efficient as PGD.
C.2 COMPARISONS AGAINST AUTO-PGD ATTACK AND GRADIENT FREE ATTACK
We also compared our proposed PGD++ variants against recently proposed Auto-PGD (APGD) with Difference of Logits Ratio (DLR) loss (Croce& & Hein (2020) and gradient free Square Attack (An driushchenko et al. (2020)) on different networks trained using ResNet-18 18 and VGG- 16 on CIFAR-I0 dataset and the results are reported in Table The attack parameters for this experiment are the same reported in the paper. It can be clearly seen that proposed variants perform much better than both APGD with DLR loss and Square Attack, consistently achieving 0% adversarial accuracy. Infact, much computationally expensive Square attack is unable to achieve 0% adversarial accuracy in any of the cases under the enforced Lex bound.
C.3 OTHER EXPERIMENTS
We provide adversarial accuracy comparisons for different attack methods CIFAR- using ResNet- 18, VGG-16, ResNet-50 and DenseNet-121 Table g Again similar to the results in the paper, our proposed PGD++ and FGSM+ outperform original form of PGD and FGSM consistently in all the experiments on floating point networks. We also provide adversarial accuracy comparison of our proposed variants against stronger attacks namely DeepFool (Moosavi-Dezfooli et al. (2016) and
14

---

Under review as a conference paper at ICLR 2021
Adversarial Accuracy (%)
Network
FGSM++
PGD++ (Ioo)
PGD++ (I2)
FGSM
PGD (Loo)
PGD (L2)
NJ8
HNS
NJS
HNS
NJS
HNS
ResNet-18
9.06
9.23
2.70
0.14
0.14
0.00
5.38
0.17
0.15
VGG-16
16.28
17.24
9.19
1.53
0.95
0.25
4.87
1.50
1.38
ResNet-50
12.95
12.95
11.94
0.12
0.00
0.00
31.01
4.43
4.14
DenseNet-121
11.41
11.41
10.74
0.00
0.00
0.00
6.10
3.09
2.76
Table 8: Adversarial accuracy on the test set of CIFAR-100 for REF (floating point networks). Both our
NIS and IINS variants consistently outperform original FGSM and PGD (Lez) L2 bounded) attacks.
Network
PGD
Deep Fool
PGD++
NJS
HNS
ResNet-18
8.57
18.92
0.81
0.03
0.04
VGG-16
78.01
12.12
0.10
0.01
0.02
Table 9: Adversarial accuracy on the test set of CIFAR-10 for BNN-WAQ. Here, we compure our proposed variants against much stronger attacks namely DeepFool (Moosavi- Dezfooli et al. (2166) and BBA (Brendel et al. (2019)) Both our variants outperform stronger attacks. Note, DeepFool and BBA are much slower in practise requiring 100-1000 iterations. BBA specifically requires even an adversarial start point that needs to be computed using another adversarial attack.
BBA (Brendel et al. (2019) on BNN-WAQ trained on CIFAR- 10 dataset Table 9 In this experiment, our proposed variants again outperform even the stronger attacks which take 100-1000 iterations with adversarial start point (instead of random initial perturbation). It should be noted that although BBA performs much better than DeepFool and PGD, it still has inferior success rate than ours considering the fact that takes multiple hours to run BBA whereas our proposed variants are almost as efficient as PGD attack.
Step Size Tuning for PGD attack. We would like to point out that step size and temperature scale have different effects in the attacks performed. Notice, PGD and FGSM attack under L bound only use the sign of imput gradients in each gradient ascent step. Thus, if the imput gradients are completely saturated (which is the case for BNNs), original forms of PGD FGSM will not work irrespective of the step size used. To illustrate this, we performed extensive step size tuning for original form of PGD attack different ResNet-18 18 models trained CIFAR-10 dataset and the adversarial accuracies reported in Fig. 4 It can be observed clearly that although tuning the step size lowers adversarial accuracy a bit some cases but still cannot reach zero for BNNS unlike our proposed variants.
Varying PGD Atlack Step Size
Adversarial training using PGD++ also investigate the potential application of PGD++ for adversarial training to improve the robustness of neural networks. PGD++
Adversanal accuracy 96)
REF
BNN-WAQ
attack is most effective when applied to network with
poor signal propagation. However, adversarial training is performed from random initialization (Glorot & Bengio (2010)) exhibiting good signal propagation. Thus, PGD and PGD++ perform similarly for adversarial training. We
Atlack Step Size
infer these conclusions from our experiments on adversarial training using PGD++.
Figure 4: Adversarial accuracy using PGD attack under Lxc bound (8/255) with varying step size (T)) on ResNet- 18 trained on CIFAR-10. Notice, PGD at- tack is unable to reach zero adversarial accuacy for BNNS with any step size.
CLEVER Scores. Recently CLEVER Scores (Weng et al. (2018) have been proposed as an empirical estimate measure robustness lower bounds for deep networks. It has been later shown that gradient masking issues cause CLEVER to overestimate the robustness bounds (Goodfellow (018)). Here we try improve the CLEVER scores using different ways choosing 3 in temperature scaling. For this experiment, CLEVER implementation of Adversarial Training Toolbox" (Nicolae et al. (0188). We set
15

---

Under review as a conference paper at ICLR 2021
Original
Heuristic
NJI8
HNS
BNN-WQ 0.8585
0.8845
0.4139
0.3450
BNN-WAQ
0.7239
3.1578
0.3120
0.2774
Table 10: CLEVER Scores Weng al. (2188) for BNN-WQ and BNN- WAQ trained on CIFAR-IO using ResNet-18. We compare CLEVER Scores returned for L1 nom perturbation using different ways of temperature scaling applied. Here, Original refers to original network without temperature scaling and Heuristic denotes temperature scale with small 3 = 0.01.
PGD++ (NJS) Varying
Methods
1c 05
le 04
1c
03
le 02
1c 01
2e
01
REF
0.00
0.00
0.00
0.00
0.00
0.00
BNN-WQ
0.00
0.00
0.00
0.00
0.00
0,00
BNN-WAQ
0.15
0.08
0.04
0.03
0.04
0.02
Table 11: Adversarial accuracy on the test set for binary neural networks using Loe bounded +GD++ attack using NIS with varying For different values of p. our approach is quite stable.
number of batches to 50, batch size to 10, radius to 5, and chose I1 norm as hyperparameters (based on the Weng et al. (2018)). We compare our variants namely NJS and HINS against heuristic choice of small 3 = 0.01 and original CLEVER Scores for BNN-WQ and BNN-WAQ (trained CIFAR- 10 using ResNet-18) in Table I0 can be clearly seen that our proposed variants improve the robustness bounds computed using CLEVER whereas heuristic choice of B 0.01 performs even worse.
C.4 STABILITY OF PGD++ WITH NIS WITH VARIATIONS IN
We perform ablation studies with varying for PGD++ with NIS in Table 11 for CIFAR-10 dataset using ResNet-18 18 architecture. It clearly illustrates that NJS variant is quite robust to the choice of p as we are able to achieve near perfect success rate with PGD++ with different values of p. As long as value of p is large enough to avoid one-hot encoding softmax outputs (in turn avoid [|][]]]] to be zero) of correctly classified sample, our approach with NJS variant quite stable.
C.5 SIGNAL PROPAGATION AND INPUT GRADIENT ANALYSIS USING NJS AND HNS
We first provide example illustration in Fig. 5 to better understand how the input gradient norm i... |00(3)/xxx|| and norm of sign of input gradient, i.e.,
is influenced by 8. It clearly shows that both the plots have concave behavior where optimal 3 can maximize the input gradient. Also, it can be quite evidently seen in Fig. (b)) that within an optimal range of 3, gradient vanishing issue can be avoided. If B 0 or 3 - it changes all the values in input gradient matrix to zero and inturn [ssign
I|2 = 0.
We also provide the signal propagation properties well analysis input gradient norm before and after using the estimated based on NJS and HNS in Table 12 For binarized networks as well floating point networks tested CIFAR-10 dataset using ResNet- 18 architecture, our HNS and NJS variants result in larger values for | | ())33) /xx011 ||2 and [|sign(AE(()) /xx This reflects the efficacy of our method in overcoming the gradient vanishing issue. It can be also noted that our variants also improves the signal propagation of the networks by bringing the mean JSV values closer 1.
C.6 ABLATION FOR P VS. PGD++ ACCURACY
In this subsection, we provide the analysis the effect of bounding the gradients of the network output of ground truth class k i.e. @E(3) K Here, we compute 3 using Proposition for all correctly classified images such that softmax(3aa > with different values of and report the PGD++ adversarial accuracy in Table I3 It can be observed that there is optimum value of at which PGD++ success rate is maximized, especially on the adversarially trained models. This can also be seen in connection with the nineaaiyyy of the network where optimum value of
16

---

Under review as a conference paper at ICLR 2021
29
sign (atlax"))
||aaaxx
64-
332 6.5
Figure 5: Plots to show how variation in 3 affects (a) norm of input gradient, i.e. [[[(3) /0x") f 1| (b) norm of sign of input gradient, i.e. (s)gg(dd(B)//) /xxx)12 on random correctly classified image. Notice that, both inpul gradient and signed input gradient norm behave similarly, showing concave behaviour. This plot is computed for BNN-WQ network on CIFAR-10 ResNet- 18. (b) clearly illustrates how optimum 3 can avoid vamishing gradient issue since |sign (8((3) 00xx)/2 will only be zero if input gradient matrix has only zeros.
Methods
REF
Adv. Train
BNN-WQ
BNN-WAQ
Orig.
8.09e-+00
5.15e-01
3.53e+
1.11e+000
JSV (Mean)
NJS
9.51e-01
5.70e-01
9.95e-01
2.24e-01
HNS
2.38e++00
6.11e++00
1.19e+00
4.65e+000
Orig.
6,27e-400
4.10e-01
3.53e
1,97e+000
JSV (Std.)
NJS
7.58e-01
6.34e-01
9.71e-01
6.73e-01
HNS
4.41e+400
5.34e
2.13e++02
1.24e++02
Orig.
9.08e
2.33e 01
6.20e
9.46e
||
NJS
4.66e 01
2.35e-00
5.37e 01
1.20e 01
HNS
1.48e 01
2.57e 01
2.07e-10 01
2.1e 01
Orig.
2.42e 01
8.52e-00 02
2.27e-01 01
6.33e 02
||0e/xx0|2
NJS
9.52e-00 01
1.10e 01
8.91e-01
1.21e 01
HNS
7.49e-01
8.18e-01
3.70e-01
2.70e-01
Orig.
5.55e+001
5.54e+01
4.39e+001
5.55e+001
||iinn 8x
l|2
NJS
5.55e+01
5.54e+01
5.55e+01
5.55e+01
HNS
5.55e+01
5.54e+001
5.55e++01
5.55e+01
Table 12: Mean and standard deviation of Jacobian Singular Values (ISV), mean |/bll2, mean
||ae 0xx112 and mean |[sign(0 (0//xx)))))))))))))))))())))(( /xx²) for different methods on CIFAR-10 with ResNet-18 computed with 500 correctly classified samples. Note here for NIS and WNS, JSV is computed for scaled jacobian and (sign()((3) (E(()) l2 are larger for our NIS and HNS variant (for most of the networks) as compared with network with no 3, which clearly indicates better gradients for performing gradient based attacks.
i.e. 3J. Also note that, values of [|/]|2. ||0((3)/xx*
3, even for robust (locally linear) -eosavi-Dezfooli et al. (201); Qin et al. (201)) networks such as adversarially trained models, non non-linearity can be maximized and better success rate for gradient based attacks can be achieved. Our HNS variant essentially tries to achieve the same objective while trying to estimate 3 for each example.
17

---

Under review as a conference paper at ICLR 2021
PGD++ with Varying P
Methods
1e
1e
09
05
1e 01
2r.
01
REF
0.00
0.00
0.00
0.00
0.00
BNN-WQ
9.61
0.04
0.00
0.00
0.00
0.00
REF"
48.18
47.66
48.00
53.09
54.58
57.57
BNN-WQ"
40.66
40.01
40.04
45.09
46.57
49.72
Table 13: Adversarial accuracy on the test set for adversarially trained networks and binary neural networks using L.x. bounded PGD++ attack with varying lower bound on the gradient of network output for ground truth class Here denotes the adversarially trained models obtained where adversarial samples are generated using Loo bounded PGD atlack with with T iterations, 2 and e = Note, here PGD++ attack refers to PGD attack where (t(B) is bounded by p for each sample, where k is ground truth class.
18


Reviews:
[{"review_id": "s442b6v_ki", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies the robustness of quantized neural networks against adversarial attacks. The authors use some slight modification of existing methods to successfully increase the attack success rate. In general, I think the idea is interesting. But I have some concerns that need to be addressed:\n1. I am not fully convinced by the arguments made at the beginning of Section 4. The authors claim that poor signal propagation such as gradient vanishing or gradient exploding should be a problem for adversarial attacks. However, I do not think the reasonings provided here is specifically for binarized neural networks. Equations (3) and (4) also works in regular full precision networks. I do not think there are any problems which only present in BNNs, so the arguments here are not strong enough. If poor signal propagation is a problem for attacks, why we don't see that in full precision networks? More discussions on this are welcomed.\n2. ResNet and DenseNet REF models in Table 3 seem to be surprisingly robust under PGD L2 attacks (column 3). This adversarial accuracy seems to be comparable to models with adversarial training. I think the authors need to provide some explanation on this. \n3. (Minor) Please refrain from only using color to distinguish curves/bars in figures as it may not be friendly to readers with color blindness.\n4. (Minor) The authors may need to re-organize some sections to make the paper easier to follow, for example, the \"related works\" section before the \"experiments\" section.", "extra_scores": {}, "extra_text": {}}, {"review_id": "Z9NWC7k0q-4", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper identifies the gradient vanishing issue in the robustness of binary quantized networks. Therefore, it proposes to use temperature scaling approach in the attack generation. It has two methods for the temperature scale: (1) singular values of the input-output Jacobian and (2) maximizing the norm of the Hessian of the loss.\n\n------------Updates after rebuttal------------- \nThanks the authors for answering my questions. However, I don't think my comments are well addressed. Even though the paper [d] may not provide public available code, the authors could either use results from the paper [d] or implement the proposed attack on the models used by [d] to see the difference.\n\nStrengths:\n\n+ The proposed method work well on adv trained models and floating-point models.\n\n+ Practical approach by a simple modification to existing gradient based attacks.\n\n\nWeaknesses:\n\n- Binary quantization is not a well accepted method, since it can in general introduce >5% accuracy loss. There are a lot more valuable quantization schemes to investigate such as low-bit-with fixed point, power of 2, and additive power of 2 (Y. Li, X. Dong, and W. Wang, “Additive powers-of-two quantization: An efficient non-uniform discretization for neural networks,” in International Conference on Learning Representations, 2020).\n\n- The novelty is limited, since it brings the temperature scaling approach, an existing method, to the problem of attacking binary quantized models.\n\n- The paper writing is not constructed for easy understanding. \n\n\nComments and questions:\n\n1. I would like to see comparisons with other attacks that are particularly designed for quantized models.\n\n2. The third paragraph in Introduction, the paper tries to justify two techniques, but they are still not well motivated. \n\n3. The fourth paraph in Introduction mentions both full precision networks and floating-point networks. What’s the difference between these two?\n\n4. Table 1 results are surprising. Is the same observation made by other ref works? \n\n5. The method is to replace the softmax with a monotonic function (softmax with a single scalar) during the attack generation. Then for testing the attack success rate, I think the neural network should still use the original softmax (without scalar). Then the attack success rate won’t be degraded?", "extra_scores": {}, "extra_text": {}}, {"review_id": "RXMZIKm0_eP", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work starts by questioning the apparent robustness of quantized networks and demonstrates that such robustness is more so a failure of the attack algorithm in picking up the gradient signal. The authors address this by tuning a scalar multiplier applied to the network logits, which doesn’t modify the model’s decision boundary. Through analyzing the Jacobian, two approaches are proposed to determine the scalar $\\beta$ without tuning it by performing the attack. This approach is quite effective on quantized networks and even provides significant improvement on floating-point networks combining with existing attacks like FGSM and PGD. The proposed modification might seem trivial at first, but it constitutes an important factor the community hasn’t taken notice of, to the best of my knowledge.\n\nA few questions: 1) I don’t see any mentions of tuning the attack step size; if we set the new $\\eta$ to $\\eta/\\beta$, we can keep the Jacobian intact and isolate the effect of temperature scaling for XENT. 2) how about sweeping $\\beta$ and plotting against adversarial accuracy? This is surely expensive, but it would paint a clearer picture of the optimality of $\\beta$ found using the proposed approaches. This can be done in tandem with an attack step of $\\eta/\\beta$.\n\nI like the result overall, though the effect of $\\beta$ on the Jacobian and the softmax can and should be separated, if my understanding is correct. The proposed approaches for determining $\\beta$ largely depend on the Jacobian; therefore, there should be more investigation on if scaling the Jacobian correctly is more important than getting good error signals from softmax.", "extra_scores": {}, "extra_text": {}}, {"review_id": "KDQQdLCz04e", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Update**: Thanks to the authors for addressing my comments. As it was pointed out by the authors, temperature rescaling is mostly applicable to non-linear loss functions. For linear loss functions, temperature scaling only linear rescales the gradients. The difference between the proposed PGD++ attack and PGD with linear DLR loss is small (see the author's response to AR4). The improvements are most significant for FGSM but FGSM is not recommended for the robustness evaluation. Given the limited technical novelty and small improvements for linear loss functions, my score remains unchanged.\n\n###### Summary\nThe paper studies the robustness of binary neural networks (BNNS), which at first look have higher robustness than full-precious neural networks. The authors highlight the problem of poor signal propagation in BNNs, which makes gradient-based attacks difficult. To address this issue, the authors proposed a 1) single scalar rescaling of the jacobian to improve the signal propagation; 2) parameter-free hessian norm scaling technique. In the experiments, the authors demonstrated that the modified attacks reduce the accuracy of BNNs to zero and outperform existing gradient-based attacks against floating-point networks.\n\n###### Reasons for score\n\nI vote for a weak acceptance of this paper. The paper shows that BNNs are not robust and introduce an interesting gradient rescaling technique, which can also be used to attack full-precision networks. The rescaling technique is well explained, easy to apply for any existing attacks, and has low computational overhead. However, as I will discuss below, I see some problems comparing the proposed attack against a well-tuned PGD attack.\n\n###### Pros:\n1) The paper studies the robustness of BNNs. The robustness of BNNs is not well studied, and understanding the robustness of BNNs is an important research direction.\n2) The authors highlight the issue of signal propagation in BNNs. To address this issue, they devise a novel low-computational complexity technique.\n3) Experimental results for BNNs and full-precious models demonstrate that the modified attack is effective.\n\n###### Concerns and questions:\n- In the experiments, the authors use a single parameter for the step size for both PGD and FGSM attacks. On the other hand, the proposed method computes an optimal rescaling to achieve good signal propagation. Even though the proposed technique has a low computational budget, I believe the authors should do a grid search for the optimal step size for PGD and FGSM attacks for a fair comparison.\n- In the experiments, PGD L2 attack was unable to reduce the naturally trained models' accuracy to 0. This seems strange and unlikely as gradient shattering should not happen for naturally trained models. Can the authors explain these results? Is it possible that there might be an implementation issue?\n\n###### Comments and suggestions:\n- The proposed technique amplifies the error signal in a nonlinear way for nonlinear losses such as cross-entropy. However, for other losses such as multiclass-hinge loss, CW loss, the proposed method will simply linearly rescale the error signal. Attacking CW loss might be useful as it avoids the issues of saturated softmax gradients. Will this technique be useful for attacking with CW loss? \n- The authors claim that this method improves the efficiency of white-box attacks against full precision models. Is it possible for the authors to get the results for mnist_challenge and cifar10_challenge to see if the method outperforms an optimally tuned PGD attack?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "11aKGkVvBSm", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Update** : Since most of my issues have been addressed, I have changed my rating from 4 to 6\n\nSummary:\n\nThis paper studies the robustness of quantized networks against gradient-based adversarial attacks (for L2 and Linf norms), showing how quantized models suffer from gradient vanishing, giving a false sense of security via gradient masking. To circumvent this issue, the authors propose temperature scaling approaches that can overcome this masking, achieving near-perfect perfect success in crafting adversarial inputs for these models.  \n\n##########################################################################\n\nReasons for score: \n\nThe paper's ultimate goal is to get better gradient-based attack performance on quantized (binarized, in this case) networks. However, key steps that should have been tried first for benchmarking such as adaptive PGD attacks have not been performed. Moreover, it is not clear what benefit the proposed method has in this scenario compared to gradient-free attacks like Boundary++.  The paper's contributions, although including some nice analyses on temperature scaling based solutions, are too weak to be accepted in their current form.\n\n \n##########################################################################\n\nPros: \n  \n- Improvement in attack success rates for full-precision networks, even for FGSM, seems like an exciting result. Further analyses and methods on top of this could be used to further increase the strength of these first-order gradient attacks.\n\n- Jacobian and Hessian based detailed analyses of temperature scaling, and what different solutions correspond to in terms of robustness is quite insightful and interesting.  \n\n##########################################################################\n\nCons: \n\n- Gradient masking is a relatively well-known phenomenon in adversarial machine learning. In cases when normal first-order gradient attacks fail, techniques like adaptive PGD attacks, gradient-free attacks, or even black-box transfer attacks are some straightforward methods to overcome gradient masking. Thus, it is not clear why the authors did not try non-gradient attacks before jumping to a complicated algorithm. At the very least, those attacks (like Boundary++) should at least be part of benchmarks for comparison. \n  - For starters, please refer to 'Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks': they have a [publicly available implementation](https://github.com/fra31/auto-attack ) as well\n  - All of this is crucial, especially since the paper claims (Section 4) that \"we would like to point out that the focus of this paper is to improve gradient-based attacks on already trained BNNs\"\n\n- In general, investigating if a model exhibits gradient masking is not a contribution: standard checks like comparing transfer rates, multi-step to single-step performance, attack rates for increasing attack budgets, etc are often used to check for gradient masking. \n\n- Figure 1: For which norm are these numbers reported? Without knowing the norm, it is hard to say if Figure b is a sign of gradient masking or not.\n\n- Section 2.2 \"..these attacks have been further strengthened by a random initial step\". This is partially true: the real benefit comes from having multiple random restarts. Having just one random initialization by itself is not that useful. Please re-run evaluation experiments with random restarts (20 is a good number).\n\n- Section 3: What does \"adversarial accuracy\" refer to? Is it accuracy on perturbed inputs f(x') = y, or success rate of the adversary when trying to change predictions aka f(x) ~= f(x')? Please clarify\n\n- Section 3.1 \"... clearly indicate gradient masking issues..\" please elaborate: not every reader will be familiar with the set of checks used for gradient masking.\n\n- Issues with Cross-Entropy based loss and how they promote certain magnitudes of logit values are not new. The authors might want to have a look at Section 4.1 of [Reliable Evaluation of Adversarial Robustness with an Ensemble of Diverse\nParameter-free Attacks](https://arxiv.org/pdf/2003.01690.pdf) to see if there are similarities/differences in the proposed temperature-based variant, and how the proposed method is better than the one in the Difference of Logits-Ratio based loss? This work seems to be a key and relevant part of related work and should be included in comparisons/benchmarking.\n \n- \"implementation of our algorithm will be released upon publication\". Please anonymize and attach the code in response. \n\n- The benefit of using the proposed FGSM++/PGD++ attacks on full-precision models trained with adversarial robustness seems to be negligible (Table 4), and should not be overstated in results. Also, since these attacks all have random seeds, please perform experiments multiple times for statistical significance and report summary statistics. \n\n##########################################################################\n\nMinor Edits:\n\n- Section 2.2 \"..perturbations to the images...\" the definition here is for adversarial examples in general, and should thus be \"perturbations to data\"\n\n- Section 2.2 \"Gradient-based attacks can be... written as Projected Gradient Descent (PGD)\" this is true only for first-order gradient-based attacks, not all gradient-based attacks (examples JSMA). Please correct.\n\n- Section 4.1 \"...since most of the modern networks consist of ReLU nonlinearities\" this can (and often is) circumvented using Fake-ReLU. Example implementation [here](https://github.com/MadryLab/robustness/blob/89bdf8088a8f4bd4a8b86925a2801069ec281fee/robustness/tools/custom_modules.py#L5)\n\n- Section 5 \"...and they hypothesize that linear networks would be robust to adversarial attacks.\" this is not their conclusion, and seems to be out of context.\n\n- Section 6 should preferably be either towards the end or at the beginning? Not clear why it is in the middle of other sections\n\n\nPlease address and clarify the cons above ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studies the robustness of binary neural networks (BNNS), showing how quantized models suffer from gradient vanishing. To solve this issue, the authors propose temperature scaling approaches that can overcome this masking, achieving near-perfect perfect success in crafting adversarial inputs for these models. The problem is interesting and important. However, the major concerns are that the technical novelty is limited raised by two Reviewers, small improvements for linear loss functions. The most related work is not compared in the experiment. 

Decision: Reject