Improved Gradient based Adversarial Attacks for Quantized Networks

1 INTRODUCTION . Neural Network ( NN ) quantization has become increasingly popular due to reduced memory and time complexity enabling real-time applications and inference on resource-limited devices . Such quantized networks often exhibit excellent generalization capabilities despite having low capacity due to reduced precision for parameters and activations . However , their robustness properties are not wellunderstood . In particular , while parameter quantized networks are claimed to have better robustness against gradient based adversarial attacks ( Galloway et al . ( 2018 ) ) , activation only quantized methods are shown to be vulnerable ( Lin et al . ( 2019 ) ) . In this work , we consider the extreme case of Binary Neural Networks ( BNNs ) and systematically study the robustness properties of parameter quantized models , as well as both parameter and activation quantized models against gradient based adversarial attacks . Our analysis reveals that these quantized models suffer from gradient masking issues ( Athalye et al . ( 2018 ) ) ( especially vanishing gradients ) and in turn show fake robustness . We attribute this vanishing gradients issue to poor forward-backward signal propagation caused by trained binary weights , and our idea is to improve signal propagation of the network without affecting the prediction of the classifier . There is a body of work on improving signal propagation in a neural network ( e.g. , Glorot & Bengio ( 2010 ) ; Pennington et al . ( 2017 ) ; Lu et al . ( 2020 ) ) , however , we are facing a unique challenge of improving signal propagation while preserving the decision boundary , since our ultimate objective is to generate adversarial attacks . To this end , we first discuss the conditions to ensure informative gradients and then resort to a temperature scaling approach ( Guo et al . ( 2017 ) ) ( which scales the logits before applying softmax cross-entropy ) to show that , even with a single positive scalar the vanishing gradients issue in BNNs can be alleviated achieving near perfect success rate in all tested cases . Specifically , we introduce two techniques to choose the temperature scale : 1 ) based on the singular values of the input-output Jacobian , 2 ) by maximizing the norm of the Hessian of the loss with respect to the input . The justification for the first case is that if the singular values of input-output Jacobian are concentrated around 1 ( defined as dynamical isometry ( Pennington et al . ( 2017 ) ) ) then the network is said to have good signal propagation and we intend to make the mean of singular values to be 1 . On the other hand , the intuition for maximizing the Hessian norm is that if the Hessian norm is large , then the gradient of the loss with respect to the input is sensitive to an infinitesimal change in the input . This is a sufficient condition for the network to have good signal propagation as well as informative gradients under the assumption that the network does not have any randomized or non-differentiable components . We evaluated our improved gradient based adversarial attacks using BNNs with weight quantized ( BNN-WQ ) and weight and activation quantized ( BNN-WAQ ) , floating point networks ( REF ) , and adversarially trained models . We employ quantized and floating point networks trained on CIFAR10/100 datasets using several architectures . In all tested BNNs , both versions of our temperature scaled attacks obtained near-perfect success rate outperforming gradient based attacks ( FGSM ( Goodfellow et al . ( 2014 ) ) , PGD ( Madry et al . ( 2017 ) ) ) . Furthermore , this temperature scaling improved gradient based attacks even on adversarially trained models ( both high-precision and quantized ) as well as floating point networks , showing the significance of signal propagation for adversarial attacks . 2 PRELIMINARIES . We first provide some background on the neural network quantization and adversarial attacks . 2.1 NEURAL NETWORK QUANTIZATION . Neural Network ( NN ) quantization is defined as training networks with parameters constrained to a minimal , discrete set of quantization levels . This primarily relies on the hypothesis that since NNs are usually overparametrized , it is possible to obtain a quantized network with performance comparable to the floating point network . Given a dataset D = { xi , yi } ni=1 , NN quantization can be written as : min w∈Qm L ( w ; D ) : = 1 n n∑ i=1 ` ( w ; ( xi , yi ) ) . ( 1 ) Here , ` ( · ) denotes the input-output mapping composed with a standard loss function ( e.g. , crossentropy loss ) , w is them dimensional parameter vector , andQ is a predefined discrete set representing quantization levels ( e.g. , Q = { −1 , 1 } in the binary case ) . Most of the NN quantization approaches ( Ajanthan et al . ( 2019a ; b ) ; Bai et al . ( 2019 ) ; Hubara et al . ( 2017 ) ) convert the above problem into an unconstrained problem by introducing auxiliary variables and optimize via ( stochastic ) gradient descent . To this end , the algorithms differ in the choice of quantization set ( e.g. , keep it discrete ( Courbariaux et al . ( 2015 ) ) , relax it to the convex hull ( Bai et al . ( 2019 ) ) or convert the problem into a lifted probability space ( Ajanthan et al . ( 2019a ) ) ) , the projection used , and how differentiation through projection is performed . In the case when the constraint set is relaxed , a gradually increasing annealing hyperparameter is used to enforce a quantized solution ( Ajanthan et al . ( 2019a ; b ) ; Bai et al . ( 2019 ) ) . We refer the interested reader to respective papers for more detail . In this paper , we use BNN-WQ obtained using MD-tanh-S ( Ajanthan et al . ( 2019b ) ) and BNN-WAQ obtained using Hubara et al . ( 2017 ) . 2.2 ADVERSARIAL ATTACKS . Adversarial examples consist of imperceptible perturbations to the data that alter the model ’ s prediction with high confidence . Existing attacks can be categorized into white-box and black-box attacks where the difference lies in the knowledge of the adversaries . White-box attacks allow the adversaries access to the target model ’ s architecture and parameters , whereas black-box attacks can only query the model . Since white-box gradient based attacks are popular , we summarize them below . First-order gradient based attacks can be compactly written as Projected Gradient Descent ( PGD ) on the negative of the loss function ( Madry et al . ( 2017 ) ) . Formally , let x0 ∈ IRN be the input image , then at iteration t , the PGD update can be written as : xt+1 = P ( xt + η gtx ) , ( 2 ) where P : IRN → X is a projection , X ⊂ IRN is the constraint set that bounds the perturbations , η > 0 is the step size , and gtx is a form of gradient of the loss with respect to the input x evaluated at xt . With this general form , the popular gradient based adversarial attacks can be specified : • Fast Gradient Sign Method ( FGSM ) : This is a one step attack introduced in Goodfellow et al . ( 2014 ) . Here , P is the identity mapping , η is the maximum allowed perturbation magnitude , and gtx = sign ( ∇x ` ( w∗ ; ( xt , y ) ) ) , where ` denotes the loss function , w∗ is the trained weights and y is the ground truth label corresponding to the image x0 . • PGD with L∞ bound : Arguably the most popular adversarial attack introduced in Madry et al . ( 2017 ) and sometimes referred to as Iterative Fast Gradient Sign Method ( IFGSM ) . Here , P is the L∞ norm based projection , η is a chosen step size , and gtx = sign ( ∇x ` ( w∗ ; ( xt , y ) ) ) , the sign of gradient same as FGSM . • PGD with L2 bound : This is also introduced in Madry et al . ( 2017 ) which performs the standard PGD in the Euclidean space . Here , P is the L2 norm based projection , η is a chosen step size , and gtx = ∇x ` ( w∗ ; ( xt , y ) ) is simply the gradient of the loss with respect to the input . These attacks have been further strengthened by a random initial step ( Tramèr et al . ( 2017 ) ) . In this paper , we perform this single random initialization for all experiments with FGSM/PGD attack unless otherwise mentioned . 3 ROBUSTNESS EVALUATION OF BINARY NEURAL NETWORKS . We start by evaluating the adversarial accuracy ( i.e . accuracy on the perturbed data ) of BNNs using the PGD attack with L∞ bound . • PGD attack details : perturbation bound of 8 pixels ( assuming each pixel in the image is in [ 0 , 255 ] ) with respect to L∞ norm , step size η = 2 and the total number of iterations T = 20 . The attack details are the same in all evaluated settings unless stated otherwise . We perform experiments on CIFAR-10 dataset using ResNet-18 and VGG-16 architectures and report the clean accuracy and PGD adversarial accuracy with 1 and 20 random restarts in Table 1 . It can be clearly and consistently observed that binary networks have high adversarial accuracy compared to the floating point counterparts . Even with 20 random restarts , BNNs clearly outperform floating point networks in terms of adversarial accuracy . Since this result is surprising , we investigate this phenomenon further to understand whether BNNs are actually robust to adversarial perturbations or they show a fake sense of security due to some form of obfuscated gradients ( Athalye et al . ( 2018 ) ) . 3.1 IDENTIFYING OBFUSCATED GRADIENTS . Recently , it has been shown that several defense mechanisms intentionally or unintentionally break gradient descent and cause obfuscated gradients and thus exhibit a false sense of security ( Athalye et al . ( 2018 ) ) . Several gradient based adversarial attacks tend to fail to produce adversarial perturbations in scenarios where the gradients are uninformative , referred to as gradient masking . Gradient masking can occur due to shattered gradients , stochastic gradients or exploding and vanishing gradients . We try to identify gradient masking in binary networks based on the empirical checks provided in Athalye et al . ( 2018 ) . If any of these checks fail , it indicates gradient masking issue in BNNs . To illustrate this , we analyse the effects of varying different hyperparameters of PGD attack on BNNs trained on CIFAR-10 using ResNet-18 architecture . Even though varying PGD perturbation bound does not show any signs of gradient masking , varying attack iterations and black-box vs white-box results ( on ResNet-18 and VGG-16 ) clearly indicate gradient masking issues as depicted in Fig . 1 . The black-box attack outperforming white-box attack for BNNs certainly indicates gradient masking issues since the black-box attack do not use the gradient information from model being attacked . Here , our black-box model to a BNN is the analogous floating point network trained on the same dataset and the attack is the same PGD with L∞ bound . These checks demonstrate that BNNs are prone to gradient masking and exhibit fake robustness . Note , shattered gradients occur due to non-differentiable components in the defense mechanism and stochastic gradients are caused by randomized gradients . Since BNNs are trainable from scratch and does not have randomized gradients1 , we narrow down gradient masking issue to vanishing or exploding gradients . Since , vanishing or exploding gradients occur due to poor signal propagation , by introducing a single scalar , we discuss two approaches to mitigate this issue , which lead to almost 100 % success rate for gradient based attacks on BNNs .

Summary: **Update**: Thanks to the authors for addressing my comments. As it was pointed out by the authors, temperature rescaling is mostly applicable to non-linear loss functions. For linear loss functions, temperature scaling only linear rescales the gradients. The difference between the proposed PGD++ attack and PGD with linear DLR loss is small (see the author's response to AR4). The improvements are most significant for FGSM but FGSM is not recommended for the robustness evaluation. Given the limited technical novelty and small improvements for linear loss functions, my score remains unchanged.