Fast and Reliable Evaluation of Adversarial Robustness with Minimum-Margin Attack

The AutoAttack ( AA ) has been the most reliable method to evaluate adversarial robustness when considerable computational resources are available . However , the high computational cost ( e.g. , 100 times more than that of the project gradient descent attack ) makes AA infeasible for practitioners with limited computational resources , and also hinders applications of AA in the adversarial training ( AT ) . In this paper , we propose a novel method , minimum-margin ( MM ) attack , to fast and reliably evaluate adversarial robustness . Compared with AA , our method achieves comparable performance but only costs 3 % of the computational time in extensive experiments . The reliability of our method lies in that we evaluate the quality of adversarial examples using the margin between two targets that can precisely identify the most adversarial example . The computational efficiency of our method lies in an effective Sequential TArget Ranking Selection ( STARS ) method , ensuring that the cost of the MM attack is independent of the number of classes . The MM attack opens a new way for evaluating adversarial robustness and contributes a feasible and reliable method to generate high-quality adversarial examples in AT . 1 INTRODUCTION . The deep neural network ( DNN ) has attracted a large number of researchers from different disciplines such as computer science ( Goodfellow et al. , 2016 ; Castelvecchi , 2016 ; Vaswani et al. , 2017 ) , physics ( DeVries et al. , 2018 ; Huang et al. , 2019 ; Levine et al. , 2019 ) , biology ( Maxmen , 2018a ; b ; Webb , 2018 ) and medicine ( Hao et al. , 2015 ; Esteva et al. , 2017 ) . The success of DNN mainly lies in its ability to learn useful high-level features from abundant data ( Deng & Yu , 2014 ; LeCun et al. , 2015 ) . These learned features have been successfully used to address many difficult tasks . For example , DNNs can recognize images with high accuracy comparable to human beings ( LeCun et al. , 1998 ; Krizhevsky et al. , 2012 ) . In addition , DNNs are also widely used for speech recognition ( Hinton et al. , 2012 ) , natural language processing ( Andor et al. , 2016 ) , and playing games ( Mnih et al. , 2013 ; Silver et al. , 2016 ) . As the impacts of DNN increase fast , its reliability has been a key to deploy it in real-world applications ( Huang et al. , 2011 ; Kurakin et al. , 2017 ) . Recently , a growing body of research shows that DNNs are vulnerable to adversarial examples , i.e. , test inputs that are modified slightly yet strategically to cause misclassification ( Szegedy et al. , 2014 ; Nguyen et al. , 2015 ; Kurakin et al. , 2017 ; Carlini & Wagner , 2017a ; Finlayson et al. , 2019 ; Wang et al. , 2019 ; Zhang et al. , 2020b ; Gao et al. , 2021 ) . The existence of such adversarial examples undoubtedly lowers the reliability of DNNs . Meanwhile , researchers have also been considering finding a reliable way to evaluate adversarial robustness of a DNN before deploying it in the real world . The high-level idea of evaluating adversarial robustness of a DNN is quite straightforward , i.e. , generating adversarial examples and calculating the accuracy of the DNN on these examples ( this kind of accuracy is also known as adversarial robust accuracy ) . Szegedy et al . ( 2014 ) first pointed out the existence of adversarial examples and used a less powerful box-constrained L-BFGS method to generate them . Based on the studies in ( Szegedy et al. , 2014 ) , Goodfellow et al . ( 2015 ) put forward the fast gradient sign method ( FGSM ) . One common loss function they used is cross-entropy ( CE ) loss , and to maximize the loss function , FGSM uses its gradient to determine in which direction the pixel ’ s intensity should be increased or decreased . Madry et al . ( 2018 ) introduced a simple refinement of the FGSM : projected gradient descent attack ( PGD ) , where instead of taking a single step of size in the direction of the gradient sign , multiple smaller steps are taken . Existing Evaluation Methods . PGD was an effective method to evaluate adversarial robustness of a standard-trained DNN ( Madry et al. , 2018 ) since adversarial robust accuracy of a standard-trained DNN is always very low after using PGD . Nevertheless , the existence of adversarial examples has already inspired research on training a robust DNN to defend against them , which means that a standard-trained DNN is not the only DNN we might meet and we need to evaluate adversarial robustness of a robust DNN as well . Unfortunately , PGD fails to reliably evaluate adversarial robustness of a robust DNN ( Carlini & Wagner , 2017b ; Croce & Hein , 2020 ) . Carlini & Wagner ( 2017b ) observed the phenomenon of gradient vanishing in the widely used CE loss for the potential failure of L-BFGS , FGSM and PGD , and replaced the CE loss with many possible choices . Croce & Hein ( 2020 ) claimed that the fixed step size and the single attack used are the causes of poor evaluations , and they put forward an ensemble of diverse attacks ( consisting of APGD-CE , APGD-DLR , FAB and Square ) called AutoAttack ( AA ) to test adversarial robustness . Until now , AA has been the most reliable method to evaluate adversarial robustness ( Croce & Hein , 2020 ) . However , though AA performs well in reliability , it needs a large amount of computational time . As shown in Figure 1 ( a ) , for attacking a ResNet-18 model on CIFAR-10 ( following the adversarial training in ( Madry et al. , 2018 ) ) , the computational cost of AA ( or T-AA ) is 65 times ( or 100 times ) more than PGD used in ( Madry et al. , 2018 ) , where T-AA is more time-consuming since it considers each target for APGD-DLR and FAB in AA . Worse still , in the worst case as analyzed in Appendix A , the computational cost of AA ( or T-AA ) is even 109 times ( or 440 times ) more than PGD . A Dilemma Between Reliability and Computational Efficiency . The high computational cost makes AA infeasible when considerable computational resources are unavailable . Unfortunately , such scenarios are common in the real world , e.g. , for practitioners who need real-time evaluation at each epoch of the training process of a robust model , such high computational cost is unacceptable . Similarly , since a large number of adversarial examples need to be generated at each epoch during adversarial training ( AT ) , such high computational cost hinders applications of AA in AT . In consideration of the high reliability but low computational efficiency of AA , and the high computational efficiency but low reliability of PGD , we seem to encounter a dilemma : we have to consider giving up one factor ( reliability or computational efficiency ) when evaluating the adversarial robustness . Our Reliable and Fast Solution . In this paper , we are dedicated to achieving reliability and computational efficiency simultaneously . For reliability , we evaluate the quality of adversarial examples using the margin between two targets for precisely identifying the most adversarial example . For computational efficiency , we put forward an effective Sequential TArget Ranking Selection ( STARS ) method to ensure that the cost of the MM attack is independent of the number of classes . Reliability . To achieve reliability , we investigate the reasons behind the failure of PGD . We identify that CE loss , which is based on the probability of the true label py , is not an appropriate measure to the quality of adversarial examples . In Figure 2 , we provide a simple demonstration to this issue , in which we consider one targeted false label t. As we can see , it is much more reasonable to measure the quality of adversarial examples in terms of the margin of probability py−pt . The most adversarial example in Figure 2 then corresponds to the one with the minimum margin of probability instead of the minimum probability py . Detailed study of the rationality of minimum-magrin is provided in Section 3.1 . Since the search space B of high dimensional images is large ( grey area ) , previous studies use gradient descent methods to generate the adversarial example that maximizes the loss function ( Goodfellow et al. , 2015 ; Carlini & Wagner , 2017b ; Madry et al. , 2018 ) . Though it looks promising to generate adversarial examples via minimizing the margin of probability , we find that there are still two issues : ( a ) The probability p is a kind of rescaling method to the logits z. Croce & Hein ( 2020 ) heuristically rescaled the logits z using their proposed DLR ( defined at Eq. ( 6 ) ) . We investigate the performance of different rescaling methods in Section 3.1 . We numerically find that the method using natural logits zy − zt ( the meaning of margin ) with no rescaling performs the best ; ( b ) For the problem of multi-class and untargeted attacks , the margin is then zy −maxi 6=y zi . However , − ( zy − maxi 6=y zi ) is not an appropriate loss function , because the max function only considers one target at the current step while those unconsidered targets may lead to more adversarial examples . Hence , the reliable method is to minimize zy − zt for each t 6= y and take the most adversarial one , which is a widely used solution ( Croce & Hein , 2020 ) . Computational Efficiency . Although running the attack for each false target is reliable , the computational cost depends on the number of classes . For datasets with a large number of classes , e.g. , CIFAR-100 ( 100 classes ) and Imagenet ( 1,000 classes ) , the computational cost will increase accordingly . To achieve computational efficiency , we propose a Sequential TArget Ranking Selection ( STARS ) method in Section 3.2 to make the computational time independent of the number of classes . According to the ranking of predicted probability , STARS method only selects a few highest targets and runs a sequential attack . Experiments show that , benefited from STARS method , MM attack can save 76.36 % of the computational time on CIFAR-10 , 98.51 % on CIFAR-100 and 77.78 % on SVHN . By taking all the above factors into consideration , we propose a novel method , the minimummargin ( MM ) attack . Its detailed realization is provided in Section 3 . We present extensive experimental results in Section 4 , which verify that our MM attack can fast and reliably evaluate adversarial robustness . In particular , MM attack achieves comparable performance but only costs 3 % of the computational time compared with the current benchmark AA . Hence , our proposed MM attack provides a new direction of evaluating adversarial robustness and contributes a feasible and reliable method to generate high-quality adversarial examples in AT . 2 PRELIMINARY . Neural Networks . A neural network is a function fθ : Rn → [ 0 , 1 ] K , where θ is the parameters contained in fθ and K is normally the number of classes . The output of the network is computed using the softmax function , which ensures that the output is a valid probability vector . Namely , given an input x ∈ Rn , fθ ( x ) = [ p1 , . . . , pK ] = p where ∑K i=1 pi = 1 and pi is the probability that input x has on class i . Before the softmax function , the output of the network is called logits z , i.e. , p = softmax ( z ) . The classifier assigns the label y = arg maxi f ( x ) i . Projected Gradient Descent Attack ( PGD ) . Madry et al . ( 2018 ) proposed the projected gradient descent ( PGD ) attack to generate adversarial examples to mislead a well-trained classifier fθ . Specifically , they start with setting x0 = x , and then in each iteration : x′ ( t+1 ) = ΠB [ x ( 0 ) ] ( x ′ ( t ) + α sign ( ∇x′ ( t ) ` ( fθ ( x ′ ( t ) ) , y ) ) , t = 0 , 1 , 2 , . . . , ( 1 ) where B [ x ] = { x′ | d∞ ( x , x′ ) ≤ } , ( 2 ) is the closed ball of radius > 0 centered at x ; the x ( 0 ) refers to the starting point which corresponds to the natural example ( or the natural example perturbed by a small Gaussian or uniformly random noise ) ; x ( t ) is the adversarial example at step t ; ΠB [ x ( 0 ) ] ( · ) is the projection function that projects the adversarial variant back to the -ball centered at x ( 0 ) if necessary ; the ` ∞ distance metric is d∞ ( x , x ′ ) = ‖x− x′‖∞ ; and ` is cross entropy ( CE ) loss : CE ( f , x , y ) = − log ( py ) = −zy + log K∑ j=1 ezj , ( 3 ) where pi = ezi/ ∑K j=1 e zj , i = 1 , ... , K , and z is the logits of the model outputs . Carlini and Wagner attack ( CW ) . Carlini & Wagner ( 2017b ) observed the phenomenon of gradient vanishing in the widely used CE loss for potential failure . The gradient w.r.t x in Eq . ( 3 ) is given by ∇xCE ( f , x , y ) = ( −1 + py ) ∇xzy + ∑ i 6=j pi∇xzi . ( 4 ) If py ≈ 1 and consequently pi ≈ 0 for i 6= y , then ∇xCE ( x , y ) ≈ 0 in Eq . ( 4 ) . This gradient vanishing issue can lead to the failure of attacks . Motivated by this phenomenon , Carlini & Wagner ( 2017b ) replaced the CE loss with several possible choices . Among these choices , the widely used one for the untargeted attack is CW ( f , x , y ) = −zy ( x′ ) + max i 6=y zi ( x ′ ) . ( 5 ) AutoAttack ( AA ) . Croce & Hein ( 2020 ) claimed that the fixed step size and the lack of diversity in attack methods are the main reasons for the limitations of previous studies . Motivated by the line search method ( Grippo et al. , 1986 ) , they put forward adaptive projected gradient descent attack ( APGD ) . They showed that using adaptive step size significantly improves the adversarial robustness compared with using fixed step size . For the loss function at Eq . ( 5 ) , they claim that scale invariance w.r.t . z is necessary , and they proposed an alternative loss function : DLR ( f , x , y ) = −zy ( x ′ ) −maxi 6=y zi ( x′ ) zπ1 ( x ′ ) − zπ3 ( x′ ) , ( 6 ) where π is the permutation of the components of z in decreasing order . For the targeted attack , they propose another alternative loss function : Targeted-DLR ( f , x , y , t ) = − zy ( x ′ ) − zt ( x′ ) zπ1 ( x ′ ) − 12 · ( zπ3 ( x′ ) + zπ4 ( x′ ) ) . ( 7 ) For the lack of diversity , they claimed that diverse attacks are beneficial for reliability , and then they put forward an ensemble of various parameter-free attacks called AutoAttack ( AA ) to test adversarial robustness , where AA contains APGD-CE , APGD-DLR , FAB and Square . Targeted AutoAttack ( T-AA ) replaces APGD-DLR with the targeted APGD-DLR , and replaces FAB with targeted FAB .

Summary: This paper proposes a minimum-margin (MM) attack to evaluate defenses. The authors report detailed results on the effects of different loss functions. Experiments are done on CIFAR-10/100 and SVHN, against the adversarially trained models.