Fast and Reliable Evaluation of Adversarial Robustness with Minimum-Margin Attack

The AutoAttack ( AA ) has been the most reliable method to evaluate adversarial robustness when considerable computational resources are available . However , the high computational cost ( e.g. , 100 times more than that of the project gradient descent attack ) makes AA infeasible for practitioners with limited computational resources , and also hinders applications of AA in the adversarial training ( AT ) . In this paper , we propose a novel method , minimum-margin ( MM ) attack , to fast and reliably evaluate adversarial robustness . Compared with AA , our method achieves comparable performance but only costs 3 % of the computational time in extensive experiments . The reliability of our method lies in that we evaluate the quality of adversarial examples using the margin between two targets that can precisely identify the most adversarial example . The computational efficiency of our method lies in an effective Sequential TArget Ranking Selection ( STARS ) method , ensuring that the cost of the MM attack is independent of the number of classes . The MM attack opens a new way for evaluating adversarial robustness and contributes a feasible and reliable method to generate high-quality adversarial examples in AT . 1 INTRODUCTION . The deep neural network ( DNN ) has attracted a large number of researchers from different disciplines such as computer science ( Goodfellow et al. , 2016 ; Castelvecchi , 2016 ; Vaswani et al. , 2017 ) , physics ( DeVries et al. , 2018 ; Huang et al. , 2019 ; Levine et al. , 2019 ) , biology ( Maxmen , 2018a ; b ; Webb , 2018 ) and medicine ( Hao et al. , 2015 ; Esteva et al. , 2017 ) . The success of DNN mainly lies in its ability to learn useful high-level features from abundant data ( Deng & Yu , 2014 ; LeCun et al. , 2015 ) . These learned features have been successfully used to address many difficult tasks . For example , DNNs can recognize images with high accuracy comparable to human beings ( LeCun et al. , 1998 ; Krizhevsky et al. , 2012 ) . In addition , DNNs are also widely used for speech recognition ( Hinton et al. , 2012 ) , natural language processing ( Andor et al. , 2016 ) , and playing games ( Mnih et al. , 2013 ; Silver et al. , 2016 ) . As the impacts of DNN increase fast , its reliability has been a key to deploy it in real-world applications ( Huang et al. , 2011 ; Kurakin et al. , 2017 ) . Recently , a growing body of research shows that DNNs are vulnerable to adversarial examples , i.e. , test inputs that are modified slightly yet strategically to cause misclassification ( Szegedy et al. , 2014 ; Nguyen et al. , 2015 ; Kurakin et al. , 2017 ; Carlini & Wagner , 2017a ; Finlayson et al. , 2019 ; Wang et al. , 2019 ; Zhang et al. , 2020b ; Gao et al. , 2021 ) . The existence of such adversarial examples undoubtedly lowers the reliability of DNNs . Meanwhile , researchers have also been considering finding a reliable way to evaluate adversarial robustness of a DNN before deploying it in the real world . The high-level idea of evaluating adversarial robustness of a DNN is quite straightforward , i.e. , generating adversarial examples and calculating the accuracy of the DNN on these examples ( this kind of accuracy is also known as adversarial robust accuracy ) . Szegedy et al . ( 2014 ) first pointed out the existence of adversarial examples and used a less powerful box-constrained L-BFGS method to generate them . Based on the studies in ( Szegedy et al. , 2014 ) , Goodfellow et al . ( 2015 ) put forward the fast gradient sign method ( FGSM ) . One common loss function they used is cross-entropy ( CE ) loss , and to maximize the loss function , FGSM uses its gradient to determine in which direction the pixel ’ s intensity should be increased or decreased . Madry et al . ( 2018 ) introduced a simple refinement of the FGSM : projected gradient descent attack ( PGD ) , where instead of taking a single step of size in the direction of the gradient sign , multiple smaller steps are taken . Existing Evaluation Methods . PGD was an effective method to evaluate adversarial robustness of a standard-trained DNN ( Madry et al. , 2018 ) since adversarial robust accuracy of a standard-trained DNN is always very low after using PGD . Nevertheless , the existence of adversarial examples has already inspired research on training a robust DNN to defend against them , which means that a standard-trained DNN is not the only DNN we might meet and we need to evaluate adversarial robustness of a robust DNN as well . Unfortunately , PGD fails to reliably evaluate adversarial robustness of a robust DNN ( Carlini & Wagner , 2017b ; Croce & Hein , 2020 ) . Carlini & Wagner ( 2017b ) observed the phenomenon of gradient vanishing in the widely used CE loss for the potential failure of L-BFGS , FGSM and PGD , and replaced the CE loss with many possible choices . Croce & Hein ( 2020 ) claimed that the fixed step size and the single attack used are the causes of poor evaluations , and they put forward an ensemble of diverse attacks ( consisting of APGD-CE , APGD-DLR , FAB and Square ) called AutoAttack ( AA ) to test adversarial robustness . Until now , AA has been the most reliable method to evaluate adversarial robustness ( Croce & Hein , 2020 ) . However , though AA performs well in reliability , it needs a large amount of computational time . As shown in Figure 1 ( a ) , for attacking a ResNet-18 model on CIFAR-10 ( following the adversarial training in ( Madry et al. , 2018 ) ) , the computational cost of AA ( or T-AA ) is 65 times ( or 100 times ) more than PGD used in ( Madry et al. , 2018 ) , where T-AA is more time-consuming since it considers each target for APGD-DLR and FAB in AA . Worse still , in the worst case as analyzed in Appendix A , the computational cost of AA ( or T-AA ) is even 109 times ( or 440 times ) more than PGD . A Dilemma Between Reliability and Computational Efficiency . The high computational cost makes AA infeasible when considerable computational resources are unavailable . Unfortunately , such scenarios are common in the real world , e.g. , for practitioners who need real-time evaluation at each epoch of the training process of a robust model , such high computational cost is unacceptable . Similarly , since a large number of adversarial examples need to be generated at each epoch during adversarial training ( AT ) , such high computational cost hinders applications of AA in AT . In consideration of the high reliability but low computational efficiency of AA , and the high computational efficiency but low reliability of PGD , we seem to encounter a dilemma : we have to consider giving up one factor ( reliability or computational efficiency ) when evaluating the adversarial robustness . Our Reliable and Fast Solution . In this paper , we are dedicated to achieving reliability and computational efficiency simultaneously . For reliability , we evaluate the quality of adversarial examples using the margin between two targets for precisely identifying the most adversarial example . For computational efficiency , we put forward an effective Sequential TArget Ranking Selection ( STARS ) method to ensure that the cost of the MM attack is independent of the number of classes . Reliability . To achieve reliability , we investigate the reasons behind the failure of PGD . We identify that CE loss , which is based on the probability of the true label py , is not an appropriate measure to the quality of adversarial examples . In Figure 2 , we provide a simple demonstration to this issue , in which we consider one targeted false label t. As we can see , it is much more reasonable to measure the quality of adversarial examples in terms of the margin of probability py−pt . The most adversarial example in Figure 2 then corresponds to the one with the minimum margin of probability instead of the minimum probability py . Detailed study of the rationality of minimum-magrin is provided in Section 3.1 . Since the search space B of high dimensional images is large ( grey area ) , previous studies use gradient descent methods to generate the adversarial example that maximizes the loss function ( Goodfellow et al. , 2015 ; Carlini & Wagner , 2017b ; Madry et al. , 2018 ) . Though it looks promising to generate adversarial examples via minimizing the margin of probability , we find that there are still two issues : ( a ) The probability p is a kind of rescaling method to the logits z. Croce & Hein ( 2020 ) heuristically rescaled the logits z using their proposed DLR ( defined at Eq. ( 6 ) ) . We investigate the performance of different rescaling methods in Section 3.1 . We numerically find that the method using natural logits zy − zt ( the meaning of margin ) with no rescaling performs the best ; ( b ) For the problem of multi-class and untargeted attacks , the margin is then zy −maxi 6=y zi . However , − ( zy − maxi 6=y zi ) is not an appropriate loss function , because the max function only considers one target at the current step while those unconsidered targets may lead to more adversarial examples . Hence , the reliable method is to minimize zy − zt for each t 6= y and take the most adversarial one , which is a widely used solution ( Croce & Hein , 2020 ) . Computational Efficiency . Although running the attack for each false target is reliable , the computational cost depends on the number of classes . For datasets with a large number of classes , e.g. , CIFAR-100 ( 100 classes ) and Imagenet ( 1,000 classes ) , the computational cost will increase accordingly . To achieve computational efficiency , we propose a Sequential TArget Ranking Selection ( STARS ) method in Section 3.2 to make the computational time independent of the number of classes . According to the ranking of predicted probability , STARS method only selects a few highest targets and runs a sequential attack . Experiments show that , benefited from STARS method , MM attack can save 76.36 % of the computational time on CIFAR-10 , 98.51 % on CIFAR-100 and 77.78 % on SVHN . By taking all the above factors into consideration , we propose a novel method , the minimummargin ( MM ) attack . Its detailed realization is provided in Section 3 . We present extensive experimental results in Section 4 , which verify that our MM attack can fast and reliably evaluate adversarial robustness . In particular , MM attack achieves comparable performance but only costs 3 % of the computational time compared with the current benchmark AA . Hence , our proposed MM attack provides a new direction of evaluating adversarial robustness and contributes a feasible and reliable method to generate high-quality adversarial examples in AT . 2 PRELIMINARY . Neural Networks . A neural network is a function fθ : Rn → [ 0 , 1 ] K , where θ is the parameters contained in fθ and K is normally the number of classes . The output of the network is computed using the softmax function , which ensures that the output is a valid probability vector . Namely , given an input x ∈ Rn , fθ ( x ) = [ p1 , . . . , pK ] = p where ∑K i=1 pi = 1 and pi is the probability that input x has on class i . Before the softmax function , the output of the network is called logits z , i.e. , p = softmax ( z ) . The classifier assigns the label y = arg maxi f ( x ) i . Projected Gradient Descent Attack ( PGD ) . Madry et al . ( 2018 ) proposed the projected gradient descent ( PGD ) attack to generate adversarial examples to mislead a well-trained classifier fθ . Specifically , they start with setting x0 = x , and then in each iteration : x′ ( t+1 ) = ΠB [ x ( 0 ) ] ( x ′ ( t ) + α sign ( ∇x′ ( t ) ` ( fθ ( x ′ ( t ) ) , y ) ) , t = 0 , 1 , 2 , . . . , ( 1 ) where B [ x ] = { x′ | d∞ ( x , x′ ) ≤ } , ( 2 ) is the closed ball of radius > 0 centered at x ; the x ( 0 ) refers to the starting point which corresponds to the natural example ( or the natural example perturbed by a small Gaussian or uniformly random noise ) ; x ( t ) is the adversarial example at step t ; ΠB [ x ( 0 ) ] ( · ) is the projection function that projects the adversarial variant back to the -ball centered at x ( 0 ) if necessary ; the ` ∞ distance metric is d∞ ( x , x ′ ) = ‖x− x′‖∞ ; and ` is cross entropy ( CE ) loss : CE ( f , x , y ) = − log ( py ) = −zy + log K∑ j=1 ezj , ( 3 ) where pi = ezi/ ∑K j=1 e zj , i = 1 , ... , K , and z is the logits of the model outputs . Carlini and Wagner attack ( CW ) . Carlini & Wagner ( 2017b ) observed the phenomenon of gradient vanishing in the widely used CE loss for potential failure . The gradient w.r.t x in Eq . ( 3 ) is given by ∇xCE ( f , x , y ) = ( −1 + py ) ∇xzy + ∑ i 6=j pi∇xzi . ( 4 ) If py ≈ 1 and consequently pi ≈ 0 for i 6= y , then ∇xCE ( x , y ) ≈ 0 in Eq . ( 4 ) . This gradient vanishing issue can lead to the failure of attacks . Motivated by this phenomenon , Carlini & Wagner ( 2017b ) replaced the CE loss with several possible choices . Among these choices , the widely used one for the untargeted attack is CW ( f , x , y ) = −zy ( x′ ) + max i 6=y zi ( x ′ ) . ( 5 ) AutoAttack ( AA ) . Croce & Hein ( 2020 ) claimed that the fixed step size and the lack of diversity in attack methods are the main reasons for the limitations of previous studies . Motivated by the line search method ( Grippo et al. , 1986 ) , they put forward adaptive projected gradient descent attack ( APGD ) . They showed that using adaptive step size significantly improves the adversarial robustness compared with using fixed step size . For the loss function at Eq . ( 5 ) , they claim that scale invariance w.r.t . z is necessary , and they proposed an alternative loss function : DLR ( f , x , y ) = −zy ( x ′ ) −maxi 6=y zi ( x′ ) zπ1 ( x ′ ) − zπ3 ( x′ ) , ( 6 ) where π is the permutation of the components of z in decreasing order . For the targeted attack , they propose another alternative loss function : Targeted-DLR ( f , x , y , t ) = − zy ( x ′ ) − zt ( x′ ) zπ1 ( x ′ ) − 12 · ( zπ3 ( x′ ) + zπ4 ( x′ ) ) . ( 7 ) For the lack of diversity , they claimed that diverse attacks are beneficial for reliability , and then they put forward an ensemble of various parameter-free attacks called AutoAttack ( AA ) to test adversarial robustness , where AA contains APGD-CE , APGD-DLR , FAB and Square . Targeted AutoAttack ( T-AA ) replaces APGD-DLR with the targeted APGD-DLR , and replaces FAB with targeted FAB .

Summary: The paper proposed a strong adversarial attack, i.e., an attack that can generate strong adversarial examples and thus can better evaluate the adversarial robustness of given deep learning models. Compared with the SOTA attack, the proposed attack is much faster and thus easier to be applied in practice. The idea is novel and the results are solid.