Local Reweighting for Adversarial Training

1 INTRODUCTION . A growing body of research shows that neural networks are vulnerable to adversarial examples , i.e. , test inputs that are modified slightly yet strategically to cause misclassification ( Carlini & Wagner , 2017a ; Finlayson et al. , 2019 ; Kurakin et al. , 2017 ; Nguyen et al. , 2015 ; Szegedy et al. , 2014 ; Wang et al. , 2019 ; Zhang et al. , 2020b ) . It is crucial to train a robust neural network to defend against such examples for security-critical computer vision systems , such as autonomous driving and medical diagnostics ( Chen et al. , 2015 ; Ma et al. , 2021 ; Miyato et al. , 2017 ; Nguyen et al. , 2015 ; Szegedy et al. , 2014 ) . To mitigate this issue , adversarial training methods have been proposed in recent years ( Balunovic & Vechev , 2019 ; Goodfellow et al. , 2015 ; Madry et al. , 2018 ; Shafahi et al. , 2020 ; Wang et al. , 2020a ; Wu et al. , 2020 ) . By injecting adversarial examples into training data , adversarial training methods seek to train an adversarial-robust deep network whose predictions are locally invariant in a small neighborhood of its inputs ( Bai et al. , 2019 ; He et al. , 2018 ; Papernot et al. , 2016 ; Raghunathan et al. , 2020 ; Tsipras et al. , 2019 ; Yang et al. , 2020 ) . Due to the diversity and complexity of adversarial data , over-parameterized deep networks have insufficient model capacity in adversarial training ( AT ) ( Zhang et al. , 2021 ) . To obtain a robust model given fixed model capacity , Zhang et al . ( 2021 ) suggest that we do not need to pay much attention to an instance that is already safe under the attack , and propose instance-reweighted adversarial training ( IRAT ) , which performs global reweighting with a given attack . To identify safe/non-robustness instances , they propose a geometric distance ( GD ) between natural data points and current class boundary . Instances being closer to/farther from the class boundary is more/less vulnerable to the given attack , and should be assigned larger/smaller weights during AT . This geometry-aware IRAT ( GAIRAT ) significantly boosts the robustness of the trained models when facing the given attack . However , when tested on attacks that are different from the given attack simulated in IRAT , the robustness of IRAT drops significantly ( e.g. , even worse than no reweighting ) . First , we find that a large number of instances are actually overlooked during IRAT . Figure 1 ( a ) shows that , for approximately four-fifths of the instances , their corresponding adversarial variants are assigned very low weights ( less than 0.2 ) . Second , we find that the robustness of the IRAT-trained classifier drops significantly when facing an unseen attack . Figures 1 ( b ) and 1 ( c ) show that the classifier trained by GAIRAT ( with projected gradient descent ( PGD ) attack ( Madry et al. , 2018 ) ) has lower robustness when attacked by the unseen Carlini and Wagner attack ( CW ) ( Carlini & Wagner , 2017b ) compared with the robustness of standard adversarial training ( SAT ) ( Madry et al. , 2018 ) . Since the robustness of a model is not determined by any single attack , the improved performance for a kind of given attack is not sufficient to justify robustness . As shown in Figure 2 , the ranking of attack intensity varies with different datasets , adversarial training methods and so on , which means there is not the strongest attack in all situations . For another , previous studies of the adversarial detection ( Lee et al. , 2018 ; Ma et al. , 2018 ; Gao et al. , 2021 ) show that the stronger adversarial examples for robust classifiers to predict , the easier for the adversarial detectors to detect . This means improving performance not only on stronger attacks , but also on weaker ones , as some of attackers may choose to use weaker attacks to escape from the adversarial detection . Hence , robustness needs the network to consider a variety of potential attacks . In this paper , we investigate the reasons behind the failure of IRAT on unseen attacks . Unlike the common scenario of the classification problem where the training and testing data are fixed , there are different adversarial variants for the same instance in AT , e.g. , PGD-based or CW-based adversarial variants . A natural question comes with this—whether there are inconsistent vulnerabilities in the view of different attacks ? The answer is affirmative . Figure 3 visualizes this phenomenon using t-SNE ( Van der Maaten & Hinton , 2008 ) . The four subfigures in Figure 3 visualize inconsistent vulnerable instances in different views using t-SNE . The red dots in all subfigures represent consistently vulnerable instances between different views , while the blue dots represent the inconsistent vulnerable instances . From the SAT-trained classifier ( Figure 3 ( a ) -3 ( d ) ) , we can clearly see that a large number of vulnerable instances are inconsistent ( the blue dots dominate in all the subfigures ) . Given the above investigation , we argue that the safeness of instances is attack-dependent , that is , for the same instance , its weight can change given different attacks based on the same model . Thus , if the attack simulated in training is mis-specified , the weights of IRAT are misleading . In order to ameliorate this pessimism of IRAT , we propose our solution—locally reweighted adversarial training ( LRAT ) . As shown in Figure 4 , LRAT pairs each instance with its adversarial variants and performs local reweighting inside each pair , while performing no global reweighting . The rationale of LRAT is to fit the instance itself if it is immune to the attack , and in order to passively defend different attacks in future , LRAT does not skip the pair . In terms of the common attribute of adversarial attacks that misleading the classifier with a low probability of predicting the correct label and a high probability of predicting the false label ( Madry et al. , 2018 ; Carlini & Wagner , 2017b ) , we propose a general reweighting strategy that is applicable to various representatively attacks . Our experimental results show that compared with AT , LRAT can achieve better results on various simulated attacks . Other than previous global reweighting training that GAIRAT , LRAT can boost the performance of AT on PGD with no decline on other attacks . 2 ADVERSARIAL TRAINING . In this section , we briefly review existing adversarial training methods ( Madry et al. , 2018 ; Zhang et al. , 2021 ) . Let ( X , d∞ ) be the input feature space X with a metric d∞ ( x , x′ ) = ‖x− x′‖∞ , and B [ x ] = { x′ | d∞ ( x , x′ ) ≤ } be the closed ball of radius > 0 centered at x in X . The dataset S = { xi , yi } ni=1 , where xi ∈ X , yi ∈ Y = { 0 , 1 , . . . , K−1 } . We use fθ ( x ) to denote a deep neural network parameterized by θ . Specifically , fθ ( x ) predicts the label of an input instance x via : fθ ( x ) = arg max k∈K pk ( x ; θ ) , ( 1 ) where pk ( x ; θ ) denotes the predicted probability ( softmax on logits ) of x belonging to class k . 2.1 STANDARD ADVERSARIAL TRAINING . The objective function of SAT ( Madry et al. , 2018 ) is min fθ∈F 1 n n∑ i=1 ` ( fθ ( x̃i ) , yi ) , ( 2 ) where x̃i = arg maxx̃i∈B [ x ] ` ( fθ ( x̃i ) , yi ) and F = { f : X → Y } . The selected x̃ is the most adversarial variant within the -ball center at x . The loss function ` : RK × Y → R is a composition of a base loss ` B : 4K−1 × Y → R ( e.g. , the cross-entropy loss ) and an inverse link function ` L : RK → 4K−1 ( e.g. , the soft-max activation ) , where 4K−1 is the corresponding probalility simplex . Namely , ` ( fθ ( · ) , y ) = ` B ( ` L ( fθ ( · ) ) , y ) . PGD ( Madry et al. , 2018 ) is the most common approximation method for searching the most adversarial variant . Starting from x ( 0 ) ∈ X , PGD ( with step size α > 0 ) works as follows : x ( t+1 ) = ΠB [ x ( 0 ) ] ( x ( t ) + α sign ( ∇xt ` ( fθ ( x ( t ) , y ) ) ) , t ∈ N , ( 3 ) where N is the number of iterations ; x ( 0 ) refers to the starting point that natural instance ( or a natural instance perturbed by a small Gaussian or uniformly random noise ) ; y is the corresponding label for x ( 0 ) ; x ( t ) is the adversarial variant at step t ; ΠB [ x ( 0 ) ] ( · ) is the projection function that projects the adversarial variant back into the -ball centered at x ( 0 ) if necessary . 2.2 GEOMETRY-AWARE INSTANCE-REWEIGHTED ADVERSARIAL TRAINING . GAIRAT is a typical IRAT proposed by Zhang et al . ( 2021 ) . GAIRAT argues that natural training data farther from/close to the decision boundary are safe/non-robustness , and should be assigned with smaller/larger weights . Let ω ( x , y ) be the geometry-aware weight assignment function on the loss of the adversarial variant x̃ , where the generation of x̃ follows SAT . GAIRAT aims to min fθ∈F 1 n n∑ i=1 ω ( xi , yi ) ` ( fθ ( x̃i ) , yi ) . ( 4 ) Eq . ( 4 ) rescales the loss using a function ω ( x , y ) . This function is non-increasing w.r.t . GD , which is defined as the least steps that the PGD method needs to successfully attack the natural instances . The method then normalizes ω to ensure that ω ( x , y ) ≤ 0 and 1n ∑n i=1 ω ( xi , yi ) = 1 . Finally GAIRAT employs a bootstrap period in the initial part of the training by setting ω ( xi , yi ) = 1 , thereby performing regular training and ignoring the geometric-distance of input ( xi , yi ) . 3 THE LIMITATION OF IRAT : INCONSISTENT VULNERABILITY IN DIFFERENT VIEWS . The difference between Eq . ( 2 ) and Eq . ( 4 ) is the addition of the geometry-aware weight ω ( x , y ) . According to GAIRAT ( Zhang et al. , 2021 ) , more vulnerable instances should be assigned larger weights . However , the relative vulnerability between instances may vary in different situations , such as for different adversarial variants . V represents the selected variable to measure the vulnerability between the classifier and adversarial variants , and the smaller V , the more vulnerable is the instance . For the PGD-based adversarial variant , the lower predicted probability it has on true class , the smaller V . Here , we give the definition of VPGD . Definition 1 ( Vulnerability in the view of PGD ) . In the view of PGD , the vulnerability VPGD regarding x̃ ( generated by PGD ) is defined as VPGD ( x̃ , y ) : = py ( x̃ ) , ( 5 ) where py denotes the predicted probability ( softmax on logits ) of x̃ belonging to the true class y . For the CW-based adversarial variant , the relatively higher predicted probability it has on the false class , the larger VCW . Here , we give the definition of VCW . Definition 2 ( Vulnerability in the view of CW ) . In the view of CW , the vulnerability VCW regarding x̃ ( generated by CW ) is defined as VCW ( x̃ , y ) : = zy ( x̃ ) −max i6=y zi ( x̃ ) , ( 6 ) where zy denotes the predicted logits of x̃ ( CW ) belonging to the true class y . The maxi6=y zi ( x̃ ) denotes the maximum predicted logits of x̃ ( CW ) belonging to the false class i ( i 6= y ) . As shown in Figure 5 , the dark yellow and dark blue are top-20 % vulnerable instances in the view of PGD and CW , respectively . The frequency distribution of dark yellow in Figure 5 ( b ) and dark blue in Figure 5 ( c ) is clearly different , and the frequency distribution of dark blue in Figure 5 ( e ) and dark yellow in Figure 5 ( f ) is also different . This phenomenon means that the most vulnerable 10 , 000 PGD adversarial variants and the most vulnerable 10 , 000 CW adversarial variants are not from the same 10 , 000 instances . As a consequence of this inconsistency , if the attack simulated in training is mis-specified , the weights of IRAT will be misleading .

Summary: The paper studies the reweighting of adversarial training. It first points out the problem of GAIRAT: GAIRAT only improves the robust accuracy under PGD and lowers the accuracy of other attacks like AutoAttack. Based on the findings, the authors propose a new method: LRAT, which performs adversarial training using different attacks and gives different weights to the instances generated by different attacks.