Local Reweighting for Adversarial Training

1 INTRODUCTION . A growing body of research shows that neural networks are vulnerable to adversarial examples , i.e. , test inputs that are modified slightly yet strategically to cause misclassification ( Carlini & Wagner , 2017a ; Finlayson et al. , 2019 ; Kurakin et al. , 2017 ; Nguyen et al. , 2015 ; Szegedy et al. , 2014 ; Wang et al. , 2019 ; Zhang et al. , 2020b ) . It is crucial to train a robust neural network to defend against such examples for security-critical computer vision systems , such as autonomous driving and medical diagnostics ( Chen et al. , 2015 ; Ma et al. , 2021 ; Miyato et al. , 2017 ; Nguyen et al. , 2015 ; Szegedy et al. , 2014 ) . To mitigate this issue , adversarial training methods have been proposed in recent years ( Balunovic & Vechev , 2019 ; Goodfellow et al. , 2015 ; Madry et al. , 2018 ; Shafahi et al. , 2020 ; Wang et al. , 2020a ; Wu et al. , 2020 ) . By injecting adversarial examples into training data , adversarial training methods seek to train an adversarial-robust deep network whose predictions are locally invariant in a small neighborhood of its inputs ( Bai et al. , 2019 ; He et al. , 2018 ; Papernot et al. , 2016 ; Raghunathan et al. , 2020 ; Tsipras et al. , 2019 ; Yang et al. , 2020 ) . Due to the diversity and complexity of adversarial data , over-parameterized deep networks have insufficient model capacity in adversarial training ( AT ) ( Zhang et al. , 2021 ) . To obtain a robust model given fixed model capacity , Zhang et al . ( 2021 ) suggest that we do not need to pay much attention to an instance that is already safe under the attack , and propose instance-reweighted adversarial training ( IRAT ) , which performs global reweighting with a given attack . To identify safe/non-robustness instances , they propose a geometric distance ( GD ) between natural data points and current class boundary . Instances being closer to/farther from the class boundary is more/less vulnerable to the given attack , and should be assigned larger/smaller weights during AT . This geometry-aware IRAT ( GAIRAT ) significantly boosts the robustness of the trained models when facing the given attack . However , when tested on attacks that are different from the given attack simulated in IRAT , the robustness of IRAT drops significantly ( e.g. , even worse than no reweighting ) . First , we find that a large number of instances are actually overlooked during IRAT . Figure 1 ( a ) shows that , for approximately four-fifths of the instances , their corresponding adversarial variants are assigned very low weights ( less than 0.2 ) . Second , we find that the robustness of the IRAT-trained classifier drops significantly when facing an unseen attack . Figures 1 ( b ) and 1 ( c ) show that the classifier trained by GAIRAT ( with projected gradient descent ( PGD ) attack ( Madry et al. , 2018 ) ) has lower robustness when attacked by the unseen Carlini and Wagner attack ( CW ) ( Carlini & Wagner , 2017b ) compared with the robustness of standard adversarial training ( SAT ) ( Madry et al. , 2018 ) . Since the robustness of a model is not determined by any single attack , the improved performance for a kind of given attack is not sufficient to justify robustness . As shown in Figure 2 , the ranking of attack intensity varies with different datasets , adversarial training methods and so on , which means there is not the strongest attack in all situations . For another , previous studies of the adversarial detection ( Lee et al. , 2018 ; Ma et al. , 2018 ; Gao et al. , 2021 ) show that the stronger adversarial examples for robust classifiers to predict , the easier for the adversarial detectors to detect . This means improving performance not only on stronger attacks , but also on weaker ones , as some of attackers may choose to use weaker attacks to escape from the adversarial detection . Hence , robustness needs the network to consider a variety of potential attacks . In this paper , we investigate the reasons behind the failure of IRAT on unseen attacks . Unlike the common scenario of the classification problem where the training and testing data are fixed , there are different adversarial variants for the same instance in AT , e.g. , PGD-based or CW-based adversarial variants . A natural question comes with this—whether there are inconsistent vulnerabilities in the view of different attacks ? The answer is affirmative . Figure 3 visualizes this phenomenon using t-SNE ( Van der Maaten & Hinton , 2008 ) . The four subfigures in Figure 3 visualize inconsistent vulnerable instances in different views using t-SNE . The red dots in all subfigures represent consistently vulnerable instances between different views , while the blue dots represent the inconsistent vulnerable instances . From the SAT-trained classifier ( Figure 3 ( a ) -3 ( d ) ) , we can clearly see that a large number of vulnerable instances are inconsistent ( the blue dots dominate in all the subfigures ) . Given the above investigation , we argue that the safeness of instances is attack-dependent , that is , for the same instance , its weight can change given different attacks based on the same model . Thus , if the attack simulated in training is mis-specified , the weights of IRAT are misleading . In order to ameliorate this pessimism of IRAT , we propose our solution—locally reweighted adversarial training ( LRAT ) . As shown in Figure 4 , LRAT pairs each instance with its adversarial variants and performs local reweighting inside each pair , while performing no global reweighting . The rationale of LRAT is to fit the instance itself if it is immune to the attack , and in order to passively defend different attacks in future , LRAT does not skip the pair . In terms of the common attribute of adversarial attacks that misleading the classifier with a low probability of predicting the correct label and a high probability of predicting the false label ( Madry et al. , 2018 ; Carlini & Wagner , 2017b ) , we propose a general reweighting strategy that is applicable to various representatively attacks . Our experimental results show that compared with AT , LRAT can achieve better results on various simulated attacks . Other than previous global reweighting training that GAIRAT , LRAT can boost the performance of AT on PGD with no decline on other attacks . 2 ADVERSARIAL TRAINING . In this section , we briefly review existing adversarial training methods ( Madry et al. , 2018 ; Zhang et al. , 2021 ) . Let ( X , d∞ ) be the input feature space X with a metric d∞ ( x , x′ ) = ‖x− x′‖∞ , and B [ x ] = { x′ | d∞ ( x , x′ ) ≤ } be the closed ball of radius > 0 centered at x in X . The dataset S = { xi , yi } ni=1 , where xi ∈ X , yi ∈ Y = { 0 , 1 , . . . , K−1 } . We use fθ ( x ) to denote a deep neural network parameterized by θ . Specifically , fθ ( x ) predicts the label of an input instance x via : fθ ( x ) = arg max k∈K pk ( x ; θ ) , ( 1 ) where pk ( x ; θ ) denotes the predicted probability ( softmax on logits ) of x belonging to class k . 2.1 STANDARD ADVERSARIAL TRAINING . The objective function of SAT ( Madry et al. , 2018 ) is min fθ∈F 1 n n∑ i=1 ` ( fθ ( x̃i ) , yi ) , ( 2 ) where x̃i = arg maxx̃i∈B [ x ] ` ( fθ ( x̃i ) , yi ) and F = { f : X → Y } . The selected x̃ is the most adversarial variant within the -ball center at x . The loss function ` : RK × Y → R is a composition of a base loss ` B : 4K−1 × Y → R ( e.g. , the cross-entropy loss ) and an inverse link function ` L : RK → 4K−1 ( e.g. , the soft-max activation ) , where 4K−1 is the corresponding probalility simplex . Namely , ` ( fθ ( · ) , y ) = ` B ( ` L ( fθ ( · ) ) , y ) . PGD ( Madry et al. , 2018 ) is the most common approximation method for searching the most adversarial variant . Starting from x ( 0 ) ∈ X , PGD ( with step size α > 0 ) works as follows : x ( t+1 ) = ΠB [ x ( 0 ) ] ( x ( t ) + α sign ( ∇xt ` ( fθ ( x ( t ) , y ) ) ) , t ∈ N , ( 3 ) where N is the number of iterations ; x ( 0 ) refers to the starting point that natural instance ( or a natural instance perturbed by a small Gaussian or uniformly random noise ) ; y is the corresponding label for x ( 0 ) ; x ( t ) is the adversarial variant at step t ; ΠB [ x ( 0 ) ] ( · ) is the projection function that projects the adversarial variant back into the -ball centered at x ( 0 ) if necessary . 2.2 GEOMETRY-AWARE INSTANCE-REWEIGHTED ADVERSARIAL TRAINING . GAIRAT is a typical IRAT proposed by Zhang et al . ( 2021 ) . GAIRAT argues that natural training data farther from/close to the decision boundary are safe/non-robustness , and should be assigned with smaller/larger weights . Let ω ( x , y ) be the geometry-aware weight assignment function on the loss of the adversarial variant x̃ , where the generation of x̃ follows SAT . GAIRAT aims to min fθ∈F 1 n n∑ i=1 ω ( xi , yi ) ` ( fθ ( x̃i ) , yi ) . ( 4 ) Eq . ( 4 ) rescales the loss using a function ω ( x , y ) . This function is non-increasing w.r.t . GD , which is defined as the least steps that the PGD method needs to successfully attack the natural instances . The method then normalizes ω to ensure that ω ( x , y ) ≤ 0 and 1n ∑n i=1 ω ( xi , yi ) = 1 . Finally GAIRAT employs a bootstrap period in the initial part of the training by setting ω ( xi , yi ) = 1 , thereby performing regular training and ignoring the geometric-distance of input ( xi , yi ) . 3 THE LIMITATION OF IRAT : INCONSISTENT VULNERABILITY IN DIFFERENT VIEWS . The difference between Eq . ( 2 ) and Eq . ( 4 ) is the addition of the geometry-aware weight ω ( x , y ) . According to GAIRAT ( Zhang et al. , 2021 ) , more vulnerable instances should be assigned larger weights . However , the relative vulnerability between instances may vary in different situations , such as for different adversarial variants . V represents the selected variable to measure the vulnerability between the classifier and adversarial variants , and the smaller V , the more vulnerable is the instance . For the PGD-based adversarial variant , the lower predicted probability it has on true class , the smaller V . Here , we give the definition of VPGD . Definition 1 ( Vulnerability in the view of PGD ) . In the view of PGD , the vulnerability VPGD regarding x̃ ( generated by PGD ) is defined as VPGD ( x̃ , y ) : = py ( x̃ ) , ( 5 ) where py denotes the predicted probability ( softmax on logits ) of x̃ belonging to the true class y . For the CW-based adversarial variant , the relatively higher predicted probability it has on the false class , the larger VCW . Here , we give the definition of VCW . Definition 2 ( Vulnerability in the view of CW ) . In the view of CW , the vulnerability VCW regarding x̃ ( generated by CW ) is defined as VCW ( x̃ , y ) : = zy ( x̃ ) −max i6=y zi ( x̃ ) , ( 6 ) where zy denotes the predicted logits of x̃ ( CW ) belonging to the true class y . The maxi6=y zi ( x̃ ) denotes the maximum predicted logits of x̃ ( CW ) belonging to the false class i ( i 6= y ) . As shown in Figure 5 , the dark yellow and dark blue are top-20 % vulnerable instances in the view of PGD and CW , respectively . The frequency distribution of dark yellow in Figure 5 ( b ) and dark blue in Figure 5 ( c ) is clearly different , and the frequency distribution of dark blue in Figure 5 ( e ) and dark yellow in Figure 5 ( f ) is also different . This phenomenon means that the most vulnerable 10 , 000 PGD adversarial variants and the most vulnerable 10 , 000 CW adversarial variants are not from the same 10 , 000 instances . As a consequence of this inconsistency , if the attack simulated in training is mis-specified , the weights of IRAT will be misleading .

Summary: Authors pointed out that instances-reweighted adversarial training (IRAT) suffer from vulnerability against the unseen types of attacks. Authors claimed that a large number of instances are actually overlooked in IRAT which may cause that phenomenon. Therefore, the authors suggest locally reweighted adversarial training (LRAT) which performs local reweighting between the adversarial variants pair.