Probabilistic Margins for Instance Reweighting in Adversarial Training

1 Introduction . Deep neural networks are susceptible to adversarial examples that are generated by changing natural inputs with malicious perturbation [ 17 , 20 , 37 , 39 ] . Those examples are imperceptible to human eyes but can fool deep models to make wrong predictions with high confidence [ 3 , 35 ] . As deep learning has been deployed in many real-life scenarios and even safety-critical systems [ 15 , 16 , 28 ] , it is crucial to make such deep models reliable and safe [ 19 , 25 , 30 ] . To obtain more reliable deep models , adversarial training ( AT ) [ 2 , 10 , 29 ] was proposed as one of the most effective methodologies against adversary attacks ( i.e. , maliciously changing natural inputs ) . Specifically , during training , it simulates adversarial examples ( e.g. , via project gradient descent ( PGD ) [ 1 , 29 , 46 , 41 ] ) and train a classifier ∗Equal contribution . †Corresponding author . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . # epoch a c c u ra c y ( % ) # step lo s s # e x a m p le LPS success fail PGD LM-PGD CW GAIRAT CW ( last ) AT PGD GAIRAT PGD ( last ) AT ( a ) Loss Curve c a c c u ra c y ( % ) st lo s s # e x a m p le success fail L - AI AT ( last ) AT P AI AT P ( last ) AT ( b ) Histogram # epoch a c c u ra c y ( % ) lo s s CW GAIRAT CW ( last ) AT PGD GAIRAT PGD ( last ) AT ( c ) Accuracy Curve with the simulated adversarial examples [ 29 , 46 ] . Since such a model has seen some adversarial examples during its training process , it can defend against certain adversarial attacks and is more adversarial-robust than traditional classifiers trained with natural data [ 7 , 14 , 36 , 49 ] . Recently , researchers have found that over-parameterized deep networks still have the insufficient model capacity , due to the overwhelming smoothing effect of AT [ 47 , 33 ] . As a result , they proposed instance-reweighted adversarial training , where adversarial data should have unequal importance given limited model capacity [ 47 ] . Concretely , they suggested that data closer to the decision boundaries are much more vulnerable to be attacked [ 45 , 47 ] and should be assigned larger weights during training . To characterize these geometric properties of data ( i.e. , the closeness between the data and decision boundaries ) , Zhang et al . [ 47 ] suggested an estimation in the input space , i.e. , the least PGD steps ( LPS ) , to identify non-robust ( i.e. , easily-be-attacked ) data . Specifically , LPS is the number of steps to make the adversarial variant of such an instance cross the decision boundaries , starting from a natural instance . Based on LPS , they achieved state-of-the-art performance via a general framework termed geometry-aware instance-reweighted adversarial training ( GAIRAT ) . PM=-0.2 : close to class 𝑗 PM=0.2 : close to class 𝑖 However , existing methods [ 18 , 22 , 38 ] in measuring the geometric properties of data are path-dependent , i.e. , they may change even given the same start ( a natural example ) and end point ( the adversarial variant ) ; and they are discrete with only a few valid values ( Figure 2 ) . The path-dependency makes the computation unstable , where the results may change given different attack paths . The discreteness makes the measurement ambiguous since each value would have several ( even contradictory ) meanings . We take LPS as an example to demonstrate a path-dependent ( Figure 1a ) and discrete measurement ( Figure 1b ) . As we can see in Figure 1c , the consequence is non-negligible : although adopting LPS ( i.e. , GAIRAT ) reveals improvement for the PGD-100 attack [ 29 ] , its performance is well below AT regarding the CW attack [ 9 ] . In this paper , we propose the probabilistic margin ( PM ) , which is continuous and path-independent , for reweighting adversarial data during AT . PM is a geometric measurement from a data point to the closest decision boundary , following the multi-class margin [ 23 ] in traditional geometry-aware machine learning . Note that , instead of choosing the input space as LPS , PM is defined by the estimated class-posterior probabilities from the model outputs , e.g. , such the probability of the true label minus the probability of the most confusing label given some natural data . Therefore , PM is computed in a low-dimensional embedding space with normalization , alleviating the troubles in comparing data from different classes . The definition of PM is general , where we consider three specifications , namely , PMnat , PMadv , and PMdif . Concretely , PMnat and PMadv are the PM scores regarding natural and adversarial data , respectively . They assume that the vulnerability of data is revealed by the closeness regarding either the natural data or the adversarial variants . However , the definition of PMdif is slightly different , which is the distance of a natural data point to its adversarial variant . PMdif is viewed as a conceptual counterpart of LPS , but is critically different since it is continuous and path-independent . In our paper , we verified the effectiveness of PMnat and PMadv and showed that they can represent the geometric properties of data well . Note that , though these types of PMs depict different geometric properties , they all share a negative correlation with the vulnerability of data—larger/smaller PMs indicate that the corresponding data are safer/riskier and thus should be assigned with smaller/larger weights . Eventually , PM is employed for reweighting adversarial data during AT , where we propose the MarginAware Instance reweighting Learning ( MAIL ) . With a non-increased weight assignment function in Eq . ( 8 ) , MAIL pays much attention to those non-robust data . In experiments , MAIL was combined with various forms of commonly-used AT methods , including traditional AT [ 29 ] , MART [ 40 ] , and TRADES [ 45 ] . We demonstrated that PM is more reliable than previous works in geometric measurement , irrelevant to the forms of the adopted objectives . Moreover , in comparison with advanced methods , MAIL revealed its state-of-the-art performance against various attack methods , which benefits from our path-independent and continuous measurement . 2 Preliminary . 2.1 Traditional Adversarial Training . For a K-classification problem , we consider a training dataset S = { ( xi , yi ) } ni=1 independently drawn from a distribution D and a deep neural network h ( x ; θ ) parameterized by θ . This deep classifier h ( x ; θ ) predicts the label of an input data via h ( x ; θ ) = arg maxk pk ( x ; θ ) , with pk ( x , θ ) being the predicted probability ( softmax on logits ) for the k-th class . The goal of AT is to train a model with a low adversarial risk regarding the distribution D , i.e. , R ( θ ) = E ( x , y ) ∼D [ maxδ∈∆ ` ( x+ δ , y ; θ ) ] , where ∆ is the threat model , defined by an Lp-norm bounded perturbation with the radius : ∆ = { δ ∈ Rd | ||δ||p ≤ } . Therein , AT computes the new perturbation to update the model parameters , where the PGD method [ 29 ] is commonly adopted : for a ( natural ) example xi , it starts with random noise ξ and repeatedly computes δ ( t ) i ← Proj [ δ ( t−1 ) i + αsign ( ∇θ ` ( xi + δ ( t−1 ) i , yi ; θ ) ) ] , ( 1 ) with Proj the clipping operation such that δ ( t ) is always in ∆ and sign the signum function . Due to the non-convexity , we typically approximate the optimal solution by δ ( T ) i with T being the maximally allowed iterations . Accordingly , δ ( T ) i is viewed as the perturbation for the most adversarial example [ 47 ] , and the learning objective function is formulated by arg min θ ∑ i ` ( xi + δ ( T ) i , yi ; θ ) . ( 2 ) Intuitively , AT corresponds to the worst-case robust optimization , continuously augmenting the training dataset with adversarial variants that highly confuse the current model . Therefore , it is a practical learning framework to alleviate the impact of adversarial attacks . Unfortunately , it leads to insufficient network capacity , resulting in unsatisfactory model performance regarding adversarial robustness . The reason is that , AT has an overwhelming smoothing effect in fitting highly adversarial examples [ 45 ] , and thus consumes large model capacity to learn from some individual data points . 2.2 Geometry-Aware Adversarial Training . Zhang et al . [ 47 ] claimed that training examples should have unequal significance in AT , and proposed the geometry-aware instance-reweighted adversarial training ( GAIRAT ) . It is a general framework to reweight adversarial data during training , where Eq . ( 2 ) is modified as arg min θ ∑ i ωi ` ( xi + δ ( T ) i , yi ; θ ) s.t . ωi ≥ 0 and ∑ i ωi = 1 . ( 3 ) Note that , the constraints are required since the risk after weighting is consistent with the original one without weighting . Further , the generation of perturbation still follows Eq . ( 1 ) . They revealed that data near decision boundary are much vulnerable to be attacked and require large weights . LPS as a geometric measurement . In assigning weights , GAIRAT needs a proper measurement for the distance to the decision boundaries . They suggested the estimation in high-dimensional input space via LPS ( Fig . 2 ) , which is the least PGD iterations for a perturbation that leads to a wrong prediction . Intuitively , a small LPS indicates that the data point can quickly cross the decision boundary and thus close to it . The drawbacks of LPS . Although promising results have been verified in experiments , LPS is path-dependent and limited by a few discrete values , where the consequence is non-negligible . In Figure 1 ( a ) , we showed that the PGD method will get stuck . Therefore , using LPS as a geometric measurement , we might identify non-robust examples as robust ones . Here , we modified the vanilla PGD method with the line-searched learning rate [ 43 ] and Nesterov momentum [ 31 ] ( see Appendix A ) , termed Line-search & Momentum-PGD ( LM-PGD ) , and we compared the loss curve of PGD with that of LM-PGD on one example . The maximal iterations of PGD was 50 . As we can see , PGD almost converged at the 14-th step , and the loss value did not ascend anymore . As a result , LPS of this example was 50 , and this example was taken as a robust one in the view of LPS . However , LM-PGD still ascended after the 14-th step and successfully attacked the instance at the 15-th step . This result means that such an instance is not a robust one , but LPS made a wrong judgment in its robustness . The main reason is that , LPS is heavily dependent on attack paths , even though both paths are highly similar . Now , we demonstrate that the limited range of LPS would cause problems as well . In Figure 1b , we show the histogram of LPSs for data on CIFAR-10 [ 24 ] . Higher LPS values mean that these examples were more robust and required smaller weights during AT . It could be seen that LPS has a confusing meaning when it equaled the maximal value , which is 10 following [ 29 ] . For data whose LPSs were 10 , they would be the most robust/safe ones . However , it could be seen that they still contained the critical data points ( the blue part ) . Although the proportion of the critical data seems low , ignoring them during AT ( i.e. , assigning small weights for them ) will cause problems . For example , the trained classifier ’ s accuracy will drop significantly when facing the CW attack [ 9 ] ( Figure 1c ) .

Summary: Reweighting adversarial data has been recently shown to be effective, with larger weights assigned for data closer to the boundaries. Previous work used the LPS to measure the closeness, but the authors suggested that the LPS might not be reliable since they are discrete and path-dependent. To this end, this paper proposed three types of probabilistic margin, which could overcome the drawbacks of the LPS. The probabilistic margins were used to measure the importance of adversarial data, which induced convincing results in experiments.