Sensible adversarial learning

1 INTRODUCTION . With many impressive successes of deep learning , there are a multitude of applications of deep neural networks ( DNNs ) that permeate in our everyday life . As DNNs are applied in securitycritical systems such as malware detection , face identification , and autonomous driving , robustness of DNNs against adversarial attacks , i.e. , the intently perturbed inputs to fool the system , has become an important research topic ( Szegedy et al. , 2013 ; Papernot et al. , 2016 ; Biggio et al. , 2013 ) . One of the most widely studied classes of adversarial perturbations is ` p-norm constrained adversarial perturbations ( Szegedy et al. , 2013 ) . Madry et al . ( 2017 ) formalize the adversarial learning against this class of perturbations as a minimization problem of adversarial risk defined in a following way . Let ( X , y ) ∈ X × Y be from some unknown distribution PX , Y . Given a loss function ` : Y × Y → R and a constraint constant > 0 , the adversarial robust risk is Rrob ( f ) = EX , Y [ max ‖δ‖p≤ ` ( f ( X + δ ) , y ) ] . ( 1 ) Many adversarial learning methods can be interpreted as empirical minimization of ( 1 ) ( Goodfellow et al. , 2014 ; Kurakin et al. , 2016b ; Ruitong Huang & Szepesvari , 2015 ; Madry et al. , 2017 ) . For this optimization problem , Madry et al . ( 2017 ) propose to train a robust model with the augmented data generated by the projected gradient descent method ( PGD ) . On this adversarial training , they make two important observations . First , it costs natural accuracy . A network trained with adversarial examples tends to have a lower natural accuracy than a naturally trained network . This trade-off is observed even with a small training . Second , the adversarial training requires a larger model capacity than the natural training does . If the model capacity is only sufficient for the natural learning , the adversarial training can converge to a constant function . For a large , the optimization problem of ( 1 ) itself may pose the trade-off . For instance , Tsipras et al . ( 2018 ) show an example of an inherent tension between pursuits of accuracy and robustness when is large enough to change the true class . For a smaller , however , the formulation in ( 1 ) does not explicitly pose any conflict between the pursuit of robustness and accuracy . Note that Rrob ( f ) is an upper bound of the standard risk of f . A perfectly robust model f with Rrob ( f ) = 0 is also perfectly accurate for natural learning . If the perfect classifier exists in a given model class , the trade-off may be caused by the large sample complexity of adversarially robust generalization ( Schmidt et al. , 2018 ; Yin et al. , 2018 ; Stutz et al. , 2019 ) . Without sufficiently large amount of data , the empirical minimization of ( 1 ) may result in a large standard risk by converging to a model of a poor robust risk . On the other hand , if robust learning converges to a constant function , it can not achieve natural accuracy . In this sense , to resolve the trade-off problem , we may need to deal with the increased requirement on the model capacity . In this paper , we propose a novel framework , called sensible adversary , in order to overcome the trade-off between natural accuracy and robustness . In particular , we restrict adversarial perturbations not to cross the Bayes decision boundary besides the -ball constraint , so that the perturbation ball is adaptively modified for every single data point . Our main contributions are : • Under the framework of sensible adversary , the pursuit of robustness and accuracy given an enough model capacity can align with each other , i.e. , there is no trade-off . We theoretically establish the Bayes rule is most robust against the sensible adversary . If the Bayes decision boundary can be far from data manifolds at least by , our pursuit of sensible robustness does not cost any adversarial robust risk . • We propose an efficient algorithm for sensible adversarial training , which utilizes sensible adversaries in the absence of the true Bayes rule . This sensible adversarial training enjoys robustness without a significant drop of natural accuracy . Furthermore , the algorithm is not sensitive to the model capacity . When insufficient model capacity is given , our algorithm does not collapse to a constant function . Instead , it trains a model as robust as possible . • We experimentally demonstrate that sensible adversarial training enables to stably learn a robust and accurate model . In particular , on CIFAR10 , we achieve 91.51 % natural test accuracy and 57.23 % robust test accuracy against ` ∞ PGD attacks constrained to = 8/255 . To the best of our knowledge , there is no approaches known to achieve natural accuracy more than 90 % , while achieving more than 55 % of robust accuracy against PGD attacks of = 8/255 . Moreover , no previous approaches pursuing robustness against this attack achieved the natural accuracy more than . 1.1 RELATED WORK . Madry et al . ( 2017 ) formalize adversarial learning as a mini-max problem given perturbation restriction , and theoretically and empirically established the feasibility of the optimization . Our sensible adversary redefines the set of perturbation in the inner maximization problem on which their theoretical result is directly applicable . Tsipras et al . ( 2018 ) investigate the possible source of robust trade-off . The key idea is that when there are features that are useful for natural classification but vulnerable to adversarial perturbations , a robust model would abandon these features because otherwise all of these features can adversarially move to promote incorrect prediction . Our work explores the possibility of learning a robust model while not allowing such collective adversarial migration . While a class change by adversarial examples typically has been prevented by using a small , Suggala et al . ( 2018 ) explicitly ignore an adversarial perturbation that crosses the decision boundary of a Bayes rule . Zhang et al . ( 2019 ) also investigate the Bayes decision boundary to resolve the trade-off problem . They search for a model f having a small weighted sum of the natural risk and a probability that an adversarial example can cross the decision boundary of f . Gilmer et al . ( 2018 ) show that in high dimensional setting , even small test error can imply the existence of adversarial examples for most of data points . Our effort to prioritize natural accuracy to find a robust model is consistent to the view in Gilmer et al . ( 2018 ) . More related work will be presented in Appendix A . 1.2 NOTATION . Let F denote the class of functions represented by DNNs with a fixed architecture . An optimal robust model w.r.t . PX , Y is denoted by frob = arg minf∈F Rrob ( f ) . Denote the standard risk w.r.t PX , Y by Rstd ( f ) = EX , Y [ ` ( f ( X ) , y ) ] and its optimal natural model by fstd = arg minf∈F Rstd ( f ) . Let P̃X , Y = PX , Y |X̃×Y denote a restricted distribution of PX , Y on a subset X̃ × Y ⊂ X × Y . Likewise , denote the standard risk w.r.t P̃X , Y by R̃std ( f ) and the robust risk w.r.t P̃X , Y by R̃rob ( f ) . For a set A , the -neighborhood in ` p-norm is defined as B ( A , ) = { y|‖y − x‖p ≤ , x ∈ A } , and the interior is denoted by int ( A ) . Denote the - neighborhood of the decision boundary of f by DB ( f , ) = { x ∣∣ ∃x′ ∈ B ( x , ) s.t . f ( x ) 6= f ( x′ ) } . Let p̂f , y ( x ) denote the predicted probability of the label of x being y by a model f . 2 ADVERSARIAL LEARNING MAY HELP STANDARD LEARNING . In this section , we use a toy example to investigate the synergistic effect between the pursuit of robustness and natural accuracy . Example : We can easily find an example of trade-off wherever an -ball attack can cross the manifold boundary between classes . For example , consider a two-dimensional random vector X = ( X1 , X2 ) on X = ( 0 , 1 ) × ( 0 , 1 ) with a binary class Y ∼ Ber ( p ) , where p > 0.5 . Let the conditional distribution be ( X1 , X2 ) |Y = 0 ∼ Unif ( ( 0 , 12 ) × ( 0 , 1 ) ) and ( X1 , X2 ) |Y = 1 ∼ Unif ( ( 12 , 1 ) × ( 0 , 1 ) ) . Then the Bayes Rule is f B ( x ) = sign ( x1−0.5 ) , and it is a perfect classifier , in thatRstd ( fB ) = 0 with ` as the 0-1 loss . If p > 0.5 , the robust classifier against -ball attacks is frob ( x ) = sign ( x1− ( 0.5− ) ) . Its decision boundary is deviated by− from that of the Bayes rule , and this deviation costs natural accuracy byRstd ( frob ) −Rstd ( f ) = ( 1− p ) . However , when the data points reside in a high-dimensional space , the samples are too sparse to represent the underlying true distribution . To take this phenomenon into account , consider a distribution P̃X , Y = PX , Y |X̃×Y on a subset X̃ × Y ⊂ X × Y . Assume that we only observe data generated from P̃X , Y , and the training and test sets do not provide any information on X̃ c × Y . Now by applying this support restriction to the example above , we demonstrate how adversarial learning dramatically changes from harming to benefiting natural accuracy . The Cheese hole distribution : For the above example , we restrict the support on X̃ = ∪3i=1 ∪3j=1 Hij , whereHij = ( ( α2 , 3α 2 ) +2α ( i−1 ) ) × ( ( α 2 , 3α 2 ) +2α ( j−1 ) ) , α = 1 6 , and j = 1 , 2 , 3 . Therefore the sampling support X̃ comprises of nine small squares equally spaced by α . This is illustrated in Figure 1 ( a ) . Among all classifiers which predict the same as fB on X̃ , the worst one , denoted by f̃B∗ , predicts exact opposite on x ∈ X̃ c as illustrated in Figure 1 ( b ) . For this worst case , the true standard risk w.r.t PX , Y is 34 , although the standard risk w.r.t . P̃X , Y is zero . We argue that pursuing robustness can mitigate this discrepancy . Among minimizers of R̃rob , let f̃∗rob be the worst classifier in that it predicts incorrectly outside X̃ as depicted in Figure 1 ( c ) . On B ( X̃ , ) , f̃∗rob should correctly classify , except on A = { ( x1 , x2 ) |0.5− ≤ x1 ≤ 0.5 } ∩ B ( X̃ , ) . In this situation , the inaccuracy of f̃∗rob is compensated by its increased accuracy on B ( X̃ , ) \ A . Moreover , the composition ofRrob ( f̃B∗ ) is particularly interesting , and it is easy to show that Rrob ( f̃B∗ ) = P ( f̃B∗ ( Y ) = X , X ∈ DB ( f̃B∗ , ) ) + 3/4 , where 3/4 is from the standard inaccuracy on X̃ c. The major part of Rrob ( f̃B∗ ) comes from the inaccuracy of f̃B∗ outside of the sampling support X̃ . Therefore by simply reducing this inaccuracy on X̃ c , we can lower the robust risk by 3/4 . Another interesting observation is that B = { ( x1 , x2 ) |0.5 − < x1 < 0.5 + } is the only area where the Bayes rule fB has its robust risk . Note that Rrob ( fB ) = P ( X ∈ B , fB ( X ) = Y ) = 2 which is greater than the optimal robust risk . The reason why fB is not most robust is simply because it is accurate on B . This prevents fB from being a robust function under the current adversarial robustness framework . These observations have two implications . First , the robust pursuit can help to increase natural accuracy outside of X̃ where the samples are poorly representative . Second , by accurately correcting the model outside of X̃ in a standard sense , the robust risk can be significantly decreased . This is where the pursuit of robustness and accuracy coincides . Toward respecting the original data structure : What if we regard the adversarial robust risk near the class border as reasonable gullibility ? This corresponds to pursuit of robustness only against adversarial examples which do not cross between the class manifolds , i.e. , as long as this does not harm the natural accuracy on X̃ . We call this robustness as sensible robustness , which respects the structure represented by data . In this example , the sensible robustness can increase both robustness and natural accuracy . Let f̃s∗rob denote the worst case sensibly robust classifier w.r.t P̃X , Y . The decision boundary of f̃s∗ should not deviate to the left on B ( X̃ , ) no matter how large is , as depicted in Figure 1 ( d ) . Note that when ≥ α/2 , f̃∗rob is the unique minimizer of R̃rob ( f ) as B ( X̃ , ) covers X .

Summary: The paper proposes the notion of a "sensible" adversary that does not perturb data points on which the Bayes-optimal classifier is incorrect. The authors then provide theory showing that minimizing robust risk against such a sensible adversary yields the Bayes-optimal classifier, which addresses the question about standard vs. robust risk posed in prior work. On the experimental side, the authors then introduce a simple yet effective variation of adversarial training / robust optimization. Instead of maximizing the loss over the perturbation set, the proposed variant stops as soon as the loss exceeds a certain threshold. This can be seen as a variant of gradient clipping that reduces the influence of examples with a very high loss. The authors show that their modification yields an 8 - 9% improvement in robust accuracy on CIFAR-10, which gives state-of-the-art performance.