Sensible adversarial learning

1 INTRODUCTION . With many impressive successes of deep learning , there are a multitude of applications of deep neural networks ( DNNs ) that permeate in our everyday life . As DNNs are applied in securitycritical systems such as malware detection , face identification , and autonomous driving , robustness of DNNs against adversarial attacks , i.e. , the intently perturbed inputs to fool the system , has become an important research topic ( Szegedy et al. , 2013 ; Papernot et al. , 2016 ; Biggio et al. , 2013 ) . One of the most widely studied classes of adversarial perturbations is ` p-norm constrained adversarial perturbations ( Szegedy et al. , 2013 ) . Madry et al . ( 2017 ) formalize the adversarial learning against this class of perturbations as a minimization problem of adversarial risk defined in a following way . Let ( X , y ) ∈ X × Y be from some unknown distribution PX , Y . Given a loss function ` : Y × Y → R and a constraint constant > 0 , the adversarial robust risk is Rrob ( f ) = EX , Y [ max ‖δ‖p≤ ` ( f ( X + δ ) , y ) ] . ( 1 ) Many adversarial learning methods can be interpreted as empirical minimization of ( 1 ) ( Goodfellow et al. , 2014 ; Kurakin et al. , 2016b ; Ruitong Huang & Szepesvari , 2015 ; Madry et al. , 2017 ) . For this optimization problem , Madry et al . ( 2017 ) propose to train a robust model with the augmented data generated by the projected gradient descent method ( PGD ) . On this adversarial training , they make two important observations . First , it costs natural accuracy . A network trained with adversarial examples tends to have a lower natural accuracy than a naturally trained network . This trade-off is observed even with a small training . Second , the adversarial training requires a larger model capacity than the natural training does . If the model capacity is only sufficient for the natural learning , the adversarial training can converge to a constant function . For a large , the optimization problem of ( 1 ) itself may pose the trade-off . For instance , Tsipras et al . ( 2018 ) show an example of an inherent tension between pursuits of accuracy and robustness when is large enough to change the true class . For a smaller , however , the formulation in ( 1 ) does not explicitly pose any conflict between the pursuit of robustness and accuracy . Note that Rrob ( f ) is an upper bound of the standard risk of f . A perfectly robust model f with Rrob ( f ) = 0 is also perfectly accurate for natural learning . If the perfect classifier exists in a given model class , the trade-off may be caused by the large sample complexity of adversarially robust generalization ( Schmidt et al. , 2018 ; Yin et al. , 2018 ; Stutz et al. , 2019 ) . Without sufficiently large amount of data , the empirical minimization of ( 1 ) may result in a large standard risk by converging to a model of a poor robust risk . On the other hand , if robust learning converges to a constant function , it can not achieve natural accuracy . In this sense , to resolve the trade-off problem , we may need to deal with the increased requirement on the model capacity . In this paper , we propose a novel framework , called sensible adversary , in order to overcome the trade-off between natural accuracy and robustness . In particular , we restrict adversarial perturbations not to cross the Bayes decision boundary besides the -ball constraint , so that the perturbation ball is adaptively modified for every single data point . Our main contributions are : • Under the framework of sensible adversary , the pursuit of robustness and accuracy given an enough model capacity can align with each other , i.e. , there is no trade-off . We theoretically establish the Bayes rule is most robust against the sensible adversary . If the Bayes decision boundary can be far from data manifolds at least by , our pursuit of sensible robustness does not cost any adversarial robust risk . • We propose an efficient algorithm for sensible adversarial training , which utilizes sensible adversaries in the absence of the true Bayes rule . This sensible adversarial training enjoys robustness without a significant drop of natural accuracy . Furthermore , the algorithm is not sensitive to the model capacity . When insufficient model capacity is given , our algorithm does not collapse to a constant function . Instead , it trains a model as robust as possible . • We experimentally demonstrate that sensible adversarial training enables to stably learn a robust and accurate model . In particular , on CIFAR10 , we achieve 91.51 % natural test accuracy and 57.23 % robust test accuracy against ` ∞ PGD attacks constrained to = 8/255 . To the best of our knowledge , there is no approaches known to achieve natural accuracy more than 90 % , while achieving more than 55 % of robust accuracy against PGD attacks of = 8/255 . Moreover , no previous approaches pursuing robustness against this attack achieved the natural accuracy more than . 1.1 RELATED WORK . Madry et al . ( 2017 ) formalize adversarial learning as a mini-max problem given perturbation restriction , and theoretically and empirically established the feasibility of the optimization . Our sensible adversary redefines the set of perturbation in the inner maximization problem on which their theoretical result is directly applicable . Tsipras et al . ( 2018 ) investigate the possible source of robust trade-off . The key idea is that when there are features that are useful for natural classification but vulnerable to adversarial perturbations , a robust model would abandon these features because otherwise all of these features can adversarially move to promote incorrect prediction . Our work explores the possibility of learning a robust model while not allowing such collective adversarial migration . While a class change by adversarial examples typically has been prevented by using a small , Suggala et al . ( 2018 ) explicitly ignore an adversarial perturbation that crosses the decision boundary of a Bayes rule . Zhang et al . ( 2019 ) also investigate the Bayes decision boundary to resolve the trade-off problem . They search for a model f having a small weighted sum of the natural risk and a probability that an adversarial example can cross the decision boundary of f . Gilmer et al . ( 2018 ) show that in high dimensional setting , even small test error can imply the existence of adversarial examples for most of data points . Our effort to prioritize natural accuracy to find a robust model is consistent to the view in Gilmer et al . ( 2018 ) . More related work will be presented in Appendix A . 1.2 NOTATION . Let F denote the class of functions represented by DNNs with a fixed architecture . An optimal robust model w.r.t . PX , Y is denoted by frob = arg minf∈F Rrob ( f ) . Denote the standard risk w.r.t PX , Y by Rstd ( f ) = EX , Y [ ` ( f ( X ) , y ) ] and its optimal natural model by fstd = arg minf∈F Rstd ( f ) . Let P̃X , Y = PX , Y |X̃×Y denote a restricted distribution of PX , Y on a subset X̃ × Y ⊂ X × Y . Likewise , denote the standard risk w.r.t P̃X , Y by R̃std ( f ) and the robust risk w.r.t P̃X , Y by R̃rob ( f ) . For a set A , the -neighborhood in ` p-norm is defined as B ( A , ) = { y|‖y − x‖p ≤ , x ∈ A } , and the interior is denoted by int ( A ) . Denote the - neighborhood of the decision boundary of f by DB ( f , ) = { x ∣∣ ∃x′ ∈ B ( x , ) s.t . f ( x ) 6= f ( x′ ) } . Let p̂f , y ( x ) denote the predicted probability of the label of x being y by a model f . 2 ADVERSARIAL LEARNING MAY HELP STANDARD LEARNING . In this section , we use a toy example to investigate the synergistic effect between the pursuit of robustness and natural accuracy . Example : We can easily find an example of trade-off wherever an -ball attack can cross the manifold boundary between classes . For example , consider a two-dimensional random vector X = ( X1 , X2 ) on X = ( 0 , 1 ) × ( 0 , 1 ) with a binary class Y ∼ Ber ( p ) , where p > 0.5 . Let the conditional distribution be ( X1 , X2 ) |Y = 0 ∼ Unif ( ( 0 , 12 ) × ( 0 , 1 ) ) and ( X1 , X2 ) |Y = 1 ∼ Unif ( ( 12 , 1 ) × ( 0 , 1 ) ) . Then the Bayes Rule is f B ( x ) = sign ( x1−0.5 ) , and it is a perfect classifier , in thatRstd ( fB ) = 0 with ` as the 0-1 loss . If p > 0.5 , the robust classifier against -ball attacks is frob ( x ) = sign ( x1− ( 0.5− ) ) . Its decision boundary is deviated by− from that of the Bayes rule , and this deviation costs natural accuracy byRstd ( frob ) −Rstd ( f ) = ( 1− p ) . However , when the data points reside in a high-dimensional space , the samples are too sparse to represent the underlying true distribution . To take this phenomenon into account , consider a distribution P̃X , Y = PX , Y |X̃×Y on a subset X̃ × Y ⊂ X × Y . Assume that we only observe data generated from P̃X , Y , and the training and test sets do not provide any information on X̃ c × Y . Now by applying this support restriction to the example above , we demonstrate how adversarial learning dramatically changes from harming to benefiting natural accuracy . The Cheese hole distribution : For the above example , we restrict the support on X̃ = ∪3i=1 ∪3j=1 Hij , whereHij = ( ( α2 , 3α 2 ) +2α ( i−1 ) ) × ( ( α 2 , 3α 2 ) +2α ( j−1 ) ) , α = 1 6 , and j = 1 , 2 , 3 . Therefore the sampling support X̃ comprises of nine small squares equally spaced by α . This is illustrated in Figure 1 ( a ) . Among all classifiers which predict the same as fB on X̃ , the worst one , denoted by f̃B∗ , predicts exact opposite on x ∈ X̃ c as illustrated in Figure 1 ( b ) . For this worst case , the true standard risk w.r.t PX , Y is 34 , although the standard risk w.r.t . P̃X , Y is zero . We argue that pursuing robustness can mitigate this discrepancy . Among minimizers of R̃rob , let f̃∗rob be the worst classifier in that it predicts incorrectly outside X̃ as depicted in Figure 1 ( c ) . On B ( X̃ , ) , f̃∗rob should correctly classify , except on A = { ( x1 , x2 ) |0.5− ≤ x1 ≤ 0.5 } ∩ B ( X̃ , ) . In this situation , the inaccuracy of f̃∗rob is compensated by its increased accuracy on B ( X̃ , ) \ A . Moreover , the composition ofRrob ( f̃B∗ ) is particularly interesting , and it is easy to show that Rrob ( f̃B∗ ) = P ( f̃B∗ ( Y ) = X , X ∈ DB ( f̃B∗ , ) ) + 3/4 , where 3/4 is from the standard inaccuracy on X̃ c. The major part of Rrob ( f̃B∗ ) comes from the inaccuracy of f̃B∗ outside of the sampling support X̃ . Therefore by simply reducing this inaccuracy on X̃ c , we can lower the robust risk by 3/4 . Another interesting observation is that B = { ( x1 , x2 ) |0.5 − < x1 < 0.5 + } is the only area where the Bayes rule fB has its robust risk . Note that Rrob ( fB ) = P ( X ∈ B , fB ( X ) = Y ) = 2 which is greater than the optimal robust risk . The reason why fB is not most robust is simply because it is accurate on B . This prevents fB from being a robust function under the current adversarial robustness framework . These observations have two implications . First , the robust pursuit can help to increase natural accuracy outside of X̃ where the samples are poorly representative . Second , by accurately correcting the model outside of X̃ in a standard sense , the robust risk can be significantly decreased . This is where the pursuit of robustness and accuracy coincides . Toward respecting the original data structure : What if we regard the adversarial robust risk near the class border as reasonable gullibility ? This corresponds to pursuit of robustness only against adversarial examples which do not cross between the class manifolds , i.e. , as long as this does not harm the natural accuracy on X̃ . We call this robustness as sensible robustness , which respects the structure represented by data . In this example , the sensible robustness can increase both robustness and natural accuracy . Let f̃s∗rob denote the worst case sensibly robust classifier w.r.t P̃X , Y . The decision boundary of f̃s∗ should not deviate to the left on B ( X̃ , ) no matter how large is , as depicted in Figure 1 ( d ) . Note that when ≥ α/2 , f̃∗rob is the unique minimizer of R̃rob ( f ) as B ( X̃ , ) covers X .

Summary: Motivated from the so-called trade-off between robustness and standard accuracy in the existing adversarial learning, this paper has proposed a "sensible" adversarial example framework without losing  significantly  performance in natural accuracy. Some toy examples have been presented, showing its reasonableness of the model. The proposed algorithm looks very simple, but it appears that it could be effective through some experiments on two data sets.