Provably robust classification of adversarial examples with detection

1 INTRODUCTION . Despite popularity and success of deep neural networks in many applications , their performance declines sharply in adversarial settings . Small adversarial perturbations are shown to greatly deteriorate the performance of neural network classifiers , which creates a growing concern for utilizing them in safety critical application where robust performance is key . In adversarial training , different methods with varying levels of computational complexity aim at robustifying the network by finding such adversarial examples at each training steps and adding them to the training dataset . While such methods exhibit empirical robustness , they lack verifiable guarantees as it is not provable that a more rigorous adversary , e.g. , one that does brute-force enumeration to compute adversarial perturbations , will not be able to cause the classifier to misclassify . It is thus desirable to provably verify the performance of robust classifiers without restricting the adversarial perturbations by inexact solvers , while restraining perturbations to a class of admissible set , e.g. , within an ` ∞ norm-bounded ball . Progress has been made by ‘ complete methods ’ that use Satisfiability Modulo Theory ( SMT ) or Mixed-Integer Programming ( MIP ) to provide exact robustness bounds , however , such approaches are expensive , and difficult to scale to large networks as exhaustive enumeration in the worst case is required ( Tjeng et al. , 2017 ; Ehlers , 2017 ; Xiao et al. , 2018 ) . ∗Work was done when the author was an intern at Bosch Center for Artificial Intelligence , Pittsburgh , PA. ‘ Incomplete methods ’ on the other hand , proceed by computing a differential upper bound on the worst-case adversarial loss , and similarly for the verification violations , with lower computational complexity and improved scalability . Such upper bounds , if easy to compute , can be utilized during the training , and yield provably robust networks with tight bounds . In particular , bound propagation via various methods such as differentiable geometric abstractions ( Mirman et al. , 2018 ) , convex polytope relaxation ( Wong & Kolter , 2018 ) , and more recently in ( Salman et al. , 2019 ; Balunovic & Vechev , 2020 ; Gowal et al. , 2018 ; Zhang et al. , 2020 ) , together with other techniques such as semidefinite relaxation , ( Fazlyab et al. , 2019 ; Raghunathan et al. , 2018 ) , and dual solutions via additional verifier networks ( Dvijotham et al. , 2018 ) fall within this category . In particular , recent successful use of Interval Bound Propagation ( IBP ) as a simple layer-by-layer bound propagation mechanism was shown to be very effective in Gowal et al . ( 2018 ) , which despite its light computational complexity exhibits SOTA robustness verification . Additionally , combining IBP in a forward bounding pass with linear relaxation based backward bounding pass ( CROWN ) Zhang et al . ( 2020 ) leads to improved robustness , although it can be up to 3-10 times slower . Alternative to robust classification , detection of adversarial examples can also provide robustness against adversarial attacks , where suspicious inputs will be flagged and the classifier “ rejects/abstains ” from assigning a label . There has been some work on detection of out-of-distribution examples Bitterwolf et al . ( 2020 ) , however the situation in the literature on the detection of adversarial examples is quite different from above . Most techniques that attempt to detect adversarial examples , either by training explicit classifiers to do so or by simply formulating “ hand-tuned ” detectors , still largely look to identify and exploit statistical properties of adversarial examples that appear in practice Smith & Gal ( 2018 ) ; Roth et al . ( 2019 ) . However , to provide a fair evaluation , a defense must be evaluated under attackers that attempt to fool both the classifier and the detector , while addressing particular characteristics of a given defense , e.g. , gradient obfuscation , non-differentability , randomization , and simplifying the attacker ’ s objective for increased efficiency . A non-exhaustive list of recent detection methods entails randomization and sparsity-based defenses ( Xiao et al. , 2019 ; Roth et al. , 2019 ; Pang et al. , 2019b ) , confidence and uncertainty-based detection ( Smith & Gal , 2018 ; Stutz et al. , 2020 ; Sheikholeslami et al. , 2020 ) , transformation-based defenses ( Bafna et al. , 2018 ; Yang et al. , 2019 ) , ensemble methods ( Verma & Swami , 2019 ; Pang et al. , 2019a ) , generative adversarial training Yin et al . ( 2020 ) , and many more . Unfortunately , existing defenses have largely proven to have poor performance against adaptive attacks ( Athalye et al. , 2018 ; Tramer et al. , 2020 ) , necessitating provable guarantees on detectors as well . Recently Laidlaw & Feizi ( 2019 ) have proposed joint training of classifier and detector , however it also does not provided any provable guarantees . Our contribution . In this work , we propose a new method for jointly training a provably robust classifier and detector . Specifically , by introducing an additional “ abstain/detection ” into a classifier , we show that the existing certified defense mechanisms can be modified , and by building on the detection capability of the network , classifier can effectively choose to either robustly classify or detect adversarial attacks . We extend the light-weight Interval Bound Propagation ( IBP ) method to account for our new robust objective , enabling verification of the network for provable performance guarantees . Our proposed robust training objective is also effectively upper bounded , enabling its incorporation into the training procedure leading to tight provably robust performance . While tightening of the bound propagation may be additionally possible for tighter verification , to the best of our knowledge , our approach is the first method to extend certification techniques by considering detection while providing provable verification . By stabilizing the training , as also used in similar IBP-based methods , experiments on MNIST and CIFAR-10 empirically show that the proposed method can successfully leverage its detection capability , and improves traditional IBP used in isolation , especially for large perturbation sizes . 2 BACKGROUND AND RELATED WORK . Let us consider an L-layer feed-forward neural network , trained for a K-class classification task . Given input x , it will pass through a sequential model , with hl denoting the mapping at layer l , recursively parameterized by zl = hl ( zl−1 ) = σl ( W > l zl−1 + bl ) , l = 1 , · · · , L Wl ∈ Rnl−1×nl , bl ∈ Rnl ( 1 ) where σl ( . ) is a monotonic activation function , z0 denotes the input , and zL ∈ RK is the preactivation unnormalized K-dimensional output vector ( nL = K and σL ( . ) as identity operator ) , referred to as the logits . Robust classifiers can be obtained by minimizing the worst-case ( adversarial ) classification loss , formally trained by the following min-max optimization Madry et al . ( 2017 ) minimize θ E ( x , y ) ∼D [ max δ∈∆ ` ( fθ ( x + δ ) , y ) ] . ( 2 ) where θ denotes network parameters , vector fθ ( x ) = zL is the logit output for input x , ` ( . ) is the misclassfication loss , e.g. , ` xent ( . ) defined as the cross-entropy loss , and ∆ denotes the set of permissible perturbations , e.g. , for ` ∞-norm ball of radius giving ∆ : = { δ | ‖δ‖∞ ≤ } . Although augmenting the training set with adversarial inputs , obtained by approximately solving the inner maximization in Eq . 2 , empirically leads to improved adversarial robustness Madry et al . ( 2017 ) ; Shafahi et al . ( 2019 ) ; Wong et al . ( 2019 ) ; Zhang et al . ( 2019 ) , inexact solution of the inner maximization prevents such methods from providing provable guarantees . In critical applications however , provable verification of classification accuracy against a given threat model is crucial . 2.1 PERFORMANCE VERIFICATION AND NETWORK RELAXATION . Given input ( x , y ) , a classification network is considered verifiably robust if all of its perturbed variations , that is x + δ for ∀δ ∈ ∆ , are correctly classified as class y . Such verification can be effectively obtained by p∗i = min zL∈ZL c > y , izL where , ZL : = { zL|zl = hl ( zl−1 ) , l = 1 , ... , L , z0 = x + δ , ∀δ ∈ ∆ } where cy , i = ey − ei for i = 1 , 2 , .. , K , i 6= y , and ei is the standard ith canonical basis vector . If p∗i > 0 ∀i 6= y , then the classifier is verifiably robust at point ( x , y ) as this guarantees that zi ≤ zy ∀i 6= y for all admissible perturbations δ ∈ ∆ . The feasible set ZL is generally nonconvex , rendering obtaining p∗i intractable . Any convex relaxation ofZL however , will provide a lower bound on p∗i , and can be alternatively used for verification . As outlined in Section 1 , various relaxation techniques have been proposed in the literature . Specifically , IBP in ( Mirman et al. , 2018 ; Gowal et al. , 2018 ) proceeds by bounding the activation zl of each layer by propagating an element-wise bounding box using interval arithmetic for networks with monotonic activation functions . Despite its simplicity and relatively small computational complexity ( computational requirements for bound propagation for a given input using IBP is equal to two forward passes of the input ) , it can provide tight bounds once the network is trained accordingly . Specifically , starting from the input layer z0 , it can be bounded for the perturbation class δ ∈ ∆ as z0 = x− 1 and z0 = x + 1 , and zl for the following layers can be bounded as zl = σl ( W > l zl−1 + zl−1 2 − |W > l | zl−1 − zl−1 2 ) , zl = σl ( W > l zl−1 + zl−1 2 + |W > l | zl−1 − zl−1 2 ) , ( 3 ) where | · | is the element-wise absolute-value operator . The verification problem over the relaxed feasible set ẐL : = { zL | zL , i ≤ zL , i ≤ z̄L , i } , where ZL ⊆ ẐL is then easily solved as p∗i = min zL∈ZL c > y , izL ≥ min zL∈ẐL c > y , izL = zL , y − z̄L , i . ( 4 ) 2.2 ROBUST TRAINING OF VERIFIABLE NETWORKS . It has been shown that convex relaxation ofZL can also provide a tractable upper bound on the inner maximization in Eq . 2 . While this holds for various relaxation techniques , focusing on the IBP let us define JIBP , θ ( x , y ) : = [ J IBP 1 , J IBP 2 , ... , J IBP K ] where J IBP i : = min zL∈ẐL c > y , izL ( 5 ) with ( θ , ) implicitly influencing ẐL ( dropped for brevity ) , and upperbound the inner-max in Eq . 2 max δ∈∆ ` xent ( fθ ( x + δ ) , y ) ≤ ` xent ( −JIBP , θ ( x , y ) , y ) , ` xent ( z , c ) : = − log ( exp ( zc ) ∑ i exp ( zi ) ) ( 6 ) By using this tractable upper bound of the robust optimization , network can now be trained by minimize θ ∑ ( x , y ) ∈D ( 1− κ ) ` xent ( −JIBP , θ ( x , y ) , y ) + κγ ` xent ( fθ ( x ) , y ) , ( 7 ) where γ trades natural versus robust accuracy , and κ is scheduled through a ramp-down process to stabilize the training and tightening of IBP Gowal et al . ( 2018 ) ( where γ = 1 is selected therein ) .

Summary: In this paper, the authors propose an additional "abstain/detection" loss term into training, so that the classifier can either robustly classify or detect an adversarial attack. They extend the interval bound propagation method for certified robustness under L_infity perturbations (a simple bounding in changes of the weight after each NN layer application, using a box constraint). Hyperparameters in the objective trade-off between clean and adversarial accuracy.