Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent

Evading adversarial example detection defenses requires finding adversarial examples that must simultaneously ( a ) be misclassified by the model and ( b ) be detected as non-adversarial . We find that existing attacks that attempt to satisfy multiple simultaneous constraints often over-optimize against one constraint at the cost of satisfying another . We introduce Selective Projected Gradient Descent and Orthogonal Projected Gradient Descent , improved attack techniques to generate adversarial examples that avoid this problem by orthogonalizing the gradients when running standard gradient-based attacks . We use our technique to evade four stateof-the-art detection defenses , reducing their accuracy to 0 % while maintaining a 0 % detection rate . 1 INTRODUCTION . Generating adversarial examples ( SZS+14 ; BCM+13 ) , inputs designed by an adversary to cause a neural network to behave incorrectly , is straightforward . By performing input-space gradient descent ( CW17b ; MMS+17 ) , it is possible to maximize the loss of arbitrary examples at test time . This process is both efficient and highly effective . Despite great efforts by the community , attempts at designing defenses against adversarial examples have been largely unsuccessful and gradient-descent attacks continue to circumvent new defenses , even those that attempt to make finding gradients difficult or impossible ( ACW18 ; TCBM20 ) . As a result , many defenses aim to make generating adversarial examples more difficult by requiring additional constraints on inputs for them to be considered successful . Defenses that rely on detection , for example , will reject inputs if a secondary detector model determines the input is adversarial ( MGFB17 ; XEQ17 ) . Turning a benign input x into an adversarial example x′ thus now requires fooling both the original classifier , f , and the detector , g , simultaneously . Traditionally , this is done by constructing a single loss function L that jointly penalizes the loss on f and the loss on g ( CW17a ) , e.g. , by defining L ( x′ ) = L ( f ) +λL ( g ) and then minimizing L ( x′ ) with gradient descent . Unfortunately , many evaluations using this strategy have had limited success—not only must λ be tuned appropriately , but the gradients of f and g must also be well behaved . Our contributions . We develop a new attack technique designed to construct adversarial examples that simultaneously satisfy multiple constraints . Our attack approach is a modification of standard gradient descent ( MMS+17 ) and requires changing just a few lines of code . Given two objective functions f and g , instead of taking gradient descent steps that optimize the joint loss function f +λg , we selectively take gradient descent steps on either f or g. This makes our attack both simpler and easier to analyze than prior attack approaches . We use our technique to evade four state-of-the-art and previously-unbroken defenses to adversarial examples : the Honeypot defense ( CCS ’ 20 ) ( SWW+20 ) , Dense Layer Analysis ( IEEE Euro S & P ’ 20 ) ( SKCB19 ) , Sensitivity Inconsistency Detector ( AAAI ’ 21 ) ( TZLD21 ) , and the SPAM detector presented in Detection by Steganalysis ( CVPR ’ 19 ) ( LZZ+19 ) . In all cases , we successfully reduce the accuracy of the protected classifier to 0 % while maintaining a detection AUC of less than 0.5—meaning the detector performs worse than random guessing . 2 BACKGROUND . 2.1 NOTATION . We consider classification neural networks f : Rd → Rn that receive a d-dimensional input vector ( in this paper , images ) x ∈ Rd and output an n-dimensional prediction vector f ( x ) ∈ Rn . We let g : Rd → R denote some other function which also must be considered , where g ( x ) ≤ 0 when the constraint is satisfied and g ( x ) > 0 if it is violated . Without loss of generality , in a detection defense this function g is the detector and higher values corresponding to higher likelihood of the input being an adversarial example . To denote the true label of x is given by y we write c ( x ) = y . In an abuse of notation , write y = f ( x ) to denote the arg-max most likely label under the model f . 2.2 ADVERSARIAL EXAMPLES . Adversarial examples ( SZS+14 ; BCM+13 ) have been demonstrated in nearly every domain in which neural networks are used . ( ASE+18 ; CW18 ; HPG+17 ) Given an input x corresponding to label c ( x ) and classifier f , an adversarial example is a perturbation x′ of the input such that d ( x , x′ ) < and f ( x′ ) 6= c ( x ) for some metric d. Additionally , an adversarial example can be targeted if , given a target label t 6= c ( x ) we have f ( x′ ) = t with d ( x , x′ ) < . The metric d is most often that induced by a p-norm , typically either || · ||2 or || · ||∞ . With small enough perturbations under these metrics , the adversarial example x′ is not perceptibly different from the original input x. Datasets . We attack each defense on the dataset that it performs best on . All of our defenses operate on images . For three of these defenses , this is the CIFAR-10 dataset ( KH09 ) , and for one , it is the ImageNet dataset ( DDS+09 ) . For each defense we attack , we constrain our adversarial examples to the threat model originally considered to perform a fair re-evaluation , but also generate adversarial examples with standard norms used extensively in prior work in order to make crossdefense evaluations meaningful . We perform all evaluations on a single GPU . Our attacks on CIFAR-10 require just a few minutes , and for ImageNet a few hours ( primarily due to the defense having a throughput of one image per second ) . 2.3 DETECTION DEFENSES . We focus our study on detection defenses . Rather than directly improve the robustness of the model ( MMS+17 ; RSL18 ; LAG+19 ; CRK19 ) , detection defenses classify inputs as adversarial or benign ( MGFB17 ; XEQ17 ) so they can be rejected . While there have been several different strategies attempted to detect adversarial examples in the past ( GSS15 ; MGFB17 ; FCSG17 ; XEQ17 ; MC17 ; MLW+18 ; RKH19 ) , many of these approaches were broken with adaptive attacks that designed new loss functions tailored to each defense ( CW17a ; TCBM20 ) . 2.4 GENERATING ADVERSARIAL EXAMPLES WITH PROJECTED GRADIENT DESCENT . Projected Gradient Descent ( MMS+17 ) is a powerful first-order method for finding such adversarial examples . Given a loss L ( f , x , t ) that takes a classifier , input , and desired target label , we optimize over the constraint set S = { z : d ( x , z ) < } and solve x′ = arg min z∈S L ( f , z , t ) ( 1 ) by taking steps xi+1 = ΠS ( xi − α∇xiL ( f , xi , t ) ) . Here ΠS denotes projection onto the set S , α is the step size , and x0 is randomly initialized ( MMS+17 ) . This paper adapts PGD in order to solve optimization problems which involve minimizing multiple objective functions simultaneously . For notational simplicity , in the remainder of this paper we will omit the projection operator ΠS . Attacks using PGD . Recent work has shown that it is possible to attack models with adaptive attacks that target specific aspects of defenses . For detection defenses this process is often ad hoc , involving alterations specific to each given defense ( TCBM20 ) . An independent line of work develops automated attack techniques that are reliable indicators of robustness ( CH20 ) ; however , in general , these attack approaches are difficult to apply to detection defenses . 3 RETHINKING ADVERSARIAL EXAMPLE DETECTION . Before we develop our improved attack technique to break adversarial example detectors , it will be useful to understand why evaluating adversarial example detectors is more difficult than evaluating standard classifiers . Early work on adversarial examples often set up the problem slightly differently than we do above in Equation 1 . The initial formulation of an adversarial example ( SZS+14 ; CW17b ) asks for the smallest perturbation δ such that f ( x+ δ ) is misclassified . In the targeted case , this means solving arg min ‖δ‖2 such that f ( x+ δ ) = t for a given target t that is not equal to the original correct label for x . Solving this problem as stated is intractable . It requires searching over a nonlinear constraint set , which is not feasible for standard gradient descent . As a result , detection papers typically ( SWW+20 ; SKCB19 ) reformulate the search with a Lagrangian relaxation arg min ‖δ‖2 + λL ( f , x+ δ , t ) ( 2 ) This formulation is simpler , but still ( a ) requires tuning λ to work well , and ( b ) is only guaranteed to be correct for convex functions L—that it works for non-convex models like deep neural networks is not theoretically justified . It additionally requires carefully constructing loss functions L ( CW17b ) . Equation 1 simplifies the setup considerably by exchanging the constraint and objective . Whereas in Equation 2 we search for the smallest perturbation that results in misclassification , Equation 1 instead finds an input x + δ with δ < that minimizes the classifier ’ s loss . This is a simpler formulation because now the constraint is convex , and so we can run standard gradient descent optimization . Evading detection defenses is difficult because there are now two non-linear constraints . Not only must the input be constrained by a distortion bound and be misclassified by the base classifier , but we must also have that they are not detected , i.e. , with g ( x ) < 0 . This new requirement is nonlinear , and now it becomes impossible to side-step the problem by merely swapping the objective and the constraint as we did before : there will always be at least one constraint that is a non-linear function , and so standard gradient descent techniques can not directly apply . In order to resolve this difficulty , the existing literature applies the same Lagrangian relaxation as was previously applied to constructing minimum-distortion adversarial examples . That is , breaking a detection scheme involves solving arg min x∈S L ( f , x , t ) + λg ( x ) ( 3 ) where λ is a hyperparameter that controls the relative importance of fooling the classifier versus fooling the detector . However , this formulation again brings back all of the reasons why the community moved past minimum-distortion adversarial examples . 3.1 A MOTIVATING EXAMPLE . Let f ( ~x ) = exp ( −1 ) − exp ( −‖~x − ~e‖22 ) − ε and g ( ~x ) = exp ( −1 ) − exp ( −‖~x + ~e‖22 ) − ε where ~e ∈ RN and ‖~e‖ = 1 , as visualized in Figure 1 . By setting ε to a small constant , the only solution that satisfies both f ( ~x ) < 0 and g ( ~x ) < 0 can be made arbitrarily close to the origin ~x = ~0 . However , no standard Lagrangian formulation will be able to find this solution . Consider the sum h ( ~x ; λ ) = f ( ~x ) + λg ( ~x ) ; then we can show that for all λ we will have arg min~xh ( ~x ; λ ) 6= 0 . To see this , observe that while it is possible for the gradient ∇h ( ~0 ; λ ) = 0 ( one of the conditions for a value to be a local minima ) , the loss surface is always “ concave down ” at the origin . It will always be possible to move slightly closer to ~e or −~e and decrease the loss . Therefore , minimizing h ( x ) will never be able to find a valid stable solution to this extremely simple problem , as it will always collapse to finding a solution of either ~e or −~e , which only satisfies one of the two equations .

Summary: The authors propose an attack that could break 4 adversarial detection methods published recently. Traditionally, attacks against detection methods have attempted to maximize the loss for both classification and detection simultaneously. However, using a toy example the authors show that this is suboptimal, as it may not find the worst-case adversaries. The authors propose to minimize the loss (targeted attack setting) iteratively by optimizing either only for the classification pipeline or the detection pipeline at a time.  The attack first considers the classification loss and further tries to fool the detection pipeline until the classification prediction remains incorrect. The authors also propose a variant of the attack by considering gradient steps for the classification pipeline to be orthogonal to the gradients of the detection pipeline and vice versa. Finally, the paper shows that these two proposed attacks completely circumvent four recent adversarial detection methods.