Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent

Evading adversarial example detection defenses requires finding adversarial examples that must simultaneously ( a ) be misclassified by the model and ( b ) be detected as non-adversarial . We find that existing attacks that attempt to satisfy multiple simultaneous constraints often over-optimize against one constraint at the cost of satisfying another . We introduce Selective Projected Gradient Descent and Orthogonal Projected Gradient Descent , improved attack techniques to generate adversarial examples that avoid this problem by orthogonalizing the gradients when running standard gradient-based attacks . We use our technique to evade four stateof-the-art detection defenses , reducing their accuracy to 0 % while maintaining a 0 % detection rate . 1 INTRODUCTION . Generating adversarial examples ( SZS+14 ; BCM+13 ) , inputs designed by an adversary to cause a neural network to behave incorrectly , is straightforward . By performing input-space gradient descent ( CW17b ; MMS+17 ) , it is possible to maximize the loss of arbitrary examples at test time . This process is both efficient and highly effective . Despite great efforts by the community , attempts at designing defenses against adversarial examples have been largely unsuccessful and gradient-descent attacks continue to circumvent new defenses , even those that attempt to make finding gradients difficult or impossible ( ACW18 ; TCBM20 ) . As a result , many defenses aim to make generating adversarial examples more difficult by requiring additional constraints on inputs for them to be considered successful . Defenses that rely on detection , for example , will reject inputs if a secondary detector model determines the input is adversarial ( MGFB17 ; XEQ17 ) . Turning a benign input x into an adversarial example x′ thus now requires fooling both the original classifier , f , and the detector , g , simultaneously . Traditionally , this is done by constructing a single loss function L that jointly penalizes the loss on f and the loss on g ( CW17a ) , e.g. , by defining L ( x′ ) = L ( f ) +λL ( g ) and then minimizing L ( x′ ) with gradient descent . Unfortunately , many evaluations using this strategy have had limited success—not only must λ be tuned appropriately , but the gradients of f and g must also be well behaved . Our contributions . We develop a new attack technique designed to construct adversarial examples that simultaneously satisfy multiple constraints . Our attack approach is a modification of standard gradient descent ( MMS+17 ) and requires changing just a few lines of code . Given two objective functions f and g , instead of taking gradient descent steps that optimize the joint loss function f +λg , we selectively take gradient descent steps on either f or g. This makes our attack both simpler and easier to analyze than prior attack approaches . We use our technique to evade four state-of-the-art and previously-unbroken defenses to adversarial examples : the Honeypot defense ( CCS ’ 20 ) ( SWW+20 ) , Dense Layer Analysis ( IEEE Euro S & P ’ 20 ) ( SKCB19 ) , Sensitivity Inconsistency Detector ( AAAI ’ 21 ) ( TZLD21 ) , and the SPAM detector presented in Detection by Steganalysis ( CVPR ’ 19 ) ( LZZ+19 ) . In all cases , we successfully reduce the accuracy of the protected classifier to 0 % while maintaining a detection AUC of less than 0.5—meaning the detector performs worse than random guessing . 2 BACKGROUND . 2.1 NOTATION . We consider classification neural networks f : Rd → Rn that receive a d-dimensional input vector ( in this paper , images ) x ∈ Rd and output an n-dimensional prediction vector f ( x ) ∈ Rn . We let g : Rd → R denote some other function which also must be considered , where g ( x ) ≤ 0 when the constraint is satisfied and g ( x ) > 0 if it is violated . Without loss of generality , in a detection defense this function g is the detector and higher values corresponding to higher likelihood of the input being an adversarial example . To denote the true label of x is given by y we write c ( x ) = y . In an abuse of notation , write y = f ( x ) to denote the arg-max most likely label under the model f . 2.2 ADVERSARIAL EXAMPLES . Adversarial examples ( SZS+14 ; BCM+13 ) have been demonstrated in nearly every domain in which neural networks are used . ( ASE+18 ; CW18 ; HPG+17 ) Given an input x corresponding to label c ( x ) and classifier f , an adversarial example is a perturbation x′ of the input such that d ( x , x′ ) < and f ( x′ ) 6= c ( x ) for some metric d. Additionally , an adversarial example can be targeted if , given a target label t 6= c ( x ) we have f ( x′ ) = t with d ( x , x′ ) < . The metric d is most often that induced by a p-norm , typically either || · ||2 or || · ||∞ . With small enough perturbations under these metrics , the adversarial example x′ is not perceptibly different from the original input x. Datasets . We attack each defense on the dataset that it performs best on . All of our defenses operate on images . For three of these defenses , this is the CIFAR-10 dataset ( KH09 ) , and for one , it is the ImageNet dataset ( DDS+09 ) . For each defense we attack , we constrain our adversarial examples to the threat model originally considered to perform a fair re-evaluation , but also generate adversarial examples with standard norms used extensively in prior work in order to make crossdefense evaluations meaningful . We perform all evaluations on a single GPU . Our attacks on CIFAR-10 require just a few minutes , and for ImageNet a few hours ( primarily due to the defense having a throughput of one image per second ) . 2.3 DETECTION DEFENSES . We focus our study on detection defenses . Rather than directly improve the robustness of the model ( MMS+17 ; RSL18 ; LAG+19 ; CRK19 ) , detection defenses classify inputs as adversarial or benign ( MGFB17 ; XEQ17 ) so they can be rejected . While there have been several different strategies attempted to detect adversarial examples in the past ( GSS15 ; MGFB17 ; FCSG17 ; XEQ17 ; MC17 ; MLW+18 ; RKH19 ) , many of these approaches were broken with adaptive attacks that designed new loss functions tailored to each defense ( CW17a ; TCBM20 ) . 2.4 GENERATING ADVERSARIAL EXAMPLES WITH PROJECTED GRADIENT DESCENT . Projected Gradient Descent ( MMS+17 ) is a powerful first-order method for finding such adversarial examples . Given a loss L ( f , x , t ) that takes a classifier , input , and desired target label , we optimize over the constraint set S = { z : d ( x , z ) < } and solve x′ = arg min z∈S L ( f , z , t ) ( 1 ) by taking steps xi+1 = ΠS ( xi − α∇xiL ( f , xi , t ) ) . Here ΠS denotes projection onto the set S , α is the step size , and x0 is randomly initialized ( MMS+17 ) . This paper adapts PGD in order to solve optimization problems which involve minimizing multiple objective functions simultaneously . For notational simplicity , in the remainder of this paper we will omit the projection operator ΠS . Attacks using PGD . Recent work has shown that it is possible to attack models with adaptive attacks that target specific aspects of defenses . For detection defenses this process is often ad hoc , involving alterations specific to each given defense ( TCBM20 ) . An independent line of work develops automated attack techniques that are reliable indicators of robustness ( CH20 ) ; however , in general , these attack approaches are difficult to apply to detection defenses . 3 RETHINKING ADVERSARIAL EXAMPLE DETECTION . Before we develop our improved attack technique to break adversarial example detectors , it will be useful to understand why evaluating adversarial example detectors is more difficult than evaluating standard classifiers . Early work on adversarial examples often set up the problem slightly differently than we do above in Equation 1 . The initial formulation of an adversarial example ( SZS+14 ; CW17b ) asks for the smallest perturbation δ such that f ( x+ δ ) is misclassified . In the targeted case , this means solving arg min ‖δ‖2 such that f ( x+ δ ) = t for a given target t that is not equal to the original correct label for x . Solving this problem as stated is intractable . It requires searching over a nonlinear constraint set , which is not feasible for standard gradient descent . As a result , detection papers typically ( SWW+20 ; SKCB19 ) reformulate the search with a Lagrangian relaxation arg min ‖δ‖2 + λL ( f , x+ δ , t ) ( 2 ) This formulation is simpler , but still ( a ) requires tuning λ to work well , and ( b ) is only guaranteed to be correct for convex functions L—that it works for non-convex models like deep neural networks is not theoretically justified . It additionally requires carefully constructing loss functions L ( CW17b ) . Equation 1 simplifies the setup considerably by exchanging the constraint and objective . Whereas in Equation 2 we search for the smallest perturbation that results in misclassification , Equation 1 instead finds an input x + δ with δ < that minimizes the classifier ’ s loss . This is a simpler formulation because now the constraint is convex , and so we can run standard gradient descent optimization . Evading detection defenses is difficult because there are now two non-linear constraints . Not only must the input be constrained by a distortion bound and be misclassified by the base classifier , but we must also have that they are not detected , i.e. , with g ( x ) < 0 . This new requirement is nonlinear , and now it becomes impossible to side-step the problem by merely swapping the objective and the constraint as we did before : there will always be at least one constraint that is a non-linear function , and so standard gradient descent techniques can not directly apply . In order to resolve this difficulty , the existing literature applies the same Lagrangian relaxation as was previously applied to constructing minimum-distortion adversarial examples . That is , breaking a detection scheme involves solving arg min x∈S L ( f , x , t ) + λg ( x ) ( 3 ) where λ is a hyperparameter that controls the relative importance of fooling the classifier versus fooling the detector . However , this formulation again brings back all of the reasons why the community moved past minimum-distortion adversarial examples . 3.1 A MOTIVATING EXAMPLE . Let f ( ~x ) = exp ( −1 ) − exp ( −‖~x − ~e‖22 ) − ε and g ( ~x ) = exp ( −1 ) − exp ( −‖~x + ~e‖22 ) − ε where ~e ∈ RN and ‖~e‖ = 1 , as visualized in Figure 1 . By setting ε to a small constant , the only solution that satisfies both f ( ~x ) < 0 and g ( ~x ) < 0 can be made arbitrarily close to the origin ~x = ~0 . However , no standard Lagrangian formulation will be able to find this solution . Consider the sum h ( ~x ; λ ) = f ( ~x ) + λg ( ~x ) ; then we can show that for all λ we will have arg min~xh ( ~x ; λ ) 6= 0 . To see this , observe that while it is possible for the gradient ∇h ( ~0 ; λ ) = 0 ( one of the conditions for a value to be a local minima ) , the loss surface is always “ concave down ” at the origin . It will always be possible to move slightly closer to ~e or −~e and decrease the loss . Therefore , minimizing h ( x ) will never be able to find a valid stable solution to this extremely simple problem , as it will always collapse to finding a solution of either ~e or −~e , which only satisfies one of the two equations .

Summary: This paper proposes two techniques for generating adversarial examples: Selective Projected Gradient Descent (SPGD) and Orthogonal Projected Gradient Descent (OPGD). In order to fool both the victim model $f$ and a detector $g$, SPGD selectively optimise either $f$ or $g$ depending on whether the modified input is misclassified as the target class, while OPGD further orthogonalizes the gradients. Evaluation on four previously unbroken, state-of-the-art defence methods demonstrate the effectiveness of the proposed attacks.