Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraints

1 Introduction . Learning algorithms are vulnerable to adversarial examples , i.e. , intentionally-perturbed inputs aimed to mislead classification at test time [ 24 , 3 ] . While adversarial examples have received much attention , evaluating the robustness of deep networks against them remains a challenge . Adversarial attacks solve a non-convex optimization problem and are thus prone to finding suboptimal solutions ; in particular , all attacks make certain assumptions about the underlying geometry and properties of the optimization problem which , if violated , can derail the attack and may lead to premature conclusions regarding model robustness . That is why the vast majority of defenses published in recent years have later shown to be ineffective against more powerful white-box attacks [ 5 , 1 ] . Having an arsenal of diverse attacks that can be adapted to specific defenses is one of the most promising avenues for increasing confidence in white-box robustness evaluations [ 6 , 25 ] . While it may seem that the number of attacks is already large , most of them are just small variations of the same technique , make similar underlying assumptions and thus tend to fail jointly ( see , e.g. , [ 25 ] , in which projected-gradient attacks all fail similarly against the “ Ensemble Diversity ” defense ) . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this work , we focus on minimum-norm attacks for evaluating adversarial robustness , i.e. , attacks that aim to mislead classification by finding the smallest input perturbation according to a given norm . In contrast to maximum-confidence attacks , which maximize confidence in a wrong class within a given perturbation budget , the former are better suited to evaluate adversarial robustness as one can compute the accuracy of a classifier under attack for any perturbation budget without re-running the attack . Within the class of gradient-based minimum-norm attacks , there are three main sub-categories : ( i ) soft-constraint attacks , ( ii ) boundary attacks and ( iii ) projected-gradient attacks . Soft-constraint attacks like CW [ 5 ] optimize a trade-off between confidence of the misclassified samples and perturbation size . This class of attacks needs a sample-wise tuning of the trade-off hyperparameter to find the smallest possible perturbation , thus requiring many steps to converge . Boundary attacks like BB [ 4 ] and FAB [ 10 ] move along the decision boundary towards the closest point to the input sample . These attacks converge within relatively few steps , but BB requires an adversarial starting point , and both attacks need to solve a relatively expensive optimization problem in each step . Finally , recent minimum-norm projected-gradient attacks like DDN [ 23 ] perform a maximum-confidence attack in each step under a given perturbation budget , while iteratively adjusting to reduce the perturbation size . DDN combines the effectiveness of boundary attacks with the simplicity and per-step speed of soft-constraint attacks ; however , it is specific to the ` 2 norm and can not be readily extended to other norms . To overcome the aforementioned limitations , in this work we propose a novel , fast minimum-norm ( FMN ) attack ( Sect . 2 ) , which retains the main advantages of DDN while generalizing it to different ` p norms ( p = 0 , 1 , 2 , ∞ ) . We perform large-scale experiments on different datasets and models ( Sect . 3 ) , showing that FMN is able to significantly outperform current minimum-norm attacks in terms of convergence speed and computation time ( except for ` 2-norm attacks , for which FMN achieves comparable results ) , while finding equal or better optima , on average , across almost all tested scenarios and ` p norms . FMN thus combines all desirable traits a good adversarial attack should have , providing an important step towards improving adversarial robustness evaluations . We conclude the paper by discussing related work ( Sect . 4 ) and future research directions ( Sect . 5 ) . 2 Minimum-Norm Adversarial Examples with Adaptive Projections . Problem formulation . Given an input sample x ∈ [ 0 , 1 ] d , belonging to class y ∈ { 1 , . . . , c } , the goal of an untargeted attack is to find the minimum-norm perturbation δ ? such that the corresponding adversarial example x ? = x+ δ ? is misclassified . This problem can be formulated as : δ ? ∈ arg min δ ‖δ‖p , ( 1 ) s.t . L ( x+ δ , y , θ ) < 0 , ( 2 ) x+ δ ∈ [ 0 , 1 ] d , ( 3 ) Algorithm 1 Fast Minimum-norm ( FMN ) Attack Input : x , the input sample ; t , a variable denoting whether the attack is targeted ( t = +1 ) or untargeted ( t = −1 ) ; y , the target ( true ) class label if the attack is targeted ( untargeted ) ; γ0 and γK , the initial and final -step sizes ; α0 and αK , the initial and final δ-step sizes ; K , the total number of iterations . Output : The minimum-norm adversarial example x ? . 1 : x0 ← x , 0 = 0 , δ0 ← 0 , δ ? ←∞ 2 : for k = 1 , . . . , K do 3 : g ← t · ∇δL ( xk−1 + δ , y , θ ) // loss gradient 4 : γk ← h ( γ0 , γK , k , K ) // -step size decay ( Eq . 6 ) 5 : if L ( xk−1 , y , θ ) ≥ 0 then 6 : k = ‖δk−1‖p + L ( xk−1 , y , θ ) /‖g‖q if adversarial not found yet else k = k−1 ( 1 + γk ) 7 : else 8 : if ‖δk−1‖p ≤ ‖δ ? ‖p then 9 : δ ? ← δk−1 // update best min-norm solution 10 : end if 11 : k = min ( k−1 ( 1− γk ) , ‖δ ? ‖p ) 12 : end if 13 : αk ← h ( α0 , αK , k , K ) // δ-step size decay ( Eq . 6 ) 14 : δk ← δk−1 + αk · g/‖g‖2 // gradient-scaling step 15 : δk ← Π ( x0 + δk ) − x0 16 : δk ← clip ( x0 + δk ) − x0 17 : xk ← x0 + δk 18 : end for 19 : return x ? ← x0 + δ ? where || · ||p indicates the ` p-norm operator . The loss L in the constraint in Eq . ( 2 ) is defined as : L ( x , y , θ ) = fy ( x , θ ) −max j 6=y fj ( x , θ ) , ( 4 ) where fj ( x , θ ) is the confidence given by the model f for classifying x as class j , and θ is the set of its learned parameters . Assuming that the classifier assigns x to the class exhibiting the highest confidence , i.e. , y ? = arg maxj∈1 , ... , c fj ( x , θ ) , the loss function L ( x , y , θ ) takes on negative values only when x is misclassified . Finally , the box constraint in Eq . ( 3 ) ensures that the perturbed sample x+ δ lies in the feasible input space . The aforementioned problem typically involves a non-convex loss function L ( w.r.t . its first argument ) , due to the non-convexity of the underlying decision function f . For this reason , it may admit different locally-optimal solutions . Note also that the solution is trivial ( i.e. , δ ? = 0 ) when the input sample x is already adversarial ( i.e. , L ( x , y , θ ) < 0 ) . Extension to the targeted case . The goal of a targeted attack is to have the input sample misclassified in a given target class y′ . This can be accounted for by modifying the loss function in Eq . ( 4 ) as Lt ( x , y′ , θ ) = maxj 6=y′ fj ( x , θ ) − fy′ ( x , θ ) = −L ( x , y′ , θ ) , i.e. , changing its sign and using the target class label y′ instead of the true class label y . Solution algorithm . To solve Problem ( 1 ) - ( 3 ) , we reformulate it using an upper bound on ‖δ‖p : min , δ , s.t . ‖δ‖p ≤ , ( 5 ) and to the constraints in Eqs . ( 2 ) - ( 3 ) . This allows us to derive an algorithm that works in two main steps , similarly to DDN [ 23 ] , by updating the maximum perturbation size separately from the actual perturbation δ , as represented in Fig . 1 ( a ) . In particular , the constraint size is adapted to reduce the distance of the constraint to the boundary ( -step ) , while the perturbation δ is updated using a projected-gradient step to minimize the loss function L within the given -sized constraint ( δ-step ) . This essentially amounts to a projected gradient descent algorithm that iteratively adapts the constraint size to find the minimum-norm adversarial example . The complete algorithm is given as Algorithm 1 , while a more detailed explanation of the two aforementioned steps is given below . -step . This step updates the upper bound on the perturbation norm ( lines 4-12 in Algorithm 1 ) . The underlying idea is to increase if the current sample is not adversarial ( i.e. , L ( xk−1 , y , θ ) ≥ 0 ) , and to decrease it otherwise , while reducing the step size to dampen oscillations around the boundary and reach convergence . In the former case ( -increase ) , the increment of depends on whether an adversarial example has been previously found or not . If not , we estimate the distance to the boundary with a first-order linear approximation , and set k = ‖δk−1‖p + L ( xk−1 , y , θ ) /||∇L ( xk−1 , y , θ ) ||q , where q is the dual norm of p. This approximation allows the attack point to make faster progress towards the decision boundary . Conversely , if an adversarial sample has been previously found , but the current sample is not adversarial , it is likely that the current estimate of is only slightly smaller than the minimum-norm solution . We thus increase by a small fraction as k = k−1 ( 1 + γk ) , being γk a decaying step size . In the latter case ( -decrease ) , if the current sample is adversarial , i.e. , L ( xk−1 , y , θ ) < 0 , we decrease as k = k−1 ( 1− γk ) , to check whether the current solution can be improved . If the corresponding k value is larger than the optimal ‖δ ? ‖p found so far , we retain the best value and set k = ‖δ ? ‖p . These multiplicative updates of exhibit an oscillating behavior around the decision boundary , due to the conflicting requirements of minimizing the perturbation size and finding an adversarial point . To ensure convergence , as anticipated before , the step size γk is decayed with cosine annealing : γk = h ( γ0 , γK , k , K ) = γK + 1 2 ( γ0 − γK ) ( 1 + cos ( kπ K ) ) , ( 6 ) being k the current step , K the total number of steps , and γ0 and γK the initial and final step sizes . δ-step . This step updates δ ( lines 13-17 in Algorithm 1 ) . The goal is to find the adversarial example that is misclassified with maximum confidence ( i.e. , for which L is minimized ) within the current -sized constraint ( Eq . 5 ) and bounds ( Eq . 3 ) . This amounts to performing a projected-gradient step along the negative gradient of L. We consider a normalized steepest descent with decaying step size α to overcome potential issues related to noisy gradients while ensuring convergence ( line 14 ) . Note that this step only rescales the gradient by its ` 2 norm , while preserving its direction . The step size α is decayed using cosine annealing ( Eq . 6 ) . Once δ is updated , we project it onto the given -sized ` p-norm constraint via a projection operator Π ( line 15 ) , to fulfill the constraint in Eq . ( 5 ) . The projection is trivial for p =∞ and p = 2 . For p = 1 , we use the efficient algorithm by Duchi et al . [ 14 ] . For p = 0 , we retain only the first components of δ exhibiting the largest absolute value . We finally clip the components of δ that violate the bounds in Eq . ( 3 ) ( line 16 ) . Execution example . In Fig . 1 ( b ) , we report an example of execution of our algorithm on a bidimensional problem . The initial sample is updated to follow the negative gradient of L towards the decision boundary . When an adversarial point is found , the algorithm reduces to find a better solution . The point is thus projected back onto the non-adversarial region , and increased ( by a smaller , decaying amount ) . These oscillations allow the point to walk on the boundary towards a local optimum , i.e. , an adversarial point lying on the boundary , where the gradient of the loss function and that of the norm constraint have opposite direction . FMN tends to quickly converge to a good local optimum , provided that the step size is reduced to a sufficiently-small value and that a sufficiently-large number of iterations are performed . This is also confirmed empirically in Sect . 3 . Adversarial initialization . Our attack can be initialized from the input sample x , or from a point xinit belonging either to a different class ( if the attack is untargeted ) or to the target class ( if the attack is targeted ) . When initializing the attack from xinit , we perform a 10-step binary search between x and xinit , to find an adversarial point which is closer to the decision boundary . In particular , we aim to find the minimum such that L ( x + Π ( xinit − x ) , y , θ ) < 0 ( or Lt < 0 for targeted attacks ) . Then we run our attack starting from the corresponding values of xk , k , δk and δ ? . Differences with DDN . FMN applies substantial changes to both the algorithm and the formulation of DDN . The main difference is that ( i ) DDN always rescales the perturbation to have size . This operation is problematic when using other norms , especially sparse ones , as it hinders the ability of the attack to explore the neighboring space and find a suitable descent direction . Another difference is that ( ii ) FMN does not use the cross-entropy loss , but it uses the logit difference as the loss function L , since the latter is less affected by saturation effects . Moreover , ( iii ) FMN does not need an initial value for , as is dynamically estimated ; and ( iv ) γ is decayed to improve convergence around better minimum-norm solutions , by more effectively dampening oscillations around the boundary . Finally , we include the possibility of ( v ) initializing the attack from an adversarial point , which can greatly increase the convergence speed of the algorithm , as it uses a fast line-search algorithm to find the boundary and the remaining queries to refine the result .

Summary: This paper proposes a new adversarial attack algorithm for finding the smallest perturbation necessary to fool a classifier under various $\ell_p$ norms. The algorithm is similar to DDN, but includes a few improvements, including using the logit margin loss instead of cross-entropy, using a more complex algorithm for updating the bound $\epsilon$, and not always projecting the adversarial example to the boundary. They evaluate the new attack, FMN, on a variety of classifiers on MNIST and CIFAR-10.