Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraints

1 Introduction . Learning algorithms are vulnerable to adversarial examples , i.e. , intentionally-perturbed inputs aimed to mislead classification at test time [ 24 , 3 ] . While adversarial examples have received much attention , evaluating the robustness of deep networks against them remains a challenge . Adversarial attacks solve a non-convex optimization problem and are thus prone to finding suboptimal solutions ; in particular , all attacks make certain assumptions about the underlying geometry and properties of the optimization problem which , if violated , can derail the attack and may lead to premature conclusions regarding model robustness . That is why the vast majority of defenses published in recent years have later shown to be ineffective against more powerful white-box attacks [ 5 , 1 ] . Having an arsenal of diverse attacks that can be adapted to specific defenses is one of the most promising avenues for increasing confidence in white-box robustness evaluations [ 6 , 25 ] . While it may seem that the number of attacks is already large , most of them are just small variations of the same technique , make similar underlying assumptions and thus tend to fail jointly ( see , e.g. , [ 25 ] , in which projected-gradient attacks all fail similarly against the “ Ensemble Diversity ” defense ) . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this work , we focus on minimum-norm attacks for evaluating adversarial robustness , i.e. , attacks that aim to mislead classification by finding the smallest input perturbation according to a given norm . In contrast to maximum-confidence attacks , which maximize confidence in a wrong class within a given perturbation budget , the former are better suited to evaluate adversarial robustness as one can compute the accuracy of a classifier under attack for any perturbation budget without re-running the attack . Within the class of gradient-based minimum-norm attacks , there are three main sub-categories : ( i ) soft-constraint attacks , ( ii ) boundary attacks and ( iii ) projected-gradient attacks . Soft-constraint attacks like CW [ 5 ] optimize a trade-off between confidence of the misclassified samples and perturbation size . This class of attacks needs a sample-wise tuning of the trade-off hyperparameter to find the smallest possible perturbation , thus requiring many steps to converge . Boundary attacks like BB [ 4 ] and FAB [ 10 ] move along the decision boundary towards the closest point to the input sample . These attacks converge within relatively few steps , but BB requires an adversarial starting point , and both attacks need to solve a relatively expensive optimization problem in each step . Finally , recent minimum-norm projected-gradient attacks like DDN [ 23 ] perform a maximum-confidence attack in each step under a given perturbation budget , while iteratively adjusting to reduce the perturbation size . DDN combines the effectiveness of boundary attacks with the simplicity and per-step speed of soft-constraint attacks ; however , it is specific to the ` 2 norm and can not be readily extended to other norms . To overcome the aforementioned limitations , in this work we propose a novel , fast minimum-norm ( FMN ) attack ( Sect . 2 ) , which retains the main advantages of DDN while generalizing it to different ` p norms ( p = 0 , 1 , 2 , ∞ ) . We perform large-scale experiments on different datasets and models ( Sect . 3 ) , showing that FMN is able to significantly outperform current minimum-norm attacks in terms of convergence speed and computation time ( except for ` 2-norm attacks , for which FMN achieves comparable results ) , while finding equal or better optima , on average , across almost all tested scenarios and ` p norms . FMN thus combines all desirable traits a good adversarial attack should have , providing an important step towards improving adversarial robustness evaluations . We conclude the paper by discussing related work ( Sect . 4 ) and future research directions ( Sect . 5 ) . 2 Minimum-Norm Adversarial Examples with Adaptive Projections . Problem formulation . Given an input sample x ∈ [ 0 , 1 ] d , belonging to class y ∈ { 1 , . . . , c } , the goal of an untargeted attack is to find the minimum-norm perturbation δ ? such that the corresponding adversarial example x ? = x+ δ ? is misclassified . This problem can be formulated as : δ ? ∈ arg min δ ‖δ‖p , ( 1 ) s.t . L ( x+ δ , y , θ ) < 0 , ( 2 ) x+ δ ∈ [ 0 , 1 ] d , ( 3 ) Algorithm 1 Fast Minimum-norm ( FMN ) Attack Input : x , the input sample ; t , a variable denoting whether the attack is targeted ( t = +1 ) or untargeted ( t = −1 ) ; y , the target ( true ) class label if the attack is targeted ( untargeted ) ; γ0 and γK , the initial and final -step sizes ; α0 and αK , the initial and final δ-step sizes ; K , the total number of iterations . Output : The minimum-norm adversarial example x ? . 1 : x0 ← x , 0 = 0 , δ0 ← 0 , δ ? ←∞ 2 : for k = 1 , . . . , K do 3 : g ← t · ∇δL ( xk−1 + δ , y , θ ) // loss gradient 4 : γk ← h ( γ0 , γK , k , K ) // -step size decay ( Eq . 6 ) 5 : if L ( xk−1 , y , θ ) ≥ 0 then 6 : k = ‖δk−1‖p + L ( xk−1 , y , θ ) /‖g‖q if adversarial not found yet else k = k−1 ( 1 + γk ) 7 : else 8 : if ‖δk−1‖p ≤ ‖δ ? ‖p then 9 : δ ? ← δk−1 // update best min-norm solution 10 : end if 11 : k = min ( k−1 ( 1− γk ) , ‖δ ? ‖p ) 12 : end if 13 : αk ← h ( α0 , αK , k , K ) // δ-step size decay ( Eq . 6 ) 14 : δk ← δk−1 + αk · g/‖g‖2 // gradient-scaling step 15 : δk ← Π ( x0 + δk ) − x0 16 : δk ← clip ( x0 + δk ) − x0 17 : xk ← x0 + δk 18 : end for 19 : return x ? ← x0 + δ ? where || · ||p indicates the ` p-norm operator . The loss L in the constraint in Eq . ( 2 ) is defined as : L ( x , y , θ ) = fy ( x , θ ) −max j 6=y fj ( x , θ ) , ( 4 ) where fj ( x , θ ) is the confidence given by the model f for classifying x as class j , and θ is the set of its learned parameters . Assuming that the classifier assigns x to the class exhibiting the highest confidence , i.e. , y ? = arg maxj∈1 , ... , c fj ( x , θ ) , the loss function L ( x , y , θ ) takes on negative values only when x is misclassified . Finally , the box constraint in Eq . ( 3 ) ensures that the perturbed sample x+ δ lies in the feasible input space . The aforementioned problem typically involves a non-convex loss function L ( w.r.t . its first argument ) , due to the non-convexity of the underlying decision function f . For this reason , it may admit different locally-optimal solutions . Note also that the solution is trivial ( i.e. , δ ? = 0 ) when the input sample x is already adversarial ( i.e. , L ( x , y , θ ) < 0 ) . Extension to the targeted case . The goal of a targeted attack is to have the input sample misclassified in a given target class y′ . This can be accounted for by modifying the loss function in Eq . ( 4 ) as Lt ( x , y′ , θ ) = maxj 6=y′ fj ( x , θ ) − fy′ ( x , θ ) = −L ( x , y′ , θ ) , i.e. , changing its sign and using the target class label y′ instead of the true class label y . Solution algorithm . To solve Problem ( 1 ) - ( 3 ) , we reformulate it using an upper bound on ‖δ‖p : min , δ , s.t . ‖δ‖p ≤ , ( 5 ) and to the constraints in Eqs . ( 2 ) - ( 3 ) . This allows us to derive an algorithm that works in two main steps , similarly to DDN [ 23 ] , by updating the maximum perturbation size separately from the actual perturbation δ , as represented in Fig . 1 ( a ) . In particular , the constraint size is adapted to reduce the distance of the constraint to the boundary ( -step ) , while the perturbation δ is updated using a projected-gradient step to minimize the loss function L within the given -sized constraint ( δ-step ) . This essentially amounts to a projected gradient descent algorithm that iteratively adapts the constraint size to find the minimum-norm adversarial example . The complete algorithm is given as Algorithm 1 , while a more detailed explanation of the two aforementioned steps is given below . -step . This step updates the upper bound on the perturbation norm ( lines 4-12 in Algorithm 1 ) . The underlying idea is to increase if the current sample is not adversarial ( i.e. , L ( xk−1 , y , θ ) ≥ 0 ) , and to decrease it otherwise , while reducing the step size to dampen oscillations around the boundary and reach convergence . In the former case ( -increase ) , the increment of depends on whether an adversarial example has been previously found or not . If not , we estimate the distance to the boundary with a first-order linear approximation , and set k = ‖δk−1‖p + L ( xk−1 , y , θ ) /||∇L ( xk−1 , y , θ ) ||q , where q is the dual norm of p. This approximation allows the attack point to make faster progress towards the decision boundary . Conversely , if an adversarial sample has been previously found , but the current sample is not adversarial , it is likely that the current estimate of is only slightly smaller than the minimum-norm solution . We thus increase by a small fraction as k = k−1 ( 1 + γk ) , being γk a decaying step size . In the latter case ( -decrease ) , if the current sample is adversarial , i.e. , L ( xk−1 , y , θ ) < 0 , we decrease as k = k−1 ( 1− γk ) , to check whether the current solution can be improved . If the corresponding k value is larger than the optimal ‖δ ? ‖p found so far , we retain the best value and set k = ‖δ ? ‖p . These multiplicative updates of exhibit an oscillating behavior around the decision boundary , due to the conflicting requirements of minimizing the perturbation size and finding an adversarial point . To ensure convergence , as anticipated before , the step size γk is decayed with cosine annealing : γk = h ( γ0 , γK , k , K ) = γK + 1 2 ( γ0 − γK ) ( 1 + cos ( kπ K ) ) , ( 6 ) being k the current step , K the total number of steps , and γ0 and γK the initial and final step sizes . δ-step . This step updates δ ( lines 13-17 in Algorithm 1 ) . The goal is to find the adversarial example that is misclassified with maximum confidence ( i.e. , for which L is minimized ) within the current -sized constraint ( Eq . 5 ) and bounds ( Eq . 3 ) . This amounts to performing a projected-gradient step along the negative gradient of L. We consider a normalized steepest descent with decaying step size α to overcome potential issues related to noisy gradients while ensuring convergence ( line 14 ) . Note that this step only rescales the gradient by its ` 2 norm , while preserving its direction . The step size α is decayed using cosine annealing ( Eq . 6 ) . Once δ is updated , we project it onto the given -sized ` p-norm constraint via a projection operator Π ( line 15 ) , to fulfill the constraint in Eq . ( 5 ) . The projection is trivial for p =∞ and p = 2 . For p = 1 , we use the efficient algorithm by Duchi et al . [ 14 ] . For p = 0 , we retain only the first components of δ exhibiting the largest absolute value . We finally clip the components of δ that violate the bounds in Eq . ( 3 ) ( line 16 ) . Execution example . In Fig . 1 ( b ) , we report an example of execution of our algorithm on a bidimensional problem . The initial sample is updated to follow the negative gradient of L towards the decision boundary . When an adversarial point is found , the algorithm reduces to find a better solution . The point is thus projected back onto the non-adversarial region , and increased ( by a smaller , decaying amount ) . These oscillations allow the point to walk on the boundary towards a local optimum , i.e. , an adversarial point lying on the boundary , where the gradient of the loss function and that of the norm constraint have opposite direction . FMN tends to quickly converge to a good local optimum , provided that the step size is reduced to a sufficiently-small value and that a sufficiently-large number of iterations are performed . This is also confirmed empirically in Sect . 3 . Adversarial initialization . Our attack can be initialized from the input sample x , or from a point xinit belonging either to a different class ( if the attack is untargeted ) or to the target class ( if the attack is targeted ) . When initializing the attack from xinit , we perform a 10-step binary search between x and xinit , to find an adversarial point which is closer to the decision boundary . In particular , we aim to find the minimum such that L ( x + Π ( xinit − x ) , y , θ ) < 0 ( or Lt < 0 for targeted attacks ) . Then we run our attack starting from the corresponding values of xk , k , δk and δ ? . Differences with DDN . FMN applies substantial changes to both the algorithm and the formulation of DDN . The main difference is that ( i ) DDN always rescales the perturbation to have size . This operation is problematic when using other norms , especially sparse ones , as it hinders the ability of the attack to explore the neighboring space and find a suitable descent direction . Another difference is that ( ii ) FMN does not use the cross-entropy loss , but it uses the logit difference as the loss function L , since the latter is less affected by saturation effects . Moreover , ( iii ) FMN does not need an initial value for , as is dynamically estimated ; and ( iv ) γ is decayed to improve convergence around better minimum-norm solutions , by more effectively dampening oscillations around the boundary . Finally , we include the possibility of ( v ) initializing the attack from an adversarial point , which can greatly increase the convergence speed of the algorithm , as it uses a fast line-search algorithm to find the boundary and the remaining queries to refine the result .

Summary: This paper presents an adversarial attack (FMN) that generates minimum-norm adversarial examples. The attack builds on the DDN attack but generalizes to $l_p$ norms beyond the $l_2$ norm that the DDN attack is specialized for.  - For the $l_0$, $l_1$ and $l_\infty$ norms, the attack has better performance: it has a consistently lower execution time per query, is more query efficient, and results in adversarial examples of comparable or smaller norm compared to the state-of-the-art. - For the $l_2$ norm, the attack has comparable performance to the DDN attack: it has a slightly higher execution time per query, has comparable query efficiency, and results in adversarial examples of comparable norm compared to the state of the art (DDN).  In addition to the advantages in execution time and query efficiency, the method does not require adversarial starting points as some other methods do.