Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks

1 Introduction . It was demonstrated that despite their impressive performance in various tasks , neural networks are susceptible to small imperceptible perturbations in the input called adversarial examples [ 4 ] . This is a concerning issue for real-world deployment of deep learning approaches , especially in safety-critical domains such as autonomous driving [ 5 ] . But apart from practical concerns , adversarial examples serve as a tool to better understand the true nature of artificial neural networks and capture their inherent properties and differences from their biological counterparts [ 6 ] . Evaluating adversarial robustness is usually formulated as a constrained optimization problem [ 4 ] . However , finding the exact solution is typically intractable [ 7 , 8 ] . Therefore , in practice one often resorts to approximate methods called adversarial attacks that try to find adversarial examples within a small number of iterations . A number of different adversarial attacks were proposed [ 8–10 ] that can be distinguished into white- and black-box attacks . In white-box attacks , one assumes full access to the model architecture and weights [ 8 ] . However , as white-box attacks typically rely on the gradient , gradient obfuscation , which does not eliminate the existence of adversarial examples but makes it significantly harder to find them gradient-based , can be a severe obstacle for certain attacks [ 11 ] . 1The code is available at https : //github.com/boschresearch/meta-rs 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Therefore , black-box attacks that use only limited information from the model such as the class scores or the final decision often provide an additional perspective on the true robustness of the model [ 12–21 ] and are thus recommended for reliable robustness evaluation [ 2 , 22 ] . A very promising direction in the field of black-box adversarial attacks are randomized search schemes for crafting adversarial examples [ 1 , 23 , 24 ] . Combining random search with specific update proposal distributions allows to achieve state-of-the-art black-box efficiency for different threat models such as ` ∞ and ` 2 [ 1 ] , ` 1 [ 25 ] , ` 0 , adversarial patches , and adversarial frames [ 24 ] . Despite the conceptual simplicity of these methods , the main disadvantage of random search based methods is that the construction of a suitable proposal distribution requires significant manual design and is crucial for competitive performance ( see also Figure 1 ) . In this work , we propose a method that allows to circumvent the fine-tuning and reduce the amount of manual design in random search based attacks . We use gradient-based meta-learning to automatically optimize controllers for schedules and proposal distribution on models with white-box access . After meta-training , the controllers can be plugged into a random search attack substituting manually designed schedules and proposal distributions . Importantly , once meta-trained , the controllers do not require any gradient access and can thus be used in a fully black-box setting and without being affected by gradient obfuscation . We consider the proposed methodology for the case of Square Attack for the ` ∞ and ` 2 threat models [ 1 ] . We meta-train controllers for update size and color on an adversarially trained [ 26 ] ResNet18 [ 27 ] model with white-box access and apply them to many different models from the RobustBench model zoo [ 28 ] that we treat as black-boxes . Since query efficiency is of crucial importance in black-box adversarial attacks , we also study the method for different query regimes ranging from several hundreds to several thousands . Depending on the query regime and the attacked model we obtain up to 20 % improvement with respect to the baseline schedules proposed by Andriushchenko et al . [ 1 ] and Croce and Hein [ 2 ] . In short , we make the following contributions : • We frame adversarial attack optimization as a meta-learning problem ( Section 3.2 ) . • We formalize the gradient-based meta-learning for the state-of-the-art Square Attack and propose Meta Square Attack ( Section 3.3 ) . • We meta-train Meta Square Attack ( MSA ) on a CIFAR10 [ 48 ] model with white-box access and show that MSA improves robust accuracy by up to 5.6 % on a vast range of CIFAR10 models with black-box access with respect to the hand-designed search distributions proposed in previous work [ 1 , 2 ] for the ` ∞ and ` 2 threat models ( Section 4 ) . • We show that Meta Square Attack generalizes well to different datasets and to the targeted attack setting . It achieves up to 20 % better robust accuracy compared to the state-of-the art baseline [ 1 ] for attacking models on CIFAR100 and ImageNet ( Section 4.2 ) . 2 Related Work . Black-box adversarial attacks : Existing approaches to black-box robustness evaluation can be divided into several groups based on their mode of interaction with the target model . Transfer-based attacks [ 17 , 29 ] rely on having a model with white-box access that is similar to the targeted model . If this is the case , one can generate adversarial perturbations for this substitute model and transfer them in a black-box setting . A downside of such approaches is that similarity between the models is a crucial factor . Without it the efficiency of transfer attacks is drastically reduced [ 29 ] . Decisionbased attacks [ 13 , 20 , 30 ] applied to classifiers assume that one can submit queries to the model and receive the predicted class label . Score-based attacks [ 1 , 12–19 ] consider broader access to the model and assume the attack receives predicted scores for all the classes . Prior work [ 15 , 31 , 32 ] use this information to estimate a gradient and apply gradient descent . Alzantot et al . [ 19 ] propose a derivative-free approach based on genetic algorithms . Several works use the empirical observation that successful ` ∞ perturbations are located at the corners of the color cube , therefore instead of continuous optimization one can reduce the problem to a discrete one [ 1 , 12 , 24 , 33 ] . Meta-learning and adversarial robustness : The closely related fields of meta-learning [ 34 ] and learning to optimize [ 35 ] have been employed in the field of adversarial robustness . The idea of learned optimizers [ 36 , 37 ] was applied to finding adversarial examples in white-box [ 38 ] and blackbox [ 39 ] settings . Meta-learning [ 40 , 41 ] was also used to improve zeroth-order gradient estimation and allow better query efficiency of black-box attacks [ 42 ] . Besides that there is the recent work [ 43 ] on automating the existing AutoAttack [ 2 ] framework for robustness evaluation . Meta-learning has also been used as part of adversarial training [ 38 , 44 ] for increasing adversarial robustness . Square Attack : Square Attack ( SA ) was proposed by Andriushchenko et al . [ 1 ] and combines classical random search with a heuristic design of the update rule . This design depends on the geometry of the perturbation set and therefore differs for ` ∞- and ` 2-attacks and was later extended also to ` 1 [ 25 ] . As the adversarial attack problem is highly non-convex , a good initialization can significantly improve query efficiency . SA uses a stripe initialization motivated by empirical findings [ 45 ] . The design principle of the attack is based on the observation that the strongest perturbations are usually found on the boundary of the feasible set [ 18 ] . For the ` ∞-case , updates are sampled as squares parametrized by square size , square color , and position of the square . Performance of SA depends heavily on how color , position , and square size are chosen , which requires manual design . The square size schedules employed by previous work are relatively sophisticated ( indicating nontrivial manual design ) : Andriushchenko et al . [ 1 ] proposed a schedule parametrized by p0 ∈ [ 0 , 1 ] ( the fraction of image pixels to be modified by a square in the first query ) and the total query budget T , where pt is halved at { 0.1 , 0.5 , 2 , 10 , 20 , 40 , 60 , 80 } % of the total query budget . For evaluation on CIFAR10 , Andriushchenko et al . [ 1 ] suggest different values of p0 and report p0 = 0.3 as a default choice . Croce and Hein [ 2 ] proposed a different schedule for SA to be used in AutoAttack : they use p0 = 0.8 and T = 5000 but fix the halving points of pt as if T = 10000 . An illustration of the two schedules for T = 500 and T = 5000 can be seen in Figure 1 . The second aspect that characterizes SA is the distribution from which position and color of the next square are sampled . For positions , the distribution is uniform over all positions for which a square of a given size would be fully contained in the input . For colors , SA always generates points on the boundary of the perturbation set ( corners of the color cube ) and uses a uniform distribution over the 2c different colors ( with c being number of channels , typically c = 3 ) . Thus SA either relies on extensive manual design ( for the square size schedules ) or resorts to simple baseline choices ( such as the uniform distributions over colors and positions ) which might be suboptimal . In this work , we show that meta-learning SA consistently improves the already strong performance of SA ( see Section 4.2 ) with very little manual design and identify non-trivial patterns that increase attack efficiency ( see Section 4.3 ) . 3 Method . In the following , we briefly introduce the formulation of the optimization problem for an adversarial attack , rephrase it in the from of a meta-learning problem , and introduce our method for learning the search distribution of a specific random search based black-box adversarial attack , namely Square Attack [ 1 ] . We denote this method as Meta Square Attack ( MSA ) . 3.1 Adversarial Robustness Evaluation . Let K be the number of classes in a classification problem and f : [ 0 , 1 ] d → ∆K−1 be a classifier which maps a d-dimensional input x to ∆K−1 = { ( p0 , ... , pK−1 ) ∈ RK ∣∣∣K−1∑ i=0 pi = 1 , and pi ≥ 0 for i = 0 , ... , K − 1 } , which denotes the set of probability distributions over the K discrete possible outcomes . For a label y ∈ { 0 , ... , K − 1 } , a loss function l : ∆K−1 × { 0 , ... , K − 1 } → R , a perturbation set S , and operator a , we define the robustness evaluation problem as : V ( f , x , y ) = max ξ∈S l ( f ( a ( x , ξ ) ) , y ) ( 1 ) For S = { ξ | ||ξ||p ≤ } and a ( x , ξ ) = Π [ 0,1 ] d ( x+ ξ ) ( where Π [ 0,1 ] d is the projection onto [ 0 , 1 ] d ) , one obtains the standard ` p ball threat model for images . Assuming that the loss function l and operator a are fixed , we denote L ( f , x , y , ξ ) : = l ( f ( a ( x , ξ ) ) , y ) as a functional that one needs to maximize in robustness evaluation . Since exact maximization of Equation ( 1 ) is intractable in the general case [ 7 ] , we consider a ( potentially non-deterministic ) procedure Aω ( L , f , x , y ) called adversarial attack . This attack Aω is parametrized by hyperparameters ω and designed with the intention that ξω ∼ Aω ( L , f , x , y ) with ξω ∈ S becomes an approximate solution ( tight lower bound ) of V ( f , x , y ) , that is V ( f , x , y ) − L ( f , x , y , ξω ) should become small ( in expectation ) . Optimizing the hyperparameters of the attack via max ω E ξω∼Aω ( L , f , x , y ) L ( f , x , y , ξω ) can allow a tighter lower bound of V ( f , x , y ) . Unfortunately , this maximization is still intractable typically , for instance when f is a black-box ( no gradient information is available ) , the number of queries to f per data ( x , y ) is limited , or Aω has high variance .

Summary: This paper proposes meta-learning a black box adversarial attack strategy. Specifically, the authors propose meta-learning controllers that output the parameters of a square attack. Experiments on CIFAR-10 and ImageNet demonstrate the effectiveness of a meta-learned square attack relative to a non meta-learned baseline square attack.