Query-efficient Meta Attack to Deep Neural Networks
Authors: Jiawei Du
Hu Zhang
Joey Tianyi Zhou
Yi Yang
Jiashi Feng
Venue:  (2020)
Keywords: Adversarial attack
Meta learning

Abstract: Black-box attack methods aim to infer suitable attack patterns to targeted DNN models by only using output feedback of the models and the corresponding input queries. However, due to lack of prior and inefficiency in leveraging the query and feedback information, existing methods are mostly query-intensive for obtaining effective attack patterns. In this work, we propose a meta attack approach that is capable of attacking a targeted  model with much fewer queries. Its high query-efficiency stems from effective utilization of  meta learning approaches in learning generalizable prior abstraction from the previously observed attack patterns and exploiting  such prior to help infer attack patterns from only a few queries and outputs. Extensive experiments on MNIST, CIFAR10 and tiny-Imagenet demonstrate that our meta-attack method can remarkably reduce the number of model queries without sacrificing the attack performance. Besides, the obtained meta attacker is not restricted to a particular model but can be used easily with a fast adaptive ability to attack a variety of models. Our code will be released to the public.

Published as a conference paper at ICLR 2020
QUERY-EFFICIENT META ATTACK TO DEEP NEURAL NETWORKS
Jiawei 1.3 D Hu Zhang Joey Tianyi Zhou', Yi Yang", Jiashi Feng' Deptt. ECE, National University of Singapore, Singapore RRLERR, University of Technology Sydney, Australia Institute of High performance Computing. STTAR Singapore dujiaweisu. nus. edu, Hu. Zhang 10s--denttdtttt.mm ut edu. au boey.tiangs zhouêgmail com, Yi. Vang@uts.edu.au edu. e elefjia@nce edu. sg
ABSTRACT
Black-box attack methods aim infer suitable attack patterns to targeted DNN models by only using output feedback of the models and the corresponding input queries. However, due to lack of prior and inefficiency in leveraging the query and feedback information, existing methods are mostly query-intensive for obtaining effective attack patterns. In this work, we propose a meta attack approach that is capable of attacking a targeted model with much fewer queries. Its high query- efficiency stems from effective utilization of meta learning approaches in learning generalizable prior abstraction from the previously observed attack patterns and exploiting such prior to help infer attack patterns from only a few queries and outputs. Extensive experiments on MNIST, CIFARIO and tiny-Imagenet demon- strate that our meta-attack method can remarkably reduce the number of model queries without sacrificing the attack performance. Besides, the obtained meta attacker is not restricted to a particular model but can be used easily with fast adaptive ability to attack a variety of models. The code of our work is available https: htt.tttt.ub....g// com htt.t///...///ydy//////ttttaaaaa
INTRODUCTION
Despite the great success in various tasks, deep neural networks (DNNs) are found to be suscepti- ble to adversarial attacks and often suffer dramatic performance degradation in front of adversarial examples, even if only tiny and invisible noise is imposed the input (Szegedy et al.) 2014). To in- vestigate the safety and robustness of DNNs, many adversarial attack methods have been developed, which apply either white-box (Goodfellow al. 2015 Moosavi-Dezfooli al. 2016 Carlini
& Wagner| [0177: Madry et al.] (018)) or a black-box setting (Papernot et al.) [017]] Brendel et al. 2018 Narodytska & Kasiviswanathan 2017). In the white-box attack setting, the target model transparent to the attacker and impercepuble adversarial noise can be easily crafted to mislead this model by leveraging its gradient information (Goodfellow et al. 2015). In contrast, in the black- box setting, the structure and parameters of the target DNN model are invisible, and the adversary can only access the input-output pair in each query. With a sufficient number queries, black-box methods utilize the returned information attack the target model generally by estimating gradient 2017 Cheng et al. 2019).
Chen et al. 2017 llys et al. 2018a Narodytska Kasiviswanathan
Black-box attack is more feasible in realistic scenarios than white-box attack but it is much more query-intensive. Such a drawback is largely attributed to the fact that returned information for each queried example is sparse and limited. During inferring attack patterns, existing black-box methods simply integrate the information between two sequential iterations brutally and ignore the implicit but profound message, thus not fully exploiting the returned information. Although query-efficient algorithms for generating attack examples are very meaningful ful in practice (Ilyas et al. 2018a), how enhance query-efficiency for black-box attack remains underexplored.
"Equal contribution.

---

Published as a conference paper at ICLR 2020
In this work, we address a query -ffereefficceney concerned attack problem. Partieularly, consider only top-k probability scores accessible from the target black-box model. With this practical but challenging scenario, aim three important objectives: lower query number, higher success rate and smaller noise magnitude. We develop a meta-learning based attack method, which applies meta learning obtaining prior information from the successful attack patterns, and uses the prior for efficient optimization. Specifically, we propose to train a meta attacker model through meta learning (Nichol al. 2018), inspired by its success in solving few-shot learning problems. We first deploy several existing classification models to get pairs of (images, gradients) with the max- max-margin logit classification loss. Then we use the data pairs of each classification model to train the meta attacker. After obtaining the attacker, we use it to attack new black-box model for accelerating the search process for adversarial examples by optimizing it with coordinate-wisg gradient estimation. Different from previous methods, we use the estimated gradient not only to update adversarial noise but to fine-tune the well-trained attacker. After few-shot fine-tuning, the attacker is able to simulate the gradient distribution of the target model.
We evaluate our method on MNIST, CIFARIO and tiny-ImageNet datasets by comparing with state-of-the-art black-box attack methods including Zoo (Chen et al.; (017). Decision- Boundary (Brendel et al. 2018), AutoZoom (Tu et al.] 2019). Opt-attack (Cheng et al.) (019) and Bandits (Ilyas et al. (018b). In both targeted and untargeted settings, our proposed method achieves comparable attack success rate and adversarial perturbation to all baselines but with a significantly reduced query number. The detailed experiment results demonstrate our superior query-efficiency.
2 RELATED WORK
Classical white-box attack methods include Fast-Gradient Sign Method (FGSM) (Goodfellow et al,, 2013), IFGSM (Madry et al.) 2018), DeepFool (Moosavi-Deztooll et al., 2016) and C&W at- tack (Carlini& & Wagner 2017). following a setting where detailed information about the target model (gradients and losses) is provided. Comparatively, the black-box setting better accords with the real world scenarios in that little information about the target model is visible to the attacker. The pi- oneer work on black-box attack (Papernot al. (017) tries to construct substitute model with augmented data and transfer the black-box attack problem to a white- box one. However, its attack performance is very poor due to the limited transferability of adversarial examples between two dif- ferent models. (Brendel et al. 2018) considers a more restricted case where only top-1 prediction classes are returned and proposes a random- walk based attack method around the decision boundary. It dispenses class prediction scores and hence requires extensive model queries. Zoo (Chen et al.) 2017 is a black-box version of C&W W attack, achieving similar attack success rate and comparable visual quality as many white-box attack methods. However, its coordinate-we wise gradient estimation requires extensive model evaluations. More recently, (Ilyas et al.] (018a) proposes a query- murry-limited setting with L.o noise considered, and uses a natural evolution strategy (NES) to enhance query effi- ciency. Though this method successfully controls the query number, the noise imposed is larger than average. (Narodytska & Kasiviswanathan 2017) proposes novel local-seard based technique to construct numerical approximation to the network gradient, which is then carefully used to construct small set of pixels in an image to perturb. It suffers a similar problem as in (Chen al. 2017) for pixel-wise attack. (Cheng et al., 2019) considers a hard-label black-box setting and formulates the problem as real-valued optimization that is solved by zeroth order optimization algorithm. [yas et al. (2018b) reduce the queries by introducing two gradient priors, the time-independent prior and the data-dependent prior, and reformulating the optimization problem.
We then briefly introduce some works on meta-learning related to our work. Meta-learning is pro- cess of learning how to learn. meta-learning algorithm takes in a distribution of tasks, each being a learning problem, and produces a quick learner that can generalize from a small number of exam- ples. Meta-learning very popular recently for its fast adaptive ability. MAML (Finn et al.. 2017) is the first to propose this idea. Recently, a simplified algorithm Reptile (Nichol et al.) 2018)) which is an approximation to the first-order MAML is proposed, achieving higher efficiency in computation and consuming less memory. With these superior properties, meta learning is applied to adversarial attack methods (Zgner& & Gnnemann 2019: Edmunds et al. 2017). Zgner& & Gnnemann (2019) try to attack the structure of graph model in the training process to decrease the model generalization performance. |Emunds et al. (2017) investigate the susceptibility of MAML to adversarial attacks and the transferability of the obtained meta model to specific task.
2

---

Published as a conference paper at ICLR 2020
3 METHOD
3.1 PRELIMINARIES: BLACK-BOV ATTACK SCHEMES
We first formulate the black-box attack problem and introduce the widely used solutions. We use (z, t) to denote the pair of natural image and its true label, and & and É to denote the adversarial perturbed version of z and the returned label by the target classification model Maar- The black-brr box attack aims to find an adversarial example & with imperceivable difference from æ to fail the target model. i.e., i A through querying the target model for multiple times. can be formulated as
min ( Mear (t),t)
(1)
s.t. a - =1, -
P. #queries S Q.
Here || I|p denotes the E, norm that measures how much perturbation is imposed. Maar (+) the returned logit or probability by the target model Mear. The loss function ((,, (aa t) measures the degree of certainty for model Miar assigning the input & into class t. One common used adversarial loss is the probability of class t: f( Mear (i),t) t) = PMrr (f) The first constraint enforces high similarity between the clean image a and the adversarial one Fods and the second
imposes fixed budget Q for the number of queries allowed in the optimization.
In the white-box attack setting, the adversary can access the true gradient Va, ((tz)) and perform gradient descent .t+1 = c. Va, ((t)). But in the black-box setting, the gradient information this case, the attacker can estimate the gradient using only queried
((t(( is not attainable. information from model evaluation such as hard label, logits and probability scores. This kind of estimator is the backbone of so-called zeroth-ore order optimization approaches (Chen et al., [017] Narodytska & Kasiviswanathan, 2017; TTu al., 20199; llyas al., 2018a/b). The estimation is done via finite difference method (Chen et al. [017]] Narodytska & Kasiviswanathan, 2017: Tu al., (2019), which finds the & components of the gradient by estimating the inner products of the gradients with all the standard basis vector Ei, Cj::
f(x + he,) f(x - -hei)
V((x)) ~
Ci.
(2)
2h
where step size h controls the quality of the estimated gradient. Another strategy is to reformulate the loss function (Ilyas et al., 2018a b). Instead of computing the gradient of E(xr) itself, the
pected value of loss function {()) under the search distribution is minimized and when the search
distribution of random Guassian noise is adopted, the gradient estimation problem transfers into
zeroth-order estimation problem,
VEL((())
(( + oō.)6,
OTI
where is the amount of noise sampled from the distribution. After obtaining the estimated gradient, classical optimization algorithms [Nsterov] 2013: Johnson Zhang (013) can be used to infer the adversarial examples. Though the estimated gradient may not be accurate, it is still proved useful enough in adversarial attack. The convergence of these zeroth-order methods is guaranteed under
mild assumptions (Ghadimi & Lan, 2013) Nesterov & Spokoiny 2017] Hazan et 2016).
Since each model evaluation consumes a query, naively applying the above gradient estimation to black-box attack is quite query expensive due to its coordinate or noise sampling nature. Take the first strategy on tiny-Imagenet dataset for example. It consumes more than 20,000 queries for each image to obtain a full gradient estimate, which such a limitation via developing a query-efficient meta-learning based attack model.
affordable in practice. this work, we address
3.2 LEARNING OF META ATTACKER
To reduce the query cost for black-box attack, we apply meta learning to training a meta attacker model, inspired by its recent success in few-shot learning problems (Finn et al. 2017 Nichol al.) 2018). The meta attacker learns to extract useful prior information of the gradient of a variety of models w.r.t. specific input samples. It can infer the gradient for new target model using only
3

---

Published as a conference paper at ICLR 2020
Algorithm Meta Attacker Training
Input: Input images X. groundtruth gradients G; generated from classification models M, serve as task T;
Randomly initialize
2: while not done do
3: for all do
4: Sample K samples from (X, G.) for training, denoted as (%,, G();
5: Evaluate (AC)(AA)) = Vell VellAA(Xx (X,) Gfll with respect to (Xg, G);
6: Update 0 = Ve AA));
7: end for
8: Update 0 = 0 c E (0) 0);
9: end while
Output: Parameters 0 of meta model A.
few queries, After obtaining such a meta attacker, we replace the zeroth-order gradient estimation in traditional black box attack methods with it to directly estimate the gradient.
We collect a set of existing classification models Mi,..,,,, to generate gradient information for universal meta attacker training. Specifically, feed each image into the models M1,.... M,, respectively and compute losses f,,... Lu by using following max- max-mangin logit classification loss:
((x) = max log[M,(())l max (o)(M, j7t
(4)
Here is the groundtruth label and j indexes other classes. [M.( the probability score of the true label predicted by the model Mi, and [M),; denotes the probability scores of other classes. By performing one step back-propagation of losses f,...., Ey w.r.t the input images æ, the corresponding gradients 9, ( 1, I,..., obtained. Finally, collect n groups of data X = (x), G2 = {g.}. i 1,...,n n to train the universal meta attacker.
We design meta attacker A which has similar structure metric convolution and de-convolution layers and outputs gradient map with the same size as the input. Meta attacker model A is parameterized with parameters 0.
autoencoder, consisting of sym-
Due to the intrinsic difference hetween selected classification models, each obtained set (X, G.) is treated a task Ti in meta attacker training. During the training process, for each iteration only draw K samples from task Tú and feedback the loss L; to update model parameters from 0 to 0%. 0f is then computed through one or multiple gradient descents: 0g := 0 a (Ago). For a sensitive position of the meta attacker, the meta attacker parameters are optimized by combining each f across all tasks (Te)==1..... following the update strategy of Reptile (Nichol et 2018)
in meta learning,
0=
Σ i=1
0' 0).
(5)
We adopt mean-squared error (MSE) as the training loss in the inner update,
(,(A)) [%,) 1
(6)
The set (X,, G3) denotes the K samples used for each inner update from 0 to 0% Since the number of K sampled each time is very small, the update strategy above tries to find good meta attacker parameters as an initial point, from which the meta attacker model can fast adapt to new data distribution through gradient descent based fine-tuning within limited samples. Therefore, this char- acteristic can naturally leveraged attacking new black-box models by estimating their gradient information through few queries. Detailed training process of our meta attacker is described in Algorithm
3.3 QUERY-EFFICIEN ATTACK VIA META ATTACKER
An effective adversarial attack relies optimizing the loss function equation I the input image to find the adversarial example of the target model M Differently, our proposed method applies the meta attacker A to predicting the gradient map of test image directly.
4

---

Published as a conference paper at ICLR 2020
Algorithm 2 Adversarial Meta Attack Algorithm
Input: Test image vo with label t, meta attacker Ag, target model Maars iteration interval m, selected top-q coordinates;
for = 0, 2, ... do
2: if (t +++1, 1) mod m 0 then
3: Perform zeroth-order gradient estimation on top y coordinates, denoted as and obtain 9::
4: Fine-tune meta attacker A with (Izz, 9t) on 4 by loss L (x:))r 1g.1,112
5: else
6:
Generate the gradient map directly from meta attacker A with It, select coordinates I;;
7:
end if
8: Update [x']r, = [ax,1. + +1g..
9: if Mear (zx) o then
10: Bade = x';
break;
12: else
a++11 = x²;
14: end if
15:
end for Output: adversarial example Todv-
We use the obtained meta attacker model A to predict useful gradient map for attacking, which should be fine tuned to adapt to the new gradient distribution under our new target model. Par- ticularly, instead of finetuning once, for each given image æ, we update by leveraging query information with the following periodic scheme. Suppose the given image is perturbed to æ; R2 at iteration L. (t ++ 1) mod m = O, our method performs zeroth-order gradient estimation to ob- tain gradient map g: for fine-tuning. As each pixel value for estimated gradient map consumes two queries, for further saving queries, we just select of the coordinates to estimate, q < p. instead of the full gradient map through all p coordinates. The indexes of chosen coordinates are determined by the gradient map gt-1 obtained in iteration t- 1. We sort the coordinate indexes by the value of g:-1 and select top-q indexes. The of these indexes are denoted as 4t. We feed image æ, iteration t into meta attacker A and compute the MSE loss on indexes I, i.e. L
(æ1))4, [g 1, Then we perform gradient descent for the MSE loss with few steps to update the parameters 0 of meta attacker A. For the rest iterations, we just use the periodically updated attacker As directly generate the gradient g: Aa(x:). When we have the estimated gradient map in iteration I, we update to get the adversarial sample xn by (æ:(), = [r,),, A[ge)., where | is hyperparameter to be tuned. The details are summarized in Algorithm 2
In our method, the following operations contribute to reducing the query number needed by the attacker. First, though we just q coordinates to fine-tune our meta attacker A every iterations, the meta attacker A is trained to ensure that can abstract the gradient distribution of different æ, and learn to predict the gradient from few samples with simple fine-tuning. Secondly, the most query-consuming part lies in zeroth-order gradient estimation, due to its coordinate nature. In algorithm, only do this every iterations. When directly, no query is consumed in gradient estimation in these iterations. Intuitively, larger m implies less gradient estimation computation and fewer queries. Besides, just as mentioned above, even in zeroth-order gradient estimation, only top-q coordinates are required. Normally, g is much smaller than dimension p of the input.
the finetuned meta attacker A
EXPERIMENTS
We compare meta attacker with state-of-the-art black-box attack methods including Zoo (Chen Decision-Boundary
et al. 2017). Decision- Boundary (Brendel et al.] (018)). AutoZoom (Tu et al. 2019), Opt- attack (Cheng et al., 2019), FW-black (Chen et al. 2018) and Bandits (Ilyas al. (018b) to evaluate its query efficiency. We also study its generalizability and transferability through a Meta transfer attacker, detailed in the following sections.
5

---

Published as a conference paper at ICLR 2020
4.1 SETTINGS
Datasets and Target Models We evaluate the attack performance on MNIST (LeCun 1998) for handwritten digit recognition, CIFARIO (Krizhevsky & Hinton 2009) and tiny- Imagenet (Rus- sakovsky et al., 2015)) for object classification. The architecture details of meta attack models on MNIST. CIFARIO and tiny-Imagenet are given in Table 6 For MNIST, we train a separate meta attacker model since the images have different channel numbers from other natural image datasets. For CIFARIO and tiny- Imagenet, we use a common meta attacker model. On CIFAR-10, 10, we choose ResNet18 8 (He et al.) 2016) as the target model Mear and use VGG13, VGG16 (Simonyan & Zisser- mann| 2014) and GoogleNet (Szegedy et al. 2015) for training our meta attacker. On tiny-Imagenet, we choose VGG19 and ResNet34 as the target model separately, and use VGG13, VGG16 and ResNet18 for training the meta attacker together.
Attack Protocols For target black-box model Mear obtaining a pair of (input-output) is consid- ered as one query. We use the mis-classification rate as attack success rate; we randomly select 1000 images from each dataset as test images. To evaluate overall noise added by the attack methods, we use the mean L2 distance across all the samples noise(Mear) = Σi=1 || i.A.Mor 2z where adde Marr denotes the adversarial version for the authentic sample æ...
Meta-training Details For all the experiments, use the same architecture for the meta attacker A as shown in Table 6 We use Reptile (Nichol et al. 2018) with 0.01 learning rate to train meta attackers. We use 10000 randomly selected images from the training set to train the meta-attackers in three datasets. The proportion of the selected images to the whole training set are 16%, 20%, and 10% respectively. Fine-tuning parameters are set as m 5 for MNIST and CIFARIO, and m for ting-emag Top = 128 coordinates are selected as part coordinates for attacker fine-tuning and model attacking on MNIST; and q = 500 on CIFARIO and -magennae
4.2 COMPARISON WITH BASELINES
We compare our meta attacker with baselines for both the untargeted and targeted black-box attack on the three datasets. The results are reported in detail as below.
Untargeted Attack Untargeted attack aims generate adversarial examples that would be mis- classified by the attacked model into any category different from the ground truth one. The overall results are shown in Table T in which, Meta transfer denotes that meta attacker trained on one dataset is used to attack target models on another dataset Our method is competitive with baselines in terms of adversarial perturbation and success rate, but our query number is reduced.
We also compare the results of our method with Zoo (Chen et al. 2017) and AutoZoom (Tu et al. 2019) from a query-effciennyy perspective. We use these models to conduct untargeted attack on CIFARIO 10 and tiny-Imagened by limiting a maximum number of queries for each adversarial example and compare their success rate. The results are shown in Fig. [ We notice that for different query thresholds, the success rate of our method is always higher than 7.0 and AutoZoom. This is possibly because the testing samples have different L2 distances to the decision boundary. Higher success rate of our method indicates meta attacker predict correct gradient even when the query information is limited. These results give strong evidence on effectiveness of our proposed method for enhancing query efficiency. Eny-Imagennt
Sucpess fats: Quaries top selection uo
Sygcess Rato, by. Quories vs. selection
0.5
0.2
6.4
0.1
Quary
899
1op natee
Figure 1: Comparison with limited queries.
Figure 2: Top-q and / selection.
Targeted Attack Targeted attack aims to generate adversarial noise such that the perturbed sample would be mis-classified into any pre-specified category. It is more strict setting than the untargeted

---

Published as a conference paper at ICLR 2020
Table 1: MNIST, CIFARTO and tiny-ImageNer untargeted attack comparison: Meta attacker attains compara-
success rate and l2 distortion as baselines, and significantly reduces query numbers.
Dataset / Target model
Method
Success Rate
Avg. L2
Avg. Queries
Zoo (Chen et al. 2017
1.00
1.61
21,760
Decision Boundary [Brendel et al. 2018
1.00
1.85
13,630
Opt-attack (Cheng cf al. 2019
1.00
1.85
12,925
MNIST Net4
AutoZoom [Tu
2019
1.00
1.86
2,412
Bandits (1lyas el al. 2018b
0.73
1.99
3,771
Meta attack (ours)
1.00
1.77
749
7on (Chen et al. 2017
1.00
0.30
8,192
Deciston Boundary [Brendel ct al. 2018
1.00
0.30
17,010
Opt-attack (Cheng el al 2019
1.00
0.33
20,407
CIFARIO/ Resnet18
AutoZoom (Tu al 2019
1.00
0.28
3,112
Bandits (llyas el al. 2018b
0.33
4,491
FW-black (Chen el al. 2018
0.43
5,021
Meta transter (OUTS)
0.92
0.35
1,765
Meta attack (ours)
0.94
0.34
1,583
Zoo (Chen et al. 2017
1.00
0.52
27.827
Decision Boundary [Beende
2018
0.52
49,942
Opt-attack (Cheng
2019
1.00
0.53
71,016
tiny-ImageNet/ VGG19
AutoZoom Tu 20186
1.00
0.54
8,904
Bandits (llyas ct al.
0.78
0.54
9,159
Meta transter (ours)
0.99
0.56
3,624
Meta attack (ours)
0.99
0.53
3,278
Zoo (Chen al. 2017
1.00
0.47
25,344
Decision Boundary (Brendel
2018
1.00
0.48
49,982
AutoZoom (Tu al. 2019
1.00
0.45
9,770
lmagelmageNet// RResne334
Opt-attack (Chenn ef crain 20 19
1.00
0.52
60,437
Bandits Illyas et al. 2018b
0.73
0.49
9,978
Meta transter (ours)
0.99
0.56
3,540
Meta attack (ours)
0.99
0.53
3,268
one. For fair comparison, we define the target label for each sample -a sample with label l gets the target label (f+1) mod #classes. We deploy our meta attacker the MNIST, CIFARIO and tiny-ImageNet are shown in Table 2 Similar to results of untargeted attack, achieve comparable noise and success rate baselines but with reduced query numbers.
above. The results
4.3
MODEL ANALYSIS
Meta Training We first test the benefits of meta training by comparing performance of meta- trained attacker with Gaussian randomly initialized attacker without meta training on the three datasets. Fig. shows their success rate, L2 distortion and query count results for initial success. The meta pre-trained attacker achieves averagely 7% higher success rate with 16% lower L2 distortion and 30% less queries, compared with the randomly initialized one. This justifies the contributions of meta training to enhancing query efficiency and also attack performance.
Guaranteed by fine-tuning, the randomly initialized attacker succeeds many testing samples. The fine-tuning works like an inner training in meta training. With sufficient fine-tuning iterations, the randomly initialized attacker functions like well-trained meta attacker. This explains the effec- tiveness of the randomly initialized meta attacker on many testing samples compromised by more queries. However, it could not predict gradient as accurate the well-trained meta attacker during earlier iterations. Such inaccuracy leads to larger Le distortion at the beginning. On the contrary, the meta training process enables the well-trained meta attacker fast-adapt to current testing samples. These results highlight the significant advantages of our meta model towards to black-box attack. The process of meta training makes it familiar with gradient patterns of various models.
Generalizability Here we show that our meta attacker trained other datasets. We conduct this experiment between CIFAR1O and tiny- iny-Imagenet, denoted as Meta transfer in Table [ and 2 We first apply the meta attacker trained CIFARI0 attack VGG19, ResNet34 on tiny- Emy-entgenet respectively, which are different from models used for training meta attacker. Note the meta attacker tested CIFAR10 has privileged prior and is not familiar with
dataset can be transferred to

---

Published as a conference paper at ICLR 2020
Table 2: MNIST, CIFARIO and liny-ImageNet targeted attack comparison: Meta attack significantly outper-
forms other black-box methods query numbers.
Dataset / Target model
Method
Success Rate
Avg. Lg
Avg. Queries
Zoo (Chen et al. 2017
1.00
2.63
23,552
Decision Boundary [Brendel el al. 2018
0,64
2.71
19,951
MNIST / Netd
AutoZoom TTu et al 2019
0.95
2.52
6,174
Opt-attack Cheng ala 2019
1.00
2.33
99,661
Meta attack (ours)
1.00
2.66
1,299
Zoo (Chen et al. 2017
1.00
0.55
66,400
Decision Boundary (Brendel ct al. 2018
0.58
0.53
16,250
CIFARI0 Resnet18
AutoZoom (Tu et al
2019
1.00
0.51
9,082
Opt-attack (Cheng el al 2019
1.00
0.50
121,810
FW-black Chen cf al. 2018
0.90
0.73
6.987
Meta transier (ours)
0.74
Meta attack (ours)
0.93
0.77
3,667
Zoo (Chen el al. 2017
0,74
1.26
119,648
AutoZoom et
0.87
1.45
53,778
liny-ImageNet// VGG19
Opt-attack Cheng ct al. 19
1.14
252.009
Meta transfer (ours)
0,55
1.37
12,275
Mcta attack (ours)
0.54
1.24
11,498
7on (Chen et al. 2017
0.60
1.03
88,966
AutoZoom [Tu cl. 2019
1.15
52,174
lingelmaeNNe// R4snet33
Opt-attack (Cheng el al. 20
214,015
Meta transfer (ours)
0.69
1.40
13,435
Mcta attack (ours)
0.54
1.21
12,897
Mnist Net4
Cifar10 Resnet18
tiny-lmagenet VGG19
2.00
1.0
Attacker
1.0
1.75
0.59
Pretrainad Attacker
0.89
1.2
0.64
1.00
1.0
0.5
0.5
D.55
0.34
0.50
2183
0.2
0.2
0.00
0.0
0.0
Success Rate
L2 Dist
Queries
Success Rate
L2 Dist
Querles
Success Rate
L2 Dist
Querles
Figure 3: Comparison of randomly initialized and well-trained meta attackers.
neither tiny-Imagenet dataset nor the corresponding classification models. Similarly, we also use the meta attacker trained on Imagennt to attack the target ResNet18 model on CIFARIO. The results show the good generalizability and robustness of our proposed meta attacker.
Parameters Selection We the selection of top-g and on CIFAR1O. We choose q ranging from 350 to 600 and give the results with 8 ranging from 3e-3 to 5e-3, as shown in Fig. 2 When increases, query number, success rate and L2 will all increase. order balance the overall result, we choose to be 500. When & increases, success rate and L2 will increase and query number decreases. In order to balance success rate and query, we choose 3 be 4e-3 in the experiment.
CONCLUSION
We propose meta-based black-box attack method that largely reduces demanded query numbers without compromising in attack success rate and distortion. We train meta attacker to learn useful prior information about gradient and incorporate it into the optimization process to decrease the number of queries. Specifically, the meta attacker is finetuned to fit the gradient distribution of target model and each update is based on the output of finetuned meta attaker. Extensive experimental results confirm the superior query-efficiency of our method over baselines.
8

---

Published as a conference paper at ICLR 2020
ACKNOWLEDGEMENT
This research is partially supported by Programmatic grant no. A1687b0033 from the Singapore government's 's Research, Innovation and Enterprise 2020 plan (Advanced Manufacturing and Engi- neering domain).
Hu Zhang (No. 201 706340188) is partially supported by the Chinese Scholarship Council.
Jiashi Feng was partially supported by NUS IDS R-265-000-C67-666, ECRA R-263-000-C37-133,, MOE Tier-Il R-263-000-D17-122 and AL,SG R-263-000-D99-490.

---

Published as a conference paper at ICLR 2020
REFERENCES
W. Brendel, J. Rauber, and M. Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In International Conference on Learning Representations, 2018. Utttt:s::
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium Security and Privacy (SP), pp. 39-57, IEEE, 2017.
Jinghui Chen, Jinfeng Yi, and Quanquan Gu. A wolfe framework for efficient and effective adversarial attacks. arXiv preprint arXiv: 1811.0880 2018.
Pin-Yu Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order opti- mization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 15-26. ACM, 2017.
Minhao Cheng, Thong Le, Pin- Yu Chen, Huan Zhang, JinFeng Yi, and Cho-Jui Hsieh. Query- efficient hard-label black-box attack: An optimization based approach. In International Confer- ence on Learning Representations, 2019. URL htw.t:tpppp........... http:::://///eeeviiiiee ew. net fott/fooummdii
rJ (0lk6iRqK)
Riley Edmunds, Noah Golmant, Vinay Ramasesh, Phillip Kuznetsov, Piyush Patil, and Raul Puri. Transfcrability of adversarial attacks in agnosttg meta-lcarning. Deep Learning and Secu- rity Workshop (DLSW) in Singapore, 2017.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference Machine Learning- Volume 70, pp. 1126-1135. JMLR. org, 2017.
Saced Ghadimi and Guanghui Lan. Stochastic first-and zeroth-order methods for nonconvex stochas- tic programming. SIAM Journal on Optimization, 23(4):2341-2366 2013.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015. URL http://
arxiv. org. wtt.:r//aabba......../. 6572
Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends in Opti- mization, 2(3-4): 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, July 2018a. URL
wwtpa/wwwwwwg..///a///aaaa//.//. 08598
Andrew Ilyas, Logan Engstrom, and Aleksander Madry. Prior convictions: Black- box adversarial attacks with bandits and priors. arXiv preprint 18X7:1880..07978. 2018b.
Rie Johnson and Tong Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In Advances in neural information processing systems, pp. 315-323, 2013.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, Citeseer, 2009.
Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exxd/nmist//, 1998.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks. In International Conference on Leam- ing Representations, 2018.
10

---

Published as a conference paper at ICLR 2020
Seyed-Mohsen Moosavi-Dezfool Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on com- puter vision and pattern recognition, pp. 2582, 2016.
Nina Narodytska and Shiva Kasiviswanathan. Simple black-box adversarial attacks on deep neural networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1310-1318. IEEE, 2017.
Yurii Nesterov. Introductory lectures on convex optimization: A basic course, volume 87, Springer Science & Business Media, 2013.
Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics, 17(2):577-666, 2017.
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta- meta-learning algorithms. arXiv preprint 1rXivv1803. 183.0999,,, 2018.
Nicolas Papernot, Patrick McDaniel, Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pp- 506-519. ACM, 2017.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 11(3)2211-252, 2015. doi: 1007/s 1263001--01---
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 1nXiv:1409.15555 2014.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014. URL http::tttxiiii.tt///// wwwpal/v.rggaa//////aaaa 6199
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9, 2015.
Chun-Cher Tu, Paishun Ting, Pin-Yu Chen, Sijia Liu, Huan Zhang, Jinfeng Yi, Cho-Jui Hsieh, and Shin-Ming Cheng. Autozoom: Autoencoder-be based zeroth order optimization method for attack- ing black-box neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 742-749, 2019.
Daniel Zgner and Stephan Gnnemann. Adversarial attacks graph neural networks via meta learning. In International Conference on Learning Representations, 2019. URL https:

---

Published as a conference paper at ICLR 2020
6 APPENDIX
6.1 MORE EXPERIMENTAL RESULTS
6.1.1 COSINE SIMILARITY BETWEEN ESTIMATED GRADIENTS AND WHITE-BOX GRADIENTS
To demonstrate the ability of estimating gradient of meta attacker, we have conducted exper- iments with the Resnet-34 model on the tiny-Imagens dataset to compare the cosine similarity between the estimated gradients from our proposed meta-attacker and the accurate white-box gradi- ents. We also compare the cosine similarity between ZOO (Chen et al. 2017) estimated gradients and the white-box gradients as reference. The results of cosine similarity and required number of queries are shown in Table 3 We can observe that our meta-attacker does fast-adapt to the target model and generate accurate gradients. Not only it estimates gradients with positive cosine similar- ity to the true gradients, but it also performs closer to the ZOO estimated results with the same small standard deviation.
Table 3: Cosine similarity between estimated gradients and white-box gradients.
Ours meta-attacker
ZOO (Chen et al.
2017
Task Type Untargeted
Similarity
Queries
Similarity
Queries
0.356 + 0.074
3,268
0.395 = 0.079
25,344
Targeted
0.225 + 0.090
12,897
0.363 = 0.069
88,966
6.1.2 VANILLA TRAINING AUTOENCODER AND META-ATTACKER LEARNING FROM ESTIMATED GRADIENTS
We have conducted experiments in section 4.3 to demonstrate the benefits of meta training. However, we only compare the performance of meta-trained attacker with a Gaussian randomly initialized attacker. We conduct two more experiments meta training here to further investigate the benefits of meta training. First we compare our meta-trained attacker with a vanilla autoencoder that learns to map images to gradients of one white-box model. The vanilla autoencoder has the same architecture with our aeta-attacker, but it is trained in one white-box model. Then, we train a new meta-attacke in four black-box models, i.e., the gradients used for training estimated via ZOO (Chen al. 2017). The experiment results are presented in Table
Table 4: MNIST untargeted attack comparison.
Method
Success Rate
Avg. La
Avg. Queries
(urs(reported)
1.000
1.78
1,103
ZOO estimated
1.000
1.77
1,130
Vanilla Autoencoder
1.000
1.93
1,899
Initialised attacker
0.912
1.96
1.721
Table 5: MNIST targeted attack comparison.
Method
Success Rate
Avg. La
Avg. Queries
Ours(reported)
1,000
2.66
1,971
ZOO estimated
1,000
2.42
2,105
Vanilla Autoencoder
1.000
2.80
2,905
Initialised attacker 0.895
2.81
3,040
The two experiments are conducted on 1000 randomly selected images from the MNIST testing set. As for the four approach settings: "Ours (reported)" is the results we report in our paper; "Z00 esti- mated" the meta-attacker trained from ZOO estimated gradients; "Vanilla Autoencoder" is the au- toencoder maps image to gradient trained in one different MNIST classification model; "Initialised attacker" is the meta-attacker with randomly initialized weights. We can see the "ZOO estimated" model performs closer much worse (with larger L2-norm and more queries), which performs similarly to randomly initial- ized meta-attacker. The two experiments verify the effectiveness of our first meta-training phase.
reported meta-attacker. However, the "Vanilla Autoencoder" performs
12

---

Published as a conference paper at ICLR 2020
6.2 STRUCTURE OF META ATTACKER
Table 6: Structure meta attacker. Conv: convolutional layer, Convt: de-convolutional layer.
Mcta attacker (MNIST)
Meta attacker (CIFARIO, tny-ImageNet)
Conv(16, 3, 1) + ReLu + bn
Conv(32, 3, 3, 1) + ReLu + bn
Conv(32, 4, 4, 2) + ReLu bn
Conv(64, 4,4,
ReLu + bn
Conv(64, 4, 2) + ReLu bn
Conv(128, 4, 4, 2) ReLu bn
Conv(64, 4, 4, 2) + ReLu + bn
Conv(256, 4, 4, 2) + ReLu + bn
Convi(64, 4,4, 2) + Rel. + bn
Convi(256, 4, 2)
Rel.u + bn
Convi(32, 4,4, 2)
ReLu bn
Convi(128, 4, 2) Rel.u bn
Convt( 16, 4, 4, 2) + ReLa bn
Convi(644 4, 4, 2) ReLa bn
Convt(8, 3. + ReLu + bu
Convi(32, 3. 1)+
ReLu + bn
6.3 STRUCTURE OF TARGET MODEL USED MNIST
Table 7: Neural network architecture used on MNIST.
MNIST Model (Conv: convolutional layer, FC: fully connected layer.)
Conv(128, 3, 3) Tanh
MaxPool(2.22 Conv(64, 3, 3) + Tanh
MaxPool(2.2)
FC(128) Relu
FC(10) Softmax
6.4 ACCURACY OF TARGET MODELS ON ORIGINAL DATASETS
Table 8: Accuracy of each target model on each dataset
Ditasct
MNIST
CIFARIO
Imy-InngeNer
Model
MNIST Model
Resnet18
VGG19
Resnet34
Accuracy
0
0.9501
0.648]
0.6972
6.5 ADVERSARIAL EXAMPLES GENERATED BY OUR METHOD
13

---

Published as a conference paper at ICLR 2020
Target class
0
1
2
3
4
5
6
7
8
9
1
2
2
2
2
2
Q
2
2
2
3
3
3
3
3
3
3
3
3
3 4 5 6
4
4
4
4
4
4 5
5
8
5
5
5
5
5
5
6
6
6
6
6
6
6
6
6
7
8
8
8
8
8
8
8
9
9
9
9
99 9
Figure 4: Adversarial examples generated by our method on MNIST. The groundtruth images
shown in the diagonal and the rest are adversarial examples that are misclassified to the targeted class shown on the top.
14

---

Published as a conference paper at ICLR 2020
Target class
0
1
2
3
4
5
6
7
8
9
Figure 5: Adversarial examples generated by our method on CIFARIO. The groundtruth images shown the diagonal and the are adversarial examples that are misclassified the targeted class shown on the top.
15


Reviews:
[{"review_id": "rkxMuTp4cr", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed a meta attack approach that is capable of attacking a targeted model with fewer queries. It effectively utilizes meta learning approaches to abstract generalizable prior from previous observed attack patterns. Experiments on several benchmark datasets demonstrates it performance in reducing model queries.\n\n- The proposed meta learning approach seems to learn a universal gradient direction of an image x for all networks, which seems to be an ambitious goal. The authors did not provide any intuition or demonstrative explanations towards this. I hope the author could provide some more evidence showing that why this is achievable and is this indeed achieved by the meta learner (e.g., comparing the cosine similarity between the true gradient and the meta learner generated gradient?) \n\n- In Algorithm 2, the authors seem to use coordinate-wise zeroth-order gradient estimation as in ZOO. I wonder have the authors considered using NES type Gaussian noise to estimate the gradient? As it has been shown to be more efficient than coordinate-wise zeroth-order gradient estimation in (Ilyas et al. 2018).\n\n- I notice that the authors only consider L2 norm attack case and did not include more popular L-infinity norm case. Is there any reason why it can not be applied to L-infinity norm? I didn’t find anywhere in the algorithm that would restrict the choice of norm type. The author might also need to compare with the following recent papers regarding black-box attacks.\n\nYan, Ziang, Yiwen Guo, and Changshui Zhang. \"Subspace Attack: Exploiting Promising Subspaces for Query-Efficient Black-box Attacks.\" arXiv preprint arXiv:1906.04392 (2019).\nMoon, Seungyong, Gaon An, and Hyun Oh Song. \"Parsimonious Black-Box Adversarial Attacks via Efficient Combinatorial Optimization.\" ICML 2019.\nChen, Jinghui, Jinfeng Yi, and Quanquan Gu. \"A Frank-Wolfe Framework for Efficient and Effective Adversarial Attacks.\" arXiv preprint arXiv:1811.10828 (2018).\n\n- In experiments part, only 100 images from each dataset may not be representative enough. I would suggest the authors to test more samples. Also please consider only the images that can be correctly classified without perturbation, as there is no need to attack those already misclassified images.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "SyxMU8GpYr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThe authors of this paper propose a novel approach for query-efficient black-box adversarial attacks by leveraging meta-learning to approximate the gradient of given clean images. \n\nPaper strength:\n1.\tThe overall idea of this paper is interesting to improve the transferability of adversarial examples through meta-learning. \n2.\tThe paper is well-organized and easy to follow.\n3.\tAdequate experiment results demonstrate that the meta-attacker is efficient to decrease the number of queries for the adversarial attacks. It is interesting to see the meta attacker trained on certain source domain can be transferred to other domains.\n\nPaper weakness:\n\n1.\tThe authors miss some important details about training meta attackers like how many images you choose to get the image and its gradient pairs. Are the images from the training set or test set. Is there an overlap between the meta-attacker training set and the test set? \n2.\tWhat if you try to learn the adversarial perturbation directly instead of learning the gradient of images?\n3.\tHow about changing the meta learner to vanilla training and learning an autoencoder directly to map the image to its gradient? There could be a problem of how to incorporate the existing classifiers, but I think this could be an important baseline.\n4.\tThe authors try to use the gradient from the ZOO to guide the pre-trained meta attacker to adapt to a new classifier in Algorithm 2 (line 3). What if you take multi-step ZOO to give a much stronger prior to finetuning?\n5.\tTake a look at the confusion matrix in [1] which studies the transferability of white-box attack and black-box attack. I think it could be more interesting if the authors change the ground truth from gradient to estimated gradient from ZOO for learning meta-attacker. \n\n\nLi, Yandong, et al. \"NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks.\" arXiv preprint arXiv:1905.00441 (2019).\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Hyl8uEoTKr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors proposed a meta-learning based black-box adversarial attack method to reduce the number of queries. Their proposed meta attacker is trained via a meta-learning approach, and it could utilize historical query information and fast adapted to different target models. The authors conduct experiments in three datasets, and the results show that their proposed meta attacker is more efficient than other black-box attack methods. \n\nStrong points of this paper:\n1) novelty. The paper combines meta-learning and black-box attack and develops a deep convolutional model (meta attacker) to predict the gradients of another DNN model. Most of the other query-efficient attack methods focus on utilizing estimated gradients, while this paper focuses on predicting accurate gradients. That makes this paper novel. \n2) The results are good. The proposed method could attain comparable l2 norm and attack success rate with much fewer queries. And the analysis in experiments is interesting. The generalizability experiment of the meta-attacker(meta transfer) shows that gradients in different models and different datasets have some similar patterns. \n\nWeak points of this paper:\n1) The authors could investigate more in the theoretical part. For example, the authors could gives the theoretical support of why a convolutional network could stimulate other networks' gradient. \n2) The deploy of the meta-attacker is not as easy as other black-box attack methods. It would not be practical if we need to meta training the attacker before we use it. \n\nQuestions:\n1) Is it hard to meta training the meta-attacker? How long will it take for training? \n2) Is it necessary to finetune the meta attacker in algorithm 2 every m iteration? How to determine the \"m\"?\n3) Could the proposed meta-attacker embedded with other optimized black-box attack methods? \n\nOverall, this paper is well-structured, novel, and ideas are well motivated. The query-efficient black-box attack problem is practical and meaningful. I would encourage the authors to release their codes. Last, I would recommend acceptance for this paper. \n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a meta attack approach based on meta learning approaches to learn generalizable prior from the previously observed attack patterns. The proposed approach is able to attack targeted models with much fewer queries. After author response, all reviewers are very positive about the paper. Thus I recommend accept. 
Decision: Accept (Poster)