Q UERY - EFFICIENT M ETA A TTACK TO D EEP N EURAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Black-box attack methods aim to infer suitable attack patterns to targeted DNN models by only using output feedback of the models and the corresponding input queries.
However, due to lack of prior and inefﬁciency in leveraging the query and feedback information, existing methods are mostly query-intensive for obtaining effective attack patterns.
In this work, we propose a meta attack approach that is capable of attacking a targeted model with much fewer queries.
Its high queryefﬁciency stems from effective utilization of meta learning approaches in learning generalizable prior abstraction from the previously observed attack patterns and exploiting such prior to help infer attack patterns from only a few queries and outputs.
Extensive experiments on MNIST, CIFAR10 and tiny-Imagenet demonstrate that our meta-attack method can remarkably reduce the number of model queries without sacriﬁcing the attack performance.
Besides, the obtained meta attacker is not restricted to a particular model but can be used easily with a fast adaptive ability to attack a variety of models.
Our code will be released to the public.
I NTRODUCTION
Despite the great success in various tasks, deep neural networks (DNNs) are found to be susceptible to adversarial attacks and often suffer dramatic performance degradation in front of adversarial examples, even if only tiny and invisible noise is imposed on the input (Szegedy et al., 2014).
To investigate the safety and robustness of DNNs, many adversarial attack methods have been developed, which apply to either a white-box (Goodfellow et al., 2015;
Moosavi-Dezfooli et al., 2016; Carlini & Wagner, 2017; Madry et al., 2018) or a black-box setting (Papernot et al., 2017; Brendel et al., 2018; Narodytska & Kasiviswanathan, 2017).
In the white-box attack setting, the target model is transparent to the attacker and imperceptible adversarial noise can be easily crafted to mislead this model by leveraging its gradient information (Goodfellow et al., 2015).
In contrast, in the blackbox setting, the structure and parameters of the target DNN model are invisible, and the adversary can only access the input-output pair in each query.
With a sufﬁcient number of queries, black-box methods utilize the returned information to attack the target model generally by estimating gradient (Chen et al., 2017; Ilyas et al., 2018a; Narodytska & Kasiviswanathan, 2017; Cheng et al., 2019).
Black-box attack is more feasible in realistic scenarios than white-box attack but it is much more query-intensive.
Such a drawback is largely attributed to the fact that returned information for each queried example is sparse and limited.
During inferring attack patterns, existing black-box methods simply integrate the information between two sequential iterations brutally and ignore the implicit but profound message, thus not fully exploiting the returned information.
Although query-efﬁcient algorithms for generating attack examples are very meaningful in practice (Ilyas et al., 2018a), how to enhance query-efﬁciency for black-box attack remains underexplored.
In this work, we address a query-efﬁciency concerned attack problem.
Particularly, we consider only top- k probability scores accessible from the target black-box model.
With this practical but challenging scenario, we aim at three important objectives: lower query number, higher success rate and smaller noise magnitude.
We develop a meta-learning based attack method, which applies meta learning to obtaining prior information from the successful attack patterns, and uses the prior for efﬁcient optimization.
Speciﬁcally, we propose to train a meta attacker model through meta learning (Nichol et al., 2018), inspired by its success in solving few-shot learning problems.
We ﬁrst deploy several existing classiﬁcation models to get pairs of ( images, gradients ) with the max-margin logit classiﬁcation loss.
Then we use the data pairs of each classiﬁcation model to train the meta attacker.
After obtaining the attacker, we use it to attack a new black-box model for accelerating the search process for adversarial examples by optimizing it with coordinate-wise gradient estimation.
Different from previous methods, we use the estimated gradient not only to update adversarial noise but to ﬁne-tune the well-trained attacker.
After few-shot ﬁne-tuning, the attacker is able to simulate the gradient distribution of the target model.
We evaluate our method on MNIST, CIFAR10 and tiny-ImageNet datasets by comparing it with state-of-the-art black-box attack methods including Zoo (Chen et al., 2017), DecisionBoundary (Brendel et al., 2018), AutoZoom (Tu et al., 2019), Opt-attack (Cheng et al., 2019) and
Bandits (Ilyas et al., 2018b).
In both targeted and untargeted settings, our proposed method achieves comparable attack success rate and adversarial perturbation to all baselines but with a signiﬁcantly reduced query number.
The detailed experiment results demonstrate our superior query-efﬁciency.
R ELATED W ORK
Classical white-box attack methods include Fast-Gradient Sign Method (FGSM)
(Goodfellow et al., 2015), IFGSM (Madry et al., 2018), DeepFool (Moosavi-Dezfooli et al., 2016) and C&W attack (Carlini & Wagner, 2017), following a setting where detailed information about the target model (gradients and losses) is provided.
Comparatively, the black-box setting better accords with the real world scenarios in that little information about the target model is visible to the attacker.
The pioneer work on black-box attack (Papernot et al., 2017) tries to construct a substitute model with augmented data and transfer the black-box attack problem to a white-box one.
However, its attack performance is very poor due to the limited transferability of adversarial examples between two different models.
(Brendel et al., 2018) considers a more restricted case where only top-1 prediction classes are returned and proposes a random-walk based attack method around the decision boundary.
It dispenses class prediction scores and hence requires extensive model queries.
Zoo (Chen et al., 2017) is a black-box version of C&W attack, achieving a similar attack success rate and comparable visual quality as many white-box attack methods.
However, its coordinate-wise gradient estimation requires extensive model evaluations.
More recently, (Ilyas et al., 2018a) proposes a query-limited setting with L ∞ noise considered, and uses a natural evolution strategy (NES) to enhance query efﬁciency.
Though this method successfully controls the query number, the noise imposed is larger than average.
(Narodytska & Kasiviswanathan, 2017) proposes a novel local-search based technique to construct numerical approximation to the network gradient, which is then carefully used to construct a small set of pixels in an image to perturb.
It suffers a similar problem as in (Chen et al., 2017) for pixel-wise attack.
(Cheng et al., 2019) considers a hard-label black-box setting and formulates the problem as real-valued optimization that is solved by a zeroth order optimization algorithm.
Ilyas et al.
(2018b) reduce the queries by introducing two gradient priors, the time-independent prior and the data-dependent prior, and reformulating the optimization problem.
We then brieﬂy introduce some works on meta-learning related to our work.
Meta-learning is a process of learning how to learn.
A meta-learning algorithm takes in a distribution of tasks, each being a learning problem, and produces a quick learner that can generalize from a small number of examples.
Meta-learning is very popular recently for its fast adaptive ability.
MAML (Finn et al., 2017) is the ﬁrst to propose this idea.
Recently, a simpliﬁed algorithm Reptile (Nichol et al., 2018) which is an approximation to the ﬁrst-order MAML is proposed, achieving higher efﬁciency in computation and consuming less memory.
With these superior properties, meta learning is applied to adversarial attack methods (Zgner & Gnnemann, 2019; Edmunds et al., 2017).
Zgner & Gnnemann (2019) try to attack the structure of a graph model in the training process to decrease the model generalization performance.
Edmunds et al.
(2017) investigate the susceptibility of MAML to adversarial attacks and the transferability of the obtained meta model to a speciﬁc task.
M ETHOD 3. P RELIMINARIES : B LACK - BOX A TTACK S CHEMES
We ﬁrst formulate the black-box attack problem and introduce the widely used solutions.
We use ( x , t ) to denote the pair of a natural image and its true label, and ˆ x and ˆ t to denote the adversarial perturbed version of x and the returned label by the target classiﬁcation model M tar .
The black-box attack aims to ﬁnd an adversarial example ˆ x with imperceivable difference from x to fail the target model , i.e., ˆ t (cid:54) =
t through querying the target model for multiple times.
It can be formulated as
[Equation] [Equation]
Here (cid:107) · (cid:107) p denotes the (cid:96) p norm that measures how much perturbation is imposed.
M tar ( ˆ x ) is the returned logit or probability by the target model M tar .
The loss function (cid:96) ( ˆ x , M tar ( ˆ x ) , t ) measures the degree of certainty for model M tar assigning the input ˆ x into class t .
One common used adversarial loss is the probability of class t : (cid:96) ( ˆ x , M tar ( ˆ x ) , t ) =
p M tar ( t | ˆ x ) .
The ﬁrst constraint enforces high similarity between the clean image x and the adversarial one x adv and the second imposes a ﬁxed budget Q for the number of queries allowed in the optimization.
In the white-box attack setting, the adversary can access the true gradient ∇ ˆ x t (cid:96) ( ˆ x t ) and perform gradient descent ˆ x t +
=
ˆ x t − ∇ ˆ x t (cid:96) ( ˆ x t ) .
But in the black-box setting, the gradient information ∇ ˆ x t (cid:96) ( ˆ x t ) is not attainable.
In this case, the attacker can estimate the gradient using only queried information from model evaluation such as hard label, logits and probability scores.
This kind of estimator is the backbone of so-called zeroth-order optimization approaches (Chen et al., 2017;
Narodytska & Kasiviswanathan, 2017;
Tu et al., 2019; Ilyas et al., 2018a;b).
The estimation is done via ﬁnite difference method (Chen et al., 2017; Narodytska & Kasiviswanathan, 2017; Tu et al., 2019), which ﬁnds the k components of the gradient by estimating the inner products of the gradients with all the standard basis vector e 1 , ..., e k :
[Equation]
where step size h controls the quality of the estimated gradient.
Another strategy is to reformulate the loss function (Ilyas et al., 2018a;b).
Instead of computing the gradient of (cid:96) ( x ) itself, the expected value of loss function (cid:96) ( x ) under the search distribution is minimized and when the search distribution of random Guassian noise is adopted, the gradient estimation problem transfers into a zeroth-order estimation problem,
[Equation]
where n is the amount of noise sampled from the distribution.
After obtaining the estimated gradient, classical optimization algorithms (Nesterov, 2013; Johnson & Zhang, 2013) can be used to infer the adversarial examples.
Though the estimated gradient may not be accurate, it is still proved useful enough in adversarial attack.
The convergence of these zeroth-order methods is guaranteed under mild assumptions (Ghadimi & Lan, 2013; Nesterov & Spokoiny, 2017; Hazan et al., 2016).
Since each model evaluation consumes a query, naively applying the above gradient estimation to black-box attack is quite query expensive due to its coordinate or noise sampling nature.
Take the ﬁrst strategy on tiny-Imagenet dataset for example.
It consumes more than 20 , 000 queries for each image to obtain a full gradient estimate, which is not affordable in practice.
In this work, we address such a limitation via developing a query-efﬁcient meta-learning based attack model.
3. L EARNING OF M ETA A TTACKER
To reduce the query cost for black-box attack, we apply meta learning to training a meta attacker model, inspired by its recent success in few-shot learning problems (Finn et al., 2017; Nichol et al., 2018).
The meta attacker learns to extract useful prior information of the gradient of a variety of models w.r.t.
speciﬁc input samples.
It can infer the gradient for a new target model using only a few queries.
After obtaining such a meta attacker, we replace the zeroth-order gradient estimation in traditional black box attack methods with it to directly estimate the gradient.
We collect a set of existing classiﬁcation models M 1 , ..., M n to generate gradient information for universal meta attacker training.
Speciﬁcally, we feed each image x into the models M 1 , ..., M n respectively and compute losses (cid:96) 1 , ..., (cid:96) n by using following max-margin logit classiﬁcation loss:
[Equation]
Here t is the groundtruth label and j indexes other classes.
[ M i ( x )] t is the probability score of the true label predicted by the model M i , and [ M i ( x )]
j denotes the probability scores of other classes.
By performing one step back-propagation of losses (cid:96) 1 , ..., (cid:96) n w.r.t.
the input images x , the corresponding gradients g i = ∇ x (cid:96) i ( x ) , i = 1 , ..., n are obtained.
Finally, we collect n groups of data X = { x } , G i = { g i } , i = 1 , ..., n to train the universal meta attacker.
We design a meta attacker A which has a similar structure as an autoencoder, consisting of symmetric convolution and de-convolution layers and outputs a gradient map with the same size as the input.
Meta attacker model
A is parameterized with parameters θ .
Due to the intrinsic difference between selected classiﬁcation models, each obtained set ( X , G i ) is treated as a task T i in meta attacker training.
During the training process, for each iteration we only draw K samples from task T i and feedback the loss L i to update model parameters from θ to θ (cid:48)
i .
θ (cid:48)
i is then computed through one or multiple gradient descents: θ (cid:48)
i := θ − α ∇ θ L i ( A θ ) .
For a sensitive position of the meta attacker, the meta attacker parameters are optimized by combining each θ (cid:48)
i across all tasks {T i } i =1 ,...,n , following the update strategy of Reptile (Nichol et al., 2018) in meta learning,
[Equation]
We adopt mean-squared error (MSE) as the training loss in the inner update,
[Equation]
The set ( X s , G si ) denotes the K samples used for each inner update from θ to θ (cid:48)
i .
Since the number of K sampled each time is very small, the update strategy above tries to ﬁnd good meta attacker parameters θ as an initial point, from which the meta attacker model can fast adapt to new data distribution through gradient descent based ﬁne-tuning within limited samples.
Therefore, this characteristic can be naturally leveraged in attacking new black-box models by estimating their gradient information through a few queries.
Detailed training process of our meta attacker is described in
Algorithm 1.
3. Q UERY - EFFICIENT A TTACK VIA M ETA A TTACKER
An effective adversarial attack relies on optimizing the loss function equation 1 w.r.t.
the input image to ﬁnd the adversarial example of the target model M tar .
Differently, our proposed method applies the meta attacker A to predicting the gradient map of a test image directly.
We use the obtained meta attacker model A to predict useful gradient map for attacking, which should be ﬁne tuned to adapt to the new gradient distribution under our new target model.
Particularly, instead of ﬁnetuning once, for each given image x , we update A by leveraging query information with the following periodic scheme.
Suppose the given image is perturbed to x t ∈ R p at iteration t .
If ( t + 1) mod m = 0 , our method performs zeroth-order gradient estimation to obtain gradient map g t for ﬁne-tuning.
As each pixel value for estimated gradient map consumes two queries, for further saving queries, we just select q of the p coordinates to estimate, q (cid:28) p , instead of the full gradient map through all p coordinates.
The indexes of chosen coordinates are determined by
Perform zeroth order gradient estimation q coordinates denoted
I t obtain g
Fine meta g
I t g
I
; 5: else 6:
Generate the gradient map g directly meta attacker A x coordinates I
Update
9: if M
10: x adv
11: break; 12: else 13:
14: end if 15: end for the gradient map g t − 1 obtained in iteration t − 1 .
We sort the coordinate indexes by the value of g t −and select top- q indexes.
The set of these indexes are denoted as I t .
We feed image x t in iteration t into meta attacker A and compute the MSE loss on indexes I t , i.e. L = (cid:107)
[ A θ ( x t )]
I t − [ g t ] I t (cid:107) 22 .
Then we perform gradient descent for the MSE loss with a few steps to update the parameters θ of meta attacker A .
For the rest iterations, we just use the periodically updated attacker A θ to directly generate the gradient g t =
A θ ( x t ) .
When we have the estimated gradient map g t in iteration t , we update to get the adversarial sample x (cid:48) t by [ x (cid:48) t ] I t =
[ x t ] I t + β
[ g t ] I t where β is a hyperparameter to be tuned.
The details are summarized in Algorithm 2.
In our method, the following operations contribute to reducing the query number needed by the attacker.
First, though we just use q coordinates to ﬁne-tune our meta attacker A every m iterations, the meta attacker A is trained to ensure that it can abstract the gradient distribution of different x t and learn to predict the gradient from a few samples with simple ﬁne-tuning.
Secondly, the most query-consuming part lies in zeroth-order gradient estimation, due to its coordinate-wise nature.
In our algorithm, we only do this every m iterations.
When we use the ﬁnetuned meta attacker A directly, no query is consumed in gradient estimation in these iterations.
Intuitively, larger m implies less gradient estimation computation and fewer queries.
Besides, just as mentioned above, even in zeroth-order gradient estimation, only top- q coordinates are required.
Normally, q is much smaller than dimension p of the input.
E XPERIMENTS
We compare our meta attacker with state-of-the-art black-box attack methods including Zoo (Chen et al., 2017), Decision-Boundary (Brendel et al., 2018), AutoZoom (Tu et al., 2019), Optattack (Cheng et al., 2019), FW-black (Chen et al., 2018) and Bandits (Ilyas et al., 2018b) to evaluate its query efﬁciency.
We also study its generalizability and transferability through a meta guided attacker and a meta transfer attacker, as detailed in the following sections.
4. S ETTINGS
Datasets and Target Models We evaluate the attack performance on MNIST (LeCun, 1998) for handwritten digit recognition, CIFAR10 (Krizhevsky & Hinton, 2009) and tiny-Imagenet (Russakovsky et al., 2015) for object classiﬁcation.
The architecture details of meta attack models on
MNIST, CIFAR10 and tiny-Imagenet are given in Table 6.
For MNIST, we train a separate meta attacker model since the images have different channel numbers from other natural image datasets.
For CIFAR10 and tiny-Imagenet, we use a common meta attacker model.
On CIFAR-10, we choose
ResNet18 (He et al., 2016) as the target model M tar and use VGG13, VGG16 (Simonyan & Zisser- man, 2014) and GoogleNet (Szegedy et al., 2015) for training our meta attacker.
On tiny-Imagenet, we choose VGG19 and ResNet34 as the target model separately, and use VGG13, VGG16 and
ResNet18 for training the meta attacker together.
Attack Protocols For a target black-box model M tar , obtaining a pair of ( input-output ) is considered as one query.
We use the mis-classiﬁcation rate as attack success rate; we randomly selectimages from each dataset as test images.
To evaluate overall noise added by the attack methods, we use the mean L 2 distance across all the samples noise ( M tar )
= 1 n (cid:80) ni =1 (cid:107) x advi, A , M tar − x i (cid:107) 2 , where x advi, A , M tar denotes the adversarial version for the authentic sample x i .
Meta-training Details For all the experiments
, we use the same architecture for the meta attacker
A as shown in Table 6.
We use Reptile (Nichol et al., 2018) with 0 .
01 learning rate to train meta attackers.
We use 10000 randomly selected images from the training set to train the meta-attackers in three datasets.
The proportion of the selected images to the whole training set are 16%, 20%, and 10% respectively.
Fine-tuning parameters are set as m = 5 for MNIST and CIFAR10, and m =for tiny-Imagenet.
Top q = 128 coordinates are selected as part coordinates for attacker ﬁne-tuning and model attacking on MNIST; and q = 500 on CIFAR10 and tiny-Imagenet.
4. C OMPARISON WITH B ASELINES
We compare our meta attacker with baselines for both the untargeted and targeted black-box attack on the three datasets.
The results are reported in detail as below.
Untargeted Attack Untargeted attack aims to generate adversarial examples that would be misclassiﬁed by the attacked model into any category different from the ground truth one.
The overall results are shown in Table 1, in which, the Meta transfer denotes that meta attacker trained on one dataset is used to attack target models on another dataset and Meta guided denotes using the output of meta attacker to guide the update of original Zoo method.
Our method is competitive with baselines in terms of adversarial perturbation and success rate, but our query number is reduced.
We also compare the results of our method with Zoo (Chen et al., 2017) and AutoZoom (Tu et al., 2019) from a query-efﬁciency perspective.
We use these models to conduct untargeted attack on
CIFAR10 and tiny-Imagenet by limiting a maximum number of queries for each adversarial example and compare their success rate.
The results are shown in Fig.
We notice that for different query thresholds, the success rate of our method is always higher than Zoo and AutoZoom.
This is possibly because the testing samples have different L 2 distances to the decision boundary.
Higher success rate of our method indicates our meta attacker can predict correct gradient even when the query information is limited.
These results give strong evidence on effectiveness of our proposed method for enhancing query efﬁciency.
[Figure]
[Figure]
Targeted Attack Targeted attack aims to generate adversarial noise such that the perturbed sample would be mis-classiﬁed into any pre-speciﬁed category.
It is a more strict setting than the untargeted one.
For fair comparison
[Figure]
we deﬁne the target label for each sample —a sample with label (cid:96) gets the target label ( (cid:96) +1) mod # classes .
We deploy our meta attacker the same as above.
The results on
MNIST, CIFAR10 and tiny-ImageNet are shown in Table 2.
Similar to results of untargeted attack, we achieve comparable noise and success rate to baselines but with reduced query numbers.
4. M ODEL A NALYSIS
[Table]
Meta Training We ﬁrst test the beneﬁts of meta training by comparing performance of a metatrained attacker with a Gaussian randomly initialized attacker without meta training on the three datasets.
Fig.
3 shows their success rate, L 2 distortion and query count results for initial success.
The meta pre-trained attacker achieves averagely 7% higher success rate with 16% lower L 2 distortion and 30% less queries, compared with the randomly initialized one.
This justiﬁes the contributions of meta training to enhancing query efﬁciency and also attack performance.
Guaranteed by ﬁne-tuning, the randomly initialized attacker succeeds over many testing samples.
The ﬁne-tuning works like an inner training in meta training.
With sufﬁcient ﬁne-tuning iterations, the randomly initialized attacker functions like a well-trained meta attacker.
This explains the effectiveness of the randomly initialized meta attacker on many testing samples compromised by more queries.
However, it could not predict gradient as accurate as the well-trained meta attacker during earlier iterations.
Such inaccuracy leads to larger L 2 distortion at the beginning.
On the contrary, the meta training process enables the well-trained meta attacker to fast-adapt to current testing samples.
These results highlight the signiﬁcant advantages of our meta model towards to black-box attack.
The process of meta training makes it familiar with gradient patterns of various models.
Generalizability Here we show that our meta attacker trained on one dataset can be transferred to other datasets.
We conduct this experiment between CIFAR10 and tiny-Imagenet, denoted as meta transfer attacker in Table 1 and 2.
We ﬁrst apply the meta attacker trained on CIFAR10 to attack
VGG19, ResNet34 on tiny-Imagenet respectively, which are different from models used for training meta attacker.
Note the meta attacker tested on CIFAR10 has no privileged prior and is not familiar with neither tiny-Imagenet dataset nor the corresponding classiﬁcation models.
Similarly, we also use the meta attacker trained on tiny-Imagenet to attack the target ResNet18 model on CIFAR10. The results show the good generalizability and robustness of our proposed meta attacker.
[Table]
[Figure]
et
[Figure]
Parameters Selection
We test the selection of top- q and β on CIFAR10.
We choose q ranging from 350 to 600 and give the results with β ranging from 3e-3 to 5e-3, as shown in Fig.
When q increases, query number, success rate and L 2 will all increase.
In order to balance the overall result, we choose q to be 500.
When β increases, success rate and L 2 will increase and query number decreases.
In order to balance success rate and query, we choose β to be 4e-3 in the experiment [Figure]
C ONCLUSION
We propose a meta-based black-box attack method that largely reduces demanded query numbers without compromising in attack success rate and distortion.
We train a meta attacker to learn useful prior information about gradient and incorporate it into the optimization process to decrease the number of queries.
Speciﬁcally, the meta attacker is ﬁnetuned to ﬁt the gradient distribution of target model and each update is based on the output of ﬁnetuned meta attaker.
Extensive experimental results conﬁrm the superior query-efﬁciency of our method over baselines.
A PPENDIX 6. M ORE EXPERIMENTAL RESULTS 6.1. C OSINE SIMILARITY BETWEEN ESTIMATED GRADIENTS AND WHITE - BOX GRADIENTS
To demonstrate the ability of estimating gradient of our meta attacker
[Table]
we have conducted experiments with the Resnet-34 model on the tiny-Imagenet dataset to compare the cosine similarity between the estimated gradients from our proposed meta-attacker and the accurate white-box gradients.
We also compare the cosine similarity between ZOO (Chen et al., 2017) estimated gradients and the white-box gradients as reference.
The results of cosine similarity and required number of queries are shown in Table 3.
We can observe that our meta-attacker does fast-adapt to the target model and generate accurate gradients.
Not only it estimates gradients with positive cosine similarity to the true gradients, but it also performs closer to the ZOO estimated results with the same small standard deviation.
[Table]
V ANILLA TRAINING AUTOENCODER AND META - ATTACKER LEARNING FROM ESTIMATED GRADIENTS
We have conducted experiments in section 4.3 to demonstrate the beneﬁts of meta training.
However, we only compare the performance of a meta-trained attacker with a Gaussian randomly initialized attacker.
We conduct two more experiments on meta training here to further investigate the beneﬁts of meta training.
First we compare our meta-trained attacker with a vanilla autoencoder that learns to map images to gradients of one white-box model.
The vanilla autoencoder has the same architecture with our meta-attacker, but it is trained in one white-box model.
Then, we train a new meta-attacker in four black-box models, i.e., the gradients used for training are estimated via ZOO (Chen et al., 2017).
The experiment results are presented in Table 4, 5 .
[Table] [Table]
The two experiments are conducted on 1000 randomly selected images from the MNIST testing set.
As for the four approach settings: ”Ours (reported)” is the results we report in our paper; ”ZOO estimated” is the meta-attacker trained from ZOO estimated gradients; ”Vanilla Autoencoder” is the autoencoder maps image to gradient trained in one different MNIST classiﬁcation model; ”Initialised attacker” is the meta-attacker with randomly initialized weights.
We can see the ”ZOO estimated” model performs closer to our reported meta-attacker.
However, the ”Vanilla Autoencoder” performs much worse (with larger L2-norm and more queries), which performs similarly to randomly initialized meta-attacker.
The two experiments verify the effectiveness of our ﬁrst meta-training phase.
TRUCTURE OF
[Table]
S
OF
ARGET
ODEL
SED IN MNIST
[Table]
A CCURACY OF T ARGET M ODELS ON O RIGINAL D ATASETS
[Table]
CIFAR tiny
[Table]
6. A DVERSARIAL E XAMPLES G ENERATED BY O UR M ETHOD