B LACK - BOX A DVERSARIAL A TTACKS WITH B AYESIAN O PTIMIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
We focus on the problem of black-box adversarial attacks, where the aim is to generate adversarial examples using information limited to loss function evaluations of input-output pairs.
We use Bayesian optimization (BO) to speciﬁcally cater to scenarios involving low query budgets to develop query efﬁcient adversarial attacks.
We alleviate the issues surrounding BO in regards to optimizing high dimensional deep learning models by effective dimension upsampling techniques.
Our proposed approach achieves performance comparable to the state of the art black-box adversarial attacks albeit with a much lower average query count.
In particular, in low query budget regimes, our proposed method reduces the query count up to 80% with respect to the state of the art methods.
I NTRODUCTION
Neural networks are now well-known to be vulnerable to adversarial examples : additive perturbations that, when applied to the input, change the network’s output classiﬁcation (Goodfellow et al., 2014).
Work investigating this lack of robustness to adversarial examples often takes the form of a back-and-forth between newly proposed adversarial attacks , methods for quickly and efﬁciently crafting adversarial examples, and corresponding defenses that modify the classiﬁer at either training or test time to improve robustness.
The most successful adversarial attacks use gradient-based optimization methods (Goodfellow et al., 2014; Madry et al., 2017), which require complete knowledge of the architecture and parameters of the target network; this assumption is referred to as the white-box attack setting.
Conversely, the more realistic black-box setting requires an attacker to ﬁnd an adversarial perturbation without such knowledge: information about the network can be obtained only through querying the target network, i.e., supplying an input to the network and receiving the corresponding output.
In real-world scenarios, it is extremely improbable for an attacker to have unlimited bandwidth to query a target classiﬁer.
In evaluation of black box attacks, this constraint is usually formalized via the introduction of a query budget : a maximum number of queries allowed to the model per input, after which an attack is considered to be unsuccessful.
Several recent papers have proposed attacks speciﬁcally to operate in this query-limited context (Ilyas et al., 2019; 2018;
Chen et al., 2017;
Tu et al., 2019; Moon et al., 2019); nevertheless, these papers typically consider query budgets on the order of 10,000 or 100,000.
This leaves open questions as to whether black-box attacks can successfully attack a deep network based classiﬁer in severely query limited settings, e.g., with a query budget of 100-200.
In such a query limited regime, it is natural for an attacker to use the entire query budget, so we ask the pertinent question:
In a constrained query limited setting, can one design query efﬁcient yet successful black box adversarial attacks?
This work proposes a black-box attack method grounded in Bayesian optimization (Jones et al., 1998;
Frazier, 2018), which has recently emerged as a state of the art black-box optimization technique in settings where minimizing the number of queries is of paramount importance.
Straightforward application of Bayesian optimization to the problem of ﬁnding adversarial examples is not feasible: the input dimension of even a small neural network-based image classiﬁer is orders of magnitude larger than the standard use case for Bayesian optimization.
Rather, we show that we can bridge this gap by performing Bayesian optimization in a reduced-dimension setting and upsampling to obtain our ﬁnal perturbation.
We explore several upsampling techniques and ﬁnd that a relatively simple nearest-neighbor upsampling method allows us to sufﬁciently reduce the optimization prob-
lem dimension such that Bayesian optimization can ﬁnd adversarial perturbations with more success than existing black-box attacks in query-constrained settings.
We compare the efﬁcacy of our adversarial attack with a set of experiments attacking three of the most commonly used pretrained ImageNet (Deng et al., 2009) classiﬁers: ResNet50 (He et al., 2015), Inception-v3 (Szegedy et al., 2015), and VGG16-bn (Simonyan & Zisserman, 2014).
Results from these experiments show that with very small query budgets (under 200 queries), the proposed method B AYES -A TTACK achieves success rates comparable to or exceeding existing methods, and does so with far smaller average and median query counts.
Further experiments are performed on the MNIST dataset to compare how various upsampling techniques affect the attack accuracy of our method.
Given these results we argue that, despite being a simple approach (indeed, largely because it is such a simple and standard approach for black-box optimization), Bayesian Optimization should be a standard baseline for any black-box adversarial attack task in the future, especially in the small query budget regime.
R ELATED W ORK
Within the black-box setting, adversarial attacks can be further categorized by the exact nature of the information received from a query.
The most closely related work to our approach are score-based attacks, where queries to the network return the entire output layer of the network, either as logits or probabilities.
Within this category, existing approaches draw from a variety of optimization ﬁelds and techniques.
One popular approach in this area is to attack with zeroth-order methods via some method of derivative-free gradient estimation, as in methods proposed in Ilyas et al.
(2019), which uses time-dependent and data-dependent priors to improve the estimate, as well as Ilyas et al.
(2018), which replaces the gradient direction found using natural evolution strategies (NES).
Other methods search for the best perturbation outside of this paradigm; Moon et al.
(2019) cast the problem of ﬁnding an adversarial perturbation as a discrete optimization problem and use local search methods to solve.
These works all search for adversarial perturbations within a search space with a hard constraint on perturbation size; other work (Chen et al., 2017; Tu et al., 2019) incorporates a soft version of this constraint and performs coordinate descent to decrease the perturbation size while keeping the perturbed image misclassiﬁed.
The latter of these methods incorporates an autoencoderbased upsampling method with which we compare in Section 5.3.1.
One may instead assume that only part of the information from the network’s output layer is received as the result of a query.
This can take the form of only receiving the output of the top k predicted classes (Ilyas et al., 2018), but more often the restrictive decision-based setting is considered.
Here, queries yield only the predicted class, with no probability information.
The most successful work in this area is in Cheng et al.
(2018), which reformulates the problem as a search for the direction of the nearest decision boundary and solves using a random gradient-free method, and in Brendel et al.
(2017) and Chen et al.
(2019), both of which use random walks along the decision boundary to perform an attack.
The latter work signiﬁcantly improves over the former with respect to query efﬁciency, but the number of queries required to produce adversarial examples with small perturbations in this setting remains in the tens of thousands.
A separate class of transfer-based attacks train a second, fully-observable substitute network, attack this network with white-box methods, and transfer these attacks to the original target network.
These may fall into one of the preceding categories or exist outside of the distinction: in Papernot et al.
(2016), the substitute model is built with score-based queries to the target network, whereas Liu et al.
(2016) trains an ensemble of models without directly querying the network at all.
These methods come with their own drawbacks: they require training a substitute model, which may be costly or time-consuming, and overall attack success tends to be lower than that of gradient-based methods.
Finally, there has been some recent interest in leveraging Bayesian optimization for constructing adversarial perturbations.
Bayesian optimization (BO) has played a supporting role in several methods. For example, Zhao et al.
(2019) use BO to solve the δ -step of an alternating direction of method multipliers (ADMM) approach, Co et al.
(2018) search within a set of procedural noise perturbations using BO and Gopakumar et al.
(2018) use BO to ﬁnd maximal distortion error by optimizing perturbations deﬁned using 3 parameters.
On the other hand, prior work in which Bayesian optimization plays a central role performs experiments only in relatively low-dimensional problems, highlighting the main challenge of its application: Suya et al.
(2017) examines an attack on a spam email classiﬁer with 57 input features, and in Co (2017) image classiﬁers are attacked but notably do not scale beyond MNIST classiﬁers.
In contrast to these past works, the main contribution of this paper is to show that Bayesian Optimization presents a scalable, query-efﬁcient approach for large-scale black-box adversarial attacks, when combined with upsampling procedures.
P ROBLEM F ORMULATION
The following notation and deﬁnitions will be used throughout the remainder of the paper.
Let F be the target neural network.
We assume that F : R d → [0 , 1] K is a K -class image classiﬁer that takes normalized inputs: each dimension of an input x ∈ R d represents a single pixel and is bounded between 0 and 1 , y ∈ { 1 , · · · K } denotes the original label, and the corresponding output F ( x ) is a
K -dimensional vector representing a probability distribution over classes.
Rigorous evaluation of an adversarial attack requires careful deﬁnition of a threat model : a set of formal assumptions about the goals, knowledge, and capabilities of an attacker (Carlini & Wagner, 2017).
We assume that, given a correctly classiﬁed input image x , the goal of the attacker is to ﬁnd a perturbation δ such that x + δ is misclassiﬁed, i.e., arg max k F ( x + δ ) (cid:54) =
arg max k F ( x ) .
We operate in the score-based black-box setting, where we have no knowledge of the internal workings of the network, and a query to the network F yields the entire corresponding K -dimensional output.
To enforce the notion that the adversarial perturbation should be small, we take the common approach of requiring that (cid:107) δ (cid:107) p be smaller than a given threshold (cid:15) in some (cid:96) p norm, where (cid:15) varies depending on the classiﬁer.
This work considers the (cid:96) ∞ norm, but our attack can easily be adapted to other norms.
Finally, we denote the query budget with t ; if an adversarial example is not found after t queries to the target network, the attack fails.
As in most work, we pose the attack as a constrained optimization problem.
We use an objective function suggested by Carlini & Wagner (2017) and used in Tu et al.
; Chen et al.
(2017): max δ f ( x , y, δ ) subject to (cid:107) δ (cid:107) p ≤ (cid:15) and ( x + δ ) ∈ [0 , 1] d , (1) where
[Equation]
Most importantly, the input x + δ to f is an adversarial example for F if and only if f ( x , y, δ ) > 0 .
We brieﬂy note that the above threat model and objective function were chosen for simplicity and for ease of directly comparing with other black box attacks, but the attack method we propose is compatible with many other threat models.
For example, we may change the goals of the attacker or measure δ in (cid:96) 1 or (cid:96) 2 norms instead of (cid:96) ∞ with appropriate modiﬁcations to the objective function and constraints in equation 1.
M ODEL F RAMEWORK
In this section, we present the proposed black-box attack method.
We begin with a brief description of Bayesian optimization (Jones et al., 1998) followed by its application to generate black-box adversarial examples.
Finally, we describe our method for attacking a classiﬁer trained with highdimensional inputs (e.g. ImageNet) in a query-efﬁcient manner.
4. B AYESIAN O PTIMIZATION
Bayesian Optimization (BO) is a method for black box optimization particularly suited to problems with low dimension and expensive queries.
Bayesian Optimization consists of two main components: a Bayesian statistical model and an acquisition function.
The Bayesian statistical model, also referred to as the surrogate model, is used for approximating the objective function: it provides a
Bayesian posterior probability distribution that describes potential values for the objective function at any candidate point.
This posterior distribution is updated each time we query the objective function at a new point.
The most common surrogate model for Bayesian optimization are Gaussian processes (GPs) (Rasmussen & Williams, 2005), which deﬁne a prior over functions that are cheap to evaluate and are updated as and when new information from queries becomes available.
We model the objective function h using a GP with prior distribution N ( µ 0 , Σ 0 ) with constant mean function µ 0 and Matern kernel (Shahriari et al., 2016; Snoek et al., 2012) as the covariance function Σ 0 , which is deﬁned as:
[Equation]
where d (cid:48) is the dimension of input and { θ i } d (cid:48)
i =0 and µ 0 are hyperparameters.
We select hyperparameters that maximize the posterior of the observations under a prior (Shahriari et al., 2016; Frazier, 2018).
The second component, the acquisition function A , assigns a value to each point that represents the utility of querying the model at this point given the surrogate model.
We sample the objective function h at x n = arg max x A ( x |D 1: n − 1 ) where D 1: n − 1 comprises of n − 1 samples drawn from h so far.
Although this itself may be a hard (non-convex) optimization problem to solve, in practice we use a standard approach and approximately optimize this objective using the LBFGS algorithm.
There are several popular choices of acquisition function; we use expected improvement (EI) (Jones et al., 1998), which is deﬁned as
[Equation]
where E n [ · ] =
E [ ·|D 1: n − 1 ] denotes the expectation taken over the posterior distribution given evaluations of h at x 1 , · · · , x n − 1 , and h ∗ n is the best value observed so far.
Bayesian optimization framework as shown in Algorithm 2 runs these two steps iteratively for the given budget of function evaluations.
It updates the posterior probability distribution on the objective function using all the available data.
Then, it ﬁnds the next sampling point by optimizing the acquisition function over the current posterior distribution of GP.
The objective function h is evaluated at this chosen point and the whole process repeats.
In theory, we may apply Bayesian optimization directly to the optimization problem in equationto obtain an adversarial example, stopping once we ﬁnd a point where the the objective function rises above 0 .
In practice, Bayesian optimization’s speed and overall performance fall dramatically as the input dimension of the problem increases.
This makes running Bayesian optimization over high dimensional inputs such as ImageNet (input dimension 3 × 299 × 299 = 268203 ) practically infeasible; we therefore require a method for reducing the dimension of this optimization problem.
4. B AYES -A TTACK : G ENERATING A DVERSARIAL E XAMPLES USING B AYESIAN O PTIMIZATION
Images tend to exhibit spatial local similarity i.e. pixels that are close to each other tend to be similar.
Ilyas et al.
(2019) showed that this similarity also extends to gradients and used this to reduce query complexity.
Our method uses this data dependent prior to reduce the search dimension of the perturbation.
We show that the adversarial perturbations also exhibit spatial local similarity and we do not need to learn the adversarial perturbation conforming to the actual dimensions of the image.
Instead, we learn the perturbation in a much lower dimension.
We obtain our ﬁnal adversarial perturbation by interpolating the learned, low-dimension perturbation to the original input dimension.
We deﬁne the objective function for running the Bayesian optimization in low dimension in Algorithm 1.
We let Π pB ( 0 ,(cid:15) ) be the projection onto the (cid:96) ∞ ball of radius (cid:15) centered at origin.
Our method ﬁnds a low dimension perturbation and upsamples to obtain the adversarial perturbation.
Since this upsampled image may not lie inside the ball of radius (cid:15) centered at the origin, we project back to ensure (cid:107) δ (cid:107) ∞ remains bounded by (cid:15) .
With the perturbation δ in hand, we compute the objective function of the original optimization problem deﬁned in equation 1.
We describe the complete algorithm our complete framework in Algorithm 2 where x 0 ∈ R d and y 0 ∈ { 1 , . .
, K } denote the original input image and label respectively.
The goal is to learn an adversarial perturbation δ ∈ R d (cid:48) in much lower dimension, i.e., d (cid:48) << d .
We begin with a small dataset D
= { ( δ 1 , v 1 ) , · · · , ( δ n 0 , v n 0 ) } where each δ n is a d (cid:48) dimensional vector sampled from a
1: procedure O BJ -F UNC ( x 0 , y 0 , δ ) 2: // (cid:15) is the given perturbation 3: δ (cid:48)
← Upsample ( δ ) (cid:46)
Upsampling low dimension perturbation to input dimension 4: δ (cid:48) ← Π pB ( 0 ,(cid:15) )
( δ (cid:48) )
(cid:46)
Projecting perturbation on (cid:96) p -ball around x5: v ← f ( x 0 , y 0 , δ (cid:48) )
(cid:46) Quering the model 6: return v
1: procedure B AYES -A TTACK ( x 0 , y 0 ) 2:
D = { ( δ 1 , v 1 ) , · · · , ( δ n 0 , v n 0 ) } (cid:46) Quering randomly chosen n 0 points.
Update the GP on D (cid:46) Updating posterior distribution using available points 4: t ← n(cid:46)
Updating number of queries till now 5: while t ≤ T do 6: δ t ← arg max δ A ( δ | D ) (cid:46)
Optimizing the acquisition function over the GP 7: v t ← O BJ -F UNC ( x 0 , y 0 , δ ) (cid:46)
Querying the model 8: t ← t +9: if v t ≤ 0 then 10:
D ← D ∪ ( δ t , v t ) and update the GP (cid:46) Updating posterior distribution 11: else 12: return δ t (cid:46) Adversarial attack successful 13: return δ t (cid:46) Adversarial attack unsuccessful given distribution and v n is the function evaluation at δ n i.e v n = O BJ -F UNC ( x 0 , y 0 , δ n ) . We iteratively update the posterior distribution of the GP using all available data and query new perturbations obtained by maximizing the acquisition function over the current posterior distribution of GP until we ﬁnd an adversarial perturbation or run out of query budget.
The Bayesian optimization iterations run in low dimension d (cid:48) but for querying the model we upsample, project and then add the perturbation to the original image as shown in Algorithm 1 to get the perturbed image to conform to the input space of the model.
To generate a successful adversarial perturbation, it is necessary and sufﬁcient to have v t > 0 , as described in Section 3.
We call our attack successful with t queries to the model if the Bayesian optimization loop exits after t iterations (line 12 in Algorithm 2), otherwise it is unsuccessful.
Finally, we note that the ﬁnal adversarial image can be obtained by upsampling the learned perturbation and adding to the original image as shown in Figure 1.
In this work, we focus on (cid:96) ∞ -norm perturbations, where projection is deﬁned as:
[Equation]
where (cid:15) is the given perturbation bound.
The upsampling method can be linear or non-linear.
In this work, we conduct experiments using nearest neighbor upsampling.
A variational autoencoder (Kingma & Welling, 2014) or vanilla autoencoder could also be trained to map the low dimension perturbation to the original input space.
We compare these different upsampling schemes in Section 5.3.1.
The initial choice of the dataset D to form a prior can be done using standard normal distribution, uniform distribution or even in a deterministic manner (e.g. with Sobol sequences).
E XPERIMENTS
Our experiments focus on the untargeted attack setting where the goal is to perturb the original image originally classiﬁed correctly by the classiﬁcation model to cause misclassiﬁcation.
We primarily consider performance of B AYES -A TTACK on ImageNet classiﬁers and compare its performance to other black-box attacks in terms of success rate over a given query budget.
We also perform ablation studies on the MNIST dataset (Lecun et al., 1998) by examining different upsampling techniques and varying the latent dimension d (cid:48) of the optimization problem.
We deﬁne success rate as the ratio of the number of images successfully perturbed for a given query budget to the total number of input images.
In all experiments, images that are already misclassiﬁed
[Figure]
by the target network are excluded from the test set; only images that are initially classiﬁed with the correct label are attacked.
For each method of attack and each target network, we compute the average and median number of queries used to attack among images that were successfully perturbed.
5. E MPIRICAL P ROTOCOLS
We treat the latent dimension d (cid:48) used for running the Bayesian optimization loop as a hyperparameter.
For MNIST, we tune the latent dimension d (cid:48) over { 16 , 64 , 100 , 256 , 784 } .
Note that 784 is the original input dimension for MNIST.
While for ImageNet, we search for latent dimension d (cid:48) and shape over the range { 48(3 × 4 × 4) , 49(1 × 7 × 7) , 100(1 × 10 × 10) , 108(3 × 6 × 6) , 400(1 × 20 × 20) , 432(3 × 12 × 12) , 576(1 × 24 × 24) , 588(3 × 14 × 14) , 961(1 × 31 × 31) , 972(3 × 18 × 18) } .
For
ImageNet, the latent shapes with ﬁrst dimension as 1 indicate that the same perturbation is added to all three channels while the ones with 3 indicate that the perturbation across channels are different.
In case of ImageNet, we found that for ResNet50 and VGG16-bn different perturbation across channels work much better than adding the same perturbation across channels.
While for Inception-v3, both seem to work equally well.
We initialize the GP with n 0 = 5 samples sampled from a standard normal distribution.
For all the experiments in next section, we use expected improvement as the acquisition function.
We also examined other acquisition functions (posterior mean, probability of improvement, upper conﬁdence bound) and observed that our method works equally well with other acquisition functions.
We independently tune the hyper-parameters on a small validation set and exclude it from our ﬁnal test set.
We used BoTorch 1 packages for implementation.
5. E XPERIMENTS ON I MAGE N ET
We compare the performance of the proposed method B AYES -A TTACK against NES (Ilyas et al., 2018), B ANDITS - TD (Ilyas et al., 2019) and P ARSIMONIOUS (Moon et al., 2019), which is the current state of the art among score-based black-box attacks within the (cid:96) ∞ threat model.
On ImageNet, we attack the pretrained 2 ResNet50 (He et al., 2015), Inception-v3 (Szegedy et al., 2015) and
VGG16-bn (Simonyan & Zisserman, 2014).
We use 10,000 randomly selected images (normalized to [0, 1]) from the ImageNet validation set that were initially correctly classiﬁed.
We set the (cid:96) ∞ perturbation bound (cid:15) to 0 .
and evaluate the performance of all the methods for low query budgets.
We use the implementation 3 and hyperparameters provided by Ilyas et al.
for 1 https org/
Pretrained models available at https://pytorch
[Figure]
[Table]
NES and B ANDITS - TD .
Similarly for P ARSIMONIOUS , we use the implementation 4 and hyperparameters given by Moon et al.
Figure 2 compares the performance of the proposed method B AYES -A TTACK against the set of baseline methods in terms of success rate at different query budgets.
We can see that B AYES -A TTACK consistently performs better than baseline methods for query budgets < 200 .
Even for query budgets > 200 , B AYES -A TTACK achieves better success rates than B ANDITS - TD and NES on ResNet and VGG16-bn.
Finally, we note that for higher query budgets ( > 1000) , both P ARSIMONIOUS and
B ANDITS - TD method perform better than B AYES -A TTACK .
To compare the success rate and average/median query, we select a point on the plots shown in
Figure 2.
Table 1 compares the performance of all the methods in terms of success rate, average and median query for a query budget of 200 .
We can see that B AYES -A TTACK achieves higher success rate with 80% less average queries as compared to the next best P ARSIMONIOUS method.
Thus, we argue that although the Bayesian Optimization adversarial attack approach is to some extent a “standard” application of traditional Bayesian Optimization methods, the performance over the existing state of the art makes it a compelling approach particularly for the very low query setting.
We also compare the average (cid:96) 2 distortion of the generated adversarial perturbations in Appendix B.
[Figure]
[Figure]
5. E XPERIMENTS ON MNIST
For MNIST, we use the pretrained network (used in Carlini & Wagner (2017)) with 4 convolutional layers, 2 max-pooling layers and 2 fully-connected layers which achieves 99 .
5% accuracy on
MNIST test set.
We conduct (cid:96) ∞ untargeted adversarial attacks with perturbation bound (cid:15)
= 0 .
2 on a randomly sampled 1000 images from the test set.
All the experiments performed on MNIST follow the same protocols.
5.3. U PSAMPLING M ETHODS
The proposed method requires an upsampling technique for mapping the perturbation learnt in the latent dimension to the original input dimension.
In this section, we examine different linear and non-linear upsampling schemes and compare their performance on MNIST.
The approaches we consider here can be divided into two broad groups: Encoder-Decoder based methods and Interpolation methods.
For interpolation-based methods, we consider nearest-neighbor, bilinear and bicubic interpolation.
For encoder-decoder based approaches, we train a variational autoencoder (Kingma & Welling, 2014; Rezende et al., 2014) by maximizing a variational lower bound on the log marginal likelihood.
We also consider a simple autoencoder trained by minimizing the mean squared loss between the generated image and the original image.
For both the approaches, we run the Bayesian optimization loop in latent space and use the pretrained decoder (or generator) for mapping the latent vector into image space.
For these approaches, rather than searching for adversarial perturbation δ in the latent space, we learn the adversarial image x
+ δ directly using the Bayesian optimization.
Figure 3a compares the performance of different upsampling methods.
We can see that Nearest
Neighbor ( NN ) interpolation and VAE-based decoder perform better than rest of the upsampling schemes.
However, the NN interpolation achieves similar performance to the VAE-based method but without the need of a large training dataset which is required for accurately training a VAEbased decoder.
5.3. L ATENT D IMENSION S ENSITIVITY A NALYSIS
We perform a sensitivity analysis on the latent dimension hyperparameter d (cid:48) used for running the
Bayesian optimization.
We vary the latent dimension over the range d (cid:48) ∈ { 9 , 16 , 64 , 100 , 256 , 784 } .
Figure 3b shows the performance of nearest neighbor interpolation method for different latent dimension.
We observe that lower latent dimensions achieve better success rates than the original input dimension d (cid:48)
= 784 for MNIST.
This could be because with increase in search dimension,
Bayesian optimization needs more queries to ﬁnd successful perturbation.
We also note that for the case of latent dimension d (cid:48)
= 9 , B AYES -A TTACK achieves lower success rates which could mean that it is hard to ﬁnd adversarial perturbations in such low dimension.
We show the convergence plots of B AYES -A TTACK on ImageNet and MNIST in Appendix A.
C ONCLUSIONS
We considered the problem of black-box adversarial attacks in settings involving constrained query budgets.
We employed Bayesian optimization based method to construct a query efﬁcient attack strategy.
The proposed method searches for an adversarial perturbation in low dimensional latent space using Bayesian optimization and then maps the perturbation to the original input space using the nearest neighbor upsampling scheme.
We successfully demonstrated the efﬁcacy of our method in attacking multiple deep learning architectures for high dimensional inputs.
Our work opens avenues regarding applying BO for adversarial attacks in high dimensional settings
A C ONVERGENCE OF B AYESIAN O PTIMIZATION
In this section, we show the convergence of Bayesian Optimization (BO) in terms of objective value versus the number of queries.
Note that we have framed our objective of ﬁnding adversarial perturbation to be a maximization problem and we stop the iteration loop of BO once the objective value reaches a positive value.
A positive objective value corresponds to a successful adversarial perturbation as described in Section 3.
Figure 4 shows the convergence of objective function in the B AYES -A TTACK on R ES N ET 50 trained on ImageNet as described in Section 5.2.
We run the BO in 972 dimensions (3 × 18 × 18) and upsample the perturbation to the original input dimension of 150 , 528(3 × 224 × 224) .
The plot shows ten randomly chosen images from the validation set, with different colors representing different images.
[Figure]
We also show the convergence of the BO on MNIST by varying the latent dimension.
Speciﬁcally, we compare the convergence with latent dimension 16(4 × 4) and the original input dimension 784(28 × 28) .
The plot is shown in Figure 5.
Each color represents a test image, while dashed lines and solid lines represent runs of BO using 16 and 784 dimensions, respectively.
As we can see from the graph, BO in 784 dimensions does not converge to a successful attack (i.e., objective value > 0 ) in 500 iterations on either of the images, while BO with 16 dimensions on the same images ﬁnds the adversarial perturbation in less than 200 iterations.
This aligns with our observation that with increase in latent dimension, it becomes harder for BO to ﬁnd successful perturbation and indeed it would require much more queries than running BO in lower dimensions.
[Figure]
B D ISTORTION E FFICIENCY
We compare the average (cid:96) 2 distortion per image of the proposed method B AYES -A TTACK with the current state-of-the-art methods including gradient-based approaches on R ES N ET 50 trained on
ImageNet.
We ﬁx the query budget at 200 similar to the experiments described in Section 5.2 and compute the distortion using only the successful adversarial perturbations.
As we can see from
Table 2, the (cid:96) 2 distortion of adversarial examples generated using B AYES -A TTACK is almost similar to the current state-of-the-art methods but achieves better attack success rate in low query budget regimes.
Having said that, as in B ANDITS - TD (Ilyas et al., 2019) and P ARSIMONIOUS (Moon et al., 2019), our approach focuses on ﬁnding successful adversarial perturbations subject to a pre-deﬁned maximum distortion speciﬁed in terms of (cid:96) ∞ distance.
[Table]