Year: 2019

Abstract:
Deep neural network image classifiers are reported to be susceptible to adversarial evasion attacks, which use carefully crafted images created to mislead a classifier. Recently, various kinds of adversarial attack methods have been proposed, most of which focus on adding small perturbations to input images. Despite the success of existing approaches, the way to generate realistic adversarial images with small perturbations remains a challenging problem. In this paper, we aim to address this problem by proposing a novel adversarial method, which generates adversarial examples by imposing not only perturbations but also spatial distortions on input images, including scaling, rotation, shear, and translation. As humans are less susceptible to small spatial distortions, the proposed approach can produce visually more realistic attacks with smaller perturbations, able to deceive classifiers without affecting human predictions. We learn our method by amortized techniques with neural networks and generate adversarial examples efficiently by a forward pass of the networks. Extensive experiments on attacking different types of nonrobustified classifiers and robust classifiers with defence show that our method has state-of-the-art performance in comparison with advanced attack parallels.

text: Deep neural network image classifiers are reported to be susceptible to adversarial evasion attacks, which use carefully crafted images created to mislead a classifier. Recently, various kinds of adversarial attack methods have been proposed, most of which focus on adding small perturbations to input images. Despite the success of existing approaches, the way to generate realistic adversarial images with small perturbations remains a challenging problem. In this paper, we aim to address this problem by proposing a novel adversarial method, which generates adversarial examples by imposing not only perturbations but also spatial distortions on input images, including scaling, rotation, shear, and translation. As humans are less susceptible to small spatial distortions, the proposed approach can produce visually more realistic attacks with smaller perturbations, able to deceive classifiers without affecting human predictions. We learn our method by amortized techniques with neural networks and generate adversarial examples efficiently by a forward pass of the networks. Extensive experiments on attacking different types of nonrobustified classifiers and robust classifiers with defence show that our method has state-of-the-art performance in comparison with advanced attack parallels.

heading: 1 INTRODUCTION
text: Recently, Deep Neural Networks (DNNs) have enjoyed great success in many areas such as computer vision and natural language processing. Nevertheless, DNNs are demonstrated to be vulnerable to adversarial attacks (Goodfellow et al., 2014b; Nguyen et al., 2015; Kurakin et al., 2016), which are data samples carefully crafted to be misclassified by DNN classifiers. For example, in image classification, an adversarial example may imperceptibly look like a legitimate data sample in a ground-truth class but misleads a DNN classifier to predict it into a maliciously-chosen target class or any class different from the ground truth. The former and latter are referred to as targeted attack and untargeted attack, respectively. In addition, adversarial attacks can be categorised into poisoning attacks (attacks during the training phase of classifiers) vs evasion attacks (attacks during the testing/inference phase) and whitebox attacks vs blackbox attacks. For whitebox attacks, the attacker has full access to the model architecture and parameters of a classifier, while those kinds of information are invisible to blackbox attackers. In this paper, we are particularly interested in untargeted, evasion, and whitebox attacks. However, many of the attacks discussed in this paper (including the proposed ones) can be easily adapted into targeted or blackbox settings.
Shown in Goodfellow et al. (2014b), image classifiers based on DNNs (e.g., Convolutional Neural Networks (CNNs) (LeCun et al., 1998)) are susceptible to small but carefully-chosen perturbations. Therefore, significant research efforts have been devoted into perturbation-based adversarial attacks, such as those in Goodfellow et al. (2014b); Moosavi-Dezfooli et al. (2016); Carlini & Wagner (2017); Xiao et al. (2018a). Usually, a classifier is more easily fooled with larger perturbations, but this may result in adversarial examples that are less similar to original/real images. In practice, it is equally important that adversarial examples should mislead the classifier as well as “look like” real images. Therefore, the general task for perturbation-based methods can be formulated as misleading a classifier with a minimum amount of perturbation.
It is reported that the intermediate feature maps (convolutional layer activations) in a CNN are not actually invariant to spatial transformations of the input data, due to its limited, pre-defined pooling mechanism for dealing with spatial variations (Jaderberg et al., 2015; Cohen & Welling, 2015; Lenc & Vedaldi, 2015). Therefore, one can imagine generating adversarial examples by incorporating spatial distortions to original images. For example, Figure 1a shows that a CNN classifier successfully classifies the digits of MNIST (LeCun & Cortes, 1998). However, it is misled by the adversarial examples with properly-chosen spatial distortions. In contrast, humans are usually less influenced by such distortions. Motivated by this demonstration, by combining spatial distortions with perturbations, we may be able to generate more realistic adversarial examples with smaller perturbations, which can challenge classifiers without affecting human predictions.
To further demonstrate this idea, given some samples of original MNIST images shown in Figure 1b, we apply a widely-used perturbation-based attack, Projected Gradient Descent (PGD) (Madry et al., 2018), with the maximum perturbations of = 0.3 (the standard setting of the attack), to attack the previously-mentioned classifier, the corresponding adversarial examples of which are shown in Figure 1c. It can be observed that those adversarial examples, though they fool the classifier, can be detected by humans, meaning that one can easily distinguish real and adversarial examples. On the other hand, shown in Figure 1d, the approach introduced in this paper, combines spatial distortions and perturbations to achieve similar attack performance as PGD, but with much smaller perturbation ( = 0.1). Our adversarial examples are clearly less distinguishable from real images by humans.
Motivated by the above idea, in this paper, we propose a new kind of adversarial attack on DNNbased image classifiers, which generalises the conventional perturbation-based attacks with additional spatial distortions. We name our framework SdpAdv (Spatial distortion + perturbation Adversary). Specifically, to attack a pretrained classifier, our approach leverages a trainable joint process of two major steps, where it first generates a spatially distorted intermediate image by performing affine-transformations on a real image and then generates the final adversarial image by adding perturbations to the intermediate image.
The proposed SdpAdv has the following appealing properties:
• With the help of spatial distortions, our method is able to achieve state-of-the-art adversarial attack performance with much less perturbations than existing approaches purely based on perturbations, which generates less distinguishable adversarial examples.
• With a differentiable framework, the learning of SdpAdv can be done efficiently in an amortized way, where two neural networks are learned to generate the specific parameters of the spatial distortions and perturbations for an input image. After SdpAdv is trained, it only takes one-step forward propagation in the two neural networks to generate adversarial examples, which enjoys better efficiency in the testing phase.
• As most of existing robust classifier with defences, like those in Samangouei et al. (2018); Matyasko & Chau (2018), are designed to defend against perturbation-based attacks, they can be less effective to our attacks based on spatial distortions (Xiao et al., 2018b).
• Unlike many other whitebox attack methods, which usually require full access to the model structures and parameters of the classifier, SdpAdv only needs to be trained with access to the predicted probabilities of the classifier and directly generates adversarial examples from input images in the testing phase. That is to say, SdpAdv fits the settings of the semi-whitebox attack (Xiao et al., 2018a), which can be more applicable in practice.
To demonstrate the superiority of our proposed method, we conduct extensive comparisons with state-of-the-art adversarial attacks. The experimental results show that SdpAdv is able to achieve better attack performance against both unprotected and robust classifiers. More interestingly, using less perturbation, our adversarial examples are much less distinguishable from real images.

heading: 2 BACKGROUND AND RELATED WORK
text: We now introduce the background and related work on adversarial attacks. Suppose that an image x in the ground-truth label y is the input of a neural network classifier f . The predicted label of f given x is denoted as f(x). We assume f(x) = y for a well-trained classifier and will use them interchangeably hereafter. The general goal of adversarial attacks is to generate an adversarial example
xA, which should “look like” x but change the prediction of f , i.e., f(x) 6= f(xA). Conversely, the goal of a defender is to train a robust classifier to defend against adversarial attacks.

heading: 2.1 PERTURBATION-BASED ADVERSARIAL ATTACKS
text: As the name implies, perturbation-based attacks generate xA by adding small perturbations η to x: xA = x + η. In general, η can be constructed by either: η := argmaxη′:‖η′‖≤ `f (xA, y) or η := argminη′:f(x)6=f(xA) ‖η
′‖, where `f (·) denotes the cross-entropy loss of f and ‖ · ‖ can be the L∞ in accordance with Madry et al. (2018); Athalye et al. (2018) or other norms. As finding the closed-form solution for the above problem can be hard, Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2014b) is a one-step attack that applies a first-order approximation: η = · sign(∇x`f (x, y)). Several extensions and variants to FGSM have been proposed, such as Randomized FGSM (Tramèr et al., 2018), Basic Iterative Method (BIM) (Kurakin et al., 2016), and Projected Gradient Descent (PGD) (Madry et al., 2018). In addition to FGSM, there are numerous perturbation-based attacks that use different approximations to the above problem. For example, DeepFool (Moosavi-Dezfooli et al., 2016) generates adversarial perturbations by taking a step in the direction of the closest decision boundary in an iterative manner and CW (Carlini & Wagner, 2017) is an attack based on the optimisation of a modified loss function with implicit box-constraints.
Despite their success, the optimisation process of the above methods may have to be done for every test image, which could be inefficient in practice. To alleviate this problem, several attack methods have been proposed to directly generate perturbations by feeding real images into a generator: η = g(x), which is usually implemented by neural networks. For example, Adversarial Transformation Networks (ATNs) (Baluja & Fischer, 2018) trains g by minimising the combination of the re-ranking loss and an L2 norm loss, so as to constrain xA to be close to x in terms of L2. Instead of using an L2 norm, the AdvGAN (Xiao et al., 2018a) attack adopts a Generative Adversarial Network (GAN) (Goodfellow et al., 2014a) framework with a discriminator to encourage the perceptual quality of the generated attacks. Moreover, the Rob-GAN (Liu & Hsieh, 2019) attack also uses a GAN framework, where the discriminator is trained to distinguish between the attacks generated by PGD and those by the generator.

heading: 2.2 NON-PERTURBATION-BASED ADVERSARIAL ATTACKS
text: Here we consider non-perturbation-based methods as attacks that do not purely rely on manipulating the pixel values of original images. Compared with perturbation-based attacks, to our knowledge, research on non-perturbation-based methods is relatively rare. Recently, by adding perturbations to the latent space learned by Auxiliary Classifier GAN (AC-GAN) (Odena et al., 2017), the attack in Song et al. (2018) generates adversarial examples for a specific label from scratch without taking a real image as input. Alternatively, there are attacks that apply spatial transformations to original images. Spatial Transformation Method (STM)1 used in the CleverHans library (Papernot et al., 2018) constructs adversarial candidates by rotation and translation then selects the one that chal-
1https://cleverhans.readthedocs.io/en/latest/source/attacks.html
lenges the classifier most. As the selection process is non-differentiable, STM relies on the SPSA adversary (Uesato et al., 2018), which is a gradient-free optimization method. Given a real image, stAdv (Xiao et al., 2018b) is differentiable method that finds a flow field, each cell of which captures the transformation direction of one pixel of an input image.

heading: 3 METHODS

heading: 3.1 PROBLEM DEFINITION
text: Here we first present the problem definition of adversarial attacks, which generalises the one of perturbation-based methods, discussed in Section 2. Suppose that a real image and its ground-truth label is denoted by x ∈ RL and y ∈ {1, · · · ,K}, respectively, where L represents the image dimension and K is the number of unique labels. We consider a pretrained classifier f taking x as input and implemented with multilayer neural networks, where the last layer has K output units, denoted by fl(x). The output label of f(x) : RL → {1, · · · ,K} is obtained by: f(x) := argmaxl softmax(fl(x)). Given image x, we would like to find an adversarial example, xA ∈ O(x), that challenges the classifier f most, i.e., f(xA) 6= f(x), where O(x) denotes the set of candidate adversarial images. Given the notation, the most challenging adversarial example xA can be generated by the following optimisation:
xA := argmax x′∈O(x)
`f (x ′, y). (1)
In particular, O(x) is usually defined along with the way of generating attacks. For example, for perturbation-based methods, O(x) can be a L2-ball (or other norms) centred at x with the radius of : O(x) = {x′, ‖x′ − x‖ ≤ }. To make the candidates “perceptually close” to x, is set to a small value. However, this definition clearly does not fit the proposed approach with spatial distortions, as a small spatial distortion usually keeps an image looking similar but can result in a large L2 difference. Before introducing our definition of O(x), we present the proposed way of generating adversarial examples in SdpAdv. Given an real image x, SdpAdv conducts two major steps: 1) the spatial distortion step, where a spatial transformation t : RL → RL: is applied to generate a distorted image xT := t(x); 2) the perturbation step, where a perturbation η is imposed on xT to generate the final adversarial example: xA := xT + η.
In the spatial distortion step, we consider the affine transformation, which is a widely-used geometric transformation for images. Simply parameterised by a matrix of six real numbers, an affine transformation can be a composition of four linear transformations including scaling, rotation, shear, and translation. Given the position of a pixel on a 2D image, (µ, ν), an affine transformation with a parameter matrix θ transforms the pixel into a new position (µ′, ν′) as follows:[
µ′ ν′
1
] = [ a b e c d f 0 0 1 ] ︸ ︷︷ ︸
θ
[ µ ν 1 ] , (2)
where if a = d = 1 and b = e = c = f = 0, we get the parameter matrix for the identity transformation, denoted by θI . In the case of an image transformation, as the new position (µ′, ν′) can be fractional numbers and so not necessarily lie on the integer image grid, we apply bilinear interpolation to the original image before transformation, following (Jaderberg et al., 2015). With this notation, to generate a distorted image xT from x, we apply an affine transformation t with a parameter matrix θx specific to x: xT := tθx(x). Recall that xT has to be perceptually close to x, thus we would like to avoid over-transforming xT . Therefore, we enforce θx to be in the L2-ball of θI : ‖θx−θI‖ ≤ γ. After the spatial distortion step, we add the perturbation of ηx into xT to generate xA in the perturbation step: xA := xT +ηx, where ‖ηx‖ ≤ . This step is similar to the conventional perturbation-based methods, except that the perturbations are added into the intermediate image xT instead of the input image x.
Finally, the optimisation problem of SdpAdv can be described as: Given an individual image x, finding the optimal θx and ηx under the constraints of ‖θx − θI‖ ≤ γ and ‖ηx‖ ≤ , which can also be formulated as:
Ex∼Pd [
max η′:‖η′‖≤ ,θ′:‖θ′−θI‖≤γ
`f (tθ′(x) + η ′, y) ] , (3)
where Pd denotes the data distribution of the image training set. It is also noteworthy that the constraints on θ and η define the set of candidate adversarial examples, i.e., O(x).

heading: 3.2 AMORTIZED OPTIMISATION SOLUTION
text: In general, directly solving the problem in Eq. (3) involves the optimisation for every input image x, as in many existing perturbation-based algorithms such as CW (Carlini & Wagner, 2017). Such an optimisation process can be challenging and inefficient in practical cases, where real-times attacks may be important. Alternatively, we leverage the amortized optimisation with neural networks to bypass this problem.
First, we introduce two families of neural networks: Hsd := {hsd : θ′ := hsd(x) ∧ ‖θ′ − θI‖ ≤ γ} and Hp := {hp : η′ := hp(x) ∧ ‖η′‖ ≤ }. In particular, the family of Hsd consists of neural networks with the same network architecture, each of which, hsd, takes x as input and outputs the affine parameter matrix θ′ in the L2-ball with θI as the centre. Similarly, hp outputs the perturbation of η′. Due to the infinite capacity of neural networks, they can be used to approximate any continuous function up to any level of precision. Therefore, our goal is to find two neural networks h∗sd ∈ Hsd and h∗p ∈ Hp, which are used to approximate the optimisation in Eq. (3), based on the following theorem:
Theorem 1. If the familyH defined above has infinite capacity, the optimization problem in Eq. (3) is equivalent to the following:
max hsd∈Hsd,hp∈Hp
Ex∼Pd [ `f ( thsd(x)(x) + hp(thsd(x)(x)), y )]︸ ︷︷ ︸ −Ladv . (4)
The proof is in the appendix.

heading: 3.3 MODEL ARCHITECTURE OF SDPADV
text: With Theorem 1, we can amortize the optimisation of SdpAdv by training two neural networks: hsd and hp. Equipped with the two neural networks, we are able to build an end-to-end adversarial generator, g, that takes x as input and outputs xA by imposing spatial distortions and perturbations: xA := g(x). The architecture of the proposed generator of SdpAdv is shown in Figure 2. In the spatial distortion step, we adopt the architecture of STN (Jaderberg et al., 2015). Specifically, following Jaderberg et al. (2015), we name hsd to the “localisation network”, which takes x as input and outputs the optimal parameter matrix of the affine transformation, θx. After that, θx is fed into the “grid generator” to create a sampling grid, which is a set of points where x should be sampled to produce xT (Jaderberg et al., 2015). In the perturbation step, the adversarial generator takes the output of the previous step, xT , and then adds the optimal perturbation, ηx to obtain xA. In particular, ηx is generated from hp, named the “perturbation network”: ηx := hp(xT ). Finally, the operations of the adversarial generator can be summarised as:
g(x) := thsd(x)(x) + hp(thsd(x)(x)). (5)

heading: 3.4 LEARNING OF SDPADV
text: Given the architecture of the adversarial generator, the learning of SdpAdv is equivalent to that of the two neural networks: hsd and hp, by minimising the loss of Ladv in Eq. (4). To train the model with stochastic gradient descent (SGD), it is important to show that the model construction is differentiable so that the loss gradients can be backpropagated. It is not hard to see that hp is trainable, which follows a similar construction to residual neural networks (He et al., 2016). Shown in Jaderberg et al. (2015), the construction of STN also allows the loss gradients to flow back to the grid generator as well as the localisation network hsd.
In addition to Ladv, recall that we need to enforce ‖θx − θI‖ ≤ γ and ‖ηx‖ ≤ . Therefore, we introduce two regularisation loss functions, respectively, as follows:
Lθ := Ex∼Pd [‖θx − θI‖], (6) Lη := Ex∼Pd [max(0, ‖ηx‖ − )], (7)
where ‖ ·‖ is the L2 norm and Eq. (7) is the hinge loss used in Xiao et al. (2018a). In our implementation, after generating θx and ηx, we apply θx := min(θI + γ, θx) and ηx := min( , ηx) to keep xA in the candidate set O(x) defined by our method. Inspired by the GAN construction in Xiao et al. (2018a), we also introduce a neural network-based discriminator d to encourage that adversarial images are perceptually close to real images. However, different from Xiao et al. (2018a), our d distinguishes between xT (positive sample) and xA(negative sample) and does not care about the spatial distortion step. This is because xT is expected to naturally “look similar” to the original image x under small affine transformations. Therefore, it is unnecessary to use the discriminator to check this. Following Goodfellow et al. (2014a), the GAN loss is as follows:
LGAN := E x∼Pd, xT :=thsd(x)(x) [logD(xT )] + E x∼Pd, xT :=thsd(x)(x),
xA:=xA+hp(xT )
[log(1−D(xA))] . (8)
In conclusion, the learning of the adversarial generator can be done by minimising the following loss function:
Lg := Ladv + αLθ + βLη + λLGAN, (9)
where α, β, γ are the weight parameters for the losses. In addition, the discriminator can be trained by maximising the GAN loss, similar to Goodfellow et al. (2014a); Xiao et al. (2018a).

heading: 3.5 COMPARISON TO OTHER METHODS
text: Among the many related adversarial attack methods discussed in Section 2, we consider AdvGAN (Xiao et al., 2018a), STM implemented in CleverHans (Papernot et al., 2018), and stAdv (Xiao et al., 2018b) as the most related ones. To our knowledge, SdpAdv is the first technique that combines both spatial distortion and perturbation to generate adversarial examples, while others only consider either of the two attacks. Our method can be viewed as a generalisation to AdvGAN. That is to say, if we set γ to 0, then the affine transformations in our method will approach to the identity transformation and our SdpAdv reduces to AdvGAN. Despite the fact that STM and stAdv only consider spatial transformations, the way of conducting spatial transformations in our model is different from theirs. Specifically, STM only allows two kinds of transformations: rotation and translation while affine transformations used in our model are more flexible. More importantly, STM proposes spatially distorted candidates and uses a non-differentiable approach to select adversarial examples from those candidates, while ours is a differentiable method, which is more flexible and easier to train. In stAdv, each pixel in the input image has its specific flow vector to capture the transformation direction of the pixel, making the optimisation of the flow vectors potentially inefficient in practice. In contrast, ours uses one affine transformation that is specific to one image, for all the pixels of that image. Consequently, we only need to optimise for the six cells in the affine parameter matrix for each image, which can be efficiently done by a forward pass in the localisation network.

heading: 4 EXPERIMENTS
text: In this section, we conduct experiments to compare the performance of SdpAdv with other state-ofthe-art adversarial attack methods on attacking both non-robustified image classifiers (raw classifiers without defences) and robust classifiers with defences.

heading: 4.1 EXPERIMENTAL SETTINGS
text: Here we mainly consider the MNIST (LeCun & Cortes, 1998) and Fashion MNIST (Xiao et al., 2017) datasets, each of which consists of 60,000 images of ten classes.
Settings of the classifiers: For non-robustified classifiers, we consider: Model-A, a CNN-based classifier with “X-Conv(64, 8×8, 2)-ReLU-Conv(128, 6×6, 2)-ReLU-Conv(128, 5×5, 1)-ReLUFC(10)-Softmax” and Model-B, a fully-connected (FC) classifier with “X-FC(200)-ReLU-FC(200)ReLU-FC(10)-Softmax” (Samangouei et al., 2018), where X denotes input layer of image. The two classifiers were pretrained on the standard training set (50,000 images) of MNIST and Fashion MNIST. Additionally, we consider robust classifiers based on the following three state-of-the-art defences: Defense-GAN (Samangouei et al., 2018), Adversarial-Critic (Adv-Critic) (Matyasko & Chau, 2018), and Adversarial-Training (Adv-Train) with FGSM ( = 0.3) (Tramèr et al., 2018). All three robust classifiers share the same model architectures with the non-robustified classifiers but defend adversarial attacks in different ways. We used the original implementations of DefenseGAN2 and Adv-Critic3, and implemented Adv-Train ourselves on top of CleverHans.
Settings of SdpAdv: In the experiments, we used the following architectures for the adversarial generator (g) and discriminator (d): the localisation network (hsd) with “X-FC(20)-FC(6)”; the perturbation network (hp) with “X-FC(128)-FC(128)-X”; d with “X-FC(64)-FC(32)-FC(2)-Softmax”. It is noteworthy that other architectures for the above neural networks can also be used in SdpAdv. In the training phase of SdpAdv, we held out 10,000 images in the training set as our validation set and trained our method on the remaining images in the training set with 100 iterations. We reported the attack results with the best model in the validation set. We initialised α to 5.0 and used a decay factor of 0.8 for every ten iterations until it reached to 0.8. β and λ were set to 1.0. We set γ to 0.3 and varied in the range of {0.0, 0.1, 0.2, 0.3}. If = 0.0, it means that perturbation is turned off in SdpAdv and we name this variant of our model to “SdAdv”. The learning of the adversarial generator and discriminator was done by Adam (Kingma & Ba, 2014) with the learning rate of 0.0005 and 0.00005, respectively.
Settings of the other attacks: Here we mainly focus on whitebox attacks, where the attackers have full access to the classifiers. However, our SdpAdv only needs the output logits of the classifier in its training phase and does not require any additional information in the testing phase, similar to AdvGAN (Xiao et al., 2018a). For comparison, we consider the following state-of-the-art perturbation-based attack methods: FGSM (Goodfellow et al., 2014b), PGD (Madry et al., 2018), Momentum Iterative Method (Dong et al., 2018), and AdvGAN (Xiao et al., 2018a). We used the CleverHans (Papernot et al., 2018) implementations of the first three attacks with the standard settings except varying in the range of {0.1, 0.2, 0.3}. We implemented AdvGAN by turning off the spatial distortion step in our SdpAdv. For non-perturbation-based attacks, we consider STM implemented in CleverHans and stAdv (Xiao et al., 2018b) implemented by Dumont et al. (2018)4. All the classifiers and attacks (including the proposed ones) were implemented in TensorFlow5.

heading: 4.2 RESULTS
text: For quantitative results, we report the classification accuracies on the test set of the classifiers under the attacks. For attackers, lower accuracy indicates better attack performance. The accuracy results for MNIST and Fashion MNIST are shown in Table 1 and Table 2, respectively.
We have the following remarks on our results: 1) In general, our proposed SdpAdv performs the best on attacking both non-robustified and robust classifiers in the comparison with other attacks. This
2https://github.com/kabkabm/defensegan 3https://github.com/aam-at/adversary_critic 4https://github.com/rakutentech/stAdv 5https://www.tensorflow.org
is particularly demonstrated in the “Sum” column, which can be viewed as a measure of the overall performance. 2) In comparison with attacks using perturbations, with very small perturbation magnitudes (e.g., = 0.1), other methods struggle to perform, while with the help of spatial distortions, SdpAdv is able to achieve impressive attack results, generating less noisy attacks without sacrificing performance. 3) Among the attacks with spatial distortions, our variant, SdAdv, achieves the best attack results in most cases. 4) We observe that robust classifiers such as Defense-GAN, which are usually designed to defend against perturbations, are much less effective against spatial distortions.
To further study our methods, we compare the test accuracies, L2 norm, and running time of the compared attacks in Table 3. It can be found out that our proposed approaches are able to perform attacks as efficient as FGSM, which is a one-step operation, but with much better performance. It is also noteworthy that the adversarial examples generated by our methods can result in large values of L2 norm but they can look very realistic, shown in Figure 3 and Figure 4.

heading: 5 CONCLUSION
text: In this paper, we have introduced a new adversarial attack method, named SdpAdv, named SdpAdv, which generates adversarial examples with both spatial distortions and perturbations. Specifically, given an input image, SdpAdv applies affine transformations to conduct spatial distortions and then adds perturbations to the spatially distorted image to generate the final adversarial example. As a differentiable approach, SdpAdv leverages the amortized optimisation with two neural networks to obtain the optimal parameter of affine transformations and the optimal perturbations, respectively. Extensive experiments of attacking different kinds of non-robustified classifiers and robust classifiers have shown that our method achieves the state-of-the-art performance in the comparison with advanced attack parallels. More importantly, in the use of spatial distortions, our proposed approach can produce more realistic adversarial examples with smaller perturbations, which can challenge classifiers well without affecting human predictions.

heading: A APPENDIX
text: A.1 PROOF OF THEOREM 1
Proof. Give x, suppose u (x) and v(x) are the optimal solution of the affine parameter matrix and perturbation for Eq. (3), respectively. We first prove that [u(x), v(x)] := h∗ (x) almost everywhere w.r.t the probability measure Pd, where h∗(x) := [ h∗sd(x), h ∗ p ( th∗sd(x)(x) )] . h∗(x) consists of the optimal solution of Eq. (4). By the definition of u, v, it is obvious that
`f ( th∗sd(x)(x) + h ∗ p ( th∗sd(x)(x) )) ≤ `f (tu(x)(x) + v (x)).
Furthermore, it follows that: EPd [ `f ( th∗sd(x)(x) + h ∗ p ( th∗sd(x)(x) ))] ≤ EPd [ `f (tu(x)(x) + v (x)) ] .
Since Hsd has infinite capacity, there exists husd ∈ Hsd such that husd = u. We further define k(x) = v ( t−1husd(x) (x) ) and obtain k ( thusd(x) (x) ) = v(x). Similarly, with the infinite capacity property ofHp, there exists hvp ∈ Hp such that hvp = k Referring to the definition of h∗, we have:
EPd [ `f ( th∗sd(x)(x) + h ∗ p ( th∗sd(x)(x) ))] ≥ EPd [ `f ( thusp(x) (x) + h v p ( thusp(x) (x) ))] ,
EPd [ `f ( th∗sd(x)(x) + h ∗ p ( th∗sd(x)(x) ))] ≥ EPd [ `f (tu(x) (x) + v (x)) ] ,
which further implies the equality: EPd [ `f ( th∗sd(x)(x) + h ∗ p ( th∗sd(x)(x) ))] = EPd [ `f (tu(x)(x) + v (x)) ] ,
and also [u(x), v(x)] := h∗ (x) almost every w.r.t the probability measure Pd.

Review summaries:
This paper proposes a new adversarial attack method by combining spatial transformations with perturbation-based noises. The proposed method uses two networks to generate the parameters of spatial transformation and the perturbation noise. The whole architecture is trained by a variant of GAN-loss to make the adversarial examples realistic to humans. Experiments on MNIST prove that the proposed attack method can improve the success rate of white-box attacks against several models.

This paper builds upon the work of AdvGAN and proposes to add spatial transformations on top of it. The resulting attacking framework is demonstrated to outperform AdvGAN on attacking several defense approaches, such as Defense-GAN, AdvCritic and adversarial training. Compared to previous approaches on generating spatially transformed adversarial examples, this approaches amortizes the attacking procedure and can produce spatially transformed adversarial examples much faster. This approach also simultaneously combine spatial transformations and perturbations to make the attack stronger.