# Boosting Adversarial Attacks on Neural Networks with Better Optimizer

Authors: Heng Yin; Hengwei Zhang; Jindong Wang; Ruiyu Dou
Journal: Security and Communication Networks
 (2021)

---

## Abstract

Convolutional neural networks have outperformed humans in image recognition tasks, but they remain vulnerable to attacks from adversarial examples. Since these data are crafted by adding imperceptible noise to normal images, their existence poses potential security threats to deep learning systems. Sophisticated adversarial examples with strong attack performance can also be used as a tool to evaluate the robustness of a model. However, the success rate of adversarial attacks can be further improved in black-box environments. Therefore, this study combines a modified Adam gradient descent algorithm with the iterative gradient-based attack method. The proposed Adam iterative fast gradient method is then used to improve the transferability of adversarial examples. Extensive experiments on ImageNet showed that the proposed method offers a higher attack success rate than existing iterative methods. By extending our method, we achieved a state-of-the-art attack success rate of 95.0% on defense models.


---

## Body

## 1. Introduction

In image recognition tasks, convolutional neural networks are able to classify images with an accuracy approaching that of humans [1–4]. However, researchers have found that neural networks are also vulnerable to adversarial examples. Szegedy et al. [5] first proposed the concept of adversarial examples: images added with small perturbations, which cause neural network models to output incorrect classifications with high confidence. These adversarial perturbations are often indistinguishable to the human eyes (in other words, there is no obvious visual difference between the adversarial examples and the original images).Adversarial attacks can be categorized into white- and black-box attacks. A variety of techniques can be used to generate adversarial examples and perform white-box attacks, depending on the model structure and corresponding parameters [6–10]. In addition, adversarial examples are generally transferable as data generated for one model may be able to fool other models. This facilitates black-box attacks, in which the structure and parameters of the model are not available, for various neural networks [11]. Goodfellow et al. [6] suggested that different models learn similar decision boundaries during the same image classification tasks and obtain similar parameters. These properties make it easier to generalize adversarial examples to different models.Although adversarial examples are generally transferable, the optimal approach of improving this transferability needs to be examined further. Due to a balance between attack performance and transferability, basic iterative attacks are often more effective than single-step attacks in white-box environments and weaker in black-box environments. In white-box attacks, iterative methods excessively fit specific network parameters, achieving a high success rate but preventing generalization to other models [9]. We consider that this is the result of overfitting since attack performance for adversarial examples in white- and black-box environments is similar to the neural network performance on training and test sets.Unlike white-box attacks, black-box attacks are more consistent with actual attack-defense environments and are primarily performed in one of the following three ways. (1) Decision-based attacks are conducted using information collected from the network. Although access to model structure and parameters is unavailable, the attacker can generate adversarial examples by executing multiple queries against the model [12–15]. (2) In substitute-model techniques, the attacker can input images into the model to obtain an output label. Then, the substitute model can be trained to imitate the target model and generate adversarial examples [16]. (3) Transferability attacks are conducted by improving the transferability of adversarial examples generated in a white-box setting [8, 9]. The first two black-box attacks require large quantities of model queries, which is impractical in some cases (e.g., online platforms often limit the number of queries). As such, this study investigates black-box attacks using adversarial data with strong transferability. We investigate a state-of-the-art optimizer Adam for improving black-box adversarial attacks.In this paper, we propose the Adam iterative fast gradient method (AI-FGM) to improve the transferability of adversarial examples among different models [17]. Inspired by the fact that Adam is better than momentum in optimization, we adopt Adam optimizer into the iterative gradient-based attack, so as to accelerate data update in dimensions with small gradients and achieve better convergence, by applying the second momentum term and a decreasing step size. As shown in Figure 1, being different from the momentum method that just accumulates the gradients of data points along the optimization path, our Adam method can also accumulate the square of the gradients. Such accumulation might help obtain an adaptive update direction, which leads to a better local minimum. Besides, a variable step size is also used to avoid oscillating.Figure 1
Schematic optimization path of MI-FGSM and the proposed AI-FGM. MI-FGSM accumulates the gradients of data points along the optimization path, while Adam accumulates the gradients and the square of the gradients. MI-FGSM adopts an invariable step size, while AI-FGM adopts a decayed step size that leads to better convergence.Compared with existing gradient-based methods, the proposed method offers improved attack performance in black-box settings. This approach was tested on multiple networks, including adversarially trained networks, achieving higher attack success rates on black-box models. In addition, we combined our approach with advanced methods and attacked an ensemble of multiple networks, which further improved the transferability of adversarial examples.


## 2. Related Works

### 2.1. Adversarial Attack Methods

A variety of techniques have been proposed to generate transferable adversarial examples, which can be divided into two categories: optimizer-based methods and data-augmentation-based methods. Specifically, Goodfellow et al. proposed the fast gradient sign method (FGSM) to generate adversarial examples with very low calculation costs [6]. By extending FGSM, Alexey et al. developed the iterative fast gradient sign method (I-FGSM) [7]. Dong et al. proposed the momentum iterative fast gradient sign method (MI-FGSM) to improve the transferability of adversarial examples in black-box environments [8]. Lin et al. proposed Nesterov iterative fast gradient sign method (NI-FGSM) and adopted Nesterov accelerated gradient into the iterative attacks [18]. Xie first applied data augmentation in the generation of adversarial examples and proposed the diverse inputs method (DIM) [9]. Dong et al. proposed a translation-invariant attack (TI-FGSM) and improved the transferability of adversarial examples on defense models by a large margin [19]. Lin et al. proposed the scale-invariant attack method (SIM) and combined it with existing methods, resulting in SI-NI-TI-DIM (the currently strongest black-box attack method) [18].


### 2.2. Adversarial Defense Methods

Multiple defense mechanisms have been proposed to protect deep learning models from the threat of adversarial examples [20–29]. Among these, adversarial training is the most effective way to improve model robustness [6, 30]. In this process, adversarial examples are generated and added to the training set to participate in the model training procedure. Since normal adversarial training remains vulnerable to adversarial examples, Tramèr et al. proposed ensemble adversarial training to improve robustness further [31]. In this process, adversarial data generated by multiple models are included in the training set for a single model, thereby producing a more robust classifier.


## 2.1. Adversarial Attack Methods

A variety of techniques have been proposed to generate transferable adversarial examples, which can be divided into two categories: optimizer-based methods and data-augmentation-based methods. Specifically, Goodfellow et al. proposed the fast gradient sign method (FGSM) to generate adversarial examples with very low calculation costs [6]. By extending FGSM, Alexey et al. developed the iterative fast gradient sign method (I-FGSM) [7]. Dong et al. proposed the momentum iterative fast gradient sign method (MI-FGSM) to improve the transferability of adversarial examples in black-box environments [8]. Lin et al. proposed Nesterov iterative fast gradient sign method (NI-FGSM) and adopted Nesterov accelerated gradient into the iterative attacks [18]. Xie first applied data augmentation in the generation of adversarial examples and proposed the diverse inputs method (DIM) [9]. Dong et al. proposed a translation-invariant attack (TI-FGSM) and improved the transferability of adversarial examples on defense models by a large margin [19]. Lin et al. proposed the scale-invariant attack method (SIM) and combined it with existing methods, resulting in SI-NI-TI-DIM (the currently strongest black-box attack method) [18].


## 2.2. Adversarial Defense Methods

Multiple defense mechanisms have been proposed to protect deep learning models from the threat of adversarial examples [20–29]. Among these, adversarial training is the most effective way to improve model robustness [6, 30]. In this process, adversarial examples are generated and added to the training set to participate in the model training procedure. Since normal adversarial training remains vulnerable to adversarial examples, Tramèr et al. proposed ensemble adversarial training to improve robustness further [31]. In this process, adversarial data generated by multiple models are included in the training set for a single model, thereby producing a more robust classifier.


## 3. Methodology

This section provides a detailed introduction to our proposed methodology. Letx denote an input image, and y denote the corresponding ground-truth label. The term θ represents network parameters, and Jθ,x,y describes a loss function, typically the cross-entropy loss. The primary objective is to generate an adversarial example x∗ to fool the model by maximizing Jθ,x,y, such that the prediction label ypre≠y. In this paper, we used an L∞ norm bound to limit adversarial perturbations, such that x∗−x∞≤ε, where ε refers to the size of the perturbation. The generation of adversarial examples can, therefore, be converted into the following optimization problem:(1)argmaxx∗Jθ,x∗,y,s.t.x∗−x∞≤ε.


### 3.1. Attack Methods Based on Gradient

In this section, several techniques that are used to solve the above optimization problem are introduced briefly:(i)
Fast gradient sign method (FGSM) [6]: As one of the simplest techniques, it seeks adversarial examples in the direction of the gradient of the loss function with respect to the input image x. The method can be expressed as(2)x∗=x+ε⋅sign∇xJθ,x,y.(ii)
Iterative fast gradient sign method (I-FGSM) [7]: This algorithm is an iterative version of FGSM. The approach involves dividing the FGSM gradient operation into multiple steps that can be expressed as follows:(3)x0∗=x,xi+1∗=xi∗+α⋅sign∇xJθ,xi∗,y,xi+1∗=Clipxi+1∗,x−ε,x+ε.whereα denotes the step size of each iteration and α=ε/T, in which T denotes the number of iterations. The Clip function was included to limit the adversarial example x∗ within the ε neighborhood of the original image x and satisfy the L∞ norm constraint. I-FGSM is more effective than FGSM in white-box environments but less effective in black-box environments. In other words, I-FGSM exhibits poor transferability. This iterative attack method is also known as projected gradient descent (PGD) if the algorithm is added by a random initialization on x [20].(iii)
Momentum iterative fast gradient sign method (MI-FGSM) [8]: In this method, a momentum term is applied to the iterative process to escape from poor local maxima. This produces adversarial examples with more transferability, which can be expressed as [8](4)x0∗=x,g0=0,gi+1=μ⋅gi+∇xJθ,xi∗,y∇xJθ,xi∗,y1,xi+1∗=xi∗+α⋅signgi+1,xi+1∗=Clipxi+1∗,x−ε,x+ε,where μ denotes the decay factor of the momentum term, and gi denotes the weighted accumulation of gradients in the first i rounds of iterations.


### 3.2. Adam Iterative Fast Gradient Method

The generation of adversarial examples is similar to the training of neural networks; both processes can be viewed as an optimization problem. Specifically, the adversarial example can be viewed as the training parameter, while the white-box model can be viewed as the training set, and the black-box model can be viewed as a testing set. From this point of view, the transferability of the adversarial examples is similar to the generalization of the models. Therefore, we can apply the methods used to improve the generalization of the models to the generation of adversarial examples. Many methods are proposed to improve the generalization of neural network models, which can be divided into two categories: better optimization and data augmentation. Correspondingly, these two kinds of methods can be used in the adversarial attacks, and there have been attempts to do so, for example, MI-FGSM and DIM [8, 9]. Based on the analysis above, we aim to improve the transferability of adversarial examples with Adam optimizer since it performs well in the training of neural networks.Adam [17] is an optimization algorithm combining momentum [32] and RMSProp [33]. In the iteration process, Adam accumulates not only the gradients of the loss function with respect to the input image but also the square of the gradients. This is done to accelerate loss function ascent in dimensions with small gradients. In addition, Adam uses a decreasing step size in each iteration to achieve better convergence.In contrast, both I-FGSM and MI-FGSM adopt a constant step size. In the later stages of the algorithm, oscillations occur near the local maximum, which do not converge well. To solve this problem, we have improved the Adam algorithm by normalizing the step size sequence (defined in Algorithm1) and have applied it to the iterative gradient method. The proposed Adam iterative fast gradient method (AI-FGM) is summarized in Algorithm 1.Algorithm 1: Adam iterative fast gradient method.
Input: A convolutional neural networkf and the corresponding cross-entropy loss function Jθ,x,y; an original image x and the corresponding ground-truth label y; the number of iterations T; the iteration time step t; the dimension of the input image N; the size of the perturbation ε; Adam decay factors β1 and β2; and a denominator stability factor δ.Output: An adversarial examplex∗, s.t. x−x∗∞≤ε.(1)m0=0, v0=0, x0∗=x, and t=0;(2)α=ε⋅N(3) whilet<T do:(4)gt=∇xJθ,xt∗,y/∇xJθ,xt∗,y1(5)mt+1=β1mt+1−β1gt;(6)vt+1=β2⋅vt+1−β2⋅gt2;(7)st+1=mt+1/δ+vt+1;(8)αt=α⋅1−β2t+1/1−β1t+1/∑i=0T−11−β2i+1/1−β1i+1;(9)xt+1∗=xt∗+αt⋅st+1/st+12;(10)xt+1∗=Clipxt+1∗,x−ε,x+ε;(11)t=t+1;(12) end while(13)
returnx∗=xT∗.Specifically, the gradient∇xJθ,xt∗,y in each iteration is normalized by its own L1 distance, defined in Algorithm 1, because the scale of these gradients differs widely in each iteration [8]. Similar to MI-FGSM, mt accumulates gradients of the first t iterations with a decay factor β1, defined in Algorithm 1. The result can be considered the first momentum. The term vt represents the second momentum, which accumulates the squares of gradients for the first t iterations with a decay factor β2, defined in Algorithm 1. It is noteworthy that gt2 denotes an elementwise square gt⊙gt, where ⊙ represents the Hadamard product [32]. The terms β1 and β2 are typically defined in the range 0,1. The update direction of input x is defined in Algorithm 1, where the stability coefficient δ is set to avoid a zero in the denominator. The st term is advantageous as it prompts x to escape from local maxima and accelerates the updating of x in dimensions with small gradients.Learning rate decay is often used in the training of neural networks. In this study, the step size used in the Adam optimizer is continually reduced to help improve the convergence of the algorithm. Ifβ1 and β2 are set appropriately, the value of 1−β2t/1−β1t will decrease with the increase of t. Thus, a decreasing sequence can be generated when t ranges from 1 to T. The sequence can then be normalized to obtain the weight of step sizes in each iteration, relative to the total step size α. This can be used to acquire a set of exponential decay step sizes controlled by β1 and β2, as defined in Algorithm 1.Existing techniques typically use the sign function to satisfy theL∞ norm limitation. However, if the sign function was applied in Algorithm 1, the update direction in our method would be equivalent to that of MI-FGSM. Hence, we constrain adversarial examples within the L∝ norm bound by the Clip function and apply the step size and update direction within the corresponding L2 norm bound, defined in Algorithm 1. The relationship between the L∝ norm bound (ε) and the L2 norm bound (α) is defined in Algorithm 1, where N represents the dimension of the input image x.


### 3.3. Attacking an Ensemble of Networks

The proposed method was also used to attack an ensemble of networks. If an adversarial example poses a threat to multiple networks, it is far more likely to transfer to other models [11]. We followed the ensemble strategy proposed by Dong et al., in which multiple models are attacked by fusing network logits [8]. Specifically, in order to attack an ensemble of K models, logits were fused as follows:(5)lx=∑k=1Kωklkx,where lkx refers to the logit output of the k-th model, ωk refers to the ensemble weight (ωk≥0), and ∑k=1Kωk=1.


## 3.1. Attack Methods Based on Gradient

In this section, several techniques that are used to solve the above optimization problem are introduced briefly:(i)
Fast gradient sign method (FGSM) [6]: As one of the simplest techniques, it seeks adversarial examples in the direction of the gradient of the loss function with respect to the input image x. The method can be expressed as(2)x∗=x+ε⋅sign∇xJθ,x,y.(ii)
Iterative fast gradient sign method (I-FGSM) [7]: This algorithm is an iterative version of FGSM. The approach involves dividing the FGSM gradient operation into multiple steps that can be expressed as follows:(3)x0∗=x,xi+1∗=xi∗+α⋅sign∇xJθ,xi∗,y,xi+1∗=Clipxi+1∗,x−ε,x+ε.whereα denotes the step size of each iteration and α=ε/T, in which T denotes the number of iterations. The Clip function was included to limit the adversarial example x∗ within the ε neighborhood of the original image x and satisfy the L∞ norm constraint. I-FGSM is more effective than FGSM in white-box environments but less effective in black-box environments. In other words, I-FGSM exhibits poor transferability. This iterative attack method is also known as projected gradient descent (PGD) if the algorithm is added by a random initialization on x [20].(iii)
Momentum iterative fast gradient sign method (MI-FGSM) [8]: In this method, a momentum term is applied to the iterative process to escape from poor local maxima. This produces adversarial examples with more transferability, which can be expressed as [8](4)x0∗=x,g0=0,gi+1=μ⋅gi+∇xJθ,xi∗,y∇xJθ,xi∗,y1,xi+1∗=xi∗+α⋅signgi+1,xi+1∗=Clipxi+1∗,x−ε,x+ε,where μ denotes the decay factor of the momentum term, and gi denotes the weighted accumulation of gradients in the first i rounds of iterations.


## 3.2. Adam Iterative Fast Gradient Method

The generation of adversarial examples is similar to the training of neural networks; both processes can be viewed as an optimization problem. Specifically, the adversarial example can be viewed as the training parameter, while the white-box model can be viewed as the training set, and the black-box model can be viewed as a testing set. From this point of view, the transferability of the adversarial examples is similar to the generalization of the models. Therefore, we can apply the methods used to improve the generalization of the models to the generation of adversarial examples. Many methods are proposed to improve the generalization of neural network models, which can be divided into two categories: better optimization and data augmentation. Correspondingly, these two kinds of methods can be used in the adversarial attacks, and there have been attempts to do so, for example, MI-FGSM and DIM [8, 9]. Based on the analysis above, we aim to improve the transferability of adversarial examples with Adam optimizer since it performs well in the training of neural networks.Adam [17] is an optimization algorithm combining momentum [32] and RMSProp [33]. In the iteration process, Adam accumulates not only the gradients of the loss function with respect to the input image but also the square of the gradients. This is done to accelerate loss function ascent in dimensions with small gradients. In addition, Adam uses a decreasing step size in each iteration to achieve better convergence.In contrast, both I-FGSM and MI-FGSM adopt a constant step size. In the later stages of the algorithm, oscillations occur near the local maximum, which do not converge well. To solve this problem, we have improved the Adam algorithm by normalizing the step size sequence (defined in Algorithm1) and have applied it to the iterative gradient method. The proposed Adam iterative fast gradient method (AI-FGM) is summarized in Algorithm 1.Algorithm 1: Adam iterative fast gradient method.
Input: A convolutional neural networkf and the corresponding cross-entropy loss function Jθ,x,y; an original image x and the corresponding ground-truth label y; the number of iterations T; the iteration time step t; the dimension of the input image N; the size of the perturbation ε; Adam decay factors β1 and β2; and a denominator stability factor δ.Output: An adversarial examplex∗, s.t. x−x∗∞≤ε.(1)m0=0, v0=0, x0∗=x, and t=0;(2)α=ε⋅N(3) whilet<T do:(4)gt=∇xJθ,xt∗,y/∇xJθ,xt∗,y1(5)mt+1=β1mt+1−β1gt;(6)vt+1=β2⋅vt+1−β2⋅gt2;(7)st+1=mt+1/δ+vt+1;(8)αt=α⋅1−β2t+1/1−β1t+1/∑i=0T−11−β2i+1/1−β1i+1;(9)xt+1∗=xt∗+αt⋅st+1/st+12;(10)xt+1∗=Clipxt+1∗,x−ε,x+ε;(11)t=t+1;(12) end while(13)
returnx∗=xT∗.Specifically, the gradient∇xJθ,xt∗,y in each iteration is normalized by its own L1 distance, defined in Algorithm 1, because the scale of these gradients differs widely in each iteration [8]. Similar to MI-FGSM, mt accumulates gradients of the first t iterations with a decay factor β1, defined in Algorithm 1. The result can be considered the first momentum. The term vt represents the second momentum, which accumulates the squares of gradients for the first t iterations with a decay factor β2, defined in Algorithm 1. It is noteworthy that gt2 denotes an elementwise square gt⊙gt, where ⊙ represents the Hadamard product [32]. The terms β1 and β2 are typically defined in the range 0,1. The update direction of input x is defined in Algorithm 1, where the stability coefficient δ is set to avoid a zero in the denominator. The st term is advantageous as it prompts x to escape from local maxima and accelerates the updating of x in dimensions with small gradients.Learning rate decay is often used in the training of neural networks. In this study, the step size used in the Adam optimizer is continually reduced to help improve the convergence of the algorithm. Ifβ1 and β2 are set appropriately, the value of 1−β2t/1−β1t will decrease with the increase of t. Thus, a decreasing sequence can be generated when t ranges from 1 to T. The sequence can then be normalized to obtain the weight of step sizes in each iteration, relative to the total step size α. This can be used to acquire a set of exponential decay step sizes controlled by β1 and β2, as defined in Algorithm 1.Existing techniques typically use the sign function to satisfy theL∞ norm limitation. However, if the sign function was applied in Algorithm 1, the update direction in our method would be equivalent to that of MI-FGSM. Hence, we constrain adversarial examples within the L∝ norm bound by the Clip function and apply the step size and update direction within the corresponding L2 norm bound, defined in Algorithm 1. The relationship between the L∝ norm bound (ε) and the L2 norm bound (α) is defined in Algorithm 1, where N represents the dimension of the input image x.


## 3.3. Attacking an Ensemble of Networks

The proposed method was also used to attack an ensemble of networks. If an adversarial example poses a threat to multiple networks, it is far more likely to transfer to other models [11]. We followed the ensemble strategy proposed by Dong et al., in which multiple models are attacked by fusing network logits [8]. Specifically, in order to attack an ensemble of K models, logits were fused as follows:(5)lx=∑k=1Kωklkx,where lkx refers to the logit output of the k-th model, ωk refers to the ensemble weight (ωk≥0), and ∑k=1Kωk=1.


## 4. Experiment

Extensive validation experiments were conducted to demonstrate the effectiveness of the proposed methodology. Experimental settings are provided in Section4.1. The results of attacking a single network and some parameters that influence the results are discussed in Section 4.2. Thus, the results of attacking an ensemble of models are shown in Section 4.3. Finally, Section 4.4 exhibits the results of attacks by the combination of AI-FGM and existing methods (source code is available at https://github.com/YinHeng121/Adam_attack).


### 4.1. Experimental Setup

(i)
Data set: if the original images cannot be classified correctly by the network, it is meaningless to generate adversarial examples based on these data. Therefore, we selected 1,000 images belonging to the 1,000 categories from the ImageNet validation set, all of which could be classified correctly by the tested networks. All images were scaled to299×299×3. As such, N=299×299×3 in Section 3.2.(ii)
Networks: seven different models were studied; four of which were normally trained networks (i.e., Inception-v3 (Inc-v3) [34], Inception-v4 (Inc-v4) [35], Inception-Resnet-v2 (IncRes-v2) [35], and Resnet-v2-101 (Res-101) [36]). The other three models were adversarially trained networks (i.e., ens3-adv-Inception-v3 (Inc-v3ens3), ens4-adv-Inception-v3 (Inc-v3ens4), and ens-adv-Inception-ResNet-v2 (IncRes-v2ens) [31]).(iii)
Other details: the proposed method was compared with the other methods discussed in Section3.1. All the experiments described in this paper were based on the L∞ norm. Thus, the momentum decay factor μ in Algorithm 1 was 1.0, and the stability coefficient δ in Algorithm 1 was 10−8, as suggested in [17].


### 4.2. Attacking a Single Network

We first performed adversarial attacks on a single network with FGSM, I-FGSM, MI-FGSM, and AI-FGM. Moreover, the adversarial examples were generated on four normally trained networks and tested on all seven networks. The results are shown in Table1, where the success rates are misclassification rates for the corresponding models, with adversarial examples used as input. The decay factors β1 and β2 in AI-FGM were set to 0.99 and 0.999, respectively. The maximum perturbation ε was 16. The number of iterations T for I-FGSM, PGD, MI-FGSM, and AI-FGM was 10. These methods are hereafter referred to as “iterative methods” without ambiguity. The effects of these parameter choices are discussed further in this section.Table 1
Attack success rates (%) for all seven networks included in the study.
AttackInc-v3Inc-v4Incres-v2Res-101Inc-v3ens3Inc-v3ens4Incres-v2ensInc-v3FGSM72.232.131.732.310.310.64.2I-FGSM100.027.523.020.96.24.71.8PGD99.818.914.714.46.16.23.2MI-FGSM100.054.350.644.013.913.36.5AI-FGM100.060.755.850.217.016.88.5Inc-v4FGSM39.464.929.533.012.211.15.0I-FGSM43.8100.027.423.46.16.32.4PGD32.099.817.716.66.45.83.1MI-FGSM69.9100.057.954.119.617.78.7AI-FGM72.9100.060.157.121.719.510.3Incres-v2FGSM37.631.857.931.413.712.06.8I-FGSM46.135.099.430.47.36.74.4PGD30.123.097.318.36.15.72.7MI-FGSM73.569.399.560.027.023.016.7AI-FGM74.671.199.561.831.325.720.5Res-101FGSM38.333.030.279.314.613.36.4I-FGSM35.128.325.199.58.46.73.7PGD30.922.420.999.67.37.23.4MI-FGSM60.055.350.699.522.919.811.3AI-FGM64.057.754.099.527.224.115.4The diagonal blocks indicate white-box attacks, while the off-diagonal blocks indicate black-box attacks.As shown in Table1, all four iterative methods attacked a white-box model with a near 100% success rate. AI-FGM performed better than the other three methods on all black-box models. For example, for adversarial examples generated on Inc-v3, AI-FGM had a success rate of 60.7% on Inc-v4, while MI-FGSM, PGD, I-FGSM, and FGSM reached 54.3%, 18.9%, 27.5%, and 32.1%, demonstrating the effectiveness of the proposed method. An original image and the corresponding adversarial example generated for Inc-v3 by AI-FGM are shown in Figure 2.Figure 2
Classification of a normal image and corresponding adversarial example by Inc-v3 and Inc-v4. The first row shows the top 10 confidence distributions for a clean image, for which both models provided a correct prediction. The second row shows the top 10 confidence distributions for the adversarial example generated for Inc-v3 by AI-FGM. It is evident that the adversarial example successfully attacked Inc-v3 (white-box) and Inc-v4 (black-box) with high confidence. (a) Clean. (b) Adversarial.
(a)(b)


#### 4.2.1. Decay Factors

The termsβ1 and β2 control not only the decay amplitude of the step sizes but also the accumulation intensity of gradients for mt and vt. As such, they have a direct impact on attack success rates. We applied a grid-search method to identify an optimal set of β1 and β2 values. In the experiments, the maximum perturbation ε was set to 16, and the number of iterations T was set to 10. The values of β1 and β2 ranged from 0 to 1. Notably, we not only chose the values of β1 and β2 uniformly from 0.1 to 0.9 but also selected values close to 0 and 1, as shown in Figure 3. This was done to provide a more comprehensive study on the effects of decay factors, as we propose that the relationship between the success rate and the decay factors may be nonlinear. Adversarial examples were then generated on Inc-v3 with AI-FGM and used to perform attacks on Inc-v3, Inc-v4, and Inc-v3ens4. It is evident from the figure that regardless of the values of β1 and β2, attack success rates were always near 100% in white-box environments. Beyond that, attack success rates were more sensitive to β1 for black-box models, regardless of whether networks were normally or adversarially trained. Success rates were maximized when both β1 and β2 were close to 1.Figure 3
Attack success rates (%) of adversarial examples generated for Inc-v3 with AI-FGM, applied to Inc-v3 (white-box), Inc-v4 (black-box and normally trained), and Inc-v3ens4 (black-box and adversarially trained) withβ1 and β2 in the range of 0,1. (a) Inc-v3. (b) Inc-v4. (c) Inc-v3ens4.
(a)(b)(c)


#### 4.2.2. The Number of Iterations

The effect of iteration quantities on success rates was studied by performing attacks using iterative methods. The maximum perturbationε was set to 16, and the decay factors β1 and β2 were set to 0.99 and 0.999, respectively. The number of iterations T ranged from 1 to 20. Adversarial examples generated on Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM were then used to attack Inc-v3 and Inc-v4, as shown in Figure 4.Figure 4
Attack success rates (%) of adversarial examples generated for Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM, applied to Inc-v3 (white-box) and Inc-v4 (black-box) withT ranging from 1 to 20. It is noteworthy that the curves of Inc-v3 vs. I-FGSM, Inc-v3 vs. MI-FGSM, and Inc-v3 vs. AI-FGM overlap.As shown in the figure, the success rates of black-box attacks for several iterative methods decreased with the increase of the iteration quantities. However, AI-FGM always outperformed I-FGSM and MI-FGSM for a given value ofT.


#### 4.2.3. Perturbation Size

The effect of adversarial perturbation size on attack success rates was studied by setting the number of iterationsT to 10, with decay factors of 0.99 and 0.999. The size of perturbations ε ranged from 1 to 30. Adversarial examples generated on Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM were used to attack Inc-v3 and Inc-v4, as shown in Figure 5.Figure 5
Attack success rates (%) of adversarial examples generated for Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM, applied to Inc-v3 (white-box) and Inc-v4 (black-box) withε ranging from 1 to 30. It is noteworthy that the curves of Inc-v3 vs. I-FGSM, Inc-v3 vs. MI-FGSM, and Inc-v3 vs. AI-FGM overlaps.It is evident from the figure that attack success rates can reach 100% in white-box environments. Success rates for black-box models increased steadily with increasing values ofε. AI-FGM always outperformed I-FGSM and MI-FGSM for a given value of ε. In other words, AI-FGM can achieve comparable black-box attack success rates with smaller perturbations.


### 4.3. Attacking an Ensemble of Networks

The experimental results presented above suggest that AI-FGM can increase the transferability of adversarial examples. In addition, the success rate of black-box attacks can be further improved by attacking an ensemble of networks. As discussed in Section3.3, multiple models were attacked by fusing network logits. In the experiment, all seven networks described in Section 4.1 were used, and we generated adversarial examples on the ensemble of Inc-v3, Inc-v4, IncRes-v2, and Res-101, with FGSM, I-FGSM, MI-FGSM, NI-FGSM, and AI-FGM. Attacks were then performed on the other three defense models. Decay factors were set to 0.99 and 0.999; the number of iterations was 10; the maximum perturbation was 16; and each network had an equal ensemble weight of ωk=1/4. The corresponding results are shown in Table 2, and it is evident that AI-FGM was more effective than the other four methods on adversarially trained models.Table 2
Attack success rates (%) on adversarially trained networks under the ensemble-model setting.
AttackInc-v3ens3Inc-v3ens4Incres-v2ensAverageFGSM18.617.39.515.1I-FGSM20.116.911.416.1MI-FGSM49.443.225.939.5NI-FGSM46.741.623.037.1AI-FGM (ours)56.250.633.046.6The adversarial examples are generated on the ensemble of Inc-v3, Inc-v4, IncRes-v2, and Res-101.


### 4.4. Combination with Advanced Methods

In this subsection, we combined our AI-FGM with SI-NI-TI, SI-NI-DI, and SI-NI-TI-DI [18] and compared the black-box attack success rates of our extensions with the original methods under a single-model setting. The adversarial examples were generated on Inc-v3, with the number of iterations set to 10 and the maximum perturbation to 16. It is noteworthy that our combination with other methods is more like an improvement. For example, the combination of AI-FGM and SI-NI-TI was done by replacing the Nesterov optimizer in SI-NI-TI with Adam, thus resulting in SI-AI-TI. As shown in Table 3, our method SI-AI-TI-DI achieved an average attack success rate of 65.1%, surpassing the state-of-the-art attack by 10.7%.Table 3
Comparison of combined methods and original methods on attack success rates (%) against adversarially trained networks under the single-model setting.
AttackInc-v3ens3Inc-v3ens4Incres-v2ensAverageSI-NI-TI54.752.334.447.1SI-AI-TI (ours)56.354.439.450.0SI-NI-DI39.537.219.332.0SI-AI-DI (ours)50.851.527.743.3SI-NI-TI-DI62.359.341.754.4SI-AI-TI-DI (ours)72.669.853.065.1The adversarial examples are generated on the ensemble of Inc-v3, Inc-v4, Incres-v2, and Res-101.Furthermore, we generated adversarial examples on the ensemble models using our SI-AI-TI-DI. As shown in Table4, we achieved an average attack success rate of 95.0% on adversarially trained models under the black-box setting, which raised a new security issue for the robust deep neural networks.Table 4
Comparison of SI-AI-TI-DI and SI-NI-TI-DI under the ensemble-model setting.
AttackInc-v3ens3Inc-v3ens4Incres-v2ensAverageSI-NI-TI-DI95.493.789.592.9SI-AI-TI-DI (ours)96.296.092.795.0The adversarial examples are generated on the ensemble of Inc-v3, Inc-v4, Incres-v2, and Res-101.


## 4.1. Experimental Setup

(i)
Data set: if the original images cannot be classified correctly by the network, it is meaningless to generate adversarial examples based on these data. Therefore, we selected 1,000 images belonging to the 1,000 categories from the ImageNet validation set, all of which could be classified correctly by the tested networks. All images were scaled to299×299×3. As such, N=299×299×3 in Section 3.2.(ii)
Networks: seven different models were studied; four of which were normally trained networks (i.e., Inception-v3 (Inc-v3) [34], Inception-v4 (Inc-v4) [35], Inception-Resnet-v2 (IncRes-v2) [35], and Resnet-v2-101 (Res-101) [36]). The other three models were adversarially trained networks (i.e., ens3-adv-Inception-v3 (Inc-v3ens3), ens4-adv-Inception-v3 (Inc-v3ens4), and ens-adv-Inception-ResNet-v2 (IncRes-v2ens) [31]).(iii)
Other details: the proposed method was compared with the other methods discussed in Section3.1. All the experiments described in this paper were based on the L∞ norm. Thus, the momentum decay factor μ in Algorithm 1 was 1.0, and the stability coefficient δ in Algorithm 1 was 10−8, as suggested in [17].


## 4.2. Attacking a Single Network

We first performed adversarial attacks on a single network with FGSM, I-FGSM, MI-FGSM, and AI-FGM. Moreover, the adversarial examples were generated on four normally trained networks and tested on all seven networks. The results are shown in Table1, where the success rates are misclassification rates for the corresponding models, with adversarial examples used as input. The decay factors β1 and β2 in AI-FGM were set to 0.99 and 0.999, respectively. The maximum perturbation ε was 16. The number of iterations T for I-FGSM, PGD, MI-FGSM, and AI-FGM was 10. These methods are hereafter referred to as “iterative methods” without ambiguity. The effects of these parameter choices are discussed further in this section.Table 1
Attack success rates (%) for all seven networks included in the study.
AttackInc-v3Inc-v4Incres-v2Res-101Inc-v3ens3Inc-v3ens4Incres-v2ensInc-v3FGSM72.232.131.732.310.310.64.2I-FGSM100.027.523.020.96.24.71.8PGD99.818.914.714.46.16.23.2MI-FGSM100.054.350.644.013.913.36.5AI-FGM100.060.755.850.217.016.88.5Inc-v4FGSM39.464.929.533.012.211.15.0I-FGSM43.8100.027.423.46.16.32.4PGD32.099.817.716.66.45.83.1MI-FGSM69.9100.057.954.119.617.78.7AI-FGM72.9100.060.157.121.719.510.3Incres-v2FGSM37.631.857.931.413.712.06.8I-FGSM46.135.099.430.47.36.74.4PGD30.123.097.318.36.15.72.7MI-FGSM73.569.399.560.027.023.016.7AI-FGM74.671.199.561.831.325.720.5Res-101FGSM38.333.030.279.314.613.36.4I-FGSM35.128.325.199.58.46.73.7PGD30.922.420.999.67.37.23.4MI-FGSM60.055.350.699.522.919.811.3AI-FGM64.057.754.099.527.224.115.4The diagonal blocks indicate white-box attacks, while the off-diagonal blocks indicate black-box attacks.As shown in Table1, all four iterative methods attacked a white-box model with a near 100% success rate. AI-FGM performed better than the other three methods on all black-box models. For example, for adversarial examples generated on Inc-v3, AI-FGM had a success rate of 60.7% on Inc-v4, while MI-FGSM, PGD, I-FGSM, and FGSM reached 54.3%, 18.9%, 27.5%, and 32.1%, demonstrating the effectiveness of the proposed method. An original image and the corresponding adversarial example generated for Inc-v3 by AI-FGM are shown in Figure 2.Figure 2
Classification of a normal image and corresponding adversarial example by Inc-v3 and Inc-v4. The first row shows the top 10 confidence distributions for a clean image, for which both models provided a correct prediction. The second row shows the top 10 confidence distributions for the adversarial example generated for Inc-v3 by AI-FGM. It is evident that the adversarial example successfully attacked Inc-v3 (white-box) and Inc-v4 (black-box) with high confidence. (a) Clean. (b) Adversarial.
(a)(b)


### 4.2.1. Decay Factors

The termsβ1 and β2 control not only the decay amplitude of the step sizes but also the accumulation intensity of gradients for mt and vt. As such, they have a direct impact on attack success rates. We applied a grid-search method to identify an optimal set of β1 and β2 values. In the experiments, the maximum perturbation ε was set to 16, and the number of iterations T was set to 10. The values of β1 and β2 ranged from 0 to 1. Notably, we not only chose the values of β1 and β2 uniformly from 0.1 to 0.9 but also selected values close to 0 and 1, as shown in Figure 3. This was done to provide a more comprehensive study on the effects of decay factors, as we propose that the relationship between the success rate and the decay factors may be nonlinear. Adversarial examples were then generated on Inc-v3 with AI-FGM and used to perform attacks on Inc-v3, Inc-v4, and Inc-v3ens4. It is evident from the figure that regardless of the values of β1 and β2, attack success rates were always near 100% in white-box environments. Beyond that, attack success rates were more sensitive to β1 for black-box models, regardless of whether networks were normally or adversarially trained. Success rates were maximized when both β1 and β2 were close to 1.Figure 3
Attack success rates (%) of adversarial examples generated for Inc-v3 with AI-FGM, applied to Inc-v3 (white-box), Inc-v4 (black-box and normally trained), and Inc-v3ens4 (black-box and adversarially trained) withβ1 and β2 in the range of 0,1. (a) Inc-v3. (b) Inc-v4. (c) Inc-v3ens4.
(a)(b)(c)


### 4.2.2. The Number of Iterations

The effect of iteration quantities on success rates was studied by performing attacks using iterative methods. The maximum perturbationε was set to 16, and the decay factors β1 and β2 were set to 0.99 and 0.999, respectively. The number of iterations T ranged from 1 to 20. Adversarial examples generated on Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM were then used to attack Inc-v3 and Inc-v4, as shown in Figure 4.Figure 4
Attack success rates (%) of adversarial examples generated for Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM, applied to Inc-v3 (white-box) and Inc-v4 (black-box) withT ranging from 1 to 20. It is noteworthy that the curves of Inc-v3 vs. I-FGSM, Inc-v3 vs. MI-FGSM, and Inc-v3 vs. AI-FGM overlap.As shown in the figure, the success rates of black-box attacks for several iterative methods decreased with the increase of the iteration quantities. However, AI-FGM always outperformed I-FGSM and MI-FGSM for a given value ofT.


### 4.2.3. Perturbation Size

The effect of adversarial perturbation size on attack success rates was studied by setting the number of iterationsT to 10, with decay factors of 0.99 and 0.999. The size of perturbations ε ranged from 1 to 30. Adversarial examples generated on Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM were used to attack Inc-v3 and Inc-v4, as shown in Figure 5.Figure 5
Attack success rates (%) of adversarial examples generated for Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM, applied to Inc-v3 (white-box) and Inc-v4 (black-box) withε ranging from 1 to 30. It is noteworthy that the curves of Inc-v3 vs. I-FGSM, Inc-v3 vs. MI-FGSM, and Inc-v3 vs. AI-FGM overlaps.It is evident from the figure that attack success rates can reach 100% in white-box environments. Success rates for black-box models increased steadily with increasing values ofε. AI-FGM always outperformed I-FGSM and MI-FGSM for a given value of ε. In other words, AI-FGM can achieve comparable black-box attack success rates with smaller perturbations.


## 4.2.1. Decay Factors

The termsβ1 and β2 control not only the decay amplitude of the step sizes but also the accumulation intensity of gradients for mt and vt. As such, they have a direct impact on attack success rates. We applied a grid-search method to identify an optimal set of β1 and β2 values. In the experiments, the maximum perturbation ε was set to 16, and the number of iterations T was set to 10. The values of β1 and β2 ranged from 0 to 1. Notably, we not only chose the values of β1 and β2 uniformly from 0.1 to 0.9 but also selected values close to 0 and 1, as shown in Figure 3. This was done to provide a more comprehensive study on the effects of decay factors, as we propose that the relationship between the success rate and the decay factors may be nonlinear. Adversarial examples were then generated on Inc-v3 with AI-FGM and used to perform attacks on Inc-v3, Inc-v4, and Inc-v3ens4. It is evident from the figure that regardless of the values of β1 and β2, attack success rates were always near 100% in white-box environments. Beyond that, attack success rates were more sensitive to β1 for black-box models, regardless of whether networks were normally or adversarially trained. Success rates were maximized when both β1 and β2 were close to 1.Figure 3
Attack success rates (%) of adversarial examples generated for Inc-v3 with AI-FGM, applied to Inc-v3 (white-box), Inc-v4 (black-box and normally trained), and Inc-v3ens4 (black-box and adversarially trained) withβ1 and β2 in the range of 0,1. (a) Inc-v3. (b) Inc-v4. (c) Inc-v3ens4.
(a)(b)(c)


## 4.2.2. The Number of Iterations

The effect of iteration quantities on success rates was studied by performing attacks using iterative methods. The maximum perturbationε was set to 16, and the decay factors β1 and β2 were set to 0.99 and 0.999, respectively. The number of iterations T ranged from 1 to 20. Adversarial examples generated on Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM were then used to attack Inc-v3 and Inc-v4, as shown in Figure 4.Figure 4
Attack success rates (%) of adversarial examples generated for Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM, applied to Inc-v3 (white-box) and Inc-v4 (black-box) withT ranging from 1 to 20. It is noteworthy that the curves of Inc-v3 vs. I-FGSM, Inc-v3 vs. MI-FGSM, and Inc-v3 vs. AI-FGM overlap.As shown in the figure, the success rates of black-box attacks for several iterative methods decreased with the increase of the iteration quantities. However, AI-FGM always outperformed I-FGSM and MI-FGSM for a given value ofT.


## 4.2.3. Perturbation Size

The effect of adversarial perturbation size on attack success rates was studied by setting the number of iterationsT to 10, with decay factors of 0.99 and 0.999. The size of perturbations ε ranged from 1 to 30. Adversarial examples generated on Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM were used to attack Inc-v3 and Inc-v4, as shown in Figure 5.Figure 5
Attack success rates (%) of adversarial examples generated for Inc-v3 with I-FGSM, MI-FGSM, and AI-FGM, applied to Inc-v3 (white-box) and Inc-v4 (black-box) withε ranging from 1 to 30. It is noteworthy that the curves of Inc-v3 vs. I-FGSM, Inc-v3 vs. MI-FGSM, and Inc-v3 vs. AI-FGM overlaps.It is evident from the figure that attack success rates can reach 100% in white-box environments. Success rates for black-box models increased steadily with increasing values ofε. AI-FGM always outperformed I-FGSM and MI-FGSM for a given value of ε. In other words, AI-FGM can achieve comparable black-box attack success rates with smaller perturbations.


## 4.3. Attacking an Ensemble of Networks

The experimental results presented above suggest that AI-FGM can increase the transferability of adversarial examples. In addition, the success rate of black-box attacks can be further improved by attacking an ensemble of networks. As discussed in Section3.3, multiple models were attacked by fusing network logits. In the experiment, all seven networks described in Section 4.1 were used, and we generated adversarial examples on the ensemble of Inc-v3, Inc-v4, IncRes-v2, and Res-101, with FGSM, I-FGSM, MI-FGSM, NI-FGSM, and AI-FGM. Attacks were then performed on the other three defense models. Decay factors were set to 0.99 and 0.999; the number of iterations was 10; the maximum perturbation was 16; and each network had an equal ensemble weight of ωk=1/4. The corresponding results are shown in Table 2, and it is evident that AI-FGM was more effective than the other four methods on adversarially trained models.Table 2
Attack success rates (%) on adversarially trained networks under the ensemble-model setting.
AttackInc-v3ens3Inc-v3ens4Incres-v2ensAverageFGSM18.617.39.515.1I-FGSM20.116.911.416.1MI-FGSM49.443.225.939.5NI-FGSM46.741.623.037.1AI-FGM (ours)56.250.633.046.6The adversarial examples are generated on the ensemble of Inc-v3, Inc-v4, IncRes-v2, and Res-101.


## 4.4. Combination with Advanced Methods

In this subsection, we combined our AI-FGM with SI-NI-TI, SI-NI-DI, and SI-NI-TI-DI [18] and compared the black-box attack success rates of our extensions with the original methods under a single-model setting. The adversarial examples were generated on Inc-v3, with the number of iterations set to 10 and the maximum perturbation to 16. It is noteworthy that our combination with other methods is more like an improvement. For example, the combination of AI-FGM and SI-NI-TI was done by replacing the Nesterov optimizer in SI-NI-TI with Adam, thus resulting in SI-AI-TI. As shown in Table 3, our method SI-AI-TI-DI achieved an average attack success rate of 65.1%, surpassing the state-of-the-art attack by 10.7%.Table 3
Comparison of combined methods and original methods on attack success rates (%) against adversarially trained networks under the single-model setting.
AttackInc-v3ens3Inc-v3ens4Incres-v2ensAverageSI-NI-TI54.752.334.447.1SI-AI-TI (ours)56.354.439.450.0SI-NI-DI39.537.219.332.0SI-AI-DI (ours)50.851.527.743.3SI-NI-TI-DI62.359.341.754.4SI-AI-TI-DI (ours)72.669.853.065.1The adversarial examples are generated on the ensemble of Inc-v3, Inc-v4, Incres-v2, and Res-101.Furthermore, we generated adversarial examples on the ensemble models using our SI-AI-TI-DI. As shown in Table4, we achieved an average attack success rate of 95.0% on adversarially trained models under the black-box setting, which raised a new security issue for the robust deep neural networks.Table 4
Comparison of SI-AI-TI-DI and SI-NI-TI-DI under the ensemble-model setting.
AttackInc-v3ens3Inc-v3ens4Incres-v2ensAverageSI-NI-TI-DI95.493.789.592.9SI-AI-TI-DI (ours)96.296.092.795.0The adversarial examples are generated on the ensemble of Inc-v3, Inc-v4, Incres-v2, and Res-101.


## 5. Discussion

It is commonly acknowledged that the training of neural network models is similar to the generation of adversarial examples, especially for gradient-based generation methods. Hence, techniques used in the training of neural networks, to improve model generalizability, can also be adopted to improve the transferability of adversarial examples. Since the Adam optimizer is often used in the training of neural networks, to improve convergence and achieve better performance on test sets, a second momentum term and decay step size in Adam were included in the AI-FGM algorithm. This was done to improve the transferability of adversarial examples. Furthermore, we suggest that other techniques (such as data augmentation) could be used to improve the performance of adversarial examples further in black-box environments.


## 6. Conclusions

In this study, we proposed the Adam iterative fast gradient method to improve the transferability of adversarial examples. Specifically, the Adam algorithm was modified to increase its suitability for the generation of adversarial examples. The proposed method improved both the iteration update direction and step size. The effectiveness of the proposed method was verified by an extensive series of experiments with ImageNet. Compared with previous gradient-based adversarial example generation techniques, our method improved attack success rates in black-box environments. In addition, we further improved the transferability of adversarial examples by combining our approach with existing methods and attacking ensemble models, which achieved a state-of-the-art attack success rate against adversarially trained networks. We suggest the proposed method could be used as a reference for other iterative gradient-based methods. For example, data augmentation could be combined with AI-FGM to achieve better attack performance.



---

*Source: 9983309-2021-06-07.xml*