A DVERSARIAL TRAINING WITH PERTURBATION GENERATOR NETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Despite the remarkable development of recent deep learning techniques, neural networks are still vulnerable to adversarial attacks, i.e., methods that fool the neural networks with perturbations that are too small for human eyes to perceive.
Many adversarial training methods were introduced as to solve this problem, using adversarial examples as a training data.
However, these adversarial attack methods used in these techniques are ﬁxed, making the model stronger only to attacks used in training, which is widely known as an overﬁtting problem.
In this paper, we suggest a novel adversarial training approach.
In addition to the classiﬁer, our method adds another neural network that generates the most effective adversarial perturbation by ﬁnding the weakness of the classiﬁer.
This perturbation generator network is trained to produce perturbations that maximize the loss function of the classiﬁer, and these adversarial examples train the classiﬁer with a true label.
In short, the two networks compete with each other, performing a minimax game.
In this scenario, attack patterns created by the generator network are adaptively altered to the classiﬁer, mitigating the overﬁtting problem mentioned above.
We theoretically proved that our minimax optimization problem is equivalent to minimizing the adversarial loss after all.
Beyond this, we proposed an evaluation method that could accurately compare a wide-range of adversarial training algorithms.
Experiments with various datasets show that our method outperforms conventional adversarial training algorithms.
I NTRODUCTION
Deep learning has shown the impressive performance in all areas of artiﬁcial intelligence, such as image classiﬁcation and speech recognition (Hinton et al., 2012; Krizhevsky et al., 2012).
These advances lead to a broad application of deep neural networks in various real-life tasks.
There are still, however, severe security issues such as adversarial examples, which hinder the use of machine learning system until a complete defense is constructed against multiple adversarial attacks.
Adversarial examples are data samples that are close to real data samples, which cause a given neural network to misclassify.
The basic idea of adversarial examples is to ﬁnd a sample that increases the loss value of a neural network in the neighborhood of training data (Szegedy et al., 2014).
The perturbation on the original training data is so small that it makes the adversarial examples indistinguishable from the original examples.
Many authors proposed methods that make neural networks robust to adversarial examples (Papernot et al., 2016;
Goodfellow et al., 2015; Szegedy et al., 2014;
Miyato et al., 2016).
One of the methods is an adversarial training, which re-trains the neural network with adversarial examples generated by adversarial attacks.
Adversarial training with powerful attacks would guarantee robustness, but the recent fatal attack methods (Szegedy et al., 2014; Papernot et al., 2016; Carlini & Wagner, 2017;
Moosavi-Dezfooli et al., 2016) require high computational complexity because of their iterative optimization.
Therefore, they are not compatible with adversarial training.
Methods that quickly produce adversarial examples, such as fast gradient sign (Goodfellow et al., 2015) or projected gradient descent (Kurakin et al.
(2017); Madry et al.
(2018)), have been used for practical adversarial training.
While the above adversarial training methods are empirically successful, they might be susceptible to future attackers, and this makes the defense procedure useless.
If an algorithm for generating an adversarial example is ﬁxed in adversarial training, the network could overﬁt to the speciﬁc algorithm.
In this paper, we introduce a novel adversarial training framework that increases the robustness against various adversarial attacks.
Stemming from GAN framework, we devised a method in which the classiﬁer network and a perturbation generator network are alternately trained.
To be more speciﬁc, the generator network generates a perturbation image that maximizes the loss function of the classiﬁer network, and the classiﬁer network is trained through the corresponding adversarial image with the true label.
Through this minimax optimization between the two networks, the classiﬁer network can improve robustness against many different attacks, as the attack pattern of the generator network is constantly modiﬁed depending on the classiﬁer network.
This procedure can be used in practical adversarial training since adversarial perturbations can be produced by a forwardpropagation.
We generalized Madry et al.
(2018)’s research on adversarial loss to theoretically support our technique, and we also proposed a method that can fairly evaluate the performance of adversarial training algorithms.
R ELATED WORKS
The goal of our work is to construct defensive mechanisms to adversarial attacks.
To alleviate the security problem, the adversarial robustness of neural networks has been studied in the literature.
One of the intuitive ways to increase robustness is to re-train with adversarial examples, which are called adversarial training.
This method uniformly smoothen the ground-truth label decision region close to the original data points.
In the context of smoothness, there exists adversarial examples that hold very low conﬁdence on the ground-truth label in the vanilla decision region before applying robust optimization.
Szegedy et al.
ﬁrst proposed a method to generate adversarial examples.
They use boxconstraint L-BFGS optimization to ﬁnd the examples.
This holds the exact formulation of adversarial examples, but because of its exhausted optimization procedure, it is not suitable for practical adversarial training.
Goodfellow et al.
(2015) introduced an algorithm that quickly generates adversarial examples by using one-step gradient update, which is called fast gradient sign method.
In addition, they ﬁrst proposed a realistic adversarial training method which injects the adversarial examples into the training data.
This method is not strong enough to generate high-quality examples and is far from robust optimization.
Kurakin et al.
(2017) suggested an iterative version of fast gradient method (FGM) attack called Projected Gradient Descent (PGD), which is much closer to the optimal adversarial examples.
Adversarial training can be formulated with the robust optimization problem which minimizes the loss of the optimal adversarial examples in the (cid:15) -ball of all the original data points.
This gives rise to the following minimax game, which is the main theoretical background of our work:
[Equation]
They approximated the above minimax game by PGD based adversarial training to reduce computational complexity issue.
The gradient descent based adversarial examples for robust optimization is not adaptive.
Therefore, those neural networks are vulnerable to other types of adversarial attacks (Athalye et al., 2018).
Several works studied the methods that generate stronger adversarial attacks(Athalye et al., 2018;
Lee et al., 2017; Papernot et al., 2017; Moosavi-Dezfooli et al., 2016;
Dong et al., 2018; Song et al., 2018).
Carlini & Wagner (2017) pin-points that defensive distillation network (Papernot et al.
(2016)) is not practical in that it exploits gradient masking, so they devised a powerful attack algorithm that avoids this problem.
In an attempt to eliminate the gradient masking problem of softmax function, they adopted logits Z in objective function, and discovered an appropriate adversarial noise for each image utilizing line-search technique.
However, most of these works have high computational cost, so they are difﬁcult to be applied to adversarial training.
There are many other defense methods that are not based on adversarial training (Li et al., 2019; , Junbo).
The above robust optimization problem can be generalized as convex outer adversarial polytope, which relaxes the activation function as a convex form to prevent misclassiﬁcation (Wong & Kolter, 2018).
Certiﬁed defense algorithms guarantee at least a certain bounds of the proper label probability distributions against adversarial examples (Cohen et al., 2019; Liu et al., 2019; Raghu-
[Figure]
nathan et al., 2018).
In addition, some researchers recently studied the theoretical backgrounds of adversarial robustness (Dohmatob, 2019;
Wang et al., 2018; Roth et al., 2019).
P ROPOSED METHOD 3. N OTATIONS
We denote a labeled training set by ( x , y ) ∼ P data , where x ∈ R H × W × C represents input images with height H , width W , and channel C , and y ∈ { 1 , 2 , . . .
, K } is a label for an input x .
We use two neural networks in the proposed method.
One is a standard K -class classiﬁer network F ( x ; θ ) which is deﬁned by:
[Equation]
Where F ( x ; θ ) represents the class probability vector computed using the softmax function.
The other is a perturbation generating network G ( ∇ x F, x , y ; φ ) , which is deﬁned by:
[Equation]
Note that G ( ∇ x F, x , y ; φ ) represents the perturbation of the input image x , where ∇ x F = ∇ x F ( x ; θ ) y denotes the gradient of class probability of the true label with respect to the input images x .
3. A DVERSARIAL T RAINING WITH GENERATIVE MODEL
The entire procedure of our algorithm is shown in Figure 1.
Goodfellow’s work on GAN inspired us to make the classiﬁer and the perturbation generator compete with each other.
Classiﬁer F deﬁnes the network we are aiming to train and increase the robustness, and the perturbation generator G is the network which produces the perturbations that maximize the loss function of the classiﬁer.
The classiﬁer network is trained with adversarial images produced by the generator network with the true label.
The generator network assigns image x , label y, and a gradient image ∇ x F as inputs, which is trained to maximize the loss function of the classiﬁer.
In other words, F and G play the following two-player minimax game:
[Equation]
By the time this minimax game is complete, the classiﬁer will have been trained with various attacks produced by the generator with the enhanced robustness against powerful adversarial attacks, while the generator will no longer ﬁnd any vulnerability in the classiﬁer, therefore only producing random noises.
In Equation (4), c L is a hyper-parameter that adjusts the ratio between two cost functions.
If c L is very low, it will only ﬁnd trivial solutions with extremely large perturbation power, and if c L is very high, it will only generate zero-perturbation images.
Therefore, it is crucial to determine an appropriate c L .
The theoretical meaning of c L will be discussed in Appendix A.
We believe that the strength of our algorithm is the ﬂexibility of the generator’s attack method.
Conventional adversarial training algorithms such as FGM have a ﬁxed attack method.
For example, adversarial perturbation in FGM is generated by normalizing the gradient of a data sample, which is scaled by (cid:15) .
A classiﬁer can be easily made robust against adversarial examples generated by these ﬁxed methods.
The following are two possible methods.
CASE 1.
The classiﬁer is trained to gradually reduce the gradient at data points.
Gradually reducing the gradient prevents FGM from generating meaningful adversarial example from the gradient (also known as a gradient masking problems).
CASE 2.
The classiﬁer can only be trained to reduce the loss under the surface of the (cid:15) -norm ball centered around a data point.
As a result, the network would be robust against adversarial examples generated by FGM, but could still be very vulnerable against adversarial examples located inside the (cid:15) -norm ball.
Through the above methods, a classiﬁer can be easily overﬁt to FGM attacks, but it becomes more vulnerable to other powerful attacks such as Carlini&Wagner L 2 attacks as shown in Figure 2.
Through our experiments under FGM adversarial training, we observed a CASE 1 overﬁtting problem when (cid:15) is small, and a CASE 2 overﬁtting problem when (cid:15) is large.
The above gradient masking problem has also been discussed by other papers on adversarial training (Tram ` er et al., 2018).
This problem also occurs when the iteration number of PGD is relatively low.
On the other hand, our algorithm continuously updates the parameters of a generator to maximize the classiﬁer loss, which tends to produce more general attacks to a given classiﬁer.
This means that a classiﬁer cannot easily overﬁt to attacks from a generator in our method.
Experimental results also support our arguments.
E XPERIMENT 4. E XPERIMENTAL SETUP
We used CIFAR-10 and CIFAR-100 for our datasets, to verify the robustness of our trained network.
We normalized the pixel value of the image to [0,1] prior to network training.
This section only presents the results from CIFAR-100.
The experiment with CIFAR-10 showed similar results, and interested readers can refer to the appendix for its results.
The model architecture and parameters for CIFAR are given in Table 1.
We used conventional
ConvPool-CNN as the classiﬁer network, and the generator network was designed to efﬁciently use gradients, images, and labels.
One might assume that hyperbolic tangent function should be used for the ﬁnal activation function of the generator network.
However, if a hyperbolic tangent function is used for generating the perturbation image, the adversarial image created must be clipped again to the proper value of the image, i.e. 0 < =
x i + δ i <
=
1 for all i .
This is known as a box-constraint problem, which might cause the network to get stuck in extreme regions (Carlini & Wagner, 2017).
Therefore, we practiced the following technique proposed by Carlini & Wagner (2017) to avoid the clipping issue.
[Equation]
For our baseline for comparison, we used a naive network trained only with clean examples.
For our control group, we set Goodfellow et al.
(2016)’s
adversarial training with Fast Gradient
Method (FGM), and Madry et al.
(2018)’s adversarial training with Projected Gradient Descent (PGD).
For attack methods, we used FGM, Momentum Iterative Method (MIM), DeepFool, and
Carlini&Wagner (C&W), and evaluated the robustness of the network through the accuracy of the adversarial examples and the mean and median value of the L 2 norm of the perturbaion generated by each attack.
All the attacks and adversarial training methods above are L 2 -bounded.
Detailed evaluation method will be discussed in section 4.2.
All of our experiments used a single RTX 2080 ti GPU with Cleverhans adversarial examples library (Papernot et al., 2018) to construct adversarial attacks, build defenses, and make comparison more effectively.
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
4. E VALUATION M ETHOD
We applied the following average distortion metric suggested by Carlini & Wagner (2017), in order to fairly evaluate the robustness of the network for various adversarial attacks.
[Equation]
where D is a successful adversarial example set
[Equation] [Equation]
The above ρ represents the mean value of L 2 norm of the perturbation derived from the successful adversarial examples from the attack, the same value as the area under the curve in Figure 2.
Although ρ can be measured for any attack methods, it is best to measure ρ for the most powerful attack.
Thus, we used ρ cw for Carlini&Wagner
L 2 attack on all of our experiments as the evaluation metric for robustness of the network.
However, it is not sufﬁcient to use only the above metric in evaluating the robustness of the adversarial training algorithm.
In most adversarial training process, there are some hyperparameters which could adjust the trade-offs of the accuracy of benign examples and the adversarial examples ( (cid:15) for FGM, PGD adversarial training, c L for our algorithm).
The above ρ cw tends to increase as the accuracy of the benign examples decline, as it can be shown in Figure 2.
In an extreme case, if the network classify almost all the images as a single class, benign accuracy (the accuracy of clean examples) would converge to 1% (for CIFAR-100), but the ρ cw would spike.
This trade-off occurs during the training process as well.
Figure 2 illustrates how benign accuracy increases, and ρ cw [Figure] decreases during the training process.
Thus, for fair comparison of the robustness of the networks, it is desirable to match the accuracy of the benign examples before comparing ρ cw .
To better compare the performance of different adversarial algorithms, we must ﬁrst train the models by adjusting hyper-parameters for each adversarial training, calculate benign accuracy and ρ cw for each trained model, and then draw a graph with the calculated values, connecting each relevant data point.
Naturally, the structure of the classiﬁer used for each adversarial training must be identical.
We will call this graph robustness-curve.
Figure 5 displays the robustness-curves for FGM adversarial training, PGD adversarial training, and our algorithm.
It should be noted that the method with outer curve is a better adversarial training algorithm since ρ cw is higher at the same benign accuracy.
4. A TTACK PERFORMANCE OF THE PERTURBATION GENERATOR NETWORK
In order to verify the attack performance of the proposed generator network, we used two ﬁxed classiﬁer networks adversarially trained with FGM ( (cid:15) = 0.5) and PGD ( (cid:15) =0.5), respectively.
For each classiﬁer network, we compared the performance of FGM attack, PGD attack, and PGN (proposed) attack.
PGN was fully trained on the training dataset.
We plotted the accuracy of the classiﬁers for these three attacks in terms of average perturbation distortion.
The results are shown in Figure 3.
For each model, it is observed that our algorithm is capable of much more powerful attack than PGD and FGM.
This difference in performance clearly justiﬁes the superior defense performance of the adversarial training with PGN in the following section.
The perturbation generator network can be trained to fully exploit incoming gradients, images, and labels to generate adversarial perturbation optimized for the classiﬁer.
As for FGM or PGD, they have a ﬁxed norm, with a single attack direction towards the gradient.
Now with PGN, it has no restriction on both direction and size, and can be transformed into an optimized perturbation for the classiﬁer.
4. D EFENSE PERFORMANCE ON VARIOUS ATTACKS
Based on the methods introduced in 4.2, we compared the robustness of the network trained by our suggested technique with that trained by the conventional adversarial training methods.
FGM attack, MIM attack, DeepFool, and Carlini&Wagner Attack were used as attack methods.
In whitebox attacks, adversarial examples were generated through direct access to the model’s gradient, while in black-box attack, accuracy was measured through the adversarial examples produced by an independently trained network.
Table 2 exhibits the robustness of the network when all the benign accuracy values of the adversarial networks are balanced to that of the naive network, and Figure 4 displays three perturbation L 2 power (x) - accuracy (y) graphs for C&W attack, with the benign accuracy for each adversarial networks set to 68%, 66%, and 63%, respectively. FGM and MIM are attack methods that ﬁnd the adversarial examples that can maximize the loss function of the classiﬁer network on ﬁxed L 2 norm of perturbation power, so the higher the accuracy
[Table]
[Figure]
of the adversarial examples, the more robust the network.
On the other hand, DeepFool and C&W attack ﬁnd the adversarial examples with the lowest L 2 norm of perturbation power that can fool the network; therefore, the robust network would have higher mean and median values of the adversarial perturbation power.
As you can see from Table 2, our algorithm outperforms the other adversarial training algorithms against all the attack methods.
In white-box attacks, our algorithm showed the highest accuracy of adversarial examples against FGM and MIM attacks, and the highest power of adversarial perturbations by DeepFool and Carlini&Wagner.
Also, in black-box attacks, our method proved to classify the adversarial examples with greater accuracy compared with the other adversarial training algorithms.
According to Table 2, FGM adversarial training and PGD adversarial training show a very similar performance.
This is because minimal (cid:15) was applied to match the benign accuracy of the baseline network.
Since a neural network is locally linear, this minimal (cid:15) would make PGD and FGM generated adversarial examples to be almost identical.
As you can see from Figureand Figure 5, as (cid:15) increases, Madry’s method shows a more robust performance compared with
Goodfellow’s method.
Training speed is also a crucial issue in adversarial training.
The proposed algorithm is slower than
FGM because it trains the generator after ﬁnding the gradient image, while FGM immediately uses the gradient image to train the classiﬁer.
On the other hand, our algorithm is faster than Madry’s which use PGD (multi-step gradient descent) to ﬁnd the adversarial image.
Note that the more iteration steps of PGD, the larger the speed-gap between Madry’s and our algorithm we get.
0.6 cw 0.
b e n i g n a cc u r a c y
GoodFellowMadryOurs 0.
0.6 cw 0.
b e n i g n a cc u r a c y # of filters (x1)
# of filters (x2) # of filters (x3)
4. V ARYING H YPER - PARAMETERS
As mentioned in Section 4.2, adversarial training algorithms have a trade-off relationship between benign accuracy and robustness metric ρ cw .
Figure 5 visualizes the relationship with a plot consisting of the data points of benign accuracy and ρ cw , which are collected by using various hyper-parameters for each adversarial training.
For FGM and PGD adversarial training, the data points with higher ρ cw are models trained with bigger (cid:15) , while for our algorithm, the data points with larger ρ cw are models trained with lower c L .
It should be noted that the robustness-curve is an appropriate indicator for performance evaluation of the adversarial training algorithms, since it displays a comprehensive set of ρ cw with corresponding benign accuracy.
As demonstrated in Figure 5, our algorithm outperforms all the other adversarial training algorithms under all benign accuracy.
Furthermore, we plotted the robustness-curve by proportionally increasing the number of ﬁlters in each convolutional layer of the classiﬁer.
As can be observed in the second plot of Figure 5, the robustness-curve moves to the right as the capacity of the model increases, which means that the classiﬁer may still be underﬁtted.
In other words, the classiﬁer trained with only clean examples tend to overﬁt easily to the training data with even a low capacity, whereas the classiﬁer trained with various adversarial examples tend to underﬁt instead even with higher capacity networks.
Although we were not able to deal with higher capacity due to the limits of the current hardware technology, it is expected that a far greater network capacity may be needed to achieve a human-level robustness.
C ONCLUSION
This study proposed a novel adversarial training method that boosts the robustness of a deep neural network against adversarial attacks.
Based on a GAN framework, the classiﬁer network and the generator network play a two-player minimax game, which improves the robustness of a classiﬁer against adversarial examples.
In generating adversarial examples, we use a trainable perturbation generator network instead of a ﬁxed function as in most of conventional adversarial training methods.
this method tend to overﬁt less, and strengthens the robustness against many different kinds of attacks.
Our proposed method is far more robust than existing adversarial training techniques.
Since it computes adversarial examples through one-step inference, it is also more advantageous in training speed, compared to other techniques that use multiple steps in inner maximization.
Our experiment with CIFAR datasets have also proved the advantage of our approach, as the network trained by our method showed improved robustness and the state-of-the-art performance against various attacks with different noise power.
Although the proposed approach compares favorably with other methods, it is believed that there is still room for improvement.
One future direction is to study a generator network which is most effective for adversarial training.
APPENDIX A THEORETICAL BACKGROUND
Madry et al.
(2018) has presented the following adversarial loss instead of the conventional loss in his paper.
[Equation]
This signiﬁes that among the given training data points, it ﬁnds the data point that has the maximum loss against perturbation from the allowed perturbations set S , and minimizes that speciﬁc loss.
In other words, it trains Classiﬁer F to satisfy F ( x + δ ; θ ) y
= 1 for all δ in S in allowed perturbations set S , which is only feasible when S is a very small norm ball.
However, since our perturbation generator network can create perturbations with any size of power, merely applying the above adversarial loss would generate only the trivial perturbations with extremely high power.
In order to extend the allowed perturbations set S to all possible perturbations, we assume that optimal
F ( x + δ ; θ ) y has a normal distribution over L 2 norm of the δ as shown in Figure 6.
To be more precise, we want to train Classiﬁer F to satisfy F ( x + δ ; θ ) y
= p n ( (cid:107) δ (cid:107) 2 ) , where n ∼ N (0 , σ 2 ) and p n is the Gaussian distribution function with 0 mean and σ 2 variance, and our adversarial loss corresponding to Equation (7) is as follows:
[Equation]
Suppose δ ∗ is deﬁned as δ ∗ ( F, x , y )
=
argmax δ (cid:2)
Loss ( x + δ , y ; θ ) − c L (cid:107) δ (cid:107) 22 − C (cid:3) .
then, min θ
[Equation]
G would converge to δ ∗ , assuming G has sufﬁciently high capacity and ∇ x F provides enough information on the classiﬁer F , and the constant value C could be ignored since we are only interested in ﬁnding the parameter θ of the classiﬁer.
We can derive the Equation (4), with c L = 12 σ .
The optimal point for F and G would be the point where Loss ( x + δ , y ; θ ) ≤ c L (cid:107) δ (cid:107) 22 + C in all data points, and here we can ﬁnd the classiﬁer network F that has the improved robustness against adversarial examples.
[Figure]
B PERFORMANCE FOR CIFAR-
[Table]
[Figure]
C SOME ADVERSARIAL EXAMPLES GENERATED BY THE PROPOSED GENERATOR NETWORK
[Table]
perturbation power ( || ∆ x || 2 ≈ 0 . 1)
[Table]
Original Image
[Table]
perturbation
[Table]
( || ∆ x || 2 ≈ 1 . 0)
[Table]
( || ∆ x || 2 ≈ 2 . 5)
mushroom fox palm tree
[Table]
bear squirrel snake
D E FFECT OF EACH INPUT ELEMENT ON THE ROBUSTNESS OF THE NETWORK :ABLATION STUDY
[Figure]
original without gradient without label without image