Adversarial Training: embedding adversarial perturbations into the parameter space of a neural network to build a robust system
Authors: Shixian Wen
Laurent Itti
Venue:  (2020)
Keywords: Adversarial Training
Adversarial Examples

Abstract: Adversarial training, in which a network is trained on both adversarial and clean examples, is one of the most trusted defense methods against adversarial attacks. However, there are three major practical difficulties in implementing and deploying this method - expensive in terms of extra memory and computation costs; accuracy trade-off between clean and adversarial examples; and lack of diversity of adversarial perturbations. Classical adversarial training uses fixed, precomputed perturbations in adversarial examples (input space). In contrast, we introduce dynamic adversarial perturbations into the parameter space of the network, by adding perturbation biases to the fully connected layers of deep convolutional neural network. During training, using only clean images, the perturbation biases are updated in the Fast Gradient Sign Direction to automatically create and store adversarial perturbations by recycling the gradient information computed. The network learns and adjusts itself automatically to these learned adversarial perturbations. Thus, we can achieve adversarial training with negligible cost compared to requiring a training set of adversarial example images. In addition, if combined with classical adversarial training, our perturbation biases can alleviate accuracy trade-off difficulties, and diversify adversarial perturbations.

Under review as a conference paper at ICLR 2019
ADVERSARIAL TRAINING: EMBEDDING ADVERSARIAL PERTURBATIONS INTO THE PARAMETER SPACE OF A NEURAL NETWORK TO BUILD A ROBUST SYSTEM
Anonymous authors Paper under double-blind review
ABSTRACT
Adversarial training, in which network is trained on both adversarial and clean examples, is one of the most trusted defense methods against adversarial attacks. However, there three major practical difficulties in implementing and deploying this method expensive terms of extra memory and computation costs; accu- racy trade-off between clean and adversarial examples; and lack of diversity of adversarial perturbations. Classical adversarial training uses fixed, precomputed perturbations in adversarial examples (input space). In contrast, we introduce dynamic adversarial perturbations into the parameter space of the network, by adding perturbation biases the fully connected layers of deep convolutional neural network. During training, using only clean images, the perturbation biases are updated in the Fast Gradient Sign Direction to automatically create and store adversarial perturbations by recycling the gradient information computed. The network learns and adjusts itself automatically to these learned adversarial pertur- bations. Thus, to requiring a training set of adversarial example images. In addition, combined with classical adversarial training, perturbation biases can alleviate accuracy trade-off difficulties, and diversify adversarial perturbations.
achieve adversarial training with negligible cost compared
INTRODUCTION
Neural networks have lead to series of breakthroughs in many fields, such image classification tasks (He et al. 2016), natural language processing (Devlin et al. 2018). Model performance on clean examples was the main evaluation criterion for these applications until the realization of the adversarial example phenomenon by (Szegeyy et al. (2013): Biggio et al. (2013). Neural networks were shown to be vulnerable to adversarial perturbations: carefully computed small perturbations added to legitimate clean examples (adversarial examples, Fig. [p) can cause misclassification on tfate-of-the-art machine learning models. Thus, building deep learning system that is robust to both adversarial examples and clean examples has emerged as critical requirement.
Researchers have proposed number of adversarial defense strategies increase the robustness of a deep learning system. Adversarial training, in which a network is trained on both adversarial examples (Iadv) and clean examples (Zein) with true class labels y, is one of the few defenses against adversarial attacks that withstands strong attacks. Adversarial examples are the summation of adversarial perturbations lying inside the input space (5f) and clean examples: Tads = Tein + ô, (Fig. [).. Given a classifier with classification loss function L and parameters 0, the objective function of adversarial training is:
min L( + 87, Feins yi 0) 6
For example, Goodfellow et al. (2014) used adversarial training and reduced the test error rates from 89.4% to 17.9% on adversarial examples of the MNIST dataset. HHang et al.] (2015) built a robust model against adversarial examples by punishing misclassified adversarial examples.
Despite the efficacy of adversarial training in building a robust system, there are three major practical difficulties while implementing and deploying this method. Difficulty one: adversarial training

---

Under review as a conference paper at ICLR 2019
is expensive in terms of memory and computation costs. Producing an adversarial example requires multiple gradient computations. In a practical scenario, we further produce more than one adversarial examples for each clean example (Tramèr et al. 2017). We need to at least double the amount of memory, to store those adversarial examples alongside the clean examples. In addition, during adversarial training, the network has to train both clean and adversarial examples; hence, adversarial training requires at least twice the computation power than just training on clean examples. For example, even on reasonably-sized datasets, such as CIFAR-I0 and CIFAR-I00, adversarial training can take multiple days on a single GPU. As consequence, although adversarial training remains among the most trusted defenses, it has only been within reach for research labs having hundreds of GPUs [[[ref]]]. Difficulty two: accuracy trade-off between clean examples and adversarial examples- although adversarial training can improve the robustness against adversarial examples, it sometimes hurts accuracy on clean examples. There is an accuracy trade-off between the adversarial examples and clean examples (Di et al. 2018 Raghunathan et al.) 2019 Stanforth et al. [019 Zhang et al. 2019). Because most of the test data in real applications are clean examples, test accuracy on clean examples should be as good as possible. Thus, this accuracy trade-off hinders the practical usefulness of adversarial training because it often ends up lowering performance on clean examples. Difficulty three: lack of diversity of adversarial perturbations even though one might have sufficient computation resources train network on both adversarial and clean examples, it is unrealistic and expensive to introduce all unknown attack samples into the adversarial training. For example, [Trmerr et al. (2017) proposed Ensemble Adversarial Training which can increase the diversity of adversarial perturbations in a training set by generating ad adversarial perturbations transferred from other models (they won the competition on Defenses against Adversarial Attacks). Thus, broad diversity of adversarial examples is crucial for adversarial training.
To solve the above three practical difficulties, we dive into details of adversarial training and analyze the causes of these difficulties. The cause of difficulty one: In Fig. [b. a classical deep convolutional network is trained on both clean examples and adversarial examples during adversarial training. Since the adversarial examples are the summation of clean examples and adversarial perturbations (Fig. [h), the adversarial training uses duplicate information from clean examples and the clean portion of perturbed examples. If we could use information from clean examples only once and generate corresponding adversarial perturbations during the training of clean examples, we could reduce computation costs significantly. The cause of difficulty two: reduce computation costs, one might train the neural network on only adversarial examples. Since adversarial examples are the summation of clean examples and adversarial perturbations, contains overlapped information from both clean examples and ad adversarial perturbations. However, failing to incorporate intact information of clean examples hurts test accuracy on clean examples (Di et al. 2018 Raghunathan et al.) (019). The cause of difficulty three: even though one might have sufficient computation resources and can afford training on both clean and adversarial examples, the finite amount of adversarial examples still limits the diversity of adversarial perturbations. Thus, the lack of diversity of adversarial perturbations decreases test accuracy on both clean and adversarial examples (Tramer et al. 2017).
Here, we introduce new adversarial perturbation bias (6AP) to the last few fully connected layers of the deep convolutional network, replacing the normal bias term (Fig. [... The main novelty is that instead of using a fixed, precomputed adversarial perturbations in adversarial examples (input space), we introduce dynamic adversarial perturbations into the parameter space of the network, lying inside the adversarial perturbation bias. During training on clean examples, the adversarial perturbation bias automatically creates and stores adversarial perturbations by recycling the gradient information computed and through updating using the Fast Gradient Sign Method (FGSM) Goodfellow al.
(2014) with the objective function:
min L(I yi 0, AA) 0,5AP
Where Tein is clean example with true class θ is the network parameters, 6AP is adversarial perturbation bias, and I. is the classification loss function. Both adversarial perturbations in adversarial examples and in adversarial perturbation biases are calculated using FGSM. The only difference is that adversarial perturbations in adversarial examples lie inside the input space and adversarial perturbations in adversarial perturbation bias lie inside the parameter space. Thus, the adversarial perturbation biases play a role like adversarial examples in the input space, but they can inject the learned adversarial perturbations directly to the network parameter space. Then, the network leams and adjusts itself automatically to these injected adversarial perturbations. Thus, it is a robust system against adversarial attacks. During training only clean examples, the network with adversarial
2

---

Under review as a conference paper at ICLR 2019
(a) Adversarial examples
clean examples
adversarial perturbations
+
(b)
Adversarial Training
Classifier
Inputs
Feature extraction
Neuron
Clean examples
A Adversarial examples
Outputs
Convolutional non-linear layers
Fully connested layer
Two duplicate information from dleen examples One information adversanal parturbations
(c)
Adversarial Training with negligible costs
Classifier
Adversarial
Inputs
Feature extraction
perturbation blas (bap) Neuron
Clean examples
Ouipuls
Convolutional and non-linear layers
Fully connected Adversartal perturbation blas creates and stores information of adversarial perturbations automat cally while training on clean examples
information cleen examples
Figure 1: (a) Adversarial examples are a summation of clean examples and adversarial perturbations. (b) Adversarial training for classical deep convolutional network model. Adversarial training both clean and adversarial examples requires two duplicate information from clean examples and one information on adversarial perturbations. Thus, the adversarial training procedure expensive. (c) Solving the expensive training procedure. By introducing adversarial perturbation biases to the fully connected layers of deep convolutional network, we can achieve adversarial training with negligible costs. While training only on clean examples, adversarial perturbation biases automatically create and store adversarial perturbations by recycling the gradient information computed when updating model parameters. The network learns and adjusts itself to these injected adversarial perturbations. Thus, the adversarial perturbation bias increases the robustness against adversarial examples.
perturbation bias shows largely improved test accuracy on adversarial examples, with negligible extra costs. In addition, during classical adversarial training combined with our approach, the network with adversarial perturbation bias alleviate the accuracy trade-off and diversify available adversarial perturbations. To show the efficacy of the network with adversarial perturbation bias on the above three difficulties, consider three different scenarios that correspond different abilities to access increasing computation power.
RELATED WORK AND BACKGROUND INFORMATION
Attack models and algorithms: Most of the attack models and algorithms focus on causing mis- classification of target classifiers. They find adversarial perturbation 87. Then, they create

---

Under review as a conference paper at ICLR 2019
adversarial example by adding the adversarial perturbation to clean example Zeln' Fadv Izln +ōj- The adversarial perturbation sneaks the clean example Eein out of its natural class and into another. Given fixed classifier with parameters 6, clean example Zein with true label y. and classification loss function L, the bounded non-targeted adversarial perturbation 61 is computed by solving:
max (z ô1, y: 0), subject to ||ör|b S 81
where |1-Ilp is some lp-norm distance metric, and is the adversarial perturbation budget.
In this work, we consider the most popular non-targeted method Fast Gradient Sign Method (FGSM) by (Goodfellow et al. (2014), in the context of -bounded attacks: e sign(Vzat L(Tenn y. 0))
Tade = Fetn
In white box attacks (Qiu et al. 2019). the adversaries have complete knowledge about the target model, including algorithm, data distribution and model parameters. The adversaries can generate ad adversarial perturbations by identifying the most vulnerable feature space of the target model. In this work, we use white box Fast Gradient Sign Method with 0.3 to generate adversarial perturbations.
White box attack model:
Modification of bias terms: Modifying the structure of bias terms in the fully connected layer can be beneficial. WWnn & Itti (2019) used bias units to store the beneficial perturbations (opposite to the well-known adversarial perturbations). WWen & Itti (2019) showed that the beneficial perturbations, stored inside task-dependent bias units, can bias the network outputs toward the correct classification region for each task, allowing single neural network to have multiple input to output mappings. Multiple input output mappings alleviate the catastrophic forgetting problem (McCloskey & Cohen 1989) in sequential learning scenarios (a previously learned mapping of an old task is crased during learning of new mapping for new task). Here, we leverage similar idea. During training clean examples, adversarial perturbation bias automatically creates and stores the adversarial perturbations. The network automatically learns how adjust to these adversarial perturbations. Thus, it helps us to build robust model against adversarial examples.
3 NETWORK STRUCTURE AND ALGORITHM FOR A NEURAL NETWORK WITH ADVERSARIAL PERTURBATION BIAS.
Two different structures for adversarial perturbation bias: naive adversarial perturbation bias and multimodal adversarial perturbation bias. Naive adversarial perturbation bias (5AP) is just a normal bias term of the fully connected layer. The only difference is that during backpropaga- tion, we update the naive adversarial perturbation bias using the Fast Gradient Sign Method. For multimodal adversarial perturbation bias, we design the multimodal adversarial perturbation bias (6ΛP = MAP MAPWAPP WAP) as a product of bias memories (MAP E R. x/1) and bias weight (W E Ra*"), where n is the number of neurons in the fully connected layer, and the number of bias memories. Multimodal adversarial perturbation bias has more degrees of freedom to better fit multimodal distribution. Thus it could yield better results than the naive adversarial perturbation bias. The update rules for the network with naive (blue) and multimodal (red) adversarial perturbation bias are shown in Alg. T and Alg.
Algorithm 1 Forward rules for both naive and multimodal adversarial perturbation bias Input:
Xactivations Activations from the last layer
ÓAP- Adversarial perturbation bias
Output: Y = aacticoatioss +
where: W
Normal neuron weights.
4 THE FUNCTION OF ADVERSARIAL PERTURBATION BIAS IN THREE DIFFERENT SCENARIOS
Scenario 1, adversarial training with negligible costs: company with modest computation resources can only afford training the network on clean examples, but still wants to have a moderately robust system against adversarial examples. During training on clean examples, when updating model
4

---

Under review as a conference paper at ICLR 2019
Algorithm 2 Backward rules for naive and multimodal adversarial perturbation bias
Notations: W- Normal neuron weights Xactivations Activations from the last layer
AP Adversarial perturbation bias (native or multimodal)
AAP Bias memories for multimodal adversarial perturbation bias
W AP Bias weights for multimodal adversarial perturbation bias
Adversarial perturbation budgets for Fast Gradient Sign Method
During the training:
Input: Grad Gradients from the next layer
output: dW Grad dx antivationn) // gradients for the normal neuron weights
dX activations WT .dol Grad)
// gradients for activations to last layer
For naive adversarial perturbation bias:
AP = esign() number of samples Grad)
I| gradients for the naive adversarial perturbation bias using FGSM
For multimodal adversarial perturbation bias:
dmAp = 6 ssign (W AP-dott Grad)) // gradients for the bias memories of multimodal adversarial perturbation bias using FGSM
dWAr AP Grad (m AP)
// gradients for the bias weights of multimodal adversarial perturbation bias
After the training:
For multimodal adversarial perturbation bias: Keep the AP as the product of mAp and WAP
Delete MAp and W AP to reduce memory and parameter costs
parameters with objective function Eqn. 2 in the backward pass of backpropagation, adversarial perturbation bias automatically creates and stores adversarial perturbations (calculated from clean examples). In the forward pass, the adversarial perturbation bias injects the learned adversarial perturbations the network. As result, the neural network learns the structure of these adversarial perturbations and adjusts itself to against these injected adversarial perturbations automatically during the training on clean examples. Although the network cannot be trained on the adversarial examples to access the information of adversarial perturbations directly because of the modest computation power, adversarial perturbation bias can still provide the information of adversarial perturbations to the network. Thus, adversarial perturbation bias can improve the model's robustness against adversarial examples (see Results) and maintain the highest test accuracy on clean examples while barely increase any computation costs. For naive adversarial perturbation bias, we do not introduce any extra computation costs beyond FGSM. For multimodal adversarial perturbation, the extra computation costs are further increased by matrix multiplication in the forward pass and two dots products in the backward pass per fully connected layer. The state-of-the-ee deep convolutional structure might have up three fully connected layers classifier, yielding up three extra multiplication and six extra dot products computation costs. The extra computation costs are negligible comparing to generating huge amount of extra adversarial examples and training the neural network on both clean and adversarial examples.
Scenario counteracting the adversarial perturbations lying inside of the adversarial exam- ples: to have strong robustness against adversarial examples, company with moderate computation resources can only afford generate adversarial examples and train network only on adversarial examples. Training only on adversarial examples can achieve high robustness against adversarial examples, but decreases the test accuracy on the clean examples. During training on adversarial examples, when updating model parameters, in the backward pass of backpropagation, adversarial perturbation bias automatically creates and stores adversarial perturbations (calculated from the adversarial examples) by recycling the gradient information computed. In the forward pass, the adversarial perturbation bias injects the learned adversarial perturbations to the network. There is a high chance that the adversarial perturbations (6AP calculated from adversarial examples) lying inside the parameter space of neural network (adversarial perturbation bias) would counteract the adversarial perturbations (ō, calculated from clean examples) lying inside the input space (images of adversarial
5

---

Under review as a conference paper at ICLR 2019
examples). In mathematical term, in the view of the network, Zade + Icn + Eeln where 61 counteract with the 6AP. Adversarial perturbation bias can convert some adversarial exam- ples into clean examples because of the counteraction. Thus, it largely improves the testing accuracy on the clean examples (see Results), while it still maintains a high testing accuracy against adversarial examples.
Scenario 3, diversify the adversarial perturbations: a company with abundant computation re- sources can afford to train a network on both clean and adversarial examples. In the generation of adversarial examples, the adversarial perturbations lying inside the adversarial examples are calculated from the clean examples. In comparison, during the training of the network on both clean and adversarial examples, in the backward pass, the adversarial perturbation bias creates and stores adversarial perturbations calculated from both clean and adversarial images by recycling the gradient information computed. This allows the adversarial perturbation bias to diversify the available adversarial perturbations and inject these varied adversarial perturbations to the network. In addition, adversarial perturbations lying inside the adversarial examples are fixed and precomputed. In addition, the adversarial perturbations lying inside the adversarial perturbation bias change dy- namically for every training epoch. This further increases the variations of adversarial perturbations during the training. In mathematical representation, the diversified group has: clean examples Teln+ adversarial examples Tado: perturbed clean examples Fein + AAA P and perturbed adversarial examples Lade + SA Adversarial perturbation bias can diversify the adversarial perturbations available to the neural network using adversarial training. As a result, the diversity improves the testing accuracy on both clean and adversarial examples (see Results).
5 EXPERIMENTS
5.1 DATASET AND NETWORK STRUCTURE
MNIST LeCunn et al. 1998) is dataset with handwritten digits, has training set of 60,000 examples, and test set of 10,000 examples. FashionMnist (Xiao et al. (017) is a dataset with article images, has training set of 60,000 examples, and test set of 10,000 examples. We use LeNet (LeCun jet al. 1998) as a classifier (classical LeNet). Then, we create our version of LeNet (LeNet with adversarial perturbation bias) by adding adversarial perturbation bias into the fully connected layers, replacing the normal bias. We generate the adversarial examples using Fast Gradient Sign Method for both classical LeNet and LeNet with adversarial perturbation bias.
5.2 QUANTITATIVE RESULTS
We test the LeNet with adversarial perturbation bias and classical LeNet on both clean and adversarial examples from MNIST and FashionMNIST dataset, under 3 different computation budgets: Negligible costs (Training only on clean examples), moderate extra costs (Training only on adversarial examples) and high extra costs (Training on both clean and adversarial examples).
Negligible cost- training only on clean examples: our method can largely increase test accu- racy on adversarial examples and slightly increase test accuracy on clean examples. When the neural network can only be trained on clean examples because of modest computation power, LeNet with adversarial perturbation bias achieves a slightly higher test accuracy on clean examples than classical Lenet (Fig. 2 MNIST: 99.17% vs. 99.01 %, FashionMNIST: 89.54% vs. 89.17%). In addition, for the test accuracy on adversarial examples (Fig. 2A), classical Lenet can only achieve 18.0 18.08% on the MNIST dataset and 1 11.87% on the FashionMNISTT dataset. In comparison, LeNet with adversarial perturbation bias can achieve 98.88% on the MNIST dataset and 53.88% on the FaSHionMNIST dataset. Thus, for companies with modest computation resources, adversarial pertur- bation bias can help a system achieve moderate robustness against adversarial examples, while only introducing negligible computation costs (c.g., on MNSSi, our method only uses 59% training time compared to the training time of adversarial training with just one adversarial example per clean example, saving 43.51 minutes training time for 500 training epochs NVIDIA Tesla- V100 platform. The saving would be huge on a larger dataset such as Imagenet (Deng et al. 2009)).
Moderate extra costs training only adversarial examples: our method can slightly increase the test accuracy on adversarial examples and largely increase the test accuracy on clean ex- amples. In the generation of adversarial examples, it takes multiple gradient computation through

---

Under review as a conference paper at ICLR 2019
Our method LeNet with Clean examples
adversanal al examples
Classical Network
Clean examples
adversarial examples
LeNet
adversarial perturbation
MNIST Datasel
FashionMNiST Dataset
Test accuraoy adversarial examples
98.5%
98 88.8%%
99.275 99.27%
0.1%%
Test accuracy on adversanial examples
96%
95.39%
11.0%%
97.01%
92.07%
91.49%
96%
919
54.07%
18.08%
0%
11.87%
65%%
95 54%
71. 71.544%
sccuracy on clean examples
Test accuracy on claan examples
97.32%
99.17% 09.01%
15% 09.00000000 00%
90% (89.3333 53%% 89.17%
% 40% Extra High Costs Training Costs
Nogligible Costs
Moderate Costs
High Costs Training Costs
Negligible Costs
Moderate Costs
Training on clean examples adverserial examples
Training on
clean examples &
Training on
Training on clean examples adversorial examples clean examples &
Training on
Training on
adversorial examples
adversarial examples
Figure 2: Performance of LeNet with adversarial perturbation bias (our method) versus classical network LeNet under three different computation power budgets for three different scenarios. The adversarial examples are generated using FGSM (e = 0.3). 1. Negligible costs scenario (A, B): training neural network only on clean examples. LeNet with adversarial perturbation bias demonstrates much increased test accuracy on adversarial examples, and slightly increased test accuracy on clean examples compared to classical LeNet. Moderate extra costs scenario (C, D): training neural network only on adversarial examples. LeNet with adversarial perturbation bias demonstrates largely increased test accuracy on clean examples and slightly increased test accuracy on adversarial examples compared to classical LeNet. 3. High extra costs scenario (E, F): training the neural network on both clean and adversarial examples. LeNet with adversarial perturbation bias demonstrates slightly increased test accuracy both clean and adversarial examples compared to classical LeNet.
backpropagation to calculate adversarial perturbations. As a consequence, training only on adversarial examples is slightly more expensive than training only clean examples. On classical networks, although training only on adversarial examples can achieve a high test accuracy on adversarial exam- ples (Fig. C classical LeNet: MNIST 99.01 %, FashionMNIST 91.49%), it hurts the test accuracy on clean examples. For example, for classical LeNet, training on clean examples can achieve test accuracy 99.01% % for the MNIST dataset and 89. for the FashionMNIST dataset (Fig 2 For classical LeNet, training on the adversarial examples can only achieve much worse testing accuracy 95.54% for MNIST dataset and 65.64% for FaSHionMNIST dataset (Fig. (D) In comparison, for LeNet with adversarial perturbation bias, training on the adversarial examples can achieve testing accuracy 97.72% for MNIST dataset and 71.54% for FashionMNIST dataset (Fig. (D). This accuracy is still worse than the accuracy of training only clean examples, but it is much better than the accuracy of classical LeNet training only on adversarial examples. Thus, for companies with moderate computation resources, we build strong robust system against adversarial examples while achieving better testing accuracy on clean examples.
High extra costs training on both clean and adversarial examples: method can diversify the adversarial perturbations, so it can slightly increase the test accuracy on both clean and adversarial examples. LeNet with adversarial perturbation bias can achieve slightly higher accuracy
7

---

Under review as a conference paper at ICLR 2019
on clean examples than classical Lenet (Fig. 2F MNIST 99. 19% vs. 99.09%, FashionMNIST 89.65% vs. 89.49%). In addition, LeNet with adversarial perturbation bias can achieve slightly higher accuracy on adversarial examples than classical LeNet (Fig. 2E MNIST 97.62% vs. 97.01%. 01, FashionMNIST 95.39% vs. 94.98%). Even for companies with abundant computation resources, it is still helpful to adapt adversarial perturbation bias because it diversifies the adversarial perturbations.
5.3 INFLUENCE OF THE ADVERSARIAL PERTURBATION BUDGETS AND STRUCTURE ADVERSARIAL PERTURBATION BIAS
The higher the adversarial perturbation budgets, the higher the chance it can successfully attack neural network. However, attacks with higher adversarial perturbation budgets are easier to detect by a program or by humans. For example, c 0.3 (Fig. [T) represents very high noise, which makes FashionMNIST images difficult to classify, even by humans. But the distribution differences between the adversarial examples and clean examples are so large that they can be easily captured by defense programs. Thus, S 0.15 is a good attack since the differences caused by adversarial perturbations are too small to be detected by most defense programs. For small adversarial perturbations (Fig. Bl S 0.15). by just training on clean images, LeNet with adversarial perturbation bias achieves moderate robustness against adversarial examples with negligible costs. Thus, it is really beneficial adapt our method for companies with modest computation power, who still want to achieve moderate robustness against adversarial examples. Structure of adversarial perturbation bias: under the FGSM attacks 0.3, Fig. 3D), multimodal adversarial perturbation bias works better than naive perturbation bias on MNIST. However, naive adversarial perturbation bias works better than multimodal perturbation bias on FashionMNIST. Thus, the structure of adversarial perturbation bias is hyperparameter for different datasets. The number of bias memories of multimodal adversarial perturbation bias is another hyperparameter. we have insufficient bias memories, there is not enough degrees of freedom to learn a good multimodal distribution. If we have excessive bias memories, the model overfits the multimodal distribution.
Adversarial perturbation budgets:
(a)
MNIST dataset FashionMNIST dataset
(b)
MNIST dataset
FashionMNIST dataset
1.2
1.2
Adversarial examples
0.9346
0.9888
0.9855
0.9834
0.9835
0.982
0.9834
0.9874
0.9835
0.
0.9888
1
Adversarial examples
Accuracy on
0.8
0:6764
Accuracy on
0.8
....114
0.6948
0.6875 0.6905
0.6867 0.8761
0.6
4204
0.4918
0.4805
0.5407
0.6
0.4
0.4
0.2
0.2
625
2500
5625
10000
0
n: number of bias memories of multimodal advesarial perturbation bias
0.05
0.1
0.15
0.2
0.25
0.3
value in Fast Gradient Sign Method
(n=0: naive adversarial perturbation bias)
Figure 3:
Test accuracy on adversarial examples after training only on clean examples from MNIST (Blue) or FashionMNIST (Orange) datasets. (a): Influence of different adversarial perturbation budgets (e) on LeNet with adversarial perturbation bias. (b): Influence of different structures of adversarial perturbation bias LeNet under adversarial attack: FGSM = 0.3.
CONCLUSION
In this paper, we propose a new method to solve the three major practical difficulties while imple- menting and deploying adversarial training, by embedding adversarial perturbation into the parameter space (adversarial perturbation bias) of neural network. There are three major contributions that benefit for companies with different levels of computation resources 1. Modest Computation power: adversarial training with negligible costs. 2. Moderate computation power: alleviate the accuracy trade-off between clean examples and adversarial examples. 3. Abundant computation power: diversify the adversarial perturbations available to the adversarial training.
8

---

Under review as a conference paper at ICLR 2019
REFERENCES
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Smdićć, Pavel Laskov. Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases, pp. 387-402. Springer, 2013.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255, Icee, 2009.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1rXiv:1810004805. 2018.
Xinhan Di, Penggian Yu, and Meng Tian. Towards adversarial training with moderate performance improvement for neural network classification. arXiv preprint arXivv/807.000 /807,00344 2018.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Ruitong Huang, Bing Xu, Dale Schuurmans, and Csaba Szepesvári. Learning with a strong adversary. CoRR, abs/1511 1.0004444 2015. Bttppbtp://axxxx org/ abs http://aaaa....oor////11.....0.. 03034
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient- based learning applied to document recognition. Proceedings of the IEEE, (6(11))2278-2324, 1998.
Michael McCloskey and Neal Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Shilin Qiu, Qihe Liu, Shijie Zhou, and Chunjiang Wu. Review of artificial intelligence adversarial attack and defense technologies. Applied Sciences, 9(5)9909, 2019.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang. John C Duchi, and Percy Liang. Adversarial training can hurt generalization. arXiv preprint arXiv:1906.0603: 2019.
Robert Stanforth, Alhussein Fawzi, Pushmeet Kohli, et al. Are labels required for improving adversarial robustness? arXiv preprint arXiv:1905.1 13725, 2019.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 132.66/9, 2013.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.0720. 2017.
Shixian Wen and Laurent Itti. Beneficial perturbation network for continual learning. CoRR, abs/1906. 10528, 2019. URL/ttpp://arxxiv.rrg//bb///////
10528
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: 2017.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael I Jordan. Theoretically principled trade-off between robustness and accuracy. arXiv preprint 1 2019.
9


Reviews:
[{"review_id": "S1eAMK6pKH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors try to tackle the problem of adversarial examples by introducing a special set of bias weights into the neural network. There are serious clarity issues with the writing of this paper. It cites Wen & Itti 2019 but provides few motivations and explanations of algorithmic choices. \n\nMy questions include: \n  - Is there a separate bias term for clean examples? When is the adversarial bias used? Is it only for adversarial examples? \n  - How is the adversarial bias term updated? In Algorithm 2, its gradient is a sum over samples, but over what samples? A mini-batch? \n  - What are the multi-modal weights? There is no forward equation on how they are used. We only have their updates in Algorithm 2. \n\nThese are all very confusing, given the central importance of Algorithm 2 to the paper. The authors should start with how the new bias terms affect inference and predictions, before going to their updates. \n\nThere are also limitations in the experiments. Only Fast Gradient Sign Method (FGSM) is used to generate adversarial examples. The more powerful projected gradient descent should be used for a better test against adversarial examples. Also, only MNIST and FashionMNIST are tested. The authors should consider including CIFAR10, CIFAR100, or other datasets. \n\nOverall I think the issues with clarity and experiments in the paper make it not ready for publication yet. \n\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Hyey5xwaYB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes perturbation biases as a counter-measure against adversarial perturbations. The perturbation biases are additional bias terms that are trained by a variant of gradient ascent. The method imposes less computational costs compared to most adversarial training algorithms. In their experimental evaluation, the algorithm achieved higher accuracy on both clean and adversarial examples.\n\nThis paper should be rejected because the proposed method is not well justified either by theory or practice. Experiments are weak and do not support the effectiveness of the proposed method.\n\nMajor comments:\nSince the evaluations of defense algorithms are often misleading [1], it requires throughout experiments or theoretical certifications to confirm the effectiveness of defense methods. However, the experiment configuration in this paper is not satisfactory to demonstrate the robustness of defended networks. The followings are a list of concerns.\n1) Experiments are limited to small datasets and networks. Since some phenomena only appear in larger datasets [2], there is a concern that the proposed method also works on other datasets.\n2) The attack algorithm used for the evaluation is weak. We can confirm this by observing the label leakage [2] in the experimental results. It is hard to judge which defenses are most effective, even within the tested datasets and models.\n3) The \"adversarial training\" baseline used in the experiment is weird. Adversarial training typically generates adversarial examples during the process of the neural networks' optimization instead of using precomputed adversarial examples. Baseline methods should be stronger, for example, adversarial training with PGD [3].\n\n[1] Athalye et al. \"Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples.\" ICML 2018\n[2] Kurakin et al. \"ADVERSARIAL MACHINE LEARNING AT SCALE.\" ICLR 2017\n[3] Madry et al. \"Towards Deep Learning Models Resistant to Adversarial Attacks.\" ICLR 2018", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "Hye_47mNYH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThis paper proposes to introduce adversarial perturbations into intermediate layers of a neural network, to achieve more efficient adversarial training.\nThe idea has been proposed before. The paper is based on, and perpetrates, a number of fundamental misconceptions about adversarial examples. A lack of many relevant citations indicates that the authors are not familiar with the related work done in this field over the past three years. This paper is currently clearly below the bar for ICLR.\n\nWe recommend that the authors read, understand, and cite at least the list of papers below before revising their paper:\n\n- Madry et al., \"Towards deep learning models resistant to adversarial attacks\", 2017\n- Sabour et al., \"Adversarial Manipulation of Deep Representations\", 2016\n- Tsipras et al., \"Robustness May Be at Odds with Accuracy\", 2019\n- Carlini & Wagner., \"Towards Evaluating the Robustness of Neural Networks\", 2017\n- Carlini et al., \"On evaluating adversarial robustness\", 2019\n\nIn particular, this should help clarify the following misconceptions:\n- Adversarial training does not require training on both clean and adversarial examples. The best results are often obtained by only training on adversarial examples.\n- Adversarial training does not require extra memory\n- While expensive, adversarial training does not require days of computation on hundreds of GPUs (not even on ImageNet)\n- There is no evidence that adversarial training with PGD produces \"non-diverse\" adversarial examples (for the chosen perturbation set)\n- FGSM is not a good benchmark for training or evaluation", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "N/A", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to introduce perturbation biases as a counter-measure against adversarial perturbations. The perturbation biases are additional bias terms that are trained by a variant of gradient ascent. Serious issues were raised in the comments. No rebuttal was provided.
Decision: Reject