Adversarial Interpolation Training: A Simple Approach for Improving Model Robustness
Authors: Haichao Zhang
Wei Xu
Venue:  (2020)
Keywords: adversarial training
adversarial robustness

Abstract: We propose a simple approach for adversarial training. The proposed approach utilizes an adversarial interpolation scheme for generating adversarial images and accompanying adversarial labels, which are then used in place of the original data for model training. The proposed approach is intuitive to understand, simple to implement and achieves state-of-the-art performance. We evaluate the proposed approach on a number of datasets including CIFAR10, CIFAR100 and SVHN. Extensive empirical results compared with several state-of-the-art methods against different attacks verify the effectiveness of the proposed approach. 

Under review as a conference paper at ICLR 2020
ADVERSARIAL INTERPOLATION TRAINING A SIMPLE APPROACH FOR IMPROVING MODEL ROBUSTNESS
Anonymous authors Paper under double-blind review
ABSTRACT
We propose simple approach for adversarial training. The proposed approach utilizes an adversarial interpolation scheme for generating adversarial images and accompanying adversarial labels, which are then used in place of the original data for model training. The proposed approach is intuitive to understand, simple to implement and achieves state-of-the-ar performance. We evaluate the proposed approach on a number of datasets including CIFARIO, CIFARIOI 100 and SVHN. Extensive empirical results compared with several state-of-the-an methods against different attacks verify the effectiveness of the proposed approach.
INTRODUCTION
Deep learning- based techniques have achieved outstanding performance in many tasks such as im-
age classification ( (Krizhevsky et al. 2012), speech recognition (Hinton et al., 2012) and video game
playing (Mnih et al., 2015). Despite of these encouraging progress, it has been shown that these models could be easily attacked by adversarial examples (Szegedy et al. 2014 Biggio et al. [013
Carlini & Wagner 2018 Lin et al. 2017). The ubiquitous of adversarial examples across many tasks (Szegedy al.) 2014 Lin et al. 2017 Eykholt al. 2018: Carlini & Wagner 2018 and
the fact that they are transferable between different models (Tramer et al. [017] Charles et al. al., 2017) raise great concerns on security of such models and hinder their actual deployment in real world applications. Researchers have been actively working on un- derstanding the cause of adversarial examples (Szegedy al. 2014:: Goodfellow et al. 2015 and approaches for improving model robustness against them (Madry et al. 2018 Tramer et al. 2018) Liao et al. 2018 Yan et al. 2018). number of theories have been developed for explaining the ex- istence of adversarial examples. [Szegedy et al. (2014) explain that adversarial examples are possible because the image space densely filled with low probability adversarial pockets. [Godffllow al. (2015) argue that the adversarial examples are caused by the linear nature of deep networks. Tanay Griffin (2066) provide the perspective that adversarial examples exist because the class boundary extends beyond the data sub- mub-manifold and can be lying close to it in some cases. It has been further shown in some recent work that adversarial examples can be decomposed into categories with dif- ferent causes including off-manifold ones (Stutz al. 2019 Jacobsen al. 2019) and those due to natural test error (Stutz et al.) 2019 Jacobsen et al. 2019 Ford et al. 2019).
2019 Moosavi-Deztoooi
Recently, Myas et al. (2019) provide perspective that adversarial vulnerability is caused by non- robust features. The reasoning is that there are abundant of useful correlations that exist in natural data, thus it is natural expect the models could learn exploit any of them if no preference is given (c.f. Table 7 However, models relying on superficial statistics (non-robust features) can be brittle and generalize poorly, thus suffering from adversarial attacks (Ilyas al.) 2019). The natural idea is therefore using only robust features for lcarning. However robust features are not easy to construct directly (Madaan & Hwang. 2019). In contrast, non-robust features are much easier to construct, using the standard adversarial example generation procedure (Szegedy et al.) 2014 Goodfellow et al., 2015 Madry et al., 2018). Therefore, we can leverage non-robust features instead for robust learning. This is typically achieved by constructing robustified dataset where the new (perturbed) images are constructed by adding non-robust features to the clean images, and then performing model training using the perturbed images in place of the original ones. This procedure is essentially the standard adversarial training approach (Goodfellow et al. 2015 Madry et al. 2018), which has been shown to be effective for defending against adversarial attacks (Athalye al.
2018

---

Under review as a conference paper at ICLR 2020
In this paper, we present simple approach for improving model robustness under the framework of adversarial training. We craft adversarial examples by performing a so-called adversarial inter- polation operation (Section 3) between samples, and expect the model should perform robustly against adversarial interpolation-induued perturbations within a constrained neighborhood.
The contribution of this work can be summarized as follows:
we propose an adversarial interpolation approach for generating adversarial samples, which is simple, intuitive and effective;
analyze the proposed approach and its connections with previous methods;
we leverage adversarial interpolation for adversarial training and verify its effectiveness im- proving model robustness compared with number of strong baselines across several datasets.
PRELIMINARIES
2.1
ADVERSARIAL ATTACK AND DEFENSE
Adversarial examples have been investigated in the seminal work of Biggio et al. (2013) Szegedy
et al. 40144) and have attracted increasing attention recently (Biggio et al. 2013 Goodfellow et al.)
2015
Tramer et al. 2018 Madry et al. 2018 Athalyy et al. 2018 Biggio & Roli 2017 Wang al. 2019). Szegedy al. (2014) investigated the vulnerabilities of CNNs to adversarial examples
and proposed an L-BFGGS-basee approach for attack generation. [Goodfellow et al. (2015) developed fast gradient sign method (FGSM) for generating adversarial images. After that, many types of attacks have been developed in the past few years
Darrsvvi-eefooliccoo et 2016 Carlini & Wagner, 2017 Su al.) 2017 Xiao et al. 2018 Brown et al. 2017 Brendel et al. 2018). On the defense side, many researchers are actively working on improving model robustness against adversarial at- tacks Meng & Chen (2017); Xie al. (2018); Metzen et al. (2017); Liu al. (2018); Song et al. 2217 Liao et al. (2018) Samangouei et al. 2018); Prakash et al. (2018); 8); Wang & Zhang (2119). Recently, Athalye al. (2018) pointed out a phenomenon called gradient masking and showed that many existing defence methods did not actually improve model robustness thus giving a false sense of robustness because of it. According to AAthalye et al. (2018) adversarial training (Goodfellow) cet al. [ Madry et al. 2018) is one of the effective defense method against adversarial attacks. It improves model robustness by solving a minimax problem (Goodfellow et al., 2015; Madry et al,, 2018) ming mmxx5 B(r.e) [(ă, y; 8)]. where the inner maximization is responsible for provid- ing the most challenging example for the current model while the outer optimization loop improves the model performance on these adversarial examples (Madry al. 2018). The inner optimization can be solved approximately by a one-step approach such as FGSM (Goodfellow et al., 2015). or a
multi-step projected gradient descent (PGD) method (Madry al. 2018
2.2 INCORPORATING REGULARIZATION IN MODEL TRAINING
Regularization is important for deep learning models with a large number of parameters, as there are multiple possible solutions that perform equally well on clean data (Garipov et al., 2018 Draxler al. 2018 Athiwaratkun et al.) 2019), the model by default has no particular preference over any of them if no preferences are given. In the case of model robustness, we need to have a way to convey preference on robustness to the learning process. This can be viewed as a form of regulariza- tion. Conventional regularization techniques include weight decay or other forms of constraints on component function (Arjovsky et al., (017). Data augmentation is another way to regularization by manipulating the input. The idea is that an nuisance factor can be de-emphasized by generating training data that incorporates variations of the factor, leading invariance w.r.t.hhis factor. Follow- ing this idea, conventional data augmentation transforms the training images using label preserving transformations (Krizhevsky al. 2012). Zhang et al. (2018) proposed a mixup approach which jointly transforms images and labels. It generates training examples (both image and label) by lin- ear interpolation between pairs of natural examples, thus introducing a linear inductive bias in the vicinity of training samples. Similar to standard data augmentation, adversarial training incorporates implicit regularization by perturbing the training data using non-robust features of other classes thus reducing the sensitivity of the model w.r.t.non-robus. features (c.f. Table 7). It has been shown that adversarial training is equivalent to regularized training with a data dependent regularization term (Ororiia et al. 2017). Since adversarial training incorporates preference to robustness against perceptual insensitive perturbations, the models trained this way have been shown to have better
interpretability (Engstrom et al. 2019: Stutz al. 2019)
2

---

Under review as a conference paper at ICLR 2020
(x
D
02 0f ar
min D(f,
8f
fz
T
Figure 1: Adversarial Interpolation for adversarial image generation. It takes a clean image x to be perturbed, and another distinct image z', performs feature extraction using the feature extractor fo(+) and then measures the distance in the feature space using a distance measure D. The perturbed image is generated by minimizing the feature space distance with respect to z'.
ADVERSARIAL INTERPOLATION TRAINING
We present the adversarial interpolation training method in this section. The intuition is that the semantic meaning of the image is unchanged under mild perturbations, thus the network should be robust against them as well. On the one hand, it is well-known that adding random perturbations to the imput ineffective for improving model robustness. On the other hand, adversarial training using classification loss generated perturbations suffers from label leaking (Kurakin et al. 2017) These motivate the development of the proposed method detailed in the sequel.
3.1 ADVERSARIAL. INTERPOLATION
We first review several existing types of perturbations in the sequel and then introduce our proposed approach. We will use i to denote the perturbed version of x.
Definition Random Perturbation. Given a sample x, the randomly perturbed sample is gener- ated by sampling from the neighborhood ball ( c) centered at x with radius e as H ~ ((r,e) e).
It is clear that the random perturbations are data-agnostic and isotropic in each direction. It has been shown that this types of perturbation is equivalent to Tikhonov regularization (Bishop) 1995) and is less effective in improving model robustness due to its data and model agnostic nature. Note that we have omitted the pixel value range constraint to avoid notation clutter. Instead of randomly sampling, (discriminative) adversarial perturbations are generated with the guidance of classification loss.
Definition 2 Discriminative Perturbation. Given a sample and its label y. we define the discrim- inative perturbation as: 3"
- arg minl -CE(T, y).
where CE(-) denotes the network taking as input and computes the cross-entropy (CE) loss be- tween the predicted class probabilities and the input label y. Discriminative perturbation is data- dependent and model/task-aware (i.e. white-box), representing a form of strong attacks for a given model. Naturally. it has been used in adversarial training for online generation of training data that is adversarial to the current model. It is known that the discriminative perturbations are highly cor- related with the decision boundary. The high correlation between the perturbation and class label poses the training to the danger of label leaking (Kurakin et al. (017)).
Definition 3 Adversarial Interpolation. Given two images x., x', we define the adversarial interpo- lation of x (towards x') as the solution to the following optimization problem
T² adv interp(z, x') = arg min D(ā,x'),
BEB(z,e)
where x' is another instance that is different from x. In practice, x' can be set as another instance from the current batch, thus introducing negligible computational overhead. D is a distance mea- sure in a feature space. As minimization of D(ā,2') wrr.t.i has the effect of "interpolating" x to- wards x', we therefore term the proposed approach for generating adversarial attacks as adversarial interpolation. The adversarial interpolation procedure illustrated Figure It leverages the cur- rently learned model for feature extraction and distance computation. It then generates the adverse examples by back- proopppppgggiinggg the distance induced gradient (feature space difference) through the network back to pixels. It is more effective than random perturbation as it leverages structures be- tween data points. And it is potentially more flexible than discriminative perturbations as adversarial interpolation is used for generating perturbations which is untied from the classification loss. For a particular sample, when viewed collectively over time, adversarial interpolation essentially con- structs randomly connected graph (Hein al. 2007) and performs interpolation along different edge each time randomly. More on understanding of the proposed approach is in Section 3.3
3

---

Under review as a conference paper at ICLR 2020
Algorithm 1 Adversarial Interpolation Training
Input: training epochs K, learning rate a, budget e, Ey, attack iterations L, number of classes C for k = to K do
for random batch {x,, yi yi dataset do
Fe (2i,e)), e), Di yi, x In-i+1, D4 - (1-Bn-itt)) Un-i+1)
P(.:) ms s 2|; - Rz]l, S B c) n [0, 255) #pixel
adversarial interpolation:
for = 1 to L do
5. P(; -sign 2()),
generating adversarial images
11 Dr cy (U+ V/))
generating adversarial labels
end for adversarial training: 0 0-a 1 n
E. (oC(2,,80)
updating model parameters
end for end for
Output: model parameter A.
3.2 ADVERSARIAL INTERPOLATION TRAINING
We utilize adversarial interpolation for robust model training. Given a set of data, S = {x; ys) }, we first generate perturbed examples $ {ā ū. using adversarial interpolation, and then use the generated examples for model training (Goodfellow et al. 2015 Madry et al. (2018). Formally, perform robust model training by solving the following problem:
arg min B(z,))ss 0
(T, m7).
(2)
where the loss is the standard cross-entropy based loss as Lo(x, y) CE(g o f(i), V). with I the feature extraction network shared with the adversarial interpolation process and g denotes some additional layers following f (e.g., FC layer followed by softmax layer). Here we overload @ to rep- resent the union of parameters for g and f. The training samples (ū, ÿ) are generated via adversarial interpolation as follows:
arg min D(0, x²), ((,(c.c)
(3)
V" arg max Ge ((r
(ū.0'))
(4)
which perturbs image and y towards different (or reverse) directions within a permissible neigh- borhood. D and DDy are proper distance measures for image and label respectively. By perturbing the image and label in directions reverse to each other, the new image-label pair (ø, (i) will potentially induce higher classification loss when evaluated together and represent a strong adversarial sample to the model (Wang& & Zhang 2019). Here Ey denotes the maximum of the allowed perturbation for the label and it will be properly set to ensure the entry corresponding to the ground-trute label will remain to be the largest one after perturbation. All the samples in B(y, Cy) assumed to have non-negative elements and the entries of each sample sum to one. We reformulate Eqn.(4) using the following approximate instantiation
arg
min Be (B
11 - 1'112
(5)
1-d with y' the label x'. with C' the total number of classes. Note that labels are in vector form thus y and y/ are one-hot vectors. More interpretations on (qn.55 are provided in Section 3.3 Based on Eqn.(3) and Eqn.(5), we can derive the updates as follows. For two points (z, y) and xx (')), we generate the perturbed training sample as follows
where Euclidean distance is used and T -
@D(x, x') dr 0 5 Oy
5
(6)
7
y
(1 - -y)y + Eg.
(q can be interpreted as interpolating point x towards x' for the data, while interpolating the la- bel y away from y'. The training procedure is summarized in Algorithm 1 and the implementation is
4

---

Under review as a conference paper at ICLR 2020
in Section A.1 In practice, D(z, x') is implemented using deepnet-based feature extractor combined with a standard distance measure such as Euclidean (or other metrics such as Cosine distance),
Do(2, x') = (lfo(z) - fo (2))
We have used D a shorthand for Dy in the above but without explicitly specifying the dependency on 0. Minimizing this term makes the feature thus the perturbed input image incorporate some fea- tures from the other sample, while retaining visually similar to the original image, thus acting as an effective form of adversarial perturbation. More importantly, we perturb the image towards target instance, while perturb the label against this instance (to all other classes but this target one). When the target instance is of different class with the current instance, the label interpolation operation essentially reinforces that the perturbed image is not from the target class (c.f. Table 7). which helps to further break the correlation between ō and y'. beyond decreasing the correlation between å and y. When the target instance is of the same class with the current instance, the label interpolation operation is equivalent to conventional label smoothing (proof is deferred to Appendix AA.3)
3.3 INTERPRETING AND UNDERSTANDING OF THE PROPOSED APPROACH
From the perspective of robust and non-robust features, the key robust model training is ef- fectively prevent the classifier from learning useful but non-robust features (Ilyas et al.) 2019). In order achieve this, we need a way to convey this preference to the learning algorithm. Currently, one of the most straightforward way to communicate this is by manipulating the data. In fact. by manipulate the data in different ways, we can convey different preferences thus receiving different results from the machine learning algorithm (c.f. Table (7). As our major objective is to obtain a robust model, this can be achieved by breaking the correlation between the adversarial perturbation ō and label. Conventionally, it is achieved by constructing d ~ A, where A is the set of non-robust features of all the other classes. In this way, by training the model using the original label y and perturbation 8 from all other classes, the correlation between 6 and y is reduced. Conversely, if we construct 6 as from one class (y') and use y/ as the label for training, then we essentially imply that 8 represents useful feature for the prediction of y'. Therefore the model will learn to use these non- robust features for prediction, which has been exploited by Ilyas et al. (2019) recently. The proposed approach can be interpreted from two complementary perspectives. Firstly, we also de-emphasize d as a family by constructing it from different classes while retaining the original target specified by y, thus reducing the correlation between d and y. Secondly, we further de- de-emphasize each individual 6 by reinforeing that 5 is not useful features for y/, as represented by (1-y'), ("not y/"), thus further breaking the correlations between ō and y/- While we present one concrete instantiation in this work, the reasoning is general and can be implemented with other configurations as well.
3.4 CONNECTIONS WITH PREVIOUS WORKS
A seature-scattering approach for generating adversarial perturbations for training has been proposed recently (Zhang& & Wang, 2019). advocating the usage of unsupervised adversarial generation to avoid label leaking. Intuitively, it generates perturbations by moving away from a set of similar samples, discovered through an additional matching process. Differently, adversariallinterpolttii generates adversarial examples by moving towards a distinct sample, which does not require the expensive matching step and is complementary to feature-scattering (Zhang & Wang 2019).
There is also an interesting connection with Mixup (Zhang et al. 2018). Specifically, in the case of fo(r)= with Euclidean distance, adversarial interpolation for images (Eqn.(6) is equivalent to the operation applied to image in Mixup (Zhang al., 2018) (proof deferred to Section A.4). Therefore our approach can be interpreted as feature space induced adverse image generation instead of linearly mixing the raw pixels directly. Manifold-mixup (Verma et al. (019)] has been proposed recently by generalizing the mixup operation from input to latent features. While it also has an interpolation step in feature space, its motivation different from ours and it used differently. performs feature interpolation to generate the mixed feature, which is then passed to the subsequent layers till the cross-entropy loss. The cross-entropy loss-induced gradients backpropagate through the interpolation for model training. Manifold- plays the role of regularizer in order to ob- tain neural networks with smoother decision boundaries at multiple levels of representation, which helps but does not significantly improve robustness against multi-step PGD attacks (Verma et al.) 2019) simiar to the original Mixup (Zhang et al. 2018). The authors of Verma et al. (2019) have further integrated Manifold-mixs
hanifold-mixup
with adversarial training to mitigate this issue in another recent work (Lamb et al. (019).. Different from these works, the proposed method uses features interpo- lation for inducing inpul perturbation, which is done by backpropogating the gradient from feature
5

---

Under review as a conference paper at ICLR 2020
space distance Eqn. to the input (Figure 1). The perturbed inputs are then used as new inputs in place of the original ones for model training. Furthermore, empirically different from Mixup (Zhngg et al. 2018) and Manifold-min mixup (Verma et al. 2019), the proposed adversarial image interpolation scheme clearly helps to improve model robustness as shown in Table [] Another crucial difference with Mixup (Zhang at. 2018) and Manifold-mixup (Verma al. 2019 is that the label in the pro- posed approach is interpolated in a direction "reverse" to that of the image interpolation to further reinforce robustness. This difference can be explained by the fact that mixup focuses more on the on-manifold generalization while adversarial robustness further extends the focus to off maniiflolddd robustness (Stutz et al., 2019). [Anonymous] (2020) presents an interesting interpretation of Mixup as belonging to a class highly analogous to adversarial training. Without evaluation of adversarial robustness provided in the paper, (nnnyymus (020) is mainly on interpreting and understanding of Mixup, which is different from our goal of improving model robustness. [Christopher Beckham (219)) generalizes Manifold-mixu
(Verma et al. 2019) for image generation, where feature inter- polation is used to train model that is able to combine the attributes of multiple inputs in resyn- thesised image. The purpose of feature interpolation in (Christopher Beckham, 2019) is to introduce human sensible visual attributes in the generated image, and the image is generated by forwarding the mixed feature through a decoder in the auto-encoding framework. Differently, the purpose of feature interpolation in the proposed method is to introduce human imperceptible, non-robust fea- tures in the generated image, which is generated by the feature extractor (c.f. Figure I. A number of recent works that exploit interpolation from a different angle have also been presented by |Wang et al.] (2018b a Wang & Osher (019)), with the interpolation operates at the output layer and no adversarial label interpolation.
the feature difference through
The recent Bilateral method (Wang& & Zhang 2019) also jointly perturbs image and label, but uses the classification loss directly. Our approach is very different in that the classification loss is not used in generating perturbed image or label. Instead, it uses an adversarial interpolation scheme which not directly connected with the classification loss thus mitigating the problem of label leaking or data manifold tilting and shrinking (Tanay & Griffin 2016 Zhang & Wang, 2019).
We have used an unsupervised approach to generate adversarial perturbations. As another form of utilizing unsupervised learning. [Uesato et al. (2019) improves adversarial generalization by using additional unlabeled data, which is orthogonal to our work and could potentially be used together.
4 EXPERIMENTS
The PyTorch implementation of the proposed Adv-Interp approach is provided in Section A. We conduct extensive experiments to verify the effectiveness of the proposed approach. We compare the performance of the proposed method with a number of baseline methods:
Natural: the model trained with standard approach using natural images (Krizhevsky 2009);
Madry: the PGDD-aasd approach from Madry et al. 2018), which is one of the most effective and representative defense method;
Bilateral: a method that performs adversarial training with both image and label adversarial perturbations generated using the classification loss (Wang & Zhang 2019);
Feature-Scatteer a recent method which generates adversarial perturbations using an unsu- pervised feature-scattering scheme for attack generation (Zhang & Wang 2019).
Following (Madry et al., 2018). the full network (image to logits) is implemented as the Wide ResNet (WRN-28-10) T0) (Zagoruyko Komodakis 2016) and (-) (feature extractor) is implemented by excluding the last layer for the logits. D is implemented as the Cosine distance, which is free-from additional tuning parameter. For training, the initial learning rate a is 0.1 for CIFAR and 0.01 for SVHN. We set the number of epochs for Natural and Madry methods as 100 with transition epochs set as {60, 90} following Madry et al. (2018); Wang & Zhang 2019). The training scheduling of 200 epochs with the same transition epochs is used similar to Wang & Zhang 2019). Standard data augmentation is used during training, i.e., random crops with 4 pixels of padding and random horizontal flips (Krizhevsky| 2009). We use perturbation budget of e= (Madry et al., 2018), ty= 0.5, and I.=1 in training. The trained models are evaluated by measuring the accuracy performance against different adversarial attacks. For white-box attacks, we use:
FGSM: the Fast Gradient Sign Method (Goodfellow et al. 2015). which is a one-step gradient- based approach for generating adversarial attacks;
PGD: Projected Gradient Descent Method (Madry et al. 2018), which is a multi-step step gradient- based approach for generating adversarial attacks (PGDT denotes PGD attack with 7' iterations);

---

Under review as a conference paper at ICLR 2020
Performance under White-box Attack (c
Models
Clean
Worst
FGSM
PGD20
PGD100
CW220
CW100
Natural
95.6
37.0
0.0
0.0
0.0
0.0
0.0
Madry
86.7
54.9
45.0
44.5
45.7
45.3
44.5
Bilateral
91.2
70.7
57.5
53.2
56.2
53.8
53.8
Feature-Scatter
90.0
78.4
70.5
68.6
62.4
60.6
60.6
Adv-Interp
90.3
78.0
73.5
73.0
68.7
Table 1: CIFARIO results. Comparison of performance (classification accuracy) for Natural,
Madry Madry et al.) 2018), Bilateral (Wang & Zhang]
2019). Beature-Scatter (hang &
Wang 2019) and the proposed Adv-Interp method under different attacks.
Models
Performance under increasing
(5-4
with PGD
Performance under increasing (8)+00 with CW
8
10
12
14
16
18
20
8
10
12
14
16
18
20
Natural
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Madry
45.0
34.9
27.0
20.8
16.8
13.1
10.1
45.7
35.6
27.6
21.4
16.9
12.9
9.9
Adv-Inted Interp
73.5
72.9
71.9
71.3
70.2
68.6
67.4
69.7
68.2
66.6
65.2
63.8
62.4
60.5
Table 2: Performance against stronger white-box attacks with increasing attack budgets. The models are trained with the attack budget 8=,, and are evaluated against attacks with larger budgets.
CW: Carlini- Wagner loss based attack (Carlimi & Wagner, 2017), which also a multi-step gradient-based approach for generating adversarial attacks. It is implemented using the CW- loss (Carlini & Wagner 2017) within the PGD framework following Madry et al. (2018).
For black-box attacks, we use both gradient-based and gradient-free ones detailed in Section 4.4
4.1 EVALUATION AGAINST WHITE-BOX ATTACKS ON CIFAR 10
We conduct experiments on CIFARIO 1O (rrihhvssyy 2009) this section. CIFARIO is dataset with 10 classes, 5K training images per class and 10K Test images. It has been widely use in adversarial training literature (Madry et al. 2018 Wang & Zhang 2019).
Evaluation under Standard Setting. We report the accuracy on the original test images (Clean) as well as under standard adversarial settings (Madry et al.) 2018 Carlini & Wagner 2017 Carlini al. 2019) and the results are summarized in Table From Table |
we have several observations:
it is observed that model trained with natural images (Natural) fails drastically under different white-box attacks. Madry method improves the model robustness significantly over the Natural model and achieves about 45.0% accuracy under the standard PGD20 attack. The Bilateral approach outperforms Madry and achieves performance of 57.5% under PGD20 attack. The recent Seature-Scatter method further boost the performance to 70.5% under PGD20 attack. The proposed approach achieves 73.5% accuracy under the standard 20 steps PGD attack, outper- forming all the compared methods;
it is also noted that while the Feature-Scatter method achieves competitive performance noticeable drop when the strength of the attack
under PGD20 attack, its performance increased (e.g. PGD20-PPDDI00, The proposed Adv-Interp approach, on the other hand, maintains its performance when the strength of the attack is increased (PGD20-PGDIIO) and outperforms Feature-Scatter with even large gap under PGD100 attack;
is interesting to observe that under the CW metric, the performances of Madry and Bilateral similar their respective performances under PGD attack. For Feature-Scatter there is a large drop for the performance under CW attack compared with that under PGD attack. The proposed Adv-Interp approach, on the other hand, has smaller performance gap between PGD and CW metric and outperforms Feature-Scatter and all other methods with a large margin; in terms of the worst case performance among all evaluation criteria, the proposed Adv-Interp method improves over Madry by 24.2%, Bi Bilateral by 14.9% and Peature-Scatterr by 8. %.
Performance under increasing T (op 1000) with PGD
Performance under increasing T (20 1000) with CW
Models
20
100
200
300
500
1000
50
100
300
500
1000
Natural
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Madry
45.0
44.6
44.5
44.5
44.5
44.5
44.4
45.7
45.7
45.4
45.3
45.3
45.3
45.2
Adv-Interp
73.3
73.0
73.0
73.0
729
72.8
7288
69.0
68.7
68.3
68.3
68.3
Table Performance against stronger white-box attacks with increasing attack iterations. The models are trained with the attack budget e-8.
7

---

Under review as a conference paper at ICLR 2020
Models
Clean
FRSM PaIns PODSTR CWVA CWIN
Performance under White-box Attack
Wonst
Models
Clean
FC5M
Performance under White-ho Atack
FOiDD PrISTRO
CWN
CWInTT
Wonst
Natural
78.6
V.7
030
0.8
0.D
030
000
Natural
96.6
36.1
0.3
02.2
0.3
0.0
0.0
Madry
59.9
28.5
22.6
22.2
23.2
23,1
22.2
Madry
93.9
68.4
17.9
46.0
48.7
17.5
46.0
Bilateral
22.1
Bilateral
53.9
50.3
Feature-Seatter
73.9
61.0
472
462
34.6
306
30.6
Teatore-Soatteer
96.2
83.5
67.9
52.0
61.3
50.8
50.8
Adv-Interp
73.6
58.3
4.0
40.
324
312
312
Adv-Intexp
91.1
75.6
65.8
61.0
534
60.1
60.4
Table 4: More evaluation results. Performance comparisons on (a) CIFARI00 and (b) SVHN.
Models
Clean
FGSM PCD20
Parformance under White-box Attack
PGD100
CW20 CW100
Worst
Models
Clean
FCSM
Performunce under White- box Attack
Fob20 FCISTO0 [CW0 CW
Worst
image-org
95.7
62.4
11.3
2.3
16.7
8.7
2.3
label-org
91.6
57.3
38.1
36.5
39.3
38.0
36.5
image-mixup
95.4
79.9
32.4
19.4
31.8
23.1
19.4
label-nixup
93.3
65.5
19.4
13.4
18.8
14.1
13.4
image-als
71.2
60.0
56.3
54.1
54.1
label-snocs
91,1
77.0
68.4
66.2
62.9
61.3
61.3
Adv Interp
90.3
78.0
73.5
73.0
69.7
68.7
68.7
Adv-Intexp
50.3
78.0
73.3
730
69.7
68.7
68.7
Table 5: Ablatiiio studies on different perturbation schemes for (a) image and (b) label.
Evaluation against Stronger White-box Attacks. All the robust models are trained under the stan- dard setting with maximum attack budget e 8. While it is standard to restrict the maximum budget at test-time to be c 8 as well, it is also illuminating to conduct evaluations beyond it for comprehensive understanding of the model robustness. We therefore evaluate model robustness against PGD20 and CW20 attacks with increasing attack budgets and the results are summarized in Table 2 Unsurprisingly, the performance of Natural model remains at O. Madry improves the model robustness significantly over Natural across a wide range of attack budgets beyond 8, demonstrating the effectiveness of the standard adversarial training on improving model robustness. The proposed Adv-Interp approach further improves the performance over Madry by a large mar- gin for all the evaluated attack budgets. It is interesting to note that Adv-Interp maintains a high level of robustness even beyond the regime it is trained on. Note that if the attack budget is further increased to e= 255, the performance of all methods will be eventually approaching O.
We further conduct another set of experiments towards evaluation against stronger attacks, by in- creasing the number of attack iterations with a fixed attack budget of 8. The results are summarized in Table 3 As can be observed from Table 3] Madry method performs stable when the number of attack iterations is increased. The proposed Adv-Interp method can also maintain a fairly stable performance across number of attack iterations is increased. It is interesting to note that the perfor- mance of Adv-Interp under PGD1000 still outperforms the performance of all other approaches under PGD20 by large margin (c.f. [.) We will use PGD/CW attackers with c-8 and attack step 20 and 100 in the sequel as the standard setting for evaluating model robustness.
4.2
WHITE-BOR BOX ATTACK EVALUATION ON MORE DATASETS
CIFARI00. We also experiment with CIFARI00 dataset, with 100 classes, 50K training and 10K test images (Krihhessyy| 2009). This data set is more challenging as the number of training images per class is much smaller (Zhang & Wang 2019). We set training epoch 300 and c, 0.99. As shown by the results in Table 4 a). the proposed approach outperforms Madry and Bilateral by a large margin under both PGD and CW attacks. Furthermore, it achieves performance on-par with Feature-Scater under the strongest attack (CW100) on this challenging dataset.
SVHN. We further report results on the SVHN dataset (Netzer et al., 2011), which is 10-way house number classification dataset, with 73257 training images and 26032 test images. The additional training images are not used in experiment. The results are summarized in Tbble Experimental results show that the proposed method outperforms other baseline methods with a clear margin under both PGD and CW attacks. Notably, it outperforms worst case metric, further demonstrating the effectiveness of the proposed approach.
Saatter by about 10% under the
4.3 ABLATION STUDIES: THE IMPACTS OF ADVERSARIAL INTERPOLATION
We conduct a number of ablation studies to investigate the impacts of the components in the ad- versarial interpolation scheme. We replace a single component while keeping all others the same. Impacts of Adversarial Image Interpolation. We investigate the impacts of image adversar- ial interpolation by comparing with the following: i) image-org: use the original image for training; ii) image-mixup: use the mixup operation for generating perturbed images (Eqn. (I)): tti) image-cls: use classification loss for generating perturbed image; The results are presented in Table 5 a). It is observed that there is significant performance drop when removing image adver- sarial interpolation (image-org) image-mi mi xup performs better than image-org with improved robustness. image-cls is more effective due to the usage of classification loss in generating adver-
8

---

Under review as a conference paper at ICLR 2020
Attack Grudient
Performance under Black-box Attack
Gradient-Free Attack Methods
Performance under Black-box Attack
Generation Models
PGD20
PGD100
CW20
CWI00
Natural
89.3
89.1
89.4
89.2
SPSA
89.2
Adv-Interp'
82.9
82.5
81.8
82.0
Local-Bearch
90.1
Table 6: Black-box attack evaluations. Performance of the model trained with the Adv Interp method under (a) maddiett-bseed and (b) gradient-free black-box attacks.
sarial images. The proposed method, which uses adversarial interpolation for generating perturbed images, achieves the best performance, implying the importance of adversarial image interpolation. Impacts of Adversarial Label Interpolation. Similarly, we replace the adversarial label inter- polation part with different approaches: i) label-org: the original label is used for training; ti) label-mixup: a mixup operation is applied to the label (Eqn.(20p) ) iti) label-smooth:: la- bel smoothing is applied to the label (Eqn.416): The results summarized in Table T(b)e It is observed that, without adversarial interpolation for label (label-org), the performance drops sig- nificantly. label-mixup improves performance under weak attacks (FGSM), but the performance under stronger attacks is decreased, possibility due to its inability in de-emphasizing the correla- tions between the perturbation and y'. label-smooth achieves a much better performance than label-org and label-mixup The Adv-Interp method further boosts the performance over sabel-smooth, demonstrating the importance and effectiveness of adversarial label interpolation.
4.4 PERFORMANCE UNDER BLACK-BOX ATTACKS
To further verify that the improvement due to inherent improvement in model robustness instead of degenerate solution, we evaluate the robustness of the model trained with the proposed approach w.r.bblack-box attacks following previous evaluation setup (Tramèr et al. 2018 Uesato et al. 2018).
Gradient-basee Black-box Attack. We first use gradient-based approach for generating the attacks, but using models that are different from the one to be evaluated. Specifically, we use the Natural model and Adv-Interp' which is another model trained with the proposed method in different training session. The results are summarized in Table 6 (a). It is observed that the model trained with the proposed approach is robust against different types of gradient-based black-box attacks.
Gradient-free Black-box Attack. We further conducted evaluation using two gradient-free black- box attacks. SPSA is technique developed for high-dimensional optimization problem (Spall 1992) and is used in Uesato et al. (2018) for gradient-free attack-based evaluations. Local-Search is a local search-based black-booo black-box attack method (Narodytska & Kasiviswanathan 2017). The evaluation results of the proposed model against gradient-free attacks are summarized in Table 6 (b). All these results together with the white-box attack results in Table verify that the improvements are indeed due to the improved model robustness, not because of gradient-masking caused by degenerate solution (Tramer et 2018
5 CONCLUSION AND FUTURE WORK
adversarial interpolation training method is presented in this paper. This approach intuitive to understand and simple to implement, yet achieving state-of-the-art performance. Extensive exper- iments on several standard datasets have been done to evaluate and understand the performance of the proposed method. Empirical results demonstrate that the proposed approach compares favorably with existing methods in the literature. There are some limitations of the proposed approach due to the usage of unsupervised surrogate function for generating image perturbations. While this offers the potential of mitigating some issues of the discriminative perturnation, the perturbations generated this way are not necessarily the most effective ones due potential mis-alignment with the downstream task. Also, as there is no explicit force for further encouraging the clustering of features and the increasing of the separation margins between them, the feature points are typically more scattered (c.f. Figure D. offering more opportunity to feature space attacks. Nevertheless, the fact that such a simple approach can achieve encouraging performance suggests that this is a valuable direction to explore further and the observation that it behaves differently with existing methods against different attacks suggests the potential to combine their complementary strength for further improvements (Pang et al.) 2019). The proposed method is generally applicable can potentially be extended to applications beyond image classification. Furthermore, as the proposed adversarial interpolation based attack generation does not require label information, it would be in- teresting to extend this approach to other types of learning tasks such as unsupervised learning and reinforcement learning. We leave the exploration of these interesting directions future work.

---

Under review as a conference paper at ICLR 2020
REFERENCES
Anonymous.
Mixup as directional adversarial training. In Submitted to International Confer- ence on Learning Representations, 2020. URL https://openreview net. forum?iae
SkRjKR4YB under review.
Martin Arjovsky, Soumith Chintala, and Leon Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning, 2017.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of curity: Circumventing defenses to adversarial examples. In International Conference on Machine learning, 2018.
Ben Athiwaratkun, Mare Finzi, Pavel Izmailov, and Andrew Gordon Wilson. There are many con- sistent explanations of unlabeled data: Why you should average. In International Conference Learning Representations, 2019.
Battista Biggio and Fabio Roli. Wild patterns: Ten years after the rise of adversarial machine learning. CoRR, abs/1712.03144 2017.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Smdic, Pavel Laskov, Gior- gio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. Euro- pean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2013.
Chris M. Bishop. Training with noise is equivalent to tikhonov regularization. Neural Comput.." 7 (1):108-116, January 1995.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision- based adversarial attacks: Reliable attacks against black-box machine learning models. In International Conference Learning Representations, 2018
Tom B. Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. Adversarial patch. CoRR, abs/1712.09665, 2017.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, 2017.
Nicholas Carlini and David A. Wagner. Audio adversarial examples: Targeted attacks on speech-to- text. In IEEE Symposium on Security and Privacy Workshops, 2018.
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian J. Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. CoRR, abs/1902.06705,. 2019.
Zachary B. Charles, Harrison Rosenberg, and Dimitris S. Papailiopoulos. A geometric perspective on the transferability of adversarial directions. In International Conference on Artificial Intelli- gence and Statistics, 2019.
Vikas Verma Alex Lamb Farnoosh Ghadiri R Devon Hjelm Yoshua Bengio Christopher Pal Christo- pher Beckham, Sina Honari. On adversarial mixup resynthesis. In Advances in Neural Informa- tion Processing Systems, 2019.
Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred Hamprecht. Essentially no barriers in neural network energy landscape. International Conference on Machine Learning. 2018.
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Alek- sander Madry. Learning perceptually-alinn representations via adversarial robustness. CoRR, abs/1906.00945, 2019.
Kevin Eykholt, Ivan Evtimov, Earlence Fermandes, Bo Li. Amir Rahmati, Florian Tramèr, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Physical adversarial examples for object detectors. CoRR, abs/1 807.8070000 007.97769 2018.
10

---

Under review as a conference paper at ICLR 2020
Nic Ford, Justin Gilmer, Nicholas Carlini, and Ekin Dogus Cubuk. Adversarial examples natural consequence of test error in noise. CoRR, abs/1901.10513, 2019.
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew Wilson. Loss surfaces, mode connectivity, and fast ensembling of DNNs. In Advances in Neural Information Processing Systems. 2018.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
M. Hein, J-Y. Audibert, and U. von Luxburg. Graph laplacians and their convergence random neighborhood graphs. Journal of Machine Learning Research, 8:1355-1370, June 2007.
Geoffrey Hinton, Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 29,))82-97, 2012.
Andrew Hyyss, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In Advances in Neural Information Processing Systems, 2019.
Joern-Henrik Jacobsen, Jens Behrmann, Richard Zemel, and Matthias Bethge. Excessive invariance causes adversarial vulnerability. In International Conference on Learning Representations, 2019.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in Neural Information Processing Systems, 2012.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In International Conference on Learning Representations, 2017.
Alex Lamb, Vikas Verma, Juho Kannala, and Yoshua Bengio. Interpolated adversarial training: Achieving robust neural networks without sacrificing too much accuracy. CoRR, abbs/190606677 2019.
Fangzhou Liao, Ming Liang, Yinpeng Dong, and Tianyu Pang. Defense against adversarial attacks using high-level representation guided denoiser. In IEEE Conference Computer Vision and Pattern Recognition, 2018.
Yen-Chen Lin, Zhang-Wat Hong, Yuan-Hong Liao, Meng-Li Shih, Ming-Yu Yu Liu, and Min Sun. Tacties of adversarial attack deep reinforcement learning agents. In International Joint Con- ferences on Artificial Intelligence, 2017.
Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsich. Towards robust neural networks via random self-ensemble. In European Conference Computer Vision, 2018.
Divyam Madaan and Sung Ju Hwang. Adversarial Neural Pruning. CoRR,
2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Dongyu Meng and Hao Chen. MagNet: two-pronged defense against adversarial examples. In ACM SIGSAC Conference on Computer and Communications Security, 2017.
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On detecting adversarial perturbations. In International Conference on Learning Representations, 2017.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Belle- mare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wier- stra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement learning. Nature, 518((540))552- 533,, 02 2015.
11

---

Under review as a conference paper at ICLR 2020
Seyed-Mohsen Moosavi-Dezfooli; Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Seyed-Mohsen
Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Computer Vision and Pattern Recognition, 2017.
Nina Narodytska and Shiva Prasad Kasiviswanathan. Simple black-box adversarial perturbations for deep networks. CoRR, abs/1612.06299, 2017.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop Deep Learning and Unsupervised Feature Learning, 2011.
Alexander G. Ororbia, II, Daniel Kifer, and C. Lee Giles. Unifying adversarial training algorithms with data gradient regularization. Neural Comput., 2964)8877-887, 2017.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning, 2019.
Aaditya Prakash, Nick Moran, Solomon Garber, Antonella Dilillo, and James Storer. Deflecting adversarial attacks with pixel deflection. In IEEE Conference Computer Vision and Pattern Recognition, 2018.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-GAN, Protecting classifiers against adversarial attacks using generative models. International Conference on Learning Representations, 2018.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models understand and defend against adversarial examples. CoRR, abs/1710.10766. 2017.
James C. Spall. Multivariate stochastic approximation using simultaneous perturbation gradient approximation. JEEE Transactions on Automatic Control, 17:332-341, 1992.
David Stutz, Matthias Hein, and Bernt Schiele. Disentangling adversarial robustness and general- ization. In IEEE Conference on Computer Vision and Pattern Recognition, 2019.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. CoRR, abs/1770,08864, 2017.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus, Intriguing properties of neural networks. In International Conference on Learning Representations, 2014.
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Thomas Tanay and Lewis D. Griffin. A boundary tilting persepective on the phenomenon of adver- sarial examples. CoRR, fabs/1608.076m 2016.
Florian Tramèr, Nicolas Papernot, Ian J. Goodfellow, Dan Boneh, and Patrick D. McDaniel. The space of transferable adversarial examples. CoRR, abs/0304 2017.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Dan Boneh, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. In International Conference on Learning Representa- tions, 2018
Jonathan Uesato. Brendan O' O'Donoghue, Pushmeet Kohli, and Aäron van den Oord. Adversarial risk and the dangers of evaluating against weak attacks. In International Conference on Machine Learning, 2018
12

---

Under review as a conference paper at ICLR 2020
Jonathan Uesato, Jean-Baptiste Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In Advances in Neural Information Processing Systems, 2019.
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez- Paz, and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In International Conference on Machine Learning, 2019.
Bao Wang and Stanley Osher. Graph interpolating activation improves both natural and robust accuracies in data-efficient deep learning. CoRR, abs/1907.06800, 2019.
Bao Wang, Alex Tong Lin, Zuoqiang Shi, Wei Zhu, Penghang Yin, Andrea L. Bertozzi, and Stan- ley J. Osher. Adversarial defense via data dependent activation function and total variation mini- mization. CoRR, abs/1 000.0180.,,,16, 2018a.
Bao Wang, Xiyang Luo, Zhen Li, Wei Zhu, Zuoqiang Shi, and Stanley J. Osher. Deep neural nets with interpolating function as output activation. In Advances in Neural Information Processing Systems, 2018b.
Bao Wang, Binjie Yuan, Zuoqiang Shi, and Stanley J. Osher. Resnets ensemble via the feynman- kac formalism to improve natural and robust accuracies. In Advances in Neural Information Processing Systems, 2019.
Jianyu Wang and Haichao Zhang. Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks. In International Conference Computer Vision, 2019.
Chaowei Xiao, Bo Li, Jun yan Zhu, Warren He, Mingyan Liu, and Dawn Song. Generating ad- versarial examples with adversarial networks. In International Joint Conference on Artificial Intelligence, 2018.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. In International Conference on Learning Representations, 2018.
Ziang Yan, Yiwen Guo, and Changshui Zhang. Deep defense: Training DNNs with improved adversarial robustness. Advances in Neural Information Processing Systems, 2018.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference, 2016.
Haichao Zhang and Jianyu Wang. Defense against adversarial attacks using feature scattering beattering-based adversarial training. CoRR, 0tss/190..10760 2019.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. International Conference on Learning Representations, 2018.
A APPENDIX
A.1 ADVERSARIAL INTERPOLATION TRAINING CODE
Here we provide the code snippet for one epoch of adversarial interpolation training in PyTorch:
for x, in data_loader:
2
x_tilde, y_tilde adv interp (x, y, net, num_classes) logits_tilde net (x_tilde, mode="logits") loss = soft_ce_losa (logits_tilde, y_tilde) pptimizer.zero_grze
5
6
loss backwar)c optimizer.
9
def adv adv_interp (x, y, net, num_classes, epsilon=8, ppsilon_y=0.s v_min=0, v_max=255)
10
11
# image batch with shape [batch_size, c, h,
12
y: one-hot label batch with shape [batch_size,
num_classes]
13

---

Under review as a conference paper at ICLR 2020
13
inv index torch. arange (x. (x.size -1, -1) long ()
14
x_prime [inv_index, ;, ;, :].eetcch
15
y-prime = [inv_index, :]
16
x_nni x. x.detach () wwworcch.orossiiii like (x) uniiforn epsilen)
17
reqnit..
18
_eet copy. sepy.deepopyynoe (net)
19
_net.eval() ()
20
loss_adv = cos_dist Lnet (x init, moce="Leature"),
21
_ee (x prime, moge="Leature").
22
_et.. (r).zero_grad()
23
loss_adv. bac.waad
24
X_tilde x_init.data
epsilon
25
X_tilde - torch (torch.
(torchh.taa max (x_tilée,, x epsilon), + epsilen) x_tilde = torch. cormh. (x ix_tid v_min, v_max)
26
27
y_bar_primg
y_prime) / (num_classes
28
y_tilde (1 epsilon_y) s_tilde.detach/
epsilon_y y_bar_primn
29
return x
A.2
DERIVATION OF THE EQUIVALENT Loss FOR ADVERSARIAL LABEL INTERPOLATION
Our label interpolation procedure is implemented as
arg
min (u
1 7'12.
(9)
which leads to the following update equation for label
3 F
Ey
(1 cy)y Goy
(10)
Oy
where the label target j is constructed as of show that this can be derived from a complementary perspective as follows.
J with C the total number of classes, Here
Given a training sample (ar, v), the original way for model training is as follows:
min CE(a, v) min Σ (cho(x))
(11)
where : = h(()) = go f(x).
By adding perturbation 8 which fools the predictor towards label y', we have ā x + 8, which semantically still belongs to the original class. Therefore, we can use the following formulation to dis-emphasize
min CF(5, v) = min Σ (iha(ā);-
(12)
Secondly, while the perturbed sample ã fools the predictor towards label y'. ã is not valid sample of class y', which is represented as
min CE s/) min Σ (.)
(13)
By combining Eqn. (2) with Eqn. (4). we have
min (1 cy) CE( y) c, CE(a.7)) min CE(2, cy)y cgg)).
(14)
which leads to the same update equation for label as in Eqn. 10
A.3 CONNECTION OF ADVERSARIAL LABEL INTERPOLATION WITH LABEL SMOOTHING
Here we provide the proof that when the target instance is the same class with the current instance, iee. y y', the label interpolation operation is equivalent conventional label smoothing.
14

---

Under review as a conference paper at ICLR 2020
Recall that for adversarial interpolation of label, we have
リ= (1 -y)y + EyD',
(15)
where y' = -- C' is the number of classes. In the case that v' = y, we have
ÿ = (1 - Cy) cy)y + C
= (1 - ) + C- 0-1 (1-1111
Ey
(16)
= (1 C-1 vyyy + Ey
C-1
which is equivalent to label-smoothing with the strength of
v (Szegedy et al. 2016).
A.4 CONNECTION OF ADVERSARIAL IMAGE INTERPOLATION WITH MIXUP
There is a connection between the proposed method and mixup (Zhang et al. 2018) as mentioned in the main paper. Specifically, in the case of fo =x with Euclidean distance, the adversarial interpolation for images (Eqn. (6) is equivalent to mixup (Zhang et al. 2018). Concretely, for
D(x, x²) = || x 11
(17)
we have its gradient as
0D x'.
(18)
Plugging it into adversarial interpolation update Eqn. (6. we have
T = x c(x x') (1 e) e)xr + ez'.
(19)
This shows that for image manipulation, when the proposed approach performed at pixel level we recover mixup (Zhang et al. 2018). In this case, the new image 7 is obtained as a linear combination of the data points x and x¹ in the original data space, which are pixels for images. This formally shows that the image manipulation in mixup can be interpreted as gradient step for minimizing the pixel value distance between two samples. As for labels, mixup (Zhang et al. 2018) shares a similar update form as Eqn.(7) but with y' y i.e.
ÿ (1 Ey)y + Eyy'
(20)
A.5 CATEGORIZATION OF DIFFERENT TRAINING METHODS
Here we provide a categorization of different training methods as in Table 7 By manipulating image and label in different ways,
convey different messages to the learning algorithm and obtain the corresponding results. The proposed approach can be intuitively understood as leveraging two complementary aspects for improving model robustness, i.e., by reducing the the correlations between å and as well as the correlations between and y'.
Training sample
Methods
Message
Result
image
label
standard training
no preference over different features
lcarning non-robust model with high probability
Krohessky et al. 2012
Riversaral
Geodiflloo al b01s
xr .
9
prefer robust feutures by reducing the correlation between 8-y
learning robust model
non-robust Iea, leaming
1/
preter non-robust features by leveruging the correlations between d-y/ prefer robust features by reducing
learning non-robust model
Ibys et 2019
this work
x x+8 8
(L-ty)y + (1(0))))
learning robust model
the correlations between d-y and 5-1/
Table 7: Categorization and analysis of different model training methods. x denotes the original image and y denotes the original label. 5 is the perturbation added to the image x, constructed by altering the prediction towards y/. < (C - 1) where C' is the total number of classes. Note that different approaches can use different methods for generating 6.
15

---

Under review as a conference paper at ICLR 2020
A.6 LABEL ADV-INTERPOLATIO PARAMETER
fy
Clean
FGSM
Performance under White-box Attack (e
PGD20
PGD1O
CW20
CWTIT
38.0
We investigate the impacts of the adversarial label in- terpolation parameter cy and the results are summa- rized in the table on the right. It is observed from the results that the proposed approach performs well for range of values of 'y.
0.1
90.9
73.3
57.3
52.9
57.2
54.2
0.3
90.6
78.2
73.5
72.6
68.7
68.0
0.4
90.8
77.8
73.7
73.0
67.7
66.6
0.5
90.3
78.0
73.5
73.0
69.7
68.7
0.6
90.5
74.9
61.6
58.4
63.5
61.5
A.7 INVESTIGATION OF THE ADVERSARIAL LOSSES
Following Madry et al. (2018), we investigate and visualize the variations the losses caused by the adversary (adversaria losses). We run PGD adversary with random starts for 10 times, against Natural model as well as the proposed Adv Interp model. The results are presented in Figure It can be observed that the loss achieved with PGD adversary against model increases in fairly consistent way and plateaus rapidly for 10 different runs with random starts, with relatively smaller variance and final loss value, compared with those of Natural model. These observations are consistent with properties of robust models observed in Madry et al. (018). further verifying the inherent robustness improvement of the proposed model.
Adversarial loss curves of Natural model
130
120
130
10
Adversarial loss curves Adv-Interp model
0.03
346
0..
Low Vogan
0.2
0.5
20
0.63
20
Figure 2: Adversarial loss curves of different models on different images under PGD attack with increasing iterations (1-100).. Each column corresponds the loss curves generated using the same input image on the Natural model (top) and the Adv-Interp model (bottom).
A.8 FEATURE DISTRIBUTION VISUALIZATION
We visualize the distribution of features extracted using Natural and Adv-Interp models with the t-SNE visualization technique and the results are shown in Figure 3 Specifically, we show the feature distributions for Clean image as well as perturbed images generated using PGD and observed that the Natural model changes drastically in the presence of attacks. The proposed Adv Interp model, on the other hand, can better preserve the feature distribution against adversarial attacks.
adversarial interpolation.
A.9 ADVERSARIAL INTERPOLATION IMAGE VISUALIZATION
We provide the visualization of original natural images and the perturbed images generated by ad- versarial interpolation using the proposed model in Figure A.9 Here conduct the visualization with different number attack steps I. 1,3, ... Note that model training, we have used I==1 1.
A.
MORE RESULTS AND COMPARISONS
We have discussed the connection of the proposed approach with a number of related recent works in Section 3.4 Here provide more comparison with them in Table & Specifically, we compared
16

---

Under review as a conference paper at ICLR 2020
Natural
Adv-Interp
Images
PGID Attack Images
Ady-Interp Images
Figure 3: Feature distribution visualization of Natural and Adv-Interp models on (top) clean images, (middle) PGD attack perturbed images, and (bottom) adversarial interpolation perturbed images. observed that the distribution features extracted using the Natural model changes drastically in the presence of attacks. On the other hand, the Adv-Interp model can better preserve the feature distribution against attacks.
with Mixup (Zhang et al. 2018). Mani Manifold-mixup Veema et al. 2019) as well as their adversarial extensions Mup and IAT-Mani IAT-Manifold-mian xup (Lamb et 22119) and UAT (Uesato al. 2019), which is an adversarial training method augmented with an unsupervised loss computed on additional unlabeled data. The results in Table for UAT obtained using 20k additional images
class for training (Uesato et al. 2019).
Models Mix-Mixup
Attack Methods
Mixup
ManiFold-mixup
IAP-Mani fold-mixup
DA?
Adv Intern
Natural
96.8
96.9
93.6
93.5
85.9
90.3
FGSM
67.4
61.6
66.2
64.8
78.0
PGD20
1.7
50.1
62.2
73.5
Table
More results and comparisons with several recent methods on CIFARIO.
A.11 EVALUATIONS AGAINST ADAPTIVE ATTACKS
Here we provide more evaluations against adaptive attacks following reviewer's suggestions as well as Carlini et al. (201)). We include the following aspects of "adaptiveness" in our evaluations: I) we assume the full knowledge of the model as well as other necessary information and perfrom evaluations with multiple runs; 2) include feature- based attack, which has the full knowledge of the model as well as the defense method. Specifically, we use the following attacks for evaluation:
PGD-min:
the evaluation agasint PGD20 attack 10 times with different random starts for the same test sample and record the worst performance for each sample; the average performance the whole test set is reported as the final result.
17

---

Under review as a conference paper at ICLR 2020
Natural
L=1
L=3
L= 5
L=
L=
Figure 4: Visualization of original natural images and the perturbed images generated by ad-
versarial interpolation with different number of attack steps (L) for the perturbation budget of c=
we run the evaluation agasint CW20 attack 10 times with different random starts for the same test sample and record the worst performance for each sample;
CW-min:
Feature-min attack: we run feature attack 10 times by using different random starts, different target images and 20 attack steps, and record the worst performance for each sample.
We have also provided the results on original clean images for reference. We conducted evaluations Natural, Madry, and Adv-Interp. Adv-Interp* denotes the model with exactly the same trained parameters as Adv-Interp, but only add a random perturbation to the input | The results shown in Table g We have the following observations from the results. The Adv-Interp perorms well agasint PGD-min and CW-min attacks with multiple random starts, suggesting its robustness against these attacks. It performs less favorably against Madry Feature attack, but the gap is further bridged by Adv-Interp*. More specifically, Adv-Interp* still achieves higher performance against PGD-min and CW-min attacks, and performs comparably Madry on Feature- min attack. Essentially, the random perturbation does not impact the effectiveness of PGD-min and CW-min attacks (as expected), while shows more impacts on Feature-min attaek. These results imply that Feature attack is less generic type of attack and generally less robust as it could be easily affected by (typically non- non-imponn con-impactful) random perturbations. The results also suggest that proposed model could potentially benefit further by improving the surrogate function, e.g., by incorporating explicit terms for enlarging the separation margin between classes, which is an interesting direction that is worthwhile to be explored further in the future.
Models
Attack Methods
Natural
Madry
Adv-Interp Adv-Interps
Clean
95.6
86.7
90.3
90.1
PGD- min
0.0
44.5
71.5
70.0
CW-min
0.0
45.2
67.2
65.4
Feature-min
36.4
43.4
Table More results against adaptive attacks on CIFAR1O.
'x + clamp(æ + e rand_sign (z)) 0, 255), where rand signn.ike((2 returns a tensor with the same shape and the elements are randomly set to -1 or 1. clamp ensures the input is within the specified range. denotes the perturbation budget which is the used in training.
18


Reviews:
[{"review_id": "SylNE5kstr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This is an interesting work proposing a new robust training method using the adversarial example generated from adversarial interpolation. The experimental results seem surprisingly promising. The ablation studies show that both image and label interpolating help the robustness improvement. \n\nI think it is important to provide a running time comparison between the proposed method and SOTA robust training method such as Madry's. Since the feature extractor is implemented by excluding the last layer for the logits, the backprop goes through almost the entire model. It seems that the proposed interpolating method has a similar amount of computation as PGD, so the training should take similar time as Madry's if it can converge quickly. \n\nAlso, there are too many works on robustness defense that have been proven ineffective (consider the works by Carlini). Since this is a new way of robust training and there is no certified guarantee, I would be very conservative and suggest the authors refer the checklist in [1] to evaluate the effectiveness of the defense more thoroughly to convince the readers that it really works. Especially, a robustness evaluation under adaptive attacks is necessary. In other words, if the attacker knows the strategy used by the defender, it may be possible to break the model. PGD and CW are non-adaptive since no defender information is provided. \n\n[1] Carlini, Nicholas, et al. \"On evaluating adversarial robustness.\" arXiv preprint arXiv:1902.06705 (2019).", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "S1esl_BpcH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes adversarial interpolation training, which perturbs the images and labels simultaneously. The perturbed image $\\tilde x$ is around $x$ and interpolated towards another image $x'$ while the corresponding $\\tilde y = (1-\\epsilon_y)y +\\epsilon_y\\frac{1-y'}{C-1}$ is near $y$ but away from $y'$. The distance of interpolating images is L2 distance in the feature space and that for labels is L2 distance in the label space. The paper provides an interpretation of the proposed approach from the perspective of robust and non-robust features. Thorough experiments on different types of attacks and different datasets are performed. Although the results are impressive, I still have some concerns on the method itself:\n\n1. The method seems like a combination of manifold mixup [1] and adversarial training. The interpolation in the feature space is not a new idea and has been explored in Manifold Mixup [1]. The method resembles manifold mixup if we focus on $x$ because $\\tilde x$ and $\\tilde y$ both retain the original image and target $(x, y)$. The \"adversarial\" interpolation part is from $(x', y')$ in the sense that $\\tilde y$ is away from $y'$.\n\n2. The paper lacks a theoretical explanation, which makes it less convincing how it works so well.\n\n3. I noticed several papers with similar ideas, e.g. [2,3,4]. Could you please discuss the connections with them? I also suggest adding related work on semi-supervised learning in the paper (see [4] for examples). It would be better to compare with Manifold Mixup [1], UAT [4] in the experiments.\n\n\nMinor:\n\nPage 5, \" further break the correlation between $\\delta$ and $y'$\", what is $\\delta$ here? I did not find the definition above the sentence. The notation is directly used without any explanation in advance.\n\n\nReferences\n[1] Manifold Mixup: Better Representations by Interpolating Hidden States, ICML 2019\n[2] MixUp as Directional Adversarial Training\n[3] On Adversarial Mixup Resynthesis, NeurIPS 2019\n[4] Are Labels Required for Improving Adversarial Robustness?, NeurIPS 2019\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "HJxv7evfYr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Contribution: This paper proposed an adversarial interpolation approach for generating adversarial samples and then training robust deep nets by leveraging the generated adversarial samples. However, I have the following concerns:\n\n1. In adversarial interpolation training, why \\tilde{y}_i' is set to 1/(C-1)(1-y_{n-i+1})? \n\n2. This work lack of interpretation of why the proposed method is more effective than PGD adversarial training.\n\n3. How about training deep nets with replicas of the training data but replace the true labels with random labels? I want to see such a result.\n\n4. Can the authors provide the black-box attack results also? I want to see the performance of PGD adversarially trained deep nets on the adversarial images crafted by attacking Adv-Interp trained deep nets, and vice versa.\n\n5. Can the authors provide the visualization of a few adversarial interpolated images?\n\n6. The authors should compare with the existing efforts that using interpolation to improve adversarial robustness. Below are some Related works on using interpolation in deep nets to improve their robustness\n\n1). Bao Wang, Xiyang Luo, Zhen Li, Wei Zhu, Zuoqiang Shi, Stanley J. Osher. Deep Neural Nets with Interpolating Function as Output Activation, NeurIPS, 2018\n\n2). Bao Wang, Alex T. Lin, Zuoqiang Shi, Wei Zhu, Penghang Yin, Andrea L. Bertozzi, Stanley J. Osher. Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization, arXiv:1809.08516, 2018\n\n3). B. Wang, S. Osher. Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning, arXiv:1907.06800, 2019\n\n7. Moreover, the following paper provides a theoretical interpretation of adversarial vulnerability of deep nets, and proposed a nice ensemble of neural SDEs to improve deep nets' robustness.\n\n1). Bao Wang, Binjie Yuan, Zuoqiang Shi, Stanley J. Osher. ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies, arXiv:1811.10745, NeurIPS, 2019\n\n\nPlease address the previously mentioned concerns in rebuttal, and this paper can be acceptable if all my concerns are addressed.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers agree that the proposed method is interesting and achieves impressive results. Clarifications were needed in terms of motivating and situating the work. Thee rebuttal helped, but unfortunately not enough to push the paper above the threshold. We encourage the authors to further improve the presentation of their method and take into accounts the comments in future revisions.
Decision: Reject