Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks
Authors: Tianyu Pang*
Kun Xu*
Jun Zhu
Venue:  (2020)
Keywords: Trustworthy Machine Learning
Adversarial Robustness
Inference Principle
Mixup

Abstract: It has been widely recognized that adversarial examples can be easily crafted to fool deep networks, which mainly root from the locally non-linear behavior nearby input examples. Applying mixup in training provides an effective mechanism to improve generalization performance and model robustness against adversarial perturbations, which introduces the globally linear behavior in-between training examples. However, in previous work, the mixup-trained models only passively defend adversarial attacks in inference by directly classifying the inputs, where the induced global linearity is not well exploited. Namely, since the locality of the adversarial perturbations, it would be more efficient to actively break the locality via the globality of the model predictions. Inspired by simple geometric intuition, we develop an inference principle, named mixup inference (MI), for mixup-trained models. MI mixups the input with other random clean samples, which can shrink and transfer the equivalent perturbation if the input is adversarial. Our experiments on CIFAR-10 and CIFAR-100 demonstrate that MI can further improve the adversarial robustness for the models trained by mixup and its variants.

Published as a conference paper at ICLR 2020
MIXUP INFERENCE: BETTER EXPLOITING MIXUP TO DEFEND ADVERSARIAL ATTACKS
Tianyu Pang", Kun Xu', , Jun Zhu Dept. of Comp. Sci. & Tech., BNRist Center, Institute for AI, Tsinghua University; RealAI {sty17, xu-k16 @mai is. tsinghua. edu. en, deszj@tsinghus, edu,
ABSTRACT
It has been widely recognized that adversarial examples can be easily crafted to fool deep networks, which mainly root from the locally unreasonable behavior nearby input examples. Applying mixup in training provides an effective mechanism to improve generalization performance and model robustness against adversarial perturbations, which introduces the globally linear behavior in- in-between training examples. However, in previous work, the mixup-trained models only passively defend adversarial attacks in inference by directly classifying the inputs, where the induced global linearity not well exploited. Namely, since the locality of the adversarial perturbations, it would be more efficient to actively break the locality via the globality of the model predictions. Inspired by simple geometric intuition, we develop an inference principle, named mixup inference (MI), for mixup-trained models. MI mixups the input with other random clean samples, which can shrink and transfer the equivalent perturbation if the input is adversarial. Our experiments on CIFAR-10 10 and CIFAR- 1-00 demonstrate that MI can further improve the adversarial robustness for the models trained by mixup and its variants.
INTRODUCTION
Deep neural networks (DNNs) have achieved state-of-the-art performance various tasks (Good- fellow et al., 2016). However, counter-intuitive adversarial examples generally exist in different domains, including computer vision (Szegedy al., 2014), natural language processing (Jin et al., 2019), reinforcement learning (Huang et al., 2017), speech (Carlini & Wagner, 2018) and graph data (Dai et al., 2018). As DNNs are being widely deployed, it is imperative to improve model robustness and defend adversarial attacks, especially in cafety-critical cases. Previous work shows that adversarial examples mainly root from the locally unstable behavior of classifiers on the data manifolds (Goodfellow et al., 2015; Fawzi et al., 2016; 2018; 8; Pang et al., 201 80)8b), where small adversarial perturbation in the input space can lead to an unreasonable shift in the feature space.
On the one hand, many previous methods try to solve this problem in the inference phase, by introducing transformations on the input images. These attempts include performing local lincar transformation like adding Gaussian noise (Tabacof & Valle, 2016), where the processed inputs are kept nearby the original ones, such that the classifiers can maintain high performance on the clean inputs. However, as shown in Fig. (a)), the equivalent perturbation, i.e., the crafted adversarial perturbation, is still 8 and this strategy is easy to be adaptively evaded since the randomness of Tp w.r.t To is local (Athalye et al., 2018). Another category of these attempts is apply various non-linear transformations, e.g., different operations of image processing (Guo et al., 2018; Xic et al., 2018; Raff et al.. 2019). They are usually off off-the-shelf for different classifiers, and generally aim disturb the adversarial perturbations, as shown in Fig. (b). Yet these methods are not quite reliable since there is no illustration or guarantee on what extent they can work.
On the other hand, many efforts have been devoted to improving adversarial robustness in the training phase. For examples, the adversarial training (AT) methods (Madry et al., 2018: 8: Zhang et al., 2019; Shafahi et al., 2019) induce locally stable behavior via data augmentation on adversarial examples. However, AT methods are usually computationally expensive, and will often degenerate model
"qqual contribution. *Corresponding author.

---

Published as a conference paper at ICLR 2020
(a)
(b)
X
X
Xo
X
X
X
Mixup Inference
Local Linear Transformation
Non-Linear Transformation
(Global Linear Transformation)
Observed inputs Virtual inpuls
The processed inputs fod into classifiers
Intuitive mechanisms the input space difterent input-procasing based defonses, crafted adversarial example, the original clean exumple, which is virtual and unknown for the classitiers. is the adversarial perturbation.
Figure 1:
performance on the clean inputs or under general-purpose purpose transformations like rotation (Engstrom al., 2019). In contrast, the mixup training method (Zhang et al., 2018) introduces globally linear behavior en-between the data manifolds, which can also improve adversarial robustness (Zhang al., 2018; Verma et al., 2019a). Although this improvement usually less significant than resulted by AT methods, mixup-trained models can keep state-of-the-are performance on the clean inputs; mianwhile, the mixup training is computationally more efficient than AT. The interpolated AT method (Lamb et al., 2019) also shows that the mixup mechanism can further benefit the AT methods.
However, most the previous work only focuses on embedding the mixup mechanism in the training phase, while the induced global linearity of the model predictions is not well exploited in the inference phase. Compared to passive defense by directly classifying the inputs (Zhang et al., 2018; Lamb et al., 2019), it would be more effective to actively defend adversarial attacks by breaking their locality via the globally linear behavior of the mixup-trained models. In this paper, we develop an inference principle for mixup-trained models, named mixup inference (MT). In each execution, MI performs global linear transformation on the inputs, which mixups the input x with a sampled clean example F. i.e., Ax (1 A)x, (detailed in Alg. 1), and feed z into the classifier as the processed input.
There are two basic mechanisms for robustness improving under the MI operation (detailed in Sec. 3.2.1), which can be illustrated by simple geometric intuition in Fig. Icc). One is perturbation shrinkage: if the input is adversarial, i..., = 2o + 8, the perturbation 8 will shrink by factor À after performing MI, which is exactly the mixup ratio MI according to the similarity between triangles. Another one is inpul transfer: after the MI operation, the reduced perturbation X5 acts on random To. Comparing to the spatially or semantically local randomness introduced by Gaussian noise or image processing, Tg introduces spatially global and semantically diverse randomness w.r. Zo. This makes less effective to perform adaptive attacks against MI (Athalye et al., 2018). Furthermore, the global linearity of the trauiet models ensures that the information of zo remained in ão is proportional to X, such that the identity of Zo can be recovered from the statistics of Zo.
In experiments, evaluate MI CIFAR-10 and CIFAR- 100 (Krizhevsky & Hinton, 2009) under the oblivious attacks (Carlini & Wagner, 2017) and the adaptive attacks (Athalye et al., 2018). The results demonstrate that MI method is efficient in defending adversarial attacks in inference, and is also compatible with other variants of mixup. e.g., the interpolated AT method (Lamb et al., 2019). Note that Shimada al. (2019) also propose to mixup the input points in the test phase, but they do not consider their method from the aspect of adversarial robustness.
2 PRELIMINARIES
In this section, we first introduce the notations applied in this paper, then we provide the formula of mixup in training. We introduce the adversarial attacks and threat models in Appendix A.
2.1 NOTATIONS
Given an input-label pair (I, v), a classifier F returns the softmax prediction vector F(x) and the predicted label ý = arg max,e(L; Fz(x), where L is the number of classes and [4] (1,-.. L}. The classifier F makes correct prediction on x if y ÿ. In the adversarial setting, we augment the
2

---

Published as a conference paper at ICLR 2020
data pair (z,y) to triplet (z, y, 2) with an extra binary variable 2, i...,
1, if x is adversarial,
(1)
0, if x: is clean.
The variable z usually considered as hidden in the inference phase, so an input x (either clean or adversarially corrupted) can be generally denoted as Zo + å 11=1- Here To is clean sample from the data manifold p(z) with label yo. 1==1 is the indicator function, and is a potential perturbation crafted by adversaries. It is worthy to note that the perturbation 6 should not change the true label of the input, i.e., y = Yo. For & f--norm adversarial attacks (Kurakin et al., 2017; Madry et al., 2018), we have ||ôl| S e, where is a preset threshold. Based on the assumption that adversarial examples are off the data manifolds, we formally have xo + 5 & supp(p(<r) (Pang et al., 201 20)
2.2 MIXUP IN TRAINING
In supervised learning, the most commonly used training mechanism is the empirical risk minimiza- tion (ERM) principle (Vapnik, 2013), which minimizes _ Σ.= (xF();), y:) on the training dataset D (( (+) Wi) with the loss function L. While computationally efficient, ERM could lead to memorization of data (Zhang et al., 2017) and weak adversarial robustness (Szegedy et al., 2014).
As alternative, Zhang et al. (2018) introduce the mixup training mechanism, which minimizes 1 - ((F(ū)),))) M2). Here ž = λæzo + (1 (1-A)Tz1: Uy = Ayjo + (1 (1- A)y11 the input-label pairs (1y0. U50) and (.z1-M/ı) are randomly sampled from the training dataset, À ~ Beta(o, a) and a is hyperparameter. Training by mixup will induce globally linear behavior of models in-between data manifolds, which can empirically improve generalization performance and adversarial robustness (Zhang et al., 2018: Tokozume et al., 2018a:b; Verma al., 2019a;b). Compared the adversarial training (AT) methods (Goodfellow et al., 2015; Madry et al., 2018) 8), trained by mixup requires much less computation and can keep state-of-the-ar performance on the clean inputs.
m
METHODOLOGY
Although the mixup mechanism has been widely shown to be effective in different domains (Berthelot et al., 2019; Beckham et al., 2019; Verma et al., 2019a;b), most of the previous work only focuses on embedding the mixup mechanism in the training phase, while in the inference phase the global linearity of the trained model is not well exploited. Compared to passively defending adversarial examples by directly classifying them, it would be more effective to actively utilize the globality of mixup-trained models in the inference phase to break the locality of adversarial perturbations.
3.1
MIXUP INFERENCE
The above insight inspires us to propose the mixup inference (MI) method, which is specialized inference principle for the mixup-trained models. In the following, we apply colored y, & and ys to visually distinguish different notations. Consider advance. When directly feeding x into the classifier F, we can obtain the predicted label ÿ. In the adversarial setting, we are only interested in the cases where is correctly classified by F if it is clean, or wrongly classified if it is adversarial (Kurakin et al., 2018). This can be formally denoted as
input triplet (z,, y, 2). where z is unknown in
1 18z4
1==1-
(2)
The general mechanism of MI works as follows. Every time execute MI, we first sample label Ys ~ Pa (v),, then we sample x, from Pa (e)) and mixup it with x as = Az + (1 2) pa (2, y) denotes the sample distribution, which is constrained to be on the data manifold, i.e.. supp(p. (x)) C supp(p(z)). In practice, we execute MI for N times and average the output predictions to obtain (x) as described in Alg. 1. Here fix the mixup ratio X in MI as a hyperparameter, while similar properties hold if X comes from certain distribution.
3.2 THFORETICAL ANALYSES
Theoretically, with unlimited capability and sufficient clean samples, well mixup trript model F can be denoted as a linear function on the convex combinations of clean examples (Hornik et al., 1989 Guo et al., 2019), i.e., Vas, Ky p(z) and X E [0, 1). there is
H(x;; + (1 A)ar)) (x:) + (1 (z.)
(3)
3

---

Published as a conference paper at ICLR 2020
Algorithm 1 Mixup Inference (MI)
Input:
The trxuup-traiedd classifier F; the input z.
Hyperparameters: The sample distribution ps; the mixup ratio X; the number of execution N.
Initialize PM (x:) O:
for k = to .N do
Sample Vs.k Po()))) (. Lo.k PPa(zaluak)):
Mixup x with k, as 0x = Ax + A)
Update FM)(2)= (2) + F (#x);
end for
Return: The prediction (x) of input x.
Specially, we consider the case where the training objective C is the erosspyntropy loss, then H (xx) + is
should predict the one-hot vector of label /i. i.e., (x4) = ymm. If the input adversarial, then there should be an extra non-linear part G(d; mo) of F, since off the data manifolds. Thus for any input x, the prediction vector can be compactly denoted as
F(x)= F(xo + 6. 1,=1) H (za) + G(ó; mo) 1,=1.
(4)
According to Eq. (3) and Eq. (4), the output of 2 in MI is given by: F( ) = (āØ)) + G(A;; ko) 1==1
(5)
(z)) (1 (z,) + (() -
where 2o Azn + (1 A)ι, is virtual unperturbed counterpart of ā as shown in Fig. I(c). Note that (x) Alg. a Monte Carlo approximation of [f()]
(M)(
F(
Ep. [F())]
(6)
N
where _ represents the limitation when the execution times N - Now we separately investigate the y-th and ý-th (could be the same one) components of F()) according to Eq. (5), and see how these two components differ from those of F(x). These two components are critical because they decide whether we can correctly classify or detect adversarial examples (Goodfellow et al., 2016). Note that there H (To) and Hy, (x:.) = 1, thus we have the y-th components as
(x) (+) = - 2) 1y=y, + GA,A6;;
Gy (6;200) - 12m1:
1,=1-
Furthermore, according to Eq. (2), there is 1y=ÿ 1,=0- We can represent the ÿ-th components
(a(x)) 1 (0; 1,=1;
(8)
(ă) = A 1.-0
A) 1g= Gg(36; 1,=1-
From the above formulas can find that, except for the hidden variable z, the sampling label y is another variable which controls the MI output F(ū) for each execution. Different distributions of sampling vs result in different versions of MI. Here consider two casy-to-implement cases:
MI with predicted label (MI-PL): In this case, the sampling label y is the same as the predicted label g. i.c., P. (y) Ly= a Dirac distribution on ğ.
MI with other labels (MI-OL): In this case, the label ys is uniformly sampled from the labels other than g. i.e., Ps 24g (v) is discrete uniform distribution on the set {y E [L]ly 0).
We list the simplified formulas of Eq. (7) and Eq. (8) under different cases in Table 1 for clear representation. With the above formulas, we can evaluate how the model performance changes with and without MI by focusing on the formula of
AF(x;p.) (x)(x) - F(x) و [p,
Specifically, in the general-puppor setting where we aim to correctly classify adversarial exam- ples (Madry al., 2018), we claim that the MI method improves the robustness if the prediction
4

---

Published as a conference paper at ICLR 2020
Table 1: The the simplified lommulas of Eq. (7) and Eg, (8) in different versions of ML Here MI PL indicates misup inference with predicted
label; MI-OL indicates mixup inference with other labels.
MI-PL
MI-OL
2
Fa(x)
+ (6:
1
+ Gy(ó;
F((
X (A6),80)
À
A (1 A) 1umy, (A()
Fs(a)
1
1
(5(6;0))
Fz(
(1- GA(;
A
value on the true label y increases while it on the adversarial label I decreases after performing MI when the input is adversarial (z = 1). This can be formally denoted
F y (o)p.) 0; A (=)==>>==1 < 0.
(10)
We refer to this condition in Eq. (10) as robustness improving condition (RIC). Further, in the detection-purposs setting where we want detect the hidden variable and filter out adversarial inputs, we can take the gap of the ÿ-th component of predictions before and after the MI operation, i.e., AF (2: p.) as the detection metric (Pang et al., 201 To formally measure the detection ability on 2, we use the detection gap (DG), denoted as
IDG = (T;P)))) (a: P=)
(11)
higher value of DG indicates that (x: is better as a detection metric. In the following sections, specifically amalyze the properties of different versions of MI according to Table 1, and we will see that the MI methods can be used and benefit in different defense strategies.
3.2.1 MIXUP INFERENCE WITH PREDICTED LABEL
In the MI-PL case, when the input is clean (i.e., = = 0), there is F(x) = F()), which means ideally the MI-PL operation does not influence the predictions the clean inputs. When the input is adversarial (i.e., 2 = 1), MI-PL can be applied as a purpose defense or a detection-purpost defense, as we separately introduce below:
General-purpose defense: If MI-PL can improve the general-purpont robustness, it should satisfy RIC in Eq. (10). By simple derivation and the results of Table 1, this means that
1 if = 5. < 1-
(x10) [G& (8; zo) Gx (A6;
(12)
if = y.
Since an adversarial perturbation usually suppress the predicted confidence on the true label and pro- mote on the target label (Goodfellow et al., 2015), there should be Gy uo) > 0 and (5; 0. Note that the left part of Eq. (12) can be decomposed into
E E eamppa (xl)) [G&(;; zo) G& (6; 2o)] + (x10)) [G& (6: 2o) (A6;
(13)
input transfer
perfurbation shrinkage
Here Eq. (13) indicates the two basic mechanisms of the MI operations defending adversarial attacks, shown Fig. Icc). The first mechanism input transfer, i.e., the clean input that the adversarial perturbation acts on transfers from the deterministic 2o to stochastic T.. Compared to the Gaussian noise different image processing methods which introduce spatially semantically local randomness, the stochastic žy induces spatially global and semantically diverse randomness. This will make it harder to perform an adaptive attack in the white-box setting (Athalye al., 2018).
The second mechanism is perturbation shrinkage, where the original perturbation 8 shrinks by factor X. This equivalently shrinks the perturbation threshold since | = X||6||p S Ae, which means that MI generally imposes a tighter upper bound on the potential attack ability for a crafted perturbation. Besides, empirical results in previous work also show that a smaller perturbation threshold largely weakens the effect of attacks (Kurakin et al., 2018). Therefore, if an adversarial attack defended by these two mechanisms leads to a prediction degradation as in Eq. (12), then applying MI-PL would improve the robustness against this adversarial attack. Similar properties also hold for MI-OL as described in Sec. 3.2.2. In Fig. 2, we empirically demonstrate that most of the existing adversarial attacks, e.g., the PGD attack (Madry al., 2018) satisfies these properties.
5

---

Published as a conference paper at ICLR 2020
6,002
5.00
MI-PL
5,06 G- (8:g)
540%)
46
8,60
MI-OL
8.
(1
6,0
5,00
5,00
F,06)
Clean Inputs
Adversarial inputs
AG G, (18;2g)
-G (8:
4Gy G)(0x)))
Gy (A6:Fg)
Figure 2: The results averaged on 100 randomly test clean samples of CIFAR-10. The adversarial attack is untargeted PGD-10. Note that
the AG calculated here is minus value of it in Eq. (12) and Eq. (15).
Detection-purpose defense: According to Eq. (11), the formula of DG for MI-PL is
DG (Bp,-p(==() (ō; G (A6;
(1 A).
(14)
By comparing Eq. (12) and Eq. (14), means that for a given adversarial attack, MI-PL can better defend it ideally MI-PL can also better detect the crafted adversarial examples.
find that they are consistent with each other, which
then
3.2.2 MIXUP INFERENCE
OTHER LABELS
As to MI-OL, when the input clean (z = 0), there would be degeneration on the optimal clean prediction as Fy, (+) Fg (i) X, since the sampled 2, does not come from the true label y. As compensation, MI-OL can better improve robustness compared to MI-PL when the input is adversarial 1), since the sampled xi also does not come from the adversarial label ý in this case.
General-purpose defense: Note that in the MI-OL formulas of Table 1, there is a term of 1 1==.. Since we uniformly select Vs from the set [A] \ (0). there is E(1y=,.) = --. According to the RIC, MI-OL can improve robustness against the adversarial attacks if there satisfies
if = 9.
By (in (x) Be, (plll.)) (G: vo) - G (A:;50))
(A-1)(L-) L-1
if y.
(15)
Note that the conditions in Eq. (15) is strietly looser than Eq. (12), which means MI-OL defend broader range attacks than MI-PL, as verified in Fig. 2.
Detection-purposs defense: According to Eq. (11) and Table 1, the DG for MI-OL is
DG E.y,,44 (v) ER (x.( [G (6; Gy(Ad; (1 A).
(16)
is interesting to note that DG PL DL thus the two variants MI have the same theoretical performance in the detection-purpose defenses. However, in practice we find that MI-PL performs better than MI-OL detection, since empirically mixup-trained models cannot induce ideal global linearity (cf. Fig. 2 in Zhang al. (2018)). Besides, according Eq. (6), statistically make sure that the clean inputs will be correctly classified after MI-OL, there should Vk e [L] (v).
E Be
Fx ト
(17)
EXPERIMENTS
In this section. we provide the experimental results on CIFAR-10 and CIFAR-100 (Krizhevsky Hinton, 2009) to demonstrate the effectiveness of our MI methods on defending adversarial attacks.
Our codes are available https: / / thub. c03/P233 /i xup- Inference.

---

Published as a conference paper at ICLR 2020
Table 2: Classlication securacy (%) on the oblivious udversarial examples cralled on 1,000 randemly sampled test peints of CIEAR-10. Perturbation = 8/255 with step size 2/ 2/255. The subseripts indicate the number of iteration steps when perfonning attacks. The notation represents accuracy less than 1%. The parameter settings for each method can be lound Table 4.
Untargeted Mode
Targeted Mode
Methods
Cle.
PGD10
PGDso
PGD200
PGD10
PGDs0
PGD200
Mixup
93.8
3.6
3.2
3.1
S
$
S
Mixup Gaussian noise
84.4
13.5
9.6
8.8
37.7
28.6
27.9
Mixup + Random rotation
82.0
21.8
18.7
18.2
38,9
32.5
26.5
Mixup Xie et al. (2018)
82.1
23.0
19.6
19.1
38.4
31.1
25.2
Mixup + Guo (2018)
83.3
31.2
28.8
28.3
57.8
49.1
48.9
ERM MI-OL (ablation study)
81.6
7.4
6.4
6.1
33.0
26.7
23.2
Mixup MI-OL
83.9
26.1
18.8
18.3
55.6
51.2
50.8
Mixup MI-Combined
82.9
33.7
31.0
30.7
56.1
49.7
49.4
Interpolated AT
89.7
46.7
43.5
42.5
65.6
62.5
61.9
Interpolated AT Gaussian noise
84.7
55.6
53.7
53.5
70.1
69.1
69.0
Interpolated AT Random rotation
83.4
57.8
56.7
55.9
69.8
68.2
67.4
Interpolated AT Xic al. (2018)
82.1
59.7
58.4
57.9
71.1
69.7
69.3
Interpolated AT + Guo et al. (2018)
83.9
60.9
60.7
60.3
73.2
72.1
71.6
AT MI-OL (ablation study)
81.2
56.2
55.8
55.1
67.7
67.2
66.4
Interpolated AT MI-OL
84.2
64.5
63.8
63.3
75.3
74.7
74.5
50
0.58
mixap
45
mixap
mixap +Xii (2018) (201.) micup
ERN MM-OL
35
moxpp + MI--L
Acousay advessatts
30
micpp MIl Mil-Coziinnd
20
0.08 0.05
15
Mixup
Mixup MI-PL
PGD-10 luntargeted]
PGD-50 (untargeted)
100
PGD-10 (targeted)
PGD-50 (targeted)
Accuracy clean esamples (9)
AUC scores
(b) Adversarial accuracy w.r. clean accuracy
Figure 3: Results on CIFAR-10. (a) AUC scores on 1,000 randomly selected lest cleun samples 1,000 adversarial counterpurts crafted these clean samples. (b) The adversarial accuracy clean accurney 1,000 randomly selected test samples. The adversarial attack untargeted PGD-10, 10, with 8/ 8/255 and step size 2/ 2/255. Each point for a certain method corresponds to set of hyperparmmeessss..
4.1 SETUP
In training,
ResNet-50 (He al., 2016) and apply the momentum SGD optimizer (Qian, 1999) on both CIFAR- 10 and CIFAR- 100. We run the training for 200 epochs with the batch size of 64. The initial learning rate 0.01 for ERM, mixup and AT; 0.1 for interpolated AT (Lamb al., 2019). The learning rate decays with factor of 0.1 at 100 and 150 epochs. The attack method for AT and interpolated AT untargeted PGD-10 10 with
8/255 and step size 2/255 (Madry et al., 2018), and the ratio of the clean examples and the adversarial ones in each mini-batch is 1 (Lamb al., 2019). The hyperparameter a for mixup and interpolated AT 1.0 (Zhang et al., 2018), All defenses with randomness are executed 30 times to obtain the averaged predictions (Xie et al., 201 2).
4.2 EMPIRICAL VERIFICATION OF THEORETICAL ANALYSES
To verify and illustrate our theoretical analyses in Sec. 3, we provide the empirical relationship between the output predictions of MI and the hyperparameter X in Fig. 2. The notations and formulas annotated in Fig. 2 correspond to those introduced in Sec. We can see that the results follow our theoretical conclusions under the assumption of ideal global linearity. Besides, both MI-PL and MI-OL empirically satisfy RIC this case, which indicates that they can improve robustness under the untargeted PGD-10 attack CIFAR- 10, as quantitatively demonstrated in the following sections.
7

---

Published as a conference paper at ICLR 2020
Table 3: Clasilication accuracy (%) on the obiivious adversarial examples crafled on 1,000 ramdoniy sampled less points of CIEAR-100 Perturbation = 8/255 with step size 2/ 2/255. The subseripts indicate the number of iteration steps when perfonning attacks. The notation represents accuracy less than 1%. The parameter settings for each method can be found Table 5.
Untargeted Mode
Targeted Mode
Methods
Cle.
PGD10
PGDso
PGD200
PGD10
PGDs0
PGD200
Mixup
74.2
5.5
5.3
5.2
S
$
S
Mixup Gaussian noise
65.0
5.5
5.3
5.3
10.0
4.3
4.1
Mixup + Random rotation
66.2
7.8
6,7
6.3
21.4
13.5
15.2
Mixup Xie et al. (2018)
66.3
9.6
7.6
7.4
30.2
22.5
22.3
Mixup + Guo (2018)
66.1
13.1
10.8
10.5
33.3
26.3
26.1
Mixup MI-OL
68.8
12.6
9.4
9.1
37.0
29.0
28.7
Mixup + MI-Combined
67.0
14.8
11.7
11.3
31.4
26.9
26.7
Interpolated AT
64.7
26.6
24.1
24.0
52.0
50.1
49.8
Interpolated AT Gaussian noise
60.4
32.6
31.6
31.4
50.1
50.0
49.6
Interpolated AT Random rotation
62.6
34.5
32.4
32.1
51.0
49.9
49.7
Interpolated AT + Xic et al. (2018)
62.1
42.2
41.5
41.3
57.1
56.3
55.8
Interpolated AT Guo al. (2018)
61.5
36.2
33.7
33.3
53.8
52.4
52.2
Interpolated AT + MI-OL
62.0
43.8
42.8
42.5
58.1
56.7
56.5
4.3 PERFORMANCE UNDER OBLIVIOUS ATTACKS
In this subsection, we evaluate the performance of our method under the oblivious-bol attacks (Carlini Wagner, 2017). The oblivious threat model assumes that the adversary is not aware of the existence of the defense mechanism, e.g., MI, and generate adversarial examples based on the unsecured classification model. We separately apply the model trained by mixup and interpolated AT as the classification model. The AUC scores for the detection-purpos defense are given in Fig. 3(a). The results show that applying MI-PL in inference can better detect adversarial attacks, while directly detecting by the returned confidence without MI-PL performs even worse than a random guess.
We also compare MI with previous general- gencral-purpose purposs defenses applied in the inference phase, c.g., adding Gaussian noise random rotation (Tabacof & Valle, 2016); performing random padding or resizing after random cropping (Guo et al., 2018; 8; Xie et al., 2018). The performance of our reported in Table and Table 3, respectively.
method and baselines on CIFAR-I0 and CIFAR-100 Since for each defense method, there is trade-off between the accuracy on clean samples and adversarial samples depending on the hyperparameters, e.g., the standard deviation for Gaussian noise, we carefully select the hyperparameters to ensure both our method and baselines keep a similar performance on clean data for fair comparisons. The hyperparameters used in our method and baselines are reported in Table 4 and Table 5. In Fig. 3(b), we further explore this trade-off by grid searching the hyperparameter space for each defense demonstrate the superiority of our method.
As shown in these results, our MI method can significantly improve the robustness for the trained mod- els with induced global linearity, and is compatible with training-phase defenses like the interpolated AT method. As practical strategy, we also evaluate a variant of MI, called MI-Combined, which applics MI-OL if the input is detected as adversarial by MI-PL with a default detection threshold; otherwise returns the prediction the original input. We also perform ablation studies of ERM/ AT + MI-OL Table 2, where no global linearity is induced. The results verify that our MI methods indeed exploit the global linearity of the mixunetrained models, rather than simply introduce randomness.
4.4 PERFORMANCE UNDER WHITE-BOX ADAPTIVE ATTACKS
Following Athalye et al. (2018), we test our method under the white-box adaptive attacks (detailed in Appendix B.2). Since we mainly adopt the PGD attack framework, which synthesizes adversarial examples iteratively, the adversarial noise will be clipped to make the input image stay within the valid range. It results in the fact that with mixup different training examples, the adversarial perturbation will be clipped differently. To address this issue, we average the generated perturbations the adaptive samples as the final perturbation. The results of the adversarial accuracy w.r.t the number of adaptive samples are shown in Fig. 4. We can see that even under a strong adaptive attack, equipped with MI still improve the robustness for the classification models.
8

---

Published as a conference paper at ICLR 2020
Advenarial sccumey
Uniargelod mode
Targeted mode
Untargeted mode
Targeted mode
Number of adaptive samples
Number of adaptive samples
Number of adaptive samples
Number of adaptive samples
Mixup + MI-OL
Interpolated AT MI-OL
Adapiive PGD-10 10
Adaptive PGD-50
Adapiive PGD-200
Figure Classification accuracy under adaptive PGD attacks CIFAR-I0. The number adaptive samples refers execution times sampling each lteration step adaptive PGD. The dash lines the accuracy of trained models without MI OL under PGD attacks.
5 CONCLUSION
In this paper, we propose the MI method, which specialized for the trained models with globally linear behaviors induced by, e.g., mixup or interpolated AT. As analyzed in Sec. 3, MI can exploit this induced global linearity in the inference phase to shrink and transfer the adversarial perturbation, which breaks the locality of adversarial attacks and alleviate their aggressivity. In experiments, empirically verify that applying MI can return more reliable predictions under different threat models.
ACKNOWLEDGEMENTS
This work was supported by the National Key Research and Development Program of China (No. 2017YFA0700904), NSFC Projects (Nos. 61620106010, U19B2034, U1811461), Beijing NSF Project (No. L172037), Beijing Academy of Artificial Intelligence (BAAI), Tsinghua-Huuwer Joint Research Program, grant from Tsinghua Institute for Guo Qiang. Tiangong Institute for Intelligent Computing, the JP Morgan Faculty Research Program and the NVIDIA NVAIL Program with GPU/DGX Acceleration.
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International Conference on Machine Learning (ICML). 2018.
Christopher Beckham, Sina Honari, Alex Lamb, Vikas Verma, Farnoosh Ghadiri, R Devon Hjelm, and Christopher Pal. Adversarial mixup resynthesizers. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach semi-supervised learning. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Nicholas Carlini and David Wagner. Adversarial examples not easily detected: Bypassing ten detection methods. ACM Workshop on Artificial Intelligence and Security (AISec), 2017.
Nicholas Carlini and David Wagner. Audio adversarial examples: Targeted attacks on speech-to-text. In 2018 IEFE Security and Privacy Workshops (SPW), pp. 1-7. IEEE, 2018.
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. arXiv preprint arXiv:1902.06700 2019.
Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. Adversarial attack graph structured data. In International Conference on Machine Learning (ICML), 2018.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

---

Published as a conference paper at ICLR 2020
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and a translation suffice: Fooling enns with simple transformations. In International Conference Machine Learning (ICML), 2019.
Alhussein Fawzi, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Robustness of classi- fiers: from adversarial to random noise. In Advances in Neural Information Processing Systems (NeurlPS), pp. (632-1640) 2016.
Alhussein Fawzi, Hamza Fawzi, and Omar Fawzi Adversarial vulnerability for any classifier. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
lan Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. http: //www. deep la rn igbook.org.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and hamnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten. Countering adversarial images using input transformations. In International Conference Learning Representations (ICLR), 2018.
Hongyu Guo, Yongyi Mao, and Richong Zhang. Mixup as locally linear out-of-manifold regular- ization. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 33, pp. 1722,, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European Conference on Computer Vision (ECCV), pp. 630-645. Sprinee, 2016.
Kurt Horiik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks universal approximators. Neural networks, 2(5): 3666, 1989.
Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, and Picter Abbeel. Adversarial attacks neural network policies. arXiv preprint arXiv: 022284, 2017.
Hiroshi Inoue. Data augmentation by pairing samples for images classification. arXiv preprint arXiv:1801.02929, 2018
Di Jin, Zhijing Jin, Tianyi Zhou, and Peter Szolovits. bert really robust? natural language attack on text classification and entailment. arXiv preprint arXiv: 1932, 2019.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In The International Conference on Learning Representations (ICLR) Workshops, 2017.
Alexey Kurakin, Ian Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang, Jun Zhu, Xiaolin Hu, Cihang Xie, et Adversarial attacks and defences competition. arXiv preprint 1:Xiv:1804.0009 2018.
Alex Lamb, Vikas Verma, Juho Kannala, and Yoshua Bengio. Interpolated adversarial training: Achieving robust neural networks without sacrificing accuracy. arXiv preprint arXiv: 1906: 1ww..0000668888 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks easily fooled: High confidence predictions for unrecognizable images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 427-436 2015.
Tianyu Pang, Chao Du, Yinpeng Dong, and Jun Zhu. Towards robust detection of adversarial examples. In Advances in Neural Information Processing Systems (NeurIPS), pp. 4579-4589, 2018a.
10

---

Published as a conference paper at ICLR 2020
Tianyu Pang, Chao Du, and Jun Zhu. Max-mahalanobia linear discriminant analysis networks. In International Conference on Machine Learning (ICML), 2018b.
Ning Qian, On the momentum term in gradient descent learning algorithms. Neural networks, 12(1): 145-151, 1999.
Edward Raff, Jared Sylvester, Steven Forsyth, and Mark McLean, Barrage of random transforms for adversarially robust defense. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6528 6537, 2019.
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Takuya Shimada, Shoichiro Yamaguchi, Kohei Hayashi, and Sosuke Kobayashi. Data interpolating prediction: Alternative interpretation of mixup. arXiv preprint arXiv: 08412, 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference Learning Representations (ICLR), 2014.
Pedro Tabacof and Eduardo Valle. Exploring the space of adversarial images. In 2016 International Joint Conference on Neural Networks (JCNN). pp. 426 433. IEEE, 2016.
Yuji Tokozume, Yoshitaka Ushiku, and Tatsuya Harada. Learning from between-class examples for deep sound recognition. International Conference on Learning Representations (ICLR), 2018a.
Yuji Tokozume, Yoshitaka Ushiku, and Tatsuya Harada. Between-class learning for image classi- fication. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5486-5494, 2018b.
Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 2013.
Vikas Verma, Alex Lamb, Christopher Beckham, Aaron Courville, Ioannis Mitliagkis, and Yoshua Bengio. Manifold mixup: Encouraging meaningful on-manifold interpolation as regularizer. In International Conference on Machine Learning (ICML), 2019a.
Vikas Verma, Alex Lamb, Juho Kannala, Yoshua Bengio, and David Lopez- Paz. Interpolation consistency training for semi-supervised learning. arXiv preprint arXiv: (ati::9993.0530 1903003855 2019b.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. In International Conference on Learning Representations (ICLR), 2018.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understand- ing deep learning requires rethinking generalization. In International Conference on Learning Representations (ICLR), 2017.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference on Machine Learning (ICML), 2019.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations (ICLR), 2018.

---

Published as a conference paper at ICLR 2020
A MORE BACKGROUNDS
In this section, we provide more backgrounds which are related to our work in the main text.
A.I ADVERSARIAL. ATTACKS AND THREAT MODELS
Adversarial attacks. Although deep learning methods have achieved substantial success in different domains (Goodfellow et al., 2016), human imperceptible adversarial perturbations can be casily crafted fool nigh-performance models, e.g., deep neural networks (DNNss (Nguyen et al., 2015).
One of the most commonly studied adversarial attack is the projected gradient descent (PGD) method (Madry et al., 2018). Let be the number of iteration steps, Ty be the original clean example, then PGD iteratively crafts the adversarial example as
x = clip,, + 6, sign (V27 (()]-119)),
(18)
where clip,,e(-) is the clipping function. Here 2á is randomly perturbed image in the neighborhood of zg. i.e. Ü (20, c), and the finally returned adversarial example is x = x 2o + 6, following our notations in the main text.
Threat models. Here we introduce different threat models the adversarial setting. As suggested in Carlini al. (2019), threat model includes set of assumptions about the adversarys goals, capabilities, and knowledge.
Adversary's goals could be simply fooling the classifiers to misclassify, which is referred as untargeted mode, Alternatiely, the goals can be more specific to make the model misclassify certain examples from source class into a target class, which is referred to as largeted mode. In our experiments, we evaluate under both modes, as shown in Table 2 and Table 3.
Adversary's capabilitiess describe the constraints imposed on the attackers. Adversarial examples require the perturbation ò to be bounded by a small threshold e under e--norm, i.c., |0l|p S e. For example, in the PGD attack, we consider under the -nom.
Adversary's 's knowledge describes what knowledge the adversary is assumed to have. Typically, there are three settings when evaluating a defense method:
Oblivious adversaries are not aware of the existence of the defense D and generate adver- sarial examples based on the unsecured classification model F (Carlini & Wagner, 2017).
White-box adversaries know the scheme and parameters of D, and can design adaptive methods to attack both the model F and the defense simultancously (Athalye et al., 2018).
Black-box adversaries have access the parameters of the defense D or the model F with varying degrees of black-box access (Dong et al., 2018).
In experiments, mainly test under the oblivious setting (Sec. 4.3) and white-box setting (Sec. 4.4), since previous work has already demonstrated that randomness itself is efficient on defending black-box attacks (Guo et al., 2018; Xie et al., 2018).
A.2 INTERPOLATED ADVERSARIAL TRAINING
To date, the most widely applied framework for adversarial training (AT) methods is the saddle point framework introduced in Madry et al. (2018):
min p(0), where p(0) ((ra)) [max C(x 8, 0)].
(19)
Here 0 represents the trainable parameters in the classifier F, and S a set of allowed perturbations. In implementation. the inner maximization problem for each input-label pair (z, y) is approximately solved by, e.g., the PGD method with different random initialization (Madry et al., 2018).
As a variant of the AT method, Lamb et al. (2019) propose the interpolated AT method, which combines AT with mixup. Interpolated AT trains on interpolations of adversarial examples along with interpolations of unperturbed examples (cf. Alg. 1 in Lamb et al. (2019)). Previous empirical results demonstrate that interpolated AT can obtain higher accuracy on the clean inputs compared to the AT method without mixup, while keeping the similar performance of robustness.
12

---

Published as a conference paper at ICLR 2020
TTable 4: 'hee parameter settings for the methods in Table 2. The number of execution for each random method is 30.
Methods
Parameter Settings
Mixup
Mixup
Gaussian noise
Noise standard deviation o = 0.04
Mixup Random rotation
Rotation degree range [-40", ,400)
Mixup +
(2018)
The random crop size is randomly selected from [16, 24] The random crop size is randomly selected from [22, 30)
Mixup Guo et al. (2018)
ERM MI-OL (ablation study)
The Åor. = 0.6
Mixup MI-OL
The Aor. 0.5
Mixup MI-Combined
The λoL. = 0.5, Aoe. = 0.4, threshold is 0.2
Interpolated AT
Interpolated AT + Gaussian noise
Noise standard deviation a 0.075
Interpolated AT Random rotation
Rotation degree range
Interpolated AT + Xic ct al. (2018)
The random crop size is randomly selected from [20, 28]
Interpolated AT Guo et al. (2018)
The random crop size is randomly selected from [20, 28]
AT + MI-OL (ablation study)
The AOL = 0.8
Interpolated AT MI-OL
The Aor. 0.6
B TECHNICAL DETAILS
We provide more technical details about our method and the implementation of the experiments.
B. MORE DISCUSSION ON THE MI METHOO
Generality. According to Sec. 3, except for the mixup-trained models, the MI method generally compatible with any trained model with induced global linearity. These models could be trained by other methods, e.g., manifold mixup (Verma et al., 201 9a; Inoue, 2018; Lamb et al., 2019). Besides, better defend white-box adaptive attacks, the mixup ratio in MI could also be sampled from certain distribution to put in additional randomness.
Empirical gap. As demonstrated in Fig. there is gap between the empirical results and the theo- retical formulas in Table 1. This is because that the mixup mechanism mainly acts as regularization in training, which means the induced global linearity may not satisfy the expected behaviors. To improve the performance of MI, stronger regularization can be imposed, e.g., training with mixup for more epochs, or applying matched À both training and inference.
B.2 ADAPTIVE ATTACKS FOR MIXUP INFERENCE
Following Athalye al. (2018), we design the adaptive attacks for our MI method. Specifically, according to Eq. (6), the expected model prediction returned by MI is:
(x) Ep. [F(r + (1- -))x))]
(20)
Note that generally the À in MI comes from certain distribution. For simplicity, fix À hyperparameter in our implementation. Therefore, the gradients of the prediction w.r.t. the input x is:
a Fz)
TaF(Ax + (1 --)x.))
Ep.
(21)
dx
dr
(uF(u)
0Ar + (1 - A)x
Ep.
(22)
Ou u= 1))x
Ox
TaF(u)
AEP.
Ou
11=
(23)
13

---

Published as a conference paper at ICLR 2020
TTable 5: 'Tee parameter settings for the methods in Table 3. The number of execution for each random method is 30.
Methods
Parameter Settings
Mixup
Mixup Gaussian noise
Noise standard deviation a 0.025
Mixup Random rotation
Rotation degree range [-20", 20"
Mixup +
(2018)
The random crop size is randomly selected from [18, 26]
Mixup Guo et al. (2018)
The random crop size is randomly selected from [24, 32]
Mixup + MI-OL
The Aor. = 0.5
Mixup MI-Combined
The λoL. 0.5, Aos. 0.4, threshold is 0.2
Interpolated AT
Interpolated AT Gaussian noise
Noise standard deviation 0.06
Interpolated AT + Random rotation
Rotation degree range [-20", -00)
Interpolated AT Xie al. (2018)
The random crop size is randomly selected from [22, 30]
Interpolated AT + Guo et al. (2018)
The random crop size is randomly selected from [24, 32]
Interpolated AT MI-OL
The λot 0.6
Clean
Adversarial
Figure S: Adversarial examples crafted by aduptive attacks with c 16/255 on CIFAR 10, against the defense of Interpolated + MI-OL.
In the implementation of adaptive PGD attacks, we first sample a series of examples {æ,,k k= where NA is the number of adaptive samples in Fig. 3. Then according to Eq. (18), the sign of gradients used in adaptive PGD can be approximated by
Na
F (2:)
OF(u)
sign
sign
(24)
Ör
Du 1u=dx+ -A)E
B.3 HYPERPARAMETER SETTINGS
The hyperparameter settings of the experiments shown in Table 2 and Table are provided in Table and Table 5, respectively. Since the original methods Xie et al. (2018) and Guo et al. (2018) are both designed for the models ImageNet. we adapt them for CIFAR-10 and CIFAR-100. Most of our experiments are conducted on the NVIDIA DGX-1 server with eight Tesla P100 GPUs.
14


Reviews:
[{"review_id": "SkgiTp66YB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a novel use of mixup, which is originally a data augmentation method incorporating two training samples and their corresponding labels. The authors utilize mixup not for training but for inference (MI; Mixup Inference). Experimental results on Cifar 10, and Cifar 100 show that MI can boost the classification performance in combination with interpolated AT (Adversarial  Training) and mixup.\n\nI lean to accept this paper. The proposed method is simple but effective, moreover well-motivated. The experimental results, including several ablation studies, show a high versatility with existing methods.\n\nMy minor concerns are, however, consisting of two points.\n- The authors should repeat the experiments several times and show the averages and standard errors to make the significance clear.\n- Both Cifar 10 and Cifar 100 are relatively small scale datasets. I would like the authors to investigate larger ones.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Hylb_3L5uS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Notes: \n\n-Paper claims that adversarial examples stem from locally non-linear behavior.  However, wasn't this the exact opposite of the conclusion of \"Explaining and Harnessing Adversarial Examples\" (Goodfellow 2015)?  It is cited in this paper but I think the proper conclusion is the opposite of what is written here.  \n\n-Linearity however may still be an important component since it simplifies the problem of adversarial robustness.  \n\n-Many techniques try to introduce adversarial robustness through transformations during inference time.  \n\n-Paper claims that adversarial training induces locally linear behavior - but I'm rather skeptical of this claim.  Think about something like KNN with k=1 and euclidean distance.  This should be L2-robust on the training set, yet very non-linear.  \n\n-I understand the contents of Figure 1, but I'm not sure exactly what conclusion I should draw from it.  \n\n-The novel procedure presented is \"Mixup Inference\".  This involves classifying using interpolations of test inputs.  \n\n-Two mechanisms are proposed for why this could help.  One is that the magnitude of the perturbation will shrink after doing mixup (although the signal in the original image will also shrink, so I'm not sure if I like this argument).  The second argument is that the adversarial perturbation will have to appear with different random examples which will force the attack to be more \"universal\" to succeed.  This second argument I find much more compelling.  \n\n-The notation $y_s \\sim p_s(y_s)$ is rather abusive since the random variable and the same have the same name but this is common in machine learning.  \n\n-The technique if I understand correctly (Algorithm 1) amounts to using an average of the prediction at the original point along with an average of the mixes going to all other points in the dataset.  \n\n-The paper is a bit slow to explain what distribution lambda will come from - but it effects the algorithm a lot (especially if the distribution is symmetric or asymmetric).  \n\nComments: \n\n-There is another paper (Shimada 2019) that also uses interpolation at test time and should be cited here, although I admit that paper is written in a confusing way so the connection may not be immediately obvious: https://arxiv.org/pdf/1906.08412v1.pdf\n\n-I have a suggestion for the organization of the paper that I think would improve it.  I would suggest to first introduce the method in a clear fashion (after motivating it), along with the equations.  Then, *after that*, clearly and separately introduce the analysis of the \"optimal linear model\": \"a well mixup-trained model F can be denoted as a linear function H on the convex combinations of clean examples\".  I think that would make the paper much clearer.  \n\n-For example, how can we actually know what G_k() is unless we know the adversarial perturbation (which shouldn't generally be possible during inference)?  I found this discussion to be rather confusing (basically section 3.1) although admittedly it might be my own fault.  \n\n-Why does mixup-inference hurt the clean accuracy by 10% (table 2)?  This seems like quite a lot to me.  Still the degree of robustness does seem impressive.  And I also believe that the obtained robustness of this technique along with AT is state of the art.  \n\n-It might be nice to see examples of attacks on the resulting model, especially the one with the best robustness.  It's possible that Linf-bounded attacks against *this model* will be perceptible, which would support lowering the epsilon-attack-budget (which is actually a good thing for the research field as it's evidence that maybe this epsilon shouldn't be considered practically imperceptible).  \n\nReview: \n\nThis paper was interesting, because it has nice experimental results and seems like a good idea, but I feel like the paper needs to be improved.  The biggest issue is that the paper repeatedly claims that adversarial robustness can be improved by making networks more linear, yet I believe that this is the opposite of what prior work has found.  I also found the exposition of the idea to be confusing as it simultaneously introduces the idea and an analysis of the technique - I would much prefer if the technique were introduced first and the analysis under some optimal assumptions moved to a different section.  ", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "B1g-lU5W9B", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a novel method for an adversarial attack named mixup inference (MI).  Most of the work focuses on embedding mixup mechanism in the training phase, but MI uses the mixup in the inference phase. MI method has two main effects for the adversarial attack: one is perturbation shrinkage, and the other one is input transfer because MI can exploit\nthe induced global linearity. The experimental results show that MI can return more reliable predictions under different threat models.\n\nThis paper should be accepted because the proposed method is super simple but effective for defending from adversarial attacks under different threat conditions. This paper is well-written, including theoretical insights on why the MI method works.\n\nThe reviewer has some questions or comments to clarify the paper:\n1) In the explanation of the MI method, the authors assume only the cases where the input data is correctly classified if it is clean, or wrongly classified if it is adversarial. In a realistic situation, the classifier sometimes outputs mislabels. Thus is the discussion in Sec.3 valid if the clean input data misclassified?\n\n2) To predict the category of the input, MI methods must perform inference N times. It is not efficient. Are there any ideas to reduce the number of inferences?\n\n3) MI-Combined seems ad-hoc. It would be better to state its justification by theory.\n\n4) The same idea of mixup was proposed at the same conference (ICLR2018). It should be cited.\nTokozume et al., Learning from Between-class Examples for Deep Sound Recognition. ICLR, 2018.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed a mixup inference (MI) method, for  mixup-trained models, to better defend adversarial attacks.  The idea is novel and is proved to be effective on CIFAR-10 and CIFAR-100.  All reviewers and the AC agree to accept the paper.
Decision: Accept (Poster)