Noisy Feature Mixup
Authors: Soon Hoe Lim
N. Benjamin Erichson
Francisco Utrera
Winnie Xu
Michael W. Mahoney
Venue: ICLR 2022 Poster (2022)
Keywords: Data augmentation
implicit regularization
mixup
noise injection
model robustness

Abstract: We introduce Noisy Feature Mixup (NFM), an inexpensive yet effective method for data augmentation that combines the best of interpolation based training and noise injection schemes. Rather than training with convex combinations of pairs of examples and their labels, we use noise-perturbed convex combinations of pairs of data points in both input and feature space. This method includes mixup and manifold mixup as special cases, but it has additional advantages, including better smoothing of decision boundaries and enabling improved model robustness. We provide theory to understand this as well as the implicit regularization effects of NFM. Our theory is supported by empirical results, demonstrating the advantage of NFM, as compared to mixup and manifold mixup. We show that residual networks and vision transformers trained with NFM have favorable trade-offs between predictive accuracy on clean data and robustness with respect to various types of data perturbation across a range of computer vision benchmark datasets.

Published as a conference paper at ICLR 2022
NOISY FEATURE MIXUP
Soon Hoe Lim"
N. Benjamin Erichson" University of Pittsburgh er cchson@pitt.ee..
Francisco Utrera
Nordita,
University of Pittsburgh and ICSI
KTH and Stockholm University soon hoe. simiumuuuu...umm edu
ut.rer@berkeeey edd
Winnie Xu
Michael W. Mahoney
University of Toronto
ICSI and UC Berkeley
wi iniexuucs toronto. edu
umahhney@stat..comm bertetttbbbrreeey........ .ddu
ABSTRACT
We introduce Noisy Feature Mixup (NFM), inexpensive yet effective method for data augmentation that combines the best of interpolation based training and noise injection schemes. Rather than training with convex combinations of pairs of examples and their labels, we use woise-perturben of data points in both input and feature space. This method includes mixup and manifold mixup special cases, but it has additional advantages, including better smoothing of decision boundaries and enabling improved model robustness. We provide theory understand this well the implicit regularization effects of NFM. Our theory is supported by empirical results, demonstrating the advantage of NFM, compared mixup and manifold mixup. We show that residual networks and vision transformers trained with NFM have favorable trade-offs between predictive accuracy on clean data and robustness with respect various types of data perturbation across a range of computer vision benchmark datasets.
convex combinations of pairs
INTRODUCTION
Mitigating over- weer-fitting and improving generalization on test data are central goals in machine learning, One approach to accomplish this is regularization, which can be either data-agnostic or data-dependent (e.g., explicitly requiring the use of domain knowledge or data). Noise injection is typical example of data-agnostic regularization (Bishopl 1995), where noise can be injected into the input data (An 1996), or the activation functions (ulcchrre et al. (016)), or the hidden layers of deep neural networks (Camuto et al. 2020 Lim et al. 2021). Data augmentation constitutes a different class of regularization methods (Baird) 1992 Chapelle et al.) 2001 DeCoste & Schölkopf 2002), which can also be either tata-agnostic or data-dependent. Data augmentation involves training model with not just the original data, but also with additional data that is properly transformed, and it has led to state-of-the-art results in image recognition (Cireşan et al. 2010; Krizhevsky et al. 2012). The prropoey-pooposed data-agnostic method, mixup (Zhang al. 2017). trains model linear interpolations of a random pair of examples and their corresponding labels, thereby encouraging the model to behave linearly in-between training examples. Both noise injection and mixup have been shown to impose smoothness and increase model robustness to data perturbations (Zhang al. 2020 Carratino et al.] 2020 Lim et al. 2021). which is critical for many safety and sensitive applications (Goodfellow et al.
Madry et al. 2017).
In this paper, we propose and study simple yet effective data augmentation method, which we call Noisy Feature Mixup (NFM). This method combines mixup and noise injection, thereby inheriting the benefits of both methods, and can seen as a generalization of input mixup (Zhang et al. 2017) and manifold mixup (Verma al. 2019). When compared to noise injection and mixup, NFM imposes regularization on the largest natural region surrounding the dataset (sce Fig. [.. which may help improve robustness and generalization when predicting out of distribution data. Conveniently, NFM can be implemented on top of manifold mixup, introducing minimal computation overhead.
"Equal contribution

---

Published as a conference paper at ICLR 2022
Contributions. Our main contributions follows.
Amy +
A)mg
We study NFM via the lens of implicit regulariza- tion, showing that NFM amplifies the regularizing effects of manifold mixup and noise injection, im- plicitly reducing the feature-output Jacobians and Hessians according to the mixing level and noise levels (see Theorem T
2/2
x
Am + A)mz
We provide mathematical analysis to show that NFM can improve model robustness when com- pared manifold mixup and noise injection. In particular, we show that, under appropriate assump- tions, NFM training approximately minimizes an upper bound on the sum of an adversarial loss and feature-dependee regularizers (see Theorem (
Figure 1: An illustration of how two data points, x and X2. are transformed in mixup (top) and noisy feature mixup (NFM) with S = {0} (bottom).
We provide empirical results in support of our theoretical findings, showing that NFM improves robustness with respect to various forms of data perturbation across a wide range of state-of-the- architectures on computer vision benchmark tasks.
In the Supplementary Materials (SM), we provide proofs for our theorems along with additional theoretical and empirical results to gain more insights into NFM. In particular, we show that NFM
can implicitly increase classification margin (see Proposition ii SMC) and the noise injection procedure in NFM can robustify manifold mixup in probabilistic sense (see Theorem 5 in SM
We also provide and discuss generalization bounds for NFM (see Theorem and and 7 in SM
Notation. I denotes identity matrix, [K] := f1..... .......Ky K}. the superseript T denotes transposition, denotes composition, C denotes Hadamard product, 1l denotes the vector with all components equal one. For a vector v denotes its kth component and ||æ||p denotes its Ip norm for > O. conv(XX) denote the convex hull of X. Mx (a. b) == Xa + A)b, for random variables a, b, X. 8g denotes the Dirac delta function, defined as 6x(r)
if z and (z) 0 otherwise. 1 denotes indicator function of the set A. For a. > D2 == a- Beta(a + 1, a+g
B Beta + 1, a:) denotes
a uniform mixture of two Beta distributions. For two vectors a, b, cos(a, b) == (a, b) ///laa|aallbla denotes their cosine similarity. (a, b) is Gaussian distribution with mean and covariance 6.
2 RELATED WORK
Regularization, Regularization refers to any technique that reduces overfitting in machine learning; see (Mahoney & Orecchia 2011 Mahoney]
and references therein, in particular for a discussion of implicit regularization, a topic that has received attention recently in the context of stochastic gradient optimization applied to neural network models. Traditional regularization techniques such as ridge regression, weight decay and dropout do not make use of the training data reduce the model capacity. A powerful class of techniques data augmentation, which constructs additional examples from the training set, e.g., by applying geometric transformations the original data (Shorten & Khoshgoftaar 2019). A recently proposed technique is mixup (Zhang et al. (017). where the examples are created by taking convex combinations of pairs of inputs and their labels. Verma et al. (0119) extends mixup to hidden representations in deep neural networks. Subsequent works by Greenewald et al. (202T): Yin al. (202T): : Engstrom et al. 2019 Kim et al. (2020ab; Yun et al. Hendrryks et al. (2019) introduce different variants and extensions of mixup. Regularizzn Regularization is also intimately connected to robustness (Hoffman
2019 Sokolić et al. (2017; et al.)
2018 Elsayed et al.) 2018 Moosavi-Deztooll et al.) 2019). Adding to the list is NFM, a powerful regularization method that we propose to improve model robustness.
Robustness. Model robustness is an increasingly important issue in modern machine learning. Robustness with respect to adversarial examples (Kurakin et al. 2016) can be achieved by adversarial training (Goodfellow et al. 2014| Madry et al. [017 Utrera al. 2020). Several works present theoretical justifications to observed robustness and how data augmentation can improve it (Hein &
Andriushchenko 2017; Yang al.
2020b Couellan 202 Pinot et al. 2019a 2021 Zhang et al.
2020; 2021]]22221211 Carratino
Carratioo et al. 2020; Kimura, 2020: Dao et al. [019:] Wu et al. 2020 Gong et al.
2020
Chen al. (0220). Re Relatedly, Fawzi al. (2016): Franceschi et al. (0018) Lim et al. (222)
2

---

Published as a conference paper at ICLR 2022
investigate how noise injection be used improve robustness. Parallel this line of work, provide theory to understand how NFM can improve robustness. Also related is the study of the trade-off's between robustness and accuracy (Min et al. 2020; Zhang et al. 2019 Tsipras et al. 2018 Schmidt et al.
2018 Su et al. 2018 Raghunathan et al. 2020 Yang et al. 2020a).
3 NOISY FEATURE MIXUP
Noisy Feature Mixup is a generalization of input mixup Zhang et al. 2017) and manifold mixup (Verma et al., 2019). The main novelty of NFM against manifold mixup lies in the injection of noise when taking convex combinations of pairs of input and hidden layer features. Fig. Iillstraates at high level, how this modification alters the region in which the resulting augmented data resides. Fig. shows that NFM most effective at smoothing the decision boundary of the trained classifier; compared noise injection and mixup alone, it imposes the strongest smoothness this dataset.
Formally, we consider multi-class classification with labels. Denote the input space by X' C Rd and the output space by y = RRK The classifier, g. constructed from learnable map f : - R mapping an input x to its label, g(x) = arg max; (xx E [K]. We are given a training set, Zn = (I y1 H= consisting of n pairs of input and one-hot label, with each training pair 2, := (zz Us) E E x y drawn i.i.d. from ground-truth distribution D. We consider training a deep neural network f == fx o 9k- where gk X (X) maps an input to a hidden representation at layer k. and fa (X)) - (r())) == maps the hidden representation to one-hot label at layer L.
Here, 9x (2) C Rd for E [L], dr. == K, (o and fo(x) = f(x).
Training f using NFM consists of the following steps:
1. Select a random layer from a set, S c {0) U [4], of cligible layers in the neural network.
2. Process two random data minibatches (z. y) and (x', y²) as usual, until reaching layer k. This gives us two immediate minibatches (gx(1),, y) and (z)),,),
3. Perform mixup on these intermediate minibatches, producing the mixed minibatch:
(Øx. V) = (Mx (z), Iu (x))), M)().1)).
(1)
where the mixing level À Beta(a, 3), with the hyper-parameters a, B 0.
4. Produce noisy mixed minibatch by injecting additive and multiplicative noise:
(ā. ú) = ((1 + Tmaut O Mx(gn (I), (x')) + aadaEk eadd MM v/),,
(2)
where the eged and 5% emautt are valuval independent random variables modeling the additive and multiplicative noise respectively, and addd, mmult 2 0 pre- rre-speifeeed noise levels.
5. Continue the forward pass from layer & until the output using the noisy mixed minibatch (īx, Ø).
6. Compute the loss and gradients that update all the parameters of the network.
Baseline (85.5%).
Dropout (87.0%).
Weight decay (88.0%).
Noise injections (87.0%).
Mixup (84.5%).
Manifold mixup (88.5%). Noisy mixup (89.0%).
NFM (90.0%).
Figure 2: The decision boundaries and test accuracy (in parenthesis) for different training schemes on toy dataset in binary classification (see Subsection F2 for details).
3

---

Published as a conference paper at ICLR 2022
At the level of implementation, following (Verma et al. 2019), we backpropagate gradients through the entire computational graph, including those layers before the mixup layer k.
In the case where aadd muit O, NFM reduces to manifold mixup (Verma et al. 2019). If in addition S = {0}. reduces to the original mixup method (Zhang et 2017). The main difference between NFM and manifold mixup lies in the noise injection of the fourth step above. Note that NFM is equivalent to injecting noise into 9& (2), It (x') first, then performing mixup on the resulting pair, i.e., the order that the third and fourth steps occur does not change the resulting noisy mixed minibatch. For simplicity, we have used the same mixing level, noise distribution, and noise levels for all layers in S in our formulation.
Within the above setting, we consider the expected NFM loss:
L (f) -ewpp.eerr////---------------- (/t
M)) My. v)),
where : RK x RRK - [0, oo) is a loss function (note that here we have suppressed the dependence of both and on the learnable parameter 0 in the notation), Ex (egdd .EK enoult) are drawn from some probability distribution Q with finite first two moments, and
Mr.ea (x). 9n (z')) (1 + muutt Smaitt @ MMx (z), gx(a')) aadaSk codd
NFM seeks to minimize a stochastic approximation of L INFF (J) by sampling finite number of k X, Ex values and using minibatch gradient descent to minimize this loss approximation.
4 THEORY
In this section, we provide mathematical analysis to understand NFM. We begin with formulating NFM in the framework of vicinal risk minimization and interpreting NFM stochastic learning strategy in Subsection 4.1 Next, we study NFM via the lens of implicit regularization in Subsection 4.2 Our key contribution is Theorem [ which shows that minimizing the NFM loss function is approximately equivalent to minimizing sum of the original loss and feature-dependent regularizers, amplifying the regularizing effects of manifold mixup and noise injection according the mixing and noise levels. In Subsection 4.3] we focus on demonstrating how NFM can enhance model robustness via the lens of distributionally robust optimization. The key result of Theorem 2 shows that NFM loss is approximately the upper bound on a regularized version of an adversarial loss, and thus training with NFM not only improves robustness but can also mitigate robust over-fitting, a dominant phenomenon where the robust test accuracy starts to decrease during training (Rice et al. 2020).
4.1 NFM: BEYOND EMPIRICAL RISK MINIMIZATION
2003) is to select a hypothesis The standard approach in statistical learning theory (Bousquet et al. function f X' + y from a pre-defined hypothesis class F to minimize the expected risk with respect to D and to solve the risk minimization problem: jer R(f) := E(z,y)>D [f(()) v)). for a suitable choice of loss function In practice, we do not have access to the ground-truth distribution. Instead, we find an approximate solution by solving the empirical risk minimization (ERM) problem, in which case D is approximated by the empirical distribution F,, = 1 Σi= 8z, In other words, in
ERM we solve the problem: fe Rr (f) == + ΣT ((f()))))))
However, when the training set is small or the model capacity is large (as is the case for deep neural networks), ERM may suffer from overfitting. Vicinal risk minimization (VRM) is a data augmentation principle introduced in (Vapnik 2013) that goes beyond ERM, aiming better estimate expected risk and reduce overfitting. In VRM. model is trained not simply on the training set, but on samples drawn from vicinal distribution, that smears the training data to their vicinity. With appropriate choices for this distribution, the VRM approach has resulted in several effective regularization schemes (Chapelle et al.) 2001). Input mixup (Zhang et al., 2017) can be viewed as an example of VRM, and it turns out that NFM can be constructed within VRM framework at the feature level (see Section A in SM). On a high level, NFM can be interpreted as a random procedure that introduces feature- dependent noise into the layers of the deep neural network. Since the noise injections are applied only during training and not inference, NFM is an instance of a stochastic learning strategy.
Note that the injection strategy of NFM differs from those of An (1996); Camuto et al. (2020) Lim
4

---

Published as a conference paper at ICLR 2022
al. 2021). Here, the structure of the injected noise differs from iteration to iteration (based on the layer chosen) and depends on the training data in a different way. We expect NFM to amplify the benefits of training using either noise injection mixup alone, will be shown next.
4.2 IMPLICIT REGULARIZATION OF NFM
We consider loss functions of the form I([(z), - uf(x), which includes standard choices such the logistic loss and the cross-entropy loss, and recall that f := fx o gk- Denote L := 1 Ei=1 (z4), Yi and let D₂ be the empirical distribution of training samples {I, [ej)) We shall show that NFM exhibits natural form of implicit regularization, i.e., regularization imposed implicitly by the stochastic learning strategy, without explicitly modifying the loss.
A), aadd H coadd mmult malt, and denote V and V as the first and second directional derivative of fR with respect to 9t respectively, for S. By working in the small parameter regime, we can relate the NFM empirical loss L. to the original loss L and identify the regularizing effects of NFM.
Let > be small parameter. In the sequel, we rescale
+ c(1 (1)
Theorem 1. Let E > 0 be a small parameter, and assume that h and f are twice differentiable. Then, NFM(() where
LNFM Lk
L (K std + teR + + R Exple),
(3)
with (e(h) = (2 to 12 nadd(ik to? mult
12 Rrmule(k))
and R nk(k) + to nadd(()) to mutt R2
(mmtt(())
add
where
Radd(k)
R2
("(((a))VV Es (SR (
(4)
2n 1=1
R Rmuut(())
(((((z)) (Es)) (E<x jemuilt (c O (
2n
i=1
(5)
Rg
Rgdd((k)
(M(f(x:)) y.) (()) ( (1)(((
(6)
2n
R Rmmult())
(H((((x))) - -.) e.) @ g
@ (x,)))
2n
Here, R,, Rh and R4J are the regularizers associated with the loss of manifold mixup (see Theorem in SM for their explicit expression), and up is some function such that lim (p()) =
Theorem Timplies that, when compared to manifold mixup, NFM introduces additional smoothness, regularizing the directional derivatives, V&f (z;)) and (zx()))))) with respect to gs (x;). according the noise levels dadd and mmult, and amplifying the regularizing effects of manifold mixup and noise injection. In particular, making f2)(x)) small can lead to smooth decision boundaries (at the input level), while reducing the confidence of model predictions. On the other hand, making the V&J(9k) (x:)) small can lead to improvement in model robustness, which we discuss next.
4.3 ROBUSTNESS OF NFM
We show that NFM improves model robustness. We do this by considering the following three lenses: (1) implicit regularization and classification margin; (2) distributionally robust optimization; and (3) a probabilistic notion of robustness. We focus on (2) in the main paper. See Section C in SM and
the last paragraph in this subsection for details on (1) and (3).
We now demonstrate how NFM helps adversarial robustness. By extending the analysis of Zhang al. (201)); Lamb et al. (2019), we relate the NFM loss function to the one used for adversarial training, which can be viewed as an instance of distributionally robust optimization (DRO) Kwon
et al. 2020; Kuhn et al. 2019 Rahimian & Mehrotral 2019) (see also Proposition 3.1 in (Staib & Jegelkal (017)). DRO provides a framework for local worst-case risk minimization, minimizing supremum of the risk in ambiguity set, such as in the vicinity of the empirical data distribution.
5

---

Published as a conference paper at ICLR 2022
Following (Lamb et al. 2019), consider the binary cross-entropy loss, setting h(2) = log(1 + +.), with the labels y taking value in {0, 1 and the classifier model f R + R. In the following, we assume that the model parameter E θ == {0: fx,(x,, + (y: (x,) 2 0 for all E [n]). Note that this set contains the set of all parameters with correct classifications of training samples (before applying NFM), since {0: [[(=))200) y: for all E [n]] C O. Therefore, the condition of A E θ is satisfied when the model classifies all labels correctly for the training data before applying NFM. Since, in practice, the training error often becomes zero in finite time, we study the effect of NFM on model robustness in the regime of 0 E θ
Working the data- data-dependent parameter space O, we have the following result.
Theorem 2. Let 0 E Q == {0: yuf (x,) + (1)(()) 2 for all i [n]} such that V (gx(())))) and (g)()))) exist for all i E [n]. k E S. Assume that (1(g)(r))) = (aI(gn( gu (r;). Vi (1a))) = 0 for all E [n]] S. In addition, suppose that (V=)))l >> ffrrr all [n].
ErwD, (...).I((())) [r)] 0 and [x:)(>;))l2 2 Ca Vdr all i E [m]], k S. Then,
NNN n
Σ
max 1(f(x; + 0.), + L + (p((): 1
(8)
n
where <n
cF ANDA [1 {] FA ri Cx
(k) / Vite and T.
[V
1
E [h" [f'(((=)))(( (0y) with (k) = cos Vaf (z;)) g (z:)) and
2n
(c,00)3 := ( Ex[(1 A) A |l| ( )1 Cos(VV/(((((t))) (n(r))
oada Ee, Illsad | coss (xdd)
mmutt Ege Ille%
Illl
@ gk (1x) cos (V (xx), egalt s((a:))1))
and & is some function such that lom (cc) O.
The second assumption stated in Theorem 2 is similar to the made in Lamb et al. (2019) Zhang ct (2020), and is satisfied by linear models and deep neural networks with ReLU activation function and max-pooling. Theorem 2 shows that the NFM loss is approximately adversarial loss with l2 attack of size e mär = min ( ci plus a feature-dependent regularization term L (see SM for further discussions). Therefore, we see that minimizing the NFM loss not only results in a small adversarial loss, while retaining the robustness benefits of manifold mixup, but it also imposes additional smoothness, due noise injection, on the adversarial loss. The latter can help mitigate robust overfitting and improve test performance (Rice et al. 2020 Rebuffi et al. 2021).
upper bound of the
NFM can also implicitly increase the classification margin (see Section of SM). Moreover, since the main novelty of NFM lies in the introduction of noise injection, it would be insightful to isolate the robustness boosting benefits of injecting noise on top of manifold mixup. We demonstrate these advantages via the lens of probabilistic robustness in Section of of S.M.
5 EMPIRICAL RESULTS
In this section, we study the test performance of models trained with NFM, and examine to what extent NFM can improve robustness to input perturbations. We demonstrate the tradeoff' between predictive accuracy on clean and perturbed test sets. We consider input perturbations that are common in the literature: (a) white noise; (b) salt and pepper; and (c) adversarial perturbations (see Section F)
We evaluate the average performance of NFM with different model architectures CIFAR- 10 (Krizhevsky) 2009). CIFAR- 100 (Krizhevsky) 2009), ImageNet (Deng et al., 2009), and CIFAR- 10c (Hendrycks & Dietterich 2019) We a pre- activated residual network (ResNet) with depth 18 (He et al. (0166) on small scale tasks. For more challenging tasks, we consider the performance of wide ResNet- 18 Zagoruyko & Komodakis 2016) and ResNet-50 architectures, respectively.
Baselines. We evaluate against related data augmentation schemes that have shown performance improvements in recent years: mixup (Zhang al. 2017); manifold mixup (Verma et al., 2019);

---

Published as a conference paper at ICLR 2022
cutmix (Yun et al. 2019); Further, we compare to vanilla models trained without data augmentation (baseline), models trained with label smoothing, and those trained white noise perturbed imputs.
puzzle mixup (Kim et al. 2020b); and noisy mixup YYngg et al. 2020b).
Experimental details. All hyperparameters are consistent with those of the baseline model across the ablation experiments. In the models trained the different data augmentation schemes, keep a fixed, i.e. the parameter defining Beta(a, a), from which the parameter controlling the convex combination between data point pairs is sampled. Across all models trained with NFM, control the level of noise injections by fixing the additive noise level to aadd 0.4 and multiplicative noise mullt 0.2. To demonstrate the significant improvements on robustness upon the introduction of these small input perturbations, we show second model (***) that was injected with higher noise levels (i.e., add 1.0, mualt 0.5). See SM (Section F55) for further details and comparisons against NFM models trained on various other levels of noise injections.
5.1 CIFAR10
Pre-activated ResNet-18. Table I summarizes the performance improvements and indicates a consistent robustness across different values. The model trained with NFM outperforms the baseline model on the clean test set, while being more robust to input perturbations (Fig. 3 left). This advantage is also displayed in the models trained with mixup and manifold mixup, though in less pronounced way. Notably, the NFM model is also robust to salt and pepper perturbations and could be significantly more so by further increasing the noise levels (Fig. 3 right).
5.2 CIFAR-100
Wide ResNet-18, Previous work indicates that data augmentation has a positive effect on performance for this dataset (Zhang et al., 2017). Fig. F (left) confirms that mixup and manifold mixup improve the generalization performance on clean data and highlights the advantage of data augmentation. The NFM training scheme is also capable of further improving the generalization performance. In
95
90
Test Accuracy
Baseline
85
Baseline
Mixup
Mixup
CutMix
80
CutMix
PuzzleMi Mix
PuzzleMix
Noisy Mixup
Noisy Mixup
Manifold Mixup
Manifold Mixup
NFM
(*)
NFM
NFM
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.00
0.02
0.04
0.06
0.08
White Noise (a)
Salt and Pepper Noise (7)
Figure 3: Pre-actived ResNet-18 18 evaluated on CIFAR-10 with different training schemes. Shaded regions indicate one standard deviation about the mean. Averaged across 5 random seeds.
Table 1:
Robustness of ResNet- w.r.t. white noise (α) and salt and pepper (n) perturbations evaluated on CIFAR-10. The results are averaged over 5 models trained with different seed values.
Scheme
Clean (%)
(%)
7 (%)
0.02
0.04
0.1
Baseline
94.6
90.4
76.7
86.3
76.1
55.2
Baseline Noise
94.4
94.0
87.5
71.2
89.3
82.5
64.9
Baseline Label Smoothing
95.0
91.3
87.7
79.2
Mixup (ax = 1.0) Zhang et al. 2017
95.6
93.2
85.4
71.8
87.1
76.1
55.2
CutMix Yun et al. 22119
96.3
86.7
60.8
32.4
90.9
81.7
54.7
PuzzleMix Kim al, 2020h)
96.3
91.7
78.1
59.9
91.4
81.8
54.4
Manifold Mixup
L.0y Verma et al. 2019
95.7
92.7
82.7
67.6
88.9
80.2
57.6
Noisy Mixup (a = 1.0) Yang al. 20206
78.9
78.6
66.6
46.7
66.6
53.4
25.9
Noisy Feature Mixup (a = 1.0)
95.4
95,0
91.6
83.0
91.9
87.4
73.3

---

Published as a conference paper at ICLR 2022
80
Test Accuracy
70
70
Baseline
Mixup
60
CutMix
60
RuzziaMix
Noisy Mixup
50
Mixup
50
NFM (*)
40
40
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.00
0.02
0.04
0.06
0.08
White Noise (o)
Salt and Pepper Noise (7)
Figure 4: Wide ResNets evaluated CIFAR-100. Averaged across 5 random sceds.
Table 2: Robustness of Wide-ResNet-I 18 w.r.t. white noise (σ) and salt and pepper (7) perturbations
evaluated on CIFAR-100. The results are averaged over 5 models trained with different seed values.
Scheme
Clean (%)
a (%)
7 (%)
0.2
0.3
0.02
0.04
0.1
Baseline
76.9
64.6
42.0
23.5
58.1
39.8
15.1
Baseline Noise
76.1
75.2
60.5
37.6
64.9
51.3
23.0
Mixup (ax 1.0) Zhang al. 2017
80.3
72.5
54.0
33.4
62.5
43.8
16.2
CutMix Yun al. 2019
77.8
58.3
28.1
13.8
70.3
58.
24.8
PuzzleMix (200 epochs) Kim et al. 20200b
78.6
66.2
41.1
22.6
69.4
56.3
23.3
PuzzleMix (1200 epochs) Kim
2020b
80.3
53.0
19.1
6.2
69.3
51.9
15.7
Manifold Mixup (a 1.0) Verma el al. 2019
79.7
70.5
45.0
23.8
62.1
42.8
14.8
Noisy Mixup (a = 1.0) Yang ata 20206
78.9
78.6
66.6
46.7
66.6
53.4
25.9
Noisy Feature Mixup (a 1.0)
80.9
80.1
72.1
55.3
72.8
62.1
34.4
Table 3: Robustness of ResNet-50 w.r.t white noise (a) and salt and pepper (n) perturbations evaluated on ImageNet. Here, the NFM training scheme improves both the predictive accuracy on clean data and robustness with respect to data perturbations.
Scheme
Clean (%)
a (%)
Y (%)
0.25
0.06
0.1
0.15
Baseline
76.0
73.5
67.0
50.1
53.2
50.4
45.0
Manifold Mixup (a 0.2) Verma et al. 2019
76.7
74.9
70.3
57.5
58.1
54.6
49.5
Noisy Feature Mixup (a 0.2) Noisy Feature Mixup (a 1.0)
77.0
76.5
72.0
60.1
58.3
56.0
52.3
76.8
76.2
71.7
60.0
60.9
58.8
54.4
addition, that the model trained with NFM is less sensitive both white noise and salt and pepper perturbations. These results are surprising, robustness is often thought to be at odds with accuracy (Tsipras et al. 2018). However, we demonstrate NFM has the ability improve both accuracy and robustness. Table indicates that for the same a, NFM can achieve an average test accuracy of 80.9% compared to only 80.3% in the mixup setting.
5.3 IMAGENET
ResNet-50. Table similarly shows that NFM improves both the generalization and robustness capacities with respect to data perturbations. Although less pronounced in comparison to previous datasets, NFM shows favorable trade-off without requiring additional computational resources. Note that due computational costs, do not average across multiple seeds and only compare NEM to the baseline and manifold mixup models.
5.4
CIFAR-10C
In Figure we use the CIFAR- 10C dataset (Hendrycks & Dietterich 2019) to demonstrate that models trained with NFM more robust range of perturbations natural images. Fleure shows
8

---

Published as a conference paper at ICLR 2022
Test Accuracy
80
Baseline
60
Mixup
CutMix
MuxzicMix
Noisy Mixup
60
Mixup
NFM (*)
30
NFM
20
0.00
0.02
0.04
0.06
0.08
0.10
0.12
0.000
0.025
0.075 0.100 0.125 Adverserial Noise (c)
0.050
0.150
Adverserial Noise (c)
Figure 5: Pre-actived ResNet- 18 evaluated on CIFAR-10 (left) and Wide ResNet-18 evaluated on CIFAR-100 100 (right) with respect to adversarially perturbed inputs.
100
100
Baseline
CutMix M. Mixup
Mixup
PuzzleMix
90
NFM
NFM (*)
Test Accuracy
80
70
40
5
20
gaussian jpeg
impulse shot
speckle
Severities
Noise type
Figure 6: Pre-actived ResNet-18 evaluated on CIFAR-10c.
the average test accuracy across six selected perturbations and demonstrates the advantage NFM being particularly pronounced with the progression of severity levels. The right figure shows the performance on the same set of six perturbations for the median severity level NFM excels on Gaussian, impulse, speckle and shot noise, and is competitive with the rest on the snow perturbation.
5.5 ROBUSTNESS TO ADVERSARIAL EXAMPLES
So far we have only considered white noise and salt and pepper perturbations. We further consider adversarial perturbations. Here, we use projected gradient decent (Madry al. 2017) with 7 iterations and various levels to construct the adversarial perturbations. Fig. higghlights the improved resilience of ResNets trained with NFM adversarial input perturbations and shows this consistently on both CIFAR-10 (left) and CIFAR- 100 (right). Models trained with both mixup and manifold mixup do not show a substantially increased resilience to adversarial perturbations.
we compare NFM to models that are adversarially trained. There, we see that adversarially trained models are indeed more robust to adversarial attacks, while at the same time being less accurate on clean data. However, models trained with NFM show an advantage compared to adversarially trained models when faced with salt and pepper perturbations.
In Section F.6
CONCLUSION
introduce Noisy Feature Mixup, an effective data augmentation method that combines mixup and noise injection. We identify the implicit regularization effects of NFM, showing that the effects amplifications those of manifold mixup and noise injection. Moreover, we demonstrate the benefits of NFM in terms of superior model robustness, both theoretically and experimentally. Our work inspires a range of interesting future directions, including theoretical investigations of the trade-offs between accuracy and robustness for NFM and applications of NFM beyond computer vision tasks. Further, it will be interesting to study whether NFM may also lead to better model calibration by extending the analysis of Thulasiddaan al. (2019) Zhang al. (022T.

---

Published as a conference paper at ICLR 2022
CODE OF ETHICS
We acknowledge that we have read and commit to adhering to the ICLR Code of Ethics.
REPRODUCIBILITY
The codes that can be used to reproduce the empirical results, well as description of the data processing steps, presented in this paper are available as zip file in Supplementary Material at OpenReview.net. The codes also available httpps: //gp:::///tttttt com/ er whwsn/ //os/M/M For the theoretical results, all assumptions, proofs and the related discussions are provided SM.
ACKNOWLEDGMENTS
S. H. Lim would like to acknowledge the WINQ Fellowship and the Knut and Alice Wallenberg Foundation for providing support this work. N. B. Erichson and M. W. Mahoney would like to acknowledge IARPA (contract W91 IN1200000)35) NSF, and ONR for providing partial support of this work. Our conclusions do not necessarily reflect the position or the policy our sponsors, and no official endorsement should be inferred. We are also grateful for the generous support from Amazon AWS.
REFERENCES
Guozhong An. The effects of adding noise during backpropagation training a generalization performance. Neural Computation, 6(3):643-674 1996.
Henry S Baird. Document image defect models. In Structured Document Image Analysis, pp. 546-556. Springer, 1992.
Chris M Bishop. Training with noise is equivalent to Tikhonov regularization, Neural Computation, 7(1):108-116, 1995.
Olivier Bousquet, Stéphane Boucheron, and Gábor Lugosi. Introduction to statistical learning theory. In Summer school on machine learning, pp- 169-207. Springer, 2003.
Alexander Camuto, Matthew Willetts, Umut Şimşekli, Stephen Roberts, and Chris Holmes. Explicit regularisation in Gaussian noise injections. arXiv preprint arXiv:2007.07368, 2020.
Luigi Carratino, Moustapha Cissé, Rodolphe Jenatton, and Jean Jean-Philippe Vert. On mixup regulariza- tion. arXiv preprint 0etr20006000449) 2020.
Olivier Chapelle, Jason Weston, Léon Bottou, and Vladimir Vapnik. Vicinal risk minimization. Advances in Neural Information Processing Systems, pp. 416-422, 2001.
Shuxiao Chen, Edgar Dobriban, and Jane H Lee. A group-theoretic framework for data augmentation. Journal of Machine Learning Research, 21 2020.
Dan Claudiu Cireşan, Ueli Meier, Luca Maria Gambardella, and Jürgen Schmidhuber. Deep, big. simple neural nets for handwritten digit recognition. Neural Computation, 22(12):327-33220, 2010.
Nicolas Couellan. Probabilistic robustness estimates for feed-forward neural networks. Neural Networks, 142:138-147, 2021.
Tri Dao, Albert Gu, Alexander Ratner, Virginia Smith, Chris De Sa, and Christopher Ré. A kernel theory of modern data augmentation. In International Conference on Machine Learning, pp. 3528-1537, PMLR, 2019.
Dennis DeCoste and Bernhard Schölkopf. Training invariant support vector machines. Machine Learning, 46(1):161-190, 2002.
10

---

Published as a conference paper at ICLR 2022
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A aeage-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, Icee, 2009.
Luc Devroye, Abbas Mehrabian, and Tommy Reddad. The total variation distance between high- dimensional Gaussians, arXiv preprint arXiv: 1Xi::8810008692 2018.
Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci., (33-4)211-4077 10707 2014.
Gamaleldin F Elsayed, Dilip Krishnan, Hossein Mobahi, Kevin Regan, and Samy Bengio. Large margin deep networks for classification. arXiv preprint 183.5598,3 2018.
Logan Engstrom, Justin Gilmer, Gabriel Goh, Dan Hendryeks, Andrew Hyass, Aleksander Madry, Reiichiro Nakano, Preetum Nakkiran, Shibani Santurkar, Brandon Tran, Dimitris Tsipras, and Eric Wallace. A discussion of ''avvrsarial examples not bugs, they are features'. Distill, 2019.
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Alek- sander Madry. Adversarial robustness prior for learned representations. ArXiv preprint 190.009906 2020.
Alhussein Fawzi, Seyed- -ohhe-
and Pascal Frossard. Robustness of classifiers: from adversarial to random noise. arXiv preprint arXiv:1608. (8667, 2016.
Jean-Yven Franceschi, Alhussein Fawzi, and Omar Fawzi, Robustness of classifiers to uniform 1p and Gaussian noise. In International Conference on Artificial Intelligence and Statistics, pp- PMLR, 2018.
Alison L Gibbs and Francis Edward Su. On choosing and bounding probability metrics, International Statistical Review, 1093):419-435, 2002.
Chengyue Gong, Tongzheng Ren, Mao Ye, and Qiang Liu. Maxup: A simple way to improve generalization of neural network training. arXiv preprint arXiv:2002.09024, 2020.
Ian Goodfellow, Patrick McDaniel, and Nicolas Papernot. Making machine learning robust against adversarial inputs. Communications of the ACM, 61(7):56-66, 2018.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv: 1Xiv::412.66522 2014.
Kristjan Greenewald, Anming Gu, Mikhail Yurochkin, Justin Solomon, and Edward Chien. k-mixup regularization for deep learning via optimal transport. arXiv preprint 0tti::20060093333 2021.
Caglar Gulcehre, Marcin Moczulski, Misha Denil, and Yoshua Bengio. Noisy activation functions. In International Conference on Machine Learning, pp. 3059-3068. PMLR, 2016.
Ali Hassani, Steven Walton, Nikhil Shah, Abulikemu Abuduweili, Jiachen Li, and Humphrey Shi. Escaping the big data paradigm with compact transformers. arXiv preprint arXiv:2104.05704, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European Conference on Computer Vision, pp. 630-645. Springer, 2016.
Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. arXiv preprint arXiv: 170.vv1700.0880 08455 2017.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corrup- tions and perturbations. Proceedings of the International Conference on Learning Representations, 2019.
Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv: 1912.0278) 2019.
11

---

Published as a conference paper at ICLR 2022
Judy Hoffman, Daniel A Roberts, and Sho Yaida. Robust learning with Jacobian regularization. arXiv 2019.
preprint
Jang- Hyun Kim, Wonho Choo, and Hyun Oh Song. Puzzle mix: Exploiting saliency and local statistics for optimal mixup. In International Conference on Machine Learning, pp. 5275-5285. PMLR, 2020a.
Jang-Hyun Kim, Wonho Choo, and Hyun Oh Song. Puzzle mix: Exploiting saliency and local statistics for optimal mixup. In International Conference on Machine Learning, 2020b.
Masanari Kimura. Mixup training the complexity reduction. arXiv preprint arXiv:2006.0622) 2020.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical Report, 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks, Advances in Neural Information Processing Systems, 25:1097-1105, 2012.
Daniel Kuhn, Peyman Mohajerin Esfahani, Viet Anh Nguyen, and Soroosh Shafieczadeh- Wasserstein distributionally robust optimization: Theory and applications in machine learning. In Operations Research & Management Science in the Age of Analytics, pp. 130-166. INFORMS, 2019.
Alexey Kurakin, Ian Goodfellow, Samy Bengio, et al. Adversarial examples in the physical world, 2016.
Yongchan Kwon, Wonyoung Kim, Joong-Ho Won, and Myunghee Cho Paik. Principled learning method for Wasserstein distributionally robust optimization with local perturbations. In Interna- tional Conference on Machine Learning, pp. 5567-5576. PMLR, 2020.
Alex Lamb, Vikas Verma, Juho Kannala, and Yoshua Bengio. Interpolated adversarial training: Achieving robust neural networks without sacrificing too much accuracy. In Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security, pp. 95-103, 2019.
Mathias Lecuyer, Vaggelis Atlidakis. Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified robustness to adversarial examples with differential privacy. In 2019 IEEE Symposium on Security and Privacy (SP), pp. 656-672. IEEE, 2019.
Soon Hoe Lim, N Benjamin Erichson, Liam Hodgkinson, and Michael Mahoney. Noisy recurrent neural networks. arXiv preprint arXiv:2102.04877 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 1XXx:1106.00088) 2017.
M. W. Mahoney. Approximate computation and implicit regularization for very large-scale data analysis. In Proceedings of the 31 ACM Symposium Principles of Database Systems, pp. 143-154, 2012.
M. W. Mahoney and L. Orecchia. Implementing regularization implicitly via approximate eigenvector computation. In International Conference on Machine Learning, pp. 121-128, 2011.
Yifei Min, Lin Chen, and Amin Karbasi. The curious case of adversarially robust models: More data can help, double descend, or hurt generalization. arXiv preprint arXiv:2002. 1080, 2020.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Robust- ness via curvature regularization, and vice versa. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9078-9088 2019.
Roman Novak, Yasaman Bahri, Daniel A Abolafia, Jeffrey Pennington, and Jascha Sohl- Dickstein. Sensitivity and generalization in neural networks: an empirical study. arXiv preprint arXiv: 180. 2018.
12

---

Published as a conference paper at ICLR 2022
Sayak Paul and Yu Chen. arXiv:210S.0758, 2021.
Vision transformers are robust learners.
arXiv preprint
Rafael Pinot, Laurent Meunier, Alexandre Araujo, Hisashi Kashima, Florian Yger, Cédric Gouy- Pailler, and Jamal Atif. Theoretical evidence for adversarial robustness through randomization. arXiv preprint arXiv:1902.011 148, 2019a.
Rafael Pinot, Florian Yger, Cédric Gouy-Pailler, and Jamal Atif. A unified view on differential privacy and robustness to adversarial examples. arXiv preprint arXiv:1906.07985) 2019b.
Rafael Pinot, Laurent Meunier, Florian Yger, Cédric Gouy-Pailler, Yann Chevaleyre, and Jamal Atif. On the robustness of randomized classifiers to adversarial examples. arXiv preprint arXiv:2102.10875, 2021.
Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John Duchi, and Percy Liang, Understanding and mitigating the tradeoff between robustness and accuracy. arXiv preprint arXiv:2002. 107 16, 2020.
Hamed Rahimian and Sanjay Mehrotra. Distributionally robust optimization: A review. arXiv preprint 0rXiv:/908.056333 05659, 2019.
Sylvestre-Alviss Rebuffi, Sven Gowal, Dan A Calian, Florian Stimberg, Olivia Wiles, and Tim- othy Mann. Fixing data augmentation to improve adversarial robustness. arXiv preprint arXiv:2103.01946, 2021.
Leslie Rice, Eric Wong, and Zico Kolter. Overfitting in adversarially robust deep learning. In International Conference on Machine Learning, pp. 8093-8104. PMLR, 2020.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adver- sarially robust generalization requires more data. arXiv preprint arXiv: 1/285, 2018.
Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pin- Yu Chen, and Cho-Jui Hsieh. On the adversarial robustness of visual transformers. arXiv preprint arXiv:2103./500 2021.
Connor Shorten and Taghi M Khoshgoftaar. A survey image data augmentation for deep learning. Journal of Big Data, 6(1):1-48, 2019.
Jure Sokolić, Raja Giryes, Guillermo Sapiro, and Miguel RD Rodrigues. Robust large margin deep neural networks. IEEE Transactions on Signal Processing, (5(16))4265-4280. 2017.
Matthew Staib and Stefanie Jegelka. Distributionally robust deep learning as generalization of adversarial training. In NIPS workshop on Machine Learning and Computer Security, volume 3, pp. 4, 2017.
Dong Su, Huan Zhang, Hongge Chen, Jinfeng Yi, Pin- Yu Chen, and Yupeng Gao. Is robustness the cost of accuracy? comprehensive study on the robustness of 18 deep image classification models. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 631-648, 2018.
Sunil Thulasidasan, Gopinath Chennupati, Jeff Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. arXiv preprint arXiv: 1100,, 2019.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry, Robustness may be at odds with accuracy. arXiv preprint arXiv:1803 12152, 2018.
Francisco Utrera, Evan Kravitz, N Benjamin Erichson, Rajiv Khanna, and Michael W Mahoney, Adversarially-trained deep nets transfer better. arXiv preprint arXiv:2007. 008800077.80,,,... 2020.
Vladimir Vapnik. The Nature of Statistical Learning Theory. Springer Science & Business Media, 2013.
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez-Paz, and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In International Conference Machine Learning, pp. 6438-6447. PMLR, 2019.
13

---

Published as a conference paper at ICLR 2022
Colin Wei and Tengyu Ma. Data-dependent sample complexity of deep neural networks via Lipschitz augmentation. arXiv preprint arXiv:1905.000 2019a.
Colin Wei and Tengyu Ma. Improved sample complexities for deep networks and robust classification via an all-layer margin. arXiv preprint arXiv: (9:::::90000000 044244, 2019b.
Sen Wu, Hongyang Zhang, Gregory Valiant, and Christopher Ré. On the generalization effects of linear transformations in data augmentation. In International Conference on Machine Learning. pp. 10410-10420. PMLR, 2020.
Yao- Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Ruslan Salakhutdinov. and Kamalika Chaud- huri. A closer look at accuracy vs. robustness. arXiv preprint arXiv:2003.02466 2020a.
Yaoqing Yang, Rajiv Khanna, Yaodong Yu, Amir Gholami, Kurt Keutzer, Joseph Gonzalez, Kannan Ramchandran, and Michael W Mahoney. Boundary thickness and robustness in learning models. In H. Larochelle, M. Ranzato, Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 6223-6234, 2020b.
Wenpeng Yin, Huan Wang, Jin Qu, and Caiming Xiong. BatchMixup: Improving training by interpo- lating hidden states of the entire mini-batch. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 4900-4912, 2021.
Sangdoo Yun, Dongyoon Han, Scong Joon Oh. Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In International Conference Computer Vision, pp. 6023-6032, 2019.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.077 1000000071166 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference Machine Learning, pp. 7472-7482. PMLR, 2019.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. Mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412, 2017.
Linjun Zhang, Zhun Deng, Kenji Kawaguchi, Amirata Ghorbani, and James Zou. How does mixup help with robustness and generalization? arXiv preprint arXiv:2010.048/9 2020.
Linjun Zhang, Zhun Deng, Kenji Kawaguchi, and James Zou. When and how mixup improves calibration. arXiv preprint arXiv:2102.06289) 2021.
14

---

Published as a conference paper at ICLR 2022
Supplementary Material (SM) for "Noisy Feature Mixup"
Organizational Details. This SM is organized as follows.
In Section A
we study the regularizing effects of NFM within the vicinal risk minimization
framework, relating the effects to those of mixup and noise injection.
In Section we restate the results presented in the main paper and provide their proof.
In Section C
we study robutsness of NFM through the lens of implicit regularization,
showing that NFM can implicitly increase the classification margin.
In Section D
we study robustness of NFM via the lens of probabilistic robustness, showing that noise injection can improve robustness top of manifold mixup while keeping track maximal loss accuracy incurred under attack by tuning the noise levels.
In Section E
provide results on generalization bounds for NFM and their proofs, identifying the mechanisms by which NFM can lead to improved generalization bound.
In Section F
we provide additional experimental results and their details.
We recall the notation that we use in the main paper well as this SM.
Notation. / denotes identity matrix, [K] := {I...., J. the superscript denotes transposition, denotes composition, C denotes Hadamard product, 1 denotes the vector with all components equal one. For a vector v, e denotes its kth component and ||인|] denotes its Lp norm for > O. conv denote the convex hull of X. MA b) = λa + (1 A)b, for random variables a, b, X. ö₂ denotes the Dirac delta function, defined as (x) = if x = and 6_(x) = 0 otherwise. 1 denotes indicator function of the set A. For a. B > 0, Dx := a+ Beta(a + 1, + a+8 Beta B + a)., uniform mixture of two Beta distributions. For two vectors 6, cos b) == (a, ////||||/||b|| denotes their cosine similarity. N(a, b) denotes the Gaussian distribution with mean a and covariance b.
A NFM THROUGH THE LENS OF VICINAL RISK MINIMIZATION
In this section, we shall show that NFM can be constructed within vicinal risk minimization (VRM) framework at the level of both input and hidden layer representations.
To begin with, we define a class of vicinal distributions and then relate NFM to such distributions.
Definition 1 (Randomly perturbed feature distribution). Let 2n = (ZI...., Zn} } be a feature set. We say that Pt, is an e--randomly perturbed feature distribution if there exists a set {z(,...,, 2/, such that P', = >x Zim1 52c, with z + 6, for some random variable e; (possibly dependent on 2,,) drawn from a probability distribution.
Note that the support of an e--randomly perturbed feature distribution may be larger than that of Z.
If 2,, is an input dataset and the e; are bounded variables such that ||c.|| S for some 3 2 then P is a 8-locally perturbed data distribution according to Definition 2 in (Kwon et al. 2020). Examples of B-locally perturbed data distribution include that associated with denoising autoencoder, input mixup, and adversarial training (see Example 1-3 in (K Kwon et al.) 2020). Definition [ be viewed as an extension of the definition in (Kwon et al., 2020 relaxing the boundedness condition on the e, to cover a wide families of perturbed feature distribution. One simple example is the Gaussian distribution, i.e., when e; ~ (0, of), which models Gaussian noise injection into the features. Another example is the distribution associated with NFM, which now discuss.
To keep the randomly perturbed distribution close to the original distribution, the amplitude of the perturbation should be small. In the sequel, we let > be small parameter and rescale 1 - A + <1 - X), Gadd + edadd and mmult + 60 mmtut
Let F& be the family of mappings from (x) y and consider the VRM: E (o (=)) P ((f((o(())))) v/)],
inf Ru(ff))
(10)
FEFF
where (P())
Σ (1s) with (x:) = (xx) + ce NFM(K) and s/ ys + cej for some FF) and o/.
random variables eg
15

---

Published as a conference paper at ICLR 2022
In NFM, we approximate the ground-truth distribution DD using the family of distributions (P(()) JRES. with particular choice of (c; (F) ). In the sequel, we denote NFM at the level of kth layer as NFM (k) (i.e., the particular case when S == {k)).
The following lemma identifies the (eg (K) ,e," associated with (k) and relates the effects of (FM (R) to those of mixup and noise injection, for any perturbation level > 0.
Lemma 1. Let O and denote zı(k)= = go(zi). Learning the neural network map f using (k) is a VRM with the (ee; FF)(() eEe" -randomyy perturbed feature distribution, p(k)
NFM(() y. == y: + =ee, the vicinal distribution. Here, R Di=ı ((((()))0)) with 24(k) == 2, (k) tee,
= (1 ()(3i V.).
NFM = (1 + mmmlttSmutt) 2 e
mizup(k) + e,
moise(k)
(11)
(k) = -1 -1-1MM (k) z,(k)) and e, noise(k) mmultEmatt @ (k(k) + OaddEadd with where e, (k),, (k) E (x)),, Beta (x, 3) and yes Di E y. Here, (2, Us) are drawn randomly from the training
Therefore, the random perturbation associated NFM is data-dependent, and it consists of randomly weighted sum of that from injecting noise into the feature and that from mixing pairs of feature samples. As simple example, one take Eadd: Smult Emudt be independent standard noise(k) NO, o² pars! + omuudiigg (2,
Gaussian random variables, which case we have E; and e; /O0, add + mum (2, (k)))) in Lemma
We now prove Lemma]
Proof of Lemma 7
Let be given and set 1 without loss of generality. For every E [n]. N M injects noise on top of mixed sample 2<(kk) and outputs:
z²' = (1 + mmuttemutt)
@ 2k) + aaddLadd
(12)
(1 + mulleEmute
@ (A2, (A) + A)z;(k)) + Tadahodd
(13)
= (k) + e,
NFM(()
(14)
(FM where 6
(1 (k) 23 (k:) mmultemutt @ (Az:(k) + +11 -x);(()))) + aaddEadd-
Now, note that applying mixup to the pair (k), (k)) results in z{(k) = 2;(k) + ei mizup(() with mirup(k) (1 (k) (k)), where 2.)k), (k) E 9b (X) and À Beta 3), whereas Ci
applying noise injection to 2, (k) results in +1+0 m1+umut(maat) @ (k) + aaddSadd (k) + 6, moise(k) NFM( in terms of mixup(k) mmttt 3 (k) + GnddEodd- Rewriting 6;
with e, nkise(k)
e and
(kise(k) e,
gives
NFM(K) (1 + mmultEmalt) @ 6,
mixup((k) + e
(oise(k))
(15)
Similarly, we can derive the expression for ex using the same argument. The results in the lemma follow upon applying the rescaling + c(1 A), Fadd r coadd and amult r> 60 for <> 0.
B
STATEMENTS AND PROOF OF THE RESULTS IN THE MAIN PAPER
B.1 COMPLETE STATEMENT OF THEOREM 1 IN THE MAIN PAPER AND THE PROOF
We first state the complete statement of Theorem I in the main paper.
Theorem (Theorem in the main paper). Let E > 0 be a small parameter, and assume that h and
I are twice differentiable. Then, L FFM E NFM((K) where
T NFM(K) atd + +eR( + 2 (k) 2 ((hh) p)(())
(16)
16

---

Published as a conference paper at ICLR 2022
with
(bhh) R2g o aadaltyudg
+ mule R2 Remaalt()))
(17)
R(gh) R2) oadully (edd(h)
o B2 (mmulei))
(18)
where
F EADD
(1 21
12(h))
Σ " i=1
gx()))]
(19)
11
E ENDD [11 - -
2n
EN (00(((((xE)))) -
(20)
E ENWD [11 - 1]
(1)
(M(f(z)) vi)
2n
E DD (gx((1)())))) (1)( (x,)) ( (x,)))))
(21)
R2
Radd(())
h² {"(f(==))xxf(o((x)) 15g (C0 (( (1)
(22)
2n
R2
Ruatt(k))
["(((=)))A/(oo((t) (EEs, (sk (Sx o tn (x(m)))(())) Va
2n
==1
(23)
Rg
Radd(k))
- us (x) (
(24)
2n
pmuul((k)
Σ (h((f(x))) y.) [ [[grult @ (V
(x;))), (25)
2n
and pp() = -ENDD FANND (em_e(p(()). with some function such that lim (p()) = 0.
Following the setup of Zhang et al. 2020), we provide empirical results to show that the second order Taylor approximation for the NFM loss function is generally accurate (see Figure 7p.
Recall from the main paper that the NFM loss function be minimized I.NFM E I where
FF)
I
(PM(K)
1
n n
E Ra Ma(ye. v,)), (26)
n
==
25
with NFM with approximated loss
with NFM
with approximated loss
training loss
20-
testing loss
0.6
1.5
0.4
1.0
0.3
05
0.1
100
20
100
epoch
epoch
Figure 7: Comparison of the original NFM loss with the approximate loss function during training and testing for layer ReLU neural network trained on the toy dataset of Subsection F2
17

---

Published as a conference paper at ICLR 2022
where 1 RK R - [0, oo) is loss function of the form l(f(x),)) y) = h(f(z)) yf(z),
Ek == (Sg (x are drawn from some probability distribution @ with finite first two moments (with zero mean), and
My.e, (x) == (1 + Smult Smalt) C (x)
JadaSk Sodd
(27)
Before proving Theorem we note that, following the argument of the proof of Lemma 3.1 in Zhang al. (022), the loss function minimized by NFM can be written as follows. For completeness, we provide all details of the proof.
Lemma 2. The NFM loss (6) can be equivalently written as LNFM = Enn LFMML wwere
a
In
Σ
F AND IF Ez ND F. ( +ee ( ys Ia +ee, (FF(())))
}}
(28)
with
NFM(K) = (1 m0 S @ es 6
mixup(k) noise(k)) = multsk multehhule @ Da (2,) + aadash sadd; with
+ +, +e
moise(k)
(29)
Here 6;
mixupp())
(gx(()))) (g (x(r))) and eg
=
go (z,),, (x,,) E (x) and X Beta(a, 8).
Proof of Lemma
From 26 have:
F Fx Mollaa/eee.eelllaMaaaeao.o.l... (x))) Ma (.t,,,,)) (30)
We rewrite:
ExxBetala,s)
(M.. ((
ExwBotata.bt [M (4) Mi (ox()))
(MA (31)
=ExBaeaaa [A( (x -o
+(1 (M (ttp:://hhaaalo.o////.c.occcccaa y: (Ma (1 (xz)))
(32)
= -Ex~Beta(a,g) Be
(M +11 (B( (gt (x)))) - wy (ox (2re, 9h (2, )))))
(33)
where (ern(A) denotes the Bernoulli distribution with parameter À (i.e. PIB
A and
P[B = 0] = 1 - A).
Note that À Beta(a, B) and BIλ Bern(A). By conjugacy, we can switch their order:
B ~ Bern a AlB ~ Bela(a + B, + B),
(34)
a
and arrive
Ex 1 [[a (n Mx,,,
E E -it))
WiSe BL (MN))
+ (1 a
(35)
a
8 a a (b [at fn
(36)
Using the facts that Beta(B 1, 1, and Beta(a, 8 + are of the same distribution and Mi-x(1, = Ma(ay, 2.), we have:
E (u
Σ 1 (. (M (37)
ig
18

---

Published as a conference paper at ICLR 2022
Therefore, denoting Dx 3 Beta(a + 1, + 2 Beta + 1,) and D3 2,2 Σÿ=ı dzz the empirical distribution induced by the training samples {az}senn) we have:
L (FM()) Σ E ANDA EI.ND. Es (Ml.aaanllliffMMMaaaaaaaaaa.a.a
(MA. ok (x,))))
(38)
The statement of the lemma follows upon substituting the fact that Mx.e. (2x) gz(x.) NFM(R) into the above equation.
ec
With this lemma in hand, now prove Theorem
Proof of Theorem Denote (e(()) = h(fiela (24) ee; (F (eff(9x(z;) + ee, ee, (
(RM is given in 29h. Since h and f& are twice differentiable by assumption, th is twice where 6; differentiable in 6, and
6² (be(e) = (0) + (0) + (0) + P((c),
(39)
where Po is some function such that lim (p((c) 0. Therefore, by Lemma E NFF ((K(K) where
LN
n
I NFM(()
Σ
E EAWD. EI,ND, ((x))(
(40)
Σ EANDs EXIND, Esane 1=1
21(00++ +
2 (0 + p
(41)
Σ
E AwDD EuGNDD Ecane
+ 2 (0 ep(()
(42)
==1
Ladd + cR(()) 2 (R( R(hh)) 2 (c)),
(43)
where pcc) = Ei= E EZ,ND, [cr~olo((e))
It remains to compute (0) and (0) in order to arrive at the expression for the 12(a), R((h) and
R(() presented in Theorem
Denoting Iu (x:) := (x,) tee (K))) we compute, applying chain rule:
(E((E) ('(fAcoo(22)) (gx(2:)
(xf(((()z)) (gx(2;)
(44)
de
de
(x))f 9) (z. (T. dE = (W((A(G - 9: (x)e(G (x:)) NFM((A)
(45)
(46)
(x))) V Va (T
(I (1-)) 92 (x(x))) + GaddSh cadd
+ +mmutth emutt
3 gk (x+) e(1 A)omatt grault o (gh(x=) gx(x:))),
(47)
where we have used (g.()))
(F) in the second last line and substituted the expression for c
de
(FM() from (9) in the last line above. C,
Therefore.
(b(()) = (h' (h'(Fa(u(()))) - (e) - (x) gx(xx)) + Jada mmutth emalt (x.)),
cadd
(48)
19

---

Published as a conference paper at ICLR 2022
and
E (f(()((xx))))) (.) (xx) - -g((x))],
(49)
where we have used the assumptions that Fg cadd and Fe o5
emult = 0. The expression for the ( in the theorem then follows from substituting (9) into 42
Next, using chain rule, we have:
0 (I((I((((())))) -.VV de
Ogn (xxe) de
(e) =
(50)
(gx(2:))
-w)) - (x::))
dc
dc
0 + (M:)a((((x))) vi) ge (V.Ji(((z.)))" Ogx
(51)
86
Note that, applying chain rule,
0 e NFM())
(52)
de
de
dc
0 (Ie VVA
(53)
de
( (gk(xx)
(54)
de
(E; (eNTMNO (xx NFM((K
NFM
(55)
Also, using chain rule again,
O de
2 (M))(( wi) -v)) =N((G(())))))+()C))) (g.().)
(56)
de
-N NFM(()
(57)
Therefore, we have:
(c) = N"(AAA(( NPM (c: NFM((K) T ( (+
+ (I(()((()))) - NFM( (x:))) NFM(H)
(58)
= (
- (ar - gx(x:)) + anda eadd
+ Tmnitsk Emmt
emult 3 (xx) c( A)omuae craute o (gn(xr) gx(x)))]
[(1 -)) gb (x:)) + aadd Tmuleeh Smulle gh (xx)
+ +c(1- -A)omatts emall @ (gx (x+) (x:))) Vx.e))))
+ (Ox - 11)((1 (xx gx(x=)) + GadaEa cadd + omult @ gk (x:)
+ <1 A)omate
smult @ (0x(xx) (x:))) Vi ( - ()x gle (x:))
+ Sada + muulthh grail o (xx)) c(1 A)o (gx(xr) (xx))) (59)
(M(((( Pz(c) (60)
NFM into the first line to arrive at the last line
where we have substituted the expression for the e above.
20

---

Published as a conference paper at ICLR 2022
Note that,
-Eei~ll( - (x;)))) + Jadask Cad + mmultSk Shantt 2 0g (2xe)
+e(1 AlomultSh Stantt (x+) - 9u (x:))) x 1(1 () - gb (a:4) + JaddSk Sadd
+ Smmttth Smatt @ gk (xi) + t(1 - muttt gmant o (gx - gl((2;))]]
(61)
(1 (xx((()))))))))))))))))))))))))) - (ax) (2++ - IN (x;))² + oaddFEn-el5R 15x (S (5]
の mait E e e o gx (1x (x())))² +o())
(62)
(1 (g - (x:) (2,) - 94 (x;))² o oadaFeu-ella (odeedddddccddd (47 o 9u (x:))) (x;) o(c),
(Sk (Sadd)T)
+ mmtte E
el(5x [(5 (S
(63)
O, where have used the assumption that S and F Emult 0 in the second last line above.
Similarly,
TEiooP(())
=Egi~e(11- -))()
aaddER + maatth Emmule @ (x(x))
+c(1 - -AjomuttSk Smmult
emalt O (ox 9u V2 - -
+ FaddSh sadd + muulthk
emult o (x()), + c(1 m
emult @ (g)(xx)) gx(2)))
(64)
-1- 1-2)
gh (x.)) ( -xx(z;)
f ( (
+ miatt Egen elsEk
mmmlt @ (x;)))
o [x(x)))]++o
(65)
as 6 0. O.
Now, recall from Eq.
42 that we have
I. (PM))
1
Σ
E Dr FR, (i(00) + evi( + 2² v5(00)
+ +e(((e)
(66)
2 EP(( + ( + R(()) + 2 p())
i=1
=: T
(67)
where (²) = -((fx(((z()) (gfa(g((x;)) Also, we have:
(cu~el(((ee]
= -WKKGaalllKffdddllaEEERRAAAAAAC.
+
(68)
= (aap:ac)))|(ffiiiii//ooo///aaaad -(t)))
+ F cadd (S (1.)
F (orault (( (xx=
+ (Nx))(( - -
(xx)))
o² 2 E [emalt (x:)) ( (xx))) + o()))
(69)
21

---

Published as a conference paper at ICLR 2022
Therefore, setting O,
[EL-OlV((0) -² (i -1) (xi)) 9u (I:)))
+ Pe (cdd + mueel Em~ (Er (S (smmult) (r) (x;)))
x (ox)))
+ (W' (Si (mn (2:))) y:) hatp:///ha/llllaa///o//////////r gx (x:))
+ add 2 (( V (sx (2
+ 0 mitt F e((
(gmultt @ ge(x))) ( (1sa( @ [x()))]]]
(70)
The expression for the k and ((k) in the theorem follows upon substituting 77) into (66)
B.2 THEBRRM IN THE MAIN PAPER AND THE PROOF
We first restate Theorem in the main paper and then provide the proof. Recall that we consider the binary cross cross-entropy loss, setting h(2) = log(1 + e*), with the labels y taking value in {0, 1) and the classifier model f R2d - R.
Theorem 4 (Theorem in the main paper). Ler E + (0: yef(a, + (y,-1)((z; 2 for all i E [n]} be a point such that Val (x:)) and (gl( exist for all i i [n], k E E. Assume that (x:))()))) = V&J (x:)) ga (2i), (Ix()()))) = 0 for all [n], k S. In addition, suppose that | [x:))l2 >> 0 foror all E [n]. Er-D, Jon (r)) = 0 and 11 (x:)ll2 2 ea yds for all E [r]]
E Then,
1
LN
NFM
max (f(x;; + 6.), + L +
(71)
Se
where
Vuf(gH(z:)Il dd
mix
eE CENDD, [1- A1
EsNO ri (k)
(72)
[V((x))l2
(k)
|osss
V (esV( ((n())())))))))))))))))))))()))) (x()))
(73)
I.reg 1
((((
(74)
2n
i=1
with
(6/00)2 e² V
Ex[(1 2)) (ot
+ ad FellSSadell cos( ( (x;)) Eada)
+ mumt i EglllEmutte C 9 (x;) 3 cos V& Emalt O (x(())))))
(75)
and & is some function such that lim <(c) = 0.
Theorem says that LNFM is approximately an upper bound of sum of an adversarial loss with by- attack of size emiz min; emiir and a feature-depender regularizer with the strength of min,(c,'3) Therefore, minimizing the NFM loss would result in small regularized adversarial loss. We note that both emsiz and rey depend on the cosine similarities between the directional derivatives and the features which the derivatives are evaluated at, whereas the < additionally depend on the cosine similarities between the directional derivatives and the injected noise.
Before proving Theorem we remark that the assumption that fa (gx( = (x.))) gu (z.).
())(()))) = 0 for all E [m.] & E S is satisfied by fully connected neural networks with ReLU activation function or max-pooling For a proof this, we refer to Section B.2 in Zhang et al. (2020). The assumption that E (r)) 0 could be relaxed the cost of obtaining more complicated formula (see Remark for the formula) for the e in the bound, which could be derived straightforward manner.
22

---

Published as a conference paper at ICLR 2022
Proof of Theorem 1 For h(2) = log(1 + e²), have (2(2) + S(2) 2 and h" (2)
(I+e (1=) 2 0. Substituting these expressions into the equation of Theorem and
using the assumptions that fa (x:)) (x)(( (x,) and E,wD, (r)) O, we have, for k E S,
ExTo [1 11 A1 Σ (u) S(f(2))))
(
R1
(76)
n1
and we compute:
E EA 111 - 1)
R
Σ
2n
x Ex, (E, -a
(77)
E Ex~ 111 2)12
2n
x Eg, DD2,1(9((1, -aaaaa/alNNNoo------))))////////
(78)
F SuB [11 - 2)12
2n
X
(79)
E ENd [11 [11 2 n
2n
EXNDA 111 2
Σ (10e: (zx
2n
(80)
E ENWD [11 - [1])
(((())))( - /
2n
1 "
x (Cos(Va/((n(ti))))))(())
2n
x (x((1-))) |a
(81)
1
a
355C0)11-5((/)))1///////0001)))) ((1-x)]]ddarrr)) )
2n
B5tt:))1--S///))//////aa//) LEN(1 - E)) (z+)
2n
cos
(82)
1
2n
(=.()x (r (.,]()))))))) (0))))
(EWD, [(1 2)
[x:)(=))11
(0/00)11---//)))/////////// -E((1 - -x((11)1(1)))))
2n
cos V ( 9u (x)())))))) (2.
(83)
In the above, we have used the facts that E [ = E[2] + Var(Z) 2 E[2] and S, - 5) 2 to
obtain (78), the assumption that F,wD, (r) = 0 to arrive at (79 the assumption that [g (x; )l2 2
(k) ydz for all é E [n], &o E to arrive at 82 and the assumption that [[V > for all
Cz
i [n] to justify the last equation above.
23

---

Published as a conference paper at ICLR 2022
Next, bound [[])) using the assumption that E O. O. Note that from our assumption 0,
have yr (x:) + (y/ 1x)(2)) 2 0, which implies that [(z;) 2 0 if y: = 1 and (xx I
3: 0. Thus, yt 1, then (y: fa
2 since fx;) 2 0 and (1 S(f(x.)) 2 0 due to the fact that (zf():) E (0, 1). A similar argument leads
= (1 ((
to (y: ((f(x:)))f((on
20 y/ O. So, we have (yi (f (x.) for all
i [n].
Therefore, noting that ENDD X] 2 0, we compute:
E ENWD [1- - A1 n
R2(4)
Σ
1/ S(((=)))]I((((m(x))
(84)
ANDD [1 21
ya (a cos( V (xx))
(85)
21 -
(wDD - A)r Vda)
(86)
( - (FA-Da(1 ->] [x,(gx( c
[V:((1)) (1) vd
(87)
Note that R/ = 0 as consequence of our assumption that (zx((())))) = 0 for all i [n], k e S,
and similar argument leads
R2 Radd(k)
(t((a)((--////))))/////kkk (sadd BooldT/iVvan V (88)
2n
2n
x Es, Ills
cos (z)) sedd))
(89)
R
pmmul(k))
1
(Eealeh (egdd (eaaddT] gu (z))g)((Z=)))
2n
x vaf(gn(E)))
2n
x Eg, IIlemalttt
(x:)))) ( Emale O (x.)))))
(90)
24

---

Published as a conference paper at ICLR 2022
Using Theorem 3 and the above results, obtain:
L NNM
((f(z)).()
2 E [
(K)
+ + 2R² (h) + 2 + +e n + ((t()
(91)
[S(((x))) ys 3111V((3)
(92)
+ 2n
1
(Sf/ ExI(1
2n
COS
(93)
1
+
-
( moisey2 2p(c),
(94)
2n
where ennix = clE ANDA [1 - A|E
(e (k) co Vd and [Vf(2,)ll
( = [ ae IllER add 12 cos(( (V))( )
+ o o Ee Ille (2: co ( (z))))) smalt o su(a)))1))
(95)
On the other hand, for any small parameters e, > and any inputs .,...., 2n, we can, using second-order Taylor expansion and then applying our assumptions, compute:
Σ (((()))))))
max [([(=: + 8.). 1
1
s(f(2)) - will [[( +
-
2n 1=1
max
(
(96)
n
|
S(((2))) ( 1
- (((()))))-
n
2n
1
(c),
(97)
where the vp are functions such that (((=)) 0, vl"(es) = max ((((,) and lim (p((2) 0.
Combining 94 and 97 we see that
1
J.N 1.
[(f(a; + 5mmir), ys) + 1.reg + (
(c) mix (c!
max
1=1
(98)
max 88 Semio
1[f(x; + (.mmin). ).y:) + Lieg + (c)(),
(99)
where [reg is defined in the theorem. Noting that lim (be) 0, the proof is done.
n
25

---

Published as a conference paper at ICLR 2022
Remark 1. Had assumed that E [gx(r)] + 0, then the statements of Theorem unchanged, but with (c,-9)² replaced by
remain
(c,09)² = Vl
[0[1--]] [1l0
+ ad Fe ElllEadd 11 cos (Vi. (V Enda)
+ mure E EelllSmute 3 gu gx; CV (V Smult O gu (3((z)))²)
- - (/nal (n
(x:)). (100)
C NFM THROUGH THE LENS OF IMPLICIT REGULARIZATION AND CLASSIFICATION MARGIN
First, we define classification margin at the input level. We shall show that minimizing the NFM loss can lead to an increase in the classification margin, and therefore improve model robustness in this sense.
Definition 2 (Classification Margin). The classification margin of training input-label sample 8, = (I'z Q;) measured by the Euclidean metric d is defined as the radius of the largest d-metric ball in X' centered x4 that is contained in the decision region associated with the class label Gs i.e., Vz}.
is: (x:) = sup x) E g(x)
Intuitively, larger classification margin allows classifier to associate larger region centered on point v; the input space to the same class. This makes the classifier less sensitive input perturbations, and perturbation of is still likely to fall within this region, keeping the classifier prediction. In this sense, the classifier becomes more robust. the typical case, the networks are trained by a loss (cross-entropy) that promotes separation of different classes in the network output. This, in turn, maximizes a certain notion of score of each training sample (Sokolić et al.) 2017). Definition 3 (Score), For an input-label training sample s; = (z), we define its score as o(s:) minjæe, V2(ee, e fxx;) 2 0, where E RR is the Kronecker delta vector (one-hot vector) with eç and eq 0 for j.
positive score implies that at the network output, classes are separated by margin that corresponds to the score. A large score may not imply large classification margin, but score can be related to classification margin via the following bound.
Proposition 1. Assume that the score o(s:) > 0 and let k E S. Then, the classification margin the training sample s; can be lower bounded as:
C(ss)
7 (8:)
(101)
sup (2) [V)(((()))]1
where C(s;)) = o(s:)/ sup supzeconu(X)
| gh (x)ll.
Since NFM implicitly reduces the feature-output Jacobians V (including the input-output Jacobian) according to the mixup level and noise levels (see Proposition 3). this, together with Theorem [ suggests that applying NFM implicitly increases the classification margin, thereby making the model more robust to input perturbations. We note that similar, albeit more involved, bound can also be obtained for the all-layer margin, more refined version of classification margin introduced in (Wei & Ma 2019b), and the conclusion that applying NFM implicitly increases the margin also holds.
We now prove the proposition.
Proof of Proposition Note that, for any S, (f(x) (v)()(()))V by the chain rule,
and so
\ < (
(102)
sup =Econe(2)
[va((
sup (Ecomm(X))
(103)
26

---

Published as a conference paper at ICLR 2022
The statement in the proposition follows from straightforward application of Theorem 4 in Sookoii et al. 2017) together with the above bound.
D NFM THROUGH THE LENS OF PROBABILISTIC ROBUSTNESS
Since the main novelty NFM lies the introduction of noise injection, it would be insightful to isolate the robustness boosting benefits of injecting noise on top of manifold mixup. We shall demonstrate the isolated benefit in this section.
The key idea is based on the observation that manifold mixup produces minibatch outputs that lic in the convex hull of the feature space at each iteration. Therefore, for & S, (K) can be viewed injecting noise to the layer k features sampled from some distribution convigx (2)), and so the (K) neural network Fx can be viewed as a probabilistic mapping from comv(gk (2)) to P(Y). the space of probability distributions on y
To isolate the benefit of noise injection, we adapt the approach of (Pinot et al. 9019a 202 1 to our setting to show that the Gaussian noise injection procedure in NFM robustifies manifold mixup in probabilistic sense. At its core, this probabilistic notion of robustness amounts to making the model locally Lipschitz with respect to some distance on the input and output space, ensuring that a small perturbation in the input will not lead to large changes (as measured by some probability metric) in the output. Interestingly, it related to a notion of differential privacy (Lecuyer et al. 2019 Dwork et al. 2014), as formalized in (Pinot et al. (019b).
We now formalize this probabilistic notion of robustness.
Let > 0. We say that standard model f : X + Y is ap-robust if for any (z, y) ~ D such that f(x) = y, one has, for any data perturbation T E X,
|| S Op = f(x) = TT).
(104)
Analogous definition can be formulated when output of the model is distribution-valued
Definition 4 (Probabilistic robustness). A probabillisicc model F - P()) is called (ap. robust with respect to D if. for any x, T E X, one has
|r S Upp D(F(s), Se,
(105)
where D is metric or divergence between two probability distributions.
We refer to the probabilistic model (built on top of manifold mixup classifier) that injects Gaus- sian noise the layer & features as probabilistic FM model, and we denote it by Enoisy(k) conv(gn (X)) + P()).. We denote G as the classifier constructed from (roisy(()) i.e. G arg |
In the sequel, we take to the total variation distance Drv, defined as:
Drv(P.Q) == sup IP(S) Q(S)).
(106)
SCX
for any two distributions P and Q over X. Recall that if P and Q have densities Pp and Py respectively, then the total variation distance is half of the L distance, i.c., Drv (P, Q) 1 fxx [pp(z) - (=)) The choice of the distance depends on the problem on hand and will give rise to different notions of robustness. One could also consider other statistical distances such as the Wasserstein distance and
Renyi divergence, which can be related to total variation (see (Pinot et al. 2021 Gibbs & Su 2002)
for details).
Before presenting our main result in this section, we need the following notation. Let E(x) := a2 T + o² " For x, T X, let II be d, by de matrix whose columns form a basis for the subspace orthogonal to (a(2 + 7) gie (x)),, and [pi(gx (x), r)) be the eigenvalues of (II'') E(g (I)) " the entries in the vector F(x).
(a + HI I. Also, let topt (x) denote the keth highest value of
Viewing N (K) classifier probabilistic FM classifier, we have the following result.
27

---

Published as a conference paper at ICLR 2022
Theorem 5 (Gaussian noise injection robustifies FM classifiers). Let k E S, dg 1. and assume that 9& (x()))((()) 2 B 0 for all x E como(X)) for some constant Bx. Then, Fnoiey(k) is (Opp 6 (D, d, Op: aadd, 7 mullt) -robust with respect to Drv against lp adversaries, with
9
Ex d, ap, Gadd, mmult)
min max A, B}}
(107)
2
where
p mule
(11 Ve + tr)dtt
2
+ +2l1gx 11.'
A = Ap(ap)
Vg + tr)dt
add + of mule 32
(108)
B = (x(T) Ap Ap(=pe(0.2' + d2/2-1112 p1 p2(2000) + ddlp=oo)
(109)
Voldd add + 0t m
with
ap apla,<t + aglo,221 p/2-1/p (appi(appppallp aa + agtap21); /dlapIapet + af
i E (0, 2],
Ap(ap) =
i E (2, x));
(110)
if xc,
and
(11 Vgu sup xEconv(2)
Von(z + tr)dt
de-1
Bx(T)=
Σ
p. p((on(z)
(111)
Moreover, i X is such that [Fnoisy(k))topp (x:) [ 2e(p, d, apr a mmult).
then for any T X. we have
|||pp Sa = G(x)= = GTr
(112)
for any p 0.
Theorem 5 implies that we can inject Gaussian noise into the feature mixup representation to improve robustness of FM classifiers in the sense of Definition 4 while keeping track of maximal loss in accuracy incurred under attack, by tuning the noise levels oodd and mmult. To illustrate this, suppose that muutt 0 and consider the case of p 2, in which case A 0, B aad/oed and so injecting additive Gaussian noise can help controlling the change in the model output, keeping the classifier's prediction, when the data perturbation of size 0:2-
We now prove Theorem Before this, we need the following lemma.
Lemma 3. Let I1 == 2 Rda and Ø2 == 2 + E Rd=,, with T > and dy > 1, and E(x) == for some constant for all æ. Let 11 be da by dx- -1
32)1 + L + o2 Ta > (o"ad to matrix whose columnsfore a basis for the subspace orthogonal to T, and let p1(2, T), T).... Pd, - (2,7)
denote the eigenvalues of [(=:) 'III" E(22) II I.
Define the function C( B2 Σ) = max(A, A, B). where
02 muult
A
11
(113)
o add + o muul
|T/|
di Σ i=1
B
T2)
(114)
loudd + mm
Then, the total variation distance between M(x, E(x1)) and N N( E(x2)) admits the following bounds:
Drv(N(xi, E(z))), E(x2)) 9
S
(115)
200
min C( x2, E)}
Proof of Lemma 3 The result follows from a straightforward application of Theorem 1.2 in (Devroye iet al., 2018). which provides bounds on the total variation distance between Gaussians with different means and covariances.
28

---

Published as a conference paper at ICLR 2022
With this lemma in hand, now prove Theorem
Proof of Theurem We denote the noise injection procedure by the map I - (sr, E(x)).
where [(x) o aada mmatt
Let E X be test datapoint and T E X' be a data perturbation such that ||||p S aa for >
Note that
Drv PFy((F(((ou(())) FATTO
S Drv (I) +
(116)
S Drv (I(g(())))) + I + -x(
(117)
Drv (I(ge(z)), (xx(x) + +w))
(118)
s min(1, (1, ( (I), Thi aadd, Tmults 9k)}
(119)
where 7h: gx(x 7) gk(xx)= (Io Vgg(x + +T) T by the generalized fundamental theorem of calculus, and
(ggh (x), Tki Godd; O muult 3k)
d2
mult
32 (1[Tel22+ 2(Th, (x))), Vorda + ommnee 32
(((
== max
2
+ o²
mult!
(120)
where the pr(g (z), r) are the eigenvalues given in the theorem.
In the first line above, we have used the data preprocessing inequality (Theorem 6 in (Pinot al.] 202I)), and the last line follows from applying Lemma 3 together with the assumption that yl )o (z:)² 2 B > 0 for all x.
Using the bounds
|| S 1//2 Von (x ( |T||2
(121)
and
[Te, (x)))( s (>)x())lla / Von + + 1
(122)
have
(gg (x); Thi Gadd; Omult, Ba) max {A, B}.
(123)
where
(11. g + tr)dt
2 1| + 2119x(())l2 1/." Vg + tr)dt
A
mult
||T||2
2
o2 munte B
addd
2
(124)
and
1o Vz + tr)dt
||T||2
de
B
((()(()).T)
(125)
Volas + o ommierz 32
(11' fou(o+trig) < sup xEcono(X)
||
Σ Aloa(t))T))
(126)
Vouua + o 2 82
11
=: Bx( Volda + 2
(127)
29

---

Published as a conference paper at ICLR 2022
The first statement of the theorem then follows from the facts that || - |17|pp ap for (0, 2]. || S d T S
dl/2-1/9a, for q > 2, and || S vdllTlo I dd for any RR
In particular, these imply that A < CAp. where
aplape agtap21 (a + a 20,, v Vdlaptla, + o0)))
if E (0, 2), if E (2, x)) if = 0o,
Ap
(128)
and
Poutt
V (11 V.n + + (T)dd
1110 Vin (2r +
C===
+ 211un (2))l2 21)un()))
(129)
Pa add + o m
The last statement in the theorem essentially follows from Proposition 3 in Pinot et al. 2021)
E ON GENERALIZATION BOUNDS FOR NFM
Let F. be the family of mappings x + [(x) and Zn == ((I;, y4 (e)m)- Given loss function 1, the
Rademacher complexity of the set o o = {(x, v) + I(f(x).)) [ F) is defined as:
Σ cdl(((r))) |EF
o = Ezn,o
sup
(130)
where σ = (o1...., Tn), with the o, independent uniform random variables taking values in {-1,1).
Following (Lamb et al. 2019). we can derive the following generalization bound for the NFM loss function, i.e., the upper bound the difference between the expected error unseen data and the NFM loss. This bound shows that NFM can reduce overfitting and give rise to improved generalization.
Theorem 6 (Generalization bound for the NFM loss). Assume that the loss function satisfies [1 y) - 1(x', y)1 S M for all x, x' and y. Then, for every 8 > O, with probability at least 1 a draw of i...d. samples {(1,, 9:) we have the following generalization bound: for all maps I E F,
In(1/5)
Eey((((=)),))) -L LNF [ 2M
Qc(f),
(131)
2n
where
Q.(5)= = E(cR (k) + 2R)
(k) +22 + (xp(())
(132)
for some function &p such that limg-000 (0x)) = 0.
To compare the generalization behavior of NFM with that without using NFM, we also need the following generalization bound for the standard loss function.
Theorem (Generalization bound for the standard loss). Assume that the loss function satisfies [1x y) -((x', v)1 S M for all x, x' and y. Then, for every 8 > with probability at least 1 over a draw of i..dd samples 9:)3 we have the following generalization bound: for all
maps! f E
In(1/6)
Ex v\ L 21 + 2M
(133)
2n
By comparing the above two theorems and following the argument of (Lamb et al.) 2019). we see that the generalization benefit of NFM comes from two mechanisms. The first mechanism is based on the term (f). Assuming that the Rademacher complexity term is the same for both methods, then NFM has a better generalization bound than that of standard method if Q. > 0. The second mechanism is based on the Rademacher complexity term R,, (I F). For certain families of neural networks, this term can be bounded by the norms of the hidden layers of the network and the norms of the Jacobians of each layer with respect to all previous layers (Wei & Ma 2019a b). Therefore,
30

---

Published as a conference paper at ICLR 2022
this term differs for the case of training using NFM and the case of standard training. Since NFM implicitly reduces the feature-output Jacobians (see Theorem 3). we can argue that NFM leads to a smaller Rademacher complexity term and hence a better generalization bound.
We now prove Theorem
The proof of Theorem 7 follows the same argument as that of Theorem
Proof of Theorem 6 Let Zn == {( 34) He(n and Z := {(a).. b4) be two test datasets, where 7h, differs from 7.p by exactly one point of an arbitrary index ip-
Denote (Z,, == seP E ((f(()) v)] INN where LNFM is computed using the dataset Zn, and likewise for GE Then,
M(2n -1
2M S
GE - GE(Z,) S
(134)
n²
where we have used the fact that FN has n² terms and there are 2n 1 different terms for Zn and Z/ Similarly, we have GE((z)) GE 2M
Therefore, by McDiarmid's inequality, for any O, with probability at least 1 6,
In(1/6)
GE < Ez, (GE(Z)) 2M
(135)
2n
Applying Theorem
we have
In(1/8)
GE(Za) < Ez,
sup Ez
1((()))
NNFM
2M
(136)
fEF
n
2n
Iz.
sup Ez; FEF
Σ ((f(z)).1)) v1)
(f((z)) vi)
Q.(f)
71
In(1/0)
+ +2M
(137)
2n
In(1/8)
< E Zn.Z% Zi
sup ((f(4)) -1((())))) Q.(f) + 2M SEF n2
(138)
2n
In(1/5)
< F. Zn.
sup Σ (((((()))))))) - - 2c()) + 2M
[EF n.
2n
(139)
(n(1/8)
< 2F Zn.a
sup a, vi)
Q.( - 2M
(140)
[ET n
2n
In(1/o)
-2R,(oFF)-Q.0
2M
(141)
2n
where 136 uses the definition of GF 137 uses EE (((())) y:) inside the expectation
and the Innearity of expectation, (138) follows from the lensen's inequality and the convexity of
the supremum, (139) follows from the fact that o ((((()))) vt) [[f(x,), 41) and ((f(z)), v\)
(ff(x)), y/) have the same distribution for each o; E (1 (since Zn. 2, are drawn i.i.d. with the same distribution), and (140) follows from the subadditivity of supremum.
The bound in the theorem then follows from the above bound.
31

---

Published as a conference paper at ICLR 2022
F ADDITIONAL EXPERIMENTS AND DETAILS
F.1 INPUT PERTURBATIONS
We consider the following three types of data perturbations during inference time:
White noise perturbations are constructed as 2 = + Ar, where the additive noise is sampled from Gaussian distribution Ar (O0 σ). This perturbation strategy emulates measurement errors that can result from data acquisition with poor sensors (where corresponds to the severity of these errors).
Salt and pepper perturbations emulate defective pixels that result from converting analog signals to digital signals. The noise model takes the form (PX = X) --,, 7, and P(X = max) = P(X = min) = 7/2, where (i. denotes the corrupted image and min, max denote the minimum and maximum pixel values, respectively. Y parameterizes the proportion of defective pixels.
"worst-case" non-random perturbations that maximize the loss ((gg + AX), y) subject to the constraint || S on the norm of the perturbation. We consider the projected gradient decent for constructing these perturbations (Madry al. 2017).
Adversarial perturbations
F.2 ILLUSTRATION OF THE EFFECTS OF NFM ON TOY DATASETS
We consider binary classification task for the noise corrupted 2D dataset whose data points form two concentric circles. Points on the same circle corresponds to the same label class. We generate 500 samples, setting the scale factor between inner and outer circle to be 0.05 and adding Gaussian noise with zero mean and standard deviation of 0.3 to the samples. Fig. 8 shows the training and test data points. We train a fully connected feedforward neural network that has four layers with the ReLU activation functions on these data, using 300 points for training and 200 for testing. All models are trained with Adam and learning rate 0.1, and the seed fixed across all experiments. Note that the learning rate can be considered regularization itself. Hence, we choose a learning rate that is large for this problem to better illustrate the regularization effects imposed by the different schemes that consider.
temperature parameter which introduces some amount of
Fig. illustrates how different regularization strategies affect the decision boundaries the neural network classifier. The decision boundaries and the test accuracy indicate that white noise injections and dropout (we explore dropout rates in the range [0.0, 0.9] and we finds that 0.2 yields the best performance) introduce favorable amount of regularization. Most notably is the effect of weight decay (we use 9)-3), i.e., the decision boundary nicely smoothed and the test accuracy is improved. In contrast, the simple mixup data augmentation scheme shows benefits here, whereas manifold mixup is improving the predictive accuracy considerably. Combining mixup (manifold mixup) with noise injections yields the best performance in terms of both smoothness of the decision boundary and predictive accuracy. Indeed, NFM is outperforming all other methods here.
The performance could be further improved by combining NFM with weight decay or dropout. This shows that there are interaction effects between different regularization schemes. In practice, when
(a) Data points for training.
(b) Data points for testing.
Figure 8: The toy dataset in R² that use for binary classification.
32

---

Published as a conference paper at ICLR 2022
90
90
Test Accuracy
80
Baseline
80
Baseline
Mixup (a= 0.1)
Mixup (a 0.1)
Noisy Mixup (a = 0.1)
Noisy Mixup =a= 0.1)
Manifold Mixup (a 0.1)
Manifold Mixup (a 0.1)
Nolsy Feature Mixup (+)
Nolsy Feature Mixup (*)
Noisy Feature Mixup (a= 0.1)
Noisy Feature Mixup (ar 0.1)
0.00
0.05 0.10 0.15
0.20 0.25
0.30
0.35
0.00
0.05
0.10
0.15
0.20
White Noise (ar)
Salt and Pepper Noise (r)
Figure 9: Vision transformers evaluated on CIFAR-10 10 with different training schemes.
Table 4: Robustness of Wide-ResNNt--Bs 18 w.rt. white noise (a) and salt and pepper (t) perturbations evaluated on CIFAR-100. The results are averaged over models trained with different seed values.
Scheme
Clean (%)
(%)
7 (%)
0.1
0.2
0.3
0.08
0.12
0.2
Baseline
91.3
89.4
77.0
56.7
83.2
74.6
48.6
Mixup (a 0.1)
Zhang al.
2017
91.2
89.5
77.6
57.7
82.9
74.6
48.6
Mixup (a 0.2) Zhang
201
91.2
89.2
77.8
58.9
82.6
74.5
47,9
Noisy Mixup (cr
0.1]
Yang èl
2020b
90.9
90.4
87.5
80.2
84.0
79.4
63.8
Noisy Mixup (a 0.2) Yang ef
202(6
90.9
90.4
87.4
79.8
83.8
79.3
63.4
Manifold Mixup (a 0.1) Verma ct al.
2019
91.2
89.2
77.2
56.9
83.0
74.3
47.1
Manifold Mixup (a = 1.0) Verma el al.
2019
90.2
88.4
76.0
55.1
81.3
71.4
42.7
Manifold Mixup (a 2.0) Verma al.
2019
89.0
87.0
74.3
53.7
79.8
70.3
41.9
Noisy Feature Mixup (a: = -0.1)
91.4
90.2
88.2
84.8
84.4
81.2
74.4
Noisy Feature Mixup (a: = 1.0)
89.8
89.1
86.6
82.7
82.5
79.0
71.4
Noisy Feature Mixup (a = 2.0)
88.4
87.6
84.6
80.1
80.4
76.5
68.6
one trains deep neural networks, different regularization strategies are considered as knobs that are fine-tuned. From this perspective, NFM provides additional knobs to further improve model.
F.3 ADDITIONAL RESULTS FOR VISION TRANSFORMERS
Here we consider compact vision transformer (ViT-lite) with 7 attention layers and heads (Hassani al. 2021). Fig. (left) compares vision transformers trained with different data augmentation strategies. Again, NFM improves the robustness of the models while achieving state-of-the-an accuracy when evaluated on clean data. However, mixup and manifold mixup do not boost the robustness. Further, Fig. (right) shows that that the vision transformer is less sensitive to salt and pepper perturbations as compared to the ResNet model. These results are consistent with the high robustness properties of transformers recently reported in Shao et al. (2021): Paul & Chen (2221). Table 4 provides additional results for different a values.
Table 4 shows results for vision transformers trained with different data augmentation schemes and different values of a. It can be seen that NFM with a = 0.1 helps to improve the predictive accuracy on clean data while also improving the robustness of the models. For example, the model trained with NFM shows about 25% improvement compared to the baseline model when faced with salt and paper perturbations ( 0.2). Further, our results indicate that larger values of have negative effect on the generalization performance of vision transformer.
F.4 ABLATION STUDY
In Table we provide a detailed ablation study where we vary several knobs. First, we can see that just injecting noise helps improve robustness, but the test accuracy is only marginally improving. On the other hand, just mixing inputs and hidden features improves the testing performance of the model, but it does not significantly improve the robustness of a model. In contrast, the NFM scheme combines best both worlds and shows that both accuracy and robustness can increased. Varying the noise levels indicate that there is trade-off between test accuracy clean data and robustness
33

---

Published as a conference paper at ICLR 2022
perturbations. We also vary the mixup parameter to show that the good performance is consistent across a range of different values.
Table 5: Ablation study using Wide-ResNet-18 trained and evaluated on CIFAR-100.
Mixup
Munifold
Noise Injections
Noise Levels
Clean (%)
(%)
0.1
0.25
0.5
0.06
0.1
0.15
x
X
0
76.9
64.6
42.0
23.5
58.1
39.8
15.1
x
0.4
0.2
78.1
76.2
65.7
46.6
70.0
58.8
28.4
X
X
0
80.3
72.5
54.0
33.4
62.5
43.8
16.2
x
0.4
0.2
78.9
78.6
66.6
46.7
66.6
53.4
25.9
/
X
0.2
0
79.7
70.6
46.6
25.3
62.1
43.0
X
0
79.7
70.5
45.0
23.8
62.1
42.8
14.8
7
X
2
0
0
79.2
69.3
43.8
23.0
62.8
44.2
16.0
7
0.1
0.1
81.0
76.2
56.6
36.4
66.8
49.7
21.4
0.2
0.4
0.2
80.6
79.2
70.2
51.7
71.5
60.4
30.3
0.2
80.9
80.1
72.1
72.8
62.1
34.4
/
0.2
80.7
71.5
53.9
72.7
36.6
0.8
0.4
80.3
80.1
75.5
66.4
74.3
66.5
44.6
F.5 ADDITIONAL RESULTS FOR RESNETS WITH HIGHER LEVELS OF NOISE INJECTIONS
In the experiments in Section we considered models trained with NFM that use noise injection levels add 0.4 and muit 0.2, whereas the ablation model uses addd 1.0 and mult 0.5. Here, we want to better illustrate the trade-off between accuracy and robustness. We saw that there exists potential sweet-spot where we are able to improve both the predictive accuracy and the robustness of the model. However, if the primary aim is to push the robustness of the model, then need to sacrifice some amount accuracy.
Fig. 10 is illustrating this trade-off for pre-actived ResNet- 18s trained on CIFAR-1O. We can see that increased levels noise injections considerably improve the robustness, while the accuracy on clean data points drops. In practice, the amount of noise injection that the chooses depend on the situation. If robustmess is critical, than higher noise levels can be used. If adversarial examples are the main concern, than other training strategies such as adversarial training might be favorable. However, the advantage of NFM over adversarial training is that (a) we have a more favorable trade-off between robustness and accuracy in the small noise regime, and (b) NFM is computationally inexpensive, when compared to most adversarial training schemes. This is further illustrated in the next section.
F.6 COMPARISON WITH ADVERSARIAL TRAINED MODELS
Here, we compare NFM to adversarial training in the small noise regime, i.e., the situation where models do not show a significant drop on the clean test set. Specifically, we consider the projected gradient decent (PGD) method (Madry et al. 2017) using attack iterations and varying la per-
95
95
Test Accuracy
90
90
85
Gast 0.4, 0mul 0.2
85
aase 0.6, 0mut 0.3
Tax 0.8, dnut 0.4
Oas 1.0, 0mu
Gas 1.2, amui 0.6
70
70
0.00
0.15 0.20 White Noise (a))
0.30
0.35
0.000
0.075 0.125 0.150 Salt and Pepper Noise (7)
Figure 10: Pre-actived ResNet-18 evaluated on CIFAR- 10 trained with NFM and varying levels of additive (Aada) and multiplicative (amutt) noise injections. Shaded regions indicate one standard deviation about the mean. Averaged across 5 random seeds.
34

---

Published as a conference paper at ICLR 2022
95
95
Test Accuracy
90
90
0.001
0.002
75
75
= 0.01
70
70
NFM (dad = 0.4, 0.2)
NFM (Cacs 1.2, Gmutt 0.6)
60
0.000
0.025 0.050 0.075 0.100 0.125
0.150
0.00
0.02
0.04
0.06
0.08
0.10
0.12
Adverserial Noise (c)
Adverserial Noise (c)
Figure 11: Pre-actived ResNet-18 evaluated on CIFAR- 10 (left) and Wide ResNet- 18 evaluated on CIFAR-100 (right) with respect adversarial perturbed inputs. Shaded regions indicate standard deviation about the mean. Averaged across 5 random seeds.
turbation levels to train adversarial robust models. First, we compare how resilient the different models are with respect adversarial input perturbations during inference time (Fig. [1 left). Again adversarial examples are constructed using the PGD method with attack iterations. Not very surprisingly, the adversarial trained model with 0.01 features the best resilience while sacrificing about 0.5% accuracy as compared to the baseline model (here not shown). In contrast, the models trained with NFM are less robust, while being about 1 1.5% more accurate on clean data.
Next, we compare (Fig. right) the robustness with respect to salt and pepper perturbations, i.e., perturbations that both models have not before. Interestingly, here NEM scheme with high noise injection levels as compared to the adversarial trained models.
advantage of the
F.7 FEATURE VISUALIZATION COMPARISON
In this subsection, we concern ourselves with comparing the features learned by three ResNet- 50 models trained on Restricted Imagenet (Tsipras et al. 2018): without mixup. manifold mixup (Verma et al. 2019), and NFM. We can compare features by maximizing randomly chosen pre-logit activations of each model with respect to the input, as described by [Engstrm et al. (2020). We do so for all models with Projected Gradient Ascent over 200 iterations, a step size of 16, and an E norm constraint of 2,000. Both the models trained with manifold mixup and NFM NFM model uses in addition aadd = 2.4 and muilt 1.2. The result, as shown in Fig. [2] is that the features learned by the model trained with NFM are slightly stronger (i.e., different from random noise) than the clean model.
0.2, and the
F.8 TRAIN AND TEST ERROR FOR CIFAR-100
Figure shows models trained with different training schemes on CIFAR-100. Compared to the baseline model, the models trained with manifold mixup and NFM have a similar convergence behavior. However, they are able to achieve a smaller test error. This shows that both manifold mixup and NFM have favorable implicit regularization effect, where the effect is more pronounced for the NEM scheme.
35

---

Published as a conference paper at ICLR 2022
Seed
No Mixup
Manifold Mixup
NFM (Ours)
Figure 12: The features learned by the NFM classifier are slightly stronger (i.e., different from random noise) than the clean model. See Subsection F.7 for more details.
100
100
Baseline Manifold Mixup ta 1.0)
Baseline
Manifold Mixup 1.0)
NFM (a= 1.0)
NFM (a= 1.0)
Train error
Test error
40-
40
20
20
100 Number epochs
100 Number epochs
(a) Train error.
(b) Tost error.
Figure 13: Train (a) and test (b) error for a pre-actived Wide-ResNet--F 18 trained CIFAR-100.
36


Reviews:
[{"review_id": "hvs2zt-pOb", "reviewer": "Reviewer_46ws", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Theorem 1 is one of the most important contributions of this paper. It shows that minimizing the proposed NFM loss is equivalent to minimizing the sum of the original loss and feature-dependent regularizers. The form of the regularizers is very complicated. I have checked part of them and seems to be correct. Too much work to follow the rest part. This theorem shows that the regularizers actually reduce the Jacobians and Hessians, which is reasonable and fit everyone's perception. In my opinion, although there is no surprising conclusion, this contribution provides evidence for everyone’s understanding of the mixup type methods.\n\nThe experiment part provides the necessary experiments to support the argument. But I am curious about the different behaviours of the proposed methods for features of different levels. According to conjecture, when the feature level is different, the meaning of adding noise and mixup may become completely different. It seems that this part of the discussion is missing.", "summary_of_the_paper": "The paper proposes a new and inexpensive mixup method named Noisy Feature Mixup (NFM) to mitigate over-fitting and improve generalization. NFM combines mixup and noise injection, which inherits the benefits of these methods. Due to its conciseness, it is convenient to apply NFM in model training. More importantly, the authors prove NFM's regularization effect on model optimization and robustness improvement with mathematical derivation. They build the implicit connection between the NFM empirical loss and the original loss. Then they identify the regularizing effects of the proposed NFM. To demonstrate that NFM helps robustness, the authors relate the NFM loss to the one used for adversarial training, which can be regarded as an example belonging to distributionally robust optimization. With integrated experiments, NFM shows its superiority on various models and datasets. The further discussion shows the interesting tradeoff between predictive accuracy on clean and perturbed test sets. The Supplementary Material provides detailed proof and additional experimental results to show NFM's firmed ground.", "summary_of_the_review": "It is difficult to rate this paper. Theorem 1 has theoretical value. The proposed method is relatively simple (both pros and cons). But I would vote positively for theorem 1.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "cfBGBiiRXsv", "reviewer": "Reviewer_gzuL", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall this paper is well written and easy to follow. The strengths of this paper include:\n1. This paper has a strong motivation, i.e., exploits the benefits of mixup and noise injection to further improve the generalization ability of a trained model.\n2. The proposed method, i.e., Noisy Feature Mixup (NFM), is simple in principle and easy to implement in practice.\n3. Theoretical analyses are offered to illustrate the underlying mechanisms of NFM. The theorems in the paper are built with reasonable assumptions, and the conclusions are sound and can explain the practical behaviors of NFM to some degree. Overall I find the theorems offer a good insight for NFM.\n4. The proposed NFM is evaluated on different datasets, with two representative neural network architectures. The experimental designs are reasonable, and the results can well demonstrate the advantages of NFM over different Mixup variants.\n\nFor this paper, I have the following concerns/questions:\n1. In NFM, the noisy mixup is performed on a random layer $k \\in [L]$ in each iteration. But is it really necessary to apply NFM to every layer? In other words, is it possible that for some layers, the noisy mixup would lead to negative (or neutral) effects and hurt the performance?\n2. From the experimental results, it is unclear if the stochasticity introduced by NFM during training would lead to a slow convergence of model training. Is it possible that such stochasticity would make the model difficult to converge, or require more time for the model to converge?\n3. In Theorem 2, it is assumed that $E_{r \\sim D_x}[g_k(r)] = 0$. However, I find this assumption could be too strong in practice.", "summary_of_the_paper": "This paper proposes a new method for data augmentation, named Noisy Feature Mixup (NFM). This method combines the advantages of both the interpolation based training and noise injection schemes. In particular, this method is simple and easy to implement. Empirically, this paper shows that NFM achieves a favorable trade-off between the test accuracy on the clean set and the model robustness. Theoretically, it is shown that NFM enables smoother decision boundary, and amplifies the regularization effects of manifold mixup and noise injection. Additional theoretical analysis also shows that NFM training is approximately minimizing an upper bound related to the adversarial loss, thereby leading to more robust model.", "summary_of_the_review": "The problem this paper focuses on is important, i.e., improve the generalization performance of deep models. To this end, this paper proposes a new method that exploits the benefits of mixup and noise injection, and is empirically demonstrated to be more effective at improving the accuracy and robustness of trained models. The proposed method is simple and easy to implement. Notably, it has sound theoretical analyses to illustrate the underlying mechanisms. Although for this paper there are still some points which are unclear, currently I think it is overall a good paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "cJlymCo1ZQ2", "reviewer": "Reviewer_o7KT", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Pros**\n- The paper is easy to read and the writing is very clear. \n- The paper provides thorough theoretical analyses, which the original manifold mixup paper lacks. \n- The proposed method is effective in noisy test environments with various models and datasets. \n\n**Cons**\n- The performance gain under the clean setting is marginal. \n- The experimental setting is limited. It will be valuable to evaluate methods in more general and natural corruption settings. (please refer https://github.com/hendrycks/robustness)\n- Baselines in experiments are not sufficient. It will be informative to compare some recent mixup techniques enhancing robustness (e.g., Puzzle Mix [1] performs mixup-aware adversarial training, and AugMix [2] enhances corruption robustness). \n\n**Additional comments**   \n- Does the theory can be generalized to the more general perturbations (such as natural data augmentation)? \n- For me, the statement in Theorem 2 is counter-intuitive (L^NFM upper bounds the worst-case). It will be informative to add a more intuitive explanation for the theorem in the paper. (e.g., when the statement is valid). \n- In appendix Figure 10, it is hard to interpret which one is better. If possible, the formal evaluation of the features will be also interesting. \n\n[1] Kim et. al, Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup, ICML 2020.  \n[2] Hendrycks et. al, AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty, ICLR 2020\n\n**Post rebuttal**   \nI appreciate the authors' rebuttal and believe the revision improves the paper. However, the robustness experiments are still not convincing and require stronger baselines to be convincing (e.g., AugMix or other methods with stronger augmentations). Overall, I believe this is a clear paper and I maintain my score. ", "summary_of_the_paper": "This paper proposes noise added version of the manifold mixup. The authors verify the theoretical properties of the method in terms of regularization and robustness. The paper verifies the effectiveness of the proposed method on image classification tasks with noisy test environments. ", "summary_of_the_review": "This paper is well-written and provides thorough theoretical properties. However, the experiment settings are a bit limited and require more baselines to be compared. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Bm8UYLG1H6", "reviewer": "Reviewer_WVNT", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths:\n\n- The main contribution of this paper is extending input mixup (Zhang et al. (ICLR'18)) and manifold mixup (Varma et al. (ICML'19)) to Noisy Feature Mixup, which applies mixup randomly to *all layers of the neural network* and *adding random noise to the mixup examples*.\n- Through this extension, the authors find that Noisy Feature Mixup improves the robustness to several kinds of noise models comp\u0010ared to previous mixup variants.\n- Additionally, a theoretical analysis of the regularization effects of Noisy Feature Mixup is presented, building on top of the results by Zhang et al. (ICLR'21).\n\n**Related works**\n\nThe related works are comprehensive in this area.\n\n## Weakness:\n\nThe main weakness of this paper is the accessibility of the technical contribution. Thus, I think several revisions towards mitigating this weakness could improve the presentation, including, for example\n\n- Illustrative examples and figures to convey Theorem 1 and Theorem 2, for example, similar to Figure 2 in Zhang et al. (ICLR'2021).\n- The proofs are in line with recent works including Zhang et al. (ICLR'2021) which seem correct to me. The main idea is using the Taylor's expansion to the Noisy Feature Mixup objective. Then, the authors note that the random noise injection can be simplified on top of the regularization terms from Zhang et al. (ICLR'2021). However, the steps are difficult to verify; in particular, more detailed derivations should be included in the appendix.\n    - For example, above equation (30), it's stated that \"we compute, using linearity and chain rule.\" However, it's not clear to me how this is computed.\n    - Then after equation (33), it's stated that \"The equation in the theorem follows upon setting...\" It's not immediately clear to me how this step is carried out either.\n    - The rest of the appendix has similar issues as above. They should be expanded with more clear explanations and derivations.\n\n**Missing ablation studies**\n\n- It's unclear why the authors set the hyper parameters of the additive noise $\\sigma_{add}=0.4$ and the multiplicative noise $\\sigma_{mult}=0.2$. This should be justified with an ablation study that varies each hyper parameter.\n- Since there are two components in the proposed approach (all-layer mixup and random noise injection), It's also unclear which part is contributing more or less. Again there should be an ablation study to justify the role of each component in the proposed approach.\n\n**Missing baselines**\n\n- The authors mainly consider Mixup and Manifold Mixup as baselines in the experiments. This is marginally acceptable. To make the experiments stronger, I think the authors could consider other related methods for improving robustness such as *label smoothing*.", "summary_of_the_paper": "This paper studies data augmentation methods for improving the robustness of supervised learning. The main contribution is presenting Noisy Feature Mixup, extending input mixup and manifold mixup to all layers of a neural net. The experimental results show that the proposed approach improves the robustness of supervised learning under several noise attacks on the input data set. The theoretical results derive the Taylor's expansion of the Noisy Feature Mixup optimization objective.", "summary_of_the_review": "This is a technically solid and empirically strong paper. The theoretical results follow from recent works for mixup (Zhang et al. (ICLR'21)). The experimental results are conducted on three popular image datasets, with three input perturbation models, and do support the validity of the proposed approach. The main weakness of this paper is the accessibility of the technical contribution and the lack of ablation studies. I think several revisions could help mitigate these weakness.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces Noisy Feature Mixup: an extension of input mixup and manifold mixup to all layers of a neural net, for the purpose of improving robustness and generalization in supervised learning. Experimental validation supports the increased robustness to attacks on the input data. The reviewers find the paper well written and they appreciate the theoretical analysis as well as the empirical results. The reviewers did not identify any big problems, and their minor concerns were sufficiently addressed in the author reponse. I'm therefore happy to recommend accepting this paper.
Decision: Accept (Poster)