Adversarial Unlearning: Reducing Confidence Along Adversarial Directions
Authors: Amrith Setlur
Benjamin Eysenbach
Virginia Smith
Sergey Levine
Venue: NeurIPS 2022 Accept (2022)
Keywords: supervised learning
overfitting
regularization
adversarial examples
spurious correlations

Abstract: Supervised learning methods trained with maximum likelihood objectives often overfit on training data. Most regularizers that prevent overfitting look to increase confidence on additional examples (e.g., data augmentation, adversarial training), or reduce it on training data (e.g., label smoothing). In this work we propose a complementary regularization strategy that reduces confidence on self-generated examples. The method, which we call RCAD (Reducing Confidence along Adversarial Directions), aims to reduce confidence on out-of-distribution examples lying along directions adversarially chosen to increase training loss. In contrast to adversarial training, RCAD does not try to robustify the model to output the original label, but rather regularizes it to have reduced confidence on points generated using much larger perturbations than in conventional adversarial training. RCAD can be easily integrated into training pipelines with a few lines of code. Despite its simplicity, we find on many classification benchmarks that RCAD can be added to existing techniques (e.g., label smoothing, MixUp training) to increase test accuracy by 1-3% in absolute value, with more significant gains in the low data regime. We also provide a theoretical analysis that helps to explain these benefits in simplified settings, showing that RCAD can provably help the model unlearn spurious features in the training data.

Adversarial Unlearning: Reducing Confidence Along Adversarial Directions
Amrith Setlur!.
Benjamin Eysenbach' Carnegie Mellon University
Virginia Smith UC Berkeley
Sergey Levine
Abstract
Supervised learning methods trained with maximum likelihood objectives often overfit on training data. Most regularizers that prevent overfitting look to increase confidence additional examples (e.g., data augmentation, adversarial training), or reduce on training data (e.g., label smoothing). complementary regularization strategy that reduces confidence on self-generated examples. The method, which we call RCAD (Reducing Confidence along Ad- versarial Directions), aims reduce confidence on out-of-distribution examples lying along directions adversarially chosen to increase training loss. In contrast to adversarial training, RCAD does not try robustify the model output the origi- nal label, but rather regularizes it to have reduced confidence on points generated using much larger perturbations than conventional adversarial training. RCAD can be easily integrated into training pipelines with few lines of code. Despite its simplicity, we find on many classification benchmarks that RCAD can be added existing techniques (e.g., label smoothing, Mix training) to increase test accu- racy by 1-3% absolute value, with more significant gains the low data regime. We also provide a theoretical analysis that helps to explain these benefits in simpli- fied settings, showing that RCAD can provably help the model unlearn spurious features in the training data.
this work we propose a
Introduction
Supervised learning techniques typically consider training models make accurate predictions fresh test examples drawn from the same distribution as training data. Unfortunately. well known that maximizing the likelihood of the training data alone may result in overfitting. Prior work broadly considers two approaches to combat this issue. Some methods train on additional examples, e.g., generated via augmentations [53, 10, 66] adversarial updates [14, 36, 4]. Others modify the objective by using alternative losses and/or regularization terms (e.g., label smoothing [56, 39], MixUp [69], robust objectives [65, 22]). In effect, these prior approaches either make the model's predictions more certain on new training examples or make the distribution over potential models less certain.
Existing regularization methods can be seen as providing a certain inductive bias for the model, e.g., the model's weights should be small (i.e., weight decay), the model's predictions should vary linearly between training examples (i.e., MixUp). this paper we identify a different inductive bias: the model's predictions should be less confident on out-of-distribution inputs that look nothing like the training examples. We turn this form inductive bias into a simple regularizer, whose benefits are complementary existing regularization strategies. To instantiate such method, must be able sample out-o--distributtio examples. For this, we propose a simple approach: generating adversarial examples [38, 14] using very large step sizes (orders-of-manl -aaggiiuue larger than traditional adversarial
*Correspondence can he sent to
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

training
regularzationn
Cross entropy
Cross entropy
Cross entropy
Entropy
Test Accuraty
ROMM Vin telin KCAD 10uns
Original
Adwemariii Inages
Originll
Sel-genaroted images RIAD targe stecl
CIFAR-100 DO
Tiny Imagenet
(a)
Figure 1: Reducing confidence along adversarial directions (RCAD) is simple and efficient regularization technique to improve test performance. (Left) For RCAD, examples are generated by taking a large step (10x typical for adversarial examples) along the gradient dircction. We see that generated images thus look very different from the original, with accentuated spurious components responsible for the model's flipped predictions on adversarial images, (Right) RCAD achieves greater test accuracy than data augmentation (DA), label smoothing (ISS), and methods that minimize cross-entropy on adversarial examples: adversarial training via FGSM [14] and ME-ADA [73].
training [36]). Hence, we first perturb the training points using the training loss gradient, and then maximize predictive entropy on these adversarially perturbed examples.
In contrast to adversarial training, our method does not try to robustify the model to output the original label, but rather regularizes it have reduced confidence on examples generated via a large step size (Figure la). As shown in Figure 1b, this can lead to significant improvements in in-distribution test aceuracy unlike adversarial training [36, 37, 4], which tends to decrease in-distribution test performance [48, 70, 61]. Compared with semi-supervised learning methods [15, 64), our method does not require additional unlabeled dataset (it generates one automatically), and different from prior works [64, 74] it also trains the model to be less confident on these examples.
As the self- self-generated samples in procedure longer resemble in-distribution data, uncertain of their labels and train the model to predict a uniform distribution over labels on them, following the principle of maximum entropy [25]. A few prior works [56, 43, 52, 11] have also considered using the same principle to prevent the memorization of training examples, but only increase entropy iid sampled mndistributtin labeled/unlabeled data. In contrast, study the effect of maximizing entropy on self-generated out-of-distribution examples along adversarial directions.
The main contribution of this work is training procedure call RCAD: Reducing Confidence along Adversarial Directions. RCAD improves test accuracy (for classification) and log- ligliikeidood (for regression) across multiple supervised learning benchmarks. Importantly, find that the benefits of RCAD are complementary to prior methods: Combining RCAD with alternative regularizers (c.g.. augmentation [53, 66]. label smoothing [56]. MixUp training [69]] further improves performance. Our method requires adding ~5 lines of code and is computationally efficient (with training time at most 1.3x standard). We provide theoretical analysis that helps to explain RCAD's benefits in simplified setting, showing that RCAD can unlearn spurious features in the training data, thereby improving accuracy on unseen examples.
2 Related Work
Below we survey common regularization techniques in machine learning, as well as other methods that utilize entropy maximization in training for various purposes (differing from our own).
Data augmentation. A common strategy to improve test accuracy (particularly on image tasks) is to augment the training data with corruptions [53] or surface level variations [10. 66] (e.g., rotations, random crops). Some methods [37, 36, 14] further augment data with imperceptibly different adversarial examples or interpolated samples [69]. Others [46, 42] sample new examples from generative models that model the marginal density of the input distribution. Also related are semi- supervised methods, which assume access to an extra set of unlabeled data and train the model to have more confident predictions on them [15,37] All these methods minimize entropy [16] on the augmented or unlabeled data to improve generalization [69, 73]. contrast, our method maximizes entropy perturbed samples along the adversarial direction.

---

Table 1: Regularization objectives: We summarize prior works that employ adversarial examples or directly
regularize model's predictions Pwr (y x) along with the scalar hyperparameters (in [-) associated with each.
name
objective
cross entropy
minl 2x.c Pooly x)
label smoothing [c] [39]
minl ExweD (11 log x) En'ty 181-1 Pool (v' [x)
Adv. training [o] [36]
mill IxweD log Pio
sign(V, x log x)))
min Exxyie log (y (y| x) where, for distance metric Cu:(X x 3) (2' x ) + R
ME-ADA [a, 01 173]
D' A ((r,)) A supeo log PouPull (mo y) + (mo) - ((mo: y), (xs,))); v(x.y) e ())
RCAD (ours) Ja, Al
min ExNED (- log (y x) A- Hw a- Vx log x)))
Adversarial training. Deep learning models are vulnerable to worst-case perturbations that can flip the model's predictions [14, 50]. Adversarial training was proposed to improve robustness to such attacks by reducing worst-case loss in small regions around the training samples [36, 33, 48). More recently. Zhao et al. [73] proposed maximum entropy adversarial data augmentation (ME- ADA), which uses an information bottleneck principle to identify worst-case perturbations that both maximize training loss and predictive entropy. Similar to adversarial training ME-ADA still minimizes cross-entropp to output the same label on new points. There are two key differences between above methods and RCAD; (i) rather than minimizing cross entropy loss on adversarial examples, we increase model's uncertainty on the self- self-generated examples; and (ii) we take much larger steps- so large that unlike adversarial images, the generated example is no longer similar to the original one (Figure la). While prior work has successfully used adversarial examples to improve OOD detection [2] or adversarial robustness [36, 33, 48), we show that these changes allow our method to improve the standard test accuracy, a metric that adversarial training typically makes worse [48, 60, 70J. Further, RCAD has a lower (at most 1/5th computational cost compared to multi- step adversarial training procedures [73, 70].
Robust objectives. In order to improve robustness to noise and outliers in the data, a common approach is to modify the objective by considering risk averse loss functions [e.g.. 22, 49, 57] or incorporating regularization terms such as the 12/l1 norms [31, 59]. Our method similar to these approaches in that we propose new regularization term. However, whereas most regularization terms are applied to the model's weights or activations, ours is directly applied to the model's predictions. Another effective loss function for classification problems is label smoothing [56. 8, 26]. which uniformly increases model's predictive uncertainty in a region around training samples [12]. In contrast, RCAD increases entropy only on examples generated along the adversarial direction that has been shown to comprise of spurious features [23, 7]. thereby unlearning them.
Entropy maximization. Finally, we note that our work builds upon prior work that draws connections
between entropy maximization in supervised [52, 11, 43] and reinforcement learning [17]. For example, Pereyra et al. [43] apply a hinge form of confidence penalty directly on training data which similar to label smoothing in principle and performance (Table 2 in [43]). Other works like [11, 52] also adapt the principle of entropy maximization but do so either on additional unlabeled data or subset of the training samples. More recently [45] show that maximizing entropy on interpolated samples from the same class improves -istr--dittiiuutonn uncertainty quantification. While we also minimize cross-entropy loss on training data, contrast to the above we maximize entropy on samples generated along the adversarial direction. Our experimental results also span wider set of benchmarks and presents significant gains (+1-3% complementary to methods like label smoothing. We also theoretically analyze our objective for the class of linear predictors and show how RCAD can mitigate vulnerability to spurious correlations.
3 RCAD: Reducing Confidence Along Adversarial Directions
We now introduce our regularization technique for reducing confidence along adversarial directions (RCAD). This section describes the objective and an algorithm for optimizing it; Section 5 presents a more formal discussion of RCAD and in Section 4 we provide our empirical study.
Notation. We are given a training dataset D A {x(1),
y where (i)) X, y Y, are sampled tid from joint distribution D over X' x Y. We use D to denote both the training dataset and
3

---

an empirical measure over it. The aim is to learn a parameterized distribution Pw |x), w E W where the learnt model is typically obtained by maximizing the log-likelihood over D. Such solution is referred to as the maximum likelihood estimate (MLE); Wmle arg wa EE log Pw (y x). In the classification setting, we measure the performance of any learned solution vb using its accuracy on the test (population) data: Ep (arg m Pw (y' x) = y)]
Solely optimizing the log- ligeliidioood over the training data can lead to poor test accuracy, since the estimate Wile can overfit on noise in 1 and fail to generalize to unseen samples. Many algorithms aim to mitigate overfitting by either designing suitable loss functions replacing the log-likelihood objective, or by augmenting the training set with additional data (see Section 2). Our main contribution is a new data-dependent regularization term that will depend not just on the model parameters but also on the training dataset. We reduce confidence on out-of-distribution samples that are obtained by perturbing the original inputs along the direction that adversarially maximizes training loss.
In the paragraphs that follow we describe the methodology and rationale behind our objective that uses the following definition of model's ' predictive entropy when y is a discrete set:
Hu (x)
Σ Pwo (y BEY
x) log Pwo x)-
In cases where is continuous, for example in regression tasks, we will use the differential form of predictive entropy: - -fy Puo (y x) log (y/ x) dy.
Reducing Confidence Along Adversarial Directions. The key idea behind RCAD is that models should not only make accurate predictions on the sampled data, but also make uncertain predictions examples that are very dif- ferent from training data. We use directions that adversar- ially maximize the training loss locally around the train- ing points to construct these out-of-distributio examples that are different from the training data. This is mainly be- cause adversarial directions have been known to comprise of spurious features [23] and we want to regularize the model in way that makes it uncertain on these features. We first describe how these examples are generated, and then describe how we train the model to be less confident.
RCAD: Reducing Confidence along Adver- sarial Directions
def rcad_loss (x, a, A):
loss - model (x) 1og_prob(y))
x_adv + dloss.gradee entropy model(x_ade) .ettroo)).etrroy((( entropy
return loss
We generate an out-of-distributinn example x by taking a large gradient of the MLE objective with respect to the input x, using a step size of a > 0:
x - x a Vx Pw x)
We train the model to make unconfident predictions on these self-generate examples by maximizing the model's predictive entropy. We add this entropy term H. (x) to the standard MLE objective, weighted by scalar À > O. yielding the final RCAD objective:
Wread
arg max E. [logPoo( Pro (y x) + A Hi (x - Cx- Vx Pw x))] wEW
(2)
In adversarial training, adversarial examples are generated by solving a constrained optimization problem [36, 48). Using first-order approximation, the adversarial example & generated by one of the simplest solvers [14] has a closed form resembling Equation We note that RCAD different from the above form of adversarial training in two ways. First, RCAD uses much larger step size (10x larger), so that the resulting example no longer resembles the training examples (Figure la). Second, whereas adversarial training updates the model to be more confident on the new example, RCAD trains the model to be less confident. Our experiments in Scction 4 (Figure 4b) show that these differencee are important for improving test accuracy.
Informal understanding of RCAD. For image classification, image features that are pure noise and independent of the true label can still be spuriously correlated with the labels for a finite set of iid drawn examples [54]. Such features are usually termed spurious features [72]. Overparameterized neural networks have tendency to overfit on spurious features in the training examples [68, 51]. In order to generate unrealistic duitr--distriuution samples, we pick examples that are far away from the true data point along the adversarial direction because (i) adversarial directions are comprised of noisy features that are spuriously correlated with the label on few samples [23]; and (ii) maximizing entropy on samples with an amplified feature forces the model to quickly unleam that feature. By

---

CIFAR-100-2k:08
OFFR-100--B
CIFAR-100
66
34
64
02-
80
78
76
50
74
Mixup
Mixlip
CutDuti Mbxpp
td)
Tiny Imagenet
72
99.0
base buse
base
Test Accurssy
36
68
98.0
47.5
DA
CutOut+Mix Mix
MixUp
CutOut+ Mix Mixup
DA
L5
CutOut+ Mix MixUp
Figure 2: Main results on supervised image classification benchmarks: 95% confidence intervals over 10 independent rums for models trained with base methods:
We plot the mean accuracy and
Augmentation (DA), Label Smoothing (LS), CutOut/CutMixx (CutOut+-Mix) augmentation, Mix and compare them with the test accuracies of the models trained with the RCAD objective in Equation 2, in addition to the base methods.
using a large step size a we exacerbate the spurious features in our self-generated example x. When we maximize predictive entropy over our generated examples we then force the model to unlearn these spurious correlations. Hence, the trained model can generalize better and achieve higher prediction performance on unseen test examples. In Section 5 we build on this informal understanding to present a more formal analysis on the benefits of RCAD.
4 Experiments: Does RCAD Improve Test Performance?
Our experiments aim study the effect that RCAD has on test accuracy, both comparison and in addition to existing regularization techniques and adversarial methods, so as to understand the degree to which its effect is complementary existing methods.
Benchmark datasets. We use six image classification benchmarks. In addition to CIFAR-10. CIFAR-100 [30], SVHN [41] and Tiny Imagenet [34], modify CIFAR-100 by randomly sub- sampling 2,000 and 10,000 training examples (from the original 50,000) to create LIFAR-100-22 and -OFFAR-100-10K These smaller datasets allow us study low-data settings where we expect the generalization gap to be larger. CIFAR- MAARR-100-220 (0000-2/1000) share the same test sets. If the validation split is not provided by the benchmark, we hold out 10% of our training examples for validation.
Implementation ? Unless specified otherwise, we train all methods using the ResNet- 18 [19] backbone, and accelerate training loss convergence clip gradients in the l2 norm (at 1.0) [71, 18). We train all models for 200 epochs and use SGD with an initial learning rate of 0.1 and Nesterov momentum of 0.9, and decay the learning rate by a factor of 0.1 at epochs 100, 150 and 180 [10]. We select the model checkpoint corresponding to the epoch with the best accuracy on validation samples as the final model representing given training method. For all datasets (except CIFAR-100-24 and CIFAR- 1RR-000-110 100-10k for which we used 32 and 64 respectively) the methods were trained with a batch size of 128. For details on algorithm specific hyperparameter choices refer Appendix B.
Baselines. The primary aim of our experiments is to study whether entropy maximization along the adversarial direction shrinks the generalization gap. Hence, explore baselines commonplace in deep learning that either (i) directly constrain the model's predictive distribution on observed samples (label smoothing [39]) (ii) implicitly regularize the model by training on additional images generated in an adversarial manner. For the first, our main comparisons are to label smoothing [56)), standard data augmentation [53], cutout data augmentation [66, 10], and MixUp [69] training. For the second, we compare with adversarial training[36] that uses FGSM [14] to perturb the inputs. Additionally, we compare RCAD with two recent approaches that use adversarial examples in different ways: adversarial data augmentation (ADA) [62] and maximum entropy adversarial data augmentation (ME-ADA) [73]. We summarize these baselines in Table 1.
"Code for this work can be found at https: h //tpt/ggtttttttt/// http://ttttuu...ars/e//////////e wttp://o///////--rreeeeeee
5

---

Test acc. Improvement over ERM
Acv. bruring
1S Aix Ig- 0,021
A2A
15 cnAdd
NE-ADA
RCAD
KEASI-LL
NC4D
Test Improvement ERM
advenarta treinig MEADA
RCAD
+4.0%
+2.0%
+1 +1.0%
+1%
0.0%
0%
-1.0%
-2.0%
-3.0%
CIFAR-100
CIFAR-100
Tiny Imagenet
CIFAR-10
CIFAR-100 SVHN
0.5k
1k
2k
5k
10k 50k (full)
(2k)
(10K)
Training data size 1000s)
Figure 3: (Left) How should we adversarial examples improve test accuracy? We compare RCAD with adversarial baselines including label smoothing on adversarial samples (LS Adv). measuring the improvement test accuracy relative to ERM. We also compare RCAD with ME-ADA when combined with label smoothing. (Right) RCAD is more effective in low-data regime. We compare the test accuracy improvement over ERM for baselines: Label Smoothing (I.S), Data Augmentation (DA), adversarial training, and ME-ADA with RCAD + LS on different sub-samples CIFAR- 100 training (0.5k 50k). We find RCAD achieves the largest gains the low data regime. In both plots, we plot the mean and 95% confidence intervals over 10 independent runs.
4.1 How much does RCAD improve test accuracy comparison and in addition to other regularization methods?
Figure 2 presents the main empirical findings of RCAD over six image classification benchmarks and across four baseline regularizers. Across all datasets, we observe that training with the RCAD objective improves the test accuracy over the baseline method in 22/24 24 cases. The effects are more pronounced on datasets with fewer training samples. For example, on CIFAR-100-24 adding RCAD on top of label smoothing boosts the performance by = 3%. These results also show that the benefits of RCAD complementury to prior methods- RCADRCAAD MixUp outperforms MixUp. and RCAD + augmentaaiion/smootiig outperforms both. We test for statistical significance using a 1-sided p-value, finding that p & 1e-3 in 22/2 /2 comparisons. In summary, these results show that our proposed regularizer is complementary to prior regularization techniques.
4.2 How effectively does RCAD improve test accuracy compared to adversarial training?
Our next set of experiments compares different ways of using adversarial examples. RCAD maximizes the model's predictive entropy unlabeled examples along the adversarial direction (obtained with a large step-size se 0.5). In contrast, other methods we compare against (Adversarial Training [36], ADA [62]. ME-ADA [62]] minimize the cross entropy loss on examples obtained by adversarially perturbing the inputs without changing the original labels (using a much smaller step size Rs 0.05). Additionally, look at the baseline that performs label smoothing on Adv) - relaxed version of RCAD. We evaluate all methods by measuring their test accuracy improvement over empirical risk minimization (ERM), noting that some of these methods were proposed to optimize robustness, a different metric. We show results in Figure 3a and note that RCAD outperforms adversarial training and LS on Adv with small step-sizes (a 0.02), ADA and ME-ADA by significant margins on all benchmarks. We numerically verify that RCAD statistically outperforms the best baseline ME-ADA by computing 1-sided p-values (p=0.009 on CIFAR-10, < 1e-4 on others).
these adversarial samples (LS
Next, we look at LS on Ady with large 0.5. Since this is the same value of used by RCAD for most datasets, this method is equivalent to performing label smoothing on examples generated by RCAD. Since label smoothing is relaxed form of entropy maximization it is not surprising that the performance of this method is similar to (if not better than) RCAD on a few benchmarks. Finally, when both RCAD and ME-ADA equipped with label smoothing, both methods improve, but the benefit of RCAD persists (p- = 0.0478 on CIFAR-10, p < 1e-4 on others).
4.3 How effective is RCAD in the low training data regime?
Since supervised learning methods are more susceptible to overfitting when training samples are limited [13], we analyze the effectiveness of RCAD in the low-data regime. To verify this, we sample

---

31.
Negutive likelihood (NIJ.) test data
w05n 2010
591
Method
Boston
Concrete
Energy
Wine
Yachi
19
ERM
2.85
3.25
1.85
0.97
1.66
20.06
20.07
40.04
-0.3%
ME-ADA
3.01
3.14
1.99
1.11
1.78
0.8%
2757
+0.04
+0.03
+0.06
+0.05
+0.04
27:8-
RCAD (ours)
2.60
3.12
1.75
1.09
1.54
CIAB300
263
45 cash nls de 333
#0.05
#0.04
#0.07
40.06
#0.00
(b)
(c)
Figure 4: RCAD can be used with larger networks and is also effective regression tasks. (Left) We compare the improvements in test accuracy (over ERM) for RCAD and ME-ADA when all methods use Wide ResNet 28-10 [67]. (Center) For CIFAR-100-2K, we show the effect on test accuracy of the step size a that RCAD takes while generating out-of-distribution examples along the adversarial direction. (Right) We compare the negative log-likclihood test samples for RCAD with basclines ERM and ME-ADA on five regression datasets from the UCI repository. For the above we show the mean and 95% confidence intervals across 10 runs.
small training datasets of size 0.5k, 1k, 2k, 5k and 10k from the 50,000 training samples in CIFAR- 100. The test set for each is the same that of CIFAR-100. We train the baseline regularizers and RCAD on each of these datasets and compare the observed improvements in test accuracies relative ERM. We show our results for this experiment in Figure 3b. Straight away we observe that RCAD yields positive improvements in test accuracy for any training dataset size. In contrast, in line with the robust overfitting phenomena described in Raghunathan adversarial training (with FGSM [14]) is found to be hurtful in some settings. Next we observe that compared to typical regularizers for supervised learning like label smoothing and data augmentation. the regularization effect of RCAD has a stronger impact as the size of the training data reduces. Notice that RCAD has a steeper upward trend (right-left) compared to the next best method label smoothing, while outperforming each baseline in terms of absolute performance values.
al. [48, 47], Zhang et al. [70],
4.4 Additional Results and Ablations
In addition to the experiments below on wider architectures, effect of step size a, and regression tasks, we conduct more analysis and experiments for RCAD (c.g., evaluating its robustness to adversarial perturbations and distribution shifts). For these and other experimental details refer to Appendix B, C.
Results on larger architecture. We compare the test accuracies of RCAD and ME-ADA (most competitive baseline from ResNet-18 experiments) when trained with the larger backbone Wide ResNet 28- 10 [67] (WRN) on CIFAR- 100 and its derivatives. We plot these test accuracies relative to ERM trained with WRN in Figure 4a. Clearly, the benefit of RCAD over ME-ADA still persists albeit with slightly diminished absolute performance differences compared to ResNet- 18 in Figure 3a.
Effect of step size a on test accuracy. In Figure 4b we plot the test accuracy of RCAD on CIFAR- 100-2k as we vary the step size a. We see that RCAD is effective only when the step size is sufficiently large (> 0.5). so that new examples do not resemble the original ones. When the step size is small (< 0.5). the perturbations are small and the new example is indistinguishable from the original, as in adversarial training. Our theory in Section also agrees with this observation.
Results on regression tasks. On five regression datasets from the UCI repository we compare the performance of RCAD with ERM and ME-ADA in terms of test negative log-likelihood (NLL) (4 is better). To be able to compute NLL, we model the predictive distribution Pu. (y x) as a Gaussian and each method outputs two parameters (mean, variance of (y | x)) at each input x. For the RCAD regularizer we use the differential form of entropy. Results are shown in Figure 4c from which we see that RCAD matches/improves over baselines on 4/5 regression datasets.
5 Analysis: Why Does RCAD Improve Test Performance?
In this section, we present theoretical results that provide a more formal explanation as to why we see a boost in test accuracy when we regularize a classifier using RCAD. We analyze the simplified problem of learning lz-regularized linear predictors in a fully specified binary classification setting. Specifically, our analysis will show that, while linear predictors trained with standard ERM can have
7

---

... 0,0000
16A0 Taane (a-52)
1.
300 150 375
200
25
120 325
(1)
(b)
(c)
Figure 5: RCAD corrects ERM solution: (Left) We plot the training data and high-dimensional decision
boundaries for all three estimates Werm and trad with 0.01, 0.70. Here, we plot the linear boundary projected onto the set of planes (x1,x; x;)i=2 222 spanned by the true feature x₁ and noisy features X2,... Xs-
Across the ERM and RCAD training iterations, we plot the weight norms along the true feature |aa and noisy dimensions ||22||2 in the (Center) plot, and on the (Right) we plot the train/test accuracies.
high dependence on spurious features, further optimization with the RCAD objective will cause the classifier to unlearn these spurious features.
The intuition behind our result can be summarized follows: when RCAD generates examples after taking a large step along adversarial directions, a majority of the generated examples end up lying close to the decision boundary along the true feature, and yet a portion of them would significantly depend on the noisy spurious features. Thus, when RCAD maximizes the predictive entropy on these examples, the classifier weights that align with the spurious components are driven toward smaller values. Detailed proof's for the analysis that follows can be found in Appendix A.
Setup. We consider binary classification problem with a joint distribution D over (x,y)
X x {0, 1} where X C [R+1 with x = [x1, x2] X1 E R, X2 E Rd and for some 3 > 0, D is:
y y~ (1,If X1 N(3 fi. X2 ~ NO Eg), and let E
Σ2 (3)
0
We borrow this setup from Chen et al. [7] which analyzes self- eeff-traiiin/eentropy minimization on unlabeled out-of-distribution samples. While their objective is very different from RCAD, their setup is relevant for our analysis since it captures two kinds of features: X1, which is the true univariate feature that is predictive of the label y, and X2, which is the high dimensional noise, that is not correlated with the label under the true distribution D, but w.h.p. is correlated with the label on finite
{x((1, (0))) D We wish to learn the class of homogencous linear predictors W (w Rd+1 (w,x) wi X1 + W2 X2, || < 1]. For mathematical convenience we make three modifications to the RCAD objective in Equation that are common in works analyzing logistic classifiers and entropy based objectives [55, 7]: (i) we minimize the margin loss (w; (x, y)) & max(0, y (w, x)) (instead of the negative log-likelihood) over D' with some > 0; (ii) we consider the constrained optimization problem in Equation 4 as opposed the Lagrangian form of RCAD in Equation [35]; and (iii) we use Lemma 5.1 and replace Ha (x+ V V y))) with exp(-l( kw, x+ a V Vxly (w: (x. I)) Note that the optimal w" e
sampled training dataset D
W that has the best test accuracy and lowest population margin loss given by w -
max MD(w) & ED exp(-I(w,x Volg(w; (x. y)))
ED 1(1 (w:; (x, 0) p/2, W
(4)
where Hrin()) is the entropy of Bern(p)) distribution. Thus Ho R exp( - (w. x) |), as both exhibit same tail behavior.
Lemma 5.1 (16, 55), informal). Hxr Hoin ((1 exp(-(w,
We analyze the RCAD estimate wyead A w
(T) returned after T iterations of projected gradient ascent on the non-convex objective in Equation 4, initialized with w(()) satisfying the constraints in Equation 4. Given the projection IIs onto the convex set S, and learning rate 77 0, the update rule for (t) is as follows: u (t+1)) ((() V (w)), and ((+1) H[ (o (t(+1)
8

---

Since the initialization w satisfies the constraint in Equation 4, it has a low margin loss on training samples and using margin based generalization gaps [27] we can conclude w.h.p. w (0) has low test error (< p). This, in turn tells us that w()) should have learnt the true feature 21 to some extent (Lemma 5.2). Intuitively, if the classifier is not trained at all, then adversarial directions would be less meaningful since they may include a higher component of the true feature X1, as opposed to noisy x2- To obtain w (0), we simply minimize loss L, on D using projected (onto || < 1) gradient descent and the ERM solution Üerm serves as the initialization w())) for RCAD's ' gradient ascent iterations. Note that werm still significantly depends on spurious features (see Figure 5). Furthermore, this dependence is unavoidable and worse when n is small, or when the noise dimension d is large.
Lemma 5.2 (true feature is partially learnt before RCAD). (" satisfies constraint in Equation 4, [ with probability 1 8 over D, w/ orfe- 20remn (55)
and when n z + 166
8
We are now ready to present our main results in Theorem 5.3 which states that after 7 = ((log(1)))) gradient ascent iterations of optimizing the RCAD objective Wread is close w", since it unlearns
the spurious feature (I]te ( 2|2 & E) and amplifies the true feature (fw5") > =2) Note that Theorem 5.3 suggests minimum step size in arriving at sufficient condition for RCAD to unlearn spuriousness and improve test performance This is also in line with our empirical findings (Figure 4b).
Since ((7), most of the points generated by RCAD lie close to the decision boundary along x1. except for the ones with noisy features that are correlated with the classifier weights.
Theorem 5.3 (Wread - "=)) B 22( lop)) and 3 Co, K > O, such that B 2 a 2 max ||Ellop) and y + cu m (Σ)
> 3. then with 2 er Ka// (20mmn
8² +log( 1/6) //1l: [Eil,,
1/g ct, aater T O (log(1/e))
gradient ascent iterations, [0] (T)
1/2 E 6, with probability 5 over D'. v1- and ||m0 (T)
The key idea behind our guarantee above is an inductive argument. We show that, if the class separation | and step size a are sufficiently larger than any noise variance, then RCAD monotonically increases the weight norm along the true feature and monotonically decreases it along the noisy dimensions with each gradient ascent update. This is because at any given instant the gradient of RCAD objective (w) with respect to w1 always points in the same direction, while the one with respect to W2 is sufficiently anti-correlated with the direction of wz. We formalize this argument in Lemma 5.4. It is also easy to see why the update will improve test accuracy monotonically, thus satisfying the constraint in Equation 4 with high probability.
Lemma 5.4 (w), W2 update). If a, B and sample size n satisfy the noise conditions in Theo- rem 5.3, then (MM(ww) (t) > O, and | [(((t])] > (. On the other hand, 3c1 > 0 such that Sue)
Mo (w (1), w/! < -c 1 Consequently 3r,, such that [, 1|0 1|2 < 1. (V w2
Empirically validating our theoretical results in the toy setup. Now, using the same setup as our theoretical study, we check if taking the ERM solution and updating it with RCAD truly helps in unlearning the spurious feature X2 and amplifying the true feature XI. With d = 8,7 = 2.0, B = 0.5, 01 = 0.1, Σ = La we collect training datasct D of size 20 according to the data distribution in Equation 3. First, we obtain the ERM solution werm by optimizing the margin loss on D for 100 projected gradient descent iterations. Then, with Üerm initialization we optimize the RCAD objective in Equation 4 for 100 additional projected gradient ascent iterations to obtain RCAD estimate Wrrad. For RCAD, try both small (a = 0.01) and large (a 0.7) values of the step size. Using the results of this study in Figure 5 we shall now verify our main theoretical claims:
(i) RCAD unlearns spurious features learnt by ERM (Theorem 5.3): RCAD training with large step size a = 0.7 corrects ERM's decision boundaries across all noisy dimensions i.c., wyadd has almost no dependence noisy dimensions {x, J (Figure 5a). On the other hand, the ERM solution has a significant non-zero component across all noisy dimensions (except X7).
(ii) RCAD is helpful only when step size is large enough (Theorem 5.3): In Figure 5a we see that RCAD estimate with small step size a 0.01 has higher dependence on spurious features compared to ERM. Thus optimizing RCAD with a very small a leads to an even poor solution than FRM. This is because, the perturbed data point still has a significant component of the true feature, which the model is forced to unlearn when RCAD tries to maximize entropy over the perturbed point.

---

(iii) The weight norms wwil and ||2l|2 change monotonically through RCAD iterations (Lemma 5.4): In Figure 5b we see that the norm along the true feature [2y] increases monotonically
through RCAD iterations, whereas ||w2||2 decreases monotonically, until |m]] R 1 and |/22|22 R 0. Thus, RCAD successfully recovers optimal w²" Based on this, we also see the test accuracy increase monotonically through RCAD iterations; reaching 100% by the end of it (Figure 5c). In contrast, since the ERM solution depends on spurious features, its accuracy does not improve beyond 70% in the first 100 iterations. We find that training ERM for more iterations only improves training accuracy.
The analysis above explains why RCAD improves test performance in the linear setting and it is inline with our intuition of RCAD's s ability to unlearn spurious features. To check if our intuition also generalizes to deeper networks we further study RCAD's behaviour in a non-linear setting using a different toy example in Appendix D. Furthermore, while our analysis is restricted to linear classifiers, linear models can provide a rough proxy for neural network learning dynamics via the neural tangent kernel (NTK) approximation [24]. This strategy has been used in number of prior works [40, 58, 3] and extending the analysis to the NTK setting may be possible in future work.
6 Conclusion
In this paper we propose RCAD, a regularization technique that maximizes model's predictive entropy on out-of-distribution examples. These samples lie along the directions that adversarially maximize the loss locally around the training points. Our experiments on image classification benchmarks show that RCAD not only improves test accuracy by significant margin, but that it can also seamlessly compose with prior regularization methods. We find that RCAD particularly effective when learning from limited data, Finally, we present analyses in a simplified setting to show that RCAD can help the model unlearn noisy features. Some current limitations of our work are that RCAD slows training by 30% and our theoretical analysis is limited to the linear case, which would be interesting directions to address/expand on in future work particcccllllally in light of the significant empirical benefits of RCAD for improving generalization.
Acknowledgements. The authors would like to thank Oscar Li, Ziyu Liu, Saurabh Garg at Carnegie Mellon University and members of the RAIL lab at UC Berkeley for helpful feedback and discussion.
References
[1] Arjovsky, M., Bottou, L.,, Gulrajani, I., and Lopez-Paz, D. (2020). Invariant risk minimization. stat, 1050:27.
[2] Besnier, V., Bursuc, A., Picard, D., and Briot, A. (2021). Triggering failures: duiti--dist detection by learning from local adversarial attacks in semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15701-15710.
[3] Cao, Y. and Gu, Q. (2019). Generalization bounds of stochastic gradient descent for wide and deep neural networks. Advances in neural information processing systems, 32.
[4] Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, L., Madry, A., and Kurakin, A. (2019). On evaluating adversarial robustness. arXiv preprint 17Xiv:1902.06705.
[5] Carlini, N. and Wagner, D. (2017). Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39-57. IEEE.
[6] Chen, T., Zhang, Z., Liu, S., Chang, S., and Wang, Z. (2020a). Robust overfitting may be mitigated by properly learned smoothening. In International Conference on Learning Representations.
[7] Chen, Y., Wei, C., Kumar, A., and Ma, T. (2020b). Self- self-training avoids using spurious features under domain shift. Advances in Neural Information Processing Systems, 13:21061-21077
[8] Church, K. W. and Gale, W. (1991). A comparison of the enhanced good-turing and deleted estimation methods for estimating probabilities of english bigrams. Computer Speech & Language, 5(1):19-54.
[9] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEFE conference on computer vision and pattern recognition, pages 248-255. lecc.
10

---

[10] DeVries, T. and Taylor, G. W. (2017). Improved regularization of convolutional neural networks with cutout. arXiv preprint (edi::1708.045555
[11] Dubey, A., Gupta, O., Raskar, R., and Naik, N. (2018). Maximum-entropy fine grained classification. Advances in neural information processing systems, 31.
[12] Gao, Y., Wang, W., Herold, C., Yang. Z., and Ney, H. (2020). Towards better understanding of label smoothing in neural machine translation. In Proceedings of the 1st Conference of the Asia- Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 212-223.
[13] Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep learning. MIT press.
[14] Goodfellow, I. Shlens, J., and Szegedy, C. (2014). Explaining and hamnsssing adversarial examples. arXiv preprint arXiv:1412.6572.
[15] Grandvalet, Y. and Bengio, (2004). Semi-supervised learning by entropy minimization. Advances in neural information processing systems, 17.
[16] Grathwohl, W., Wang, K.-.,, Jacobsen, J.-H., Duvenaud, D., Norouzi, M., and Swersky, K. (2019). Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint
[17] Haarojja, T., Zhou, A., Abbeel, P., and Levine, S. (2018). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861-1870 PMLR.
[18] Hardt, M., Recht, B., and Singer, Y. (2016). Train faster, generalize better: Stability of stochastic gradient descent. In International Conference on Machine Learning, pages 1225-1234. PMLR.
[19] He, K., Zhang, X.. Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778.
[20] Heinze-Deml, and Meinshausen, N. (2017). Conditional variance penalties and domain shift robustness. arXiv preprint 17X00 1469.
[21] Hendrycks, D. and Dietterich, T. (2019). Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12266
[22] Hertz, J., Krogh, A., and Palmer, R. G. (2018). Introduction to the theory of neural computation. CRC Press.
[23] Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., and Madry, A. (2019). Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32.
[24] Jacot, A., Gabriel, F., and Hongler, C. (2018). Neural tangent kemnel: Convergence and generalization neural networks. Advances neural information processing systems, 31.
[25] Jaynes, E. T. (1957). Information theory and statistical mechanics. Physical review, 106(4):620.
[26] Johnson, W. E. (1932). Probability: The deductive and inductive problems. Mind, 41((64):409- 423.
[27] Kakade, S. M., Sridharan, K., and Tewari, A. (2008). On the complexity of linear prediction: Risk bounds, margin bounds, and regularization. Advances in neural information processing systems, 21.
[28] Kalimeris, D., Kaplun, G., Nakkiran, P., Edelman, B., Yang, T., Barak, and Zhang, H. (2019). Sgd on neural networks learns functions of increasing complexity. Advances in Neural Information Processing Systems, 32:3496-3506.
[29] Koltchinskii, and Panchenko, D. (2002). Empirical margin distributions and bounding the generalization error of combined classifiers. The Annals of Statistics, 30(1):1-50.
[30] Krizhevsky, A. et al. (2009). Learning multiple layers of features from tiny images. Citeseer.

---

[31] Krogh, A. and Hertz, J. A. (1992). A simple weight decay can improve generalization. In Advances neural information processing systems, pages 950-957.
[32] Kschischang, F. R. (2017). The complementary error function. Online, April.
[33] Kurakin, A., Goodfellow, I,, and Bengio, S. (2016). Adversarial machine learning scale. arXiv preprint 1.t::1111.0130
[34] Le, Y. and Yang, X. (2015). Tiny imagenet visual recognition challenge. CS 231N,, 7(7):3.
[35] Li, D. (1995). Zero duality gap for class of nonconvex optimization problems. Journal of Optimization Theory and Applications, 85(2):309-324.
[36] Madry, A., Makelov, A., Schmidt, L.., Tsipras, D., and Vladu, A. (2017). Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 770.00683.
[37] Miyato, T., Maeda, S.-i., Koyama, M., and Ishii, S. (2018). Virtual adversarial training: regularization method for supervised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 1188):1979-1993 1993.
[38]
S.-M., Fawzi, A., and Frossard, P. (2016). Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2574-2555 -582..
[39] Müller, R., Kornblith, S., and Hinton, G. (2019). When does label smoothing help? arXiv preprint 1rXiv:1906.02662
[40] Nagarajan, V. and Kolter, Z. (2019). Uniform convergence may be unable to explain generalization deep learning. Advances in Neural Information Processing Systems, 32.
[41] Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A. Y. (2011). Reading digits in natural images with unsupervised feature learning. Citeseer.
[42] Odena, A. (2016). Semi-supervise learning with generative adversarial networks. arXiv preprint arXiv:1606.01588
[43] Pereyra, G., Tucker, G., Chorowski, J., Kaiser, t.,, and Hinton, G. (2017). Regularizing neural networks by penalizing confident output distributions. arXiv preprint S.:iv/7701.0666
[44] Peters. J., Bühlmann, P., and Meinshausen, N. (2016). Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Sociery. Series B (Statistical Methodology), pages 947-1012.
[45] Pinto, F., Yang, H., Lim, S.-N.. Torr, P., and Dokania, K. (2021). Mix-maxent: Improving accuracy and uncertainty estimates of deterministic neural networks. In NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications.
[46] Poursaeed, O., Jiang, T., Yang, H., Belongie, S., and Lim, S.-N. (2021). Robustness and generalization via generative adversarial training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 15711-1572)
[47] Raghunathan, A., Xie, S. M., Yang, F., Duchi, J., and Liang, P. (2020). Understanding and mitigating the tradeoff between robustness and accuracy. International Conference on Machine Learning, pages 7909-7919. PMLR.
[48] Raghunathan, A., Xie, S. M., Yang, F., Duchi, J. C., and Liang, P. (2019). Adversarial training hurt generalization. arXiv preprint arXiv: 000.-99066....3
[49] Rockafellar, R. T., Uryasev, S., et al. (2000). Optimization of conditional value-at-risk. Journal of risk, 2:21-42.
[50] Sabour, S., Cao, Y., Faghri, F., and Fleet, D. (2016). Adversarial manipulation of deep representations. In ICLR (Poster).
12

---

[51] Sagawa, S., Raghumathan, A., Koh, P. W., and Liang, P. (2020). An investigation of why overparamettrizzatoo exacerbates spurious correlations. In International Conference on Machine Learning, pages 8346-83365 PMLR.
[52] Saito, K., Kim, D., Sclaroff, S., Darrell, T., and Saenko, K. (2019). Semi-supervised domain adaptation via minimax entropy. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 8050-
[53] Shorten, C. and Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(1):1-48.
[54] Singla, S. and Feizi, S. (2021). Salient imagenet: How discover spurious features in deep learning? In International Conference on Learning Representations.
[55] Soudry, D., Hoffer, E., Nacson, M. S., Gunasekar, S., and Srebro, N. (2018). The implicit bias of gradient descent on separable data. The Journal of Machine Learning Research, 19()):2222-2878
[56] Szegedy, C., Vanhoucke, V., loffe, S., Shlens, J., and Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818-2826.
[57] Tamar, A., Glassner, Y., and Mannor, S. (2015). Optimizing the evar via sampling. In Twenty- Ninth AAAI Conference on Artificial Intelligence.
[58] Tancik, M., Srinivasan, P., Mildenhall, B.. Fridovich-Kan Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J., and Ng, R. (2020). Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems, 33:7537-7547.
[59] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1 286)22677-
[60] Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., and Madry, A. (2018). Robustness may be at odds with accuracy. arXiv preprint 1805::8805.22555
[61] Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., and Madry, A. (2019). Robustness may be at odds with accuracy. arXiv: Machine Learning.
[62] Volpi, R., Namkoong, H., Sener, O., Duchi, J., Murino, V., and Savarese, S. (2018). Generalizing to unseen domains via adversarial data augmentation. arXiv preprint arXiv: 12018.
[63] Wainwright, M. J. (2019). High-dimensional statistics: A non-asymptotic viewpoint, volume 48. Cambridge University Press.
[64] Wang, D., Shelhamer, E., Liu, S., Olshausen, B., and Darrell, T. (2020). Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2066.1072
[65] Wang. X., Jiang, Y., Huang, M., and Zhang. H. (2013). Robust variable selection with exponen- tial squared loss. Journal of the American Statistical Association, 108(502):632 643.643.
[66] Yun, S., Han, D., Oh, S. J., Chun, S., Choe, J., and Yoo, Y. (2019). Cutmix: Regularization strategy train strong classifiers with localizable features. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 6023-6032.
[67] Zagoruyko, S. and Komodakis, N. (2016). Wide residual networks. arXiv preprint crXiv:1605.07111
[68] Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O. (2021a). Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 6473):107-115.
[69] Zhang, H., Cisse, M., Dauphin, Y. N., and Lopez-Paz, D. (2018). mixup: Beyond empirical risk minimization. In International Conference on Learning Representations.
13

---

[70] Zhang, H., Yu, Y. Jiao, J., Xing. E., El Ghaoui, L.,, and Jordan, M. (2019a). Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pages 7472-7482. PMLR.
[71] Zhang. J. He, T., Sra, S., and Jadbabaie, A. (2019b). Why gradient clipping accelerates training:
A theoretical justification for adaptivity. arXiv preprint arXiv:I905.1 18811.
[72] Zhang, Y., Gong, M., Liu, T., Niu, G., Tian, X., Han, B., Schölkopf, B., and Zhang, K. (2021b). Adversarial robustness through the lens of causality. arXiv preprint arXiv:2106.06/
[73] Zhao, L., Liu, T., Peng. X., and Metaxas, D. (2020). Maximum-entropy adversarial data augmentation for improved generalization and robustness. Advances in Neural Information Processing Systems,
144477
[74] Zhou, A. and Levine, S. (2021). Training on test data with bayesian adaptation for covariate shift. arXiv preprint urXiv:2109. 12746.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Sec. 6.
(c) Did you discuss any potential negative societal impacts of your work? [No] While su- pervised learning might be used in applications with both positive and negative societal
impacts, our empirical and theoretical contributions do not dictate these potential appli- cations.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] We provide them in the Appendix.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We document these in the Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] For our results in Section 4, we provide error bars for the estimated test performance.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Details Appendix.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We use public datasets and cite the creators.
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either the supplemental material or as URL.? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you't 're
using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
14

---

(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
15


Reviews:
[{"review_id": "xD9cONEUN_f", "reviewer": "Reviewer_6c8K", "summary": "This paper proposes to increase the test accuracy by maximizing the entropy of out-of-distribution examples that are adversarially generated, which is named reducing confidence along adversarial directions (RCAD). Empirical results support the claim that RCAD can improve test accuracy. In addition,  the authors theoretically analyze the effectiveness of RCAD on a linear case. \n", "questions": "+ Could I know the influence of the hyperparameter $\\lambda$, since it seems that an important hyperparameter in RCAD?\n", "limitations": "The authors have pointed out the limitation of their proposed method. I have no idea about how to accelerate RCAD so far.\n", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n+ This paper is well written and organized. I can easily follow.\n+ The proposed method is easy but effective in improving test accuracy. The proposed method is an interesting case that leverages adversarial attacks for good. \n+ Experiments are comprehensive and sound. The results validate that RCAD can be compatible with various methods and indeed enhance the test accuracy. \n+ The theoretical results explain why RCAD help test accuracy to some extent (although I have not carefully checked the correctness of theoretical results) — RCAD can regularize the model to unlearn the spurious features. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "mebmQRVJUx8", "reviewer": "Reviewer_cq35", "summary": "The paper proposes RCAD, a simple regularization scheme for predictive models that aims to increase the predictive entropy for OOD points. OOD points are generated adversarially using FGSM with a large step size. The authors demonstrate that the method reliably improves the test accuracy on a set of image classification tasks, as well as tabular regression tasks. The improvement is more significant when less data is available. To understand why the method works, the authors study it on a toy linear classification task.", "questions": "- Does the \"sweet spot\" for the $\\alpha$ value that we see in Figure 4b change significantly with the dataset? Would be useful to understand how tricky it is to get this parameter \"right\" for a novel task.\n- Have authors considered the relation of the method to max margin classifiers like the SVM? I wonder if the two approaches recover similar decision boundaries. A comparison on the toy dataset would be interesting.", "limitations": "Authors discuss the impact of the additional regularization term on training time, as well as the assumptions made by the analysis. I consider the potential negative societal impact of this work to be limited.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The method is clearly introduced, and carefully compared to prior work.\n- Statistically significant improvement in test accuracy (up to 3 percentage points) across the benchmarks (image classification and regression on UCI data) when RCAD is employed, compared to multiple strong baselines.\n- Ablations to understand the impact of the adversarial step size and model architecture.\n- Theoretical analysis for the linear case that motivates the method.\n\nWeaknesses:\n- The idea to use adversarial optimization to find OOD points is of limited novelty (see \\[a\\]). To the best of my knowledge, however, it has not been previously studied theoretically, or evaluated on such a broad set of benchmarks.\n- The analysis on the linear case is sometimes hard to follow. Consider moving more detail into the appendix, to give the narrative more breathing space.\n\nReferences:\n- \\[a\\]: Besnier, V., Bursuc, A., Picard, D., & Briot, A. (2021). Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 15701-15710).\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "iGD2qWm9S5k", "reviewer": "Reviewer_wyGm", "summary": "This paper proposes a regularization method for improving the generalization in maximum likelihood learning. The main idea is to reduce confidence (or increase entropy) on inputs generated by large adversarial perturbation. The authors conduct extensive experiments to demonstrate the effectiveness of their method. They also analyze the effectiveness of their method by showing it can unlearn noisy features in a toy setting.", "questions": "The authors compare their method with baselines including those from adversarial training such as FGSM, and those from domain adaptation such as ADA and ME-ADA. But as far as I am concerned, none of these methods is specifically designed for standard in-domain generalization. FGSM is for improving adversarial robustness while ADA and ME-ADA are for domain generalization. Why didn't the authors compare their method to VAT [1], which also proposes to perform data augmentation with adversarial perturbation, and aims for generalization in standard supervised and semi-supervised settings?\n\n[1] Virtual adversarial training: a regularization method for supervised and semi-supervised learning. Miyato et al., 2018.", "limitations": "The limitations have been properly discussed in Section 6.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper is overall well-organized and clearly written. As the main contribution of this paper is to propose a new regularization method, the novelty might be minor since the proposed method resembles the existing methods in adversarial data augmentation [1,2,3], albeit the difference that this paper proposes to \"unlearn\" examples under large adversarial perturbation instead of \"learn\" examples under small adversarial perturbation. \n\nMy biggest concern in this paper is that the experiment results may not be sufficient to demonstrate the effectiveness of the proposed method. Since the authors aim to improve generalization in a very general sense instead of robustness against perturbation or domain shift, experiments on large datasets such as ImageNet would be necessary. I am not quite certain about the effectiveness of the proposed method in such a more realistic setting as the improvements on full datasets such as CIFAR-10 and CIFAR-100 are already marginal. The authors also emphasize that the proposed method is most effective in low-data regimes. In this regard, I believe it would be necessary to showcase the proposed method in a semi-supervised setting with unlabeled data available. \n\n[1] Virtual adversarial training: a regularization method for supervised and semi-supervised learning. Miyato et al., 2018.\n\n[2] Generalizing to unseen domains via adversarial data augmentation. Volpi et al., 2018.\n\n[3] Maximum-entropy adversarial data augmentation for improved generalization and robustness. Zhao et al., 2020.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Xq1hI-Nxb8T", "reviewer": "Reviewer_uiAQ", "summary": "This paper proposes a new regularization technique for deep neural networks by reducing confidence on self-generated examples that lie along directions adversarially chosen to increase training loss. The proposed regularizer can be easily integrated into training pipelines and added to existing techniques to gain 1-3% improvement in test accuracy.", "questions": "1. Justify the way to generate out-of-distirbution examples.\n2. Check how vulnerable the trained model is to adversarial attacks.\n3. Discuss the difference from contrastive learning with data augmentation.\n4. Running time comparison.\n5. Comparison with existing regularizers.", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n\nThis paper identifies a new inductive bias to reduce the confidence on out-of-distribution inputs and turns this bias into a new regularizer for training. Using adversarial direction to guide the training process is an interesting idea. The empirical evidence demonstrates the effectiveness of the proposed method, and the theoretical analysis also presents some theoretical guarantees on generalization gap. Some interesting insights are also provided with experimental validations.\n\nWeaknesses:\n\n1. The approach of generating out-of-distribution examples from large stepsize adversarial examples needs to be justified. Why is it a proper way to generate such examples? How is it compared with other generation methods, e.g., random samples in other classes?\n\n2. It would be better to point out that the objective is not to improve adversarial robustness although adversarial examples are employed for training. This could help avoid confusion. Although the proposed method does not try to robustfy the model, it is unclear if the proposed method improves the test accuracy at the cost of the adversarial robustness. Corresponding experiments could be designed to address this concern.\n\n3. The connection of the proposed method to some other ideas such as data augmentation and contrastive learning could be also discussed and compared if relevant.\n\n4. Running time comparison could be added to see how much cost paid for the performance gains.\n\n5. It would be also interesting to see how the proposed regularizer is compared with the existing ones with respect to generalization performance.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept