Learning Sample Reweighting for Adversarial Robustness
Authors: Chester Holtz
Tsui-Wei Weng
Gal Mishne
Venue: ICLR 2022 Submitted (2022)
Keywords: deep learning
adversarial attack
robust training

Abstract: There has been great interest in enhancing the robustness of neural network classifiers to defend against adversarial perturbations through adversarial training, while balancing the trade-off between robust accuracy and standard accuracy. We propose a novel adversarial training framework that learns to reweight the loss associated with individual training samples based on a notion of class-conditioned margin, with the goal of improving robust generalization. Inspired by MAML-based approaches, we formulate weighted adversarial training as a bilevel optimization problem where the upper-level task corresponds to learning a robust classifier, and the lower-level task corresponds to learning a parametric function that maps from a sample's \textit{multi-class margin} to an importance weight. Extensive experiments demonstrate that our approach improves both clean and robust accuracy compared to related techniques and state-of-the-art baselines. 

Under review as a conference paper at ICLR 2022
LEARNING SAMPLE REWEIGHTING FOR ADVERSARIAL ROBUSTNESS
Anonymous authors Paper under double-blind review
ABSTRACT
There has been great interest in enhancing the robustness of neural network classi- fiers to defend against adversarial perturbations through adversarial training, while balancing the trade-off between robust accuracy and standard accuracy. We propose a novel adversarial training framework that learns to reweight the loss associated with individual training samples based on a notion of class-condiiioeed margin, with the goal of improving robust generalization. Inspired by MAML-basee ap- proaches, we formulate weighted adversarial training as bilevel optimization problem where the upper-level task corresponds to learning robust classifier, and the lower-level task corresponds learning a parametric function that maps from sample's multi-class margin to importance weight. Extensive experiments demonstrate that our approach improves both clean and robust accuracy compared to related techniques and state-of-the-ard baselines.
INTRODUCTION
While neural networks have been extremely successful in tasks such as image classification and speech recognition, recent work (Szegedy et al. (2014) Goodfellow al. 2015) has demonstrated that neural network classifiers can be arbitrarily fooled by small, adversarially-chosen perturbations of their input. Notably, [Su et al. (017) demonstrated that neural network classifiers which can correctly classify "clean" images may be vulnerable to targeted attacks, e.g., misclassify those same images when only single pixel is changed.
Recent work has shown a common failing among techniques that uniformly encourage robustness. In particular, there exists an intrinsic tradeoff between robustness and accuracy (Zhang et al. 2019). Bao et al. (020) investigate this tradeoff from the perspective of cassification-callibra
loss theory. Rice et al. (2020) empirically showed that during adversarial training, networks often irreversibly lose robustness after training for short time. They dubbed this phenomenon adversarial overfitting while proposing early stopping as a remedy. The significance of label noise and memorization the context of adversarial overfitting was demonstrated in [Sanyal et al. (202T) in particular that poor training samples induce fragility to adversarial perturbations due to the tendency of neural networks interpolate the training data. Methods based on weight and logit smoothing have been proposed as an alternative to early stopping (Chen et al. 2021 Cohen et al. 2019 Salman et al., 2019).
In another line of work, Geometry-Aware Instance Reweighted Adversarial Training (GAIRAT: Zhang et al. (2021)), Weighted Margin-aware Minimax Risk (WMMR: Zeng et al. (2021), and Margin-
Aware Instance reweighting Learning (MAIL; Wang et al. 202T were developed address adver- sarial overfitting by controlling the influence of training examples via importance / loss weighting. Intuitively, the samples assigned low weight correspond to samples on which the classifier already sufficiently robust. However, existing methods rely on rigid approximations of the margin and employ heuristic weighting schemes that rely on careful choices of hyperparameters.
This work builds upon these observations. We present BiLAW (Bilevel Learnable Adversarial reWeighting), an approach that explicitly learns a parametric function (e.g. represented by a small feed-forward network) that assigns weights to the loss suffered by classifier, associated with individual training samples. The sample weights are learned as a function the classifier multiclass margins of the sample, according the weights' effect robust generalization. We employ bi-level optimization formulation Bracken & McGill (1973) and leverage a validation set, where the upper-level objective corresponds to learning the parameters of robust classifier, while the

---

Under review as a conference paper at ICLR 2022
lower-level objective corresponds to learning function that predicts sample weights that improve improve robustness on a validation set. Our approach alternates between iteratively updating the parametric sample weights and updating the classifier network parameters.
Contributions We note that while sample weighting (Zhang et al. 2021] et al. 2021 Wang et al. [021] has been investigated in the context of robust training, far know this is the first work to explore a method to use a validation set to learn weights to induce robust generalization. Our contributions consist of the following:
1. Our approach performs sample re- re-weighting during training. We propose a bilevel optimiza- tion formulation to learn a mapping from multi-class margins to weights according to the robust loss suffered by classifier on validation set.
2. We show that when our weighting function corresponds to a neural network, the magnitude of a sample's 's weight directly corresponds to the vulnerability of the classifier at that sample.
3. We evaluate the practical performance of BiLAW on MNIST (LeCun & Cortes 2010), F- MNIST (Xiao et al.) 2017), CIFAR-10, and EIFAR-100. (Krizhevsky et al.) and demonstrate that improves clean accuracy up 6% and robust test accuracy by up 5% compared to TRADES and other state-of-the-art sample reweighting methods on CIFAR-10.
This paper is organized as follows. In Section 2 we review the notation and background of cost-aware and robust classification. In Section we describe our method in full, including the sample weighting method. In Section 4 we provide ablative experiments and demonstrate the efficacy of our framework by comparing clean and robust test performance on MNIST, F-MNIST, CIFAR-10, 10, and CIFAR-100 to adversarial training and two recent, state-of- -fate-of-the-ann the-art sample weighting methods.
PRELIMINARIES AND RELATED WORK
In this section, we briefly present background terminology pertaining to adversarially robust classifi- cation, sample reweighting and bilevel optimization.
Notations An ReLU network is neural network such that all nonlinear activations ReLU functions, where we denote the ReLU activation by a IR + P. (xx) = max (0, Informally, we define IRd Rdd by (x) (ar)), ............ a (ra)). Let So Rd - [0, 1] be a feedforward ReLU network with hidden layers and weights 0: for example. f may map from a d-dimensional image to a k-dimensional vector corresponding likelihoods for k classes.
Given a training set of m sample-label pairs (z;, M:) drawn from a training data distribution D, we associate a weight wa with each training sample. Informally, these weights characterize the effect of the sample on the generalization of the network (i.e. samples with large weights promote robust generalization and visa versa). Given loss function É RR RR + R. we denote the empirical weighted training loss suffered by network with parameters 0 on m training samples with weights w to Car w) = =Enh wrK(y;; fo():)) such that w; 2 0 and Σ, w; = 1. For brevity, we write (0) = (yv, fo.)))). Additionally, if w is left unspecified, L corresponds to the unweighted mean over empirical losses. Likewise, the unweighted validation loss of n samples is denoted (0).
2.1 ROBUST CLASSIFICATION AND ADVERSARIAL OVERFITTING
Consider the network fo Rd + R where the input is d-dimensional and the output is a k- dimensional vector of likelihoods, with j-th entry corresponding to the likelihood the image belongs to the j-th class. The associated classification is then c(x) = arg maxje(1.M [x] (1). (zr). In adversarial machine learning, we are not just concerned that the classification be correct, but we also want to be robust against adversarial examples, i.e. small perturbations to the input which may change the classification to an incorrect class. We define the notion of e-robustness below:
Definition (c-robust) f9 called e-robust with respect to norm p at x if the classification is consistent for small ball of radius around
cxx + 5) = c(x), 48 110 1p Se.
2

---

Under review as a conference paper at ICLR 2022
Note that the c-robustness of fa z: is intimately related to the uniform and local Lipschitz smoothness of fo around r. Recall that a function ff has finite, global Lipschitz constant with respect to norm 11. if
TL 2 0 s.t. If(x) - f(z') S L --11=- - I'11, VI, E X.
(2)
An immediate consequence of Eq. [ and Fqg is that if So is uniformly I--Lipschizz then fo is e-robust with 6 27 (P. Pb) where Pa is the likelihood of the most likely outcome, and P. is the likelihood of the second most likely outcome (Salman et al., (019). The piecewise linearity of ReLU networks facilitates the extension of this consequence to the locally Lipschitz regime. L corresponds to the norm of the affine map characterized by f conditioned on input z. These properties were previously used to characterize the robustness of network a sample (and the weight associated with the sample) (Zeng et al., [021 Yi et al. [2221] Wang et al., [021).
The minimal p norm perturbation 6p required to switch an sample's label given by the solution to the following optimization problem:
0 - arg min ||0| s.t. c()) + c(x 6).
significant amount of existing work relies on a first-order approximations and Hölder's 's inequality to recover 5", justiffying the popularity of inducing robustness by controlling global and local Lipschitz constants. More concretely, given a l norm and radius c, a typical goal of robust machine learning is learn classifiers that minimize the robust loss on a training dataset:
max Jolx +50 [1811
min (Z,))DD
For brevity we will denote the robust analogue of a loss as L, indicating this is the robust counterpart of C, differentiated by the "inner" maximization problem.
2.2 MARGIN-AWARE REWEIGHTING
In the framework of cost-sensitive learning, weights are assigned the loss associated with individual samples. The goal of cost-sensitive learning is to minimize the empirical weighted training loss:
min Ca w) = weL,(0).
#
Previous work in cost-sensitive learning for adversarial robustness Zhhag et al. 2020 2021 Zeng et al., 2021) typically substitutes the robust loss Êu for Lu and largely focuses on designing heuristic functions of various notions of margin to use for the sample weight w/-
For example, in GAIRAT (Zhang et al. 2021 2020), the margin is defined as the least number of PGD steps, denoted k, that Icads the classifier to make an incorrect prediction. The sample's 's weight is computed as WGGIRAT( (z:) = 3 (1 + tanh A + 5(1
(()) with hyperparameters K and
X. A small indicates that the sample lies close to the decision boundary. Larger x values imply that associated samples lie far from the decision boundary, and are therefore more robust, requiring smaller weights. However, due the non-linearity of the loss-surface in practice, PGD-based attacks with finite iterations may suffer from the same issues that plague standard iterative first-order methods in non-convex settings. In other words, x is heavily dependent on the optimization path taken by PGD. This is demonstrated by GAIRAT's vulnerability on sophisticated attacks, e.g. AutoAttack (Croce & Hein 2020b).
Zhang et al. (2020) define the margin as the difference between the loss of a network suffered at clean sample and its adversarial variant. Zeng et al. (202T); Wang et al. (202T) propose definition of margin corresponding to taking differences between logits, as follows.
Wang et al. 2021) The margin of a classifier fo on sample (x4, y1) the difference between the classifier's confidence in the correct label yi and the maximal probability of an incorrect label t, margin fo. 24, y:) p(f(x:)) = yt) - max pofoxy.P(fo(z.) = t).
Definition 2 (Zeng et al. 2021:
Given this definition, Zeng et al. (202]): Wang et al. (202T) propose to use an exponential (WMMR) and sigmoidal (MAIL. functions respectively: WWMMR (x:) exp(-am) with parameter o, and
3

---

Under review as a conference paper at ICLR 2022
WMAIL (2;) sigmoid(- - 3)) with parameters y and B. WMMR and MAIL rely on the local linearity of ReLU networks and the fact that for samples near the margin, the relative scale of predicted class- kass-likelihoods directly corresponds to the distance to the decision boundary. However, similarly to GAIRAT's R, even for samples very close to the decision boundary, simple functions of the difference between class likelihoods may not necessarily correspond to the true distance the decision boundary. Here we propose more fine-grained notion of margin, the multi-class margin, and design a method to learn a mapping between the margin at sample and its associated weight.
Previous work has explored theoretical notions of a multi-class margin. For example, Zou (0005) defined the margin vector in the context of boosting as proxy for a vector of conditional class probabilities. However, this notion of margin is unaware of the true class of a sample. In contrast, the multi-class margin proposed by Saberian
Vasconcelos) (2019;; Cortes et al. (203)) are both closely related to Wang et al. (202T): Zeng et al. (2021), i.e. defined as the minimal distance between an arbitrary predicted logit and the logit of the true class.
In Fig. D we explore the relationship between the logits of a network evaluated at clean sample and the predicted class of the adversarially perturbed variant. Methods which rely on the canonical notions of margin reasonably assume that samples at which classifier is vulnerable have small margin according to Def. 2 i.c. the magnitude of the smallest difference between the logits of any class and the logit corresponding the true class is small. However, we demonstrate in Fig. [Tb) for both vulnerable and robust classifiers that while the majority of predictions made by networks on adversarial samples do correspond to the classes with minimal margin, significant number do not. In other words, the class for which the margin is smallest does not always correspond to the adversarial class. Furthermore, this issue exacerbated for robust networks shown by the difference in count distribution between networks whose relative robustness varies.
2.3 BI-LEVEL OPTIMIZATION AND META-LEARNING
Bilevel optimization, first introduced by Bracken & McGill (973) is a framework for mathematical optimization involving nested optimization problems. A typical bilevel optimization problem takes on the following form:
min (xx) := f(x, y (x)) s.t. y E Earg min g(x, v), RERP
(3)
yER*
where f and g are respectively denoted the upper-level and lower-level objer-cttveeeeeeeeeee.. The goal of the framework is to minimize the primary objective (xx) with respect to x where y (z) is obtained by solving the lower-level minimization problem. The framework of bilevel optimization has seen wide adoption by the machine learning community-in particular in the context of hyperparameter tuning (Jenni & Favaro 2018 Ren al., 2018) and meta-learning (Finn al., 2017 Rajeswaran al. 2019). Our proposed algorithm has some similarity to the meta-learning literature (Finn et al. 2017] Rusu et al. 2019, Rajeswaran et al. 2019 Eshratifar et al. 2018). Most notably, the Model-Agnostic Meta-Learning MAML) algorithm by Finn et al. (2017) incorporates gradient information for the meta-learning setting. The application of meta-learning as an instance of bilevel optimization has been explored in the context of sample reweighting. In particular, Ren al. (2018); ;; Jenni & Favaro (2018); (hu et al. (2019) proposed methods to address learning with noisy labels by reweighting the gradients associated with the losses at individual samples based on balancing performance curated validation set and the corrupted training set.
3 BILAW: LEARNING SAMPLES WEIGHTS FOR ADVERSARIAL TRAINING
In this section, we propose BiLAW, a new learning framework for robust training. There are 2 main novelties in our new learning scheme compared to existing robust training methods: First, we consider a more reasonable assumption leveraging the concept of multi-class margin in robust training, where good weights should be aware of both the margin associated with each class, as well as the true class associated with the sample. Second, as opposed to related work which defines an explicit formula for the weights dependent on the margin, we propose to learn the weights as part of training the classification model. Specifically, we define the weights and parameterize this function using a small auxiliary network. We formulate this as a bi-level optimization problem and learn the weights iteratively alongside the neural network parameters.
function of a multi-class margin,
4

---

Under review as a conference paper at ICLR 2022
(a)
(b)
Class
Decision boundary
A
margin
4004
Class
2014
Decision boundary
Class 3
(a) Diagram of multiclass margin. Larger samples denote samples that should be assigned large weight, e.g., are misclassified or close to the decision boundary. Green (red) arrows denote entries in the multiclass margin vector for a correctly (incorrectly) classified sampled. (b) Sorted logit order and frequency of adversarial classification. Number of instances where the prediction of an adversarial sample corresponds to its i-th largest logit in CIFAR-10 (ignoring the 0-th logit/samples for which the prediction does not change). Colors represent the perturbation budget used during adversarial training (i.e. varying degrees of robustness). Perturbations are computed using & PPGD with 10 iterations and maximum budget of 0.031.
Figure 1:
3.1 MULTI-CLASS MARGIN REWEIGHTING
We extend the logit-based definitions of margin applied in Zeng al. (202T): Wang et al. (2021 and define the multi-class margin of a classifier at sample as follows.
The multi-class margin of a classifier fo on sample (I4, y.). denoted A [o, 1] [-1,1] is a k-dimensional vector whose j-th entry, (f)(()) y.), is the difference between the classifier's 's confidence in the correct label yi and the classifier's confidence in label (Joz))) si) p(Jo(1:)))))) = yi) -[[o(xa)= j).
Definition 3
For brevity we denote A ((o( as A. Note that the multi-class margin exhibits two qualities:
1. Correct/incorrect classification is implicit in the definition, as negative values indicate incorrect classification.
2. The true class of the sample also implicit-i.e. the index with element zero (assuming the sample does not lie exactly on decision boundary separating the true class from another).
In particular, we highlight the second quality. Prior work has demonstrated that the distribution of predictions made adversarial samples is not necessarily uniform over all classes (Abbasi & Gagné] 2017). In other words, vulnerable samples and their associated adversarial perturbations may concentrate about certain classes more than others. We demonstrate in the results that networks exhibit non-uniform robustness per-class.
To learn the sample weights function of the multiclass margin, we construct an auxiliary neural network with a single hidden layer, whose parameters are denoted μ and whose inputs are the multi- class margins. The weight of the i-th training sample then computed as w; = wn (A.). In general, we denote the function used to map from margin to weight w..). A question that arises is what loss function should be used train this auxiliary network. We design a bilevel optimization approach leveraging the validation set to learn the auxiliary network parameters μ.
3.2 BILEVEL OPTIMIZATION
We exploit a validation set to jointly learn a parametric weighting function wp on the train- ing samples and classifier which jointly minimize the associated weighted robust error. Let (0, w)) Em (,,)((()), where w1,s = Wp. (A,) be the weighted robust training loss with respect parameters 0, and μr at time t. Additionally, We, and Dmh Way 1. Intuitively, the
5

---

Under review as a conference paper at ICLR 2022
samples with high weights should improve robust generalization this is quantified by the robust error evaluated on held-out validation set. Let (0.) a -i=r( (0.) be the unweighted robust validation loss associated with dz. Following the meta-learning principle, we seek weights such that
the minimizer of the weighted robust training loss maximizes robust accuracy on the unweighted validation set-i.e. solve the following bilevel optimization problem:
arg min (0, w0p (A)) s.t. j E arg min (0)
(4)
8
The workflow the reweighting procedure is illustrated in Fig. A and the reweighting algorithm is presented in Alg. [
We provide high-level overview of the procedure below.
Iteration t: Update parametric weights w(x)
Iteration Update network parameters
Step
Meta-update
Step 3: 6 GD(9
6 GDID, a.M).
Mir
Step 2: He MAME B.)
Figure 2: BiLAW Framework. (A) MAML-inspired sample weighting. Step 1: intermediate
parameters 0, are computed by pseudo-update of 0,, Step 2: Validation loss gradients (calculated via back- boock--rppagttion through the weighted training loss) are used to update the parameters of the auxiliary weighting network faz. (B) Step 3: network parameters 0, updated using new weights wt-
Our approach is composed of three steps. Steps and 2 rely on the MAML-trick (Finn et al. 2017 which substitutes one-step updates μr for μæ" and iteratively solves the upper-level problem. In this context, jt is updated according to the gradient of the unweighted robust validation loss with respect the sample weights. We note that this method necessitates computation of aveudo-upatte in order to compute this gradient:
Step 1 Pseudo update of classifier parameters 0, (Step 1 in Fig. line 5 in Alg.
0 = 8 - PV Q w;-1)
(5)
The pseudo parameters 0, are then used as surrogate for 0, in optimizing for ju: Step 2 Update parameters μr of the auxiliary network (Step 2 in Fig, line 6 in Alg.
m aB
de (0) ae
Ow
14 14-1 mn
(6)
ae
where a and 3 are the step size used in the pseudo and auxiliary network updates, respectively. Step 3 Update parameters of classifier network (Step 3 in Fig. 2 line 8 Alg. T
0z+1 02 BV V. we)
One interpretation of this procedure that we take pseudo-step using 6:-1 and Pt-1 (Step 1), calculate the best auxiliary network parameters Mr in hindsight that improve generalization, by minimizing the validation loss with 0,, (Step 2), and then derive the "real" update for 0,- by
minimizing the weighted training loss using the new weights μu (Step 3). The detailed derivation the
ae:4 ())
gradient update is provided in the appendix. Note that the term
0z-1
in Eq. (6) represents the correlation between the gradient of the j-th training sample computed on the training loss and the average gradient of the validation data calculated on the robust validation loss. As a consequence, the gradient of the loss with respect to the network parameters at time t for training sample j is aligned with the average gradient of the meta-loss, it will be considered beneficial sample for generalization and its weight will be increased. Conversely, the weight of the sample suppressed if the gradient is anticorrelated with the average validation set-gradient.

---

Under review as a conference paper at ICLR 2022
Algorithm 1 BiLAW training procedure
Input: Training data D, validation-data set D. max iterations 7, learning rates a, 8 Output: Classifier parameters 0
1: (
2: Initialize Ov. nv. wo = WHo (4)
3: for t S do
4: (X, v) ~ D. (X,
5:
0g +- 0 - 3. VoLulo
6:
ML+1 =
aV Luaal le,,us
compute V VaCmle,.. via backpropagation according to Eq.
7:
compute W1+1 = W (A)
> compute A with respect to 8 according to Def
8: Dz+1 = Oc - BV Me, 11
9:
end for
10:
return Pr
4 EXPERIMENTS
In this section, we evaluate the efficacy of our framework on a variety of datasets, and demonstrate that our technique improves robustness while preserving clean accuracy. We introduce three variants based on our reweighting technique:
Non-parametric reweighting: we learn weights using the weighted adversarial cross-entropy loss where the weight Wie for sample.7 j at iteration / is proportional to the correlation between the training
loss gradient and the average validation loss gradient: E
(
(0)
30
36
Ac-1
2) BiLAW (Parametric reweighting, Sec. trained using the weighted adversarial cross-entropy loss.
3) BiLAW can be modified BiLAW-TRADES, Parametric reweighting trained with the TRADES
loss (Zhang al. 2019). i.e. we solve
min Σ - 1/\(w; [L(fo(=))
where E() corresponds to the standard cross-entropy loss, KL corresponds to the KL-divergence, d corresponds to an adversarially perturbation, and w; = wi (A,): the parametric map applied to multi-class margin of fe at I;, For all experiments, layer fully connected ReLU network with 128 hidden units and a sigmoid activation. Furthermore, to enforce aforementioned constraints, normalize the weights per-batch-ise wi; wt/ Σ3 Wi--
set 1/A 6, and define be single hidden-
4.1 PERFORMANCE EVALUATION
We evaluate the performance of approach compared plain training, adversarial training (AT) (Madry et al.) 2018), GAIRAT (Zhang et al., 2021). WMMR (Zeng et al.) (021). and MAIL (Wang et al. 202Ip. All experiments are run on single RTX 2080 Ti. When applying our approach and variants, two validation sets of size 1000 are extracted from the training set: one is used to learn the auxiliary network parameters, and the second is used for early stopping. This results in smaller training set for BiLAW, while the training sets of competing methods are unaltered.
In Table we evaluate BiLAW using two relatively small networks on two datasets: MNIST (LeCun & Cortes 2010) and Fashion MNIST (Xiao et al. (017)) Tiny-CNN is a convolutional network with convolutional and dense layers. FCI corresponds a single hidden layer feedforward network with 1024 hidden units. The details of the architectures are given in the Appendix. We consider robustness with respect to fo distance. We use three criteria: clean test accuracy (clean), robust test accuracy (PGD) for given threshold and Attack (AA). Robust test accuracy is computed using Projected Gradient Descent (PGD) (Madry et al. 2018) with 20 iterations. In all testcases, our method matches the performance of GAIRAT and out out performs the other methods for clean and PGD accuracy and we out- perforrfomm all reweighting methods on AA accuracy. However, we note the overall distribution of both clean and robust accuracy is tight. We note potential drawback of reweighting
7

---

Under review as a conference paper at ICLR 2022
algorithms: the MNIST and F-MNIST datasets contain non-trivial number of misclassified samples which can influence performance (Müller & Markert, 2019). For algorithms which perform weighted training, possible large weights on outhies mislabeled examples may influence classification performance. We will investigate this in the context of adversarial training in future work.
Table 1: MNIST/F-MNIST comparison for plain, AT, GAIRAT, WMMR (Oitaain = 0.1, Ctest = 2). MAIL (7 5, 0.05) and BiLAW using standard robust loss. Best result underlined and bolded and second best is bolded.
Tiny-CNi CNN-
FCI
penurbution: Ex
perturbatio: i
perturbation: Foe
perturhatio:: l2
Clean
PGD
AA
Claa PGD
AA
Clean
PGD
AA
Clean PGD
AA
MWVST
0.1
= 0.3
6 = U.1
0.3
plain
99.1
21.7
9.1
99.2
96.9
36.4
98.4
1.7
0.0
98.3
90.3
16.1
AT
99.0
95.9
93.7
99.1
98.2
96.1
98.4
92.9
90.4
8.8
97.4
95.3
GAIRAT
99.1
96.77
91.1
99.2
98.8
90.3
99.0
93..2
89.7
98.8
97.6
89.2
99.0
98.5
98.9
97.2
MAIL
98.6
95.1
91.4
98.7
98.6
95.4
98.4
93.1
91.3
98.1
97.4
94.2
(OUTS))))))
99.2
96.7
91.7
99.2
98.9
95.4
99.1
93.1
91.6
98.6
97.6
94.4
F.MNIST
0..
c
0.3
0.1
<
0.3
plain
89.6
1,5
0.0
89.7
42.9
0.0
98.5
0.0
0.0
89.3
57.2
0.0
AT
86.4
91.9
79.6
77.9
87.0
68.7
76.0
GAIRAT
86.4
77.6
64.3
92.3
81.1
70.3
87.1
70,2
61.4
91.1
81.0
70.4
WMMR
86.2
77.3
64.1
92.1
80.6
71.4
86.9
68.4
61.3
91.1
78.4
70.9
MAIL
86.4
76.9
68.6
92.2
80.5
76.2
90.1
69.3
66.4
90.6
79.3
75.9
(iLA)((Ur))
86.6
77.4
68.8
92.4
81.3
76.6
87.3
70.6
66.7
91.4
$0.9
76.1
Table CIFAR- 10 comparison for AT, GAIRAT, WMMR, non-parametric weighting, and BiLAW with standard adversarial training (BiLAW) and with TRADES loss (BILAW-TRADES We perform AA on 1000 samples the test set. Best result underlined and bolded and second best is bolded.
Small-CNN
WRN- 10-32
perturhation: &o
perturbation: f.
perturbation: foc Cleun
perturhation: for
Clean PGD
AA
Clean PGD
AA
PGD
Clean PGD
AA
CIFAR-TO
0.0078
0.031
0.0078
0.031
GAIRAT
79.0
54.7
48.1
79.0
55.6
40.7
86.4
73.6
63.1
84.7
56.8
43.4
WMMR
78.7
81.7
49.1
39.1
85.9
70.9
67.4
80.6
49.5
40.6
MAIL
76.8
64.3
59.2
81.9
53.3
40.6
84.3
74.1
73.7
83.2
53.7
52.0
AT
78.7
58.7
56.6
79.6
45.6
42.9
85.9
71.3
69.5
85.9
$2.0
48.0
TRADES (1/A- = 6)
79.2
58.9
56.8
78.9
54.8
51.7
84.6
73.9
73.1
83.1
53.9
52.1
Non-purametrric weighting
79.7
47.3
81.3
52.2
40.6
86.4
73.7
62.3
86.6
52.8
42.9
BiL (ours)
79.7
63.6
56.7
80.4
55.4
45.3
87.1
74.2
71.3
87.4
58.6
51.4
BILAW-TRADES (ours)
79.1
64.8
61.5
80.2
56.2
52.6
86.2
74.8
74.2
86.1
58.9
53.6
In Table we evaluate our method using the two architec- tures used in Zhang et al. (202T) on CIFAR- 10 (Krizhevsky al.): a 6-layer convolutional network and a Wide-Resnet- 10-32 (WRN-32-10) (Zagoruyko & Komodakis 2016), with details provided in the Appendix. We run each method for 100 epochs with training and validation batch sizes sct to 128 using SGD + momentum. A standard learn- ing rate schedule is implemented with the initial learning rate of 0.1 divided by 10 at Epoch 30 and 60, respectively. BiLAW strictly outperforms AT with respect to both clean and robust accuracy and generally outperforms GAIRAT and WMMR with respect to clean and robust accuracy on
Table 3: CIFAR- 100 comparison for AT, GAIRAT, WMMR, and BiLAW with the TRADES loss. We perform AA 1000 samples in the test set. The best result is underlined and bolded and the second best is bolded.
WRN-32-10
perturbation; &x Clean PGD
AA
CIPAR-TOU
0.031
RILAW-TRADES (ours)
62.8
31.4
AT
57.9
28.9
24.7
TRADES (1/A I)
62.4
25.3
22.2
TRADES (1/A 6)
56.5
30.9
26.9
CIFAR-10 (up to 10%). In particular, BiLAW consistently achieves superior clean test accuracy in all testcases, ex- cept for the fxc small-CNN (c = 0.031). On the WRN f case, we maintain and outperform relevant methods with respect to both PGD-bbaee and AA-based robust accuracy while achieving superior clean test accuracy. We demonstrate that when used in conjunction with TRADES, BiLAW preserves and improves robustness to AA attacks by 1.5% in contrast to TRADES, while significantly enhancing clean test accuracy by up to 3% and PGD attacks by up to 5%. We also note that parametric reweighting as opposed to the non- -ar-maraett version significantly improves robust accuracy. On CIFAR- 100 (Table 3) BiLAW-TRADES other methods. Our results demonstrate the effectiveness of using a held-out validation set to learn the sample weights compared to heuristic reweighting schemes.
GAIRAT
60.2
30.4
22.6
WMMR
56.1
29.7
25.8
MAII.
62.4
30.9
26.8
ter-errfomm all
8

---

Under review as a conference paper at ICLR 2022
In the Appendix conduct three ablative experiments to analyze the effect of (1) the capacity of the auxiliary weighting network, (2) the fixed TRADES coefficient and (3) the input to the weighting network, on the performance of BiLAW.
4.2 TRAINING SAMPLE WEIGHTS
We investigate the correspondence between weights and samples, and ask the question: what are the properties of Iraining examples with high/low weights? Fig. Fig.B provides evidence that supports our claim that samples for which the auxiliary network predicts high weights correspond to vulnerable, or difficult samples close to the decision boundary. Fig. Bj))(b), we plot the distribution of weights for each class, as well the associated confusion matrix of predictions made by a robust classifier (trained with BiLAWW on adversarial samples. We note that the distribution of weights matches the distribution of misclassified adversarial examples. For example, in Fig. (ia),, samples of the "ship' and "automobile' class are assigned a higher number smaller weights and they are typically classified correctly as in Fig. 3p. In contrast, birds, cats, and other animals have a higher number of samples assigned large weight and are more frequently misclassified.
(a)
B17
(b)
E3T
Beer
hhgg
Prrdesed abal
tome
ght
Tros Lafzel
peddraa sumonoble prddrne
pROITAA It's
prOrri siventll PACTTA arerida
(c)
heethhese bey pnd trci poettrek yne fren toe pest peetaky frue sower
(d)
Figure 3: (a) Weight distribution and examples of CIFAR-10 samples. (b) Adversarial confusion matrix of robust network on CIFAR-10. (c) "Easy" CIFAR-10 samples with low weight are correctly classified, (d) "Hard" CIFAR-10 10 samples with high weight are typically incorrectly classified.
In Fig. []cc, we provide several examples of test samples that are assigned low weight. These images are typically very clear, involving centered object and plain background. In Fig. dd, we provide set of test samples assigned high weights. Many of these images are challenging for humans to identify, even when uncorrupted by adversarial noise. For example, the second and fifth image pictures of cats and birds with unusual pose. The seventh, eighth, and ninth image nearly impossible to identify due to complex backgrounds or obscured objects. Additionally, the second, third, eighth, and tenth images consist of multiple objects that could confuse the network facilitate more effective adversarial attacks. Thus, high weights can be used to identify adversarially vulnerable samples, and automatically differentiate easy and challenging samples in existing datasets.
CONCLUSION
We have introduced BiLAW, a new robust training method that learns classifiers that are robust to norm-bounded adversarial attacks and is inspired by the hypothesis that reweighting via bilevel opti- mization offers remedy to the issue of adversarial overfitting, We demonstrate that our method learns robust networks that out- out-performs competing methods, includingg related, geometrically moottriicallymootiattedd techniques. Notably, BiLAW does not rely complicated heuristics to assign weights, and demonstrate the learned weights are interpretable. Future work involves improving scalability and investigating whether the auxiliary network might be used detect adversarial corruptions.
motivated
9

---

Under review as a conference paper at ICLR 2022
REFERENCES
Mahdich Abbasi and Christian Gagné. Robustness to adversarial examples through an ensemble of specialists. ArXiv, abs/1702.06856, 2017.
Han Bao, Clay Scott, and Masashi Sugiyama. Calibrated surrogate losses for adversarially robust classification. In Proceedings of Thirty Third Conference Learning Theory volume 125 of Proceedings of Machine Learning Research, pp. 408 408-451. PMLR, 09-12 Jul 2020. URL
Jerome Bracken and James McGill. Mathematical programs with optimization problems in the constraints. Operations Research, 21(1):37-44, 1973. ISSN 0030364X, 15265463. URL
http://wwk. hwtp:/sssttootooooooooo.o.eet
Tianlong Chen, Zhenyu Zhang. Sijia Liu, Shiyu Chang, and Zhangyang Wang. Robust overfitting may be mitigated by properly learned smoothening. International Conference on Learning Representations, 2021. URL httcttp::///peeeeviee
net
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. volume 97 of Proceedings of Machine Learning Research, pp. 1310-1320, Long Beach, California, USA, 09-15 Jun 2019. PMLR.
Corinna Cortes, Mehryar Mohri, and Afshin Rostamizadeh. Multi-class classification with maximum margin multiple kernel. In Sanjoy Dasgupta and David McAllester (eds.), Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pp. 46-54, Atlanta, Georgia, USA, 17-19 Jun 2013. PMLR. URL https v/proceedings.m.. press / c2//cooree13...ooooooo html
Francesco Croce and Matthias Hein. Provable robustness against all adversarial ip t-perturbations for 2 1. In International Conference on Learning Representations, 2020a. URL https
//openreview.c .nt/ aeppem/wwwee........ooooo....dd. ySYPB
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML, 2020b.
Amir Erfan Eshratifar, David Eigen, and Massoud Pedram. Gradient agreement as optimization 2018. URL URL/tttp://ax/wvoor//////
objective for meta-learning. CoRR,
08178
Chelsea Finn, Picter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, volume 70, pp. 1126-1135. PMLR, 06-11 Aug 2017. URL httt:://proceeeinng.. mlr
p (tnn0//fiinnlln inniinnnn.... html
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
Simon Jenni and Paolo Favaro. Deep bilevel learning. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (eds.), Computer Vision - ECCV 2018, pp. 632-648, Cham, 2018.
Springer International Publishing. ISBN 978 978-3-930-0122996
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. CIFAR-10 (Canadian Institute for Advanced Research). htt::tt:///ww/ CS twronto..oooooo.. edu wwrrk/i///// /i ftr.m
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. URL httt::ttyyann
lecun. cow wwt.w//w//w//eeccc/occ http//xxbbmmnnitt /mnist/
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018.
Nicolas Michael Müller and Karla Markert. Identifying mislabeled instances in classification datasets. CoRR, abs/1912.052a. 2019. /tll:tttpe//aaww..rrraaaa/////// 05283
10

---

Under review as a conference paper at ICLR 2022
Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-leaming with implicit gradients. H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran As- sociates, Inc., 2019. URL https: /ttp::::////ccceeeooeeooo nerggssnneurre neurips....... www.eeew...ppppppeeeee wwtpppppp//.11..00ppppp 9/ffle/fl 072b033ba 0eweaa12622444444445451
pdf]
Mengye Ren, Wenyuan Zeng. Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In ICMI., 2018.
Leslie Rice, Eric Wong, and J. Zico Kolter. Overfitting in advensarially robust deep learning. CoRR, abs/2002. 11569, 2020. URL
11569
Andrei A. Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-learning with latent embedding optimization. In International Confer- ence on Learning Representations, 2019. URL /ttps::/ net/forumfid BJgkLhAcK?
Mohammad Saberian and Nuno Vasconcelos. Multiclass boosting: Margins, codewords, losses, and algorithms. Journal of Machine Learning Research, 20(137):1-68, 2019. URL http:
7/ 1/]ml. wttp:opppppppp.ppetootoeeeeeeeet /v2/ 1111221111111111 11mmmmmmmll
Hadi Salman, Greg Yang. Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya P. Razenshteyn, and Sébastien Bubeck. Provably robust deep learning via adversarially trained smoothed classifiers. CoRR, 10s/1996600454, 2019.
Amartya Sanyal, Puneet K. Dokania, Varun Kanade, and Philip Torr. How benign is benign overfitting ? In International Conference on Learning Representations, 2021. URL https
7/openreview net g wu9TMPODo
Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meta-weight- net: Learning explicit mapping for sample weighting. In NeurIPS, 2019.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. CoRR, abs/aa 2017.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv, (bb///31..699,, 2014.
Qizhou Wang, Feng Liu, Bo Han, Tongliang Liu, Chen Gong, Gang Niu, Mingyuan Zhou, and Masashi Sugiyama. Probabilistic margins for instance reweighting in adversarial training. CoRR, abs/2106.07904, 2021. URL
Eric Wong and Zico J. Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. International Conference on Machine Learning (ICML), 2018
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: novel image dataset for bench- marking machine learning algorithms. CoRR, abs/1708.07747, 2017.
Mingyang Yi, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu, and Zhi-Ming Ma. Reweighting augmented samples by minimizing the maximal expected loss. In International Conference on Learning Representations, 2021. htthttss::/ http:/::////peeeeeeeeee/ net/ http://rrrrreeeeeeeeeeeeeee
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. CoRR, 0abs/1605.077 2016.
URL httttt////a/iii http://wwww..../a////.......000 07146
Huimin Zeng. Chen Zhu, Tom Goldstein, and Furong Huang. Are adversarial examples created equal? learnable weighted minimax risk for robustness under non-uniform attacks. In AAAI, 2021.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. volume 97 of Proceedings of Machine Learning Research, pp. 7472-7482, Long Beach, California, USA, 09-15 15 Jun 2019.
11

---

Under review as a conference paper at ICLR 2022
Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankan- halli. Attacks which do not kill training make adversarial learning stronger. In ICML, 2020.
Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry-awand reweiheereweiighted adversarial training. In International Conference Learning Representations, 2021. URL httttpps///penrreieee https
Hui Zou. The margin vector, admissible loss and multi-class margin-based classifiers. 2005.
A APPENDIX
A.1 DERIVATION OF META GRADIENT
In this section we derive the update rule for the parameters of the auxiliary network in Eq.
a
m
de
dw
14 Pt-1
mn
00
18,
8
p4
Let
m Σ (0,) j=1
(o w) =
be the robust training loss with respect to parameters 0 at time and example weight wy for the j-th training example. Let (0.) Σ (.().) be the associated unweighted validation loss. Following the meta-learning framework, we to minimize this loss via gradient descent.
1
0μ
n
3p
1
pval (0)
00 ow
0e dw 0p
To compute 09 (recall, 0. (O.,, w1-1 0:-1 V w)). For example, a single gradient descent step:
we can apply the MAML technique and differentiate through the pseudo update De
08
0
(0,-1 a aV ao
Ow
Ou
V 1
m
So the complete update is:
m
R dن (0)
Sw
# =
1111111
mn
00
0.
00
1
A.2
MAIN EXPERIMENTS
A.2.1 ARCIITECTURES
We abbreviate one hidden layer fully connected network with 1024 hidden units with FC1. The tiny-CNN convolutional architecture that we use is identical to that of Wong & Kolter (2018); Croce & Hein (2020a) -onnsistigg of two convolutional layers with 16 and 32 filters of size 4 x 4 and stride 2. followed by a fully connected layer with 100 hidden units. For all experiments we use training use SGD with piecewise constant learning rate schedule with initial learning rate 0.1. The learning rate is divided by 10 at epochs 30 and 60 respectively. On all datasets (MNIST, F-MNIST, CIFAR- 10, and
and validation batch sizes of 128 and we train all models for 100 epochs. Moreover,
12

---

Under review as a conference paper at ICLR 2022
Table 4: Architectures for main experiments for number of classes n.c
FC1
tiny-CNN
small-CNN
FC(1024)
Conv(16, 4 x 4, 2)
Mmall-CNN-BLOCKEEE
ReLU
ReLU
Small-CNN-BLOCKKE
FC(nc)
Conv(32, 4 x 2)
small-CNN-BLLOCK(]]
ReLU
FC(256)
FC(100) ReLU
ReLU
FC(nc)
FC(nc)
Table 5: Architectures for main experiments for number classes nic.
small-CNN-BLOCKK
Conv(c, 3 x 3, 1)
BatchNorm
ReLU Conv(c, 3 3, 1)
BatchNorm ReLU MaxPool(2 x 2)
CIFAR- 100) we restrict the input to be in the range [0, 1]. On the CIFAR-10 dataset, following Zhang al. (2021) apply random crops and random mirroring of the images data augmentation during training. We perform adversarial training using the PGD attack of Madry et al. (2018). During training, perform 10 iterations of the PGD attack for all datasets. During evaluation, use 20 iterations for all datasets. Following /Zhang et al. (021), the step size is the perturbation radius divided by 4.
A.3 ABLATION STUDY
In this section, evaluate variations of our technique on CIFAR-10 using the WRN-32-10 archi- tecture and Ex with = 0.031. First, we explore how the capacity of the auxiliary reweighting technique influences the performance our method. Next, we evaluate our network in combination with TRADES for various values of 1/A. Finally, demonstrate the advantage of the multi-class margin over alternative inputs mapping to the sample weights-e.g using the class-unaware margin (Def. 1), the adversarial loss A and the difference between the adversarial loss and the clean loss at a sample Aun-
Table 6: Ablation experiments: capacity of the axuiliary weight prediction network
CIFARIO I0
Capacity of
Clean
PGD
PGD
Clean
128 hidden units
86.1
58.9
27.2
64
83.6
57.4
26.2
64 64
85.8
58.6
27.1
256
85.7
57.7
28
In Table 6 we evaluate the influence of the auxiliary network capacity, i.c. the choice of As with training robust classifiers, the architecture of the network influences the clean-robust tradeoff, with smaller capacity networks reducing the gap between clean and robust performance, and larger networks increasing the gap.
In Table 7
show relative robustness of our approach combined with TRADES the choice of 1/A. particular, we maintain an important advantage of TRADES: the ability to easily control the robustness tradeoff by controlling 1/2.
13

---

Under review as a conference paper at ICLR 2022
Table Ablation experiments: TRADES coefficient 1/A
CIFARIO
1/A
Clean
PGD
1/A
86.1
58.9
1/A 1
87.4
52.5
1/X
86.9
57.6
1/A 10
83.8
57.9
Table 8: Ablation experiments: A, input to the auxiliary network
CIFARIO
Network input
Clean
PGD
multiclass margin (Def. 3)
86.1
58.9
margin (Def.
84.1
54.6
86.9
57.9
8-E
85.4
53.8
In T show that the choice of input the auxiliary neural network to predict the sample weights has a significant impact. In particular, we show the necessity of using the multi-class margin achieve superior clean and robust test accuracy. Surprisingly, conditioning the weight the robust loss also leads to good performance, better than the margin, and employing a learnable map for either the class-aware and class- unasaee outperforms heuristic methods (e.g.. WMMR and MAIL).
A.4 CIFAR-10 EXAMPLE WEIGHTS
In Fig we recover the predictions made by small-CNN trained with BiLAW. We then use principal component analysis (PCA) to project 10-dimensional predicted class likelihoods into 2-dimensions and plot the corresponding embeddings. The color denotes the degree of the robustness of each data point. Samples which are assigned larger weight are darker. As expected, these samples associated with high weights lie close to the decision boundary and are more likely improve robust generalization.
In Fig 5 we investigate the dynamics of predicted weights by visualizing the progression of weights predicted at margins for training samples and their adversarial variants. We observe (1) the dynamics of the weights seem to be determined largely by the learning rate of the classifier (i.e. the first adjustment to the learning rate happens around epoch 20), (2) the majority of weights predicted for clean samples are low (i.e. most clean samples are easy), and (3) the variance of the weight distribution is quite tight for adversarial samples.
In Fig 6 we compare weights computed via the GAIRAT and MAIL heuristics weights predicted via BiLAW and show a positive correlation. In particular, BiLAW may be considered a generalization of the MAIL heuristic that additionally incorporates multi-class margin information. The similar- ity between margin- based weight estimators BiLAWW and MAII. is evident, while the PGD-based GAIRAT weighting heuristic emphasizes a bimodal weight distribution.
Replicating (Fig. 3
we plot samples with small and large weight for competitive methods GAIRAT and MAIL. As with out method, samples associated with small weights appear to be "easy" and visa versa.
A.5 MNIST WEIGHT EXAMPLES
We plot MNIST samples with small and large weight. As with CIFAR-I0 (Fig. 3), samples associated with small weights appear to "easy" in the sense that the digits are neatly written. On the other hand, digits associate with high weight are easily confused and often involve the occurrence or lack of occurrence of spaces between strokes that define certain digits (e.g. 3, 5, 0, 9, and 8).
14

---

Under review as a conference paper at ICLR 2022
(a)
(b)
Figure 4: Two orientations of 3-d plot of PCA applied to the model's likelihood predictions on training samples of three classes from the CIFAR-I0 dataset (blue: car, red: plane, & green: ship). The weight of individual samples (denoted by the shade) correlates with the margin/degree of robustness.
BUSC
8780
(a)
(b)
Figure 5: (a) Progression of weights associated with subset of clean training samples (b) Progression of weights associated with a subset of adversarially perturbed training samples
100
7080
1090
3000
(a)
(b)
0784
EO0S
telse
5300
C004
(d)
(c)
Figure 6: (a) BiLAW weight distributions per-class for CIFAR-10 samples. (b) MAIL weight distributions per-class for CIFAR-10 10 samples. (c) Scatter plot of MAIL weight vs. BiLAW weight
for robust network. (b) GAIRAT weight distributions per-class for CIFAR-10 samples. (c) Scatter plot of GAIRAT weight VS. BiLAWW weight for robust network.
15

---

Under review as a conference paper at ICLR 2022
SPAITIA deet
Encaaa Ponn
Viliie Sert
ENETTA
(b)
Figure 7: Examples taken from CIFAR-10 10 and weighted using GAIRAT (Zhang et al. Samples with low weight. (b) Samples with high weight.
202 (a)
sedena Miders
(a)
(b)
Figure 8: Examples taken from CIFAR- and weighted using MAIL (Wang et al. 2021 (a) Samples with low weight. (b) Samples with high weight.
(a)
6
2
3
7
6
2
p48.0000
PRO.S.0
s968 tiat
6
(b)
3
/
e
4
C
Figure 9: Examples taken from MNIST, (a) Samples with low weight. (b) Samples with high weight.
16


Reviews:
[{"review_id": "piSb_4EkhV", "reviewer": "Reviewer_3Xwo", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths (motivation, soundness, correctness, clarity):\n\nWriting: The writing is clear and easy to understand.\n\nMethodology: The proposed methodology is sound. The authors clearly explained how they adapted from prior works (MAML, Meta-Weight-Net, etc.). Their motivation for Def 3. and how it can be used as an input to their meta net are well explained. I also appreciate that the authors make the paper more self-contained by giving their meta gradient derivation in the appendix.\n\nExperiments: The experimental results are good, especially under harder attacks (e.g. AutoAttack).\n\nWeakness (novelty, improvements decoupling):\nAlthough the proposed methodology is new in adversarial training to my best knowledge, it essentially applies idea of Meta-Weight-Net in the adversarial training setting. On the surface, this lacks some novelty. I think the paper can benefit from explaining how the performance improvement is decoupled: 1. the bilevel framework, 2. their encoding (Def 3.), say by replacing Def 3 as inputs to the meta net by simply the raw input data or the input data's activations.\n\nMinor suggestion or typo:\n- Page 3: \"An immediate consequence of Eq. 1 and Def. 2.1\", the hyperlink in Def. 2.1 doesn't seem to point to any definition. I cannot see Def 2.1 in the main paper either.\n- Page 3: \"More concretely, given a norm p and radius $\\epsilon$\", do the authors mean to say: ... given $L_p$ norm ...\n- Page 3: I understand this notation \"$L(y; x + \\delta; \\theta)$\", but this may be more natural: $L(y; f_{\\theta}(x + \\delta))$, unless this is defined again in the notations section.\n\n\n", "summary_of_the_paper": "The paper proposes a bilevel optimization procedure for adversarial training. While this has been done for empirical risk optimization, the paper may be the first that adapts such techniques for adversarial training. The new modification is their input encoding for the meta network: Def 3 (multi-class margin).", "summary_of_the_review": "Overall, I think this is a nice contribution to the literature. The only drawback I can see is the lack of justification for their input encoding to the meta net, leaving the readers pondering where the improvements come from. I'd raise my score if the author can provide more justification on their input encoding (Def 3), or an ablation study.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ifQ-UP0w-U", "reviewer": "Reviewer_oJtx", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. It is interesting to combine MAML with adversarial training and formulate a bi-level reweighing adversarial training framework.\n2. The definition of multi-class margin is intuitive but enlightening for rethinking the boundary of samples.\n3. The structure of the article is relatively organized.\n\nWeaknesses:\n1. My biggest concern is that, from Table 2 and 3, using BiLAW alone is not competitive with others when facing more reliable attack methods like AutoAttack, especially TRADES. So when used in combination with TRADES, the gain is more likely to be brought by TRADES rather than BiLAW itself. Combined with the higher robust accuracy against PGD, it suggests that the boundary of the model trained by BiLAW may not significantly change. BiLAW is more likely to make the area near some hard samples (for PGD and its variants only) much sharper, which is virtually harmful to construct a certified robust model.\n2. It seems that the uncertainty of the source of performance improvement. It is hard to say whether such robust performance comes from the reweighting strategy or just the increase in parameters or the structural change brought by the auxiliary network. Although this network will not be activated in the inferring stage, the inaccessibility of the auxiliary network to attackers may result in a performance boost.\n3. Since the authors add an auxiliary network to train the model. It is natural to care about the running time for BiLAW in comparison with other methods.", "summary_of_the_paper": "The authors propose a bi-level adversarial training method to reweight the importance of samples in each mini-batch, aiming at building a more robust DNN model. The authors borrow the idea from meta-learning and design an auxiliary network to learn such weights. The results show their approach can improve robustness against certain attacks. ", "summary_of_the_review": "Overall, considering the weaknesses, not good enough experimental results and the limited novelty, I am inclined to reject the paper unless the authors can provide more interesting discovery and explain a compelling reason why it cannot perform well itself.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "fBOxJy5TJxc", "reviewer": "Reviewer_tCDo", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The technique of learning the sample weights during training instead of simply using a function proportional to class margins is novel and interesting. Further, similar bi-level optimization has been used for other tasks (MAML, GAN training) for sample reweighing with a fair amount of success. \n2. The claims in the paper are well supported with experiments and comparisons are adequate. \n3. The authors also demonstrate the need for a new definition of multi-class margin through a very interesting experiment ranking the adversarial logit index.  \n\nWeaknesses/Questions:\n1. How will the defense fair against an adaptive attack? In this case, if the attacker knows the relative average weights of the various classes, the attack algorithm may be able to compensate for that. \n2. Sec 4.2 shows the distribution of high and low weighed training samples for BiLAW. It would be great to see similar plots for GAIRAT and MAIL to see if the weights are a property of the dataset or that of the training method. \n3. The ablation studies in the appendix should be in the main paper in order to demonstrate the advantages of BiLAW over other methods such as TRADES. The paper would benefit from a more involved discussion of the ablation instead of Sec 4.2.\n4. I am also curious about training times for BiLAW models. Is there a significant difference in the number of epochs required in comparison with TRADES?", "summary_of_the_paper": "The authors propose a bi-level adversarial training mechanism to learn sample weights with the goal of training more robust neural networks.  In comparison with related work which derive sample weights using various definitions of class margins, the authors parametrize the weights using a neural network. They show the efficacy of their approach by testing the robustness of their model against a variety of attacks. ", "summary_of_the_review": "Overall, the paper is well written. I especially like the way the authors have motivated BiLAW, and appreciate the detailed comparisons with related work. The improvements while not significant, do merit publication and further discussion.The experiments are also comprehensive and support all claims. I am therefore inclined to recommending an accept. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ZlRLKOPb8Sv", "reviewer": "Reviewer_E9Gz", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Though the concept of reweighing training samples based on margins to improve robustness is not new, the paper presents novel approaches to reweighing by a) using multi-class margins and b) meta-learning how to reweight the samples from the class margins. The experiments show that BiLAW generally outperforms the current baselines. The paper is generally easy to follow.\n\nThe main weaknesses of the paper are the lack of 1) theoretical discussion/justification of why the use of proposed multi-class margin is better and 2) ablation studies to separately test that the two proposed components a) multi-class margin and b) meta-learned margin-to-weight mapping indeed improve the performance as claimed.\n\nComments & Questions:\nWhat is the computational cost of BiLAW versus other reweighting baselines?\nWhat could explain BiLAW underperforming some of the baselines in some settings?\n\nTypo:\nTestcases > test cases\n", "summary_of_the_paper": "The paper proposes a novel approach (BiLAW) to reweigh training samples with the aim of improving models’ adversarial robustness. Different from previous reweighing methods, BiLAW uses multi-class margins and a trained neural network that learns to map the class margins to sample weights. The author claims that these two components improve sample reweighting to bring better robustness and clean accuracy. A meta-learning algorithm (MAML) is used to train the reweighing model. Experiments are conducted in the MNIST, CIFAR-10 and -100 datasets to show that BiLAW outperforms previous robustness and reweighting baselines.", "summary_of_the_review": "Despite the weaknesses and the limited novelty from previous reweighting algorithms, I am more inclined to accept given the improved empirical results.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "VNpv9YOyEkc", "reviewer": "Reviewer_fjzH", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nStrengths:\n- As for as I know, it is the first learning-based re-weighting strategy for adversarial robustness, compared to previous heuristic methods like GAIRAT.\n- The paper is well-written and easy to follow. The proposed approach is methodologically sound. \n- The evaluation includes strong adversaries, like AutoAttack, and they show their reweighing does not suffer from performance degeneracy like GAIRAT.\n\n**Major Concerns**:\n\n**1. Novelty.** The proposed approach is a direct adaptation of the classical MAML algorithm to adversarial training, which is of limited technical novelty and hardly meets the bar of this venue. Deeper investigation of the reweighting mechanisms should be discussed, for example, the effect on the multi-class margin by the proposed reweighting, should be demonstrated.\n\n**2. Evaluation.** Unlike previous approach, the proposed BiLAW relies an additional reweighting module in the training stage. Therefore, in the (fully transparent) white-box setting, I believe this additional module should be included in the evaluation stage for crafting adaptive attack. I have read the authors’ response to other reviewers, while I cannot agree with that evaluating adaptive attack is “beyond the scope of this paper”. Instead, because the proposed method distinguishes itself as using a learned reweighting, the authors cannot use other methods for an excuse, and the evaluation of adaptive attack on the learned reweighting module is **within the scope of this work**. Even if the authors argue that the reweighting module is not included in the test stage, we can still use an independently learned reweighting head for adaptive attack. To wrap up, I believe this work would not be complete without evaluating adaptive attack, and I strongly suggest that the authors should do so in the future.\n\n**3. Performance.** The improvement is still marginal under AA, and it relies on a combination with TRADES to obtain good accuracy, and lacks comparison to SOTA baselines like AWP. MNIST/F-MNIST is too old for the current research, and small CNNs are also unnecessary for CIFAR-10. On CIFAR-100, the result of BiLAW is **not shown**. I encourage the authors to be honest with the results even if it is relatively lower.\n\nMinor concerns:\n\nThe authors spend too much space on preliminaries and lack enough discussion on introducing and justifying their own method. For example, the technical discussions of multi-class margin (Section 3.1) is unnecessarily complex, as the authors have not shown any rigorous connection between the theory and their proposed method. Also, in Step 2 (Section 3.2), the update rule is also unnecessarily complex, as the sum term could be simply replaced by the average loss.\n", "summary_of_the_paper": "This paper develops a meta-learning approach for re-weighting samples for better adversarial robustness. In particular, they parameterize the weights using an additional module and learn it with the MAML objective. Through a comparison with previous work, they show that the proposed method improves model robustness on benchmark datasets.\n", "summary_of_the_review": "Based on the limitations above, I think that this paper, although having technical contributions to the community, is still not ready for publication due to the lack of fair evaluation. Therefore, I recommend rejection. I would suggest the authors address my concerns (as well as those from other reviewers) with additional experiments and submit it to a future venue.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Rl9_QlA79Oh", "reviewer": "Reviewer_K5MU", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is well-written, clearly motivated and related work is sufficiently discussed. The comparison with the baselines is fair and the evaluation seems thorough. The improvements over the baselines are also consistent and relatively significant.\n\nMy biggest concern is that some more modern baselines may be needed in order to keep the claim in the abstract \"improves both clean and robust accuracy compared to related techniques and **state-of-the-art baselines**.\" According to the [RobustBench leaderboard](https://robustbench.github.io/) there are many papers that achieve higher robust accuracy. Most of them use either real or synthetic auxiliary data or larger architectures, so no direct comparison is possible. However, [Adversarial Weight Perturbation Helps Robust Generalization](https://arxiv.org/abs/2004.05884) is listed as having a 3% higher robust accuracy using a \"WideResNet-34-10\". Is this the same or different from what the authors call \"Wide-Resnet-10-32\"? Ultimately, I do not think that the paper absolutely needs to compare to all baselines that achieve higher robust accuracy, because the techniques may be very different, but in my opinion the paper would benefit from clarifying where it fits in the current SOTA. For example, could the SOTA be improved by combining the technique with prior methods?\n\nIt also seems that the method's novelty is not particularly high since it simply applies a previously known technique to adversarial training. Given the good results this is not an issue though, in my opinion.\n\nI didn't understand why the MNIST and FMNIST models were not evaluated using AutoAttack or at least Auto-PGD.\n\nOne fact that I found quite striking in the paper is that Figure 3 a) shows that the learned weights are in fact very close to one another. This makes it all the more surprising that such small differences could really have a significant impact. It would be quite interesting to see if it is really the weights that make the difference or the altered training dynamics, i.e. that the objective is changing throughout training. A simple ablation would be to retrain a model from scratch using a fixed weighting network that is taken from a previous BiLAW run. If the weights really make all the difference, then they should be unaltered under this change.\n\nMinor comments:\n- the paragraph headers on page 2 have inconsistent punctuations\n- visa-versa => vice versa\n- punctuations after equations are missing\n- the use of $\\mathcal{L}$ on page 3 seems inconsistent with its definition on page 2\n- on page 5 there is a notational clash between the meaning of $\\Delta_i$ and $\\Delta_j$. I would recommend using a superscript for one of them.\n- in step 2 on page 6 the parentheses refer to the wrong line in Alg. 1\n- a boldface is missing in the 6th column of Table 2\n", "summary_of_the_paper": "The paper applies ideas from meta-learning to adversarial training. They train an auxiliary network that assigns each training sample a corresponding weight which indicates how useful the training point is for robust generalization performance. They compare their robust accuracy against several baselines and demonstrate improved clean and robust accuracy as compared to their baselines.", "summary_of_the_review": "The paper is well-motivated and has a solid evaluation showing consistent improvements on a well-studied task. Even though the authors could try to push their method's performance by additionally combining it with other baselines, in my opinion the results are significant enough to warrant a publication.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a meta-learning approach for re-weighting samples for better adversarial robustness. Specifically, they parameterize the weights using an additional module and learn it with the MAML objective. I have read the paper and reviews carefully by myself and found that the paper has several weaknesses that are not well addressed in the rebuttal. 

1) limited novelty. The proposed approach is a direct adaptation of the classical MAML algorithm to adversarial training, which is of limited technical novelty as pointed by serveral reviewers. 

2) Adaptive attack experiments are incomplete. The proposed BiLAW relies an additional reweighting module in the training stage, though it will not be used in the test stage. But we can still use an independently learned reweighting head for adaptive attack, which is should be considered in the white-box attacks. We do not want to see the new proposed defense will be defeated by other attacks quickly. 

3) The true performance for BiLAW is problematic. Table 1 on MNIST is not representative for current development in adversarial community. On Table 2, comparing BiLAW with TRADES, for AA (0.031), 45.3% vs 51.7% on small-CNN and 51.4% vs 52.1% on WRN-32-10. The performance of BiLAW is lower than TRADES, while when combine the two together, the results is very natural higher than TRADES and BiLAW but we are sure which component benefit the gains. For example, we can say TRADES benefits BiLAW because BiLAW+TRADES (52.6%) is much higher than BiLAW (45.3%). Also, the author did not show the results of single BiLAW on CIFAR-100. Considering the around two times running time, this performance is not acceptable in adversarial training methods. 

Due to the above reasons, I cannot recommend acceptance in the current verison to ICLR.
Decision: Reject