Original: L EARNING S AMPLE R EWEIGHTING FOR A DVERSARIAL R OBUSTNESS
Revised: L EARNING S AMPLE R EWEIGHTING FOR A DVERSARIAL R OBUSTNESS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: There has been great interest in enhancing the robustness of neural network classiﬁers to defend against adversarial perturbations through adversarial training, while balancing the trade-off between robust accuracy and standard accuracy.
Revised: There has been great interest in enhancing the robustness of neural network classiﬁers to defend against adversarial perturbations through adversarial training, while balancing the trade-off between robust accuracy and standard accuracy.

Original: We propose a novel adversarial training framework that learns to reweight the loss associated with individual training samples based on a notion of class-conditioned margin, with the goal of improving robust generalization.
Revised: We propose a novel adversarial training framework that learns to reweight the loss associated with individual training samples based on a notion of class-conditioned margin, with the goal of improving robust generalization.

Original: Inspired by MAML-based approaches, we formulate weighted adversarial training as a bilevel optimization problem where the upper-level task corresponds to learning a robust classiﬁer, and the lower-level task corresponds to learning a parametric function that maps from a sample’s multi-class margin to an importance weight.
Revised: Inspired by MAML-based approaches, we formulate weighted adversarial training as a bilevel optimization problem where the upper-level task corresponds to learning a robust classiﬁer, and the lower-level task corresponds to learning a parametric function that maps from a sample’s multi-class margin to an importance weight.

Original: Extensive experiments demonstrate that our approach improves both clean and robust accuracy compared to related techniques and state-of-the-art baselines.
Revised: Extensive experiments demonstrate that our approach improves both clean and robust accuracy compared to related techniques and state-of-the-art baselines.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: While neural networks have been extremely successful in tasks such as image classiﬁcation and speech recognition, recent work (Szegedy et al., 2014; Goodfellow et al., 2015) has demonstrated that neural network classiﬁers can be arbitrarily fooled by small, adversarially-chosen perturbations of their input.
Revised: While neural networks have been extremely successful in tasks such as image classiﬁcation and speech recognition, recent work (Szegedy et al., 2014; Goodfellow et al., 2015) has demonstrated that neural network classiﬁers can be arbitrarily fooled by small, adversarially-chosen perturbations of their input.

Original: Notably, Su et al.
Revised: Notably, Su et al.

Original: (2017) demonstrated that neural network classiﬁers which can correctly classify “clean” images may be vulnerable to targeted attacks , e.g., misclassify those same images when only a single pixel is changed.
Revised: (2017) demonstrated that neural network classiﬁers which can correctly classify “clean” images may be vulnerable to targeted attacks , e.g., misclassify those same images when only a single pixel is changed.

Original: Recent work has shown a common failing among techniques that uniformly encourage robustness.
Revised: Recent work has shown a common failing among techniques that uniformly encourage robustness.

Original: In particular, there exists an intrinsic tradeoff between robustness and accuracy (Zhang et al., 2019).
Revised: In particular, there exists an intrinsic tradeoff between robustness and accuracy (Zhang et al., 2019).

Original: Bao et al.
Revised: Bao et al.

Original: (2020) investigate this tradeoff from the perspective of classiﬁcation-callibrated loss theory.
Revised: (2020) investigate this tradeoff from the perspective of classiﬁcation-callibrated loss theory.

Original: Rice et al.
Revised: Rice et al.

Original: (2020) empirically showed that during adversarial training, networks often irreversibly lose robustness after training for a short time.
Revised: (2020) empirically showed that during adversarial training, networks often irreversibly lose robustness after training for a short time.

Original: They dubbed this phenomenon adversarial overﬁtting while proposing early stopping as a remedy.
Revised: They dubbed this phenomenon adversarial overﬁtting while proposing early stopping as a remedy.

Original: The signiﬁcance of label noise and memorization in the context of adversarial overﬁtting was demonstrated in Sanyal et al.
Revised: The signiﬁcance of label noise and memorization in the context of adversarial overﬁtting was demonstrated in Sanyal et al.

Original: (2021)—in particular that poor training samples induce fragility to adversarial perturbations due to the tendency of neural networks to interpolate the training data.
Revised: (2021)—in particular that poor training samples induce fragility to adversarial perturbations due to the tendency of neural networks to interpolate the training data.

Original: Methods based on weight and logit smoothing have been proposed as an alternative to early stopping (Chen et al., 2021;
Revised: Methods based on weight and logit smoothing have been proposed as an alternative to early stopping (Chen et al., 2021;

Original: Cohen et al., 2019;
Revised: Cohen et al., 2019;

Original: Salman et al., 2019).
Revised: Salman et al., 2019).

Original: In another line of work, Geometry-Aware Instance Reweighted Adversarial Training (GAIRAT; Zhang et al.
Revised: In another line of work, Geometry-Aware Instance Reweighted Adversarial Training (GAIRAT; Zhang et al.

Original: , Weighted Margin-aware Minimax Risk (WMMR; Zeng et al.
Revised: , Weighted Margin-aware Minimax Risk (WMMR; Zeng et al.

Original: and MarginAware Instance reweighting Learning (MAIL; Wang et al.
Revised: and MarginAware Instance reweighting Learning (MAIL; Wang et al.

Original: (2021)) were developed to address adversarial overﬁtting by controlling the inﬂuence of training examples via importance / loss weighting.
Revised: (2021)) were developed to address adversarial overﬁtting by controlling the inﬂuence of training examples via importance / loss weighting.

Original: Intuitively, the samples assigned a low weight correspond to samples on which the classiﬁer is already sufﬁciently robust.
Revised: Intuitively, the samples assigned a low weight correspond to samples on which the classiﬁer is already sufﬁciently robust.

Original: However, existing methods rely on rigid approximations of the margin and employ heuristic weighting schemes that rely on careful choices of hyperparameters.
Revised: However, existing methods rely on rigid approximations of the margin and employ heuristic weighting schemes that rely on careful choices of hyperparameters.

Original: This work builds upon these observations.
Revised: This work builds upon these observations.

Original: We present BiLAW (Bilevel Learnable Adversarial reWeighting), an approach that explicitly learns a parametric function (e.g. represented by a small feed-forward network) that assigns weights to the loss suffered by a classiﬁer, associated with individual training samples.
Revised: We present BiLAW (Bilevel Learnable Adversarial reWeighting), an approach that explicitly learns a parametric function (e.g. represented by a small feed-forward network) that assigns weights to the loss suffered by a classiﬁer, associated with individual training samples.

Original: The sample weights are learned as a function of the classiﬁer multiclass margins of the sample, according to the weights’ effect on robust generalization .
Revised: The sample weights are learned as a function of the classiﬁer multiclass margins of the sample, according to the weights’ effect on robust generalization .

Original: We employ a bi-level optimization formulation Bracken & McGill (1973) and leverage a validation set, where the upper-level objective corresponds to learning the parameters of a robust classiﬁer, while the lower-level objective corresponds to learning a function that predicts sample weights that improve improve robustness on a validation set.
Revised: We employ a bi-level optimization formulation Bracken & McGill (1973) and leverage a validation set, where the upper-level objective corresponds to learning the parameters of a robust classiﬁer, while the lower-level objective corresponds to learning a function that predicts sample weights that improve improve robustness on a validation set.

Original: Our approach alternates between iteratively updating the parametric sample weights and updating the classiﬁer network parameters.
Revised: Our approach alternates between iteratively updating the parametric sample weights and updating the classiﬁer network parameters.

Original: Contributions We note that while sample weighting (Zhang et al., 2021;
Revised: Contributions We note that while sample weighting (Zhang et al., 2021;

Original: Yi et al., 2021; Wang et al., 2021) has been investigated in the context of robust training, as far as we know this is the ﬁrst work to explore a method to use a validation set to learn weights to induce robust generalization.
Revised: Yi et al., 2021; Wang et al., 2021) has been investigated in the context of robust training, as far as we know this is the ﬁrst work to explore a method to use a validation set to learn weights to induce robust generalization.

Original: Our contributions consist of the following:
Revised: Our contributions consist of the following:

Original: Our approach performs sample re-weighting during training.
Revised: Our approach performs sample re-weighting during training.

Original: We propose a bilevel optimization formulation to learn a mapping from multi-class margins to weights according to the robust loss suffered by a classiﬁer on a validation set.
Revised: We propose a bilevel optimization formulation to learn a mapping from multi-class margins to weights according to the robust loss suffered by a classiﬁer on a validation set.

Original: We show that when our weighting function corresponds to a neural network, the magnitude of a sample’s weight directly corresponds to the vulnerability of the classiﬁer at that sample.
Revised: We show that when our weighting function corresponds to a neural network, the magnitude of a sample’s weight directly corresponds to the vulnerability of the classiﬁer at that sample.

Original: We evaluate the practical performance of BiLAW on MNIST (LeCun & Cortes, 2010), FMNIST (Xiao et al., 2017), CIFAR-10, and CIFAR-100.
Revised: We evaluate the practical performance of BiLAW on MNIST (LeCun & Cortes, 2010), FMNIST (Xiao et al., 2017), CIFAR-10, and CIFAR-100.

Original: (Krizhevsky et al.)
Revised: (Krizhevsky et al.)

Original: and demonstrate that it improves clean accuracy up to 6% and robust test accuracy by up to 5% compared to TRADES and other state-of-the-art sample reweighting methods on CIFAR-10.
Revised: and demonstrate that it improves clean accuracy up to 6% and robust test accuracy by up to 5% compared to TRADES and other state-of-the-art sample reweighting methods on CIFAR-10.

Original: This paper is organized as follows.
Revised: This paper is organized as follows.

Original: In Section 2, we review the notation and background of cost-aware and robust classiﬁcation.
Revised: In Section 2, we review the notation and background of cost-aware and robust classiﬁcation.

Original: In Section 3 we describe our method in full, including the sample weighting method.
Revised: In Section 3 we describe our method in full, including the sample weighting method.

Original: In Section 4, we provide ablative experiments and demonstrate the efﬁcacy of our framework by comparing clean and robust test performance on MNIST, F-MNIST, CIFAR-10, and CIFARto adversarial training and two recent, state-of-the-art sample weighting methods.
Revised: In Section 4, we provide ablative experiments and demonstrate the efﬁcacy of our framework by comparing clean and robust test performance on MNIST, F-MNIST, CIFAR-10, and CIFARto adversarial training and two recent, state-of-the-art sample weighting methods.

Original:  P RELIMINARIES AND R ELATED W ORK
Revised:  P RELIMINARIES AND R ELATED W ORK

Original: In this section, we brieﬂy present background terminology pertaining to adversarially robust classiﬁcation, sample reweighting and bilevel optimization.
Revised: In this section, we brieﬂy present background terminology pertaining to adversarially robust classiﬁcation, sample reweighting and bilevel optimization.

Original: Notations. An ReLU network is a neural network such that all nonlinear activations are ReLU functions, where we denote the ReLU activation by σ : R → R , σ ( x ) = max { 0 , x } .
Revised: Notations An ReLU network is a neural network such that all nonlinear activations are ReLU functions, where we denote the ReLU activation by σ : R → R , σ ( x ) = max { 0 , x } .

Original: Informally, we deﬁne σ : R d → R d by σ ( x ) =
Revised: Informally, we deﬁne σ : R d → R d by σ ( x ) =

Original: [ σ ( x 1 ) , . . .
Revised: [ σ ( x 1 ) , . . .

Original: , σ ( x d )] .
Revised: , σ ( x d )] .

Original: Let f θ : R d → [0 , 1] k be a feedforward ReLU network with L hidden layers and weights θ ; for example, f may map from a d -dimensional image to a k -dimensional vector corresponding to likelihoods for k classes.
Revised: Let f θ : R d → [0 , 1] k be a feedforward ReLU network with l hidden layers and weights θ ; for example, f may map from a d -dimensional image to a k -dimensional vector corresponding to likelihoods for k classes.

Original: Given a training set of m sample-label pairs ( x i , y i ) drawn from a training data distribution D , we associate a weight w i with each training sample.
Revised: Given a training set of m sample-label pairs ( x i , y i ) drawn from a training data distribution D , we associate a weight w i with each training sample.

Original: Informally, these weights characterize the effect of the sample on the generalization of the network (i.e. samples with large weights promote robust generalization and visa-versa).
Revised: Informally, these weights characterize the effect of the sample on the generalization of the network (i.e. samples with large weights promote robust generalization and visa versa).

Original: Given a loss function (cid:96) : R k × R k → R , we denote the empirical weighted training loss suffered by a network with parameters θ on m training samples with weights w to be L tr ( θ, w )
Revised: Given a loss function (cid:96) : R k × R k → R , we denote the empirical weighted training loss suffered by a network with parameters θ on m training samples with weights w to be L tr ( θ, w )

Original: = (cid:80) mi
Revised: = (cid:80) mi

Original: =1 w i (cid:96) ( y i , f θ ( x i ))
Revised: =1 w i (cid:96) ( y i , f θ ( x i ))

Original: such that w i ≥ 0 and (cid:80) i
Revised: such that w i ≥ 0 and (cid:80) i

Original: w i
Revised: w i

Original: = 1 .
Revised: = 1 .

Original: For brevity, we write (cid:96) i ( θ ) =
Revised: For brevity, we write (cid:96) i ( θ ) =

Original: (cid:96) ( y i , f θ ( x i )) .
Revised: (cid:96) ( y i , f θ ( x i )) .

Original: Additionally, if w is left unspeciﬁed, L corresponds to the unweighted mean over empirical losses.
Revised: Additionally, if w is left unspeciﬁed, L corresponds to the unweighted mean over empirical losses.

Original: Likewise, the unweighted validation loss of n samples is denoted L val ( θ ) .
Revised: Likewise, the unweighted validation loss of n samples is denoted L val ( θ ) .

Original: 2. R OBUST CLASSIFICATION AND ADVERSARIAL OVERFITTING
Revised: 2. R OBUST CLASSIFICATION AND ADVERSARIAL OVERFITTING

Original: Consider the network f θ : R d → R k , where the input is d -dimensional and the output is a kdimensional vector of likelihoods, with j -th entry corresponding to the likelihood the image belongs to the j -th class.
Revised: Consider the network f θ : R d → R k , where the input is d -dimensional and the output is a kdimensional vector of likelihoods, with j -th entry corresponding to the likelihood the image belongs to the j -th class.

Original: The associated classiﬁcation is then c ( x ) =
Revised: The associated classiﬁcation is then c ( x ) =

Original: arg max j ∈ [1 ,k ] f θ,j ( x ) .
Revised: arg max j ∈ [1 ,k ] f θ,j ( x ) .

Original: In adversarial machine learning, we are not just concerned that the classiﬁcation be correct, but we also want to be robust against adversarial examples, i.e. small perturbations to the input which may change the classiﬁcation to an incorrect class.
Revised: In adversarial machine learning, we are not just concerned that the classiﬁcation be correct, but we also want to be robust against adversarial examples, i.e. small perturbations to the input which may change the classiﬁcation to an incorrect class.

Original: We deﬁne the notion of (cid:15) -robustness below:
Revised: We deﬁne the notion of (cid:15) -robustness below:

Original: Deﬁnition 1 ( (cid:15) -robust)
Revised: Deﬁnition 1 ( (cid:15) -robust)

Original: f θ is called (cid:15) -robust with respect to norm p at x if the classiﬁcation is consistent for a small ball of radius (cid:15) around x :
Revised: f θ is called (cid:15) -robust with respect to norm p at x if the classiﬁcation is consistent for a small ball of radius (cid:15) around x :

Original: [Equation]
Revised: [Equation]

Original: Note that the (cid:15) -robustness of f θ at x is intimately related to the uniform and local Lipschitz smoothness of f θ around x .
Revised: Note that the (cid:15) -robustness of f θ at x is intimately related to the uniform and local Lipschitz smoothness of f θ around x .

Original: Recall that a function f has ﬁnite, global Lipschitz constant L > 0 with respect to norm || · || , if
Revised: Recall that a function f has ﬁnite, global Lipschitz constant L > 0 with respect to norm || · || , if

Original: [Equation]
Revised: [Equation]

Original: An immediate consequence of Eq.
Revised: An immediate consequence of Eq.

Original: 1 and Def.
Revised: 1 and Eq.

Original: 2.1 is that if f θ is uniformly L -Lipschitz, then f θ is (cid:15) -robust at x with (cid:15) = 12 L
Revised: 2 is that if f θ is uniformly L -Lipschitz, then f θ is (cid:15) -robust at x with (cid:15) = 12 L

Original: ( P a − P b ) where P a is the likelihood of the most likely outcome, and P b is the likelihood of the second most likely outcome (Salman et al., 2019).
Revised: ( P a − P b ) where P a is the likelihood of the most likely outcome, and P b is the likelihood of the second most likely outcome (Salman et al., 2019).

Original: The piecewise linearity of ReLU networks facilitates the extension of this consequence to the locally Lipschitz regime.
Revised: The piecewise linearity of ReLU networks facilitates the extension of this consequence to the locally Lipschitz regime.

Original: L corresponds to the norm of the afﬁne map characterized by f conditioned on input x .
Revised: L corresponds to the norm of the afﬁne map characterized by f conditioned on input x .

Original: These properties were previously used to characterize the robustness of a network at a sample (and the weight associated with the sample)
Revised: These properties were previously used to characterize the robustness of a network at a sample (and the weight associated with the sample)

Original: (Zeng et al., 2021;
Revised: (Zeng et al., 2021;

Original: Yi et al., 2021; Wang et al., 2021).
Revised: Yi et al., 2021; Wang et al., 2021).

Original: The minimal (cid:96) p -norm perturbation δ ∗ p required to switch an sample’s label is given by the solution to the following optimization problem:
Revised: The minimal (cid:96) p -norm perturbation δ ∗ p required to switch an sample’s label is given by the solution to the following optimization problem:

Original: [Equation]
Revised: [Equation]

Original: A signiﬁcant amount of existing work relies on a ﬁrst-order approximations and Hölder’s inequality to recover δ ∗ , justifying the popularity of inducing robustness by controlling global and local Lipschitz constants.
Revised: A signiﬁcant amount of existing work relies on a ﬁrst-order approximations and Hölder’s inequality to recover δ ∗ , justifying the popularity of inducing robustness by controlling global and local Lipschitz constants.

Original: More concretely, given a norm p  and radius (cid:15) , a typical goal of robust machine learning is to learn classiﬁers that minimize the robust loss on a training dataset.
Revised: More concretely, given a (cid:96) p norm and radius (cid:15) , a typical goal of robust machine learning is to learn classiﬁers that minimize the robust loss on a training dataset:

Original: [Equation]
Revised: [Equation]

Original: For brevity we will denote the robust analogue of a loss L as ˆ L , indicating this is the robust counterpart of L , differentiated by the “inner” maximization problem.
Revised: For brevity we will denote the robust analogue of a loss L as ˆ L , indicating this is the robust counterpart of L , differentiated by the “inner” maximization problem.

Original: 2. M ARGIN - AWARE R EWEIGHTING
Revised: 2. M ARGIN - AWARE R EWEIGHTING

Original: In the framework of cost-sensitive learning, weights are assigned to the loss associated with individual samples.
Revised: In the framework of cost-sensitive learning, weights are assigned to the loss associated with individual samples.

Original: The goal of cost-sensitive learning is to minimize the empirical weighted training loss :
Revised: The goal of cost-sensitive learning is to minimize the empirical weighted training loss :

Original: [Equation]
Revised: [Equation]

Original: Previous work in cost-sensitive learning for adversarial robustness (Zhang et al., 2020; 2021; Zeng et al., 2021) typically substitutes the robust loss ˆ L tr for L tr and largely focuses on designing heuristic functions of various notions of margin to use for the sample weight w i .
Revised: Previous work in cost-sensitive learning for adversarial robustness (Zhang et al., 2020; 2021; Zeng et al., 2021) typically substitutes the robust loss ˆ L tr for L tr and largely focuses on designing heuristic functions of various notions of margin to use for the sample weight w i .

Original: For example, in GAIRAT (Zhang et al., 2021; 2020), the margin is deﬁned as the least number of PGD steps, denoted κ , that leads the classiﬁer to make an incorrect prediction.
Revised: For example, in GAIRAT (Zhang et al., 2021; 2020), the margin is deﬁned as the least number of PGD steps, denoted κ , that leads the classiﬁer to make an incorrect prediction.

Original: The sample’s weight is computed as ω GAIRAT ( x i ) =
Revised: The sample’s weight is computed as ω GAIRAT ( x i ) =

Original: 12 (1 + tanh( λ + 5(

Revised: 12 (1 + tanh( λ + 5(

Original: − 2 κ/K )))
Revised: − 2 κ/K )))

Original: with hyperparameters K and λ .
Revised: with hyperparameters K and λ .

Original: A small κ indicates that the sample lies close to the decision boundary.
Revised: A small κ indicates that the sample lies close to the decision boundary.

Original: Larger κ values imply that associated samples lie far from the decision boundary, and are therefore more robust, requiring smaller weights.
Revised: Larger κ values imply that associated samples lie far from the decision boundary, and are therefore more robust, requiring smaller weights.

Original: However, due to the non-linearity of the loss-surface in practice, PGD-based attacks with ﬁnite iterations may suffer from the same issues that plague standard iterative ﬁrst-order methods in non-convex settings.
Revised: However, due to the non-linearity of the loss-surface in practice, PGD-based attacks with ﬁnite iterations may suffer from the same issues that plague standard iterative ﬁrst-order methods in non-convex settings.

Original: In other words, κ is heavily dependent on the optimization path taken by PGD.
Revised: In other words, κ is heavily dependent on the optimization path taken by PGD.

Original: This is demonstrated by GAIRAT’s vulnerability on sophisticated attacks, e.g. AutoAttack (Croce &
Revised: This is demonstrated by GAIRAT’s vulnerability on sophisticated attacks, e.g. AutoAttack (Croce &

Original: Hein, 2020b).
Revised: Hein, 2020b).

Original: Zhang et al.
Revised: Zhang et al.

Original: (2020) deﬁne the margin as the difference between the loss of a network suffered at a clean sample and its adversarial variant.
Revised: (2020) deﬁne the margin as the difference between the loss of a network suffered at a clean sample and its adversarial variant.

Original: Zeng et al.
Revised: Zeng et al.

Original: ; Wang et al.
Revised: ; Wang et al.

Original: (2021) propose a deﬁnition of margin corresponding to taking differences between logits, as follows.
Revised: (2021) propose a deﬁnition of margin corresponding to taking differences between logits, as follows.

Original: Deﬁnition 2 ((Zeng et al., 2021; Wang et al., 2021))
Revised: Deﬁnition 2 ((Zeng et al., 2021; Wang et al., 2021))

Original: The margin of a classiﬁer f θ on sample ( x i , y i ) is the difference between the classiﬁer’s conﬁdence in the correct label y i and the maximal probability of an incorrect label t , margin ( f θ , x i , y i )
Revised: The margin of a classiﬁer f θ on sample ( x i , y i ) is the difference between the classiﬁer’s conﬁdence in the correct label y i and the maximal probability of an incorrect label t , margin ( f θ , x i , y i )

Original: = p ( f ( x i ) =
Revised: = p ( f ( x i ) =

Original: y i ) − max t (cid:54) =
Revised: y i ) − max t (cid:54) =

Original: y i p ( f θ ( x i ) = t ) .
Revised: y i p ( f θ ( x i ) = t ) .

Original: Given this deﬁnition, Zeng et al.
Revised: Given this deﬁnition, Zeng et al.

Original: ; Wang et al.
Revised: ; Wang et al.

Original: (2021) propose to use an exponential (WMMR) and sigmoidal (MAIL) functions respectively: ω WMMR ( x i ) =
Revised: (2021) propose to use an exponential (WMMR) and sigmoidal (MAIL) functions respectively: ω WMMR ( x i ) =

Original: exp( − αm ) with parameter α , and ω MAIL ( x i ) = sigmoid ( − γ ( m − β )) with parameters γ and β .
Revised: exp( − αm ) with parameter α , and ω MAIL ( x i ) = sigmoid ( − γ ( m − β )) with parameters γ and β .

Original: WMMR and MAIL rely on the local linearity of ReLU networks and the fact that for samples near the margin, the relative scale of predicted class-likelihoods directly corresponds to the distance to the decision boundary.
Revised: WMMR and MAIL rely on the local linearity of ReLU networks and the fact that for samples near the margin, the relative scale of predicted class-likelihoods directly corresponds to the distance to the decision boundary.

Original: However, similarly to GAIRAT’s κ , even for samples very close to the decision boundary, simple functions of the difference between class likelihoods may not necessarily correspond to the true distance to the decision boundary.
Revised: However, similarly to GAIRAT’s κ , even for samples very close to the decision boundary, simple functions of the difference between class likelihoods may not necessarily correspond to the true distance to the decision boundary.

Original: In this work, we propose a more ﬁne-grained notion of margin, the multi-class margin , and design a method toautomatically learn a mapping between the margin at a sample to the associated weight. 
Revised: Here we propose a more ﬁne-grained notion of margin, the multi-class margin , and design a method to learn a mapping between the margin at a sample and its associated weight. Previous work has explored theoretical notions of a multi-class margin .

Revised: For example, Zou (2005) deﬁned the margin vector in the context of boosting as a proxy for a vector of conditional class probabilities.

Revised: However, this notion of margin is unaware of the true class of a sample.

Revised: In contrast, the multi-class margin proposed by Saberian & Vasconcelos (2019); Cortes et al.

Original:  In Fig.
Revised: (2013) are both closely related to Wang et al. Zeng et al. (2021), i.e. deﬁned as the minimal distance between an arbitrary predicted logit and the logit of the true class. In Fig.

Original: 1 we explore the relationship between the logits of a network evaluated at a clean sample and the predicted class of the adversarially perturbed variant.
Revised: 1 we explore the relationship between the logits of a network evaluated at a clean sample and the predicted class of the adversarially perturbed variant.

Original: Methods which rely on the canonical notions of margin reasonably assume that samples at which a classiﬁer is vulnerable have small margin according to Def. 2, i.e. the magnitude of the smallest difference between the logits of any class and the logit corresponding the true class is small.
Revised: Methods which rely on the canonical notions of margin reasonably assume that samples at which a classiﬁer is vulnerable have small margin according to Def. 2, i.e. the magnitude of the smallest difference between the logits of any class and the logit corresponding the true class is small.

Original: However, we demonstrate in Fig. 1(b) for both vulnerable and robust classiﬁers that while the majority of predictions made by networks on adversarial samples do correspond to the classes with minimal margin, a signiﬁcant number do not.
Revised: However, we demonstrate in Fig. 1(b) for both vulnerable and robust classiﬁers that while the majority of predictions made by networks on adversarial samples do correspond to the classes with minimal margin, a signiﬁcant number do not.

Original: In other words, the class for which the margin is smallest does not always correspond to the adversarial class.
Revised: In other words, the class for which the margin is smallest does not always correspond to the adversarial class.

Original: Furthermore, this issue is exacerbated for robust networks as shown by the difference in count distribution between networks whose relative robustness varies.
Revised: Furthermore, this issue is exacerbated for robust networks as shown by the difference in count distribution between networks whose relative robustness varies.

Original: 2. B I - LEVEL O PTIMIZATION AND M ETA - LEARNING
Revised: 2. B I - LEVEL O PTIMIZATION AND M ETA - LEARNING

Original: Bilevel optimization, ﬁrst introduced by Bracken & McGill (1973) is a framework for mathematical optimization involving nested optimization problems.
Revised: Bilevel optimization, ﬁrst introduced by Bracken & McGill (1973) is a framework for mathematical optimization involving nested optimization problems.

Original: A typical bilevel optimization problem takes on the following form:
Revised: A typical bilevel optimization problem takes on the following form:

Original: [Equation]
Revised: [Equation]

Original: where f and g are respectively denoted the upper-level and lower-level objectives.
Revised: where f and g are respectively denoted the upper-level and lower-level objectives.

Original: The goal of the framework is to minimize the primary objective Φ( x ) with respect to x where y ∗ ( x ) is obtained by solving the lower-level minimization problem.
Revised: The goal of the framework is to minimize the primary objective Φ( x ) with respect to x where y ∗ ( x ) is obtained by solving the lower-level minimization problem.

Original: The framework of bilevel optimization has seen wide adoption by the machine learning community—in particular in the context of hyperparameter tuning (Jenni & Favaro, 2018;
Revised: The framework of bilevel optimization has seen wide adoption by the machine learning community—in particular in the context of hyperparameter tuning (Jenni & Favaro, 2018;

Original: Ren et al., 2018) and meta-learning (Finn et al., 2017; Rajeswaran et al., 2019).
Revised: Ren et al., 2018) and meta-learning (Finn et al., 2017; Rajeswaran et al., 2019).

Original: Our proposed algorithm has some similarity to the meta-learning literature (Finn et al., 2017;
Revised: Our proposed algorithm has some similarity to the meta-learning literature (Finn et al., 2017;

Original: Rusu et al., 2019; Rajeswaran et al., 2019; Eshratifar et al., 2018).
Revised: Rusu et al., 2019; Rajeswaran et al., 2019; Eshratifar et al., 2018).

Original: Most notably, the Model-Agnostic
Revised: Most notably, the Model-Agnostic

Original: Meta-Learning MAML) algorithm by Finn et al.
Revised: Meta-Learning MAML) algorithm by Finn et al.

Original: (2017) incorporates gradient information for the meta-learning setting.
Revised: (2017) incorporates gradient information for the meta-learning setting.

Original: The application of meta-learning as an instance of bilevel optimization has been explored in the context of sample reweighting.
Revised: The application of meta-learning as an instance of bilevel optimization has been explored in the context of sample reweighting.

Original: In particular, Ren et al.
Revised: In particular, Ren et al.

Original: (2018); Jenni & Favaro (2018);
Revised: (2018); Jenni & Favaro (2018);

Original: Shu et al.
Revised: Shu et al.

Original: (2019) proposed methods to address learning with noisy labels by reweighting the gradients associated with the losses at individual samples based on balancing performance on a curated validation set and the corrupted training set.
Revised: (2019) proposed methods to address learning with noisy labels by reweighting the gradients associated with the losses at individual samples based on balancing performance on a curated validation set and the corrupted training set.

Original:  B I LAW: L EARNINGTO R EWEIGHT S AMPLES  FOR A DVERSARIAL T RAINING
Revised:  B I LAW: L EARNING S AMPLES W EIGHTS FOR A DVERSARIAL T RAINING

Original: In this section, we propose BiLAW, a new learning framework for robust training.
Revised: In this section, we propose BiLAW, a new learning framework for robust training.

Original: There are 2 main novelties in our new learning scheme compared to existing robust training methods: First, we consider a more reasonable assumption leveraging the concept of multi-class margin in robust training, where good weights should be aware of both the margin associated with each class, as well as the true class associated with the sample.
Revised: There are 2 main novelties in our new learning scheme compared to existing robust training methods: First, we consider a more reasonable assumption leveraging the concept of multi-class margin in robust training, where good weights should be aware of both the margin associated with each class, as well as the true class associated with the sample.

Original: Second, as opposed to related work which deﬁnes an explicit formula for the weights dependent on the margin, we propose to learn the weights as part of training the classiﬁcation model.
Revised: Second, as opposed to related work which deﬁnes an explicit formula for the weights dependent on the margin, we propose to learn the weights as part of training the classiﬁcation model.

Original: Speciﬁcally, we deﬁne the weights as a function of a multi-class margin, and parameterize this function using a small auxiliary network.
Revised: Speciﬁcally, we deﬁne the weights as a function of a multi-class margin, and parameterize this function using a small auxiliary network.

Original: We formulate this as a bi-level optimization problem and learn the weights iteratively alongside the neural network parameters.
Revised: We formulate this as a bi-level optimization problem and learn the weights iteratively alongside the neural network parameters.

Original: In the following, we introduce the two key components

Original: [Figure]
Revised: [Figure]

Original: 3. M ULTI - CLASS M ARGIN R EWEIGHTING
Revised: 3. M ULTI - CLASS M ARGIN R EWEIGHTING

Original: We extend the logit-based deﬁnitions of margin applied in Zeng et al.
Revised: We extend the logit-based deﬁnitions of margin applied in Zeng et al.

Original: ; Wang et al.
Revised: ; Wang et al.

Original: and deﬁne the multi-class margin of a classiﬁer at a sample as follows.
Revised: and deﬁne the multi-class margin of a classiﬁer at a sample as follows.

Original: Deﬁnition 

Revised: Deﬁnition

Original: The multi-class margin of a classiﬁer f θ on sample ( x i , y i ) , denoted ∆ : [0 , 1] k
Revised: The multi-class margin of a classiﬁer f θ on sample ( x i , y i ) , denoted ∆ : [0 , 1] k

Original: → [ − 1 , 1] k , is a k -dimensional vector whose j -th entry, ∆  j  ( f θ ( x i ) , y i ) , is the difference between the classiﬁer’s conﬁdence in the correct label y i and the classiﬁer’s conﬁdence in label j , ∆  j  ( f θ ( x i ) , y i ) 
Revised: → [ − 1 , 1] k , is a k -dimensional vector whose j -th entry, ∆ ( j ) ( f θ ( x i ) , y i ) , is the difference between the classiﬁer’s conﬁdence in the correct label y i and the classiﬁer’s conﬁdence in label j , ∆ ( j ) ( f θ ( x i ) , y i ) =

Original: = p ( f θ ( x i ) =
Revised: p ( f θ ( x i ) =

Original: y i ) − p ( f θ ( x i ) =
Revised: y i ) − p ( f θ ( x i ) =

Original: j ) .
Revised: j ) .

Original: For brevity we denote ∆( f θ ( x i ) , y i ) as ∆ i .
Revised: For brevity we denote ∆( f θ ( x i ) , y i ) as ∆ i .

Original: Note that the multi-class margin exhibits two qualities:
Revised: Note that the multi-class margin exhibits two qualities:

Original: Correct/incorrect classiﬁcation is implicit in the deﬁnition, as negative values indicate an incorrect classiﬁcation.
Revised: Correct/incorrect classiﬁcation is implicit in the deﬁnition, as negative values indicate an incorrect classiﬁcation.

Original: The true class of the sample is also implicit—i.e. the index with element zero (assuming the sample does not lie exactly on a decision boundary separating the true class from another).
Revised: The true class of the sample is also implicit—i.e. the index with element zero (assuming the sample does not lie exactly on a decision boundary separating the true class from another).

Original: In particular, we highlight the second quality.
Revised: In particular, we highlight the second quality.

Original: Prior work has demonstrated that the distribution of predictions made on adversarial samples is not necessarily uniform over all classes (Abbasi &
Revised: Prior work has demonstrated that the distribution of predictions made on adversarial samples is not necessarily uniform over all classes (Abbasi &

Original: Gagné, 2017).
Revised: Gagné, 2017).

Original: In other words, vulnerable samples and their associated adversarial perturbations may concentrate about certain classes more than others.
Revised: In other words, vulnerable samples and their associated adversarial perturbations may concentrate about certain classes more than others.

Original: We demonstrate in the results that networks exhibit non-uniform robustness per-class.
Revised: We demonstrate in the results that networks exhibit non-uniform robustness per-class.

Original: To learn the sample weights as a function of the multiclass margin, we construct an auxiliary neural network with a single hidden layer, whose parameters are denoted µ and whose inputs are the multiclass margins.
Revised: To learn the sample weights as a function of the multiclass margin, we construct an auxiliary neural network with a single hidden layer, whose parameters are denoted µ and whose inputs are the multiclass margins.

Original: The weight of the i -th training sample is then computed as w i =
Revised: The weight of the i -th training sample is then computed as w i =

Original: ω µ (∆ i ) .
Revised: ω µ (∆ i ) .

Original: In general, we denote the function used to map from margin to weight ω ( · ) .
Revised: In general, we denote the function used to map from margin to weight ω ( · ) .

Original: A question that arises is what loss function should be used to train this auxiliary network.
Revised: A question that arises is what loss function should be used to train this auxiliary network.

Original: We design a bilevel optimization approach leveraging the validation set to learn the auxiliary network parameters µ .
Revised: We design a bilevel optimization approach leveraging the validation set to learn the auxiliary network parameters µ .

Original: 3. B ILEVEL O PTIMIZATION
Revised: 3. B ILEVEL O PTIMIZATION

Original: We exploit a validation set to jointly learn a parametric weighting function ω µ on the training samples and a classiﬁer which jointly minimize the associated weighted robust error.
Revised: We exploit a validation set to jointly learn a parametric weighting function ω µ on the training samples and a classiﬁer which jointly minimize the associated weighted robust error.

Original: Let ˆ
Revised: Let ˆ

Original: L tr ( θ t , w t ) =
Revised: L tr ( θ t , w t ) =

Original: (cid:80) mi =1 w t,i ˆ (cid:96) i ( θ t ) , where w t,i =
Revised: (cid:80) mi =1 w t,i ˆ (cid:96) i ( θ t ) , where w t,i =

Original: ω µ t (∆ i ) be the weighted robust training loss with respect to parameters θ t and µ t at time t .
Revised: ω µ t (∆ i ) be the weighted robust training loss with respect to parameters θ t and µ t at time t .

Original: Additionally, w t,i ≥ 0 and (cid:80) m
Revised: Additionally, w t,i ≥ 0 and (cid:80) m

Original: b i =1 w t,y = 1 .
Revised: b i =1 w t,y = 1 .

Original: Intuitively, the samples with high weights should improve robust generalization—this is quantiﬁed by the robust error evaluated on a held-out validation set.
Revised: Intuitively, the samples with high weights should improve robust generalization—this is quantiﬁed by the robust error evaluated on a held-out validation set.

Original: Let ˆ L val ( θ t ) =
Revised: Let ˆ L val ( θ t ) =

Original: 1 n (cid:80) ni
Revised: 1 n (cid:80) ni

Original: =1 ˆ (cid:96) i
Revised: =1 ˆ (cid:96) i

Original: ( θ t ) be the unweighted robust validation loss associated with θ t .
Revised: ( θ t ) be the unweighted robust validation loss associated with θ t .

Original: Following the meta-learning principle, we seek weights such that the minimizer of the weighted robust training loss maximizes robust accuracy on the unweighted validation set—i.e. solve the following bilevel optimization problem:
Revised: Following the meta-learning principle, we seek weights such that the minimizer of the weighted robust training loss maximizes robust accuracy on the unweighted validation set—i.e. solve the following bilevel optimization problem:

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: The workﬂow of the reweighting procedure is illustrated in Fig.
Revised: The workﬂow of the reweighting procedure is illustrated in Fig.

Original: 2(A) and the reweighting algorithm is presented in Alg.
Revised: 2(A) and the reweighting algorithm is presented in Alg.

Original: We provide a high-level overview of the procedure below.
Revised: We provide a high-level overview of the procedure below.

Original: [Figure]
Revised: [Figure]

Original: Our approach is composed of three steps.
Revised: Our approach is composed of three steps.

Original: Steps 1 and 2 rely on the MAML-trick (Finn et al., 2017), which substitutes one-step updates µ t for µ ∗ and iteratively solves the upper-level problem.
Revised: Steps 1 and 2 rely on the MAML-trick (Finn et al., 2017), which substitutes one-step updates µ t for µ ∗ and iteratively solves the upper-level problem.

Original: In this context, µ t is updated according to the gradient of the unweighted robust validation loss with respect to the sample weights.
Revised: In this context, µ t is updated according to the gradient of the unweighted robust validation loss with respect to the sample weights.

Original: We note that this method necessitates computation of a pseudo-update in order to compute this gradient:
Revised: We note that this method necessitates computation of a pseudo-update in order to compute this gradient:

Original: Step 1 Pseudo update of classiﬁer parameters ˜ θ t (Step 1 in Fig.
Revised: Step 1 Pseudo update of classiﬁer parameters ˜ θ t (Step 1 in Fig.

Original: 2, line 5 in Alg. 1)
Revised: 2, line 5 in Alg. 1)

Original: [Equation]
Revised: [Equation]

Original: The pseudo parameters ˜ θ t are then used as a surrogate for θ t in optimizing for µ :
Revised: The pseudo parameters ˜ θ t are then used as a surrogate for θ t in optimizing for µ :

Original: Step 2 Update parameters µ t of the auxiliary network (Step 2 in Fig.
Revised: Step 2 Update parameters µ t of the auxiliary network (Step 2 in Fig.

Original: 2, line 5 in Alg. 1)
Revised: 2, line 6 in Alg. 1)

Original: [Equation]
Revised: [Equation]

Original: where α and β are the step size used in the pseudo and auxiliary network updates, respectively.
Revised: where α and β are the step size used in the pseudo and auxiliary network updates, respectively.

Original: Step
Revised: Step

Original:  Update parameters of
Revised: 3 Update parameters of

Original: classiﬁer network
Revised: classiﬁer network

Original: (Step 3 in
Revised: Step in

Original: Fig
Revised: Fig.

Original: line 8 in Alg. 1)
Revised: line 8 in Alg. 1)

Original: [Equation]
Revised: [Equation]

Original: One interpretation of this procedure is that we take a pseudo-step using θ t − 1 and µ t − 1 (Step 1), calculate the best auxiliary network parameters µ t in hindsight that improve generalization, by minimizing the validation loss with ˜ θ t , (Step 2), and then derive the “real” update for θ t − 1 by minimizing the weighted training loss using the new weights µ t (Step 3).
Revised: One interpretation of this procedure is that we take a pseudo-step using θ t − 1 and µ t − 1 (Step 1), calculate the best auxiliary network parameters µ t in hindsight that improve generalization, by minimizing the validation loss with ˜ θ t , (Step 2), and then derive the “real” update for θ t − 1 by minimizing the weighted training loss using the new weights µ t (Step 3).

Original: The detailed derivation of the gradient update is provided in the appendix.
Revised: The detailed derivation of the gradient update is provided in the appendix.

Original: Note that the term 1 n (cid:80)
Revised: Note that the term 1 n (cid:80)

Original: [Equation]
Revised: [Equation]

Original: in Eq.
Revised: in Eq.

Original: (5) represents the correlation between the gradient of the j -th training sample computed on the training loss and the average gradient of the validation data calculated on the robust validation loss.
Revised: (6) represents the correlation between the gradient of the j -th training sample computed on the training loss and the average gradient of the validation data calculated on the robust validation loss.

Original: As a consequence, if the gradient of the loss with respect to the network parameters at time t for training sample j is aligned with the average gradient of the meta-loss, it will be considered a beneﬁcial sample for generalization and its weight will be increased.
Revised: As a consequence, if the gradient of the loss with respect to the network parameters at time t for training sample j is aligned with the average gradient of the meta-loss, it will be considered a beneﬁcial sample for generalization and its weight will be increased.

Original: Conversely, the weight of the sample is suppressed if the gradient is anticorrelated with the average validation set-gradient.
Revised: Conversely, the weight of the sample is suppressed if the gradient is anticorrelated with the average validation set-gradient.

Original: Input: Training data D , validation-data set ˆ D , max iterations T , learning rates α, β
Revised: Input: Training data D , validation-data set ˆ D , max iterations T , learning rates α, β

Original: Output: Classiﬁer parameters θ 1: t ←2: Initialize θ 0 , µ 0 , w 0 = ω µ 0 (∆) 3: for t ≤ T do 4: ( X, y ) ∼ D , ( ˆ X, ˆ y ) ∼ ˆ D 5: ˜ θ t ← θ t − β · ∇ θ ˆ L tr | ˜ θ t ,w t 6: µ t +1 = µ t − α ∇ µ ˆ L val | θ t ,w t (cid:46) Compute ∇ µ ˆ L val | ˜ θ t ,w t via backpropagation according to Eq.7: compute w t +1 = ω µ t +1 (∆) (cid:46) compute ∆ with respect to θ t according to Def8:
Revised: Output: Classiﬁer parameters θ 1: t ←2: Initialize θ 0 , µ 0 , w 0 = ω µ 0 (∆) 3: for t ≤ T do 4: ( X, y ) ∼ D , ( ˆ X, ˆ y ) ∼ ˆ D 5: ˜ θ t ← θ t − β · ∇ θ ˆ L tr | ˜ θ t ,w t 6: µ t +1 = µ t − α ∇ µ ˆ L val | θ t ,w t (cid:46) compute ∇ µ ˆ L val | ˜ θ t ,w t via backpropagation according to Eq.7: compute w t +1 = ω µ t +1 (∆) (cid:46) compute ∆ with respect to θ t according to Def8:

Original: θ t +1 =
Revised: θ t +1 =

Original: θ t − β ∇ θ ˆ L tr | θ t ,w t +1 9: end for 10: return θ T
Revised: θ t − β ∇ θ ˆ L tr | θ t ,w t +1 9: end for 10: return θ T

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we evaluate the efﬁcacy of our framework on a variety of datasets, and demonstrate that our technique improves robustness while preserving clean accuracy.
Revised: In this section, we evaluate the efﬁcacy of our framework on a variety of datasets, and demonstrate that our technique improves robustness while preserving clean accuracy.

Original: We introduce three variants based on our reweighting technique:
Revised: We introduce three variants based on our reweighting technique:

Original: 1) Non-parametric reweighting: we learn weights using the weighted adversarial cross-entropy loss
Revised: 1) Non-parametric reweighting: we learn weights using the weighted adversarial cross-entropy loss

Original: where the weight w j,t for sample j at iteration t is proportional to the correlation between the training loss gradient and the average validation loss gradient: 1 n (cid:80) ni
Revised: where the weight w j,t for sample j at iteration t is proportional to the correlation between the training loss gradient and the average validation loss gradient: 1 n (cid:80) ni

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: 2) BiLAW (Parametric reweighting, Sec. 3) trained using the weighted adversarial cross-entropy loss.
Revised: 2) BiLAW (Parametric reweighting, Sec. 3) trained using the weighted adversarial cross-entropy loss.

Original: 3) BiLAW
Revised: 3) BiLAW

Original: can be modiﬁed to BiLAW-TRADES:
Revised: can be modiﬁed to BiLAW-TRADES:

Original: Parametric reweighting trained with the TRADES loss (Zhang et al., 2019), i.e. we solve
Revised: Parametric reweighting trained with the TRADES loss (Zhang et al., 2019), i.e. we solve

Original: [Equation]
Revised: [Equation]

Original: where (cid:96) () corresponds to the standard cross-entropy loss, KL corresponds to the KL-divergence, δ corresponds to an adversarially perturbation, and w i =
Revised: where (cid:96) () corresponds to the standard cross-entropy loss, KL corresponds to the KL-divergence, δ corresponds to an adversarially perturbation, and w i =

Original: ω µ (∆ i ) : the parametric map applied to the multi-class margin of f θ at x i .
Revised: ω µ (∆ i ) : the parametric map applied to the multi-class margin of f θ at x i .

Original: For all experiments, we set 1 /λ = 6 , and deﬁne ω to be a single hiddenlayer fully connected ReLU network with 128 hidden units and a sigmoid activation.
Revised: For all experiments, we set 1 /λ = 6 , and deﬁne ω to be a single hiddenlayer fully connected ReLU network with 128 hidden units and a sigmoid activation.

Original: Furthermore, to enforce aforementioned constraints, we normalize the weights per-batch—i.e. w i =
Revised: Furthermore, to enforce aforementioned constraints, we normalize the weights per-batch—i.e. w i =

Original: w i / (cid:80) j w j .
Revised: w i / (cid:80) j w j .

Original: 4. P ERFORMANCE EVALUATION
Revised: 4. P ERFORMANCE EVALUATION

Original: We evaluate the performance of our approach compared to plain training, adversarial training (AT) (Madry et al., 2018), GAIRAT (Zhang et al., 2021), WMMR (Zeng et al., 2021), and
Revised: We evaluate the performance of our approach compared to plain training, adversarial training (AT) (Madry et al., 2018), GAIRAT (Zhang et al., 2021), WMMR (Zeng et al., 2021), and

Original: MAIL (Wang et al., 2021).
Revised: MAIL (Wang et al., 2021).

Original: All experiments are run on a single RTX 2080 Ti.
Revised: All experiments are run on a single RTX 2080 Ti.

Original: When applying our approach and variants, two validation sets of size 1000 are extracted from the training set: one is used to learn the auxiliary network parameters, and the second is used for early stopping.
Revised: When applying our approach and variants, two validation sets of size 1000 are extracted from the training set: one is used to learn the auxiliary network parameters, and the second is used for early stopping.

Original: This results in a smaller training set for BiLAW, while the training sets of competing methods are unaltered.
Revised: This results in a smaller training set for BiLAW, while the training sets of competing methods are unaltered.

Original: In Table 1, we evaluate BiLAW using two relatively small networks on two datasets: MNIST (LeCun & Cortes, 2010) and Fashion MNIST (Xiao et al., 2017).
Revised: In Table 1, we evaluate BiLAW using two relatively small networks on two datasets: MNIST (LeCun & Cortes, 2010) and Fashion MNIST (Xiao et al., 2017).

Original: Tiny-CNN is a convolutional network withconvolutional and 2 dense layers.
Revised: Tiny-CNN is a convolutional network with 2 convolutional and 2 dense layers.

Original: FC1 corresponds to a single hidden layer feedforward network with 1024 hidden units.
Revised: FC1 corresponds to a single hidden layer feedforward network with 1024 hidden units.

Original: The details of the architectures are given in the Appendix.
Revised: The details of the architectures are given in the Appendix.

Original: We consider robustness with respect to (cid:96) ∞ distance.
Revised: We consider robustness with respect to (cid:96) ∞ distance.

Original: We use two criteria: clean test accuracy (TA) and robust test accuracy (PGD) for a given threshold (cid:15) .
Revised: We use three criteria: clean test accuracy (clean), robust test accuracy (PGD) for a given threshold (cid:15) and AutoAttack (AA).

Original: Robust test accuracy is computed using Projected Gradient Descent (PGD) (Madry et al., 2018) with 20 iterations.
Revised: Robust test accuracy is computed using Projected Gradient Descent (PGD) (Madry et al., 2018) with 20 iterations.

Original: In all testcases, our method matches the performance of GAIRAT and out-performs the other methods.
Revised: In all testcases, our method matches the performance of GAIRAT and out-performs the other methods for clean and PGD accuracy and we out-perform all reweighting methods on AA accuracy.

Original: However, we note the overall distribution of both clean and robust accuracy is tight.
Revised: However, we note the overall distribution of both clean and robust accuracy is tight.

Original: We note a potential drawback of reweighting algorithms: the MNIST and Fashion-MNIST datasets contain a non-trivial number of misclassiﬁed samples which can inﬂuence performance (Müller & Markert, 2019).
Revised: We note a potential drawback of reweighting algorithms: the MNIST and F-MNIST datasets contain a non-trivial number of misclassiﬁed samples which can inﬂuence performance (Müller & Markert, 2019).

Original: For algorithms which perform weighted training, possible large weights on outliers or mislabeled examples may inﬂuence classiﬁcation performance.
Revised: For algorithms which perform weighted training, possible large weights on outliers or mislabeled examples may inﬂuence classiﬁcation performance.

Original: We will investigate this in the context of adversarial training in future work
Revised: We will investigate this in the context of adversarial training in future work

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: [Table]
Revised: , WMMR, and BiLAW with the TRADES loss. We perform AA onsamples in the test set. The best result is underlined and bolded and the second best is bolded.

Original: [Table]
Revised: [Table]

Original: In Table 2, we evaluate our method using the two architectures used in Zhang et al.
Revised: In Table 2, we evaluate our method using the two architectures used in Zhang et al.

Original: (2021) on CIFAR-10 (Krizhevsky et al.
Revised: (2021) on CIFAR-10 (Krizhevsky et al.

Original: ): a 6-layer convolutional network and a Wide-Resnet10-32 (WRN-10-32) (Zagoruyko & Komodakis, 2016), with details provided in the Appendix.
Revised: ): a 6-layer convolutional network and a Wide-Resnet10-32 (WRN-10-32) (Zagoruyko & Komodakis, 2016), with details provided in the Appendix.

Original: We run each method for 100 epochs with training and validation batch sizes set to 128 using SGD + momentum.
Revised: We run each method for 100 epochs with training and validation batch sizes set to 128 using SGD + momentum.

Original: A standard learning rate schedule is implemented with the initial learning rate of 0.1 divided by 10 at Epoch 30 and 60, respectively.
Revised: A standard learning rate schedule is implemented with the initial learning rate of 0.1 divided by 10 at Epoch 30 and 60, respectively.

Original: BiLAW strictly outperforms AT with respect to both clean and robust accuracy and generally outperforms GAIRAT and WMMR with respect to clean and robust accuracy on
Revised: BiLAW strictly outperforms AT with respect to both clean and robust accuracy and generally outperforms GAIRAT and WMMR with respect to clean and robust accuracy on

Original: CIFAR-10 (up to 10% ).
Revised: CIFAR-10 (up to 10% ).

Original: In particular, BiLAW consistently achieves superior clean test accuracy in all testcases, except for the (cid:96) ∞ small-CNN ( (cid:15)
Revised: In particular, BiLAW consistently achieves superior clean test accuracy in all testcases, except for the (cid:96) ∞ small-CNN ( (cid:15)

Original: = 0 .
Revised: = 0 .

Original: On the WRN (cid:96) ∞ case, we maintain and outperform relevant methods with respect to both PGD-based and AA-based robust accuracy while achieving superior clean test accuracy.
Revised: On the WRN (cid:96) ∞ case, we maintain and outperform relevant methods with respect to both PGD-based and AA-based robust accuracy while achieving superior clean test accuracy.

Original: We demonstrate that when used in conjunction with TRADES, BiLAW preserves and improves robustness to AA attacks by 1 . 5% in contrast to TRADES, while signiﬁcantly enhancing clean test accuracy by up to 3% and PGD attacks by up to 5% .
Revised: We demonstrate that when used in conjunction with TRADES, BiLAW preserves and improves robustness to AA attacks by 1 . 5% in contrast to TRADES, while signiﬁcantly enhancing clean test accuracy by up to 3% and PGD attacks by up to 5% .

Original: We also note that parametric reweighting as opposed to the non-parametric version signiﬁcantly improves robust accuracy.
Revised: We also note that parametric reweighting as opposed to the non-parametric version signiﬁcantly improves robust accuracy.

Original: On CIFAR-100 (Table 3) BiLAW-TRADES out-performs all other methods.
Revised: On CIFAR-100 (Table 3) BiLAW-TRADES out-performs all other methods.

Original: Our results demonstrate the effectiveness of using a held-out validation set to learn the sample weights compared to heuristic reweighting schemes.
Revised: Our results demonstrate the effectiveness of using a held-out validation set to learn the sample weights compared to heuristic reweighting schemes.

Revised: In the Appendix we conduct three ablative experiments to analyze the effect of (1) the capacity of the auxiliary weighting network, (2) the ﬁxed TRADES coefﬁcient and (3) the input to the weighting network, on the performance of BiLAW.

Original: 4. T RAINING SAMPLE WEIGHTS
Revised: 4. T RAINING SAMPLE WEIGHTS

Original: We investigate the correspondence between weights and samples, and ask the question: what are the properties of training examples with high/low weights?
Revised: We investigate the correspondence between weights and samples, and ask the question: what are the properties of training examples with high/low weights?

Original: Fig.
Revised: Fig.

Original: 3 provides evidence that supports our claim that samples for which the auxiliary network predicts high weights correspond to vulnerable, or difﬁcult samples close to the decision boundary.
Revised: 3 provides evidence that supports our claim that samples for which the auxiliary network predicts high weights correspond to vulnerable, or difﬁcult samples close to the decision boundary.

Original: In Fig. 3(a)-(b), we plot the distribution of weights for each class, as well the associated confusion matrix of predictions made by a robust classiﬁer (trained with BiLAW) on adversarial samples.
Revised: In Fig. 3(a)-(b), we plot the distribution of weights for each class, as well the associated confusion matrix of predictions made by a robust classiﬁer (trained with BiLAW) on adversarial samples.

Original: We note that the distribution of weights matches the distribution of misclassiﬁed adversarial examples.
Revised: We note that the distribution of weights matches the distribution of misclassiﬁed adversarial examples.

Original: For example, in Fig. 3(a), samples of the ‘ship’ and ‘automobile’ class are assigned a higher number of smaller weights and they are typically classiﬁed correctly as in Fig.
Revised: For example, in Fig. 3(a), samples of the ‘ship’ and ‘automobile’ class are assigned a higher number of smaller weights and they are typically classiﬁed correctly as in Fig.

Original: 3b.
Revised: 3b.

Original: In contrast, birds, cats, and other animals have a higher number of samples assigned large weight and are more frequently misclassiﬁed.
Revised: In contrast, birds, cats, and other animals have a higher number of samples assigned large weight and are more frequently misclassiﬁed.

Original: (c) ( a ) (b) (d)
Revised: (c) ( a ) (b) (d)

Original: In Fig. 3(c), we provide several examples of test samples that are assigned low weight.
Revised: In Fig. 3(c), we provide several examples of test samples that are assigned low weight.

Original: These images are typically very clear, involving a centered object and plain background.
Revised: These images are typically very clear, involving a centered object and plain background.

Original: In Fig. 3(d), we provide a set of test samples assigned high weights.
Revised: In Fig. 3(d), we provide a set of test samples assigned high weights.

Original: Many of these images are challenging for humans to identify, even when uncorrupted by adversarial noise.
Revised: Many of these images are challenging for humans to identify, even when uncorrupted by adversarial noise.

Original: For example, the second and ﬁfth image are pictures of cats and birds with unusual pose.
Revised: For example, the second and ﬁfth image are pictures of cats and birds with unusual pose.

Original: The seventh, eighth, and ninth image are nearly impossible to identify due to complex backgrounds or obscured objects.
Revised: The seventh, eighth, and ninth image are nearly impossible to identify due to complex backgrounds or obscured objects.

Original: Additionally, the second, third, eighth, and tenth images consist of multiple objects that could confuse the network or facilitate more effective adversarial attacks.
Revised: Additionally, the second, third, eighth, and tenth images consist of multiple objects that could confuse the network or facilitate more effective adversarial attacks.

Original: Thus, high weights can be used to identify adversarially vulnerable samples, and automatically differentiate easy and challenging samples in existing datasets.
Revised: Thus, high weights can be used to identify adversarially vulnerable samples, and automatically differentiate easy and challenging samples in existing datasets.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We have introduced BiLAW , a new robust training method that learns classiﬁers that are robust to norm-bounded adversarial attacks and is inspired by the hypothesis that reweighting via bilevel optimization offers a remedy to the issue of adversarial overﬁtting.
Revised: We have introduced BiLAW , a new robust training method that learns classiﬁers that are robust to norm-bounded adversarial attacks and is inspired by the hypothesis that reweighting via bilevel optimization offers a remedy to the issue of adversarial overﬁtting.

Original: We demonstrate that our method learns robust networks that out-performs competing methods,including related, geometrically-motivated techniques.
Revised: We demonstrate that our method learns robust networks that out-performs competing methods,including related, geometrically-motivated techniques.

Original: Notably, BiLAW does not rely on complicated heuristics to assign weights, and we demonstrate the learned weights are interpretable.
Revised: Notably, BiLAW does not rely on complicated heuristics to assign weights, and we demonstrate the learned weights are interpretable.

Original: Future work involves improving scalability and investigating whether the auxiliary network might be used to detect adversarial corruptions.
Revised: Future work involves improving scalability and investigating whether the auxiliary network might be used to detect adversarial corruptions.

Original: A A PPENDIX A. D ERIVATION OF M ETA G RADIENT
Revised: A A PPENDIX A. D ERIVATION OF M ETA G RADIENT

Original: In this section we derive the update rule for the parameters of the auxiliary network in Eq. 5:
Revised: In this section we derive the update rule for the parameters of the auxiliary network in Eq. 6:

Original: [Equation]
Revised: [Equation]

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: be the robust training loss with respect to parameters θ at time t and example weight w j for the j -th training example.
Revised: be the robust training loss with respect to parameters θ at time t and example weight w j for the j -th training example.

Original: Let ˆ L val ( θ t ) =
Revised: Let ˆ L val ( θ t ) =

Original: 1 n (cid:80) ni
Revised: 1 n (cid:80) ni

Original: =1 ˆ (cid:96) i
Revised: =1 ˆ (cid:96) i

Original: ( θ t ) be the associated unweighted validation loss.
Revised: ( θ t ) be the associated unweighted validation loss.

Original: Following the meta-learning framework, we to minimize this loss via gradient descent.
Revised: Following the meta-learning framework, we to minimize this loss via gradient descent.

Original: [Equation]
Revised: [Equation]

Original:  To compute ∂ ˜ θ∂w , we can apply the MAML technique and differentiate through the pseudo update (recall, ˜ θ t =
Revised: ∂µ To compute ∂ ˜ θ∂w , we can apply the MAML technique and differentiate through the pseudo update (recall, ˜ θ t =

Original: GD tr ( θ t − 1 , w t − 1 ) := θ t − 1 − α ∇ θ L tr ,t − 1 ( θ t − 1 , w ) ).
Revised: GD tr ( θ t − 1 , w t − 1 ) := θ t − 1 − α ∇ θ L tr ,t − 1 ( θ t − 1 , w ) ).

Original: For example, a single gradient descent step:
Revised: For example, a single gradient descent step:

Original: [Equation]
Revised: [Equation]

Original: So the complete update is:
Revised: So the complete update is:

Original: [Equation]
Revised: [Equation]

Original: A. M AIN E XPERIMENTS A.2. A RCHITECTURES
Revised: A. M AIN E XPERIMENTS A.2. A RCHITECTURES

Original: [Table]

Original: We abbreviate one hidden layer fully connected network with 1024 hidden units with FC1.
Revised: We abbreviate one hidden layer fully connected network with 1024 hidden units with FC1.

Original: The tiny-CNN convolutional architecture that we use is identical to that of Wong & Kolter (2018); Croce & Hein (2020a) —consisting of two convolutional layers with 16 and 32 ﬁlters of size 4 × 4 and stride 2 , followed by a fully connected layer with 100 hidden units.For all experiments we use training [Table]
Revised: The tiny-CNN convolutional architecture that we use is identical to that of Wong & Kolter (2018); Croce & Hein (2020a) —consisting of two convolutional layers with 16 and 32 ﬁlters of size 4 × 4 and stride 2 , followed by a fully connected layer with 100 hidden units.

Original: [Equation] [Table] and validation batch sizes of [-
-] and we train all models for 100 epochs.
Revised: For all experiments we use training and validation batch sizes of [-
-]128 and we train all models for 100 epochs.

Original: Moreover, we use SGD with a piecewise constant learning rate schedule with initial learning rate of 0 . 1 .
Revised: Moreover, we use SGD with a piecewise constant learning rate schedule with initial learning rate of 0 . 1 .

Original: The learning rate is divided by 10 at epochs 30 and 60 respectively.
Revised: The learning rate is divided by 10 at epochs 30 and 60 respectively.

Original: On all datasets (MNIST, F-MNIST, CIFAR-10, and 
Revised: On all datasets (MNIST, F-MNIST, CIFAR-10, and [Table]

Revised: [Table]

Original: CIFAR-100)
Revised: CIFAR-100)

Original: we restrict the input to be in the range [0 , 1] .
Revised: we restrict the input to be in the range [0 , 1] .

Original: On the CIFAR-10 dataset, following Zhang et al.
Revised: On the CIFAR-10 dataset, following Zhang et al.

Original: (2021), we apply random crops and random mirroring of the images as data augmentation during training.
Revised: (2021), we apply random crops and random mirroring of the images as data augmentation during training.

Original: We perform adversarial training using the PGD attack of Madry et al.
Revised: We perform adversarial training using the PGD attack of Madry et al.

Original: During training, we perform 10 iterations of the PGD attack for all datasets.
Revised: During training, we perform 10 iterations of the PGD attack for all datasets.

Original: During evaluation, we use 20 iterations for all datasets.
Revised: During evaluation, we use 20 iterations for all datasets.

Original: Following Zhang et al.
Revised: Following Zhang et al.

Original: (2021), the step size is the perturbation radius divided by 4 .
Revised: (2021), the step size is the perturbation radius divided by 4 .

Original: A. A BLATION STUDY
Revised: A. A BLATION STUDY

Original: In this section, we evaluate variations of our technique on CIFAR-10 using the WRN-32-10 architecture and (cid:96) ∞ with (cid:15)
Revised: In this section, we evaluate variations of our technique on CIFAR-10 using the WRN-32-10 architecture and (cid:96) ∞ with (cid:15)

Original: = 0 .
Revised: = 0 .

Original: First, we explore how the capacity of the auxiliary reweighting technique inﬂuences the performance of our method.
Revised: First, we explore how the capacity of the auxiliary reweighting technique inﬂuences the performance of our method.

Original: Next, we evaluate our network in combination with TRADES for various values of 1 /λ .
Revised: Next, we evaluate our network in combination with TRADES for various values of 1 /λ .

Original: Finally, we demonstrate the advantage of the multi-class margin over alternative inputs mapping to the sample weights—e.g. using the class-unaware margin (Def. 1), the adversarial loss ∆ adv , and the difference between the adversarial loss and the clean loss at a sample ∆ diff .
Revised: Finally, we demonstrate the advantage of the multi-class margin over alternative inputs mapping to the sample weights—e.g. using the class-unaware margin (Def. 1), the adversarial loss ∆ adv , and the difference between the adversarial loss and the clean loss at a sample ∆ diff .

Original: [Table]
Revised: [Table]

Original: In Table 6, we evaluate the inﬂuence of the auxiliary network capacity, i.e. the choice of ω .
Revised: In Table 6, we evaluate the inﬂuence of the auxiliary network capacity, i.e. the choice of ω .

Original: As with training robust classiﬁers, the architecture of the network inﬂuences the clean-robust tradeoff, with smaller capacity networks reducing the gap between clean and robust performance, and larger networks increasing the gap.
Revised: As with training robust classiﬁers, the architecture of the network inﬂuences the clean-robust tradeoff, with smaller capacity networks reducing the gap between clean and robust performance, and larger networks increasing the gap.

Original: [Table] In Table 7, we show relative robustness of our approach combined with TRADES to the choice of 1 /λ .
Revised: In Table 7, we show relative robustness of our approach combined with TRADES to the choice of 1 /λ .

Original: In particular, we maintain the advantage of  the TRADES loss facilitating easy control the robustness tradeoff.
Revised: In particular, we maintain an important advantage of TRADES: the ability to easily control the robustness tradeoff by controlling 1 /λ .

Original: [Table]
Revised: [Table]

Original:  In Table 8, we show that the choice of input to the auxiliary neural network to predict the sample weights has a signiﬁcant impact.
Revised: [Table] [Table] In Table 8, we show that the choice of input to the auxiliary neural network to predict the sample weights has a signiﬁcant impact.

Original: In particular, we show the necessity of using the multi-class margin to achieve superior clean and robust test accuracy.
Revised: In particular, we show the necessity of using the multi-class margin to achieve superior clean and robust test accuracy.

Original: Surprisingly, conditioning the weight on the robust loss also leads to good performance, better than the margin , and employing a learnable map for either the class-aware and class-unaware outperforms heuristic methods (e.g., WMMR and MAIL).
Revised: Surprisingly, conditioning the weight on the robust loss also leads to good performance, better than the margin , and employing a learnable map for either the class-aware and class-unaware outperforms heuristic methods (e.g., WMMR and MAIL).

Original: A. MNIST W EIGHT EXAMPLES
Revised: A. CIFAR-10 EXAMPLE WEIGHTS

Revised: In Fig 4 we recover the predictions made by a small-CNN trained with BiLAW.

Revised: We then use principal component analysis (PCA) to project 10-dimensional predicted class likelihoods into 2-dimensions and plot the corresponding embeddings.

Revised: The color denotes the degree of the robustness of each data point.

Revised: Samples which are assigned larger weight are darker.

Revised: As expected, these samples associated with high weights lie close to the decision boundary and are more likely to improve robust generalization.

Revised: In Fig 5 we investigate the dynamics of predicted weights by visualizing the progression of weights predicted at margins for training samples and their adversarial variants.

Revised: We observe (1) the dynamics of the weights seem to be determined largely by the learning rate of the classiﬁer (i.e. the ﬁrst adjustment to the learning rate happens around epoch 20), (2) the majority of weights predicted for clean samples are low (i.e. most clean samples are easy), and (3) the variance of the weight distribution is quite tight for adversarial samples.

Revised: As with CIFAR-10 (Fig. 3), we plot samples with small and large weight for competitive methods

Revised: GAIRAT and MAIL.

Revised: As with out method, samples associated with small weights appear to be “easy” and visa versa.

Revised: A. MNIST W EIGHT EXAMPLES

Original: We plot MNIST samples with small and large weight.
Revised: We plot MNIST samples with small and large weight.

Original: As with CIFAR-10 (Fig. 3), samples associated with small weights appear to be “easy” in the sense that the digits are neatly written.
Revised: As with CIFAR-10 (Fig. 3), samples associated with small weights appear to be “easy” in the sense that the digits are neatly written.

Original: On the other hand, digits associate with high weight are easily confused and often involve the occurrence or lack of occurrence of spaces between strokes that deﬁne certain digits (e.g. 3, 5, 0, 9, and 8). 
Revised: On the other hand, digits associate with high weight are easily confused and often involve the occurrence or lack of occurrence of spaces between strokes that deﬁne certain digits (e.g. 3, 5, 0, 9, and 8). [Figure] [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Revised: [Figure]