Toward Efficient Robust Training against Union of $\ell_p$ Threat Models
Authors: Gaurang Sriramanan
Maharshi Gor
Soheil Feizi
Venue: NeurIPS 2022 Accept (2022)
Keywords: Adversarial Robustness
Adversarial Defense
Adversarial Training
Multiple Threat Models
Fast Adversarial Training
Efficient Adversarial Training
Single-Step Adversarial Training

Abstract: The overwhelming vulnerability of deep neural networks to carefully crafted perturbations known as adversarial attacks has led to the development of various training techniques to produce robust models. While the primary focus of existing approaches has been directed toward addressing the worst-case performance achieved under a single-threat model, it is imperative that safety-critical systems are robust with respect to multiple threat models simultaneously. Existing approaches that address worst-case performance under the union of such threat models ($\ell_{\infty}, \ell_2, \ell_1$) either utilize adversarial training methods that require multi-step attacks which are computationally expensive in practice, or rely upon fine-tuning of pre-trained models that are robust with respect to a single-threat model. In this work, we show that by carefully choosing the objective function used for robust training, it is possible to achieve similar, or improved worst-case performance over a union of threat models while utilizing only single-step attacks, thereby achieving a significant reduction in computational resources necessary for training. Furthermore, prior work showed that adversarial training specific to the $\ell_1$ threat model is relatively difficult, to the extent that even multi-step adversarially trained models were shown to be prone to gradient-masking. However, the proposed method—when applied on the $\ell_1$ threat model specifically—enables us to obtain the first $\ell_1$ robust model trained solely with single-step adversaries. Finally, to demonstrate the merits of our approach, we utilize a modern set of attack evaluations to better estimate the worst-case performance under the considered union of threat models.

Toward Efficient Robust Training against Union of lp Threat Models
Gaurang Sriramanan, Maharshi Gor, Soheil Feizi Department of Computer Science
University of Maryland, College Park Igaurangs, mgoor, Eefeizi]ee /acs. umd. edu
Abstract
The overwhelming vulnerability of deep neural networks to carefully crafted perturbations known as adversarial attacks has led the development of various training techniques produce robust models. While the primary focus of existing approaches toward addressing the worst-case performance achieved under a single-threat model, it is imperative that safety-critical systems are robust with respect to multiple threat models simultaneously. Existing approaches that address worst-case performance under the union of such threat models (e.g., foc, f2, (1) either utilize adversarial training methods that require multi-step attacks which are computationally expensive in practice, or rely upon fine-tuning of pre-trained models that are robust with respect to single-threat model. In this work, we show that by carefully choosing the objective function used for robust training. it is union of threat
been directed
possible achieve similar, even improved worst-case performance models while utilizing only single-step attacks during the training, thereby achieving a significant reduction in computational resources necessary for training. Furthermore, prior work showed that ad adversarial training against the & threat model is relatively difficult, to the extent that even multi-ste step adversarially trained models were shown to be prone gradient-masking and catastrophic over- wer-ffttingg However, our proposed method- when applied on the & threat model trained solely with single-step adversarial attacks. Finally, to demonstrate the merits of our approach, we utilize modern set of attack evaluations hetter estimate the worst-case performance under the considered union of threat models.
us to obtain the first & robust model
Introduction
Recent years have demonstrated the success of deep learning in solving machine learning tasks spanning across various domains domains-compuree vision, natural language texts, speech, etc. In addition. has even exceeded the human level performance for certain tasks [He et al., 2016, 2015]. However, despite their successes, these systems exhibit severe vulnerabilities: Deep learning models are very susceptible imperceptible perturbations in the input at test time [Szegedy et al., 2013]. Such human- imperceptible noise, known as adversarial attacks, can be used to induce networks to confidently predict incorrect labels, and can thus have disastrous implication safety critical applications such as autonomous navigation and identity verification. To make models robust against such vulnerabilities at test time, a paradigm of adversarial robust training of machine learning models has been developed in recent years [Goodfellow et al., 2015, Madry et al., 2018, 8, Zhang et al., 2019].
These adversarial training procedures have primarily been used to train models robust to a single threat model-pertune perturrabotruuiii constrained within an ball of radius for some p. For instance, the predominant threat model of interest that has been extensively studied in existing literature corre- sponds to the too threat model (mostly Ex 8/255). However, human- camman-imprrcpptbbl adversarial perturbations can be sourced from multiple models- hence in practice, it pertinent to ensure that networks are robust against perturbations from a union of threat models simultaneously. More so, it has been observed that robust training procedures for a chosen threat model are not effective against
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

attacks from other threat models [Trammer and Boneh, 2019, Maini et al., 2020], thus necessitating the development of adversarial defenses against multiple perturbation models simultaneously.
Over recent years, training procedures have been proposed to make systems simultaneously ro- bust against perturbations constrained within union of foo. & and & balls. Systems trained in such manner are then evaluated over the worst-case performance across perturbations from all the threat-models. Tramer and Bonch [2019] proposed simple aggregations of different adversaries for adversarial training against multiple perturbation models utilizing multi-step adversarial attacks for robust training. Maini et al. [2020] further established SOTA for adversarial accuracy against union of (Roe, l1 f2) perturbations through the adversarial training procedure Multi Steepest Descent (or, MSD) that also uses multi-step (k- 50) adversarial attacks generate adversaries for training.
However, these methods, owing to their requirement of great number of adversarial training steps as compared to regular setting for multi-step adversarial training procedure (10 steps), com- putationally inefficient. This leads to our research question: Is it possible to achieve worst-case performance over union of threat models that is similar that of the SOTA methods, while utilizing training procedures that requires only single-step attacks to generate adversaries? We answer the same in affirmation: we first analyse failure modes of existing approaches during & based adversarial training, and thereby propose to use a dynamic curriculum schedule to effectively mitigate robust overfitting. Furthermore, we extend this approach develop a training routine that utilizes a single- step ad adversarial training across a union of threat models to be robust against them simultancously. In summary, we make the following contributions' in this work:
We demonstrate the first successful single-step robust training procedure, NCAT-( to achieve ε robustness by using a curriculum schedule with Nuclear Norm based training.
We extend this approach to propose training procedure NCAT, that yields SOTA-like robust accuracy under the union of multiple lp threat models, while requiring only a single-step attack budget per minibatch.
We further demonstrate that the proposed defense can scale-up to high-capacity networks and large-scale datasets such as ImageNet- 100. Additionally, NCAT trained models generalize to unseen threat models, achieving near-SOTA robustness even on Perceptual Projected Gradient Descent (PPGD), which comprises one of the strongest attacks known to date.
2 Preliminaries
Here, we lay down the notations and conventions used in this work. We denote x to be a d-dimensional image from an N-class dataset D, while its corresponding ground-trute truth label as one-hot vector y. fo represents a Deep Neural Network with parameters 6, that maps an input image x to its pre-softmax output (xr). The cross-entropy loss corresponding the network prediction on sample (1r, y) is denoted as fo (crf(((z)))) y). For a minibatch B = {(x,, y: M we denote X as the image matrix whose 5th row consists of flattened pixel intensitics of the image I,, and Y as the corresponding ground-truth array. Thus, X is matrix of size (M x d), and Y is matrix of size (M x N). Also, (ce(fo()), Y) now denotes the sum of cross-entropy losses over all data samples in the minibatch B. Further, for matrix A, let ||l denote the Nuclear Norm, the of the singular values, of A.
Adversarial Threat Model: In this work, we primarily consider the robustness of Deep Networks against the union of Lx.. é,, and la constrained adversaries. Thus under the foo threat model with an ex consttraain for a given clean image v, an adversarially perturbed counterpart 7 can differ by at most at any given pixel location. In contrast, adversaries under the & constraint may differ from the original image such that the sum of pixel-wise absolute differences are capped by E1. Similarly, for adversaries under lg constraint, the sum of squared pixel-wise differences capped by eg. Further, a network fo is said to be Ep robust under a threat model Ep on a clean sample x with label y, fo( = y, for all perturbations ă such that ||æ - Ellp I Ep-
!Our code and pre-trained models available here: https: / /ttp//ggtttttttt.// www htt//GGurannSrii rawanan//Caattn

---

3 Related Works
In this section we briefly discuss the adversarial attacks and defences that builds up to efficient multi-step adversarial training procedures, work that introduces adversarial training against the union of multiple threat models, and their limitations that we propose to alleviate.
While adversarial training methods have been observed to be the most effective defenses in recent times, early attempts of improving robustness adversarial attacks included input pre-processing based defenses [Guo et al., 2018, Xie et al., 2018, Song et al., 2018] that were computationally cheap. However, such methods primarily relied upon masking of input gradients in order to counter white-box attacks. Several such defenses of this category were circumvented using smooth approximations of the non-differentiable components, by utilizing expectation over randomized components [Athalye et al., 2018, Carlini et al., 2019].
3.1 Effectiveness of FGSM and its limitations
Perhaps the most successful defense which has stood the test of time Projected Gradient Descent or PGD adversarial training [Madry et al., 2018]. This involved minimization of cross-cntrve cross-entropy loss on the worst-case perturbations generated using multiple iterations of constrained optimization, leading to significantly higher computational cost when compared to standard training. Multi-step defenses achieve the state-of-the-an art robustness today and typically use on the order of 10 steps of optimization for attack generation, with each step requiring an additional forward and backward pass. FGSM or the Fast Gradient Sign Method [Goodfellow et al., 2015] based adversarial training alleviates the computational cost by utilizing single-step adversarial samples for training. However, in practice it is observed that during the course of FGSM training, degenerate solutions are frequently encountered, wherein the local linearity assumption of the loss surface is violated. Indeed, Kurakin et al. [2017] showed that such models exhibited the phenomenon of gradient masking, wherein stronger multi-step attacks were seen to reduce the robust accuracy drastically. Wong et al. [2020] proposed to incorporate early-stopping using R-FGSM based adversarial training [TTammrr et al., 2018], in order to identify the failure-point during robust training with single-step adversaries. However, the method was later shown to not be effective on large capacity networks such as the WideResNet [Zagoruyko and Komodakis, 2016] architecture in subsequent work [Sriramanan et al., 2020].
3.2 Nuclear Norm Adversarial Training (NuAT)
Sriramanan et al. [2021] proposes a Nuclear Norm regularizer to improve the adversarial robustness of Deep Networks through the use of single-step adversarial training under foo constraints. This Nuclear Norm Adversarial Training (NuAT) enforces function smoothing in the vicinity of clean samples by incorporating joint batch-statistics of adversarial samples, which results in enhanced robustness. Further, this limits the oscillation of function values and prevents the over-smoothing of loss surface uniformly in all directions, leading a better robustness-accuracy trade-off.
Formally, in a given minibatch B, if X is the matrix composed of row-wise vectorized pixel values of each image, A is matrix of the same dimension as X consisting of independently sampled Bernoulli noise, and Y is the matrix containing the corresponding ground truth one-hot vectors, maximization of the following loss function that utilizes the pre-softmax values fo(-) generates single-step adversaries:
arg max fEEE (Ja(X + A), + A [I + A) - so(X)]l-]
A
Subsequently, the following loss function is minimized during Nuclear Norm adversarial training:
min fCE (fo(X), + + (I + fo())]l.]
(2)
0
3.3 Union of Threat Models
While above described works trains a target network to robust against a single threat model of ba-ball, there has been recent effort in the direction of making models robust against multiple threat models simultaneously. Tramer and Boneh [2019] study the theoretical and empirical trade-offs of adversarial robustness in various settings when defending against aggregations of multiple
3

---

Accuracy
Train Cross-Entron Eooss-Entropy Loss
Val. Cross- Entropy Loss
100
2.5
80
2.0
40
R-FGSM (vel)
R-FGSM (Trair)
50
1.5
R-FGSM-C (vel)
R-FGSM-C SFFGSM-C((Taag) (Train)
40
1.0
20
NuAT
NuAT (train)
20
0.5
NCAT (Va)
(1rwi))
0.0
20
40
60 Epoches
80
100
0
20
400
601
200
100
20
40
043
100
100
Epochss
Epochs
Figure I: Catastrophic Overfitting in é Adversarial Training: To analyze the stability of single- step training. we plot accuracy (left) and cross-entropy losses (centre, right) over epochs of different single-step adversarially trained models. With R-FGSM based adversarial training [Wong et al., 2020], catastrophic overfitting occurs with extreme gradient masking (orange); ad adversarial accuracy (loss) is high (low) on the train set, while being close to zero (high) for validation images. More so, even using curriculum schedule for & adversaries during training only delays the catastrophic overfitting (blue). In contrast, the proposed training approach NCAT (green) does not display catastrophic overfitting due to gradient masking, and is stable over the entire training regime.
adversaries, proposing to train on the average (AVG) or maximizers of loss (MAX) amongst the different threat models considered for each minibatch of samples. Madaan et al. [2020] train using perturbations generated using Meta-Noise Generator, and also propose variant, Stochastic Adversarial Training wherein they utilize multi-step adversaries (10 steps for fo and fz. 20 steps for &.). though the authors note sub-optimal performance from the same. Croce and Hein [2019] propose a provable adversarial defense against all lp norms for p 2 using a regularization term for ReLU networks, by enforcing robustness against foo and l adversaries in particular. Lastly and most importantly, Maini et al. [2020] develops a generalization of the standard PGD-based approach to incorporate multiple perturbation models into single attack by introducing procedure called Multi Steepest Descent, or MSD, and further utilizing it to train standard architectures that simultaneously robust against 8oo. & and lg adversaries.
3.3.1 Multi Steepest Descent (MSD)
The core idea that MSD [Maini et al., 2020] adopts, which helps establish better worst-case accuracies against the union of adversaries, is to create a single adversarial perturbation by simultaneously maxi- mizing the worst-case loss over all perturbation models at each projected steepest descent step. Unlike previous approaches [Tramer and Boneh, 2019] that generate worst-case adversaries for each threat model, or augment adversaries from multiple threat models, MSD chooses projected steepest descent direction in each iteration that maximizes the loss over threat models. This has been established to be superior to the standard adversarial training and the simpler approaches that use comparatively myopic PGD subroutines that only use one perturbation model at a time. However, a clear limitation of MSD here is that it requires 50 adversarial attack steps for each training iteration. Additionally, for each training step, it performs three forward passes (one for each threat model) and a backward pass.
3.3.2 Extreme Norms Adversarial Training (EAT)
In order to achieve robustness against union of ép threat models, Croce and Hein [2021b] propose to fine-tune models that were originally trained to be robust against a single É, norm threat model. The authors demonstrate that fine-tuning robust models to previously unseen l, threat models is effective, in contrast to adversarial fine-tuning of normally trained networks which yields non-robust models. Furthermore, the authors propose train solely on & and &o adversaries, such that other Ep balls on interest are contained within the union of these two threat models (Extreme Norms Adversarial Training or EAT). However, this can place excessive restrictions during robust training if the pertur- bation budget of intermediate Ep adversaries is large. As with MSD [Maini et al., 2020], EAT can be computationally expensive in practice, since it relies upon multi-step adversarially pre-trained models, and further performs robust fine-tuning of such models using 10-step adversaries in the second phase.

---

4 Proposed Method
As noted by prior works [Madry et al., 2018, Croce and Hein, 2021a, Maini et al., 2020], robust training against the & threat model is significantly more complicated when compared to standard adversarial training techniques for foo or lg threat models. Croce and Hein [2021a] al note that even adversarial training using expensive 10-step adversaries generated from SLIDE [Tramer and Boneh, 2019] is prone to catastrophic overfitting [Wong et al., 2020]: Over the course of training, models overfit the adversaries generated, leading to false notion of being robust, while achieving close to 0% accuracy against stronger attacks during test evaluation. While such phenomena are frequently seen in single-step training [Goodfellow et al., 2015], the occurrence of such failure modes even with 10-step adversaries exhibits the difficulty in training 4 robust networks. Recent work [Wong al., 2020, Sriramanan et al., 2020, 2021] has demonstrated that such failure modes can be mitigated through appropriate algorithmic choices such as validation- early stopping using PGD adversaries, or relaxation terms for the overall loss to prevent collapse in training. Croce and Hein [2021a] demonstrate that using the 10-step APGD f attack, robust models can be trained by automatically tuning the sparsity level induced in the & perturbations seen during training. Building upon these in this work, we demonstrate the first successful instance of achieving non-trivial & robustness using single-step adversaries during training. Further, we extend the technique to achieve simultaneous robustness against the union of Ep threat models using only single-step training.
4.1 Analyzing Robust Training with E Adversaries
As mentioned before. Croce and Hein [2021a] point out the intricacies involved with & adversarial training. in that even multi-step training methods can begin to catastrophically overfit. In this work, we seek to identify efficient yet effective single-step training routines that achieve robustness against & ad- versaries. In order build upto that, we first focus on understanding the phenomenon of catastrophic overfitting under this setting and analyze what methods can help alleviate it in the single-step setting.
4.1.1 Nuclear Norm Attack and Curriculum Schedule
We begin by plotting the prediction accuracy and cross-entrope loss of different models over training and validation (Figure 1). We find that when trained with R-FGSM based adversaries, models suffer from catastrophic overfitting early during the training. However, we make a crucial observation that dynamically varying the perturbation budget during training, effectively setting up a curriculum, helps immensely in improving overall stability of training. For instance, with the final ε threat model of in- terest given by the ball of radius 12, we propose to linearly increase this parameter from 0 to 12 to pre- vent catastrophic overfitting. However, applying this curriculum to RFGSM-AT only leads to a delay in catastrophic failure, indicating the unsuitability of using R-FGSM adversaries for robust training.
Since the goal here is to utilize only single adversarial training step. it becomes imperative that the loss that we optimize over to generate the adversaries is smooth and does not showcase gradient masking. Hence, we build upon Nuclear-Norm Adversarial Training (NuAT) [Sriramanan et al., 2021] to generate single-step adversaries. NuAT proposed to generate -dverssrss by maximizing the Nuclear Norm regularized objective. We apply the same for generating adversaries like follows:
ì. ī= tCE (Jo(X + 2), + À [1 + A) - fx(X)ll
(3)
Sriramanan et al. [2021] note that since the Nuclear norm forms a tight convex relaxation for the rank of the predicted matrix of logit values, the corresponding attack generates diverse adversaries in a given minibatch, which then helps mitigate robust overfitting. Crucially, we observe however that this supplemental attack diversity not sufficient for single-step training on the l threat model, as even NuAT is observed to be susceptible to catastrophic failure in Fig-1. However, by utilizing the dynamic curriculum schedule, this phenomenon is successfully remedied in the proposed method, NCAT.
4.1.2 Steepest Ascent with Single-Step Optimization
Given a minibatch of samples X, we generate the Nuclear Norm attack by identifying a perturbation that maximizes the loss L as in Eq-3 and conforms to the following two constraints:
|||| S X + A e (0, 1jd
5

---

wherein the second constraint arises from the normalized range for pixel intensities of an image. Assuming a first-order Taylor series approximation for the loss incurred by the network fa. if V Vall represents the gradient direction, the steepest ascent direction A to maximize loss would be parallel to the same in the unconstrained setting. For notational convenience, without loss of generality, consider the optimization problem for an image z of the minibatch X. Thus if g represents the corresponding gradient for image I with respect to the loss L, for steepest ascent of loss we have:
max Σ 9:d,
such that
(a) 0 S x4 + 6, 14 and (b) ||8|| S 61
In the absence of constraint (b), the optimal perturbation ò given by d = M where M, denotes the deviation budget required at pixel to saturate the same to the pixel constraint [0, 1], parallel to the gradient gs, and can defined formally as:
M, - { 11
Ij
if 2 0
Iz if g: <0
When the overall available budget is limited by E1 as in constraint (b), such that ||M]| < E1, the constraint is inactive, and the solution is unaltered. On the other hand, if constraint (b) is active, the solution necessarily different, wherein a reduction in the perturbation allocated for some pixel locations is made mandatory. Thus, the inner- inner-product in Eq-4 is maximized by assigning the perturbations M; in priority-ordet for different pixel locations, based on decreasing magnitude of the absolute gradient values. Thus, if a; = Ige| represents the gradient magnitudes at different pixel locations, let a denote the sorted permutation of indices such that ds(1) 2 aa(2) 2 2 Co(d)- Further, let the cumulative budget utilized be defined as S; = = = for i (1....,d) d). with Sy = 0. Since each term [Ma(j)] in the summand is positive, S; increases monotonically. Thus with I, = max(0, E - } denoting support variables for indices which receive a lower perturbation allocation due to constraint (b), the optimal single-step perturbation 8* corresponding to image is then defined as:
{{
if S; S E1 Mo(s) 4 if
Mo(t)
(3)
E1
Thus, in an & constrained attack the gradients have to be sorted by their magnitude, which requires O(d log d) complexity where d represents the image dimensionality. However, in practice this overhead is observed to be exceedingly minimal relative to data loading times etc. Indeed, using this routine with Nuclear norm based training with the curriculum schedule (NCAT-L1), we demonstrate for the first time that single-step training can be effectively used to produce & robust models.
4.2 Scheduling Curriculum for Robust Training
In order to improve the efficacy of single-step adversarial training, we propose to utilize dynami- cally varying perturbation budget during training, to ensure that the model receives supervision on increasingly difficult adversaries as determined by a Curriculum schedule, As remarked previously from 1igure-1, robust training using single-step adversaries gains significant degree of stability with Curriculum schedule. Different curricula are seen to be effective in practice, including those that incorporate the robust accuracy on training samples observed at a given epoch to dynamically vary the perturbation budget at different rates. However, these complex curricula require additional fine-tuning of hyperparameters, and are sensitive to the lp threat model considered. We thus utilize a simple, linear curriculum wherein the perturbation size is increased from zero to E, and kept constant for the last few epochs, to facilitate adequate fine-tuned training on the final threat model of interest.
The curriculum schedule proposed here is particularly effective for single-step training given that it does not require sampling of adversarial statistics different radii. contrast, prior works largely require an inherent multi-step adversarial generation approach to sample different stopping points to implement a curriculum in practice. For instance, Zhang et al. [2020] proposed Friendly Adversarial Training (FAT), wherein early stopping is performed during the generation of the multi-step PGD attack - the perturbation updates are stopped as soon as the PGD attack successfully induces misclassification the training image.

---

Algorithm Nuclear Curriculum Adversarial Training for Ep Norm Robustness
1: Input: Network fo with parameters 0. Weight Averaged Network fa with parameters Training Data D with imput images of dimension d, Minibatch Size M, Attack Size Ep for each lp threat model, Epochs E, Learning Rate 7). Decision Function D, Curriculum Schedule C
2: for epoch = 1 to E do
3: Cp C(p)
4: for minibatch {(T, 1.)
C D do
81
1/1
5:
X
=
A
6, ~ (-an)) Y=
JM
UM
6:
-= lee (X Y) + A.
[X
A) (x)
7:
for in D(0) do
8:
A A E Proj Val, By(sp)
9:
X Clamp (X A,0,1
10:
end for
11:
I. = lCE (X), Y) x. [15o(X) f6(X)
1
12:
0 = 0 M 17 VoL
w T. w + (1 - -7
14: end for
15:
end for
4.3 Single-Step Training for &oo and l Robustness
Since the fo threat model is the most well-studied setting in existing literature, we rely upon prior works to obtain excellent baselines. To achieve robustness against foo constrained adversaries using single-step training, we utilize the current state-of-the-ar method, Nuclear Norm Adversarial Training (NuAT) [Sriramanan al., 2021]. We further seek incorporate other threat models during training, in order to obtain models with non-trivial robustness against the union of the Lı, & and Eoo threat models simultaneously. In order efficiently train against f2 adversaries, we first propose modify the NuAT training algorithm to utilize this constraint set, using & norm based projections. However, similar Croce and Hein [2021b], we make the remarkable observation that models that are trained solely on foo adversaries achieve a great degree of robustness versus l adversaries on the test set. We observe similar transfer of robustness from l trained models toward the l threat model well. Thus, the primary difficulty in achieving robustness to the union of threat models appears to be that of training networks robust the and loo threat models particular.
4.4 Sampling Procedures to Improve Efficiency
To achieve robustness against the union of the three l, threat models considered, is plausible that training with three distinct single-step attacks (constrained to f,, l and fxx) using the proposed approach in each minibatch will be effective. However, in this work, we primarily focus on reducing the training complexity further, in order to effectively utilise only a single-step attack for each minibatch. Awasthi et al. [2021] proposed to utilize the multiplicative weights algorithm, wherein the loss under different adversaries on hold-out validation set guides the sampling procedure, using a set exponential running weights wN for each threat model. However, we find that this is contingent on the efficacy of adversaries utilized on the validation set, which can be restrictive in practice due to computational constraints. In practice, we observe that it is indeed sensitive to the degree of convergence achieved by different adversaries, and requires additional tuning for the update coefficient hyperparameters.
Building upon this, we find in practice that alternating between foc and & attacks across different minibatches with a fixed frequency is remarkably effective. Thus, the proposed defense, NCAT, uses nuclear norm based single-step training following a curriculum schedule, such that different threat models are selected for attack generation in different minibatches based on a fre-fixed frequency. We present concise, summarised overview of the proposed training approaches in Algorithm-
7

---

Table 1: Consolidated Results on CIFAR-10: Prediction accuracy (%) of ResNet-18 models trained using different methods under various threat models. Robust evaluations are presented under constraint sets given by the ball of radius 12, l ball of radius 0.5 and &or ball of radius 8/255 comprising the individual threat models of interest, along with worst-case and average-case performance under the union of these threat models.
Method
Number of Clean Acc
Worst-Case
Average Acc
f Acc
f2
AT Steps
Acc
Ace
Acc
& Training Alone
APGD-&
10
85.9
22.1
48.8
59.5
64.9
22.1
NCAT-4
81.1
37.9
53.6
55.9
67.0
38.0
Training under Union of Threat Models
SAT
13.33
83.9
40.4
54.2
54.0
68.0
40.7
AVG
30
84.6
40.1
53.8
52.1
68.4
40.8
MAX
30
80.4
44.0
53.4
48.6
66.0
45.7
MSD
50
81.1
43.9
53.4
49.5
65.9
44.9
EAT
10
82.2
42.4
54.6
53.6
67.5
42.7
NCAT
80.3
42.6
53.3
46.9
67.0
46.0
NCAT
77.5
43.7
53.4
48.4
65.7
46.1
1. Here, the Decision Function D (L7, Alg-I) alternately outputs - or oc based on predetermined frequency, since such models are observed simultaneously achieve &g robustness without explicit training. As observed in prior works [Chen et al., Sriramanan et al., 2021], maintaining a exponential running average of network weights (SWA [Izmailov al., 2018J) helps improve robust performance overall as well, particularly so in this setup since different (random) minibatches are trained with adversarial perturbations arising from different threat models. Furthermore, this effectively reducing undesired bias to particular threat model due to auto-correlations that arise in training. We thus use these exponentially averaged models for final evaluation. A modified version of the proposed approach, namely NCAT-AVG, uses Decision Function D that outputs the collection of = {1, 2, ). and effectively uses a budget of three single-step attacks, one for each threat model. We present results obtained using NCAT-AVG, sampling with multiplicative weights based updates and other ablations in the Supplementary Material.
5 Experiments and Analysis
In this work, we primarily consider the CIFAR-10 [Krizhevsky et al., 2009] and ImageNet- 100 [Russakovsky et al., 2014] datasets, since they have come to form the benchmark for comparative analysis of adversarially robust models. Following prior works [Maini et al., 2020], we consider constraint sets given by the lx ball of radius 8/255, &g ball of radius 0.5 and & ball of radius 12 as the threat models of interest, and as explained previously, we attempt to train models that achieve non- trivial worst-case accuracy against the union of such lp threat models. For the ImageNet- 100 dataset, the corresponding radii for é,, l2 and loo threat models are 255, 1200/255 and 4/255 respectively, following the constraints considered by Laidlaw al. [2021].
We present results the white-box setting, wherein the adversary is cognizant the model weights, architecture and training scheme employed. To accurately estimate worst-case performance, focus our evaluation pipeline to incorporate state-of-the-an attacks such as AutoAttack [Croce and Hein, 2020] for each f, threat model. Furthermore, AutoAttack includes strong l attack evaluation baselines using techniques proposed by Croce and Hein [2021a], wherein the authors note that significant improvement in attack efficacy as compared to prior works such as SLIDE Attack [Tramee and Boneh, 2019], B&B Attack [Brendel et al., 2017] and Pointwise Attack [Schott et al., 2018]. Due to paucity of space, we include black-box evaluations, generalization to unseen domains, gradient masking checks and adaptive attacks in the Supplementary Material. We however make the important note that the suite of white-box attacks considered using AutoAttack are strictly stronger than the former set of attack evaluations, indicating the absence of gradient masking.

---

Table 2: Consolidated Results on CIFAR-10: Prediction accuracy (%) of WideResNet-28- models trained using different methods under various threat models. Robust evaluations are presented under the constraint sets given by the & ball radius 12, & ball of radius 0.5 and foo ball of radius 8/255 comprising the individual threat models of interest, along with worst-case and average-case performance under the union of these threat models.
Method
Number of Clean AT Steps
Worst-Case
Average Acc
f Acc
f2
to Acc
Acc
Acc
Ace
& Training Alone
APGD-&
10
83.7
30.7
52.5
61.6
65.1
30.7
NCAT-4
80.7
39.2
54.6
56.1
68.6
39.3
Training under Union of Threat Models
SAT
13.33
80.5
45.7
56.2
55.9
66.7
45.9
AVG
30
82.5
45,1
56.1
55.0
68.0
45.4
MAX
30
79.9
47.4
54.6
50.2
65.3
48.4
MSD
50
80.6
46.9
55.1
51.7
65.6
48.0
EAT
10
79.9
46.4
56.3
56.0
66.2
46.6
NCAT
81.5
44.6
54.8
49.9
68.3
46.3
We first present results obtained using the ResNet-18 [He et al., 2016] architecture on CIFAR-10 in Table-1. In the first partition of the table, we present models trained solely the l threat model. The current state-of-the-arr the is achieved by APGD- [Croce and Hein, 2021a], which performs 10-step APGD attack during training in order to mitigate gradient masking and catastrophic overfitting for & constrained adversaries. On the other hand, our method, NCAT-& which uses just single-step attack for adversarial training achieves l robustness much more efficiently. We note that while the multi-step approach has higher & robustness (+3.6%), ) the single-step NCAT trained model has significantly better worst-case (+15.8%) and average- case (+4.8%) accuracy under the union of all three threat models. Indeed, we note once again that NCAT-E, represents the first-ever successful single-step adversarial training on the threat model, which also generalizes well to the unseen foo. and E threat models simultaneously.
In the second partition of Table-1, we present models that are explicitly trained to be robust under the union of the Lo., le and & threat models. Namely, we present comparative analysis with respect to existing multi-step ad adversarially trained defenses such as AVG and MAX [Tramer and Boneh, 2019], MSD [Maini et al., 2020], EAT [Croce and Hein, 2021b] and SAT [Madaan et al., 2020]. For these methods, we primarily utilize robust evaluations as presented by Croce and Hein [2021b] to enable fair comparisons, which comprise of re- implemented models that obtain higher accuracies compared to values reported in the original papers. We first note that SAT' requires 13.33 adversarial attack steps during training, since utilizes 10-step attacks for foc and & adversaries, and 20 attack steps for & adversaries to mitigate gradient masking, indicating the considerable difficulty involved in achieving & robustness. In contrast, EAT relies upon 10-step fine-tuning of network that is already robust against single threat-model. The current state-of-the-an art approaches comprise of MSD and MAX that achieve 44% worst-case accuracy, while utilizing a budget of 50 and 30 attack steps respectively during training. observe that the proposed approach, NCAT achieves comparable worst-case and average-case performance over the threat models considered, while requiring a significantly smaller computational footprint during training. The relative trade-off between clean accuracy and robustness can be adjusted depending individual use-case requirements by tuning the coefficient (A) of the Nuclear norm regularization term in the overall loss objective. The proposed method facilitates this trade-off, and is presented in the last row - wherein this coefficient is increased so to achieve near-Son SOTA robust performance, at the cost of lower clean accuracy.
In Table-2, we present results on models trained with the WideResNet-22 28- [Zaoruyko and Komodakis, 2016] architecture to demonstrate the scalability of the proposed defense to high-capacity networks. We thus establish the efficacy of the curriculum schedule combined with nuclear norm based training in mitigating catastrophic overfitting, enabling efficient training these large networks. Similar to previous remarks, we again note that while the multi-step APGD-(, model has higher L

---

Table 3: Results on ImageNet-100: Prediction accuracy (%) of models trained using different methods under various threat models. Robust evaluations are presented under the constraint sets given by the 4 ball of radius 255, & ball of radius 1200/22 and fer ball of radius 4/255 comprising the individual threat models of interest, along with worst-case and average-case performance under the
union of these threat models.
Method
Number of AT Steps
Arch
Clean Acc
Worst-Case Acc
Average Acc
&
f2
fx Acc
PPGD
Acc
Acc
Acc
AT
10
RN50
81.7
0.8
20.7
0.8
3.7
55.7
1.5
PAT
10
RN50
72.6
37.8
41.2
41.2
37.7
45.0
29.2
NCAT-&
RN18
64.9
41.1
43.9
48.3
41.4
42.1
26.6
NCAT
1
RN18
63.9
41.5
44.8
46.8
41.9
45.7
29.1
robustness (+5.5%), the single-step NCAT-&2 trained model has improved worst-case (+8.5% and average-case (+2.1 accuracy under the union of all three threat models.
In Table-3, we present evaluations on the ImageNet-100 100 dataset, wherein we utilize ResNet- 18 net- works to reduce computational demands. We observe that NCAT-&, attains remarkable robust accuracy on unseen &oo and &g ad versaries, even achieving 26.6% accuracy against the Perceptual Projected Gra- dient Descent (PPGD) attack [Laidlaw et al., 2021], which forms one of the strongest attacks known to date. Further, we observe that the NCAT trained achieves ttae-oofthhe-ae worst-case lp accuracy, while attaining robustness similiar to that of Perceptual Adversarial Training [Laidlaw et al., 2021] for the PPGD attack, with the latter being model that was explicitly trained such adversaries.
Conclusions
In this work, we develop an efficient adversarial training procedure, NCAT, train networks that are robust against a union of Ep threat models, namely foo, f and l.. To do so, we first focus efficient, yet effective robust training procedure for the e threat model, by incorporating a curriculum schedule to mitigate catastrophic overfitting. Indeed, in this work we present the first & constrained robust model trained solely using single-step adversaries, achieving robustness similar to that of multi-step SOTA approaches. Furthermore, we extend the proposed method to achieve worst-case robustness under multiple lp norm constraints simultaneously. Compared to the current SOTA that uses 30 adversarial attack steps for its training procedure to achieve 44% robust accuracy on CIFAR-10, our method yields 43.7% robustness while solely utilizing single-step adversaries during the training routine. This thereby greatly reduces the computational requirements needed achieve SOTA-equivalent robust performance.
developing
Limitations and Societal Impact
The proposed approach, NCAT, facilitates efficient yet effective training of robust models, that achieve non-trivial worst-case accuracy under the union of several lp norm threat models. Furthermore, NCAT trained models seen to robust against unseen attacks and adversaries that might potentially be encountered the real world. Thus, given that NCAT helps train reliable and trustworthy models using small computational footprint, this work indeed has potential in creating a positive impact on society. We do not see any immediately foreseeable negative consequences associated with our work. Further, in order to surmount existing limitations, works could extend NCAT to include provable certificates for different Ép threat models and their union, and possibly provide stricter guarantees for the absence of gradient masking and catastrophic failure with the use of sereeiiccallymoivvtttte curriculum schedules.
hope that future
8 Acknowledgements
This project was supported in part by NSF CAREER AWARD 1942230, a grant from NIST 60NANB20D134, HR001 19S0026 (GARD), HR001 12090132 ONR YIP award N00014-22-1- 2271, Army Grant No. W91 INN21 and the NSF award CCF221 12458. We would also like to thank Tianyi Zhou and David Jacobs for fruitful discussions on initial versions of this work.
10

---

References
A. Athalye. N. Carlini, and D. Wagner. Obfuscated gradients give false seuse of security: Cireumventing defenses adversarial examples. International Conference on Muchine Learning (ICML), 2018. 3
Awasthi, G. Yu, C.S. Ferng, A. Tomkins, and D.-C. Juan. Adversarial robustness representation spaces. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7608-7616, 2021. 7
W. Brendel, J. Rauber, and M. Bethge. cerision-based adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprint
2017. 8
N. Carlini, A. Athalye, N. Papernot, W. Brendel, J. Rauber, D. Tsipras, I. Goodfellow, and A. Madry. On evaluating adversaria robustness. arXiv preprint 192.00390..06055. 2019. 3
T. Chen, Z. Zhang, S. Liu, S. Chang, and Z. Wang. Robust overfitting may be mitigated by properly learned smoothening. 8
F. Croce and M. Hein. Provable robustness against all adversarial L--pertuuraations for p 2 1. arXiv preprint arXiv:I905.1 1213, 2019.
F. Croce and M. Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning (ICML), 2020. 8
Croce and M. Hein. Mind the box: 4-APGD for sparse adversarial attacks image classifiers. 139: 2201-2211, 2021a. 8,9
Croce and M. Hein. Adversarial robustness against multiple 1,-threat models the price of and how quickly fine-tune robust models to another threat model. May 2021b. 4, 4,7,9
J. Goodfellow, Shlens, and Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015. 3,
C. Guo, M. Rana, M. Cisse, and van der Maaten. Countering adversarial images using input transformations. In International Conference on Learning Representations (ICLR), 2018. 3
K. He, X. Zhang, S. Ren, and Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. arXiv preprint arXiv: Arxiv-1502.01857 2015.
K. He, X. Zhang. S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 1,
P. Izmailov, D. Podoprikhin, T. Garipov, D. Vetrov, and A. G. Wilson. Averaging weights leads to wider optima and better generalization. arXiv preprint vrrXiv:183..5444 2018. 8
A. Krizhevsky et al. Learning multiple layers of features from tiny images. 2009. 8
A. Kurakin, I. Goodfellow, and S. Bengio, Adversarial machine learning at scalc. In International Conference on Learning Representations (ICLR), 2017. 3
C. Laidlaw, S. Singla, and S. Feizi. Perceptual ad adversarial robustness: Defense against unseen threat models. In ICLR, 2021. 8, 10
D. Madaan, J. Shin, and S. J. Hwang. Learning to generate noise for multi-attack robustness, 2020. URL dttps://arxivv org/ 2200 12135. 4,
A. Madry, A. Makelov, Schmidt, Dimitris. and A. Vladu. Towards deep learning models resistant adversarial attacks. In International Conference Learning Representations (ICLR), 2018. 3,
Maini, E. Wong, and J. Z. Kolter. Adversarial robustness against the union of multiple perturbation models. In International Conference on Machine Learning, 2020. URL https:://araie www.oogg///.o..o.ooooooo...o...r aba/ 04068. 2, 4, 5, 8, 9
O. Russakovsky, Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. Imagenet large scale visual recognition challenge. CoRR, Abs//449.0055, 2014, URL http: /eat:://aaee //wxxvv/vvvvvv/////////a/////vv org/ abe/ /1409.40..00 0075.. 8
L. Schott, J. Rauber, M. Bethge, and W. Brendel. Towards the first adversarially robust neural network model on MNIST. Sept. 2018. 8

---

Y. Song, Т. Kim, S. Nowozin, S. Ermon, and N. Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In International Conference on Learning Representations (ICLR), 2018.
G. Sriramanan, Addepalli, Baburaj, and R. Venkatesh Bahu. Guided Adversarial Attack for Evaluating and Enhancing Adversarial Defenses. In Advances Neural Information Processing Systems (NeurIPS), 2020. 3,
G. Sriramanan, Addepalli, A. Baburaj, and R. V. Babu. Towards efficient and effective adversarial training. In Advances in Neural Information Processing Systems, 2021. URL https: //tp:////////eeeerrr/ee/ee wwtp:/eeeetomiiidee kuK2VARZGnI. 7.8
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, 1. J. Goodfellow, and R. Fergus. Intriguing properties of neural networks. In International Conference Learning Representations (ICLR), 2013.
F. Tramer and D. Boneh. Adversarial training and robustness for multiple perturbations. Advances in Neural Information Processing Systems, 32, 2019. 2, 3, 4, 8,
F. Tramēr, A. Kurakin, N. Papernot, Goodfellow. D. Boneh, and McDaniel. Ensemble adversarial training: Attacks and defenses. In International Conference on Learning Reprexentations (ICLR), 2018. 3
Wong, Rice, and Kolter. Fast is better than free: Revisiting adversarial training. International Conference on Learning Representations (ICLR), 2020. 3, 4,
C. Xie, Wang, Z. Zhang, Ren, and A. Yuille. Mitigating adversarial effects through randomization. In International Conference on Learning Representations (ICLR), 2018. 3
Zagoruyko and N. Komodakis. Wide residual networks. arXiv preprint arXi::1605.07144 2016. 3, 3
Zhang. Yu, J. Jiao, E. Xing, L. El Ghaoui, and M. I. Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference on Machine Learning (ICML), 2019.
J. Zhang, X. Xu, Han, G. Niu, L. Cui, M. Sugiyama, and M. Kankanhalli, Attacks which do not kill training make adversarial learning stronger. In ICMI,, 2020. 6
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section-6,7
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section 6,7
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] Further details included in the Supplementary Material
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material URL)? [Yes] Details included in the Supplementary Material
(b) Did you specify the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Details included in the Supplementary Material
(c) Did you report error hars (e.g., with respect to the random seed after running experiments multiple times)? [Yes] Details included in the Supplementary Material
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Details included in the Supplementary Material
If you are using existing assets (e.g.. code, data, models) or curatingfreleasngg new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] Further details included in the Supplementary Material
(b) Did you mention the license of the assets? [Yes] Details included in the Supplementary Material
12

---

(c) Did you include any new assets either in the supplemental material or a URL? [Yes] Details included in the Supplementary Material
(d) Did you discuss whether and how consent was obtained from people whose data you'l re us- ing/curating? [N/A]
(c) Did you discuss whether the data you using/curating contains personally identifiable informa- tion or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "nlEqewxUrMp", "reviewer": "Reviewer_LkCw", "summary": "The paper proposes an efficient robust training method to achieve adversarial robustness on multiple threat models (l_1,l_2,l_\\inf). Specifically, it first study the adversarial training on  l_1 norm where normal adversarial training method could achieve a good performance. By combining the previously proposed nuclear-norm adversarial training and curriculum schedule, it finds it could achieve a better robust accuracy against l_1 adversarial attack. As l_2 and l_inf threat model have a similar behavior on the robustness, the paper then choose to alternatively switch the threat model to do nuclear-norm adversarial training to improve the efficiency. The experiments are conducted on cifar10 and imagenet100 on ResNet18 and wideresnet28. It shows the proposed method could achieve a better worst-case robustness while maintain a good scalabliilty.", "questions": "1. Does the weight average on the model parameter matter in Algorithm 1 line 13? I don't see any explanation on that. Also, which parameter is used in the end? theta or w?\n2. The curriculum scheduling part is not clear to me. Could you elaborate more on that?\n3. If the threat model doesn't include l_1 threat model, will the proposed method still have a good performance?", "limitations": "The papers has discussed its limitation on provable certificates on different l_p threat models and their union. However, I think one of big limitation is the paper requires the union to have l_1 threat model.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n1. The paper is in general well-written and easy to follow except for the curriculum scheduling part.\n2. The experiment result shows the proposed method could achieve a worst-case robust accuracy.\n\nCons:\n1. The novelty of proposed method is quite limited. It is a fairy combination on nuclear norm adversarial training with the curriculum schedule. And curriculum schedule is also discussed in the previous works such as [1]. It is still unclear why the curriculum schedule is useful in the adversarial training or the proposed schedule only works for the l_1 threat model.\n2. The main claim of improving the union of threat model is actually improving the adversarial training's performance on the l_1 threat model, which is not equal to improve the performance over the union. In other words, if the union doesn't include l_1 threat model, will the proposed method still outperform the baseline?\n3. Although the proposed method is better in terms of the worst-case acc, they are sometimes worse on the average. The main improvement I think is on the training efficiency.\n\nMinor:\n1. Typos: line 219 a_\\sigma(1)\\geq a_\\simga(1).. \n2. The dash line in Figure 1 (b)(c) is inconsistent with the legend.\n\n[1] Zhang, Jingfeng, et al. \"Attacks which do not kill training make adversarial learning stronger.\" International conference on machine learning. PMLR, 2020.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "NHelPKP1ZMV", "reviewer": "Reviewer_TwcU", "summary": "This paper presents a robust training against the union of $l_p$ threat models, i.e., training a robust model in l_inf, l_2, l_1, threat models. The key idea is to use a nuclear norm-based training procedure called NCAT. ", "questions": "Please report the actual computation time for training using NCAT vs. other baseline adversarial training methods.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n-----------\n- NCAT can achieve SOTA robust accuracy while requiring a single-step attack budget per mini-batch\n- NCAT can scale to large datasets\n\n\nWeaknesses\n--------------\n- The actual training performance (in terms of computation time) is not reported. Therefore, it is hard to compare the actual performance of NCAT against other robust training methods. \n\n- Prior work (Sriramanan et al.) 2021 has already used nuclear norms in the context of adversarial training, albeit for single-step attacks.   ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "N7oJjUqwEkK", "reviewer": "Reviewer_mBSh", "summary": "The paper proposes a method, NCAT, to obtain models robust wrt $\\ell_1$ or multiple $\\ell_p$-norms efficiently, that is with single step adversarial training. NCAT combines Nuclear-Norm Adversarial Training (NuAT) with a curriculum for the size of the perturbations seen at training time to prevent catastrophic overfitting in 1-step adversarial training. Further, the paper applies NCAT to the multiple norms scenario, achieving results close to those of more expensive multi-step methods.", "questions": "- While the paper is well-written, I think some more details about the curriculum (and maybe some training details) could belong to the main part, since that's one of the main components of NCAT. Also in Table 1, NCAT+ appears but I couldn't find where it's defined.\n\n- An efficient method is particularly useful when training is particularly expensive, then it'd be interesting to scale NCAT to the full version of ImageNet.", "limitations": "The limitations are sufficiently addressed.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The goal of reducing the cost of adversarial training without degrading (too much) the resulting robustness is well established, and to my knowledge this is the first work to extend single step adversarial training to $\\ell_1$ and multiple norms.\n\n- The proposed method is simple and effective in the experimental evaluation on both CIFAR-10 (with two architectures) and ImageNet-100. Several additional experiments are presented in the supplements to further analyze and support NCAT.\n\nWeaknesses\n- NCAT is built combining several existing techniques (NuAT, curriculum for $\\epsilon$, steepest ascent direction for the threat models of $\\ell_1$ with the image domain constraints, alternating $\\ell_1$ and $\\ell_\\infty$ when training for multiple norms), which to some extent limits the contributions of the paper.\n\nOverall, I think that the novelty of the method is somehow limited, showing how it's possible to achieve with single step adversarial training robustness in challenging threat models a meaningful contribution.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2oKFM6kLDW", "reviewer": "Reviewer_4QVR", "summary": "This work tackles the lucrative goal of multi-norm perturbation robustness, which is largely constrained by efficient robustness against $L_1$-norm adversaries in the current literature. The authors propose a technique inspired by the Nuclear-Norm regularizer that helps achieve robustness against a union of adversaries efficiently. The curriculum-based method has clear performance gains, especially with complex datasets and large models.", "questions": "There is a non-trivial drop in clean-model accuracy when using the proposed method. I believe this is probably a side effect of learning more robust models, thereby limiting weak dataset-specific signals that the model might otherwise be latching on to. Nonetheless, it would be nice if the authors could address this drop better and do some kind of qualitative analysis. For instance, visual/statistical inspection of the (additional) cases where the models go wrong may help identify problematic examples (mislabeled), or specific concepts that seem to be hard if aiming for robustness as well.", "limitations": "Although societal impact is addressed in Section 7, I do not see much discussion around limitations.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n- The paper is well written, with easy-to-follow reasoning for design/algorithmic choices, buildup, and review of relevant literature. Evaluation is also thorough (although I would prefer to see at last least some mention of deviation in results across runs).\n- The observation (and perhaps affirmation) of $L_1$ and $L_{\\infty}$ being the hardest threat models while aiming for union robustness is nice.\n\n# Weaknesses\nAs such, I see no visible weaknesses in the paper, apart from minor comments mentioned below (and in the questions section). Good job!\n\n# Minor comments\n- Table 1: What does the $^+$ in NCAT$^+$ stand for? Please clarify in the table description.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept