Toward Efficient Robust Training against Union of ℓ p Threat Models
Anonymous Author(s)
AffiliationAddress email
Abstract
The overwhelming vulnerability of deep neural networks to carefully crafted perturbationsknown as adversarial attacks has led to the development of various training techniques toproduce robust models.
While the primary focus of existing approaches has been directedtoward addressing the worst-case performance achieved under a single-threat model, it isimperative that safety-critical systems are robust with respect to multiple threat modelssimultaneously.
Existing approaches that address worst-case performance under the union ofsuch threat models (e.g., ℓ ∞ , ℓ 2 , ℓ 1 ) either utilize adversarial training methods that requiremulti-step attacks which are computationally expensive in practice, or rely upon fine-tuningof pre-trained models that are robust with respect to a single-threat model.
In this work,we show that by carefully choosing the objective function used for robust training, it ispossible to achieve similar, or even improved worst-case performance over a union of threatmodels while utilizing only single-step attacks during the training, thereby achieving asignificant reduction in computational resources necessary for training.
Furthermore, priorwork showed that adversarial training against the ℓ 1 threat model is relatively difficult, tothe extent that even multi-step adversarially trained models were shown to be prone togradient-masking and catastrophic over-fitting.
However, our proposed method—whenapplied on the ℓ 1 threat model specifically—enables us to obtain the first ℓ 1 robust modeltrained solely with single-step adversarial attacks.
Finally, to demonstrate the merits of ourapproach, we utilize a modern set of attack evaluations to better estimate the worst-caseperformance under the considered union of threat models.
Introduction
Recent years have demonstrated the success of deep learning in solving machine learning tasksspanning across various domains—computer vision, natural language texts, speech, etc.
In addition,it has even exceeded the human level performance for certain tasks [He et al., 2016, 2015].
However,despite their successes, these systems exhibit severe vulnerabilities: Deep learning models are verysusceptible to imperceptible perturbations in the input at test time [Szegedy et al., 2013].
Such humanimperceptible noise, known as adversarial attacks, can be used to induce networks to confidentlypredict incorrect labels, and can thus have disastrous implication in safety critical applications such asautonomous navigation and identity verification.
To make models robust against such vulnerabilitiesat test time, a paradigm of adversarial robust training of machine learning models has been developedin recent years
[Goodfellow et al., 2015, Madry et al., 2018, Zhang et al., 2019].
These adversarial training procedures have primarily been used to train models robust to a singlethreat model—perturbations constrained within an ℓ p -ball of ε p radius for some p .
For instance,the predominant threat model of interest that has been extensively studied in existing literaturecorresponds to the ℓ ∞ threat model (mostly ε ∞ = 8 / 255 ).
However, human-imperceptible adversarialperturbations can be sourced from multiple threat-models; hence in practice, it is pertinent to ensurethat networks are robust against perturbations from a union of threat models simultaneously.
More so,it has been observed that robust training procedures for a chosen threat model are not effective against attacks from other threat models [Tramer and Boneh, 2019, Maini et al., 2020], thus necessitating thedevelopment of adversarial defenses against multiple perturbation models simultaneously.
Over recent years, training procedures have been proposed to make systems simultaneously robustagainst perturbations constrained within a union of ℓ ∞ , ℓ 1 and ℓ 2 balls.
Systems trained in such manner are then evaluated over the worst-case performance across perturbations from all the threat-models.
Tramer and Boneh [2019] proposed simple aggregations of different adversaries for adversarial training against multiple perturbation models utilizing multi-step adversarial attacks for robust training.
Maini et al.
[2020] further established SOTA for adversarial accuracy against union of ( ℓ ∞ , ℓ 1 , ℓ 2 )perturbations through the adversarial training procedure Multi Steepest Descent (or, MSD) that alsouses multi-step ( k = 50 ) adversarial attacks to generate adversaries for training.
However, thesemethods, owing to their requirement of great number of adversarial training steps as compared to aregular setting for multi-step adversarial training procedure (10 steps), are computationally inefficient.
This leads to our research question: Is it possible to achieve worst-case performance over a unionof threat models that is similar to that of the SOTA methods, while utilizing training procedures thatrequires only single-step attacks to generate adversaries?
We answer the same in affirmation: wefirst analyse failure modes of existing approaches during ℓ 1 based adversarial training, and therebypropose to use a dynamic curriculum schedule to effectively mitigate robust overfitting.
Furthermore,we extend this approach to develop a training routine that utilizes a single-step adversarial trainingacross a union of threat models to be robust against them simultaneously.
In summary, we make thefollowing contributions in this work:
• We demonstrate the first successful single-step robust training procedure, NCAT- ℓ 1 , to
achieve ℓ 1 robustness by using a curriculum schedule with Nuclear Norm based training.• We extend this approach to propose a training procedure NCAT, that yields SOTA -like robustaccuracy under the union of multiple ℓ p threat models, while requiring only a single-stepattack budget per minibatch.
• We further demonstrate that the proposed defense can scale-up to high-capacity networks andlarge-scale datasets such as ImageNet-100.
Additionally, NCAT trained models generalizeto unseen threat models, achieving near- SOTA robustness even on Perceptual ProjectedGradient Descent (PPGD), which comprises one of the strongest attacks known to date.
Preliminaries
Here, we lay down the notations and conventions used in this work.
We denote x to be a d -dimensionalimage from an N -class dataset D , while its corresponding ground-truth label as a one-hot vector y .
f θrepresents a Deep Neural Network with parameters θ , that maps an input image x to its pre-softmaxoutput f θ ( x ) .
The cross-entropy loss corresponding to the network prediction on a sample ( x, y ) isdenoted as ℓ CE ( f θ ( x ) , y ) .
For a minibatch B = { ( x i , y i ) } Mi =1 , we denote X as the image matrixwhose i th row consists of flattened pixel intensities of the image x i , and Y as the correspondingground-truth array.
Thus, X is a matrix of size ( M × d ) , and Y is a matrix of size ( M × N ) .
Also,ℓ CE ( f θ ( X ) , Y ) now denotes the sum of cross-entropy losses over all data samples in the minibatch
B .
Further, for a matrix A , let ∥ A ∥ ∗ denote the Nuclear Norm, the sum of the singular values, of A .
Adversarial Threat Model : In this work, we primarily consider the robustness of Deep Networksagainst the union of ℓ ∞ , ℓ 1 , and ℓ 2 constrained adversaries.
Thus under the ℓ ∞ threat model with anε ∞ -constraint, for a given a clean image x , an adversarially perturbed counterpart ˜ x can differ by atmost ε at any given pixel location.
In contrast, adversaries under the ℓ 1 constraint may differ from theoriginal image such that the sum of pixel-wise absolute differences are capped by ε 1 .
Similarly, foradversaries under ℓ 2 constraint, the sum of squared pixel-wise differences are capped by ε 22 .
Further,a network f θ is said to be ε p -robust under a threat model ℓ p on a clean sample x with label y , iff θ (˜ x ) =
y , for all perturbations ˜ x such that ∥ x − ˜ x
∥ p ≤ ε p .
Related Works
In this section we briefly discuss the adversarial attacks and defences that builds up to efficientmulti-step adversarial training procedures, work that introduces adversarial training against the unionof multiple threat models, and their limitations that we propose to alleviate.
3.1 Early attempts towards Robust training
Early attempts of improving robustness to adversarial attacks could be broadly classified into inputpre-processing based defenses and adversarial training methods.
While the input pre-processingbased defenses were computationally cheap, they primarily involved masking of gradients to preventthe generation of strong attacks.
Such methods [Guo et al., 2018, Xie et al., 2018, Song et al., 2018]were later shown to be ineffective against attacks constructed using smooth approximations of thenon-differentiable components [Athalye et al., 2018, Carlini et al., 2019].
3.2 Effectiveness of FGSM and its limitations
Amongst the latter methods, the most successful early defense which stood the test of time was the Projected Gradient Descent or PGD adversarial training [Madry et al., 2018].
This involved minimizationof cross-entropy loss on the worst-case perturbations generated using multiple iterations of constrainedoptimization, leading to a significantly higher computational cost when compared to standard training.
Multi-step defenses achieve the state-of-the-art robustness today and typically use 10-steps of optimization for attack generation, leading to 11 times higher forward and backward propagations.
FGSMor the Fast Gradient Sign Method [Goodfellow et al., 2015] based adversarial training alleviates thecomputational cost by utilizing single-step adversarial samples for training.
However, this method iscontingent on the local linearity assumption of the loss surface, which is often compromised duringthe process of FGSM training, leading to generation of weaker adversaries as training progresses.
Indeed, Kurakin et al.
[2017] showed that such models exhibited the phenomenon of gradient masking,wherein stronger multi-step attacks were seen to reduce the robust accuracy drastically.
Wong et al.[2020] proposed to incorporate early-stopping using R-FGSM based adversarial training [Tramèr et al.
,2018], in order to identify the failure-point during robust training with single-step adversaries.
However, the method was later shown to not be effective on large capacity networks such as the WideResNet Zagoruyko and Komodakis [2016] architecture in subsequent work Sriramanan et al.
3.3 NuAT: Efficient Adversarial training for ℓ ∞ threat model
Sriramanan et al.
[2021] proposes a Nuclear Norm regularizer to improve the adversarial robustnessof Deep Networks through the use of single-step adversarial training under ℓ ∞ constraints.
This
Nuclear Norm Adversarial Training (NuAT) enforces function smoothing in the vicinity of cleansamples by incorporating joint batch-statistics of adversarial samples, which results in enhancedrobustness.
Further, this limits the oscillation of function values and prevents the over-smoothing ofloss surface uniformly in all directions, leading to a better robustness-accuracy trade-off.
Formally, in a given minibatch B , if X is the matrix composed of row-wise vectorized pixel values ofeach image, ∆ is a matrix of the same dimension as X consisting of independently sampled Bernoullinoise, and Y is the matrix containing the corresponding ground truth one-hot vectors, maximization ofthe
following loss function that utilizes the pre-softmax values f θ ( · ) generates single-step adversaries:
[Equation]
Subsequently, the following loss function is minimized during the NuAT training:
[Equation]
3.4 Union of Threat Models
While above described works trains a target to be robust against a single threat model of ℓ p -ball,there has been recent effort in the direction of making models robust against multiple threat modelssimultaneously.
Tramer and Boneh [2019] study the theoretical and empirical trade-offs of adversarialrobustness in various settings when defending against aggregations of multiple adversaries, proposingto train on the average (AVG) or maximizers of loss (MAX) amongst the different threat modelsconsidered for each minibatch of samples.
Madaan et al.
[2020] train using perturbations generatedusing a Meta-Noise Generator, and also propose a variant, Stochastic Adversarial Training whereinthey utilize multi-step adversaries (10 steps for ℓ ∞ and ℓ 2 , 20 steps for ℓ 1 ), though the authors note suboptimal performance from the same.
Croce and Hein [2019] propose a provable adversarial defense against all ℓ p norms for p ≥ 1 using a regularization term for ReLU networks, by enforcing robustnessagainst ℓ ∞ and ℓ 1 adversaries in particular.
Lastly and most importantly, Maini et al.
[2020] develops ageneralization of the standard PGD-based approach to incorporate multiple perturbation models into asingle attack by introducing a procedure called Multi Steepest Descent, or MSD, and further utilizingit to train standard architectures that are simultaneously robust against ℓ ∞ , ℓ 1 and ℓ 2 adversaries.
3.4.1 Multi Steepest Descent (MSD)
The core idea that MSD [Maini et al., 2020] adopts, which helps establish better worst-case accuraciesagainst the union of adversaries, is to create a single adversarial perturbation by simultaneously maximizing the worst-case loss over all perturbation models at each projected steepest descent step.
Unlikeprevious approaches Tramer and Boneh [2019] that generate worst-case adversaries for each threatmodel, or augment adversaries from multiple threat models, MSD chooses a projected steepest descentdirection in each iteration that maximizes the loss over all threat models.
This has been establishedto be superior to the standard adversarial training and the simpler approaches that use comparativelymyopic PGD subroutines that only use one perturbation model at a time.
However, a clear limitationof MSD here is that it requires 50 adversarial attack steps for each training iteration.
Additionally, foreach training step, it performs three forward passes (one for each threat model) and a backward pass.
3.4.2 Extreme Norms Adversarial Training (EAT)
In order to achieve robustness against a union of ℓ p threat models, Croce and Hein [2021b] proposeto fine-tune models that were originally trained to be robust against a single ℓ p norm threat model.
The authors demonstrate that fine-tuning of robust models to previously unseen ℓ p threat models iseffective, in contrast to adversarial fine-tuning of normally trained networks which yields non-robustmodels.
Furthermore, the authors propose to train solely on ℓ 1 and ℓ ∞ adversaries, such that otherℓ p balls on interest are contained within the union of these two threat models (Extreme Norms Adversarial Training or EAT).
However, this can place excessive restrictions during robust training if theperturbation budget of intermediate ℓ p adversaries is large.
As with MSD [Maini et al., 2020], EAT iscomputationally expensive in practice, since it relies upon multi-step adversarially pre-trained models,and further performs robust fine-tuning of such models using 10-step adversaries in the second phase.
Proposed Method
As noted by prior works [Madry et al., 2018, Croce and Hein, 2021a, Maini et al., 2020], robusttraining against the ℓ 1 threat model is significantly more complicated when compared to standardadversarial training techniques for ℓ ∞ or ℓ 2 threat models.
Croce and Hein [2021a] note that evenadversarial training using expensive 10-step adversaries generated from SLIDE
[Tramer and Boneh,2019] is prone to catastrophic overfitting [Wang et al., 2020]:
Over the course of training, models overfit to the adversaries generated, leading to a false notion of being robust, while achievingclose to 0% accuracy against stronger attacks during test evaluation.
While such phenomena arefrequently seen in single-step training Goodfellow et al. [2015], the occurrence of such failure modeseven with 10-step adversaries exhibits the difficulty in training ℓ 1 robust networks.
Recent work[Wong et al., 2020,
Sriramanan et al., 2020, 2021] has demonstrated that such failure modes canbe mitigated through appropriate algorithmic choices such as validation-based early stopping using
PGD adversaries, or relaxation terms for the overall loss to prevent collapse in training.
Croce and
Hein [2021a] demonstrate that using the 10-step APGD ℓ 1 attack, robust models can be trained byautomatically tuning the sparsity level induced in the ℓ 1 perturbations seen during training.
Buildingupon these in this work, we demonstrate the first successful instance of achieving non-trivial ℓ 1robustness using single-step adversaries during training.
Further, we extend the technique to achievesimultaneous robustness against the union of ℓ p threat models using only single-step training.
4.1 Analyzing Robust Training with ℓ 1 Adversaries
As mentioned before, Croce and Hein [2021a] point out the intricacies involved with ℓ 1 adversarialtraining, in that even multi-step training methods can begin to catastrophically overfit.
In this work, weseek to identify efficient yet effective single-step training routines that achieve robustness against ℓ 1 adversaries.
In order to build upto that, we first focus on understanding the phenomenon of catastrophicoverfitting under this setting and analyze what methods can help alleviate it in the single-step setting.
4.1.1 Nuclear Norm Attack and Curriculum Schedule
We begin by plotting the prediction accuracy and cross-entropy loss of different models over trainingand validation (Figure 1).
We find that when trained with R-FGSM based adversaries, models sufferfrom catastrophic overfitting early on during the training.
However, we make a crucial observation thatdynamically varying the perturbation budget during training, effectively setting up a curriculum , helpsimmensely in improving overall stability of training.
For instance, with the final ℓ 1 threat model of interest given by the ball of radius 12 , we propose to linearly increase this parameter from 0 to 12 to prevent catastrophic overfitting.
However, applying this curriculum to RFGSM-AT only leads to a delayin catastrophic failure, indicating the unsuitability of using R-FGSM adversaries for robust training.
Since the goal here is to utilize only single adversarial training step, it becomes imperative that theloss that we optimize over to generate the adversaries is smooth and does not showcase gradientmasking.
Hence, we build upon Nuclear-Norm Adversarial Training (NuAT)
[Sriramanan et al.
,2021] to generate single-step adversaries.
NuAT proposed to generate ℓ ∞ -adversaries by maximizingthe Nuclear Norm regularized objective.
We apply the same for generating adversaries like follows:
[Equation]
Sriramanan et al.
[2021] note that since the Nuclear norm forms a tight convex relaxation for the rankof the predicted matrix of logit values, the corresponding attack generates diverse adversaries in agiven minibatch, which then helps mitigate robust overfitting.
Crucially, we observe however that thissupplemental attack diversity is not sufficient for single-step training on the ℓ 1 threat model, as even
NuAT is observed to be susceptible to catastrophic failure in Fig-1.
However, by utilizing the dynamiccurriculum schedule, this phenomenon is successfully remedied in the proposed method, NCAT.
4.1.2 Steepest Ascent with Single-Step Optimization
Given a minibatch of samples X , we generate the Nuclear Norm attack by identifying a perturbation∆ that maximizes the loss (cid:101) L as in Eq-3 and conforms to the following two constraints:
[Equation]
wherein the second constraint arises from the normalized range for pixel intensities of an image.
Assuming a first-order Taylor series approximation for the loss incurred by the network f θ , if ∇ ∆ (cid:101)
Lrepresents the gradient direction, the steepest ascent direction ∆ ∗ to maximize loss would be parallelto the same in the unconstrained setting.
However, to incorporate the two constraints entailed above,we now derive the optimal perturbation (Eq-4).
For an image x in the minibatch X , let g representthe corresponding gradient.
Further, let M i denote the deviation budget required at pixel i to saturatethe same to the pixel constraint [0 , 1] parallel to the gradient g i .
Formally, M can be defined as:
[Equation]
Finally, to incorporate the ℓ 1 budget being constrained by ε 1 , we observe the optimal perturbationprioritizes pixel locations corresponding to large gradient magnitudes.
Let σ denote the sortedpermutation of indices such that a σ (1) ≥ a σ (1) ≥ · · · ≥ a σ ( d ) , where a i = | g i | represents thegradient magnitudes at different pixel locations.
Now, if the cumulative budget utilized is defined as
S i
=
Σ ij =
| M σ ( j ) | and I
i = max { 0 , ε 1 − S
i − 1 } denoting support variables, the optimal perturbationcorresponding to image x can be defined as:
[Equation]
Thus, in an ℓ 1 constrained attack the gradients have to be sorted by their magnitude, which requires
O ( d · log d ) complexity where d represents the image dimensionality.
However, in practice thisoverhead is observed to be exceedingly minimal relative to data loading times etc.
Indeed, using thisroutine with Nuclear norm based training with the curriculum schedule (NCAT- ℓ 1 ), we demonstratefor the first time that single-step training can be effectively used to produce ℓ 1 robust models.
4.2 Single-Step Training for ℓ ∞ and ℓ 2 Robustness
Since the ℓ ∞ threat model is the most well-studied setting in existing literature, we rely upon priorworks to obtain excellent baselines.
To achieve robustness against ℓ ∞ constrained adversaries usingsingle-step training, we utilize the current state-of-the-art method, Nuclear Norm Adversarial Training(NuAT)
[Sriramanan et al., 2021].
We further seek to incorporate other threat models during training,in order to obtain models with non-trivial robustness against the union of the ℓ 1 , ℓ 2 and ℓ ∞ threatmodels simultaneously.
In order to efficiently train against ℓ 2 adversaries, we first propose to modifythe NuAT training algorithm to utilize this constraint set, using ℓ 2 norm based projections.
However,similar to Croce and Hein [2021b], we make the remarkable observation that models that are trainedsolely on ℓ ∞ adversaries achieve a great degree of robustness versus ℓ 2 adversaries on the test set.
We observe similar transfer of robustness from ℓ 1 trained models toward the ℓ 2 threat model as well.
Thus, the primary difficulty in achieving robustness to the union of threat models appears to be thatof training networks robust to the ℓ 1 and ℓ ∞ threat models in particular.
4.3 Sampling Procedures to Improve Efficiency
To achieve robustness against the union of the three ℓ p threat models considered, it is plausible thattraining with three distinct single-step attacks (constrained to ℓ 1 , ℓ 2 and ℓ ∞ ) using the proposedapproach in each minibatch will be effective.
However, in this work, we primarily focus on reducingthe training complexity further, in order to effectively utilise only a single-step attack for eachminibatch.
Awasthi et al.
[2021] proposed to utilize the multiplicative weights algorithm, wherein theloss under different adversaries on a hold-out validation set guides the sampling procedure, using aset of exponential running weights w i for each threat model.
However, we find that this is contingenton the efficacy of adversaries utilized on the validation set, which can be restrictive in practice dueto computational constraints.
Building upon this, we find in practice that alternating between ℓ ∞and ℓ 1 attacks across different minibatches with a fixed frequency is remarkably effective.
Thus,the proposed defense, NCAT, uses nuclear norm based single-step training following a curriculumschedule, such that different threat models are selected for attack generation in different minibatchesbased on a pre-fixed frequency.
We present a concise, summarised overview of the proposed trainingapproaches in Algorithm-1.
Here, the Decision Function D (L7, Alg-1) alternately outputs p = 1or p =
∞
based on a predetermined frequency, since such models are observed to simultaneouslyachieve ℓ 2 robustness without explicit training.
As observed in prior works [Chen et al.
, Sriramananet al., 2021], maintaining a exponential running average of network weights (SWA [Izmailov et al.,2018]) helps improve robust performance overall as well, particularly so in this setup since different(random) minibatches are trained with adversarial perturbations arising from different threat models.
Furthermore, this effectively reducing undesired bias to a particular threat model due to autocorrelations that arise in training.
A modified version of the proposed approach, namely NCAT-AVG,
1: Input: Network f θ with parameters θ , Weight Averaged Network f ω with parameters ω , Training
Data D with input images of dimension d , Minibatch Size M, Attack Size ε p for each ℓ p threat model, Epochs E , Learning Rate η , Decision Function D , Curriculum Schedule C 2: for epoch
= 1 to E do 3: ε p =
C ( p ) 4: for minibatch { ( x i , y i ) } Mi =1 ⊂ D do 5:
X =   . . .
x. . .
x M . . .

 , ∆ =   . . .
δ. . .
δ M . . .

 , δ i ∼ Bern d ( − α, α ) , Y =   y 1 .
y M 
 6: (cid:101)
L = ℓ CE ( f θ ( X + ∆) , Y ) + λ · || f θ
( X + ∆) − f θ ( X ) ||
∗ 7: for p in D( θ ) do 8: ∆ = ∆ + ε p · Proj (cid:16) ∇ ∆
(cid:101) L, B p ( ε p ) (cid:17) 9: (cid:101) X
=
Clamp ( X + ∆ , 0 , 1) 10: end for 11:
L = ℓ CE ( f θ ( X ) , Y ) + λ · || f
θ ( (cid:101) X ) − f θ ( X ) ||
∗ 12: θ = θ − 1 M · η · ∇ θ L 13: ω = τ · ω + (1 − τ ) · θ 14: end for 15: end for uses a Decision Function D that outputs the collection of p = { 1 , 2 , ∞} , and effectively uses abudget of three single-step attacks, one for each threat model.
We present results obtained using
NCAT-AVG and other ablations in the Supplementary Material.
Experiments and Analysis
In this work, we primarily consider the CIFAR-10 [Krizhevsky et al., 2009] and {+ImageNet-
 Russakovsky+} et al. [2014] datasets, since they have come to form the benchmark for comparativeanalysis of adversarially robust models.
Following prior works [Maini et al., 2020], we considerconstraint sets given by the ℓ ∞ ball of radius 8 / 255 , ℓ 2 ball of radius 0 .
5 and ℓ 1 ball of radius 12 asthe threat models of interest, and as explained previously, we attempt to train models that achieve nontrivial worst-case accuracy against the union of such ℓ p threat models.
For the ImageNet-100 dataset,the corresponding radii for ℓ 1 , ℓ 2 and ℓ ∞ threat models are 255, 1200/255 and 4/255 respectively,following the constraints considered by Laidlaw et al.
We present results in the white-box setting, wherein the adversary is cognizant of the model weights,architecture and training scheme employed.
To accurately estimate worst-case performance, wefocus our evaluation pipeline to incorporate state-of-the-art attacks such as AutoAttack [Croce and
Hein, 2020] for each ℓ p threat model.
Furthermore, AutoAttack includes strong ℓ 1 attack evaluationbaselines using techniques proposed by Croce and Hein [2021a], wherein the authors note thatsignificant improvement in attack efficacy as compared to prior works such as SLIDE Attack [Tramerand Boneh, 2019], B&B Attack Brendel et al.
[2017] and Pointwise Attack [Schott et al., 2018].
Dueto paucity of space, we include black-box evaluations, generalization to unseen domains, gradientmasking checks and adaptive attacks in the Supplementary Material.
We however make the importantnote that the suite of white-box attacks considered using AutoAttack are strictly stronger than theformer set of attack evaluations, indicating the absence of gradient masking.
We first present results obtained using the ResNet-
[He et al., 2016] architecture on CIFAR-10 in
Table-1.
In the first partition of the table, we present models trained solely on the ℓ 1 threat model.
Thecurrent state-of-the-art is achieved by APGD- ℓ 1 [Croce and Hein, 2021a], which performs a 10-step
APGD attack during training in order to mitigate gradient masking and catastrophic overfitting forℓ 1 constrained adversaries.
On the other hand, our method, NCAT- ℓ 1 which uses just a single-stepattack for adversarial training achieves ℓ 1 robustness much more efficiently.
We note that whilethe multi-step approach has higher ℓ 1 robustness (+3.6%), the single-step NCAT trained model hassignificantly better worst-case accuracy over all three threat models (+15.8). Indeed, we note once
[Table]
the union of these threat models.
[Table]
again that NCAT represents the first-ever successful single-step adversarial training on the ℓ 1 threatmodel, which also generalizes well to the unseen ℓ ∞ and ℓ 2 threat models simultaneously.
In the second partition of Table-1, we present models that are explicitly trained to be robust underthe union of the ℓ ∞ , ℓ 2 and ℓ 1 threat models.
Namely, we present comparative analysis with respectto existing multi-step adversarially trained defenses such as AVG and MAX [Tramer and Boneh,2019], MSD [Maini et al., 2020], EAT [Croce and Hein, 2021b] and SAT [Madaan et al., 2020].
Forthese methods, we primarily utilize robust evaluations as presented by Croce and Hein [2021b] toenable fair comparisons, which comprise of re-implemented models that obtain higher accuracies ascompared to values reported in the original papers.
We first note that SAT † requires 13.33 adversarialattack steps during training, since it utilizes 10-step attacks for ℓ ∞ and ℓ 2 adversaries, and 20 attacksteps for ℓ 1 adversaries to mitigate gradient masking, indicating the considerable difficulty involvedin achieving ℓ 1 robustness.
In contrast, EAT †† relies upon 10-step fine-tuning of a network thatis already robust against a single threat-model.
The current state-of-the-art approaches {+compriseof MSD and MAX that achieve 44% worst-case accuracy, while utilizing a budget of 50 and 
+}
[Table]
attack steps respectively during training.
We observe that the proposed approach, NCAT achievescomparable worst-case and average-case performance over the threat models considered, whilerequiring a significantly smaller computational footprint during training.
Furthermore, the proposedmethod facilitates the trade-off between clean accuracy and worst-case performance as indicated by
NCAT + which achieves near-SOTA robust performance.
In Table-2, we present results on models trained with the WideResNet-28-10 [Zagoruyko and
Komodakis, 2016] architecture to demonstrate the scalability of the proposed defense to high-capacitynetworks.
We thus establish the efficacy of the curriculum schedule combined with nuclear normbased training in mitigating catastrophic overfitting, enabling efficient training of these large networks.
In Table-3, we present evaluations on the ImageNet-100 dataset, wherein we utilize ResNet-18 networks to reduce computational demands.
We observe that NCAT- ℓ 1 attains remarkable robust accuracyon unseen ℓ ∞ and ℓ 2 adversaries, even achieving 26.6% accuracy against the Perceptual Projected Gradient Descent (PPGD) attack [Laidlaw et al., 2021], which forms one of the strongest attacks knownto date.
Further, we observe that the NCAT trained achieves state-of-the-art worst-case ℓ p accuracy,while attaining robustness similiar to that of Perceptual Adversarial Training [Laidlaw et al., 2021]for the PPGD attack, with the latter being a model that was explicitly trained on such adversaries.
Conclusions
In this work, we develop an efficient adversarial training procedure, NCAT, to train networks thatare robust against a union of ℓ p threat models, namely ℓ ∞ , ℓ 1 and ℓ 2 .
To do so, we first focuson developing an efficient, yet effective robust training procedure for the ℓ 1 threat model, byincorporating a curriculum schedule to mitigate catastrophic overfitting.
Indeed, in this work wepresent the first ℓ 1 constrained robust model trained solely using single-step adversaries, achievingrobustness similar to that of multi-step SOTA approaches.
Furthermore, we extend the proposedmethod to achieve worst-case robustness under multiple ℓ p norm constraints simultaneously.
Compared to the current SOTA that uses 30 adversarial attack steps for its training procedure toachieve 44% robust accuracy on CIFAR- 10 , our method yields 43.7% robustness while solelyutilizing single-step adversaries during the training routine.
This thereby greatly reduces thecomputational requirements needed to achieve SOTA -equivalent robust performance.
Limitations and Societal Impact
The proposed approach, NCAT, facilitates efficient yet effective training of robust models, that achievenon-trivial worst-case accuracy under the union of several ℓ p norm threat models.
Furthermore, NCATtrained models are seen to robust against unseen attacks and adversaries that might potentially be encountered in the real world.
Thus, given that NCAT helps train reliable and trustworthy models usinga small computational footprint, this work indeed has potential in creating a positive impact on society.
Further, we hope that future works could extend NCAT to include provable certificates for differentℓ p threat models and their union, and possibly provide stricter guarantees for the absence of gradientmasking and catastrophic failure with the use of theoretically-motivated curriculum schedules.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect
the
paper’s contribu
tions
and scope? [Yes (b) Did you describe the limitations of your work?
[Yes] See Section-6,
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See Section-6,
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] Further details included in the
Supplementary Material
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] Details included in the Supplementary
Material
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] Details included in the Supplementary Material
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] Details included in the Supplementary Material
(d) Did you include the total amount of compute and the type of resources used (e
[Table]
g., type of GPUs, [Table] internal cluster, or cloud provider)?
[Yes] Details included in the Supplementary Material
If you are using existing assets (e
[Table]
g
[Table]
, code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] Further details included in the
Supplementary Material
(b) Did you mention the license of the assets?
[Yes]
Details included in the Supplementary Material
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
[Table]
[Table]
[Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]