Robustifying $\ell_\infty$  Adversarial Training to the Union of Perturbation Models

Classical adversarial training ( AT ) frameworks are designed to achieve high ad-1 versarial accuracy against a single attack type , typically ` ∞ norm-bounded per-2 turbations . Recent extensions in AT have focused on defending against the union3 of multiple perturbation models but this benefit is obtained at the expense of a4 significant ( up to 10× ) increase in training complexity over single-attack ` ∞ AT.5 In this work , we expand the capabilities of widely popular single-attack ` ∞ AT6 frameworks to provide robustness to the union of ( ` ∞ , ` 2 , ` 1 ) perturbations while7 preserving their training efficiency . Our technique , referred to as Shaped Noise8 Augmented Processing ( SNAP ) , exploits a well-established byproduct of single-9 attack AT frameworks – the reduction in the curvature of the decision boundary of10 networks . SNAP prepends a given deep net with a shaped noise augmentation layer11 whose distribution is learned along with network parameters using any standard12 single-attack AT . As a result , SNAP enhances adversarial accuracy of ResNet-1813 on CIFAR-10 against the union of ( ` ∞ , ` 2 , ` 1 ) perturbations by 14 % -to-20 % for14 four state-of-the-art ( SOTA ) single-attack ` ∞ AT frameworks , and , for the first15 time , establishes a benchmark for ResNet-50 and ResNet-101 on ImageNet.16 1 Introduction17 Today adversarial training ( AT ) provides state-of-the-art ( SOTA ) empirical defense against adver-18 sarial perturbations . For this , adversarial perturbations are used during training to optimize a robust19 loss function [ 20 , 41 , 30 , 35 ] . Early AT frameworks [ 20 , 41 ] were 7×-to-10× more computationally20 demanding than vanilla training . More recent works [ 30 , 35 , 40 ] have significantly reduced the21 computational demands of AT via single-step attacks and superconvergence.22 However , today ’ s AT frameworks predominantly focus on a single-attack , i.e. , they seek robustness23 to a single perturbation , typically ` ∞-bounded [ 30 , 35 , 37 , 41 , 43 , 40 , 39 , 26 , 9 , 34 , 42 , 10 , 11 , 14 ] .24 This results in low performance against other perturbations such as ` 2 , ` 1 , or the union of ( ` ∞ , ` 2 , ` 1 ) .25 Indeed , as shown in Fig . 1 , four state-of-the-art ( SOTA ) single-attack AT frameworks ( black markers ) 26 employing only ` ∞-bounded perturbations achieve low adversarial accuracy A ( U ) adv of ≈ 15 % -to-20 % 27 against the union of ( ` ∞ , ` 2 , ` 1 ) perturbations . Recent extensions in AT [ 21 , 32 , 18 ] do seek higher28 A ( U ) adv but only at the expense of 6×-to-10× increase in the total training time ( blue markers in29 Fig . 1 ) . The large training time of these AT frameworks has inhibited their application to large-scale30 datasets such as ImageNet , e.g. , Maini et al . [ 21 ] , Tramèr & Boneh [ 32 ] show results for MNIST and31 CIFAR-10 only , while Laidlaw et al . [ 18 ] only additionally show 64× 64 ImageNet-100 results.32 The high training time for AT frameworks arises from two sources : ( i ) the need to employ larger33 networks , e.g. , MSD [ 21 ] with ResNet-18 achieves higher A ( U ) adv than PAT [ 18 ] with ResNet-50 ( see34 Fig . 1 ) ; and ( ii ) the need to incorporate multiple perturbations during each attack step and a higher35 Submitted to 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Do not distribute . overall number of attack steps , e.g. , 50 in MSD [ 21 ] , 20 in AVG [ 32 ] . Obviously one can always36 reduce the number of attack steps in MSD/AVG to proportionally reduce training time . Doing so37 results in training time andA ( U ) adv to rapidly approach the training complexity andA ( U ) adv of standard AT38 frameworks , e.g. , a 5-step MSD and 2-step AVG is equivalent in training time and accuracy to PGD39 and TRADES , respectively . Notwithstanding the expensive nature of 50-step multi-attack training,40 today MSD [ 21 ] achieves a SOTA A ( U ) adv of 47 % with ResNet-18 on CIFAR-10.41 TRADES [ Zhang et al. ’ 19 ] TRADES+SNAP [ Maini et al. ’ 20 ] PGD [ Madry et al. ’ 18 ] FreeAdv [ shahafi et al. ’ 19 ] FreeAdv+SNAP PGD+SNAP De sir ed ! ! '' # ( % ) ( % ) Total training time ( minutes ) MSD-50 MSD-20 MSD-30 MSD-10 MSD-5 FastAdv [ Wong et al. ’ 20 ] FastAdv+SNAP PAT [ Laidlaw et al. ’ 21 ] AVG-50AVG-20 AVG-5 AVG-10 [ Tramer et al. ’ 19 ] AVG-2 Figure 1 : Adversarial accuracy ( A ( U ) adv ) against union of ( ` ∞ , ` 2 , ` 1 ) vs. measured wall-clock total training time on CIFAR10 with different AT frameworks on single NVIDIA TESLA P100 GPU . = ( 0.031 , 0.5 , 12 ) for ( ` ∞ , ` 2 , ` 1 ) perturbations , respectively . SNAP enhances robustness with a small increase in training time . All frameworks except PAT employ ResNet-18 . This poses a question : can we approach the high42 robustness of multiple-attack AT such as 50-step43 MSD against the union of ( ` ∞ , ` 2 , ` 1 ) perturbations44 while maintaining the low training time of fast single-45 attack AT frameworks such as FreeAdv [ 30 ] and Fas-46 tAdv [ 35 ] ? 47 In our quest to answer this question we find that noise48 augmentation using adequately shaped noise within49 standard single-attack AT frameworks employing ` ∞-50 bounded perturbations significantly improves robust-51 ness against the union of ( ` ∞ , ` 2 , ` 1 ) perturbations.52 The improvement appears to be a consequence of a53 well-established byproduct of AT frameworks – the54 reduction in the curvature of the decision boundary of55 networks trained using single-attack AT [ 6 , 23 ] . We56 confirm this connection by quantifying the impact57 of single-attack AT on the geometric orientations of58 different perturbations.59 Based on this insight , we propose Shaped Noise60 Augmented Processing ( SNAP ) – a method to en-61 hance robustness against the union of perturbation types by augmenting single-attack AT frameworks.62 SNAP prepends a deep net with a shaped noise ( SN ) augmentation layer ( see Fig . 4 ) whose dis-63 tribution parameter Σ is learned with that of the network ( θ ) within any standard single-attack AT64 framework . SNAP improves the robustness of four SOTA ` ∞-AT frameworks against the union of65 ( ` ∞ , ` 2 , ` 1 ) perturbations by 15 % -to-20 % on CIFAR-10 ( red markers in Fig . 1 ) with only a modest66 ( ∼ 10 % ) increase in training time . This expands the capabilities of widely popular single-attack ` ∞67 AT frameworks to providing robustness to the union of ( ` ∞ , ` 2 , ` 1 ) perturbations without sacrificing68 training efficiency . We validate SNAP ’ s benefits via thorough comparisons with nine SOTA adver-69 sarial training and randomized smoothing frameworks across different operating regimes on both70 CIFAR-10 and ImageNet.71 One tangible outcome of our work – we demonstrate for the first time ResNet-50 ( ResNet-101 ) 72 networks on ImageNet that achieveA ( U ) adv = 32 % ( 35 % ) against the union of ( ` ∞ ( = 2/255 ) , ` 2 ( =73 2.0 ) , ` 1 ( = 72.0 ) ) perturbations . Our code and trained models will be shared publicly on GitHub.74 2 Related Work75 We categorize works on adversarial vulnerability of DNNs as follows:76 Low-complexity adversarial training : The high computational needs of AT frameworks has spurred77 significant efforts in reducing their complexity [ 40 , 30 , 35 , 43 ] . FreeAdv [ 30 ] updates weights while78 accumulating multiple attack iterations . FastAdv [ 35 ] employs appropriate use of single-step attacks,79 while Zheng et al . [ 43 ] leverage inter-epoch similarity between adversarial perturbations . However,80 these fast AT methods seek robustness against a single perturbation type , e.g. , ` ∞ norm-bounded81 perturbations . In contrast , SNAP expands the capabilities of these AT frameworks by enhancing82 robustness to the union of three perturbation types ( ` ∞ , ` 2 , ` 1 ) , while preserving their efficiency.83 Robustness against union of perturbation models : The focus on the robustness against the union of84 multiple perturbation types is relatively new . Kang et al . [ 16 ] studied transferability between different85 perturbation types , while Jordan et al . [ 15 ] considered combination attacks with low perceptual86 distortion . Stutz et al . [ 31 ] proposed a modification in AT to detect images with different models87 of perturbations via confidence thresholding , but they don ’ t attempt to classify perturbed images88 correctly . For accurate classification in the presence of different perturbation models , Tramèr & 89 Boneh [ 32 ] studied empirical and theoretical trade-offs involved in including multiple perturbation90 types simultaneously during training . Maini et al . [ 21 ] further built upon this work to propose the91 multi steepest descent ( MSD ) AT framework which chooses one among the three perturbation models92 ( ` ∞ , ` 2 , ` 1 ) in each attack iteration during training , achieving SOTA adversarial accuracy on CIFAR-93 10 against the union of the ( ` ∞ , ` 2 , ` 1 ) perturbation models , albeit at a high ( 10× ) training time . In94 contrast , SNAP provides high robustness against the union of ( ` ∞ , ` 2 , ` 1 ) perturbation models using95 established single-attack ` ∞ AT frameworks . This enables to showcase the benefits of our approach96 on large-scale datasets such as ImageNet.97 Recently , Laidlaw et al . [ 18 ] developed a novel AT framework ( PAT ) with low perceptual distortion98 attacks to demonstrate impressive generalization to unseen attacks . In contrast , we focus on extending99 the capabilities of widely popular ` ∞-AT frameworks to providing robustness against the union of100 ( ` ∞ , ` 2 , ` 1 ) perturbations , while preserving their training efficiency.101 Noise augmentation : Multiple recent works have investigated the role of randomization in enhancing102 adversarial robustness [ 12 , 24 , 8 , 25 ] with theoretical guarantees . Another prominent line of work103 in this category is randomized smoothing [ 5 , 29 , 19 , 38 ] , where random noise is used as a tool to104 compute certification bounds . Rusak et al . [ 28 ] also explored the role of noise augmentation for105 improving the robustness against common-corruptions [ 13 ] . In contrast , in SNAP , noise augmentation106 is used as a means to enable widely popular ` ∞-AT frameworks to efficiently achieve high robustness107 against the union of multiple norm-bounded perturbations . As is the characteristic of AT works , our108 results are primarily empirical in nature . Hence , we follow recent guidelines [ 33 , 21 ] to evaluate the109 accuracy against the strongest possible adversaries . We do explicitly compare ` ∞-AT+SNAP with110 randomized smoothing approaches in the Appendix.111 3 Subspace Analysis of Adversarial Perturbations112 In this section , we employ subspace methods to compre-113 hend the distinction between ` ∞ , ` 2 and ` 1 perturbations.114 For each input xi ∈ RD in dataset X , consider adversar-115 ial perturbations αi , βi , and γi bounded within ` ∞ , ` 2,116 and ` 1 norms , respectively.117 We begin with a hypothesis ( see Fig . 2 ) : The perturbations118 α , β , and γ corresponding to input x have directions that119 differ significantly if the curvature of the decision bound-120 ary is high in the neighborhood of x. Conversely , if the121 curvature of the decision boundary is low , the perturba-122 tions α , β , and γ tend to point in similar directions.123 Since , prior works [ 6 , 23 ] have found that single-attack124 AT reduces the curvature of the decision boundary , we test125 our hypothesis by studying the following two networks126 on CIFAR-10 data : a non-robust ResNet18 f vanθ trained using vanilla training , and a robust ResNet18127 f robθ trained using the TRADES [ 41 ] AT framework employing ` ∞ perturbations.128 We compute perturbationsαi , βi , and γi for each xi ∈ X for both networks , i.e. , κ ∈ { van , rob } . We129 compute the singular vector basis Pκ for the set of ` 2 bounded perturbations ∆κ = { βκ1 , . . . , βκ|X| } .130 The normalized mean squared projections of the three types of perturbation vectors on the singular131 vector basis Pκ of vanilla trained ResNet-18 ( Pvan ) ( Fig . 3 ( a ) ) and TRADES trained ResNet-18132 ( P rob ) ( Fig . 3 ( b ) ) shows a clear contrast.133 The perturbations of a vanilla trained network roll-off gradually to occupy a larger subspace as134 indicated in Fig . 3 ( a ) . Specifically , the projections of α and γ occupy almost all 3000 directions in135 the basis Pvan since their mean squared projections are within ∼ 10 % of the maximum value mmax.136 This shows that the dominant singular vectors of β are not well-aligned with α and γ in a vanilla137 trained network . With TRADES AT ( Fig . 3 ( b ) ) , however , all three types of perturbations are squeezed138 into a much smaller subspace spanning only the top 250 singular vectors in the perturbation basis139 P rob . Outside these 250 dimensions , the mean squared projections fall to < 10 % of their maximum140 value.141 In summary , the results in Fig . 3 validate the hypothesis that single-attack AT increases the average142 alignment of different perturbation types due to the reduction in the decision boundary curvature . In143 Sec . 4 , we exploit this behavior of single-attack ` ∞ AT to improve its robustness against the union of144 multiple perturbation models via SNAP.145 4 Shaped Noise Augmented Processing ( SNAP ) 146 We show that single-attack AT can be enhanced to address147 multiple perturbations by introducing noise to appropriately148 wiggle the ` ∞-bounded perturbations ( Fig . 4 ( a ) ) . However,149 to do so , the noise distribution needs to be chosen and shaped150 appropriately to minimize its impact on natural accuracy and151 robustness to ` ∞-bounded perturbations.152 We experiment with both ` ∞ and ` 2 perturbations in single-153 attack AT frameworks and find ` ∞-AT to be suitable for154 our proposed shaped noise augmentation ( see Sec . 5.2.1 for155 details ) . Hence , in this section , we describe SNAP for single-156 attack AT frameworks employing ` ∞ perturbations.157 4.1 SNAPnet158 A deep net fθ ( x ) : RD → { 0 , 1 } C parametrized by θ maps159 the input x ∈ RD to a one-hot vector y ∈ { 0 , 1 } C over C160 classes.161 We construct a SNAP-based deep net ( SNAPnet ) fSNθ , Σ ( x ) by162 introducing an additive shaped noise ( SN ) layer ( Fig . 4 ( b ) ) ,163 where the noise distribution parameter Σ is learned during164 training . Formally,165 y = fSNθ , Σ ( x ) = fθ ( x+ n ) = fθ ( x+ V Σn0 ) , ( 1 ) where n0 ∼ L ( 0 , ID×D ) is a zero-mean isotropic Laplace166 noise vector , Σ = Diag [ σ1 , . . . , σD ] is a distribution param-167 eter denoting its per-dimension standard deviation , ID×D168 denotes the D ×D identity matrix , and V = [ v1 , . . . , vD ] denotes a basis in RD . We also studied169 Gaussian and Uniform distributed n0 , but empirically find the Laplace distribution to yield better170 results ( Sec . 5.2.1 ) . We use V = ID×D for all our experiments in the main text and study other171 options for V in the Appendix.172 The final classification decision d is computed via173 d = arg max c [ En [ y ] ] c , ( 2 ) where [ a ] c denotes the c-th element of vector a . Note , the shaped noise perturbs the input x with a174 noise source n = V Σn0 ( Eq . ( 1 ) ) . The distribution parameter Σ is learned in the presence of any175 standard AT method [ 20 , 41 , 30 ] used for learning deep net parameters θ as described next.176 Algorithm 1 Training SNAPnet Input : training set X ; basis V = [ v1 , . . . , vD ] ; total noise power Pnoise ; minibatch size r ; baseline training method BASE ; noise variance update frequency Uf ; Total number of epochs T Initialize : noise variances Σ0 = Diag [ σ1,0 , . . . , σD,0 ] . Output : robust network fSNθ , Σ , noise variances ΣT = Diag [ σ21 , T , . . . , σ2D , T ] . 1 : for epoch t = 1 . . . T do 2 : for mini-batch B = { x1 , . . . , xr } do θ ← BASE ` ∞ ( fSNθ , Σt ( { xi } ri=1 ) , θ ) . BASE ( ) Training 3 : end for 4 : if t mod Uf = 0 then . SNAP Distribution Update once every Uf epochs 5 : for mini-batch B = { x1 , . . . , xr } do 6 : { xadvi } ri=1 ← PGD ( K ) ` 2 ( fSNθ , Σt ( { xi } ri=1 ) ) ; ηi = xadvi − xi ∀ i ∈ { 1 , . . . , r } 7 : γj ← γj + ∑r i=1 ( 〈vj , ηi〉 ) 2 ∀j ∈ { 1 , . . . , D } . Accumulate projections ; See Eq . ( 3 ) 8 : end for 9 : σ2j , t+1 = Pnoise √ γj∑D k=1 √ γk ∀j ∈ { 1 , . . . , D } . Normalize accumulated projections ; See Eq . ( 3 ) 10 : else 11 : Σt+1 ← Σt 12 : end if 13 : end for 4.2 Training SNAPnet177 Algorithm 1 summarizes the procedure for training SNAPnet fSNθ , Σ ( x ) . In each epoch , an arbitrary178 AT method BASE ( ) ( line 2 ) updates network parameters θ with input perturbed by noise n. Here179 BASE ( ) can be any established AT framework [ 20 , 41 , 30 , 35 ] employing ` ∞ perturbation.180 The SNAP parameter Σ is updated once every Uf = 10 epochs via a SNAP distribution update ( lines181 4-10 ) . In this update , the per-dimension noise variance σ2j is updated proportional to the root mean182 squared projection of the adversarial perturbations η on the basis V given a total noise constraint183 ∑D j=1 σ 2 j = Pnoise , where Pnoise denotes the total noise power . Formally,184 σ2j ∝ √ Ex∈X ( 〈η , vj〉2 ) s.t . D∑ j=1 σ2j = Pnoise , ( 3 ) where η is the ` 2 norm-bounded PGD adversarial perturbation for the given input x ∈ X ( line 6 ) .185 Note that these ` 2 perturbations are employed only for noise shaping and are distinct from the ` ∞186 perturbations employed by BASE ( ) AT ( line 2 ) . Also , ` ∞ perturbations can not be used here since187 their projections are constant ∀j when V = ID×D , whereas employing ` 1 perturbations leads to poor188 shaping due to high sparsity.189 Thus , in SNAP , the average squared ` 2 norm of the noise vector n is held constant at Pnoise while190 adapting the noise variances in the individual dimensions so as to align the noise vectors with the191 adversarial perturbations on average . Intuitively , the decision boundary is pushed aggressively in192 those directions.193 4.3 Remarks194 Note that the SNAP distribution update is distinct from BASE ( ) AT . Hence , SNAP doesn ’ t require any195 hyperparameter tuning in BASE ( ) . For fairness to baselines we keep all hyperparameters identical196 when introducing SNAP in all our experiments . However , SNAP introduces a new hyperparameter197 Pnoise , which permits to trade adversarial robustness A ( U ) adv for natural accuracy Anat . This trade-off is198 explored in Sec . 5.2.2.199 The computational overhead of SNAP is small ( ∼ 10 % ) since the SNAP Distribution Update occurs200 once in 10 epochs using just 20 % of the training data to update the noise standard deviations σj . We201 provide more details about the SNAP Distribution Update in the Appendix.202 5 Experimental Results203 5.1 Setup204 Following experimental settings of prior work [ 41 , 30 , 21 ] , we employ a ResNet-18 network for205 CIFAR-10 experiments and both ResNet-50 and ResNet-101 networks for ImageNet experiments.206 Accuracy on clean test data is referred to withAnat and accuracy on adversarially perturbed test data is207 referred to via A ( ` ∞ ) adv , A ( ` 2 ) adv , and A ( ` 1 ) adv , for ` ∞ , ` 2 , and ` 1 norm bounded perturbations , respectively.208 Accuracy against the union of all three perturbations is denoted by A ( U ) adv .209 For a fair robustness comparison , our evaluation setup closely follows the setup of Maini et al . [ 21 ] 210 for CIFAR-10 data : ( 1 ) choose norm bounds = ( 0.031 , 0.5 , 12.0 ) for ( ` ∞ , ` 2 , ` 1 ) perturbations,211 respectively ; ( 2 ) scale norm bounds for images to lie between [ 0 , 1 ] ; ( 3 ) choose the PGD attack212 configuration to be 100 iterations with 10 random restarts for all perturbation types1 ; and ( 4 ) estimate213 A ( U ) adv as the fraction of test data that is simultaneously resistant to all three perturbation models.214 Following the guidelines of Tramer et al . [ 33 ] , we carefully design adaptive PGD attacks that215 target the full defense – SN layer – since SNAPnet is end-to-end differentiable . Specifically , we216 backpropagate to primary input x through the SN layer ( see Fig . 4 ) . Thus , the final shaped noise217 distribution is exposed to the adversary . We also account for the expectation En [ · ] in Eq . ( 2 ) by218 explicitly averaging deep net logits over N0 ( = 8 ) noise samples before computing the gradient,219 which eliminates any gradient obfuscation , and is known to be the strongest attack against noise220 augmented models [ 29 ] . In the Appendix we also show robustness stress tests and evaluate more221 attacks.222 On CIFAR-10 data , we compare with the following seven key SOTA AT frameworks : PGD [ 20 ] ,223 TRADES [ 41 ] , FreeAdv [ 30 ] , FastAdv [ 35 ] , AVG [ 32 ] , MSD [ 21 ] , PAT [ 18 ] . We also compare224 with two randomized smoothing frameworks [ 5 , 29 ] in the Appendix . Thanks to their GitHub code225 releases , we first successfully reproduce their results with a ResNet-18 network in our environment . In226 the case of PAT [ 18 ] , we evaluate and compare with their pretrained ResNet-50 model on CIFAR-10.227 We compare all training times on a single NVIDIA P100 GPU . On ImageNet data , we primarily228 compare to FreeAdv [ 30 ] . We train ResNet-50 and its SNAPnet version with FreeAdv on a Google229 Cloud server with four NVIDIA P100 GPUs to compare their accuracy and training times . We will230 release our pretrained models and code on GitHub.231 5.2 Ablation Studies232 5.2.1 Impact of Noise Distribution and Model of BASE ( ) AT Perturbations233 In this subsection , we first study the impact of employing ` ∞ vs. ` 2 perturbations in BASE AT ( ) ( see234 line 2 in Alg . 1 ) on A ( U ) adv . For each choice , we further experiment with three distributions for the235 SN layer in Fig . 4 ( b ) viz . Gaussian , Uniform , and Laplace . We don ’ t consider ` 1 perturbations in236 1Following Maini et al . [ 21 ] , we also run all attacks on a subset of the first 1000 test examples with 10 random restarts for CIFAR-10 data . BASE AT ( ) since Maini et al . [ 21 ] showed that employing ` 1 single-attack AT achieves very low237 robustness to all attacks . We choose PGD [ 20 ] AT as BASE AT ( ) for this ablation study . For a fair238 comparison across the noise distributions , we fix Pnoise = 160 , enforcing all noise vectors to have the239 same average ` 2 norm . For each distribution , the noise is shaped per the procedure summarized in240 Alg . 1.241 As observed in Table 1 , ` ∞-PGD AT achieves much lowerA ( U ) adv than ` 2-PGD AT , an observation also242 reported by Maini et al . [ 21 ] . With SNAP , however , we find that there is an interaction between the243 perturbation model in PGD AT and the noise distribution in SNAP . For instance , SNAP [ U ] enhances244 A ( U ) adv by 11 % with ` ∞-PGD AT while not achieving any improvement with ` 2-PGD AT . In fact,245 SNAP appears to be particularly suitable for ` ∞-AT , since it always improvesA ( U ) adv by 11 % -to-20.6 % 246 irrespective of the noise distribution.247 Finally , of the three noise distributions , we find the Laplace distribution to be distinctly superior,248 achieving the highest A ( U ) adv ( 35.6 % and 30.8 % ) due to a significant improvement in A ( ` 1 ) adv for both249 ` ∞ and ` 2 PGD AT , respectively . The superiority of the Laplace distribution in achieving high250 A ( ` 1 ) adv stems from its heavier tail compared to the Gaussian and Uniform distributions with the same251 variance . Shaped Laplace noise generates the highest fraction of extreme values in a given noise252 sample . Hence , it is more effective in improving accuracy against ` 1-bounded attacks , which are253 the strongest when perturbing few pixels by a large magnitude [ 21 , 32 ] . We discuss this further in254 the Appendix . Henceforth , unless otherwise mentioned , we choose Laplace noise for SNAP and ` ∞255 perturbations for BASE ( ) AT as the default setting since it achieves the highest A ( U ) adv .256 5.2.2 Impact of Pnoise257 ! ! `` # ( ℓ ! ) ! ' ! ( ! ! `` # ( ℓ '' ) '' ' ) * + , Ad ve rs ar ia l A cc ur ac y ( % ) Na tu ra l A cc ur ac y ( % ) ! ! '' # ( % ) PGD+SNAP PGD+Iso [ L ] ! `` $ % ( ' ) ( % ) ∼ $ % ( a ) ( b ) Figure 5 : ResNet-18 CIFAR-10 results : adversarial accuracy A ( ` 1 ) adv , A ( ` ∞ ) adv , and natural accuracy Anat vs. total noise power Pnoise for PGD+SNAP . Next , we explore the impact of the SNAP hyperpa-258 rameter Pnoise , which constrains the average squared259 ` 2 norm of the noise vector n. It enables to trade260 between adversarial and natural accuracy.261 Fig . 5 shows that , as Pnoise increases , A ( ` 1 ) adv improves262 from 31 % to 47 % , accompanied by a graceful ( 5 % ) 263 drop in Anat and a small drop of 2 % in A ( ` ∞ ) adv that264 stabilizes to ≈ 45 % . These results show : ( 1 ) SNAP265 preserves the impact of ` ∞ perturbations which is266 not surprising since PGD AT [ 20 ] explicitly includes267 those , and ( 2 ) Pnoise provides an explicit knob to268 control the Anat vs. Aadv trade-off . Henceforth , we269 choose Pnoise values that incur < 1.5 % drop in Anat270 for all SNAP+AT experiments.271 5.2.3 SNAP augmented SOTA AT Frameworks272 Table 2 shows the effectiveness of SNAP for four SOTA AT frameworks : high complexity frame-273 works , such as PGD [ 20 ] , TRADES [ 41 ] , and low complexity frameworks such as FreeAdv [ 30 ] ,274 FastAdv [ 35 ] . All are trained against ` ∞ attacks with = 0.031 . As expected , while they achieve275 high A ( ` ∞ ) adv , their A ( ` 2 ) adv and A ( ` 1 ) adv are lower.276 For high-complexity AT , SNAP enhances A ( ` 2 ) adv and A ( ` 1 ) adv by ∼ 6 % and ∼ 25 % , respectively , while277 incurring only a drop of ∼ 5 % in A ( ` ∞ ) adv . Thus overall , SNAP improves robustness ( A ( U ) adv ) by ∼ 20 % 278 against the union of the three perturbation models . Note that this robustness improvement comes279 at only a ∼ 1 % drop in Anat ( see Table 2 ) . For low-complexity ATs , SNAP improvements in union280 robustness ( A ( U ) adv ) are also significant ( ∼ 15 % ) . Again , presence of SNAP improves A ( ` 2 ) adv and A ( ` 1 ) adv .281 This time the drop in A ( ` ∞ ) adv is ∼ 7 % . We believe this is due to the fact that these frameworks employ282 weaker single-step attacks during training . Note that in the case of FreeAdv+SNAP , we actually283 observe a ∼ 2 % increase in Anat , a trend we also observe in the ImageNet experiments described284 later.285 5.3 Robustness vs. Training Complexity286 Next we quantify adversarial robustness vs. training time trade-offs . Table 3 shows that SNAP287 augmentation of single-attack AT frameworks achieves the highest A ( U ) adv , when training time is288 constrained to 12 hours ( sets B , C , D , and E ) .289 For instance , TRADES+SNAP achieves a 4 % higherA ( U ) adv ( = 41 % ) than MSD-20 with 2 hours lower290 training time ( Set B in Table 3 ) . Similarly , PGD+SNAP achieves a 2 % higher A ( U ) adv than MSD-10291 while having a similar training time ( Set C ) . Note that both PGD and TRADES here use 100 training292 epochs with standard step learning rate ( LR ) schedule , while MSD frameworks employ a cyclic293 learning rate schedule to achieve superconvergence in 50 epochs.294 In Set D , following Maini et al . [ 21 ] , we employ a cyclic learning rate schedule for PGD , TRADES,295 as well as for PGD+SNAP and TRADES+SNAP to achieve convergence in 50 epochs . Improvements296 in A ( U ) adv for PGD+SNAP and TRADES+SNAP are similar to those in Sets B and C. Most notably,297 PGD+SNAP with cyclic learning rate achieves ∼ 20 % and 11.5 % higher A ( U ) adv than MSD-5 and298 AVG-2 , respectively , while having a similar training time ( ∼ 3 hours ) . Set E augments the data from299 Table 2 with training times . FastAdv+SNAP and FreeAdv+SNAP achieve a high A ( U ) adv ∼ 30 % , while300 preserving the training efficiency of both FastAdv and FreeAdv . Notably , FastAdv+SNAP achieves301 18 % higher A ( U ) adv than MSD-5 , while being ∼ 2.7× more efficient to train.302 5.4 ImageNet Results303 Thanks to SNAP ’ s low computational overhead combined with FreeAdv ’ s fast training time , we are304 for the first time able to report adversarial accuracy of ResNet-50 and ResNet-101 against the union305 of ( ` ∞ , ` 2 , ` 1 ) attacks on ImageNet.306 We closely follow the evaluation setup of Shafahi et al . [ 30 ] . Specifically , we use 100 step PGD307 attack , one of the strongest adversaries considered by Shafahi et al . [ 30 ] , and evaluate on the entire308 test set . We first reproduce FreeAdv [ 30 ] results using the same hyperparameters and then introduce309 SNAP . All hyperparameter details are specified in the Appendix.310 In order to clearly demonstrate the contrast between robustness to different perturbation models , we311 evaluate with = ( 2/255 , 2.0 , 72.0 ) for ( ` ∞ , ` 2 , ` 1 ) attacks , respectively.2 As shown in Table 4,312 FreeAdv achieves a high A ( ` ∞ ) adv = 47.8 % with ResNet-50 , but a lower A ( ` 2 ) adv = 20 % and A ( ` 1 ) adv =313 15 % , and consequently , a low A ( U ) adv of 12.6 % against the union of the perturbations . In contrast,314 FreeAdv+SNAP improves A ( ` 2 ) adv and A ( ` 1 ) adv by 17 % and 22 % , respectively , accompanied by a 5 % 315 improvement inAnat and a small 2 % loss inA ( ` ∞ ) adv . This results in an overall robustness improvement316 of 20 % against the union of the perturbation models , setting a first benchmark for ResNet-50 on317 ImageNet . Upon increasing the network to ResNet-101 , both natural and adversarial accuracies318 improve by ≈ 4 % for FreeAdv , a trend also observed by Shafahi et al . [ 30 ] . SNAP further improves319 FreeAdv ’ s results for Anat and A ( U ) adv by 4.3 % and 19.3 % .320 6 Discussion321 Given the wide popularity of ` ∞-AT , in this paper , we propose SNAP as an augmentation that322 generalizes the effectiveness of ` ∞-AT to the union of ( ` ∞ , ` 2 , ` 1 ) perturbations . SNAP ’ s strength is323 its simplicity and efficiency . Consequently , this work sets a first benchmark for ResNet-50 and ResNet-324 101 networks which are resilient to the union of ( ` ∞ , ` 2 , ` 1 ) perturbations on ImageNet . Note that325 norm-bounded perturbations include a large class of attacks , e.g. , gradient-based [ 20 , 27 , 32 , 21 , 4 , 22 ] ,326 decision-based [ 3 ] and black-box [ 1 ] attacks.327 More work is needed to extend the proposed SNAP technique to attacks beyond norm-bounded328 additive perturbations , e.g. , functional [ 17 , 36 ] , rotation [ 7 ] , texture [ 2 ] , etc . We provide preliminary329 evaluations in this direction in the Appendix . It is important to note that SNAP is meant to be an330 efficient technique for improving ` ∞-AT , and not a new defense . Indeed defending against a large331 variety of attacks simultaneously remains an open problem , with encouraging results from recent332 efforts [ 21 , 18 ] .333 Another limitation of our approach is that its benefits are demonstrated empirically . It is an inevitable334 consequence of a lack of any theoretical guarantees for underlying AT frameworks . An interesting335 direction of future work is to explore whether any theoretical guarantees can be derived for anisotropic336 shaped noise distributions in SNAP by building upon the recent developments in randomized smooth-337 ing [ 29 , 38 ] . This could be a potential avenue for bridging the gap between certification bounds and338 empirical adversarial accuracy.339 Finally , we believe that any effort on improving adversarial robustness of deep nets has net positive340 societal impact . However , recent past in this field has shown that any improvements in defense341 techniques also lead to more effective threat models . While such a cat-and-mouse game is of great342 intellectual value in the academic setting , it does have an unintentional negative societal consequence343 of equipping malicious outside actors with a broad set of tools . This further underscores the well-344 recognized need for provable defenses.345 2Note that ` 2 and ` 1 norms of PGD perturbation with ` ∞ norm of 2/255 can be as large as ∼ 3.0 and ∼ 1100 for images of size 224× 224× 3 . References346 [ 1 ] Andriushchenko , M. , Croce , F. , Flammarion , N. , and Hein , M. Square attack : a query-efficient black-box347 adversarial attack via random search . In European Conference on Computer Vision , pp . 484–501 . Springer,348 2020.349 [ 2 ] Bhattad , A. , Chong , M. J. , Liang , K. , Li , B. , and Forsyth , D. A . Unrestricted adversarial examples via350 semantic manipulation . arXiv preprint arXiv:1904.06347 , 2019.351 [ 3 ] Brendel , W. , Rauber , J. , and Bethge , M. Decision-based adversarial attacks : Reliable attacks against352 black-box machine learning models . In International Conference on Learning Representations , 2018.353 [ 4 ] Chen , P.-Y. , Sharma , Y. , Zhang , H. , Yi , J. , and Hsieh , C.-J . Ead : elastic-net attacks to deep neural networks354 via adversarial examples . In Thirty-second AAAI conference on artificial intelligence , 2018.355 [ 5 ] Cohen , J. , Rosenfeld , E. , and Kolter , Z . Certified adversarial robustness via randomized smoothing . In356 International Conference on Machine Learning ( ICML ) , 2019.357 [ 6 ] Dezfooli , S. M. M. , Fawzi , A. , Fawzi , O. , Frossard , P. , and Soatto , S. Robustness of classifiers to universal358 pertur-bations : A geometric perspective . In International Conference on Learning Representations ( ICLR ) ,359 2018.360 [ 7 ] Engstrom , L. , Tran , B. , Tsipras , D. , Schmidt , L. , and Madry , A . Exploring the landscape of spatial361 robustness . In International Conference on Machine Learning , pp . 1802–1811 . PMLR , 2019.362 [ 8 ] Gilmer , J. , Ford , N. , Carlini , N. , and Cubuk , E. Adversarial examples are a natural consequence of test363 error in noise . In International Conference on Machine Learning , pp . 2280–2289 , 2019.364 [ 9 ] Gowal , S. , Qin , C. , Uesato , J. , Mann , T. , and Kohli , P. Uncovering the limits of adversarial training against365 norm-bounded adversarial examples . arXiv preprint arXiv:2010.03593 , 2020.366 [ 10 ] Gui , S. , Wang , H. , Yu , C. , Yang , H. , Wang , Z. , and Liu , J . Model compression with adversarial robustness:367 A unified optimization framework . arXiv preprint arXiv:1902.03538 , 2019.368 [ 11 ] Guo , M. , Yang , Y. , Xu , R. , Liu , Z. , and Lin , D. When nas meets robustness : In search of robust architectures369 against adversarial attacks . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern370 Recognition , pp . 631–640 , 2020.371 [ 12 ] He , Z. , Rakin , A. S. , and Fan , D. Parametric noise injection : Trainable randomness to improve deep neural372 network robustness against adversarial attack . In Proceedings of the IEEE Conference on Computer Vision373 and Pattern Recognition ( CVPR ) , 2019.374 [ 13 ] Hendrycks , D. and Dietterich , T. Benchmarking neural network robustness to common corruptions and375 perturbations . In International Conference on Learning Representations , 2018.376 [ 14 ] Hu , T.-K. , Chen , T. , Wang , H. , and Wang , Z . Triple wins : Boosting accuracy , robustness and efficiency377 together by enabling input-adaptive inference . arXiv preprint arXiv:2002.10025 , 2020.378 [ 15 ] Jordan , M. , Manoj , N. , Goel , S. , and Dimakis , A. G. Quantifying perceptual distortion of adversarial379 examples . arXiv preprint arXiv:1902.08265 , 2019.380 [ 16 ] Kang , D. , Sun , Y. , Brown , T. , Hendrycks , D. , and Steinhardt , J . Transfer of adversarial robustness between381 perturbation types . arXiv preprint arXiv:1905.01034 , 2019.382 [ 17 ] Laidlaw , C. and Feizi , S. Functional adversarial attacks . Advances in Neural Information Processing383 Systems , 2019.384 [ 18 ] Laidlaw , C. , Singla , S. , and Feizi , S. Perceptual adversarial robustness : Defense against unseen threat385 models . International Conference on Learning Representations ( ICLR ) , 2018.386 [ 19 ] Li , B. , Chen , C. , Wang , W. , and Duke , L. C. Certified adversarial robustness with addition gaussian noise.387 Neural Information Processing Systems ( NeurIPS ) , 2019.388 [ 20 ] Madry , A. , Makelov , A. , Schmidt , L. , Tsipras , D. , and Vladu , A . Towards deep learning models resistant389 to adversarial attacks . International Conference on Learning Representations ( ICLR ) , 2018.390 [ 21 ] Maini , P. , Wong , E. , and Kolter , J . Z. Adversarial robustness against the union of multiple perturbation391 models . In International Conference on Machine Learning ( ICML ) , 2020.392 [ 22 ] Moosavi-Dezfooli , S.-M. , Fawzi , A. , and Frossard , P. Deepfool : a simple and accurate method to fool393 deep neural networks . In Proceedings of the IEEE conference on computer vision and pattern recognition394 ( CVPR ) , 2016.395 [ 23 ] Moosavi-Dezfooli , S.-M. , Fawzi , A. , Uesato , J. , and Frossard , P. Robustness via curvature regularization,396 and vice versa . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition397 ( CVPR ) , 2019.398 [ 24 ] Pinot , R. , Meunier , L. , Araujo , A. , Kashima , H. , Yger , F. , Gouy-Pailler , C. , and Atif , J. Theoretical399 evidence for adversarial robustness through randomization : the case of the exponential family . In Advances400 in Neural Information Processing Systems , 2019.401 [ 25 ] Pinot , R. , Ettedgui , R. , Rizk , G. , Chevaleyre , Y. , and Atif , J. Randomization matters . how to defend against402 strong adversarial attacks . In International Conference on Machine Learning ( ICML ) , 2020.403 [ 26 ] Rebuffi , S.-A. , Gowal , S. , Calian , D. A. , Stimberg , F. , Wiles , O. , and Mann , T. Fixing data augmentation to404 improve adversarial robustness . arXiv preprint arXiv:2103.01946 , 2021.405 [ 27 ] Rony , J. , Hafemann , L. G. , Oliveira , L. S. , Ayed , I . B. , Sabourin , R. , and Granger , E. Decoupling direction406 and norm for efficient gradient-based l2 adversarial attacks and defenses . In Proceedings of the IEEE/CVF407 Conference on Computer Vision and Pattern Recognition , pp . 4322–4330 , 2019.408 [ 28 ] Rusak , E. , Schott , L. , Zimmermann , R. S. , Bitterwolf , J. , Bringmann , O. , Bethge , M. , and Brendel , W. A409 simple way to make neural networks robust against diverse image corruptions . In European Conference on410 Computer Vision , pp . 53–69 . Springer , 2020.411 [ 29 ] Salman , H. , Li , J. , Razenshteyn , I. , Zhang , P. , Zhang , H. , Bubeck , S. , and Yang , G. Provably robust deep412 learning via adversarially trained smoothed classifiers . In Advances in Neural Information Processing413 Systems , pp . 11289–11300 , 2019.414 [ 30 ] Shafahi , A. , Najibi , M. , Ghiasi , A. , Xu , Z. , Dickerson , J. , Studer , C. , Davis , L. S. , Taylor , G. , and Goldstein,415 T. Adversarial training for free ! Advances in Neural Information Processing Systems ( NeurIPS ) , 2019.416 [ 31 ] Stutz , D. , Hein , M. , and Schiele , B. Confidence-calibrated adversarial training : Generalizing to unseen417 attacks . In International Conference on Machine Learning , pp . 9155–9166 . PMLR , 2020.418 [ 32 ] Tramèr , F. and Boneh , D. Adversarial training and robustness for multiple perturbations . In Advances in419 Neural Information Processing Systems , pp . 5858–5868 , 2019.420 [ 33 ] Tramer , F. , Carlini , N. , Brendel , W. , and Madry , A . On adaptive attacks to adversarial example defenses.421 arXiv preprint arXiv:2002.08347 , 2020.422 [ 34 ] Vivek , B. and Babu , R. V. Single-step adversarial training with dropout scheduling . In 2020 IEEE/CVF423 Conference on Computer Vision and Pattern Recognition ( CVPR ) , pp . 947–956 . IEEE , 2020.424 [ 35 ] Wong , E. , Rice , L. , and Kolter , J . Z . Fast is better than free : Revisiting adversarial training . In International425 Conference on Machine Learning ( ICLR ) , 2020.426 [ 36 ] Xiao , C. , Zhu , J.-Y. , Li , B. , He , W. , Liu , M. , and Song , D. Spatially transformed adversarial examples . In427 International Conference on Learning Representations , 2018.428 [ 37 ] Xie , C. and Yuille , A. Intriguing properties of adversarial training at scale . In International Conference on429 Learning Representations , 2020.430 [ 38 ] Yang , G. , Duan , T. , Hu , E. , Salman , H. , Razenshteyn , I. , and Li , J. Randomized smoothing of all shapes431 and sizes . International Conference on Machine Learning ( ICML ) , 2020.432 [ 39 ] Yang , Y.-Y. , Rashtchian , C. , Zhang , H. , Salakhutdinov , R. , and Chaudhuri , K. A closer look at accuracy vs.433 robustness . Advances in Neural Information Processing Systems , 33 , 2020.434 [ 40 ] Zhang , D. , Zhang , T. , Lu , Y. , Zhu , Z. , and Dong , B . You only propagate once : Accelerating adversarial435 training via maximal principle . arXiv preprint arXiv:1905.00877 , 2019.436 [ 41 ] Zhang , H. , Yu , Y. , Jiao , J. , Xing , E. , El Ghaoui , L. , and Jordan , M. Theoretically principled trade-off437 between robustness and accuracy . In International Conference on Machine Learning ( ICML ) , 2019.438 [ 42 ] Zhang , J. , Xu , X. , Han , B. , Niu , G. , Cui , L. , Sugiyama , M. , and Kankanhalli , M. Attacks which do not439 kill training make adversarial learning stronger . In International Conference on Machine Learning , pp.440 11278–11287 . PMLR , 2020.441 [ 43 ] Zheng , H. , Zhang , Z. , Gu , J. , Lee , H. , and Prakash , A . Efficient adversarial training with transferable442 adversarial examples . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern443 Recognition , pp . 1181–1190 , 2020.444 Checklist445 1 . For all authors ... 446 ( a ) Do the main claims made in the abstract and introduction accurately reflect the paper ’ s447 contributions and scope ? [ Yes ] 448 ( b ) Did you describe the limitations of your work ? [ Yes ] See Section 6.449 ( c ) Did you discuss any potential negative societal impacts of your work ? [ Yes ] See450 Section 6.451 ( d ) Have you read the ethics review guidelines and ensured that your paper conforms to452 them ? [ Yes ] 453 2 . If you are including theoretical results ... 454 ( a ) Did you state the full set of assumptions of all theoretical results ? [ N/A ] 455 ( b ) Did you include complete proofs of all theoretical results ? [ N/A ] 456 3 . If you ran experiments ... 457 ( a ) Did you include the code , data , and instructions needed to reproduce the main experi-458 mental results ( either in the supplemental material or as a URL ) ? [ Yes ] An URL to our459 code as well as the pretrained models is provided in the Appendix.460 ( b ) Did you specify all the training details ( e.g. , data splits , hyperparameters , how they461 were chosen ) ? [ Yes ] All training hyperparameters are mentioned in the Appendix.462 Our technique does introduce a new hyperparameter , whose impact is discussed in463 Sec . 5.2.2.464 ( c ) Did you report error bars ( e.g. , with respect to the random seed after running exper-465 iments multiple times ) ? [ Yes ] We do run a subset of experiments multiple times to466 obtain error bars ( see Appendix ) . In doing so we confirm that our technique is effective467 across random initializations . However , some of the training runs in our work are too468 expensive to run multiple times.469 ( d ) Did you include the total amount of compute and the type of resources used ( e.g. , type470 of GPUs , internal cluster , or cloud provider ) ? [ Yes ] Yes , we explicitly mention the471 type of GPUs used and the training times in Section 5.472 4 . If you are using existing assets ( e.g. , code , data , models ) or curating/releasing new assets ... 473 ( a ) If your work uses existing assets , did you cite the creators ? [ Yes ] We appropriately474 cite the relevant papers while using their code to reproduce/extend their results.475 ( b ) Did you mention the license of the assets ? [ Yes ] Our own codes & models , as well as,476 all the other codes that we use are available freely in public domain . We do mention so477 explicitly in Section 5.1478 ( c ) Did you include any new assets either in the supplemental material or as a URL ? [ Yes ] 479 We do share our own code and pretrained model as a part of the supplemental material480 ( d ) Did you discuss whether and how consent was obtained from people whose data you ’ re481 using/curating ? [ N/A ] 482 ( e ) Did you discuss whether the data you are using/curating contains personally identifiable483 information or offensive content ? [ N/A ] 484 5 . If you used crowdsourcing or conducted research with human subjects ... 485 ( a ) Did you include the full text of instructions given to participants and screenshots , if486 applicable ? [ N/A ] 487 ( b ) Did you describe any potential participant risks , with links to Institutional Review488 Board ( IRB ) approvals , if applicable ? [ N/A ] 489 ( c ) Did you include the estimated hourly wage paid to participants and the total amount490 spent on participant compensation ? [ N/A ] 491

Summary: This paper produces SNAP -- that augments a noise layer to networks during standard adversarial training. On 9 different adv training methods for l_inf perturbations, the authors find that adding a noise layer helps improve robustness against the union of l1, l2, l_inf perturbations. The key hypothesis that the work is built on is that since AT reduced the curvature of the decision boundary, various attacks belonging to different perturbation types get closely aligned. Using this property, the authors attempt to modify the l_inf model boundary by adding a specifically chosen noise.