Title: RAMP: Boosting Adversarial Robustness Against Multiple $l_p$ Perturbations for Universal Robustness
Venue: NeurIPS 2024
Accepted: yes
Keywords: Adversarial Robustness

Pre-training and Fine-tuning

Distribution Shifts
Abstract: Most existing works focus on improving robustness against adversarial attacks bounded by a single $l_p$ norm using adversarial training (AT). However, these AT models' multiple-norm robustness (union accuracy) is still low, which is crucial since in the real-world an adversary is not necessarily bounded by a single norm. The tradeoffs among robustness against multiple $l_p$ perturbations and accuracy/robustness make obtaining good union and clean accuracy challenging. We design a logit pairing loss to improve the union accuracy by analyzing the tradeoffs from the lens of distribution shifts. We connect natural training (NT) with AT via gradient projection, to incorporate useful information from NT into AT, where we empirically and theoretically show it moderates the accuracy/robustness tradeoff. We propose a novel training framework \textbf{RAMP}, to boost the robustness against multiple $l_p$ perturbations. \textbf{RAMP} can be easily adapted for robust fine-tuning and full AT. For robust fine-tuning, \textbf{RAMP} obtains a union accuracy up to $53.3\%$ on CIFAR-10, and $29.1\%$ on ImageNet. For training from scratch, \textbf{RAMP} achieves a union accuracy of $44.6\%$ and good clean accuracy of $81.2\%$ on ResNet-18 against AutoAttack on CIFAR-10. Beyond multi-norm robustness \textbf{RAMP}-trained models achieve superior \textit{universal robustness}, effectively generalizing against a range of unseen adversaries and natural corruptions.

RAMP: Boosting Adversarial Robustness Against Multiple l p Perturbations for Universal Robustness

Abstract

Most existing works focus on improving robustness against adversarial attacks bounded by a single l p norm using adversarial training (AT). However, these AT models' multiple-norm robustness (union accuracy) is still low, which is crucial since in the real-world an adversary is not necessarily bounded by a single norm. The tradeoffs among robustness against multiple l p perturbations and accuracy/robustness make obtaining good union and clean accuracy challenging. We design a logit pairing loss to improve the union accuracy by analyzing the tradeoffs from the lens of distribution shifts. We connect natural training (NT) with AT via gradient projection, to incorporate useful information from NT into AT, where we empirically and theoretically show it moderates the accuracy/robustness tradeoff. We propose a novel training framework RAMP , to boost the robustness against multiple l p perturbations. RAMP can be easily adapted for robust fine-tuning and full AT. For robust fine-tuning, RAMP obtains a union accuracy up to 53 . 3% on CIFAR-10, and 29 . 1% on ImageNet. For training from scratch, RAMP achieves a union accuracy of 44 . 6% and good clean accuracy of 81 . 2% on ResNet-18 against AutoAttack on CIFAR-10. Beyond multi-norm robustness RAMP -trained models achieve superior universal robustness , effectively generalizing against a range of unseen adversaries and natural corruptions.

1 Introduction

Though deep neural networks (DNNs) demonstrate superior performance in various vision applications, they are vulnerable against adversarial examples [Goodfellow et al., 2014, Kurakin et al., 2018]. Adversarial training (AT) [Tramèr et al., 2017, Madry et al., 2017] which works by injecting adversarial examples into training for enhanced robustness, is currently the most popular defense. However, most AT methods address only a single type of perturbation [Wang et al., 2020, Wu et al., 2020, Carmon et al., 2019, Gowal et al., 2020, Raghunathan et al., 2020, Zhang et al., 2021, Debenedetti and Troncoso-EPFL, 2022, Peng et al., 2023, Wang et al., 2023]. An l ∞ robust model may not be robust against l p ( p = ∞ ) attacks. Also, enhancing robustness against one perturbation type can sometimes increase vulnerability to others [Engstrom et al., 2017, Schott et al., 2018]. On the contrary, training a model to be robust against multiple l p perturbations is crucial as it reflects real-world scenarios [Sharif et al., 2016, Eykholt et al., 2018, Song et al., 2018, Athalye et al., 2018] where adversaries can use multiple l p perturbations. We show that multi-norm robustness is the key to improving generalization against other threat models [Croce and Hein, 2022]. For instance, we show it enables robustness against perturbations not easily defined mathematically, such as image corruptions and unseen adversaries [Wong and Kolter, 2020].

̸

Two main challenges exist for training models robust against multiple perturbations: (i) tradeoff among robustness against different perturbation models [Tramer and Boneh, 2019] and (ii) tradeoff between accuracy and robustness [Zhang et al., 2019, Raghunathan et al., 2020]. Adversarial examples induce a shift from the original distribution, causing a drop in clean accuracy with AT [Xie et al., 2020, Benz et al., 2021]. The distinct distributions created by l 1 , l 2 , l ∞ adversarial examples make the problem even more challenging. Through a finer analysis of the distribution shifts caused by these adversaries, we propose the RAMP framework to efficiently boost the R obustness A gainst M ultiple P erturbations. RAMP can be used for both fine-tuning and training from scratch. It utilizes a novel logit pairing loss on a certain pair and connects NT with AT via gradient projection [Jiang et al., 2023] to improve union accuracy while maintaining good clean accuracy and training efficiency.

Logit pairing loss. We visualize the changing of l 1 , l 2 , l ∞ robustness when fine-tuning a l ∞ -AT pre-trained model in Figure 1 using the CIFAR-10 training dataset. The DNN loses substantial robustness against l ∞ attack after only 1 epoch of fine-tuning: l 1 fine-tuning and E-AT [Croce and Hein, 2022] (red and yellow histograms under Linf category) both lose significant l ∞ robustness (compared with blue histogram under Linf category). Inspired by this observation, we devise a new logit pairing loss for a l q -l r tradeoff pair to attain better union accuracy, which enforces the logit distributions of l q and l r adversarial examples to be close, specifically on the correctly classified l q subsets. In comparison, our method (green histogram under Linf and union categories) preserves more l ∞ and union robustness than others after 1 epoch. We show this technique works on larger models and datasets (Section 5.1).

Connect natural training (NT) with AT. We explore the connections between NT and AT to obtain a better accuracy/robustness tradeoff. We find that NT can help with adversarial robustness: useful information in natural distribution can be extracted and leveraged to achieve better robustness. To this end, we compare the similarities of model updates of NT and AT layer-wise for each epoch, where we find and incorporate useful NT components into AT via gradient projection (GP), as outlined in Algorithm 2. In Figure 2 and Section 5.1, we empirically and theoretically show this technique strikes a better balance between accuracy and robustness, for both single and multiple l p perturbations. We provide

0.2

0.4

0.6

0.8

Linf

L1

L2

Union

a theoretical analysis of why GP works for adversarial robustness in Theorem A.2 & 4.5.

Main contributions :

We design a new logit pairing loss to mitigate the l q -l r tradeoff for better union accuracy, by enforcing the logit distributions of l q and l r adversarial examples to be close.

We empirically and theoretically show that connecting NT with AT via gradient projection better balances the accuracy/robustness tradeoff for l p perturbations, compared with standard AT.

RAMP achieves good union accuracy, accuracy-robustness tradeoff, and generalizes better to diverse perturbations and corruptions (Section 5.1) achieving superior universal robustness ( 75 . 5% for common corruption and 26 . 1% union accuracy against unseen adversaries). RAMP fine-tuned DNNs achieve union accuracy up to 53 . 3% on CIFAR-10, and 29 . 1% on ImageNet. RAMP achieves a 44 . 6% union accuracy and good clean accuracy on ResNet-18 against AutoAttack on CIFAR-10. Our code is available at https://github.com/uiuc-focal-lab/RAMP .

2 Related Work

Adversarial training (AT). Adversarial Training (AT) usually employs gradient descent to discover adversarial examples, incorporating them into training for enhanced adversarial robustness [Tramèr et al., 2017, Madry et al., 2017]. Numerous works focus on improving robustness by exploring the trade-off between robustness and accuracy [Zhang et al., 2019, Wang et al., 2020], instance reweighting [Zhang et al., 2021], loss landscapes [Wu et al., 2020], wider/larger architectures [Gowal

et al., 2020, Debenedetti and Troncoso-EPFL, 2022], data augmentation [Carmon et al., 2019, Raghunathan et al., 2020], and using synthetic data [Peng et al., 2023, Wang et al., 2023]. However, these methods often yield DNNs robust against a single perturbation type while remaining vulnerable to other types.

̸

Robustness against multiple perturbations. Tramer and Boneh [2019], Kang et al. [2019] observe that robustness against l p attacks does not necessarily transfer to other l q attacks ( q = p ). Previous studies [Tramer and Boneh, 2019, Maini et al., 2020, Madaan et al., 2021, Croce and Hein, 2022] modified Adversarial Training (AT) to enhance robustness against multiple l p attacks, employing average-case [Tramer and Boneh, 2019], worst-case [Tramer and Boneh, 2019, Maini et al., 2020], and random-sampled [Madaan et al., 2021, Croce and Hein, 2022] defenses. There are also works [Nandy et al., 2020, Liu et al., 2020, Xu et al., 2021, Xiao et al., 2022, Maini et al., 2022] using preprocessing, ensemble methods, mixture of experts, and stability analysis to solve this problem. Ensemble models and preprocessing methods are weakened since their performance heavily relies on correctly classifying or detecting various types of adversarial examples. In certified training, Banerjee et al. [2024], Banerjee and Singh [2024] propose verification/certifiable training methods under different threat models for l p universal adversarial perturbation. However, prior works are hard to scale to larger models and datasets, e.g. ImageNet, due to the efficiency issue. Furthermore, Croce and Hein [2022] devise Extreme norm Adversarial Training (E-AT) and fine-tune a l p robust model on another l q perturbation to quickly make a DNN robust against multiple l p attacks. However, E-AT does not adapt to varying epsilon values. Our work demonstrates that the suboptimal tradeoff observed in prior studies can be improved with our proposed framework.

̸

Logit pairing in adversarial training. Adversarial logit pairing methods encourage logits for pairs of examples to be similar [Kannan et al., 2018, Engstrom et al., 2018]. People apply this technique to both clean images and their adversarial counterparts, to devise a stronger form of adversarial training. In our work, we devise a novel logit pairing loss to train a DNN originally robust against l p attack to become robust against another l q ( q = p ) attack on the correctly predicted l p subsets, which helps gain better union accuracy.

Adversarial versus distributional robustness. Sinha et al. [2018] theoretically studies the AT problem through distributional robust optimization. Mehrabi et al. [2021] establishes a pareto-optimal tradeoff between standard and adversarial risks by perturbing the test distribution. Other works explore the connection between natural and adversarial distribution shifts [Moayeri et al., 2022, Alhamoud et al., 2023], assessing transferability and generalizability of adversarial robustness across datasets. However, little research delves into distribution shifts induced by l 1 , l 2 , l ∞ adversarial examples and their interplay with the robustness-accuracy tradeoff [Zhang et al., 2019, Yang et al., 2020, Rade and Moosavi-Dezfooli, 2021]. Our work, inspired by recent domain adaptation techniques [Jiang, 2023, Jiang et al., 2023], designs a logit pairing loss and utilizes model updates from NT via GP to enhance adversarial robustness. We show that GP adapts to both single and multi-norm scenarios.

3 AT against Multiple Perturbations

We consider a standard classification task with samples { ( x i , y i ) } N i =0 from an empirical data distribution ̂ D n ; we have input images x ∈ R d and corresponding labels y ∈ R k . Standard training aims to obtain a classifier f parameterized by θ to minimize a loss function L : R k × R k → R on ̂ D n . Adversarial training (AT) [Madry et al., 2017, Tramèr et al., 2017] aims to find a DNN robust against adversarial examples. It is framed as a min-max problem where a DNN is optimized using the worst-case examples within an adversarial region around each x i . Different types of adversarial regions B p ( x, ϵ p ) = { x ′ ∈ R d : ∥ x ′ -x ∥ p ≤ ϵ p } can be defined around a given image x using various l p -based perturbations. Formally, we can write the optimization problem of AT against a certain l p attack as follows:

The above optimization is only for certain p values and is usually vulnerable to other perturbation types. To this end, prior works have proposed several approaches to train the network robust against multiple perturbations ( l 1 , l 2 , l ∞ ) at the same time. We focus on the union threat model

∆ = B 1 ( x, ϵ 1 ) ∪ B 2 ( x, ϵ 2 ) ∪ B ∞ ( x, ϵ ∞ ) which requires the DNN to be robust within the l 1 , l 2 , l ∞ adversarial regions simultaneously [Croce and Hein, 2022]. Union accuracy is then defined as the robustness against ∆ ( i ) for each x i sampled from D . In this paper, similar to the prior works, we use union accuracy as the main metric to evaluate the multiple-norm robustness. Apart from that, we define universal robustness as the generalization ability against a range of unseen adversaries and common corruptions. Specifically, we have average accuracy across five severity levels for common corruption and union accuracy against a range of unseen adversaries used in Laidlaw et al. [2020].

Worst-case defense follows the following min-max optimization problem to train DNNs using the worst-case example from the l 1 , l 2 , l ∞ adversarial regions:

MAX [Tramer and Boneh, 2019] and MSD [Maini et al., 2020] fall into this category. Finding worst-case examples yields a good union accuracy but results in a loss of clean accuracy as the distribution of generated examples is different from the clean data distribution.

Average-case defense train DNNs using the average of the l 1 , l 2 , l ∞ worst-case examples:

AVG [Tramer and Boneh, 2019] is of this type. This method generally leads to good clean accuracy but suboptimal union accuracy as it does not penalize worst-case behavior within the l 1 , l 2 , l ∞ regions.

Random-sampled defense. The defenses mentioned above lead to a high training cost as they compute multiple attacks for each sample. SAT [Madaan et al., 2021] and E-AT [Croce and Hein, 2022] randomly sample one attack out of each type at a time, contributing to a similar computational cost as standard AT on a single perturbation model. They achieve a slightly better union accuracy compared with AVG and relatively good clean accuracy. However, they are not better than worst-case defenses for multiple-norm robustness, since they do not consider the strongest attack within the union region all the time.

4 RAMP

There are two main tradeoffs in achieving better union accuracy while maintaining good accuracy: 1. Among perturbations: there is a tradeoff among different attacks, e.g., a l ∞ pre-trained AT DNN is not robust against l 1 , l 2 perturbations, which makes the union accuracy harder to attain. Also, we observe there exists a main tradeoff pair of two attacks among the union over l 1 , l 2 , l ∞ attacks. 2. Accuracy and robustness: all defenses lead to degraded clean accuracy. To address these tradeoffs, we study the problem from the lens of distribution shifts.

Interpreting tradeoffs from the lens of distribution shifts. The adversarial examples with respect to an empirical data distribution ̂ D n , adversarial region B p ( x, ϵ p ) , and DNN f θ generate a new adversarial distribution ̂ D a with samples { ( x ′ i , y i ) } N i =0 , that are correlated by adding certain perturbations but different from the original ̂ D n . Because of the shifts between ̂ D n and ̂ D a , DNN decreases performance on ̂ D n when we move away from it and towards ̂ D a . Also, the distinct distributions created by multiple perturbations, ̂ D l 1 a , ̂ D l 2 a , ̂ D l ∞ a , contribute to the tradeoff among l 1 , l 2 , l ∞ attacks. To address the tradeoff among perturbations while maintaining good efficiency, we focus on the distributional interconnections between ̂ D n and ̂ D l 1 a , ̂ D l 2 a , ̂ D l ∞ a . From the insights we get from above, we propose our framework RAMP , which includes (i) logit pairing to improve tradeoffs among multiple perturbations, and (ii) identifying and combining the useful DNN components using the model updates from NT and AT, to obtain a better robustness/accuracy tradeoff.

Identify the Key Tradeoff Pair. We study the common case with l p norms ϵ 1 = 12 , ϵ 2 = 0 . 5 , ϵ ∞ = 8 255 on CIFAR-10 [Tramer and Boneh, 2019]. The distributions generated by the two strongest attacks show the largest shifts from ̂ D n ; also, they have the largest distribution shifts between each other

because of larger and most distinct search areas. Thus, by comparing the single norm robustness of l p adversarially trained models, we select the two l p -AT models with the lowest l p robustness against themselves as the key tradeoff pair. They refer to the strongest attack since their l p robustness is low. The attack with the highest l p robustness is mostly included by the convex hull of the other two stronger attacks [Croce and Hein, 2022]. Here we identify l ∞ -l 1 as the key tradeoff pair.

4.1 Logit Pairing for Multiple Perturbations

̸

Figure 1: Finetuning a l q -AT model on l r examples reduces l q robustness. To get a finer analysis of the l ∞ -l 1 tradeoff mentioned above, we visualize the changing of l 1 , l 2 , l ∞ robustness of the training dataset when we fine-tune a l ∞ pre-trained model with l 1 examples for 1 epochs, as shown in Figure 1: x-axis represents the robustness against different attacks and y-axis is the accuracy. After 1 epoch of finetuning on l 1 examples or performing E-AT, we lose much l ∞ robustness since blue/yellow histograms are much lower than the red histogram under the Linf category. RAMP preserves both l ∞ and union robustness more effectively: the green histogram is higher than the red/yellow histogram under Linf and Union categories. Specifically, RAMP maintains 14%, 28% more union robustness than E-AT and l 1 fine-tuning. The above observations indicate the necessity of preserving more l q robustness as we adversarially fine-tune with l r adversarial examples on a l q pre-trained AT model, with l q -l r as the key tradeoff pair, which inspires us to design our loss design with logit pairing. We want to enforce the union predictions between l q and l r ( q = r ) attacks: bringing the predictions of l q and l r ( q = r ) close to each other, specifically on the correctly predicted l q subsets. Based on our observations, we design a new logit pairing loss to enforce a DNN robust against one l q attack to be robust against another l r ( q = r ) attack.

̸

̸

Enforcing the Union Prediction via Logit Pairing. The l q -l r ( q = r ) tradeoff leads us to the following principle to improve union accuracy: for a given set of images, when we have a DNN robust against some l q examples, we want it to be robust against l r examples as well. This serves as the main insight for our loss design: we want to enforce the logits predicted by l q and l r adversarial examples to be close, specifically on the correctly predicted l q subsets. To accomplish this, we design a KL-divergence (KL) loss between the predictions from l q and l r perturbations. For each batch of data ( x, y ) ∼ D , we generate l q and l r adversarial examples x ′ q , x ′ r and their predictions p q , p r using APGD [Croce and Hein, 2020]. Then, we select indices γ , which part elements of p q correctly predicts the ground truth y . We denote the size of the indices as n c , and the batch size as N . We compute a KL-divergence loss over this set of samples using KL ( p q [ γ ] ∥ p r [ γ ]) (Eq. 1). For the subset indexed by γ , we want to push its l r logit distribution towards its l q logit distribution, such that we prevent losing more l q robustness when training with l r adversarial examples.

To further boost the union accuracy, apart from the KL loss, we add another loss term using a MAX-style approach in Eq. 2: we find the worst-case example between l q and l r adversarial regions by selecting the example with the higher loss. L max is a cross-entropy loss over the approximated worst-case adversarial examples. Here, we use L ce to represent the cross-entropy loss. Our final loss L combines L KL and L max , via a hyper-parameter λ in Eq. 3.

Algorithm 1 shows the pseudocode of robust fine-tuning with RAMP that leverages logit pairing.

4.2 Connecting Natural Training with AT

To improve the robustness and accuracy tradeoff against multiple perturbations, we explore the connections between AT and NT. Since extracting valuable information in NT aids in improving robustness (Section 4.2), we use gradient projection [Jiang et al., 2023] to compare and integrate natural and adversarial model updates, which yields an improved tradeoff between robustness and accuracy.

̸

NT can help adversarial robustness. Let us consider two models f 1 and f 2 , where f 1 is randomly initialized and f 2 undergoes NT on ̂ D n for k epochs: f 2 results in a better decision boundary and higher clean accuracy. Performing AT on f 1 and f 2 subsequently, intuitively, f 2 becomes more robust than f 1 due to its improved decision boundary, leading to fewer misclassifications of adversarial examples. This effect is empirically shown in Figure 2. For AT (blue), standard AT against l ∞ attack [Madry et al., 2017] is performed, while for AT-pre (red), 50 epochs of pre-training precede the standard AT procedure. AT-pre shows superior clean and robust accuracy on CIFAR-10 against l ∞ PGD-20 attack with ϵ ∞ = 0 . 031 . Despite ̂ D n and ̂ D a are different, Figure 2 suggests valuable information in ̂ D n that potentially enhances performance on ̂ D a .

AT with Gradient Projection. To connect NT with AT more effectively, we analyze the training procedures on ̂ D n and ̂ D a . We consider model updates over all samples from ̂ D n and ̂ D a , with the initial model f ( r ) at epoch r , and models f ( r ) n and f ( r ) a after 1 epoch of natural and adversarial training from the same starting point f ( r ) , respectively. Here, we compare the natural updates ̂ g n = f ( r ) n -f ( r ) and adversarial updates ̂ g a = f ( r ) a -f ( r ) . Due to distribution shift, an angle exists between them. Our goal is to identify useful components from g n and incorporate them into g a for increased robustness in ̂ D a while maintaining accuracy in ̂ D n . Inspired by Jiang et al. [2023], we layer-wisely compute the cosine similarity between ̂ g n and ̂ g a . For a specific layer l of ̂ g l n and ̂ g l a , we preserve a portion of ̂ g l n based on their cosine similarity score (Eq.4). Negative scores indicate that ̂ g l n is not beneficial for robustness in ̂ D a . Therefore, we filter components with similarity score ≤ 0 . We define the GP (Gradient Projection) operation in Eq.5 by projecting ̂ g l a towards ̂ g l n .

Therefore, the total projected (useful) model updates g p coming from ̂ g n could be computed as Eq. 6. We use M to denote all layers of the current model update. Note that ⋃ l ∈M concatenates all layers' useful natural model update components. A hyper-parameter β is used to balance the contributions of g GP and ̂ g a , as shown in Eq. 7. By finding a proper β (0.5 as in Figure 4c), we can obtain better robustness on ̂ D a , as shown in Figure 2 and Figure 3. In Figure 2, with β = 0 . 5 , AT-GP refers to AT with GP; for AT-GP-pre , we perform 50 epochs of NT before doing AT-GP . We see AT-GP obtains a better accuracy/robustness tradeoff than AT . We observe a similar trend for AT-GP-pre vs. AT-pre . Further, in Figure 3, RN-18 l ∞ -GP achieves good clean accuracy and better robustness than RN-18 l ∞ against AutoAttack [Croce and Hein, 2020].

Algorithm 1 Fine-tuning via Logit Pairing

Algorithm 2 Connect AT with NT via GP

1: Input : model f , input samples ( x, y ) from distribution ̂ D n , fine-tuning rounds R , hyper-parameter λ , adversarial regions B q , B r with size ϵ q and ϵ r , APGD attack.

2: for r = 1 , 2 , ..., R do

3: for ( x, y ) ∼ training set D do

4: x ′ q , p q ← APGD ( B q ( x, ϵ q ) , y )

5: x ′ r , p r ← APGD ( B r ( x, ϵ r ) , y )

6: γ ← where ( argmax p q = y )

7: n c ← γ.size ()

8: calculate L using Eq. 3 and update f

9: end for

10: end for

11: Output : model f .

1: Input : model f , input images with distribution ̂ D n , training rounds R , adversarial region B p and its size ϵ p , β , natural training NT and adversarial training AT .

2: for r = 1 , 2 , ..., R do

3: f n ← NT ( f ( r ) , D )

4: f a ← AT ( f ( r ) , B p , ϵ p , D )

5: compute ̂ g n ← f n -f ( r ) , ̂ g a ← f a -f ( r )

6: compute g p using Eq. 6

7: update f ( r +1) using Eq. 7 with β and ̂ g a

8: end for

9: Output : model f .

4.3 Theoretical Analysis of GP for Adversarial Robustness

We define D n = { ( x i , y i ) } ∞ i =0 as the ideal data distribution with an infinite cardinality. Here, we consider a classifier f θ at epoch t . Wedefine D a as the distribution created by { ( x i + ϵ ( f θ , x i , y i ) , y i ) } ∞ i =0 where ( x i , y i ) ∼ D n . x i + ϵ ( f θ , x i , y i ) denotes the perturbed image, which could be both single and multiple perturbations based on f θ itself.

Assumption 4.1. We assume ̂ D n consists of N i.i.d. samples from the ideal distribution D n and ̂ D a = { ( x i + ϵ ( f θ , x i , y i ) , y i ) } N i =0 where ( x i , y i ) ∼ ̂ D n consists of N i.i.d. samples from D a .

We define the population loss as L D ( θ ) := E ( x,y ) ∼D L ( f ( x ) , y ) , and let g D ( θ ) := ∇L D ( θ ) . For simplification, we use g a := ∇L D a ( θ ) , ̂ g a := ∇L ̂ D a ( θ ) , and ̂ g n := ∇L ̂ D n ( θ ) . g GP = β · g p +(1 -β ) · ̂ g a (Definition A.3) is the aggregation using GP. We define the following optimization problem.

Definition 4.2 (Aggregation for NT and AT) . f θ is trained by iteratively updating the parameter

where µ is the step size. We seek an aggregation rule Aggr ( · ) = ̂ g Aggr such that after training, f θ minimizes the population loss function L D a ( θ ) .

We need ̂ g Aggr to be close to g a for each iteration, since g a is the optimal update on D a . Thus, we define L π -Norm and delta error to indicate the performance of different aggregation rules.

Definition 4.3 ( L π -Norm [Enyi Jiang, 2024]) . Given a distribution π on the parameter space θ , we define an inner product ⟨ g D , g D ′ ⟩ π = E θ ∼ π [ ⟨ g D ( θ ) , g D ′ ( θ ) ⟩ ] . The inner product induces the L π -norm on g D as ∥ g D ∥ π := √ E θ ∼ π ∥ g D ( θ ) ∥ 2 . We use L π -norm to measure the gradient differences under certain D .

Definition 4.4 (Delta Error of an aggregation rule Aggr ( · ) ) . We define the following squared error term to measure the closeness between ̂ g Aggr and g a under ̂ D t a (distribution at time step t ), i.e.,

Delta errors ∆ 2 AT and ∆ 2 GP measure the closesness of g GP , ̂ g a from g a in ̂ D a at each iteration.

Theorem 4.5 (Error Analysis of GP) . When the model dimension m →∞ , for an epoch t , we have an approximation of the error difference ∆ 2 AT -∆ 2 GP as follows

¯ τ 2 = E π [ τ 2 ] ∈ [0 , 1] , where τ ( θ ) is the sin( · ) value of the angle between ̂ g n and g a -̂ g n .

Theorem 4.5 shows ∆ 2 GP is generally smaller than ∆ 2 AT for a large model dimension during each iteration, as is the case for the models in our evaluation, with β = 0 . 5 , since β (1 -β ) > β 2 (0 . 75 > 0 . 25) and the small value of ¯ τ in practice (see Interpretation of Theorem A.2 in Appendix A, where we show the order of difference is between 1 e -8 and 1 e -12 ). Thus, GP achieves better robust accuracy than AT by achieving a smaller delta error; GP also obtains good clean accuracy by combining parts of the model updates from the clean distribution ̂ D n . Further, we provide an error analysis of a single gradient step in Theorem A.1 and convergence analysis in Theorem A.2, showing that a smaller Delta error results in better convergence. The full proof of all theorems is in Appendix A.

We outline the AT-GP method in Algorithm 2 and it can be extended to the multiple-norm scenario. The overhead of this algorithm comes from natural training and GP operation. Their costs are small, and we discuss this more in Section 5.2. Combining logit pairing and gradient projection methods, we provide the RAMP framework which is similar to Algorithm 2, except that we replace line 4 of Algorithm 2 as Algorithm 1 line 3-9.

5 Experiment

Datasets, baselines, and models. CIFAR-10 [Krizhevsky et al., 2009] includes 60 K images with 50 K and 10 K images for training and testing respectively. ImageNet has ≈ 14 . 2 Mimages and 1 K classes, containing ≈ 1 . 3 M training, 50 K validation, and 100 K test images [Russakovsky et al.,

0

20

40

60

80

100

120

Epochs

0.700

0.725

0.750

0.775

0.800

0.825

0.850

0.875

0.900

Natural Acc

AT

AT-GP

AT-GP-pre

AT-pre

(a) Clean Accuracy

0

20

40

60

80

100

120

Epochs

0.20

0.25

0.30

0.35

0.40

0.45

0.50

Robust Acc

AT

AT-GP

AT-GP-pre

AT-pre

(b) Robust Accuracy: PGD-20

Figure 3: l ∞ AT-GP with APGD [Croce and Hein, 2020] improves robustness against l ∞ AutoAttack [Croce and Hein, 2020] with ϵ = 8 255 . RN-18 l ∞ -GP uses AT-GP ; RN-18 l ∞ -GP-pre pre-trains 40 epochs on ̂ D n before AT-GP is applied.

2015]. We compare RAMP with following baselines: 1. SAT [Madaan et al., 2021]: randomly sample one of the l 1 , l 2 , l ∞ attacks. 2. AVG [Tramer and Boneh, 2019]: take the average of l 1 , l 2 , l ∞ examples. 3. MAX [Tramer and Boneh, 2019]: take the worst of l 1 , l 2 , l ∞ attacks. 4. MSD [Maini et al., 2020]: find the worst-case examples over l 1 , l 2 , l ∞ steepest descent directions during each step of inner maximization. 5. E-AT [Croce and Hein, 2022]: randomly sample between l 1 , l ∞ attacks. For models, we use PreAct-ResNet-18, ResNet-50, WideResNet-34-20, and WideResNet-70-16 for CIFAR-10, as well as ResNet-50 and XCiT-S transformer for ImageNet.

Implementations and Evaluation. For AT from scratch for CIFAR-10, we train PreAct ResNet18 [He et al., 2016] with a lr = 0 . 05 for 70 epochs and 0 . 005 for 10 more epochs. We set λ = 2 , β = 0 . 5 for training from scratch, and λ = 0 . 5 for robust fine-tuning. For all methods, we use 10 steps for the inner maximization in AT. For ImageNet, we perform 1 epoch of fine-tuning and use a learning rate lr = 0 . 005 , λ = 0 . 5 for ResNet-50 and lr = 1 e -4 , λ = 0 . 5 for XCiT-S models. We reduce the rate by a factor of 10 every 1 3 of the training epoch and set the weight decay to 1 e -4 . We use APGD with 5 steps for l ∞ and l 2 , 15 steps for l 1 . Settings are similar to [Croce and Hein, 2022]. We use the standard values of ϵ 1 = 12 , ϵ 2 = 0 . 5 , ϵ ∞ = 8 255 for CIFAR-10 and ϵ 1 = 255 , ϵ 2 = 2 , ϵ ∞ = 4 255 for ImageNet. We focus on l ∞ -AT models for fine-tuning, as Croce and Hein [2022] shows their higher union accuracy for the ϵ values in our evaluation. We report the clean accuracy, robust accuracy against { l 1 , l 2 , l ∞ } attacks, union accuracy, universal robustness against common corruptions and unseen adversaries, as well as runtime for RAMP . The robust accuracy is evaluated using Autoattack [Croce and Hein, 2020]. More implementation details are in Appendix B.

5.1 Main Results

Robust fine-tuning. In Table 2, we apply RAMP to larger models and datasets (ImageNet). However, the implementation of other baselines is not publicly available and Croce and Hein [2022] do not report other baseline results except E-AT on larger models and datasets, so we only compare against E-AT in Table 2, which shows RAMP consistently obtains better union accuracy and accuracyrobustness tradeoff than E-AT. We observe that RAMP improves the performance more as the model becomes larger. We obtain the SOTA union accuracy of 53 . 3% on CIFAR-10 and 29 . 1% on ImageNet.

RAMP with varying ϵ 1 , ϵ 2 , ϵ ∞ values. We provide results with 1. ( ϵ 1 = 12 , ϵ 2 = 0 . 5 , ϵ ∞ = 2 255 ) where ϵ ∞ size is small and 2. ( ϵ 1 = 12 , ϵ 2 = 1 . 5 , ϵ ∞ = 8 255 ) where ϵ 2 size is large, using PreAct ResNet-18 model for CIFAR-10 dataset: these cases have different tradeoff pair compared to

Figure 1. The pair identified using our heuristic are l 1 -l 2 and l 2 -l ∞ . In Table 1, we observe that RAMP consistently outperforms E-AT and MAX with significant margins in union accuracy, when training from scratch and performing robust fine-tuning. In Table 1, when l 2 is the bottleneck, E-AT obtains a lower union accuracy as it does not leverage l 2 examples. Similar observations are made across various epsilon values, with RAMP consistently outperforming other baselines, as detailed in Appendix B.4. Appendix B includes more training details/results, and ablation studies. Results for applying the trades loss to RAMP outperforming E-AT are detailed in Appendix B.6. Appendix B.7 presents robust fine-tuning using ResNet-18, where RAMP achieves the highest union accuracy.

Adversarial training from random initialization. Table 3 presents the results of AT from random initialization on CIFAR-10 with PreAct ResNet-18. RAMP has the highest union accuracy with good clean accuracy, which indicates that RAMP can mitigate the tradeoffs among perturbations and robustness/accuracy in this setting. The results for all baselines are from Croce and Hein [2022].

Table 3: RN-18 model trained from random initialization on CIFAR-10 over 5 trials: RAMP achieves the best union robustness and good clean accuracy compared with other baselines. Baseline results are from Croce and Hein [2022].

Universal Robustness. In Table 4, we report average accuracy against common corruptions and union accuracy against unseen adversaries from Laidlaw et al. [2020] (implementation details are in Appendix B.3). We compare against l p pretrained models, E-AT, MAX, winninghand [Diffenderfer et al., 2021] (a SOTA method for natural corruptions) using WideResNet-28-10 architecture on the CIFAR-10 dataset. Compared to E-AT and MAX, RAMP achieves 4% higher accuracy for common corruptions with five severity levels and 2 -4% better union accuracy against multiple unseen adversaries. Winninghand has high corruption robustness but no adversarial robustness. The results show that RAMP obtains a better robustness and accuracy tradeoff with stronger universal robustness. In Appendix B.3, we evaluate on ResNet-18 to support this fact further.

5.2 Ablation Study and Discussion

Sensitivities of λ . We perform experiments with different λ values in [0 . 1 , 0 . 5 , 1 . 0 , 1 . 5 , 2 , 3 , 4 , 5] for robust fine-tuning and [1 . 5 , 2 , 3 , 4 , 5 , 6] for AT from scratch using PreAct-ResNet-18 model for CIFAR-10 dataset. In Figure 4, we observe a decreased clean accuracy when λ becomes larger. We

λ

Acc

35

45

55

65

75

85

0

1

2

3

4

5

Clean

Linf

L2

L1

Union

(a)

λ

: Robust fine-tuning.

λ

Acc

40

45

50

55

60

65

70

75

80

85

2

3

4

5

6

Clean

Linf

L2

L1

Union

(b)

λ

: Train from scratch.

β

Acc

40

45

50

55

60

65

70

75

80

85

0.2

0.4

0.6

0.8

Clean

Linf

L2

L1

Union

(c)

β

: Train from scratch.

pick λ = 2 . 0 for training from scratch (Figure 4a) and λ = 0 . 5 for robust fine-tuning (Figure 4b) in our main experiments, as these values of λ yield both good clean and union accuracy.

Choices of β . Figure 4c shows the performance of RAMP with varying β values on CIFAR-10 ResNet-18 experiments. We pick β = 0 . 5 for combining natural training and AT via GP, which achieves comparatively good robustness and clean accuracy. This choice is also based on Theorem 4.5 when β (2 -β ) has the largest difference from β 2 (0.75 vs 0.25).

Fine-tune l p AT models with RAMP. Table 5 shows the robust fine-tuning results using RAMP with l ∞ -AT ( q = ∞ , r = 1 ), l 1 -AT ( q = 1 , r = ∞ ), l 2 -AT ( q = ∞ , r = 1 ) RN-18 models for CIFAR-10 dataset. For l ∞ -l 1 tradeoffs, RAMP on l ∞ -AT pre-trained model achieves the best union accuracy.

Computational analysis and Limitations. The extra training costs of AT-GP are small, e.g. for each epoch on ResNet-18, the extra NT takes 6 seconds and the

standard AT takes 78 seconds using a single NVIDIA A100 GPU, and the GP operation only takes 0 . 04 seconds on average. RAMP is more expensive than E-AT and less expensive than MAX. We have a complete runtime analysis in Appendix B.2. We notice occasional drops in clean accuracy during fine-tuning with RAMP . In some cases, union accuracy improves slightly but clean accuracy and single l p robustness reduce. Further, we find no negative societal impact from this work.

6 Conclusion

We introduce RAMP , a framework enhancing multiple-norm robustness and achieving superior universal robustness against corruptions and perturbations by addressing tradeoffs among l p perturbations and accuracy/robustness. We apply a new logit pairing loss and use gradient projection to obtain SOTA union accuracy with favorable accuracy/robustness tradeoffs against common corruptions and other unseen adversaries. Results demonstrate that RAMP surpasses SOTA methods in union accuracy across model architectures on CIFAR-10 and ImageNet.

Acknowledgments

This work was supported in part by NSF Grants No. CCF-2238079, CCF-2316233, CNS-2148583. We would like to thank Jacky Yibo Zhang for the helpful discussions and advice on the proof. Also, we thank anonymous reviewers for their valuable feedback on the paper.

References

Kumail Alhamoud, Hasan Abed Al Kader Hammoud, Motasem Alfarra, and Bernard Ghanem. Generalizability of adversarial robustness under distribution shifts. Transactions on Machine Learning Research , 2023. ISSN 2835-8856. URL https://openreview.net/forum?id= XNFo3dQiCJ . Featured Certification.

Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In International conference on machine learning , pages 284-293. PMLR, 2018.

Debangshu Banerjee and Gagandeep Singh. Relational dnn verification with cross executional bound refinement. arXiv preprint arXiv:2405.10143 , 2024.

Debangshu Banerjee, Changming Xu, and Gagandeep Singh. Input-relational verification of deep neural networks. Proceedings of the ACM on Programming Languages , 8(PLDI):1-27, 2024.

Philipp Benz, Chaoning Zhang, and In So Kweon. Batch normalization increases adversarial vulnerability and decreases adversarial transferability: A non-robust feature perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7818-7827, 2021.

Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. Advances in neural information processing systems , 32, 2019.

Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning , pages 2206-2216. PMLR, 2020.

Francesco Croce and Matthias Hein. Adversarial robustness against multiple and single l _ p -threat models via quick fine-tuning of robust classifiers. In International Conference on Machine Learning , pages 4436-4454. PMLR, 2022.

Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: a standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670 , 2020.

Francesco Croce, Maksym Andriushchenko, Naman D Singh, Nicolas Flammarion, and Matthias Hein. Sparse-rs: a versatile framework for query-efficient sparse black-box adversarial attacks. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 6437-6445, 2022.

Edoardo Debenedetti and Carmela Troncoso-EPFL. Adversarially robust vision transformers, 2022.

James Diffenderfer, Brian Bartoldson, Shreya Chaganti, Jize Zhang, and Bhavya Kailkhura. A winning hand: Compressing deep networks can improve out-of-distribution robustness. Advances in neural information processing systems , 34:664-676, 2021.

Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and a translation suffice: Fooling cnns with simple transformations. 2017.

Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint arXiv:1807.10272 , 2018.

Logan Engstrom, Andrew Ilyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library), 2019. URL https://github.com/MadryLab/robustness .

Sanmi Koyejo Enyi Jiang, Yibo Jacky Zhang. Principled federated domain adaptation: Gradient projection and auto-weighting. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview.net/forum?id=6J3ehSUrMU .

Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1625-1634, 2018.

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 , 2014.

Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593 , 2020.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 , 2019.

Enyi Jiang. Federated domain adaptation for healthcare, 2023.

Enyi Jiang, Yibo Jacky Zhang, and Oluwasanmi Koyejo. Federated domain adaptation via gradient projection. arXiv preprint arXiv:2302.05049 , 2023.

Daniel Kang, Yi Sun, Tom Brown, Dan Hendrycks, and Jacob Steinhardt. Transfer of adversarial robustness between perturbation types. arXiv preprint arXiv:1905.01034 , 2019.

Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing. arXiv preprint arXiv:1803.06373 , 2018.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security , pages 99-112. Chapman and Hall/CRC, 2018.

Cassidy Laidlaw, Sahil Singla, and Soheil Feizi. Perceptual adversarial robustness: Defense against unseen threat models. arXiv preprint arXiv:2006.12655 , 2020.

Aishan Liu, Shiyu Tang, Xianglong Liu, Xinyun Chen, Lei Huang, Zhuozhuo Tu, Dawn Song, and Dacheng Tao. Towards defending multiple adversarial perturbations via gated batch normalization. arXiv preprint arXiv:2012.01654 , 2020.

Divyam Madaan, Jinwoo Shin, and Sung Ju Hwang. Learning to generate noise for multi-attack robustness. In International Conference on Machine Learning , pages 7279-7289. PMLR, 2021.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017.

Pratyush Maini, Eric Wong, and Zico Kolter. Adversarial robustness against the union of multiple perturbation models. In International Conference on Machine Learning , pages 6640-6650. PMLR, 2020.

Pratyush Maini, Xinyun Chen, Bo Li, and Dawn Song. Perturbation type categorization for multiple adversarial perturbation robustness. In Uncertainty in Artificial Intelligence , pages 1317-1327. PMLR, 2022.

Mohammad Mehrabi, Adel Javanmard, Ryan A Rossi, Anup Rao, and Tung Mai. Fundamental tradeoffs in distributionally adversarial training. In International Conference on Machine Learning , pages 7544-7554. PMLR, 2021.

Mazda Moayeri, Kiarash Banihashem, and Soheil Feizi. Explicit tradeoffs between adversarial and natural distributional robustness. Advances in Neural Information Processing Systems , 35: 38761-38774, 2022.

Jay Nandy, Wynne Hsu, and Mong Li Lee. Approximate manifold defense against multiple adversarial perturbations. In 2020 International Joint Conference on Neural Networks (IJCNN) , pages 1-8. IEEE, 2020.

ShengYun Peng, Weilin Xu, Cory Cornelius, Matthew Hull, Kevin Li, Rahul Duggal, Mansi Phute, Jason Martin, and Duen Horng Chau. Robust principles: Architectural design principles for adversarially robust cnns. arXiv preprint arXiv:2308.16258 , 2023.

Rahul Rade and Seyed-Mohsen Moosavi-Dezfooli. Reducing excessive margin to achieve a better accuracy vs. robustness trade-off. In International Conference on Learning Representations , 2021.

Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John Duchi, and Percy Liang. Understanding and mitigating the tradeoff between robustness and accuracy. arXiv preprint arXiv:2002.10716 , 2020.

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision , 115:211-252, 2015.

Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the first adversarially robust neural network model on mnist. arXiv preprint arXiv:1805.09190 , 2018.

Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K Reiter. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the 2016 acm sigsac conference on computer and communications security , pages 1528-1540, 2016.

Aman Sinha, Hongseok Namkoong, and John Duchi. Certifiable distributional robustness with principled adversarial training. In International Conference on Learning Representations , 2018. URL https://openreview.net/forum?id=Hk6kPgZA-.

Dawn Song, Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, and Tadayoshi Kohno. Physical adversarial examples for object detectors. In 12th USENIX workshop on offensive technologies (WOOT 18) , 2018.

Florian Tramer and Dan Boneh. Adversarial training and robustness for multiple perturbations. Advances in neural information processing systems , 32, 2019.

Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07204 , 2017.

Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR , 2020.

Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 36246-36263. PMLR, 23-29 Jul 2023. URL https://proceedings.mlr. press/v202/wang23ad.html .

Eric Wong and J Zico Kolter. Learning perturbation sets for robust machine learning. arXiv preprint arXiv:2007.08450 , 2020.

Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust generalization. Advances in Neural Information Processing Systems , 33:2958-2969, 2020.

Jiancong Xiao, Zeyu Qin, Yanbo Fan, Baoyuan Wu, Jue Wang, and Zhi-Quan Luo. Adaptive smoothness-weighted adversarial training for multiple perturbations with its stability analysis. arXiv preprint arXiv:2210.00557 , 2022.

Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L Yuille, and Quoc V Le. Adversarial examples improve image recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 819-828, 2020.

Kaidi Xu, Chenan Wang, Hao Cheng, Bhavya Kailkhura, Xue Lin, and Ryan Goldhahn. Mixture of robust experts (more): A robust denoising method towards multiple perturbations. arXiv preprint arXiv:2104.10586 , 2021.

Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ R Salakhutdinov, and Kamalika Chaudhuri. A closer look at accuracy vs. robustness. Advances in neural information processing systems , 33:8588-8601, 2020.

Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Procedings of the British Machine Vision Conference 2016 . British Machine Vision Association, 2016.

Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning , pages 7472-7482. PMLR, 2019.

Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry-aware instance-reweighted adversarial training. In International Conference on Learning Representations , 2021. URL https://openreview.net/forum?id=iAX0l6Cz8ub .

A Proof of Theorems

A.1 Proof of Theorem A.2

We first show what happens during one step of optimization, where we highlight the importance of analyzing delta error.

Theorem A.1. Consider model parameter θ ∼ π and an aggregation rule Aggr ( · ) with step size µ > 0 . Define the updated parameter as

Assuming the gradient ∇L ( θ ) is γ -Lipschitz in θ for any input, and let the step size µ ≤ 1 γ , we have

Proof. The proof is the same as Theorem A.1 in [Enyi Jiang, 2024].

Theorem A.2 (Convergence of Aggr ( · ) ) . For any probability measure π over the parameter space, and an aggregation rule Aggr ( · ) with step size µ > 0 . We update the parameter for T steps by θ t +1 := θ t -µ ̂ g Aggr ( θ t ) . Assume the gradient ∇L ( θ ) and ̂ g Aggr ( θ ) are γ 2 -Lipschitz in θ such that θ t → ̂ θ Aggr . ∆ Aggr_max is the Delta error at time t ′ when ∥ ̂ g Aggr ( ̂ θ Aggr ) - ∇L D t ′ a ( ̂ θ Aggr ) ∥ 2 is maximized. Then, given step size µ ≤ 1 γ and a small enough ϵ > 0 , with probability at least 1 -δ we have

where C ϵ = E ̂ D t ′ a [1 /π ( B ϵ ( ̂ θ Aggr ))] 2 and B ϵ ( ̂ θ Aggr ) ⊂ R m is the ball with radius ϵ centered at ̂ θ Aggr . The C ϵ measures how well π covers where the optimization goes.

Proof. Denote random function f : R m → R

where the randomness comes from ̂ D a . Note that ̂ f is γ -Lipschitz by assumption. Now we consider B ϵ ( ̂ θ Aggr ) ⊂ R m , i.e., the ball with radius ϵ centered at ̂ θ Aggr . Then, by γ -Lipschitzness we have

Therefore,

Taking expectation w.r.t. ̂ D a on both sides, we have

By Markov's inequality, with probability at least 1 -δ we have a sampled dataset ̂ D a such that

Conditioned on such event, we proceed on to the optimization part.

Note that Theorem A.1 characterizes how the optimization works for one gradient update. We denote D t a as the data distribution D a at time step t . Therefore, for any time step t = 0 , . . . , T -1 , we can apply Theorem A.1 which only requires the Lipschitz assumption:

We notice that D t a changes based on θ s of different time steps. On both sides, to sum over t = 0 , . . . , T -1 , we first consider two terms:

To compare L D t a ( θ t ) and L D t -1 a ( θ t ) , since D t a optimizes one more step than D t -1 a , we assume L D t a ( θ t ) ≤ L D t -1 a ( θ t ) for ∀ t . Therefore, we have:

Summing up all time steps,

Dividing both sides by T , and with regular algebraic manipulation we derive

Note that we assume the loss function L D ( θ ) := E ( x,y ) ∼D L ( f ( x ) , y ) , is non-negative. Thus, we have

Note that we assume given ̂ D a we have θ t → ̂ θ Aggr . Therefore, for any ϵ > 0 there exist T ϵ such that

This implies that ∀ t > T ϵ :

Moreover, (11) also implies ∀ t 1 , t 2 > T ϵ :

Now, let's get back to (10). For ∀ T > T ϵ we have

Equation 14 bounds the left hand side with the maximum ∥ ̂ g Aggr ( ̂ θ Aggr ) -∇L D t a ( ̂ θ Aggr ) ∥ 2 one can get during the optimization steps. Here, we assume at time t ′ , the largest value is attained. We denote ∆ 2 Aggr_max as the delta error at time step t ′ .

Then, we can continue with what we have done at the beginning of the proof of this theorem:

Therefore, combining the above we finally have: for ∀ T > T ϵ with probability at least 1 -δ ,

To complete the proof, let us investigate the left-hand side.

Combining (15) and (16), we finally have

which completes the proof.

A.2 Proof of Theorem 4.5

To prove Theorem 4.5, we first use the following definitions and lemmas from [Enyi Jiang, 2024], to get the delta errors of Gradient Projection (GP) and standard adversarial training (AT):

Definition A.3 (GP Aggregation) . Let β ∈ [0 , 1] be the weight that balances between ̂ g a and ̂ g n . The GP aggregation operation is

where Proj + ( ̂ g a | ̂ g n ) = max {⟨ ̂ g a , ̂ g n ⟩ , 0 } ̂ g n / ∥ ̂ g n ∥ 2 is the operation that projects ̂ g a to the positive direction of ̂ g n .

Definition A.4 (AT Aggregation) . The AT aggregation operation is

standard AT only leverages the gradient update on ̂ D a .

Lemma A.5 (Delta Error of GP) . Given distributions ̂ D a , D a and ̂ D n , as well as the model updates ̂ g a , g a , ̂ g n on these distributions per epoch, we have ∆ 2 GP as follows

In the above equation, m is the model dimension and ¯ τ 2 = E π [ τ 2 ] ∈ [0 , 1] where τ ( θ ) is the sin( · ) value of the angle between ̂ g n and g a -̂ g n . ∥ · ∥ π is the π -norm over the model parameter space.

Proof. The proof is the same as Theorem 4.4 in Enyi Jiang [2024].

Lemma A.6 (Delta Error of AT) . Given distributions ̂ D a , D a and ̂ D n , as well as the model updates ̂ g a , g a , ̂ g n on these distributions per epoch, we have ∆ 2 AT as follows

where ∥ · ∥ π is the π -norm over the model parameter space.

Then, we prove Theorem 4.5.

Theorem A.7 (Error Analysis of GP) . When the model dimension is large ( m →∞ ) at time step t , we have

¯ τ 2 = E π [ τ 2 ] ∈ [0 , 1] where τ is the sin( · ) value of the angle between ̂ g n and g a -̂ g n , ∥ · ∥ π is the π -norm over the model parameter space.

When m →∞ , we have a simplified version of the error difference as follows

Interpretation. When β = 0 . 5 , we can usually show ∆ 2 AT > ∆ 2 GP , because β (2 -β ) > β 2 ¯ τ 2 (0 . 75 > 0 . 25) for the coefficients of two terms. We estimate the actual values of terms E ̂ D a t ∥ g a -̂ g a ∥ 2 π (variance), ∥ g a -̂ g n ∥ 2 π (bias), and ¯ τ using the estimation methods in Enyi Jiang [2024]. Table 6 displays the values of those terms as well as the error differences on ResNet18 experiments at epoch 5 , 10 , 15 , 20 , 60 . We plot the changing of these terms on the ResNet18 experiment in Figure 5. The order of difference is always positive and usually smaller than 1 e -08 and approaches the order of 1 e -12 in the end.

B Additional Experiment Information

In this section, we provide more training details, additional experiment results on the universal robustness of RAMP to common corruptions and unseen adversaries, runtime analysis of RAMP, additional ablation studies on different logit pairing losses, and AT from random initialization results on CIFAR-10 using WideResNet-28-10.

B.1 More Training Details

We set the batch size to 128 for the experiments on ResNet-18 and WideResNet-28-10 architectures. We use an SGD optimizer with 0 . 9 momentum and 5 e -4 weight decay. For other experiments on ImageNet, we use a batch size of 64 to fit into the GPU memory for larger models. For all training procedures, we select the last checkpoint for the comparison. When the pre-trained model was originally trained with extra data beyond the CIFAR-10 dataset, similar to Croce and Hein [2022], we use the extra 500 k images introduced by Carmon et al. [2019] for fine-tuning, and each batch contains the same amount of standard and extra images. An epoch is completed when the whole standard training set has been used.

0

20

40

60

80

100

120

Epoches

0.0

0.2

0.4

0.6

0.8

1.0

1e

8

Variances

t

a

|

ga

ga

|

2

0

20

40

60

80

100

120

Epoches

0.00000

0.00002

0.00004

0.00006

0.00008

0.00010

|

ga

gn

|

2

0

20

40

60

80

100

120

Epoches

0.0025

0.0050

0.0075

0.0100

0.0125

0.0150

0.0175

0.0200

0

20

40

60

80

100

120

Epoches

0.0

0.2

0.4

0.6

0.8

1.0

1e

10

Error Differences

2

AT

2

GP

B.2 Runtime Analysis of RAMP

We present runtime analysis results demonstrating the fact that RAMP is more expensive than E-AT and less expensive than MAX in Table 7. These results, recorded in seconds per epoch, were obtained using a single A100 40GB GPU. RAMP consistently supports that fact in all experiments.

B.3 Additional Results on RAMP Generalizing to Common Corruptions and Unseen Adversaries for Universal Robustness

In this section, we show RAMP can generalize better to other corruptions and unseen adversaries on union accuracy for stronger universal robustness.

Implementations. For the l 0 attack, we use Croce et al. [2022] with an epsilon of 9 pixels and 5 k query points. For common corruptions, we directly use the implementation of RobustBench [Croce et al., 2020] for evaluation across 5 severity levels on all corruption types used in Hendrycks and Dietterich [2019]. For other unseen adversaries, we follow the implementation of Laidlaw et al. [2020], where we set eps = 12 for the fog attack, eps = 0 . 5 for the snow attack, eps = 60 for the gabor attack, eps = 0 . 125 for the elastic attack, and eps = 0 . 125 for the jpeglinf attack with 100 iterations. For ResNet-18 experiments, we do not compare with Winninghand [Diffenderfer et al.,

2021] since it uses a Wide-ResNet architecture. Also, we select the strongest baselines (E-AT and MAX) from the Wide-ResNet experiment results to compare for ResNet-18 experiments on universal robustness.

Results. For the ResNet-18 training from scratch experiment on CIFAR-10, in Table 8 and 9, we also show RAMP generally outperforms by 0 . 5% on common corruptions and 7% on union accuracy against unseen adversaries compared with E-AT.

Models common corruptions

B.4 Additional Experiments with Different Epsilon Values

In this section, we provide additional results with different ϵ 1 , ϵ 2 , ϵ ∞ values. We select ϵ ∞ = [ 2 255 , 4 255 , 12 255 , 16 255 ] , ϵ 1 = [6 , 9 , 12 , 15] , and ϵ 2 = [0 . 25 , 0 . 75 , 1 . 0 , 1 . 5] . We provide additional RAMP results compared with related baselines with training from scratch and performing robust fine-tuning in Section B.4.1 and Section B.4.2, respectively. We observe that RAMP can surpass E-AT with significant margins as well as a better accuracy-robustness tradeoff for both training from scratch and robust fine-tuning with λ = 2 . 0 for training from scratch and λ = 0 . 5 for robust fine-tuning in most cases.

B.4.1 Additional Results with Training from Scratch

Changing l ∞ perturbations with ϵ ∞ = [ 2 255 , 4 255 , 12 255 , 16 255 ] . Table 10 and Table 11 show that RAMP consistently outperforms E-AT [Croce and Hein, 2022] on union accuracy when training from scratch.

Changing l 1 perturbations with ϵ 1 = [6 , 9 , 12 , 15] . Table 12 and Table 13 show that RAMP consistently outperforms E-AT [Croce and Hein, 2022] on union accuracy when training from scratch.

Changing l 2 perturbations with ϵ 2 = [0 . 25 , 0 . 75 , 1 . 0 , 1 . 5] . Table 14 and Table 15 show that RAMP consistently outperforms E-AT [Croce and Hein, 2022] on union accuracy when training from scratch.

B.4.2 Additional Results with Robust Fine-tuning

Changing l ∞ perturbations with ϵ ∞ = [ 2 255 , 4 255 , 12 255 , 16 255 ] . Table 16 and Table 17 show that RAMP consistently outperforms E-AT [Croce and Hein, 2022] on union accuracy when performing robust fine-tuning.

Changing l 1 perturbations with ϵ 1 = [6 , 9 , 12 , 15] . Table 12 and Table 13 show that RAMP consistently outperforms E-AT [Croce and Hein, 2022] on union accuracy when performing robust fine-tuning.

Changing l 2 perturbations with ϵ 2 = [0 . 25 , 0 . 75 , 1 . 0 , 1 . 5] . Table 14 and Table 15 show that RAMP consistently outperforms E-AT [Croce and Hein, 2022] on union accuracy when performing robust fine-tuning.

Table 19: ( ϵ ∞ = 8 255 , ϵ 1 = 15 , ϵ 2 = 0 . 5) and ( ϵ ∞ = 8 255 , ϵ 1 = 18 , ϵ 2 = 0 . 5) with robust finetuning.

Table 21: ( ϵ ∞ = 8 255 , ϵ 1 = 12 , ϵ 2 = 1 . 0 ) and ( ϵ ∞ = 8 255 , ϵ 1 = 12 , ϵ 2 = 1 . 5 ) with robust finetuning.

B.5 Different Logit Pairing Methods

In this section, we test RAMP with robust fine-tuning using two more different logit pairing losses: (1) Mean Squared Error Loss ( L mse ) (Eq. 17), (2) Cosine-Similarity Loss ( L cos ) (Eq. 18). We replace the KL loss we used in the paper using the following losses. We use the same lambda value λ = 1 . 5 for both cases.

B.6 AT from Scratch Using WideResNet-28-10

Implementations. We use a cyclic learning rate with a maximum rate of 0 . 1 for 30 epochs and adopt the outer minimization trades loss from Zhang et al. [2019] with the default hyperparameters, same as Croce and Hein [2022]; also, we set λ = 2 . 0 and β = 0 . 5 for training RAMP . Additionally, we use the WideResNet-28-10 architecture same as Zagoruyko and Komodakis [2016] for our reimplementations on CIFAR-10.

Results. Since the implementation of experiments on WideResNet-28-10 in Croce and Hein [2022] paper is not public at present, we report our implementation results on E-AT, where our results show that RAMP outperforms E-AT in union accuracy with a significant margin, as shown in Table 23. Also, we experiment with using the trade loss ( RAMP w trades ) for the outer minimization, we observe that RAMP w trades achieves a better union accuracy at the loss of some clean accuracy.

B.7 Robust Fine-tuning Using PreAct-ResNet-18

Implementations. For robust fine-tuning with ResNet-18, we perform 3 epochs on CIFAR-10. We set the learning rate as 0 . 05 for PreAct-ResNet-18 and 0 . 01 for other models. We set λ = 0 . 5 in this case. Also, we reduce the learning rate by a factor of 10 after completing each epoch.

Result. Table 24 shows the robust fine-tuning results using PreAct ResNet-18 model on the CIFAR-10 dataset with different methods. The results for all baselines are directly from the E-AT paper [Croce and Hein, 2022] where the authors reimplemented other baselines (e.g., MSD, MAX) to achieve better union accuracy than presented in the original works. RAMP surpasses all other methods on union accuracy.

B.8 Robust Fine-tuning with More Epochs

In Table 25, we apply robust fine-tuning on the PreAct ResNet-18 model for the CIFAR-10 dataset with 5 , 7 , 10 , 15 epochs, and compare it with E-AT. RAMP consistently outperforms the baseline on union accuracy, with a larger improvement when we increase the number of epochs.

C Additional Visualization Results

In this section, we provide additional t-SNE visualizations of the multiple-norm tradeoff and robust fine-tuning procedures using different methods.

C.1 Pre-trained l 1 , l 2 , l ∞ AT models

Figure 6 shows the robust accuracy of l 1 , l 2 , l ∞ AT models against their respect l 1 , l 2 , l ∞ perturbations, on CIFAR-10 using PreAct-ResNet-18 architecture. Similar to Figure ?? , l ∞ -AT model has a low l 1 robustness and vice versa. In this common choice of epsilons, we further confirm that l ∞ -l 1 is the key trade-off pair.

C.2 Robust Fine-tuning for all Epochs

We provide the complete visualizations of robust fine-tuning for 3 epochs on CIFAR-10 using l 1 examples, E-AT, and RAMP . Rows in l 1 fine-tuning (Figure 7), E-AT fine-tuning (Figure 8), and RAMP fine-tuning (Figure 9) show the robust accuracy against l ∞ , l 1 , l 2 attacks individually, of epoch 0 , 1 , 2 , 3 , respectively. We observe that throughout the procedure, RAMP manages to maintain more l ∞ robustness during the fine-tuning with more points colored in cyan, in comparison with two other methods. This visualization confirms that after we identify a l p -l r ( p = r ) key tradeoff pair, RAMP successfully preserves more l p robustness when training with some l r examples via enforcing union predictions with the logit pairing loss.

Linf

Linf - AT

L1 - AT

L2 - AT

̸

Linf

L1

L2

Linf

L1

L2

Linf

L1

L2

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction present the claims made in the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of our work in the discussion section of the paper.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We state the assumptions along with theoretical results in the theory part of the paper. The proofs are in the supplemental material.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide the implementations of our experiments in both main paper and supplementary materials.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide scripts to reproduce all experimental results for the new proposed method.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The full details are provided with the code and in appendix.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The results are accompanied by confidence intervals.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide the information on the computer resources in the discussion section of the paper.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We conduct the research with the NeurIPS Code of Ethics.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We mention the societal impacts of the work in the discussion section of the paper.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cite the original paper that produced the code package or dataset.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We provide README and scripts to run for the new asset we introduce. They are well documented.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not include crowdsourcing or research involving human subjects.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of training Machine Learning (ML) models that show good robustness against multiple l_p attacks and accuracy. This problem has already been studied in previous work [33,6], but the papers discuss the problem from the lens of distribution shift. This point of view represents the starting point for proposing a new adversarial training (AT) algorithm called RAMP based on two building blocks: (i) a new logit pairing loss that allows a model robust against a l_p norm to exhibit high robustness w.r.t. another l_q norm (with q \\neq p); (ii) a new model parameter update based on the combinations of model updates from natural training and adversarial training to train a model that observes a better accuracy/union robustness tradeoff that SOTA AT models. A deep theoretical analysis shows that, for large models and a reasonable choice of parameter values, the novel parameter update allows the adversarial training algorithm to converge faster, training a model with higher robustness and accuracy. The extensive experimental evaluation highlights that the trained large models on ImageNet and Cifar-10 exhibit a better accuracy-union robustness tradeoff than SOTA AT proposals [6].\n\nSTRENGTHS:\n**Original AT algorithm to address the accuracy-union robustness trade-off**: the novel AT training addresses both the problem of improving the robustness on multiple l_p perturbations and improving the accuracy-robustness trade-off at the same time by exploiting a novel connection with natural training. Addressing the two problems at the same time constitutes the novelty of the work.\n\n**Theoretical analysis of the impact of gradient projection on adversarial robustness**: the deep theoretical analysis shows that integrating model parameter updates from natural training in the adversarial training updates allows the training to converge better for large models.\n\n**Comprehensive experimental settings considered**: the experimental evaluation is performed on full training and fine-tuning large models used by relevant competitors in the literature [6], on significant datasets (CIFAR-10 and ImageNet) and considering several combinations of perturbation values for l_inf, l_1, and l_2.\n\nWEAKNESSES:\n**Analysis of time per epoch is not opportunely deepened (addressed by the authors)**: the analysis regarding the computational time per epoch is limited since the results are shown for only one model (ResNet-18) on only one dataset (CIFAR-10), but seven other models are employed in the experimental evaluation. Since a lot of additional results have been included in the Appendix, I would have expected to see also more results supporting the statement ``` RAMP has around two times the runtime cost of E-AT and 2/3 cost of MAX``` (Section 5.2). For example, the authors may want to report the time per epoch on AT the ResNet-18 and the WRN-28-10 from scratch on CIFAR-10 (as reported in [6]) and on training another model on ImageNet.\n\n**Trade-off between robustness and accuracy seems to not be properly addressed**: Even though RAMP allows the model to obtain better union accuracy, the clean accuracy is in any case lower than the clean accuracy obtained by using the baseline E-AT [6]. As an example, the RAMP fine-tuned RN-50-$\\ell_\\infty$ shows a clean accuracy that is 3.8 points lower than the one obtained by the E-AT fine-tuned model and a gain in union accuracy that is one point (Table 1). Moreover, more than half of the results shown in the Appendix and some results in the main body show that the gain in the union accuracy is obtained at the expense of a comparable loss in the clean accuracy. If the trade-off between robustness and accuracy had been properly addressed, I would have expected to see a comparable clean accuracy and better union accuracy than the one obtained by E-AT. These results limit the novelty of RAMP since it seems to be slower than E-AT and only allows the model to obtain better union accuracy without properly addressing the accuracy/robustness trade-off. I suggest that the authors explain why these cases happen and how to improve also the accuracy of the obtained model.\n\nQUESTIONS:\n- Can the authors show more results supporting the fact that the time per epoch required by RAMP is partially longer than the time per epoch required by E-AT? (See the Weaknesses section for more details)\n\n- Is it possible to use RAMP to obtain an accuracy close to the one obtained by using E-AT and a better union accuracy? (See the Weaknesses section for more details)\n\n**Updates after authors' response**: The authors provided additional results to address the first question, effectively addressing the first weakness. After discussion, the answer to the second question is more convincing. RAMP seems to address the accuracy-robustness trade-off better than E-AT even by choosing small values of $\\lambda$, as the authors wrote. So, choosing a small value of $\\lambda$ helps preserve the accuracy while still outperforming the models trained with E-AT regarding union accuracy. This choice could be right in general. \n\nReasons for Not Giving a Higher Score: RAMP does not allow the user to obtain an impressive gain in union accuracy as E-AT did when the corresponding paper was published. However, it allows the model to obtain a few points more of union accuracy while preserving an accuracy similar to that of models trained with E-AT. I can say that this paper will have a moderate to high impact, but I am not convinced that it will have a high impact for sure.\n\nReasons for Not Giving a Lower Score: In my opinion, the work is solid from a theoretical and methodological point of view. The experimental results are more convincing after the discussion with the authors.\n\nLIMITATIONS:\nThe authors have explained the limitations in Section 5.2 in brief. However, the limitations should be addressed better:\n- The analysis of the computational cost should be supported with more experimental results.\n- The authors should better substantiate the sentence ``` We notice occasional drops in clean accuracy during fine-tuning with RAMP```. Indeed, the drop in accuracy is exhibited also when the model is trained from scratch (see Section B.2.1 in the Appendix). Moreover, the gain in the union accuracy is often comparable to the loss in clean accuracy. Thus, the accuracy/robustness trade-off seems not to be properly addressed.\n\nThis work has no negative societal and ethical impacts since it contributes to improving the robustness of AI systems in practice. The experimental and implementation details have been documented.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a framework called RAMP to boost the adversarial robustness against multiple $l_p$ perturbations. Compared with existing methods, RAMP can better retain the robustness against the original type of perturbations when fine-tuning the model. The authors provide theoretical and empirical justifications to demonstrate the effectiveness of RAMP.\n\nSTRENGTHS:\n++ The targeted problem is well-motivated and formulated. \n\n++ The experimental results are comprehensive with reliable evaluation metrics.\n\n++ Theoretical justifications can better explain the effectiveness of the proposed methods.\n\nWEAKNESSES:\nDespite the strength in the last section, I have the following concerns:\n\n1. Novelty in the algorithm: logit pairing is not a new idea. Actually, in Equation (1), if we replace $p_r$ with the probability of the clean input, the algorithm will become TRADES. Some findings, such as trade-offs between accuracy and robustness, are not new.\n\n2. Generality of the algorithm: the manuscript only studies the joint robustness among $l_1$, $l_2$ and $l_\\infty$ perturbations, especially the $l_1$ and $l_\\infty$ ones. This is too limited. The authors should consider more generic cases, such as $l_0$ bounded perturbations and common corruptions (Gaussian noise, color re-mapping e.t.c.). I believe this will make the algorithm more generally applicable.\n\n3. Novelty and precision of the theorem: the theoretical tools employed to analyze convergence in Theorem 4.5 are standard and not unique. In Theorem 4.6, if $\\beta = 0$, $\\widehat{g}\\_a$ should be similar to $g_{GP}$, so $\\Delta_{AT}$ should be very close to $\\Delta_{GP}$. Furthermore, the left hand side of the approximation should be close to $0$, can the author explain how the right hand side approximation reflects this? In addition, using $\\simeq$ is not rigorous, the authors should at least point out the order of the difference between the left hand side and the right hand side.\n\nQUESTIONS:\nThe weakness part points out the major questions I would ask. Below are additional relatively minor questions:\n\n1. Why don't use AutoAttack as the evaluation metric in Figure 2?\n\n2. In Figure 4, what is the difference between MAX and RAMP with $\\lambda = 0$?\n\nMy major concern of this paper is about the novelty, because both the motivation (logit pairing) and many parts of the analysis are standard. In addition, the algorithm is not generic, as it is only evaluated under $l_1$, $l_2$ and $l_\\infty$ bounded perturbations. Therefore, I cannot recommend acceptance at this point. I welcome the authors to address my concerns in the rebuttal.\n\nLIMITATIONS:\nThe limitations and the societal impact are briefly discussed in the end of the paper.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-05"}

comments: SUMMARY:
This work proposes a training framework RAMP, to boost the robustness against multiple $\ell_p$ perturbations, through connecting NT with AT via gradient projection.

STRENGTHS:
- Easy to follow and clear presentation
- Enough details
- Some theoretical analyses
- Outwardly, this is a good paper

WEAKNESSES:
My main concern is why the distributions of adversarial examples generated using $\ell_1$, $\ell_2$, and $\ell_\infty$ norms are distinct?
Is this due to inconsistent (unreasonable) radius settings across different $\ell_p$-norms?
What constitutes a reasonable radius setting for various norms?
These norms can be bounded by each other theoretically, what is the essential meaning to study this problem?

QUESTIONS:
ref weakness

LIMITATIONS:
As I mentioned before, in my opinion, I think this work needs to discuss more about the essential meaning of the problem.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-08

comments: SUMMARY:
The paper introduces a new addition to the adversarial training literature -- it proposes to augment standard adversarial training [Madry et al.] with a specific logits pairing loss and, the existing technique, of gradient project. 

The paper shows strong results on the union of lp-norm adversarial accuracy (over l-1, l-inf and l-2 using the standard CIFAR-10 dataset and standard perturbation radii).

STRENGTHS:
The paper describes its contributions and motivates why each technique is introduced. The paper includes an informal discussion to motivate the logits pairing loss and some supporting theoretical analysis.

The paper obtains strong results (against many relevant baselines, evaluated correctly with AutoAttack) -- which is rare for adversarial robustness submissions. The method also seems straightforward to implement. Happy to recommend acceptance, based mainly on the strong results.

WEAKNESSES:
The one main weakness is that Table 1 must be re-generated using all the test set points in CIFAR-10, not just 1000.

Presentation-wise, Figure 1 is very confusing; there is no need to plot t-SNEs of image when one wants to simply show classification labels. Please consider re-plotting this figure (with a line plot, histogram, etc.).

The writing, overall, is ok, but could also be improved: for example, lines 162-167, are approximately repeated from earlier on in the paper.

QUESTIONS:
See above.

LIMITATIONS:
The authors have adequately addressed the limitations of their work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-11