RAMP: Boosting Adversarial Robustness Against Multiple $l_p$ Perturbations for Universal Robustness
Authors: Enyi Jiang
Gagandeep Singh
Venue: NeurIPS 2024 poster (2024)
Keywords: Adversarial Robustness
Pre-training and Fine-tuning
Distribution Shifts
Primary area: safety_in_machine_learning
TL;DR: We design a logit pairing loss and connect natural training with adversarial training via gradient projection to improve the multi-norm robustness while maintaining good clean accuracy.

Abstract: Most existing works focus on improving robustness against adversarial attacks bounded by a single $l_p$ norm using adversarial training (AT). However, these AT models' multiple-norm robustness (union accuracy) is still low, which is crucial since in the real-world an adversary is not necessarily bounded by a single norm. The tradeoffs among robustness against multiple $l_p$ perturbations and accuracy/robustness make obtaining good union and clean accuracy challenging. We design a logit pairing loss to improve the union accuracy by analyzing the tradeoffs from the lens of distribution shifts. We connect natural training (NT) with AT via gradient projection, to incorporate useful information from NT into AT, where we empirically and theoretically show it moderates the accuracy/robustness tradeoff. We propose a novel training framework \textbf{RAMP}, to boost the robustness against multiple $l_p$ perturbations. \textbf{RAMP} can be easily adapted for robust fine-tuning and full AT. For robust fine-tuning, \textbf{RAMP} obtains a union accuracy up to $53.3\%$ on CIFAR-10, and $29.1\%$ on ImageNet. For training from scratch, \textbf{RAMP} achieves a union accuracy of $44.6\%$ and good clean accuracy of $81.2\%$ on ResNet-18 against AutoAttack on CIFAR-10. Beyond multi-norm robustness \textbf{RAMP}-trained models achieve superior \textit{universal robustness}, effectively generalizing against a range of unseen adversaries and natural corruptions.

RAMP: Boosting Adversarial Robustness Against Multiple Ip Perturbations for Universal Robustness
Enyi Jiang
Gagandeep Singh
Department of Computer Science University of Illinois Urbana-Chann Chbama-Champaign
Department of Computer Science University of Illinois Jrbana-Champart
Urbana, II. 61801 enyij22illinois edu
Urbana, 61801 ggnds@illinois.cm edu
Abstract
Most existing works focus on improving robustness against adversarial attacks bounded by a single ip norm using adversarial training (AT). However, these AT models' multiple-norm robustness (union accuracy) is still low, which is cru- cial since the real-world an adversary is not necessarily bounded by a single norm. The tradeoffs among robustness against multiple lp perturbations and accu- racy/robustness make obtaining good union and clean accuracy challenging. We design logit pairing loss to improve the union accuracy by analyzing the tradcoffss from the lens of distribution shifts. We connect natural training (NT) with AT via gradient projection, to incorporate useful information from NT into AT, where empirically and theoretically show it moderates the accuracy/robustnes tradeoff. We propose novel training framework RAMP,, to boost the robustness against multiple I, perturbations. RAMP can be easily adapted for robust fine-tuning and full AT. For robust finettunin, RAMP obtains union accuracy up 53.3% on CIFAR-10, and 29.1% on ImageNet. For training from scratch, RAMP achieves a union accuracy of 44.6% and good clean accuracy of 81.2% on ResNet-18 against AutoAttack on CIFAR-10. Beyond multi-norm robustness RA MPttrandd models achieve superior universal robustness, effectively generalizing against range of unseen ad versariies and natural corruptions.
Introduction
Though deep neural networks (DNNs) demonstrate superior performance various vision appli- cations, they vulnerable against adversarial examples [Goodfellow et al., 2014 Kurakin al.
2018]. Adversarial training (AT) [Tramèr et al. 2017 Madry et al.) [2077] which works by injecting
adversarial examples into training for enhanced robustness, currently the most popular defense.
However, most AT methods address only a single type of perturbation [Wang et al. 2020 Wu
al. 2020 Carmon al.] 2019 Gowal et al. 2020 Raghunathan al. 20200 Zhang al. 2021
Debenedetti and Troncoso EPFL 2022 Peng et al.
2023 Wang et al. 2023]. An l.x robust model
may not be robust against (p * x)) attacks. Also,
enhancing robustness against
perturbation
type can sometimes increase vulnerability to others
[Engstrom et al.) 2017 Schott et al. 2018 On
the contrary. training model be robust against multiple Lp perturbations is crucial as it reflects real-world scenarios [Sharif' et al. 2016 Eykholi et al. 2018 [oon et al. 2018 Athalyc et al. 2018 where adversaries can use multiple ip perturbations. We show that multi-norm robustness is the key to improving generalization against other threat models [Croce and Hein 2022). For instance, we show it enables robustness against perturbations not easily deñned mathematically. such as image corruptions and unseen adversaries [Wogg and Kolter 2020
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Two main challenges exist for training models robust against multiple perturbations: (i) tradeoff among robustness against different perturbation models [Tramer and Boneh 2019 and (ii) tradeoff between accuracy and robustness (Zhang et al. [019 Raghunathan et a 2020 Adversarial examples
induce a shift from the original distribution, causing a drop in clean accuracy with AT [Xie et al.) 2020 [Benz et al. 2021]. The distinct distributions created by 4, 12, 4/ adversarial examples make the problem even more challenging. Through a finer analysis of the distribution shifts caused by these adversaries, we propose the RAMP framework to efficiently boost the Robustness Against Multiple Perturbations. RAMP can be used for both fine-tuning and training from scratch. It utilizes a novel logit pairing loss certain pair and connects NT with AT via gradient projection (Jiang et al.) 2023 to improve union accuracy while maintaining good clean accuracy and training efficiency.
Logit pairing loss. We visualize the changing of 4 12, lo robustness when fine-tuning a AT pre-trained model in Figure D using the CIFAR-10 training dataset. The DNN loses substantial robustness against lo attack after only 1 epoch of fine-tuning: I fine-tuning and E-AT' [Croce and Hein, 2022] (red and yellow histograms under Linf category) both lose significant loc robustness (compared with blue histogram under Linf category). Inspired by this observation, we devise new logit pairing loss for L, - 1r tradeoff pair to attain better union accuracy, which enforces the logit distributions of l and l, adversarial examples to be close, specifically on the correctly classified L, subsets. In comparison, our method (green histogram under Linf and union categories) preserves more Lo and union robustness than others after 1 epoch. We show this technique works larger models and datasets (Section }.
Connect natural training (NT) with AT. We explore the connections between NT and AT to obtain a better accuracy/robustness trade- off. We find that NT can help with adversarial robustness: useful information in natural dis- tribution can be extracted and leveraged to achieve better robustness. To this end, we compare the similarities of model updates of NT and AT layer-wise for each epoch, where we find and incorporate useful NT compo- nents into AT via gradient projection (GP), as outlined in Algorithm _ In Figure 2 and Section 5.1] we empirically and theoretically show this technique strikes a better balance between accuracy and robustness, for both sin-
Epecy Fitune
0.0
2A
Figure 1: Multiple-norm tradeoff with robust fine- tuning: We observe that fine-tuning on AT model using l examples drastically reduces Loo robustness. RAMP preserves more To0 and union robustness.
gle and multiple lp perturbations. We provide a theoretical analysis of why GP works for adversarial robustness in Theorem A.2 & 4.5
Main contributions:
We design new logit pairing loss to mitigate the lg 1, I,- tradeoff for better union accuracy, by enforcing the logit distributions of ly and 4, adversarial examples to be close.
We empirically and theoretically show that connecting NT with AT via gradient projection better balances the accuracy/robustness tradeoff for 1p perturbations, compared with standard AT.
RAMP achieves good union accuracy, accuracy-robustnes, tradeoff, and generalizes better to diverse perturbations and corruptions (Section 5.1) achieving superior universal robustness (75.5% for common corruption and 26.1% union accuracy against unseen adversaries). RAMP fine-tuned DNNs achieve union accuracy up to 53.3% on CIFAR- 10, and 29.1% on ImageNet. RAMP achieves a 44.6% union accuracy and good clean accuracy on ResNet- 18 against AutoAttack CIFAR-10. Our code is available at Ihtpps:/ /tte:///gtttn tiub........ com/uiuc focal 1ab/RAMP
2 Related Work
Adversarial training (AT), Adversarial Training (AT) usually employs gradient descent to discover adversarial examples, incorporating them into training for enhanced adversarial robustness [Tramer
et al. 2017 Madry et al. 2017]. Numerous works focus on improving robustness by exploring
the trade-off between robustness and accuracy (Zhang et al. 2019 Wang et al.) 2020), instance
reweighting (Zhang et al. 2021 loss landscapes
Wu et al. 2020 wider/larger architectures (Gowal

---

et al, 2020 Debenedetti and Troncoso EPFL
2022]. data augmentation (C [Carmon et al. 2019
Raghunathan al. [020], and using synthetic data [Png al., 2023 Wang al. 2023]. However,
these methods often yield DNNs robust against single perturbation type while remaining vulnerable to other types.
Robustness against multiple perturbations, Tramer and Bonch 2019 Kang et al. [2019] observe
that robustness against iy attacks does not necessarily transfer to other by attacks (q + p). Previous
studies Tramer and Bonch, 2019 Maini et al. 2020 Madaan et al., 2021 Croce and Hein 2022]
modified Adversarial Training (AT) enhance robustness against multiple Lp attacks, employing
average-case (Tramer and Boneh 2019], worst-case [Trmmrr and Boneh 2019 Maini et al. 2020], and
andom-ssmplee [Madaan al. 2021| Croce and Hein [022] defenses. There are also works [Nandy et al. 2020 Liu et al. [2020] Xu et al. [iio et al., 2022 Maini et al. (022] using preprocessing.
ensemble methods, mixture of experts, and stability analysis to solve this problem. Ensemble models and preprocessing methods are weakened since their performance heavily relies on correctly classifying or detecting various types of adversarial examples. certified training. [Banerjee et al.) (2024) Banerjee and Singh| [2024) propose verification/certiffbbb training methods under different threat models for ip universal adversarial perturbation. However, prior works are hard to scale to larger models and datasets, e.g. ImageNet, due to the efficiency issue. Furthermore, Croce and Hein 2022] devise Extreme norm Adversarial Training (E-AT) and tune-tune-e a Ip robust model on another l perturbation quickly make DNN robust against multiple lp attacks. However, E-AT does not adapt to varying epsilon values. Our work demonstrates that the suboptimal tradeoff observed in prior studies can be improved with our proposed framework.
Logit pairing adversarial training. Adversarial logit pairing methods encourage logits for pairs of examples to be similar [Knnnan et al. [2018] |Engstrmm et al.] 2018). People apply this technique to both clean images and their adversarial counterparts, to devise a stronger form of adversarial training. In our work, we devise novel logit pairing loss to train DNN originally robust against attack to become robust against another 1, (g + p) attack on the correctly predicted Ip subsets, which helps gain better union accuracy.
Adversarial versus distributional robustness. Sinha et al. (018 theoretically studies the AT problem through distributional robust optimization, Mchrabi et al. [202]] establishes pareto-optimal tradeoff between standard and adversarial risks by perturbing the test distribution. Other works explore the connection between natural and adversarial distribution shifts [Moayeri et al. 2022 Allhamoud jtt al. 2023). assessing transferability and generalizability of adversarial robustness across datasets. However, little research delves into distribution shifts induced by (,, 12,22,4 Lo adversarial examples and their interplay with the robustness-accuracy tradeoff [Zhang et 2019 Yang et al., 2020 Rade and Moosavi-Dezfooli 2021]. Our work, inspired by recent domain adaptation techniques [Jiang] 2023 Jiang et al. 2023), designs logit pairing loss and utilizes model updates from NT via GP to enhance adversarial robustness. We show that GP adapts to both single and multi-norm scenarios.
3 AT against Multiple Perturbations
We consider standard classification task with samples {(2, No from an empirical data dis- tribution D,, we have input images x E Rd and corresponding labels y E I Standard training aims to obtain classifier f parameterized by 9 to minimize a loss function L IR* x R R on D,.. Adversarial training (AT) [Madry et al. 2017 Tramèr et al. (017)) aims to find a DNN robust against adversarial examples. It is framed as a min-max problem where a DNN is optimized using the worst-case examples within an adversarial region around each z;, Different types of adversarial regions Bp(x, <p) = f2' E Rd : I|a' - xlIp I 6 can be defined around given image using various 1p- ba-based perturbations. Formally, we can write the optimization problem of AT against a certain 4, attack as follows:
min E
D.
max ('E,,(
((((x'),))
The above optimization is only for certain p values and is usually vulnerable to other perturbation types. To this end, prior works have proposed several approaches to train the network robust against multiple perturbations (4,, (oo) at the same time. We focus the union threat model
3

---

A=Bi (1) By(a, 62) U oz Coo) which requires the DNN to be robust within the 11, 12, loe adversarial regions simultaneously [Croce and Hem 2022]. Union accuracy is then defined as the robustness against (()) for each sampled from D. In this paper, similar to the prior works, we use union accuracy as the main metric to evaluate the multiple-norm robustness. Apart from that, we define wniversal robustness as the generalization ability against a range of unseen adversaries and common corruptions. Specifically, we have average accuracy across five severity levels for common corruption and union accuracy against a range of unseen adversaries used in Laidlaw et al. (2020).
Worst-case defense follows the following min-max optimization problem to train DNNs using the worst-case example from the [ 11,12,0c adversarial regions:
min E (2.
max C(f(x'),) pE((1.200)) x (u.ee))
max
MAX Tramer and Boneh 2019] and MSD Maini et al. 2020 fall into this category. Finding worst-case examples yields good union accuracy but results in loss of clean accuracy as the distribution of generated examples is different from the clean data distribution.
Average-case defense train DNNs using the average of the 4, 12, to worst-case examples:
min E (a.))) 0
max ((((2)))) ot (2.6,
AVG [Tramer and Boneh 2019 is of this type. This method generally leads to good clean accuracy but suboptimal union accuracy as it does not penalize worst-case behavior within the 1 12, loc regions.
Random-sampled defense. The defenses mentioned above lead high training cost they compute multiple attacks for each sample. SAT [Madaan et al.) 202T] and E-AT [Croce and Hein 2022] randomly sample one attack out of each type a time, contributing a similar computational cost as standard AT on a single perturbation model. They achieve a slightly better union accuracy compared with AVG and relatively good clean accuracy. However, they are not better than worst-case defenses for multiple-norm robustness, since they do not consider the strongest attack within the union region all the time.
4 RAMP
There are two main tradeoffs in achieving better union accuracy while maintaining good accuracy: 1. Among perturbations: there is a tradeoff among different attacks, e.g., lo pre-traines AT DNN is not robust against 4,42 perturbations, which makes the union accuracy harder to attain. Also, we observe there exists a main tradeoff pair of two attacks among the union over I, 12, Los attacks. 2. Accuracy and robustness: all defenses lead to degraded clean accuracy. To address these tradeoffs, we study the problem from the lens of distribution shifts.
Interpreting tradeofl's from the lens of distribution shifts. The adversarial examples with respect to empirical data distribution D... adversarial region (,, <p), and DNN So generate new adversarial distribution D. with samples {(<%, v:) that are correlated by adding certain perturba- tions but different from the original D,,. Because of the shifts between Dn and D.. DNN decreases performance on Dn, when we move away from it and towards D... Also, the distinct distributions created by multiple perturbations, D: D'2 a contribute to the tradeoff among I 12, loc attacks. To address the tradeoff among perturbations while maintaining good efficiency, we focus on the distributional interconnections between D,, and Di: D'a Dire. From the insights we get from above, propose our framework RAMP, which includes (i) logit pairing to improve tradeoffs among multiple perturbations, and (ii) identifying and combining the useful DNN components using the model updates from NT and AT, to obtain better robustness/accuracy tradeoff
Identify the Key Tradeoff Pair. We study the common case with I, norms é 12, <2 = 0.5, 6,0 = 255 on CIFAR-10 10 Tramer and Boneh 2019). The distributions generated by the two strongest attacks show the largest shifts from D,, also, they have the largest distribution shifts between each other

---

because of larger and most distinct search areas. Thus, by comparing the single norm robustness of lp adversarially trained models, we select the two 4- models with the lowest I, robustness against themselves as the key tradeoff pair. They refer to the strongest attack since their Ip robustness is low. The attack with the highest Ip robustness mostly included by the convex hull of the other two stronger attacks [Croe and Hein 2022 Here we identify loc - I as the key tradeoff pair.
4.1 Logit Pairing for Multiple Perturbations
Figure T Finetuning a ATT model on 1, examples reduces l robustness. get a finer analysis of the 1ox 1 tradeoff mentioned above, we visualize the changing of I1, 12, lo robustness of the training dataset when we fine-tune lo pre-trained model with I examples for epochs, as shown in Figure D x-axis represents the robustness against different attacks and y-axis is the accuracy. After 1 epoch of finetuning on ( examples or performing E-AT, we lose much (x robustness since much lower than the red histogram under the Linf category. RAMP preserves both (o0 and union robustness more effectively: the green histogram is higher than the red/yellow histogram under Linf and Union categories. Specifically, RAMP maintains 14%, 28% more union robustness than E-AT and 4 fine-tuning. The above observations indicate the necessity adversarially fine-tune with 4, adversarial examples l pre-trained AT model, with lg - I, as the key tradeoff pair, which inspires us to design our loss design with logit pairing. We want enforce the union predictions between l and (- * r) attacks: bringing the predictions of l and ( & r) close to each other, specifically on the correctly predicted l subsets. Based on our observations, we design new logit pairing loss enforce a DNN robust against one by attack to be robust against another (4 + r) attack.
blue/yellow histograms
of preserving more l robustness
Enforcing the Union Prediction via Logit Pairing. The 113 1 (+(q+1) r) tradeoff leads to the following principle to improve union accuracy: for a given set of images, when we have a DNN want it to be robust against I, examples as well. This serves as the main insight for our loss design: we want to enforce the logits predicted by ly and I, adversarial examples be close, specifically on the correctly predicted l, subsets. To accomplish this, design a KI -ivdeegreee (KI.) loss between the predictions from l and 1, perturbations. For each batch of data (ar, y) D, generate ly and adversarial examples 27, and their predictions Py: Pr using APGD [Croce and Hein 2020]. Then, we select indices y, which part elements of Pa correctly predicts the ground truth y. We denote the size of the indices The, and the batch size as N. We compute a KL-divergence loss over this set of samples using K [PP[7]]]PP-11) indexed by 7 we want to push its I, logit distribution towards its ly logit distribution, such that prevent losing more lg robustness when training with 4, adversarial examples.
robust against some lg examples,
(Eq. [ For the subset
ne
PAbEL Pr bAM
LKI.
Po PobM
log
The
To further boost the union accuracy, apart from the KL loss, we add another loss term using a MAX-style approach in Eq. we find the worst-case example between lg and I, adversarial regions by selecting the example with the higher loss. Lmax a cross-entropy loss over the approximated worst-case adversarial examples. Here, we fee to represent the cross-entropy loss. Our final loss L combines LK and Lmaz, via a hyper-parameter in Eq.
Emaz
max 1=0 pE(a.) (z,t,)
Lee Cf (f
L Lmax X LKL
N
Algorithm
shows the pseudocode of robust fine-tuning with RAMP that leverages logit pairing.
4.2 Connecting Natural Training with AT
To improve the robustness and accuracy tradeoff against multiple perturbations, we explore the connections between AT and NT. Since extracting valuable information NT aids improving robustness (Section 4.2). we use gradient projection [Jiang et al. 2023 to compare and integrate natural and adversarial model updates, which yields an improved tradeoff between robustness and accuracy.
5

---

NT can help adversarial robustness, Let us consider two models fi and fz, where fi is randomly initialized and f2 undergoes NT on Dn, for k epochs: f2 results in better decision boundary and higher clean accuracy. Performing AT on fi and f2 subsequently, intuitively, f2 becomes more robust than f1 due to its improved decision boundary, leading fewer misclassifications of adversarial examples. This effect is empirically shown in Figure 2 For AT (blue), standard AT against lee attack [Madry et al., 2017) is performed, while for AT-pre (red), 50 epochs of pre-training precede the standard AT procedure. AT-prr shows superior clean and robust accuracy on CIFAR-10 10 against loo PGD-20 attack with too = 0.031. Despite D, and Da are different, Figure suggests valuable information in D, that potentially enhances performance on Da..
AT with Gradient Projection. To connect NT with AT more effectively, we analyze the training procedures on D,, and D.. We consider model updates over all samples from D,, and Da, with the initial model f(r) at epoch r, and models f% and flr) after 1 epoch of natural and adversarial training from the same starting point f(t), respectively. Here, we compare the natural updates Gn = fn f()) and adversarial updates Ja f(² f(r), Due to distribution shift, an angle exists between them. Our goal is to identify useful components from 9n. and incorporate them into do for increased robustness in Da while maintaining accuracy in D,, Inspired by Jiang et al. [2023]. we layer-wisely compute the cosine similarity between In and Ga. For specific layer / of gn and ya we preserve portion of Hn based on their cosine similarity score (Eq 4b. Negative scores indicate that T. not beneficial for robustmess in Da. Therefore, we filter components with similarity score S We define the GP (Gradient Projection) operation in 54 by projecting Z. towards Fn-
cos( (.) Th. 0.
cos GG, -.))00)) >0 cos In (n. S
cos cos( (.
In
To
(4)
GP B'a)
(5)
I
Therefore, the total projected (useful) model updates 9p coming from In could be computed as Eq. We use M to denote all layers the current model update. Note that UIeM concatenates all layers useful natural model update components. A hyper-parametee /3 is used to balance the contributions of 9GP and Ja. as shown in Eq. 7 By finding a proper (0.5 as in Figure 4c). we can obtain better robustness on D., as shown in Figure and Figure 3 In Figure 2 with B = 0.5, AT-GP refers to AT with GP; for AT-GP-pre. we perform 50 epochs of NT before doing AT-GP. We see AT-GP obtains a better accuracy/robustness tradeoff than AT. We observe a similar trend for AT-GP-pre vs. AT-pre. Further, Figure 3 RN-18 -o achieves good clean accuracy and better robustness than RN-18 loc against AutoAttack [Croce and Hein 2020
9p U GP 3la
[Tn:
(6)
f(r-1) = + B. 9r + +(1 - B) Ta
+
TEM
Algorithm Fine-tuning via Logit Pairing
Algorithm Connect AT with NT via GP
I: Input: model f. input samples (z, y) from distribution D,,, fine-tuning rounds R, hyper-paramedee
1: Input: model f. input images with distri- bution D,,, training rounds R, adversarial region B, and its size Ep. 3, natural train- ing NT and adversarial training AT.
X, adversarial regions By B, with size Eg and 6,, APGD attack. 2: for r ror-=1, 1,2, R do
for r 1,2, ..., R do
3: for (I,, u) training set D do
3: fn. + (T) D)
4:
Py + APGD B, (x, kq), y) er.Pr + APGD (x, 6 (), 7 7 w-eerearrgma Pg y)
4: fa + (T()()) Bp. Cp. D)
5:
5: compute Gn t fn ((r)). Da Sa
6:
f((r)
7:
The - 7.size(
compute Ip using Eq. 6
8:
calculate L using Eq. 3 and update f end for
update (r-1) using Eq. with $ and
9:
end for
end for
11: Output: model f.
9: Output: model f.

---

4.3 Theoretical Analysis of GP for Adversarial Robustness
We define D,, = { yi 1 as the ideal data distribution with an infinite cardinality. Here, we con- sider a classifier fo at epoch t. We define Du as the distribution created by {(z; tez,teffa 24, Ue), ys 1 where (z yt) nn-- 2; c(fe, 24. y1) denotes the perturbed image, which could be both single and multiple perturbations based on so itself.
Assumption 4.1. We assume Dn consists of N i..dd samples from the ideal distribution Dn and D. {(2; c(f". 24, ye), Vs 1:-0 where (Iz: Vs) Dn consists of N i.i.d. samples from Da.
We define the population loss as LD(0) = -(fx)>-D(((()) v), and let gD(() = (0) For
simplification, we use Gn == VLD. (0). Ga == VL (0), and Jn == vL (0). 9GP = 3 9p + (3- Te (Definition A.3) is the aggregation using GP. We define the following optimization problem.
Definition 4.2 (Aggregation for NT and AT). fo is trained by iteratively updating the parameter
0 0 p.. Aggr( Fu);
where y is the step size. We seek an aggregation rule Aggr(.) = DAggr such that after training, Se minimizes the population loss function LD_ (0).
We need Taggr to be close to for each iteration, since Ia is the optimal update on D,- Thus, we define -ormNorm and delta error indicate the performance of different aggregation rules.
Definition 4.3 -or-Norm [Enyi Jiang 2024 Given distribution on the parameter space O, we E (gg (0), yD> (0))). The inner product induces the L."- norm on 9D as || == / (l We use I -nom to measure the gradient differences under certain D.
define an inner product (gD. 9D )x
Definition 4.4 (Delta Error of an aggregation rule Aggr(-)). We define the following squared error term measure the closeness between Gäggr and 9a under D' (distribution at time step t), i.e.,
2igr
EBg 19aa ll
Delta errors A7 and Orr measure the closesness of gGP, Gn from Ja in D., each iteration.
Theorem 4.5 (Error Analysis of GP). When the model dimension - co, for epoch I, we have an approximation of the error difference A²TT Drr as follows
2 - 2 Re B - B) I1g. - gull 32 - In
72 = E (72] E [0, 1). where r0) is the sin(.) value of the angle between In and 9a In-
Theorem 4.5 4.5 shows 2 is generally smaller than A for large model dimension during each iteration, as the case for the models our evaluation, with 8 = 0.5, since 3(1 ~32 3 > 0.25) and the small value of F in practice (see Interpretation of Theorem A.2 Appendix A where we show the order of difference between le and le 11)). Thus, GP achieves better robust accuracy than AT by achieving smaller delta error; GP also obtains good clean accuracy by combining parts of the model updates from the clean distribution D,.. Further, we provide an error analysis of single gradient step Theorem A.T] convergence analysis in Theorem A.2 showing that a smaller Delta error results in better convergence. The full proof of all theorems is in Appendix A
We outline the AT-GP method in Algorithm 2 and it can be extended the multiple-norm scenario. The overhead of this algorithm comes from natural training and GP operation. Their costs are small, and we discuss this more in Section 5.2 Combining logit pairing and gradient projection methods, we provide the RAMP framework which is similar to Algorithm except that we replace line 4 of Algorithm as Algorithm I line 3-9.
Experiment
Datasets, baselines, and models. CIFAR-10 KKizhevsky et al., 2009 includes 60K images with 50K and 10K images for training and testing respectively. ImageNet has 14.2M images and 1K classes, containing 1.3M training, 50K validation, and 100K test images [Russakovsky al.
7

---

ACC
Robust Roc
Clean
10c
RN-18 lo
84.2
47.4
AT
AT
AT-CaP
A3-PP
RN-18 t00 ..-GP
84.5
48 3
AT-GP-sre
RN-18 1s0 -.--PPppr
84.9
48 3
ATere
Epochs
Epochs
Figure 3:
1o AT-GP
with
APGD [Croce and Hein 2020
(a) Clean Accuracy
(b) Robust Accuracy: PGD-20
improves robustness against 100
Figure 2: Lo AT-GP with PGD [Mdryy et al., 2017] with == AutoAttack [Croce and Hein 2020
0.031 on CIFAR-10 improves accuracy and robustness. Pre- with 6 = 206 RN-18 GP uses training on D, for 50 epochs further boosts the performance. AT-GP; RN-18 -GP-pe pre-trains
40 epochs on D,, before AT-GP applied.
2015]. We compare RAMP with following baselines: 1. SAT MMdaan et al. 2021]: randomly
sample one of the 4, 12, Lo attacks. 2. AVG (Tramer and Bonch 2019]: : take the average of 4 1,12,44 Loc. examples. 3. MAX [Tramer and Boneh 2019 take the worst of 4,, 2, to attacks. 4. MSD MMaini (ct al. 2020]: find the worst-case examples over 4, 12, loo steepest descent directions during each step of inner maximization. 5. E-AT [Croce and Hein 2022]: :: randomly sample between 4, 1. attacks. For models, we use PreAct-ResNet 18, ResNet-50, WideResNet-34-- and WideResNet-70. for CIFAR-10, 10, as well as Re and XCIT-S transformer for ImageNet.
Implementations and Evaluation. For AT from scratch for CIFAR-10, we train PreAct ResNet- 18 (He et al., 2016) with Ir = 0.05 for 70 epochs and 0.005 for 10 more epochs. We set À = 2, 0.5 for robust fine-tuning. For all methods, we use 10 steps for the inner maximization in AT. For ImageNet,
3= 0.5 for training from scratch, and
perform epoch of fine-tuning and use a learning rate Ir = 0.005, À = 0.5 for ResNet-50 and Ir = le -, X = 0.5 for XCiT-S models. We reduce the rate by a factor of 10 every $ of the training epoch and set the weight decay to We use APGD with 5 steps for loo and l2. 15 steps for 4 Settings &
1e 1e-4. 4.
similar to CCroe and Hein 2022]. We use the standard values of 61 = 12, E2 = 0.5, Eoo = 255 for CIFAR and
255, 62 = 2, toc 255 for ImageNet. We focus on -TT models for fine-tuning, as Croce and Hein 2022] shows their higher union accuracy for the values in our evaluation. We report the clean accuracy, robust accuracy against (41.2, 12, lxc attacks, union accuracy, universal robustness against common corruptions and unseen adversaries, as well runtime for RAMP. The robust accuracy is evaluated using Autoattack [Croce and Hein 2020). More implementation details are in Appendix B
5.1 Main Results
Table Different epsilon values: RAMP consistently outperforms E-AT and MAX for both training from scratch and robust fine-tuning when the key tradeoff pair changes.
(12, 0.5,
zir
(12,
1.5, ziz
Clean
1s
l2
41
Union
Clean
1oc
12
4
Union
Training from Scratch
E-AT
87.2
73.3
64.1
55.4
55.4
83,5
41.0
25.5
52.9
25.5
MAX
85.6
72.1
63.6
56.4
56.4
74.6
42.9
35.7
50.3
35.6
RAMP
86.3
73.3
64.9
59.1
59.1
74.4
43.4
37.2
51.1
37.1
Robust Fine-tuning
E-AT
86.3
74.8
66.7
37.9
37.9
80.2
42.8
31.3
52.4
31.3
MAX
85.7
74.0
66.2
60.0
60.0
74.8
43.8
36.7
50,2
36.6
RAMP
85.8
74.0
66.2
60.1
60.1
74.9
43.7
37.0
50.2
36.9
Robust fine-tuning, In Table we apply RAMP to larger models and datasets (ImageNet). However, the implementation of other baselines is not publicly available and Croce and Hein| [2022] do not report other baseline results except E-AT on larger models and datasets, so we only compare against E-AT in Table which shows RAMP consistently obtains better union accuracy and accuracy- robustness tradeoff than E-AT. We observe that RAMP improves the performance more as the model becomes larger. We obtain the SOTA union accuracy 53.3% on CIFAR-I0 and 29.1% on ImageNet.
RAMP with varying 61, C22 Coo values. We provide results with 1. (G = 12, c2 = 0.5, coc 255 where 6 size small and 2. (e1 = 12, f2 = 1.5, 6:x 255 where f2 size large, using PreAct ResNet-18 18 model for CIFAR-10 dataset: these cases have different tradeoff pair compared to
8

---

Table 2: Robust fine-tuning on larger models and datasets ( uses extra data for pre- vre-training). We
evaluate all CIFAR- 10 and Imagenet test points. RAMP consistently achieves better union accuracy with significant margins and good uccuracy-robustng tradeoff.
Models
Methods
Clean
lac
F2
Union
WRN-70-16-7000
E-AT
58.0
51.6
RAMP
74.6
57.9
53.3
W8N-34-20--0
Gowwl et
2020
E-AT
S7.
40.0
7L6
49.8
45.1
RAMP
57.1
49.7
70.8
50.4
46.9
WWN-28-10-1.0 Canton
E-AT
T466
CIFAR-10
RAMP
T4.7
55.7
52.7
WRN-28-10-1 Gowal et 2020
E-AT
89.8
54.4
76.1
$6.0
50.5
RAMP
80.4
55.9
T4.7
56.0
52.9
RN-50-1
Bugstrom
2019
E-AT
41.6
RAMP
47.0
67.7
46.5
43.3
X Deberedet:: and Troocuso BATL 2022
EXA
65.1
38.1
51.8
23.8
23.4
ImageNet
RAMP
Engstroom 2019
39.5
17.8
RAMP
53.6
23.1
38.3
22.4
20.9
Figure The pair identified using our heuristic are 4 12 and 22- 1c. In Table I we observe that RAMP consistently outperforms E-AT and MAX with significant margins in union accuracy, when training from scratch and performing robust fine-tuning. In Table T when 12 is the bottleneck, E-AT obtains lower union accuracy as it does not leverage l2 examples. Similar observations across various epsilon values, with RAMP consistently outperforming other baselines, as detailed in Appendix B.4 Appendix B includes more training details/results, and ablation studies. Results for applying the trades loss to RAMP outperforming E-AT are detailed in Appendix B.6 Appendix presents robust fine-tuning using ResNet-18, where RAMP achieves the highest union accuracy.
made
Adversarial training from random initialization. Table presents the re- sults of AT from random initialization on CIFAR-10 with PreAct ResNet-18 18. RAMP has the highest union accu- racy with good clean accuracy, which indicates that RAMP can mitigate the tradeoff's among perturbations and ro- bustness/accuracy in this setting. The results for all baselines from from[rrocs Croce
Table RN-18 model trained from random initialization on CIFAR-10 10 over 5 trials: RAMP achieves the best union
robustness and good clean accuracy compared with other baselines. Baseline results are from Croce and Hein 2022]
Mehhods
100
1/
SAT
83.9100.
60.71:0.3
68.01004
54.01:12
40.41:077
AVG
84.6+-0.3
40.8--0.7
68.4+-0.7
52.14-0.4
40.1-10.88
80410.
2577:9
660000
48611:8
43.020077
MSD)
81:11:11
449206
650110
49.5212
43.900.88
82:21:1
42710077
67.51.0.5
$36610.1
424410.6
RAMP (A=5)
8124-033
46.04:0.5
65.84:0.2
48.3-10..6
4..6:-0.6
and Hein 2022).
(A-2)
82.14:03
45.54:033
66.64:03
48.4:0222
44.0200.2
Table 4: Individual, average, and union accuracy against common corruptions (averaged across five 28-1- 10 on CIFAR- 10 dataset.
levels) and unseen adversaries using WideResNet-
Modeis
Common Comuptions
[o
fog
sDow
gabor
clustc
jpeginf
Avg
Unicn
78.2
469
12-AT
71.2
67.5
48.7
26.1
44.1
53.2
45.4
47.5
16.2
1oo-AT
73.4
55.5
33.4
46.2
Winninghand Diffenderter al. 2011
91.1
34,1
74.5
76.5
0.0
42.7
0.0
E-AT
71.5
58.5
35.9
35.3
50.7
55.7
603
49.4
21.9
MAX
71.0
24.4
RAMP
75.5
55.5
40.5
40.2
$2.9
60.3
56.1
50,9
26.1
Universal Robustness. In Table we report average accuracy against common corruptions and union accuracy against unseen adversaries from Laidlaw et al. [2020] (implementation details are in Appendix B.3 We compare against lp pretrained models, s, E-AT, MAX, winninghand [Diffenderfer al. 2021 (a SOTA method for natural corruptions) using WideResNet-288
architecture on the CIFAR-10 dataset. Compared to E-AT and MAX, RAMP achieves 4% higher accuracy for common corruptions with five severity levels and 2-4% better union accuracy against multiple unseen adversaries. Winninghand has high corruption robustness but no adversarial robustness. The results show that RAMP obtains a better robustness and accuracy tradeoff with stronger universal robustness. In Appendix B.3 we evaluate on ResNet-18 88 to support this fact further.
5.2 Ablation Study and Discussion
Sensitivities of X. We perform experiments with different À values in (0.1,0.5, 0.0, 5, for robust fine-tuning and [1.5, 2,3, 4, 5, 6] for AT from scratch using PreAct-ResNNet-18 model for CIFAR- 10 dataset. In Figure 4 observe decreased clean accuracy when À becomes larger. We

---

Clean
AMC
46
42
A
(a) A: Robust fine-tuning.
(b) A: Train from scratch.
(c) B: Train from scratch.
Figure 4: Alabtion studies on and 3 hyper-parameters.
pick À = 2.0 for training from scratch (Figure 4a and À = 0.5 for robust fine-tuning (Figure 4b) in our main experiments, these values of yield both good clean and union accuracy.
Choices of B. Figure 4c shows the performance of RAMP with varying / values on CIFAR-10 ResNet- 18 experiments. We pick 3 0.5 for combining natural training and AT via GP, which achieves comparatively good robustness and clean accuracy. This choice is also based on Theorem 4.5 when ((2 - 3) has the largest difference from 8 (0.75 vs 0.25).
Fine-tune 1p AT models with RAMPP. Table 5 shows the robust fine-tuning results using RAMP with 30- AT (q oc, r 1), AAT (g 1, o0), 12-AT (q X2, 1) RN-18 models for CIFAR- 10 dataset. For 1xc tradeoffs, RAMP on AT pre-trained model achieves the best union accuracy.
Clean
1oo
1/
Union
RN-18 AAT RN.18 AT
81.5
45.5
66.4
47.0
42,9
42.6
RN-18 AT
84.1
41.6
39.4
Table 5: RAMP with loo, (1, 12-----------
Computational analysis and Limitations. The extra training costs of AT-GP are small, c.g. for each epoch on ResNet- 18, the extra NT takes seconds and the standard AT takes 78 seconds using a single NVIDIA A100 GPU, and the GP operation only takes 0.04 seconds on average. RAMP more expensive than E-AT and less expensive than MAX. We have a complete runtime analysis in Appendix B.2 We notice occasional drops in clean accuracy during fine-tuning with RAMP. some cases, union accuracy improves slightly but clean accuracy and single I, robustness reduce. Further, we find no negative societal impact from this work.
models on CIFAR-10 with standard epsilons.
6 Conclusion
We introduce RAMP, framework enhancing multiple-norm robustness and achieving superior universal robustness against corruptions and perturbations by addressing tradeoffs among (p perturba- tions and accuracy/robustness. We apply a new logit pairing loss and use gradient projection to obtain SOTA union accuracy with favorable accuracy/robustness tradcoff's against common corruptions and other unseen adversaries. Results demonstrate that RAMP surpasses SOTA methods in union accuracy across model architectures CIFAR-10 10 and ImageNet.
Acknowledgments
This work was supported in part by NSF Grants No. CCF-2238079, CCF-2316233, CNS-2148583. We would like to thank Jacky Yibo Zhang for the helpful discussions and advice on the proof. Also, we thank anonymous reviewers for their valuable feedback on the paper.
References
Kumail Alhamoud, Hasan Abed Al Kader Hammoud, Motasem Alfarra, and Bernard Ghanem. Generalizability of adversarial robustness under distribution shifts. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URI. hattpe://qpenreiiev.neee wtwp:eewwwww.eeeeo/emimm.iie
XNFo3dDiCJ Featured Certification.
10

---

Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In International conference on machine learning, puges 284-293. PMLR, 2018.
Debangshu Banerjee and Gagandeep Singh. Relational dnn verification with cross executional bound refinement. arXiv preprini arXiv:2405.10114 2024.
Debangshu Banerjee, Changming Xu, and Gagandeep Singh. Input-relational verification of deep neural networks. Proceedings of the ACM on Programming Languages, 8(PLDI): 2024. 2024.
Philipp Benz, Chaoning Zhang, and In So Kweon. Batch normalization increases adversarial vulnerability and decreases adversarial transferability: A non-robust feature perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7818-7827, 2021.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John Duchi, and Perey Liang. Unlabeled data improves adversarial robustness. Advances in neural information processing systems, 32, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pages 2206-2216. PMLR, 2020.
Francesco Croce and Matthias Hein. Adversarial robustness against multiple and single I p- p-threet models via quick fine-tuning of robust classifiers. In International Conference on Machine Learning, pages 4436 4454. PMLR, 2022.
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag. Edoardo Debenedetti, Nicolas Flam- marion, Mung Chiang, Prateek Mittal, and Matthias Hein, Robusthench: standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.0963 2020.
Francesco Croce, Maksym Andriushchenko, Naman D Singh, Nicolas Flammarion, and Matthias Hein. Sparse-rs: a versatile framework for query-efficient sparse black-box adversarial attacks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6437-6445, 2022.
Edoardo Debenedetti and Carmela Troncoso-EERe Adversarially robust vision transformers, 2022.
James Diffenderfer, Brian Bartoldson, Shreya Chaganti, Jize Zhang, and Bhavya Kailkhura. winning hand: Compressing deep networks can improve disti--dit robustness. Advances in neural information processing systems, 34:6644674 2021.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and translation suffice: Fooling cnns with simple transformations. 2017.
Logan Engstrom, Andrew Ilyas, and Anish Athalye. Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint arXiv: 10272, 2018.
Logan Engstrom, Andrew Ilyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library). 2019. URL URL /ttt::::s://ttthhin com/ wwt.ccoo///aaa htw///adrylabbrrbusuteesss
Sanmi Koyejo Enyi Jiang, Yibo Jacky Zhang. Principled federated domain adaptation: Gradient pro- jection and auighweigh In The Twelfth International Conference on Learning Representations, 2024. http:/ttt////peeeeeeee
Kevin Eykholt, Ivan Eytimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1625-1634, 2018
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against nornebounded adversarial examples. arXiv preprint M.Xi::2010.03593. 2020.

---

Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 770-778, 2016.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness common corruptions and perturbations. arXiv preprint arXiv:1903. 2019.
Enyi Jiang. Federated domain adaptation for healthcare, 2023.
Enyi Jiang, Yibo Jacky Zhang, and Oluwasanmi Koyejo. Federated domain adaptation via gradient projection. arXiv preprint arXiv:2302.05049, 2023.
Daniel Kang, Sun, Tom Brown, Dan Hendrycks, and Jacob Steinhardt. Transfer of adversarial robustness between perturbation types. arXiv preprint arXiv:1905.01034 2019.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing, arXiv preprint arXiv:1803.06337, 2018.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, lan J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security, pages 99-112. Chapman and Hall/CRC, 2018.
Cassidy Laidlaw, Sahil Singla, and Soheil Feizi. Perceptual adversarial robustness: Defense against unseen threat models. arXiv preprint arXiv:2006. 000.0006666665050 /2655, 2020.
Aishan Liu, Shiyu Tang, Xianglong Liu. Xinyun Chen, Lei Huang, Zhuozhuo Tu, Dawn Song, and Dacheng Tao. Towards defending multiple adversarial perturbations via gated batch normalization. arXiv preprint arXiv:2112.01654, 2020.
Divyam Madaan, Jinwoo Shin, and Sung Ju Hwang. Learning to generate noise for multi-attack robustness. In International Conference Machine Learning, pages 7279-7289. PMLR, 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 10Xx:11066.0083. 2017.
Pratyush Maini, Eric Wong, and Zico Kolter. Adversarial robustness against the union of multiple perturbation models. In International Conference on Machine Learning, pages 6640-6650. PMLR, 2020.
Pratyush Maini, Xinyun Chen, Bo Li, and Dawn Song. Perturbation type categorization for multiple adversarial perturbation robustness. In Uncertainty in Artificial Intelligence, pages 1317-1327. PMLR, 2022.
Mohammad Mehrabi, Adel Javanmard, Ryan A Rossi, Anup Rao, and Tung Mai. Fundamental tradeoffss in distributionally adversarial training. In International Conference on Machine Learning. pages 7544-7554 PMLR, 2021.
Mazda Moayeri, Kiarash Banihashem, and Soheil Feizi. Explicit tradeoff' between adversarial and natural distributional robustness. Advances in Neural Information Processing Systems, 35: 38761-38774, 2022.
Jay Nandy, Wynne Hsu, and Mong Lee. Approximate manifold defense against multiple adversarial perturbations. In 2020 International Joint Conference Neural Networks (IJCN),, pages 1-8. IEEE, 2020.
Sheng Yun Peng, Weilin Xu, Cory Comeelus, Matthew Hull, Kevin Li, Rahul Duggal, Mansi Phute, Jason Martin, and Duen Horng Chau. Robust principles: Architectural design principles for adversarially robust enns. arXiv preprint arXiv:2308. /6258, 2023.
Rahul Rade and Seyed-Mohsen Moosavi-Dezfoolli Reducing excessive margin to achieve better accuracy robustness trade-off. In International Conference on Learning Representations, 2021.
12

---

Aditi Raghunathan. Sang Michael Xie, Fanny Yang, John Duchi, and Percy Liang, Understanding and mitigating the tradeoff between robustmess and accuracy. arXiv preprint arXiv:2002.10716, 2020.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-2523 2015.
Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the first adversarially robust neural network model on mnist. arXiv preprint arXiv: 180500999, 2018.
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K Reiter. Accessorize to a crime: Real and stealthy attacks on state-of-the face recognition. In Proceedings of the 2016 acm sigsac conference on computer and communications security, pages 1528-15500 11440, 2016.
Aman Sinha, Hongseok Namkoong, and John Duchi, Certifiable distributional robustness with principled adversarial training. In Internationaa Conference on Learning Representations, 2018. .nt peet/forummidd-HHoRttt
URL /tRLtttt:////peereeeee
Dawn Song, Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, and Tadayoshi Kohno. Physical adversarial examples for object detectors. 12th USENIX workshop on offensive technologies (WOOT 18), 2018.
Florian Tramer and Dan Boneh. Adversarial training and robustness for multiple perturbations. Advances in neural information processing systems, 32, 2019.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07204 2017.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In ICLR, 2020.
Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning mlr
Research, pages 36246-36263, PMLR, 23-29 Jul 2023. URL
press/v209 vooss//v02//2 pross/v202wvang23ad wwdg/3an.nnna...m..mm. html
Eric Wong and J Zico Kolter. Learning perturbation sets for robust machine learning. arXiv preprint arXiv:2007.084550 2020.
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. Advances in Neural Information Processing Systems, 33:2958-2966 2020.
Jiancong Xiao, Zeyu Qin, Yanbo Fan, Baoyuan Wu, Jue Wang, and Zhi- Chi-Quan Luo. Adaptive smoothness weighted adversarial training for multiple perturbations with its stability analysis. arXiv preprint arXhv:2210.0055: 2022.
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan L Yuille, and Quoe V Le. Adversarial examples improve image recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 819-828, 2020.
Kaidi Xu, Chenan Wang, Hao Cheng, Bhavya Kailkhura, Xue Lin, and Ryan Goldhahn. Mixture of robust experts (more): A robust denoising method towards multiple perturbations. arXiv preprint arXiv:2104. 10586, 2021.
Yuo-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ R Salakhutdinov, and Kamalika Chaud- huri. A closer look at accuracy vs. robustness. Advances in neural information processing systems, 33:8588 8601, 2020.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Procedings of the British Machine Vision Conference 2016. British Machine Vision Association, 2016.
13

---

Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pages 7472-7482. PMLR, 2019.
Jingfeng Zhang, Jianing Zhu, Gang Niu, Bo Han, Masashi Sugiyama, and Mohan Kankanhalli. Geometry-aware mnstance-reweighter adversarial training. In International Conference on Learning Representations, 2021. URL http:::/opprreeiiieeeeeee/ www.:eww..eeeeeeeeeeeeeeeeeeeeee wwt.:eet/oouu aeee/fforumm11-11000060a8a
14

---

A Proof of Theorems
A.1 Proof of Theorem A.2
We first show what happens during one step of optimization, where we highlight the importance of analyzing delta enror.
Theorem A.1. Consider model parameter > O. Define the updated parameter as
π and an aggregation rule Aggr(.) with step size
0+ == 0 pr
Assuming the gradient VC()) is 7-Lipschitz in 0 for any input, and let the step size μe S 1 we have
B e CD, (f+) LD. (0) - (11aall A Agggr)
Proof. The proof is the same as Theorem A. in
Enyi Jiang 2024].
Theorem A.2 (Convergence Aggr(-)). For any probability measure over the parameter space, and an aggregation rule Aggr(.) with step size > O. We update the purameter for T steps by B++1 == f³ ( Assume the gradient V L()) and (aggr are 3/2 3-Lipschitz
such that f + Aggr Aaggr_naz is the Delta error at time t' when llA (0 (Oagrr V (l is maximized. Then, given step size S + and a small enough e > O, with probability at least 1 we have
82 (VCc Digrr. mar + +l0))² -0 (o
D (0")
O(c),
where C, py /1/x Be (0 s)g and (0 C TR" is the ball with radius centered at 0aggr-
The C. measures how well covers where the optimization goes.
Proof. Denote random function f: Rm -> R4 as
:R
f(0) = l1Agger (0) VLD. (f)l,,
(8)
where the randomness comes from D.. Note that iis y-Lipschitz by assumption. Now consider B. (Oager) C. R", i.e., the ball with radius centered at OAger. Then, by g-Lipschitzness have
Es~x F(0) = f(0)dT(0)
dr(()
(F(Oneer) - YE) dTT(
n3 (umcc)
(Frneer) - (Be
Therefore,
((nggr) <
x (B.g(r))))) Es f(0)+ + O(e).
Taking expectation w.r.t. D. on both sides, we have
EB, [(Oneer) S E..
-Ena +0(E)
(B. (Bnger)))
2
Bs.
E.g,
Boon [(0] + O())
(Cauchy-Schwarz))
TT (Beeer)
C. EB. [Bown. ((0)
Oce)
(by definition of C,)
$ V C. EE,, Egon [(0) 2 + 0())
(Jensen's 's inequality)
VCe A + O(e)
15

---

By Markov's inequality, with probability at least - 8 we have sampled dataset D. such that
[(Ongrr) - BB. [(0ner)) S 5 VV. Anger O
(9)
Conditioned on such event, we proceed on to the optimization part.
Note that Theorem A.1 characterizes how the optimization works for one gradient update. We denote DA as the data distribution Da time step Therefore, for any time step = 0, ...... 1, we can apply Theorem A.1 which only requires the Lipschitz assumption:
LDy (0 CD: (0') S
11 (IIV (0') Te (0') VLDD (0))
2
We notice that D changes based on Os of different time steps. On both sides, to sum over T O,....
1, we first consider two terms:
(LD; (B -D; (8")) + (Cp;-1 - (
To compare LD; (0*) and Luy 10'), since D'd optimizes one more step than D we assume
DD (0') & D (0") for Vt. Therefore, we have:
(CDg((0 - CDy - (0')) + (0p)-1 - fp.-1(0'11)) 2 (CD: (ED:(0'+1) -p:-1 + (Cp:-1 -.: (0--1))
Summing up all time steps,
LDD (0(+1) - (0)) S LD:(('t))) -pp + (pp c +... LDg (09)
T-1 Σ
VLp: (0')]|
[ (0') VLp: (0¹)
Dividing both sides by T, and with regular algebraic manipulation we derive
2 IVCp: (0)) (Crg (09) - Cor-1(0")) +
TaAeEr (0') VLD; (0')
Note that assume the loss function LD(()) (x,))) ~ ((f(x)) v). is non-negative. Thus, have
T
2C (80)
1 T-1
V p
1uggrr (0') V VDDy 112.
(10)
T
uL
T
Note that we assume given D, we have 0r Oregr. Therefore, for any 0 there exist T. such that
Vt
Te ||0
@Aggr <ec. c.
(11)
This implies that Vt > Te: #1 1Agggg (8*) = || BAger + ÖAggr 8*ll ||0
8ngge + 11Aggr 8 < 2c.
(12)
Moreover, also implies Vt,, 12 Te:
V (8*1) VL [8*2)| S 7/ 8a2
(y-Lipschitzness)
2c.
(13)
16

---

Now, let's get back to 10 For AT >
T. we have
T
2
T.
1
T
[C)=
V
TAgger (0') - VLD: (0'))1
+ T
l/aggr (0') VVDD; (0¹)12
T
T
t=
0
1
Σ
TrASE (0') VLLDD (0*)11
T
T
0
+
1
Σ
1 (0') INeErr( (OnggE) + Trage (Onezz) VLD (09)
(10nesx (0*) Jagge (Onggr) )1 + 1[āggr (Ongg=) - VVLD; (0))1) 2
0
(triangle inequality)
0
(
1
Σ
(o(e) + llāmegr (Oneer) V CLD (0'))
(by (12)
T
0
+ +0(1) +
(Iagger (ONgEr) - VL (OAgEE) + V Dp; (OAgET) VL (0 (0¹)1)
T
T
0
(
+ O(e) + T
Σ T=
T-1
(lānger (Onggr) - VLDD (OKSEz )1 + +O(e)) 2
(by (3)
0
1 T
+ O(c) + 11Angge (Fnggr) - V CDp (Ongge) )1(
(14)
Equation 14 bounds the left hand side with the maximum l| (QAgrr) VLDy (OKgE)) I| one can get during the optimization steps. Here, we assume at time t', the largest value attained. We denote A Aggrr_axx as the delta error at time step E'.
Then, continue with what have done the beginning of the proof of this theorem:
1
4
O
+ O(E) + =((nngr))
(by 8
T
1
(()) + C. A. Siger nax + O(e/8))
S 5
-
(by 9
Therefore, combining the above we finally have: for VT > T. with probability at least 1 6,
1
Σ 0=
VC (V
( + Ole) + 8z (Vc. A Diggr max + o(())²
(15)
T
17

---

To complete the proof, let us investigate the left-hand side.
1
1
Σ
(0*) 112
Σ
LD::(00)1
T
T T=
1=0
1=
= =0
-
+
| Dz (09)1
(
T-1
0
Σ
(IVLD; -
|V )l))
+
(triangle inequality)
o 7
Σ
(O(e) + +1 CDV (0
(by
13
o
+ 0 | Dg )12
(16)
Combining
15) and (6).. we finally have
VVDg (0") (0")) =o (1) () O()++
(V A Aigerrma max + O(o))
which completes the proof.
A.2 Proof of Theorem
To prove Theorem 4.5 we first use the following definitions and lemmas from (Enyi Jiang 2024 to get the delta errors Gradient Projection (GP) and standard adversarial training (AT):
Definition A.3 (GP Aggregation). Let 8 [0, 1] be the weight that balances between Gn and Gn. The GP aggregation operation is
GP Imd = ((1 (1', + )Pr))_(
where Proj. (0.j_0aai (n) max (g.. 0,), 0)3,//10..11 is the operation that projects ga the positive direction of 9n- Definition A.4 (AT Aggregation). The AT aggregation operation
AT ga-
standard AT only leverages the gradient update on D...
Lemma A.S (Delta Error of GP). Given distributions D... Da and D,, as well as the model updates Tar Bas On these distributions per epoch, we have A as follows
&GP s (11-))+22)
23
82 m
B5, 9a - -gall + 32 Ig.
1n.
In the above equation, m is the model dimension and 72 = =2 E [0, 1] where T(0) is the sin(.) value of the angle between In and go.
Gn. || || the n-normm over the model parameter space.
Proof. The proof is the same as Theorem 4.4 in Enyi Jiang 2024
Lemma A.6 (Delta Error of AT). Given distributions D D and D.,. as well as the model updates Gar Ba, In on these distributions per epoch, we have AT as follows
A 2 = F. Il9. Ia
where 1|s is the x-norm over the model parameter space.
18

---

Then, we prove Theorem 4.5
Theorem A.7 (Error Analysis of GP), When the model dimension is large (m + oo) at time step t, we have
AT AP = 3(2 B) 119. - -all - 332 In l1
72 = BT|Td E [0,1] where is the sin(-) value of the angle between Fn and Da In. || 1| is the g-norm over the model parameter space.
Proof. A 4
Eb 19u 1
((1 3)2 + 28-0²
E. 119 -a -3272119. Inll
(11 B)² + 25-0² 8² )) BBg 11ga
Do 11
f² 11 Fn 11
m
=1+ 1, m ) 3 1
3 In 1
When m + 0o, we have simplified version of the error difference as follows
A AT - drp ~ (2 - 5) I1l.
gall - / - -n
Interpretation. When B = 0.5, can usually show AAr > Aop, because B(2 3) (0.75 > 0.25) for the coefficients two terms. We estimate the actual values terms Ep_ 119. 11 (variance), l1
2 (bias), and 7 using the estimation methods in Enyi Jiang [2024]. Table A displays the values of those terms well as the error differences on ResNet18 exper- iments at epoch 5. 10, 15. 20, 60. We plot the changing of these terms on the ResNet18 experiment in Figure The order of difference is always positive and usually smaller than 1e-08 088 and approaches the order of le 12 in the end.
Table 6: Estimations the actual values of terms 1T, 119. ÿa ll (variance), llg. In ll (bias),
and A AT A (error differences) across different epochs.
Terms / epochs
5
10
15
60
D... 1u 9n 1|
4.6017e-08
2.0448e-09
6.9623e-10
6.4329e-10
2.3849e-11
119a 12
0.0007
9.9098c-05
4.4932c-05
3.7930e-05
2.8391c-06
7
0.0071
0,0052
0.0036
0,0038
0.0030
A Gi
2.5335e-08
8.5709e-10
3.7609e-10
3,4487e-10
1.1574e-11
B Additional Experiment Information
In this section, we provide more training details, additional experiment results on the universal robustness RAMP common corruptions and unseen adversaries, runtime analysis of RAMP. additional ablation studies on different logit pairing losses, and AT from random initialization results on CIFAR-10 using WideResNNt-28-10..
B.1 More Training Details
We set the batch size to 128 for the experiments on ResNet-18 and WideResNNet28-11 architectures. We use an SGD optimizer with 0.9 momentum and 54-4 weight decay. For other experiments on ImageNet, we use batch size of 64 fit into the GPU memory for larger models. For all training procedures, we select the last checkpoint for the comparison. When the pre- pre-trained model was originally trained with extra data beyond the CIFAR-10 dataset, similar to Croce and Hein [2022), we use the extra 500k images introduced by Carmon et al. (2019) for fine-tuning, and each batch contains the same amount of standard and extra images. An epoch is completed when the whole standard training set has been used.
19

---

Variances
Blas
0,00303
0.00301
0.00203
1.0
26-30
Differences
0.3230
Air
+b
N
0.8-
0.2150
0.3323
M
0.5
0.4
0.0015
0.3050
0.2
0.0
100
Epechos
Esoches
Figure 5: Plot of values of terms EB_. 119, Pa 11 (variance), 119a In ll (bias), F, and 4 2 (error differences).
B.2 Runtime Analysis of RAMP
We present runtime analysis results demonstrating the fact that RAMP is more expensive than E-AT and less expensive than MAX in Table These results, recorded in seconds per epoch, were obtained using a single A100 40GB GPU. RAMP consistently supports that fact in all experiments.
Table Analysis of time per epoch for RAMP and related baselines. RAMP more expensive than
E-AT and less expensive than MAX.
Models Methhdd
E-AT [Croce and Hein
2022
MAX
RAMP
CIFAR- 10 RN-18 18 scratch
78
219
157
CIFAR-I0 10 WRN-28-10 10 scratch
334
1048
660
CIFAR-I0 10 RN-50
188
510
388
CIFAR-10 WRN-34-20
1094
2986
2264
CIFAR-10 WRN-28-100 10 carmon
546
1420
1110
CIFAR-10 WRN-28810 gowal
698
1895
1456
CIFAR-10 WRN-70-16
3486
10330
7258
ImageNet ResNet50
15656
41689
35038
ImageNet Transformer
38003
101646
81279
B.3 Additional Results on RAMP Generalizing to Common Corruptions and Unseen Adversaries for Universal Robustness
In this section, show RAMP generalize better other corruptions and unseen adversaries union accuracy for stronger universal robustness.
Implementations. For the ly attack, we use Croce et al. 2022 with an epsilon of 9 pixels and 5k: query points. For common corruptions, we directly use the implementation of RobustBench [Croce et al. 2020) for evaluation across 5 severity levels on all corruption types used in [Hendryckss and Dietterich 2019]. For other unseen adversaries, we follow the implementation of Laidlaw [2020]. where we set eps 12 for the fog attack, eps = 0.5 for the snow attack, eps 60 for the gabor attack, eps 0.125 for the elastic attack, and eps = 0. 12. for the jpeglinf attack with 100 iterations. For ResNet- 18 experiments, we do not compare with Winninghand [Diffenderfer et al.
al.
20

---

architecture. Also, we select the strongest baselines (E-AT and MAX) from the Wide-ResNet experiment results compare for ResNet-18 experiments on universal robustness.
2021] since it uses
Results. For the ResNet-18 18 training from scratch experiment CIFAR-10, in Table 8 and 9 also show RAMP generally outperforms by 0.5% on common corruptions and 7% on union accuracy against unseen adversaries compared with E-AT.
Table 8: Accuracy against common corruptions using ResNet- 18 on CIFAR-10 dataset.
Models common corruptions
E-AT
73.8
MAX
75.1
RAMP
74.3
Table 9: Individual, average, and union accuracy against unseen adversaries using ResNet-18 on CIFAR-10 dataset.
Models
40
fog
snow
gabor
elastic
ipeglinf
Avg
Union
E-AT
58.5
41.8
30.8
45.9
55.0
59.1
48.5
18.8
MAX
70.8
40.0
34.4
45.1
54.8
56.8
50.3
20.6
RAMP
56.8
40.5
40.5
50.0
59.2
56.2
50.5
25.9
B.4 Additional Experiments with Different Epsilon Values
In this section, we provide additional results with different é1, 62, Eoo values. We select 6oo 12 16
255 255 255 255 él [6, 9, 12, 15). and = [0.25, 0.75, 1.5] We provide additional RAMP results compared with related baselines with training from scratch and performing robust fine-tuning in Section I and Section B.4.2 respectively. We observe that RAMP surpass E-AT with significant margins as well as a better accuracy- iccuracy-robustness tradeoff for both training from scratch and robust fine-tuning with À 2.0 for training from scratch and 0.5 for robust fine-tuning in most cases.
2
B.4.1 Additional Results with Training from Scratch
Changing lo perturbations with 6x = RAMP consistently outperforms E-AT (Croce and Hein 2022 on union accuracy when training from scratch.
Table 10 and Table 11 show that
255 255
Table 10: (Exo initializations.
2 255
61
12,
0.5) and (Ex
12,
0.5)
with random
Clean
Loc
1
4
Union
Clean
100
h
Union
E-AT RAMP
87.2
73.3
64.1
55.4
55.4
E-AT
86.8
58.9
66.4
54.6
53.7
86.3
73.3
64.9
59.1
59.1
RAMP
86.1
60.0
67.4
58.5
57.4
12 Table 11: (eoo = 255
= 12,62
62
= 0.5) and (exo = 255
= 12, 62 = =0.5) with random
initializations.
Clean
Loc
12
4
Union
Clean
40c
1/
4
Union
E-AT RAMP
77.5
28.8
64.0
50.1
28.7
E-AT
69.4 4
18.8
58.7
47.7
18.7
73.7
34.6
59.1
38.9
33.3
RAMP
65.0
25.7
49.8
32.6
25.0
Changing 4 perturbations with e, [6. 9, 12, 15). Table 12 and Table 13 show that RAMP
consistently outperforms E-AT [Croce and Hein 2022 on union accuracy when training from scratch.
Changing perturbations with 62 (0.25,00 0. 0575, 1.0, 1.5). Table 14 and Table 15 show that RAMP
consistently outperforms E-AT CCooce and Hein 2022 on union accuracy when traming from scratch.
21

---

Table 12: (Ex0
255 61 = 6,
62
= 0.5) and (coo
253 61 = 9, 62 = 0.5) with random initializa-
tions.
Clean
1oc
12
4
Union
Clean
10c
12
41
Union
E-AT
85.5
43.1
67.9
63.9
42.8
E-AT
84.6
41.8
67.7
57.6
41.4
RAMP
83.8
48.1
63.0
51.2
46.0
RAMP
82.6
47.5
65.7
50.8
45.9
Table 13: (ex0 initializations.
255
15,
0.5) and (Ex:
2
255
8
18, 62
0.5)
with random
Clean
loc
1
1
Union
Clean
100
1
1
Union
E-AT
81.9
40.2
66.9
48.7
39.2
E-AT
81.0
39.8
65.8
44.3
38.0
RAMP
80.9
45.0
66.4
46.7
43.3
RAMP
79.9
43.5
65.7
45.0
41.9
Table 14: initializations.
(6so
255 6
12,
C2
0.25) and (cxc
255 9
12, C2
0.75) with random
Clean
1oc
12
4
Union
Clean
100
12
1
Union
E-AT
82.8
41.3
75.6
52.9
40.5
E-AT
83.0
41.2
57.6
53.0
40.5
RAMP
81.8
46.0
74.7
48.8
44.5
RAMP
81.9
46.1
56.9
48.7
44.5
Table 15: (6xo = 255
=
12, 62 = 1.0) and (<xc = 255 6 = 12, = 1.5) with random
initializations.
Clean
loc
12
1
Union
Clean
1oc
12
1
Union
B-AT RAMP
83.4
41.0
47.3
52.8
40.3
B-AT
83.5
41.0
25.5
52.9
25.5
RAMP
74.4
43.4
37.2
51.1
37.1
(A=5)
81.5
46.0
46.5
48.1
44.1
B.4.2
Additional Results with Robust Fine-tuning
Changing loc perturbations with 6 RAMP consistently outperforms E-AT [Croce and Hein 2022 on union accuracy when performing robust fine-tuning.
Table 16 and Table 17 show that
Table 16: (fox fine-tuning.
2 61
61
12,
0.5) and (foo
255 61
12,
0.5)
with robust
Clean
Loc
4
4
Union
Clean
Loc
4
Union
E-AT
86.5
74.8
66.7
57.9
57.9
E-AT
85.9
61.4
67.9
57.6
56.8
RAMP
85.8
74.0
66.2
60.1
60.1
RAMP
85.7
60.9
67.6
59.3
58.1
Table 17:
(6x
265:50
61
12, 62
0.5) and (c
255 6
12,
62
0.5) with robust
fine-tuning.
Clean
L0c
12
Union
Clean
100
1g
Union
E-AT RAMP
75.5
30.8
62.4
44.6
30.0
E-AT
68.7
20.7
56.1
42.1
20.5
74.0
33.6
59.7
38.5
31.9
RAMP
65.6
25.0
51.5
31.2
23.8
Changing 4 perturbations with 6 = [6,9, 9, 12, 15). Table 12 and Table 13 show that RAMP
consistently outperforms E-AT [Croce and Hein 2022 union accuracy when performing robust fine-tuning.
Changing /2 perturbations with [0.25, 0. 0..5.111 1.0, 1.5). Table 14 and Table 15 show that RAMP
consistently outperforms E-AT [Croce and Hein 2022 on union accuracy when performing robust fine-tuning.
22

---

Table 18: (6x0 255
61
6,
62
= 0.5)
and (Eso = 255
c1 = 9,
62
0.5) with robust fine-tunie tieettuing.
Clean
L0c
12
4
Union
Clean
10c
1
Union
R-AT
84.2
45.8
66.8
59.0
45.0
E-AT
83.1
44.9
67.2
52.6
43.2
RAMP
RAMP
82.5
47.1
66.0
49.9
44.8
(A=1,5)
83.0
48.7
63.5
51.7
46.4
Table 19: (foc tuning.
255
8
61
15,
62
0.5) and (E50
255 61
18,
62 = 0.5) with robust fine-
Clean
loc
1
1
Union
Clean
1oc
12
1
Union
E-AT
81.3
43.5
66.6
42.8
39.0
E-AT
81.3
38.9
66.6
45.0
37.5
RAMP
80.4
44.2
66.1
44.4
41.2
RAMP
80.7
40.6
66.3
43.5
38.8
Table 20: fine-tuning.
(Cso
253 6
12, c
0.25) and (cxo
255 6
12, c2
0.75) with robust
Clean
10c
12
11
Union
Clean
100
12
1
Union
E-AT
82.3
44.2
75.3
47.2
41.4
E-AT
83.0
43.5
58.1
46.5
40.4
RAMP
81.5
45.6
74.4
47.1
43.1
RAMP
81.4
45.6
57.4
47.2
42,9
Table 21: (Eoo tuning.
255
8
61
=
12,
62
1.0) and (Exo
8 253
= 12, = 1.5) with robust fine-
Clean
1.c
1
1,
Union
Clean
1.0
12
1
Union
E-AT RAMP
82.3
41.0
49.0
51.6
E-AT 42.9
40.2
80.2
42.8
31.5
52.4
31.5
81.4
45.6
47.8
47.1
RAMP
74.9
43.7
37.0
50.2
36.9
B.5
Different Logit Pairing Methods
In this section, we test RAMP with robust fine-tuning using two more different logit pairing losses:
(1) Mean Squared Error Loss (Emse) (Eq. (7). (2) Cosine-Similarity Loss (Loos) (Eq, (8). We replace the KL loss we used paper using the following losses. We use the same lambda value 1.5 for both cases.
Lise
(ppb1 Pr h(l)))
(17)
The
2
1=0
1 ne
Lcos
(1 (1l-cospp/hiii)---rtti))
(18)
The
==0
Table 22 displays RAMP robust fine-tuning results of different logit pairing losses using PreAct- ResNet- 18 on CIFAR-10 with 1.5. We see those losses generally improve union accuracy compared with baselines in Table 24] Loos has better clean accuracy yet slightly worsened union accuracy. Lmse has the best union accuracy and the worst clean accuracy. LK in the middle of the two others. However, we acknowledge the possibility that each logit pairing loss may have its own best-tuned À value.
Table 22: RAMP fine-tuning results of different logit pairing losses using 1re-cct-ReNNet118 on CIFAR-10. 10.
Losses
Clean
1.0
1
Union
KI.
80.9
45.5
66.2
47.3
43.1
MSE
80.4
45.6
65.8
47.6
43.5
Cosine
81.6
45.4
66.7
47.0
42.9
23

---

B.6 AT from Scratch Using 2ideResNet-28-10
Implementations. We use a cyclic learning rate with a maximum rate of 0.1 for 30 epochs and adopt the outer minimization trades loss from Zhang et al. [2019] with the default hyperparameters, same as Croce and Hein [022];; also, we set X = 2.0 and 0.5 for training RAMP. Additionally,
the WideResNNt-225 28-10 architecture same Zagoruyko and Komodakis [2016] for
reimplementations on CIFAR-10.
Results. Since the implementation of experiments WideResNet-28-I1 in Croce and Hein 2022 paper not public at present, we report our implementation results on E-AT, where our results show that RAMP outperforms E-AT in union accuracy with significant margin, as shown in Table 23 Also, we experiment with using the trade loss (RAMP w trades) for the outer minimization, we observe that RAMP trades achieves better union accuracy at the loss of some clean accuracy.
Table 23: WideResNeet22-N trained from random initialization on CIFAR-10 10. RAMP outper- forms E-AT union accuracy with our implementation. Methods
Clean
Icc
1
4
Union
E-AT w trades (reported in Croce and Hein 2022
79.9
46.6
66.2
56.0
46.4
E-AT w trades (ours)
79.2
44.2
64.9
54.9
44.0
RAMP w/o trades (ours)
81.1
46.6
65.9
48.1
44.6
RAMP trades (ours)
79.9
47.1
65.1
49.0
45.8
B.7 Robust Fine-tuning Using PreAct-ResNet-I
Implementations. For robust fine-tuning with ResNet-18 18, we perform 3 epochs on CIFAR-10. We the learning rate 0.05 for PreAct-ResNet- and 0.01 for other models. We set 0.5 in this case. Also, we reduce the learning rate by a factor of 10 after completing each epoch.
Result. Table 24 shows the robust fine-tuning results using PreAct ResNet-18 18 model the CIFAR-10 dataset with different methods. The results for all baselines are directly from the E-AT paper [Croce and Hein 2022] where the authors reimplemented other baselines (e.g., MSD, MAX) to achieve better union accuracy than presented in the original works. RAMP surpasses all other methods on union accuracy.
Table 24: RN-18 AT model fine-tuned for 3 epochs (repeated for 5 seeds). RAMP has the highest union accuracy. Baseline results from Croce and Hein 2022
Methods
Clean
1n0
1
Union
RN-18- AT
83.7
48.1
59.8
7.7
38.5
SAT
83.5 0.2
43.5 - 0.2
68.0 0.4
47.4
0.5
41.0 0.3
+ AVG
84.2 0.4
43.3 + 0.4
68.4 0.6
46.9
10.9:00
0.6
40.6 0.4
MAX
82.2 0.3
45.2
0.7
46.1
0.4
42.2
+ MSD
8..2
44.9
6..1
4.6
42.6
E-AT
0.4
0.6
0.5
0.5
4.8
RAMP (A=1.5)
81.1
45.4 0.3
66.1
4.12
43.1
RAMP 0.5)
81.5 + 0.1
45.5 + 0.2
66.4 0.2
4..0
42.9 + 0.2
B.8 Robust Fine-tuning with More Epochs
apply robust fine-tuning the PreAct ResNet-18 model for the CIFAR-10 dataset with 5, 7, 10, 15 epochs, and compare it with E-AT. RAMP consistently outperforms the baseline on union accuracy, with larger improvement when we increase the number of epochs.
In Table 25
C Additional Visualization Results
In this section, we provide additional t-SNE visualizations of the multiple-norm tradeoff and robust fine-tuning procedures using different methods.

---

Table 25: Fine-tuning with more epochs: RAMP consistently outperforms E-AT on union accuracy,
E-AT results are from Croce and Hein [2022]
epochs
epochs
10 epochs
15 epochs
Clean
Union
Clean
Union
Clean
Union
Clean
Union
E-AT
83.0
43.1
83.1
42.6
84.0
42.8
84.6
43,2
RAMP
81.7
43.6
82.1
43.8
82.5
44.6
83.0
44.9
C.1 Pre-trained 11,1, Lc AT models
6
6 shows the robust accuracy of I1, 12, lo AT models against their respect 41, 12, loe perturba-
tions, on CIFAR-10 using PreAct-ResNet- 18 architecture. Similar to Figure ??, AAT model has low I robustness and vice versa. this common choice of epsilons, we further confirm that to 41 is the key trade-off pair.
C.2 Robust Fine-tuning for all Epochs
We provide the complete visualizations of robust fine-tuning for epochs on CIFAR-10 10 using I examples, E-AT, and RAMP. Rows in i fine-tuning (Figure 7 E-AT fine-tuning (Figure 8), and RAMP fine-tuning (Figure 9 show the robust accuracy against Lo 41, l2 attacks individually, of epoch 0, 2, 2, respectively. We observe that throughout the procedure. RAMP manages to maintain more Ix robustness during the fine- sene-tuning with more points colored in cyan, in comparison with two other methods. This visualization confirms that after we identify lp (p & r) key tradeoff pair, RAMP successfully preserves more Ip robustness when training with some I, examples via enforcing union predictions with the logit pairing loss.
Lind
Linf AT
L1-AT
L2-AT
Figure 11,22, loc. pre-trained RN18 AT models with correct/incorrect predictions against h 12, loo attacks. Correct predictions are colored with cyan and incorrect with magenta. Each row represents l,, la AT models, respectively. Each column shows the accuracy concerning a certain lp attack.
25

---

Figure 7: Finetune RN18 -TT model on | examples for 3 epochs. Each row represents the prediction results of epoch 1, 2, 3 respectively.
Figure Finetune RN18 RN181.g-AA model with E-AT for epochs. Each row represents the prediction results of epoch 0, 1,2, respectively.
26

---

Lnf
Figure 9: Finetune RN18 1 -T model with RAMP for epochs. Each row represents the prediction results of epoch 1, 2, 3 respectively.
27

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the
paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction present the claims made in the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitations of our work in the discussion section of the paper.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We state the assumptions along with theoretical results in the theory part of the paper. The proofs are in the supplemental material.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide the implementations of our experiments in both main paper and supplementary materials.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide scripts to reproduce all experimental results for the new proposed method.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The full details are provided with the code and in appendix.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The results are accompanied by confidence intervals.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: We provide the information on the computer resources in the discussion section of the paper.
28

---

9. Code Of Ethics
Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpp:///maarrg...cccobbb/////ep
Question:
Answer: [Yes]
Justification: We conduct the research with the NeurlPS Code of Ethics.
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We mention the societal impacts of the work the discussion section of the
paper.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite the original paper that produced the code package or dataset.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provide README and scripts to run for the new asset we introduce. They are well documented.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects. 15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human
Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not include crowdsourcing or research involving human subjects.
29


Reviews:
[{"review_id": "JUmkDeTacj", "reviewer": "Reviewer_jXrr", "summary": "The paper introduces a new addition to the adversarial training literature -- it proposes to augment standard adversarial training [Madry et al.] with a specific logits pairing loss and, the existing technique, of gradient project. \n\nThe paper shows strong results on the union of lp-norm adversarial accuracy (over l-1, l-inf and l-2 using the standard CIFAR-10 dataset and standard perturbation radii).", "questions": "See above.", "limitations": "The authors have adequately addressed the limitations of their work.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper describes its contributions and motivates why each technique is introduced. The paper includes an informal discussion to motivate the logits pairing loss and some supporting theoretical analysis.\n\nThe paper obtains strong results (against many relevant baselines, evaluated correctly with AutoAttack) -- which is rare for adversarial robustness submissions. The method also seems straightforward to implement. Happy to recommend acceptance, based mainly on the strong results.", "weaknesses": "The one main weakness is that Table 1 must be re-generated using all the test set points in CIFAR-10, not just 1000.\n\nPresentation-wise, Figure 1 is very confusing; there is no need to plot t-SNEs of image when one wants to simply show classification labels. Please consider re-plotting this figure (with a line plot, histogram, etc.).\n\nThe writing, overall, is ok, but could also be improved: for example, lines 162-167, are approximately repeated from earlier on in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- *The one main weakness is that Table 1 must be re-generated using all the test set points in CIFAR-10, not just 1000.*\n\nThank you for the valuable suggestion. We have re-evaluated both RAMP and E-AT using the entire CIFAR-10 dataset. The results demonstrate that RAMP consistently enhances union accuracy compared to E-AT (up to 2.2%), corroborating the trend observed in our initial findings. We will update these numbers in the revised version of the paper.\n\n| Models               | Methods | Clean | linf | l2   | l1   | union    |\n|----------------------|---------|-------|------|------|------|----------|\n| RN-50                | E-AT    | 85.3  | 46.5 | 68.3 | 45.3 | 41.6     |\n|                      | RAMP    | 83.9  | 47.3 | 67.4 | 47.1 | **43.8** |\n| WRN-34-20            | E-AT    | 87.8  | 49.0 | 71.6 | 49.8 | 45.1     |\n|                      | RAMP    | 87.0  | 49.8 | 70.7 | 50.8 | **47.3** |\n| WRN-28-10 (*) carmon | E-AT    | 89.3  | 51.8 | 74.6 | 53.3 | 47.9     |\n|                      | RAMP    | 88.7  | 52.3 | 73.1 | 51.7 | **48.8** |\n| WRN-28-10 (*) gowal  | E-AT    | 89.8  | 54.4 | 76.1 | 56.0 | 50.5     |\n|                      | RAMP    | 89.1  | 55.1 | 74.5 | 54.8 | **52.1** |\n| WRN-70-16 (*)        | E-AT    | 89.6  | 54.4 | 76.7 | 58.0 | 51.6     |\n|                      | RAMP    | 90.1  | 55.1 | 74.5 | 57.4 | **53.4** |\n\n- *Presentation-wise, Figure 1 is very confusing; there is no need to plot the t-SNEs of the image when one wants to show classification labels simply. Please consider re-plotting this figure (with a line plot, histogram, etc.). The writing, overall, is ok, but could also be improved: for example, lines 162-167, are approximately repeated from earlier on in the paper.*\n\nThanks for the suggestions on presentation and writing. We replot Figure 1 using histograms on accuracy (included in the rebuttal pdf), using CIFAR-10 training data: the x-axis represents the robustness against different attacks and the y-axis is the accuracy. The figure shows that after 1 epoch of finetuning on $l_1$ examples or performing EAT, we lose much $l_\\infty$ robustness since blue/yellow histograms are much lower than the red histogram under the Linf category. In comparison, RAMP preserves both $l_\\infty$ and union robustness more effectively compared to E-AT and fine-tuning on $l_1$ examples; the green histogram is higher than the red/yellow histogram under Linf and Union categories. Specifically, RAMP maintains 14%, and 28% more union robustness than E-AT and $l_1$ fine-tuning. Besides, we will make writing more concise in the updated version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jH2H8hCpto", "reviewer": "Reviewer_6TV5", "summary": "This work proposes a training framework RAMP, to boost the robustness against multiple $\\ell_p$ perturbations, through connecting NT with AT via gradient projection.", "questions": "ref weakness", "limitations": "As I mentioned before, in my opinion, I think this work needs to discuss more about the essential meaning of the problem.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Easy to follow and clear presentation\n- Enough details\n- Some theoretical analyses\n- Outwardly, this is a good paper", "weaknesses": "My main concern is why the distributions of adversarial examples generated using $\\ell_1$, $\\ell_2$, and $\\ell_\\infty$ norms are distinct?\nIs this due to inconsistent (unreasonable) radius settings across different $\\ell_p$-norms?\nWhat constitutes a reasonable radius setting for various norms?\nThese norms can be bounded by each other theoretically, what is the essential meaning to study this problem?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **The essential meaning of studying this problem:** \n\na) In real-world settings, adversarial attacks often involve multiple types of perturbations, necessitating models to be robust against $l_1$, $l_2$, and $l_\\infty$ attacks. For instance, in image recognition: $l_1$ attacks modify key pixels to bypass facial recognition; $l_2$ attacks apply slight blur or noise to fool object detection; $l_\\infty$ attacks adjust color balance or contrast to cause misclassification. These examples highlight the need for comprehensive adversarial training. Robustness against multiple $l_p$ perturbations is critical, as it mirrors real-world scenarios where adversaries use a blend of techniques, ensuring the security and reliability of machine learning systems.\t\n\nb) Training for multiple norm robustness allows generalization beyond the threat model used during training [1]. Our method achieves superior average and union accuracy compared to the SOTA common corruption model [7], $l_p$ pretrained models, E-AT, and MAX across other perturbations beyond multiple-norm perturbations. Specifically, it obtains good accuracy against common corruptions with five severity levels and shows the best average and union accuracy performance against multiple unseen adversaries. Using WideResNet 28-10 models, we observe the following results:\n\nFor common corruptions, RAMP achieves a 4% improvement compared to E-AT and MAX, averaged across five severity levels and all common corruptions.\n\n| model           | common corruptions |\n|-----------------|--------------------|\n| l1-AT           |               78.2 |\n| l2-AT           |               77.2 |\n| linf-AT         |               73.4 |\n| winninghand [7]  |           **91.1** |\n| E-AT  |               71.5 |\n| MAX |                 71 |\n| RAMP |               75.5 |\n\nFor L0 and other unseen adversaries, RAMP shows the best average and union accuracy, with a 4% improvement over E-AT and a 2% improvement over MAX.\n\n| model           | l0 (eps=9,query=5k) | fog (eps=128) | snow (eps=0.5) | gabor (eps=60) |  elastic (eps=0.125) | jpeg(eps=0.125) | avg      | union    |\n|-----------------|---------------------|---------------|----------------|----------------|----------------------|-----------------|----------|----------|\n| l1-AT           |                  79 |          41.4 |           22.9 |           40.5 |                 48.9 |            48.4 |     46.9 |     12.8 |\n| l2-AT           |                67.5 |          48.7 |           26.1 |           44.1 |                 53.2 |            45.4 |     47.5 |     16.2 |\n| linf-AT         |                55.5 |          44.7 |           32.9 |           53.8 |                 56.6 |            33.4 |     46.2 |     19.1 |\n| winninghand [7]  |                74.1 |          74.5 |           18.3 |           76.5 |                 12.6 |               0 |     42.7 |        0 |\n| E-AT  |                58.5 |          35.9 |           35.3 |           50.7 |                 55.7 |            60.3 |     49.4 |     21.9 |\n| MAX |                56.2 |          42.9 |           35.4 |           49.8 |                 57.8 |            55.7 |     49.6 |     24.4 |\n| RAMP |                55.5 |          40.5 |           40.2 |           52.9 |                 60.3 |            56.1 | **50.9** | **26.1** |\n\n\nThese results highlight the effectiveness of our method in enhancing model robustness across diverse adversarial scenarios.\n\n- **Why are distributions distinct?** The distributions are distinct because these norms measure distances differently, resulting in different adversarial example distributions. Successful defenses are often tailored to specific perturbation types, providing empirical or certifiable robustness guarantees for those types alone. These defenses typically fail to offer protection against other attacks [1]. Moreover, enhancing robustness against one perturbation type can sometimes increase vulnerability to others [4,5], which further confirms the distinction between multiple-norm distributions.\n\n- **What constitutes a reasonable radius setting?** The values we are using are standard in the previous literature [1,2,3]; the epsilon values are chosen so that input images can be adversarially perturbed to be misclassified but are “close enough” to the original example to be imperceptible to the human eye [3]. Also, in Table 2 and Appendix Tables 5-16, we show RAMP works not only for the standard choice of epsilons but varying values of epsilons. The reasonable radius setting for various norms should make each pair of two perturbations not include one another. For given epsilon values of $\\epsilon_1$, $\\epsilon_2$, $\\epsilon_\\infty$ and $d$ (dimension of the inputs), we need the following relationships to achieve this:\n$$0 \\leq \\epsilon_1 \\leq d \\cdot \\epsilon_\\infty$$\n$$0 \\leq \\epsilon_2 \\leq \\sqrt{d} \\cdot \\epsilon_\\infty$$\n$$\\frac{\\epsilon_1}{\\sqrt{d}} \\leq \\epsilon_2 \\leq \\epsilon_1$$\n\nWe check the standard choices of CIFAR10 $\\epsilon_1=12, \\epsilon_2=0.5, \\epsilon_\\infty=\\frac{8}{255}, d = 3 \\cdot 32^2$ and Imagenet $\\epsilon_1=255, \\epsilon_2=2, \\epsilon_\\infty=\\frac{4}{255}, d = 3 \\cdot 224^2$. For CIFAR10, we have $0 \\leq 12 \\leq 3072\\cdot\\frac{8}{255}, 0 \\leq 0.5 \\leq \\sqrt{3072}\\cdot\\frac{8}{255}, \\frac{12}{\\sqrt{3072}} \\leq 0.5 \\leq 12$. For Imagenet, we have $0 \\leq 255 \\leq 150528\\cdot\\frac{4}{255}, 0 \\leq 2 \\leq \\sqrt{150528}\\cdot\\frac{4}{255}, \\frac{255}{\\sqrt{150528}} \\leq 2 \\leq 255$. Both hold so our choices make three perturbations not include one another. However, **the key tradeoff pair always exists**. The two perturbations with the largest volumes refer to the strongest attack as the attacker has more search area.\n\n[1] https://arxiv.org/abs/2105.12508\n\n[2] https://arxiv.org/abs/1904.13000\n\n[3] https://arxiv.org/abs/1909.04068\n\n[4] https://openreview.net/forum?id=BJfvknCqFQ\n\n[5] https://arxiv.org/abs/1805.09190\n\n[6] https://arxiv.org/abs/1903.12261\n\n[7] https://arxiv.org/abs/2106.09129", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Df0RlKauMr", "reviewer": "Reviewer_3Hj9", "summary": "The paper proposes a framework called RAMP to boost the adversarial robustness against multiple $l_p$ perturbations. Compared with existing methods, RAMP can better retain the robustness against the original type of perturbations when fine-tuning the model. The authors provide theoretical and empirical justifications to demonstrate the effectiveness of RAMP.", "questions": "The weakness part points out the major questions I would ask. Below are additional relatively minor questions:\n\n1. Why don't use AutoAttack as the evaluation metric in Figure 2?\n\n2. In Figure 4, what is the difference between MAX and RAMP with $\\lambda = 0$?\n\nMy major concern of this paper is about the novelty, because both the motivation (logit pairing) and many parts of the analysis are standard. In addition, the algorithm is not generic, as it is only evaluated under $l_1$, $l_2$ and $l_\\infty$ bounded perturbations. Therefore, I cannot recommend acceptance at this point. I welcome the authors to address my concerns in the rebuttal.", "limitations": "The limitations and the societal impact are briefly discussed in the end of the paper.", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "++ The targeted problem is well-motivated and formulated. \n\n++ The experimental results are comprehensive with reliable evaluation metrics.\n\n++ Theoretical justifications can better explain the effectiveness of the proposed methods.", "weaknesses": "Despite the strength in the last section, I have the following concerns:\n\n1. Novelty in the algorithm: logit pairing is not a new idea. Actually, in Equation (1), if we replace $p_r$ with the probability of the clean input, the algorithm will become TRADES. Some findings, such as trade-offs between accuracy and robustness, are not new.\n\n2. Generality of the algorithm: the manuscript only studies the joint robustness among $l_1$, $l_2$ and $l_\\infty$ perturbations, especially the $l_1$ and $l_\\infty$ ones. This is too limited. The authors should consider more generic cases, such as $l_0$ bounded perturbations and common corruptions (Gaussian noise, color re-mapping e.t.c.). I believe this will make the algorithm more generally applicable.\n\n3. Novelty and precision of the theorem: the theoretical tools employed to analyze convergence in Theorem 4.5 are standard and not unique. In Theorem 4.6, if $\\beta = 0$, $\\widehat{g}\\_a$ should be similar to $g_{GP}$, so $\\Delta_{AT}$ should be very close to $\\Delta_{GP}$. Furthermore, the left hand side of the approximation should be close to $0$, can the author explain how the right hand side approximation reflects this? In addition, using $\\simeq$ is not rigorous, the authors should at least point out the order of the difference between the left hand side and the right hand side.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **Novelty in the algorithm**: Firstly, $p_r$ represents the predictions of $l_r$ adversarial examples, not the probability of the clean input. Our logit pairing method differs from TRADES as we only regularize the logits on the correctly predicted $l_r$ adversarial example subset, which TRADES does not consider. This approach enhances union accuracy by aligning the logit distributions of $l_r$ and $l_q$ attacks on the correctly predicted $l_r$ subset. Additionally, in Appendix Table 18, we demonstrate that combining RAMP with TRADES loss improves union accuracy to 45.2% (up from 44.6%) in the WideResNet-28-10 experiment training from scratch. Secondly, Our gradient projection method is more generalizable than traditional methods mitigating the trade-off between accuracy and robustness, such as TRADES [1], MART [2], and HAT [3]. Previous works limit to a certain type of perturbation for better robustness-accuracy tradeoff. However, our gradient projection method is effective for both single (Figures 2 and 3) and multiple perturbations/corruptions (Table 2 and 3, also see in our rebuttal to Reviewer Qhq5).\n\n- **Generality of the algorithm**: \nTraining for multiple norm robustness allows generalization beyond the threat model used during training [1]. Our method achieves superior average and union accuracy compared to the SOTA common corruption model [5], $l_p$ pretrained models, E-AT, MAX across other perturbations beyond multiple-norm perturbations. Specifically, it obtains good accuracy against common corruptions with five severity levels and shows the best average and union accuracy performance against multiple unseen adversaries. Using WideResNet 28-10 models, RAMP achieves a 4% improvement over E-AT and MAX on common corruptions, averaged across five severity levels. For other attack types, such as $l_0$ and unseen adversaries, RAMP shows the best average and union accuracy, with a 4% improvement over E-AT and 2% over MAX. These results highlight that RAMP effectively enhances model robustness across diverse adversarial scenarios.\n\n| model           | common corr. | l0 (eps=9,query=5k) | fog (eps=128) | snow (eps=0.5) | gabor (eps=60) |  elastic (eps=0.125) | jpeg(eps=0.125) | avg      | union    |\n|-----------------|--------------------|---------------------|---------------|----------------|----------------|----------------------|-----------------|----------|----------|\n| l1-AT           |               78.2 |                  79 |          41.4 |           22.9 |           40.5 |                 48.9 |            48.4 |     46.9 |     12.8 |\n| l2-AT           |               77.2 |                67.5 |          48.7 |           26.1 |           44.1 |                 53.2 |            45.4 |     47.5 |     16.2 |\n| linf-AT         |               73.4 |                55.5 |          44.7 |           32.9 |           53.8 |                 56.6 |            33.4 |     46.2 |     19.1 |\n| winninghand [5]  |           **91.1** |                74.1 |          74.5 |           18.3 |           76.5 |                 12.6 |               0 |     42.7 |        0 |\n| E-AT  |               71.5 |                58.5 |          35.9 |           35.3 |           50.7 |                 55.7 |            60.3 |     49.4 |     21.9 |\n| MAX |                 71 |                56.2 |          42.9 |           35.4 |           49.8 |                 57.8 |            55.7 |     49.6 |     24.4 |\n| RAMP |               75.5 |                55.5 |          40.5 |           40.2 |           52.9 |                 60.3 |            56.1 | **50.9** | **26.1** |\n\n- **Novelty and precision of the theorem**:\n\nThe novelty of Theorem 4.5 lies in its consideration of the changing adversarial example distribution $D_{a^t}$ at each time step $t$ to get the convergence proof. To sum over $t=0,\\dots,T-1$, we perform some relaxations to obtain the summation for changing distrbutions. Also, for Equation 14, we bound the left-hand side with the maximum $\\|\\widehat{g_{\\texttt{Aggr}}}(\\widehat{\\theta_{\\texttt{Aggr}}})-\\nabla L_{D_{a^t}}(\\widehat{\\theta_{\\texttt{Aggr}}})\\|^2$ one can get during the optimization steps. Theorem 4.5 links the convergence of the aggregation rule with the delta error analysis in Theorem 4.6. Specifically, Theorem 4.6 demonstrates that a smaller delta error results in better convergence with a tighter bound.\n\nFurther, we have carefully checked the theorem and Theorem 4.6 will be updated as follows:\n\n$\\Delta^2_{AT} -\\Delta^2_{GP} \\approx \\beta (2 - \\beta) E_{\\widehat{D_{a^t}}} \\|g_{a} -\\widehat{g_{a}} \\|^2_\\pi - \\beta^2 \\bar\\tau^2 \\|g_{a} - \\widehat{g_{n}}\\|^2_\\pi$\n\nWhen $\\beta = 0$, the difference between two aggregation errors becomes 0. Further, we estimate the actual values of terms $E_{\\widehat{D_{a^t}}} \\|g_{a} - \\widehat{g_{a}} \\|^2_\\pi$ (variance), $\\|g_{a} -\\widehat{g_{n}}\\|^2_\\pi$ (bias), and $\\bar\\tau$ using the estimation methods in [4]. We show across different epochs, the delta error difference **is always positive**, largely because of the small value of $\\bar\\tau$. We plot the changing of these terms on the ResNet18 experiment with the table and figure in the rebuttal pdf. **The order of difference** is usually smaller than 1e-8 and approaches the order of 1e-12 in the end. \n\n- **The evaluation metric in Figure 2.** We want to show our methods can show a better robustness and accuracy tradeoff for both PGD and AutoAttack for a more comprehensive evaluation. As shown in Figure 3, gradient projection also shows a better robustness accuracy tradeoff evaluated by AutoAttack.\n\n- **The difference between MAX and RAMP with lambda=0**. RAMP with $\\lambda = 0$ and not considering the key tradeoff pair will become the same as MAX. Therefore, MAX is a special case of RAMP. \n\n[1] https://arxiv.org/abs/1901.08573\n\n[2] https://openreview.net/forum?id=rklOg6EFwS\n\n[3] https://openreview.net/forum?id=Azh9QBQ4tR7\n\n[4] https://arxiv.org/abs/2302.05049\n\n[5] https://arxiv.org/abs/2106.09129", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1Ob40S8pYO", "reviewer": "Reviewer_Qhq5", "summary": "This paper addresses the challenge of training Machine Learning (ML) models that show good robustness against multiple l_p attacks and accuracy. This problem has already been studied in previous work [33,6], but the papers discuss the problem from the lens of distribution shift. This point of view represents the starting point for proposing a new adversarial training (AT) algorithm called RAMP based on two building blocks: (i) a new logit pairing loss that allows a model robust against a l_p norm to exhibit high robustness w.r.t. another l_q norm (with q \\neq p); (ii) a new model parameter update based on the combinations of model updates from natural training and adversarial training to train a model that observes a better accuracy/union robustness tradeoff that SOTA AT models. A deep theoretical analysis shows that, for large models and a reasonable choice of parameter values, the novel parameter update allows the adversarial training algorithm to converge faster, training a model with higher robustness and accuracy. The extensive experimental evaluation highlights that the trained large models on ImageNet and Cifar-10 exhibit a better accuracy-union robustness tradeoff than SOTA AT proposals [6].", "questions": "- Can the authors show more results supporting the fact that the time per epoch required by RAMP is partially longer than the time per epoch required by E-AT? (See the Weaknesses section for more details)\n\n- Is it possible to use RAMP to obtain an accuracy close to the one obtained by using E-AT and a better union accuracy? (See the Weaknesses section for more details)\n\n**Updates after authors' response**: The authors provided additional results to address the first question, effectively addressing the first weakness. After discussion, the answer to the second question is more convincing. RAMP seems to address the accuracy-robustness trade-off better than E-AT even by choosing small values of $\\lambda$, as the authors wrote. So, choosing a small value of $\\lambda$ helps preserve the accuracy while still outperforming the models trained with E-AT regarding union accuracy. This choice could be right in general. \n\nReasons for Not Giving a Higher Score: RAMP does not allow the user to obtain an impressive gain in union accuracy as E-AT did when the corresponding paper was published. However, it allows the model to obtain a few points more of union accuracy while preserving an accuracy similar to that of models trained with E-AT. I can say that this paper will have a moderate to high impact, but I am not convinced that it will have a high impact for sure.\n\nReasons for Not Giving a Lower Score: In my opinion, the work is solid from a theoretical and methodological point of view. The experimental results are more convincing after the discussion with the authors.", "limitations": "The authors have explained the limitations in Section 5.2 in brief. However, the limitations should be addressed better:\n- The analysis of the computational cost should be supported with more experimental results.\n- The authors should better substantiate the sentence ``` We notice occasional drops in clean accuracy during fine-tuning with RAMP```. Indeed, the drop in accuracy is exhibited also when the model is trained from scratch (see Section B.2.1 in the Appendix). Moreover, the gain in the union accuracy is often comparable to the loss in clean accuracy. Thus, the accuracy/robustness trade-off seems not to be properly addressed.\n\nThis work has no negative societal and ethical impacts since it contributes to improving the robustness of AI systems in practice. The experimental and implementation details have been documented.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "**Original AT algorithm to address the accuracy-union robustness trade-off**: the novel AT training addresses both the problem of improving the robustness on multiple l_p perturbations and improving the accuracy-robustness trade-off at the same time by exploiting a novel connection with natural training. Addressing the two problems at the same time constitutes the novelty of the work.\n\n**Theoretical analysis of the impact of gradient projection on adversarial robustness**: the deep theoretical analysis shows that integrating model parameter updates from natural training in the adversarial training updates allows the training to converge better for large models.\n\n**Comprehensive experimental settings considered**: the experimental evaluation is performed on full training and fine-tuning large models used by relevant competitors in the literature [6], on significant datasets (CIFAR-10 and ImageNet) and considering several combinations of perturbation values for l_inf, l_1, and l_2.", "weaknesses": "**Analysis of time per epoch is not opportunely deepened (addressed by the authors)**: the analysis regarding the computational time per epoch is limited since the results are shown for only one model (ResNet-18) on only one dataset (CIFAR-10), but seven other models are employed in the experimental evaluation. Since a lot of additional results have been included in the Appendix, I would have expected to see also more results supporting the statement ``` RAMP has around two times the runtime cost of E-AT and 2/3 cost of MAX``` (Section 5.2). For example, the authors may want to report the time per epoch on AT the ResNet-18 and the WRN-28-10 from scratch on CIFAR-10 (as reported in [6]) and on training another model on ImageNet.\n\n**Trade-off between robustness and accuracy seems to not be properly addressed**: Even though RAMP allows the model to obtain better union accuracy, the clean accuracy is in any case lower than the clean accuracy obtained by using the baseline E-AT [6]. As an example, the RAMP fine-tuned RN-50-$\\ell_\\infty$ shows a clean accuracy that is 3.8 points lower than the one obtained by the E-AT fine-tuned model and a gain in union accuracy that is one point (Table 1). Moreover, more than half of the results shown in the Appendix and some results in the main body show that the gain in the union accuracy is obtained at the expense of a comparable loss in the clean accuracy. If the trade-off between robustness and accuracy had been properly addressed, I would have expected to see a comparable clean accuracy and better union accuracy than the one obtained by E-AT. These results limit the novelty of RAMP since it seems to be slower than E-AT and only allows the model to obtain better union accuracy without properly addressing the accuracy/robustness trade-off. I suggest that the authors explain why these cases happen and how to improve also the accuracy of the obtained model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- **Analysis of time per epoch**: We present additional results demonstrating the fact that RAMP is more expensive than E-AT and less expensive than MAX. These results, recorded in seconds per epoch, were obtained using a single A100 GPU. RAMP consistently supports that fact in all experiments. \n\n| Models \\ Methods         | E-AT  | MAX    | RAMP  |\n|--------------------------|-------|--------|-------|\n| CIFAR RN-18 scratch      | 78    | 219    | 157   |\n| CIFAR WRN-28-10 scratch  | 334   | 1048   | 660   |\n| CIFAR RN-50              | 188   | 510    | 388   |\n| CIFAR WRN-34-20          | 1094  | 2986   | 2264  |\n| CIFAR WRN-28-10 carmon   | 546   | 1420   | 1110  |\n| CIFAR WRN-28-10 gowal    | 698   | 1895   | 1456  |\n| CIFAR WRN-70-16          | 3486  | 10330  | 7258  |\n| ImageNet ResNet50        | 15656 | 41689  | 35038 |\n| ImageNet Transformer     | 38003 | 101646 | 81279 |\n\n- **The trade-off between robustness and accuracy**: \n\n**It is possible to use RAMP to obtain an accuracy close to the one obtained by E-AT and a better union accuracy by setting a small $\\lambda$ value**. The clean accuracy drops because our main goal is to improve union accuracy. To improve the clean accuracy of the obtained model, we can set a smaller $\\lambda$ value (as shown in Figure 4(a) (b)). For instance, in Figure 4(b), when $\\lambda=1$, RAMP has a clean accuracy of 81.9% and union accuracy of **44.4%**, compared with E-AT's 82.2% and 42.4%. Thus, one can usually achieve an accuracy close to E-AT and a better robustness by setting a relatively small $\\lambda$ value.\n\nFurther, we show as follows that **RAMP obtains a better robustness and accuracy tradeoff by generalizing better to common corruptions and other unseen adversaries, and the overall gain in robustness is more than the drop in accuracy**. Compared with SOTA common corruption model [2] and other pretrained $l_p$ models on wideresnet-28-10 experiment training from scratch on CIFAR-10, RAMP shows the best tradeoff between robustness and accuracy against various kinds of perturbations and corruptions. \n\nFor common corruptions, RAMP achieves a 4% improvement compared to E-AT and MAX, averaged across five severity levels and all common corruptions.\n\n| model           | common corruptions |\n|-----------------|--------------------|\n| l1-AT           |               78.2 |\n| l2-AT           |               77.2 |\n| linf-AT         |               73.4 |\n| winninghand [2]  |           **91.1** |\n| wideresnet E-AT  |               71.5 |\n| wideresenet MAX |                 71 |\n| wideresnet RAMP |               75.5 |\n\n\nFor other attack types, such as $l_0$ and other unseen adversaries, RAMP shows the best average and union accuracy, with a 4% improvement over E-AT and a 2% improvement over MAX.\n\n| model           | l0 (eps=9,query=5k) | fog (eps=128) | snow (eps=0.5) | gabor (eps=60) |  elastic (eps=0.125) | jpeg(eps=0.125) | avg      | union    |\n|-----------------|---------------------|---------------|----------------|----------------|----------------------|-----------------|----------|----------|\n| l1-AT           |                  79 |          41.4 |           22.9 |           40.5 |                 48.9 |            48.4 |     46.9 |     12.8 |\n| l2-AT           |                67.5 |          48.7 |           26.1 |           44.1 |                 53.2 |            45.4 |     47.5 |     16.2 |\n| linf-AT         |                55.5 |          44.7 |           32.9 |           53.8 |                 56.6 |            33.4 |     46.2 |     19.1 |\n| winninghand [2]  |                74.1 |          74.5 |           18.3 |           76.5 |                 12.6 |               0 |     42.7 |        0 |\n| wideresnet E-AT  |                58.5 |          35.9 |           35.3 |           50.7 |                 55.7 |            60.3 |     49.4 |     21.9 |\n| wideresenet MAX |                56.2 |          42.9 |           35.4 |           49.8 |                 57.8 |            55.7 |     49.6 |     24.4 |\n| wideresnet RAMP |                55.5 |          40.5 |           40.2 |           52.9 |                 60.3 |            56.1 | **50.9** | **26.1** |\n\nFor Resnet-18 training from scratch experiment on CIFAR-10, we also show RAMP outperforms by 0.5% on common corruptions and 7% on union accuracy against unseen adversaries compared with E-AT.\n| model | common corruptions |\n|-------|--------------------|\n| E-AT   |               73.8 |\n| RAMP  |           **74.3** |\n\n| model | l0 (eps=9,query=5k) | fog (eps=128) | snow (eps=0.5) | gabor (eps=60) |  elastic (eps=0.125) | jpeg(eps=0.125) | avg      | union    |\n|-------|---------------------|---------------|----------------|----------------|----------------------|-----------------|----------|----------|\n| E-AT   |                58.5 |          41.8 |           30.8 |           45.9 |                   55 |            59.1 |     48.5 |     18.8 |\n| RAMP  |                56.8 |          40.5 |           40.5 |             50 |                 59.2 |            56.2 | **50.5** | **25.9** |\n\nThese results highlight that RAMP effectively enhances model robustness across diverse adversarial scenarios and attains a better robustness/accuracy tradeoff evaluated on various kinds of perturbations and corruptions.\n\n\n[1] Adversarial Robustness against Multiple and Single lp-threat Models via Quick Fine-Tuning of Robust Classifiers.\n\n[2] A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank the reviewers for their valuable feedback and constructive suggestions. We hope our responses have adequately addressed the questions and concerns raised by the reviewers. Here we summarize our response to the main concerns of the reviewers.

- **The essential meaning of this problem** (Reviewer 6TV5)

**Enhancing robustness against one type is not enough**: Successful defenses are often tailored to specific perturbation types, providing empirical or certifiable robustness guarantees for those types alone. These defenses typically fail to offer protection against other attacks [1]. Moreover, enhancing robustness against one perturbation type can sometimes increase vulnerability to others [4,5]. 

**The choices of epsilon values are standard and comprehensive**: The values we are using are standard in the previous literature [1,2,3]; the epsilon values are chosen so that input images can be adversarially perturbed to be misclassified but are “close enough” to the original example to be imperceptible to the human eye [3]. Also, in Table 2 and Appendix Table 5-16, we show that RAMP works not only for the standard choice of epsilons but also for the varying values of epsilons.

**Training a model to be robust against multiple $l_p$ perturbations is crucial as it mirrors real-world scenarios where adversaries use a blend of techniques**. In image recognition, an $l_1$ attack might modify key pixels to bypass facial recognition, an l_2 attack could apply slight blur or noise to fool an object detection system, and an $l_\infty$ attack might adjust the color balance or contrast to cause misclassification. This holistic approach enhances the resilience, security, and reliability of machine learning systems in practical applications.

- **The novelty and generality of RAMP** (Reviewer 3Hj9 and 6TV5)

**Logit pairing is not the same as TRADES [6]**: $p_r$ represents the predictions of $l_r$ adversarial examples, not the clean input probability. Our logit pairing method differs from TRADES by regularizing only the logits on the correctly predicted $l_r $adversarial subset, enhancing union accuracy by aligning $l_r$ and $l_q$ attack logit distributions. Combining RAMP with TRADES loss improves union accuracy to 45.2% from 44.6% in the WideResNet-28-10 experiment.

**New findings on accuracy and robustness tradeoff**: Our gradient projection method generalizes better than traditional methods like TRADES [6], MART [7], and HAT [8], which focus on specific perturbations. Our approach is effective for both single and other various types of perturbations, offering improved robustness-accuracy tradeoff.

- **Generality of RAMP**: Training for multiple norm robustness allows generalization beyond the threat model used during training [1]. Our method achieves superior average and union accuracy compared to $l_p$ pretrained models, [9], E-AT, and MAX across other perturbations beyond multiple-norm perturbations. Compared with MAX and E-AT, it excels 4% in accuracy against common corruptions with five severity levels and shows 2-4% better union accuracy against multiple unseen adversaries on the WideResNet experiment (Tables in response to reviewer 3Hj9), showing our method’s robustness and adaptability to diverse adversarial scenarios.

- **Robustness-accuracy tradeoff** (Reviewer Qhq5): It is possible to use RAMP to obtain an accuracy close to E-AT and a better union accuracy by setting a small $\lambda$. For instance, in Figure 4(b), when $\lambda=1$, RAMP has a clean accuracy of 81.9% and union accuracy of **44.4%**, compared with E-AT's 82.2% and 42.4%. Further, RAMP obtains a better robustness-accuracy tradeoff by generalizing better to common corruptions and other unseen adversaries, and the overall gain in robustness is more than the drop in accuracy. Compared with E-AT in the WRN-28-10 and RN-18 experiments on CIFAR-10, RAMP shows a better tradeoff between robustness and accuracy against various kinds of perturbations and corruptions (Tables in response to reviewer Qhq5).

- **New time complexity results, generated results of Table 1 using all test points, regenerated Figure 1** (Reviewer Qhq5 and jXrr). 

a) We present additional time cost results on CIFAR-10 and Imagenet for wideresnet and transformer experiments, supporting that RAMP is more expensive than E-AT and less expensive than MAX. 

b) We have re-evaluated both RAMP and E-AT using the entire CIFAR-10 dataset (see Table in response to reviewer jXrr). RAMP consistently enhances union accuracy compared to E-AT (up to 2.2%), corroborating the trend observed in our initial findings. Also, we observe that the gain in the union accuracy is usually larger than the loss in the clean accuracy.

c) We replot Figure 1 using histograms on accuracy (included in the rebuttal pdf): RAMP maintains 14%, and 28% more union robustness than E-AT and $l_1$ fine-tuning after 1 epoch of robust fine-tuning on CIFAR-10 training data.
	
- **Novelty and precision of the theory** (Reviewer 3Hj9) : The novelty of Theorem 4.5 lies in its consideration of the changing adversarial example distribution $\mathcal{D}_{a^t}$ at each time step $t$ to get the convergence proof. Theorem 4.5 links the convergence of the aggregation rule with the delta error analysis in Theorem 4.6, showing that a smaller delta error leads to better convergence with a tighter bound. Further, we updated Theorem 4.6 and plotted the changing of delta error differences in the rebuttal PDF to show GP always has a smaller delta error than standard adversarial training, which supports the experimental results.

[1] https://arxiv.org/abs/2105.12508

[2] https://arxiv.org/abs/1904.13000

[3] https://arxiv.org/abs/1909.04068

[4] https://openreview.net/forum?id=BJfvknCqFQ

[5] https://arxiv.org/abs/1805.09190

[6] https://arxiv.org/abs/1901.08573

[7] https://openreview.net/forum?id=rklOg6EFwS 

[8] https://openreview.net/forum?id=Azh9QBQ4tR7 

[9] https://arxiv.org/abs/2106.09129

Meta-review: This paper studies the question of jointly achieving robustness against multiple adversarial perturbations. The main idea is to use a logit-pairing loss that makes the logits achieved on multiple perturbations match each other. Then, the authors used gradient projection to further align the updates from adversarial examples and ordinary non-adversarial examples.

Two reviewers are concerned about the paper. For example, one reviewer believes that logit pairing sounds like a “standard” idea and lacks novelty. Furthermore, focusing on the L1, L2, and L-inf attacks are not representative enough, and more studies on other types of perturbations should be considered, such as common corruptions, which the authors have addressed during the rebuttal. There is another concern that the paper does not provide precise characterizations of adversarial distributions for different norms.

Two other reviewers favored accepting the paper due to the solid theoretical and empirical results. One reviewer even championed the paper despite the diverging scores. 

IMHO, the problem of studying the tradeoff between different norms is legitimate because that’s a good starting point for the community to go beyond a single perturbation norm. For example, it’s quite intuitive that L2 and L-infinity attacks will lead to different distributions of adversarial examples, and training against one type of perturbation can lead to a distributional shift on the other. I think this is a common problem, and focusing on the L1, L2, and L-infinity is probably due to the convention in this field and the consideration of tractability. 

The reviewers provided some good suggestions for improving the paper. For example, instead of showing a T-SNE plot in Figure 1, one can just show a line plot. After discussing with the reviewer, the authors showed that RAMP achieves an improved accuracy-robustness tradeoff than a previous baseline, which convinced one reviewer.

In summary, I recommend accepting the paper despite the diverging scores.
Decision: Accept (poster)