Adversarial robustness against multiple $l_p$-threat models at the price of one and how to quickly fine-tune robust models to another threat model
Authors: Francesco Croce
Matthias Hein
Venue: ICLR 2022 Submitted (2022)
Keywords: adversarial robustness
multiple norms
adversarial training
fine-tuning

Abstract: Adversarial training (AT) in order to achieve adversarial robustness wrt single $l_p$-threat models has been discussed extensively. However, for safety-critical systems adversarial robustness should be achieved wrt all $l_p$-threat models simultaneously. In this paper we develop a simple and efficient training scheme to achieve adversarial robustness against the union of $l_p$-threat models. Our novel E-AT scheme is based on geometric considerations of the different $l_p$-balls and costs as much as normal adversarial training against a single $l_p$-threat model. Moreover, we show that using our E-AT scheme one can fine-tune with just 3 epochs \emph{any} $l_p$-robust model (for $p \in \{1,2,\infty\}$) and achieve multiple norm adversarial robustness. In this way we boost the state-of-the-art for multiple-norm robustness to more than $51\%$ on CIFAR-10 and report up to our knowledge the first ImageNet models with multiple norm robustness. Moreover, we study the general transfer of adversarial robustness between different threat models and in this way boost the previous SOTA $l_1$-robustness on CIFAR-10 by almost $10\%$.

Under review as a conference paper at ICLR 2022
ADVERSARIAL ROBUSTNESS AGAINST MULTIPLE 1p- THREAT MODELS AT THE PRICE OF ONE AND HOW TO QUICKLY FINE-TUNE ROBUST MODELS TO ANOTHER THREAT MODEL
Anonymous authors Paper under double-bling review
ABSTRACT
Adversarial training (AT) in order to achieve adversarial robustness wrt single 1 threat models has been discussed extensively. However, for safety-critical systems adversarial robustness should be achieved wrt all 1p-threat models simultaneously. In this paper we develop simple and efficient training scheme to achieve adver- sarial robustness against the union of 1p- --tthrett models. Our novel E-AT scheme is based on geometric considerations of the different 1,-balls and costs as much as normal adversarial training against single th--tt model. Moreover, we show that using our E-AT scheme one can fine-tune with just epochs any tp-robust model (for p E 21,2, )o)) and achieve multiple norm adversarial robustness. In this way we boost the state-of-the-art for multiple-norm robustness to more than 51% on CIFAR-10 1-0 and report up to our knowledge the first ImageNet models with multiple norm robustness. Moreover, we study the general transfer of adversar- ial robustness between different threat models and in this way boost the previous SOTA 1 robustsss on CIFAR-10 by almost 10%.
INTRODUCTION
The problem adversarial examples, that small adversarial perturbations of the input (Szegedy) et al. [214]] Kurakin et al.) 2017) changing the decision of a classifier, is serious obstacle for the use of machine learning in safety-critical systems. Many adversarial defenses have been proposed but most of them could be broken either by stronger attacks with a higher computational budget (Carlini & Wagner 2017; AAthayy al. 2018 Mosbach et al.] 2018) or using adaptive attacks (Tramer et al.] (020). Apart from provable adversarial defenses which are however still restricted to rather simple CNNs (Wong et al. 2018 Gowal al., 2018), the only successful technique so far
remains adversarial training (Madry et al. 2018) and its improvements (Zhang et al.) 2019 Carmon (ct al.] 2019 Wu et al.) 2020; (222]). We refer to Gowal et al. (2020) for a recent overview of "tricks of the trade" for improving adversarial training yielding the currently most robust models for l2 and 3o on CIFAR-16 10. While the community initially focused on adversarial examples for de-pperturbttions, there has been recently more interest in other 4n-attacks, such as 4 and l2, or perceptual threat models (Stutz et al.) 2019 Wong & Kolter [2021] Laidlaw et al. 2021). It is well known that robustness in one ball does not necessarily generalize to some other by-ball for p A q (Kang et al. 2019a Tramer & Boneh 2019). However, in safety-critical systems we need robustness against all norms simultaneously which has triggered recent extensions of adversarial training for
multiple norms (Tramèr & Boneh 2019 Maini et al. 2020) and provable defenses for all I, with p 2 (Croce Hein| 2020b).
In this paper we show that, using the geometry of the ball, the computationally expensive multi- ple norm training procedures of Tramèr & Boneh (2019): Maini et al. (2020), which cost up to three times as much normal adversarial training, can be replaced by very effective and simple form of adaptively alternating between the two extreme norms, namely I and 1xo. This simple scheme achieves similar robustness for the union of the threat models more costly previous approaches. Moreover, we show that already 3 epochs of fine-tuning with our extreme norms adversarial train- ing (E-AT) enough to turn any L robusttt model for p {1,2, 0o) into model which is robust

---

Under review as a conference paper at ICLR 2022
robust acc. (%)
70.2
robust acc. (%)
65.6
66.9
3 epochs
epochs of E-AT fine-tuning
acc.
51.2
L1 fine-tunm line-tuning
8.1
8.1
Linf L2
L1
union
Linf
L22
L1
union
Lin
L1
union
New SOTA L1-robustness
Linf-robust model (Gowal et 2020)
New SOTA multiple robustness
We fine-tune for 3 epochs the Ne0--6s CIFAR-10 from Gowal et al. (0220) with highest loc rrobustesss to be either robust wrt l (left) or with our E-AT to be robust wrt to the union of the lx--, 12., and l1-threat models (right). We achieve state- ofate-of-the-ar results in both threat models. The plots show the robust accuracy in the individual threat models and in their union for the initial Lxx -robust classifier (middle) and the fine-tuned ones, with the target threat model highlighted.
Figure I:
against all 1p-threat models for p E f1. ( even if the original classifier was completely non- robust against one of the threat models. We apply the proposed method to many existing models originally trained to be robust in a single norm: fine-tuning the currently most robust network in the lox -threat model from from Gowal et al. (2020) for CIFAR-10 we improve the current state-of-the-art performance for multiple norm robustness by more than 6% (robustness over the union of 4, l2 and employ our E-AT fine-tuning scheme with just a single epoch to yield the first ImageNet model which is robust against multiple attacks at the same time. Finally, we show that fine-tuning with just 3 epochs for CIFAR-10 and one epoch for ImageNet is sufficient transfer robustness from one threat model to another one which very quickly yields baselines for all threat models. In this way we achieve SOTA I -oorbuustssss on CIFAR-10 by fine-tuning the most robust mx-mmdel from Gowal et al. (2020) to become rrobust and get an ImageNet model robust wrt 4T- These results are quite striking as the original classifiers show no or only very little t-robustness. Fig. summarizes the results of robust fine-tuning on CIFAR- 10 (see Table 2 and foro details).
-allls..). Morcover,
2 RELATED WORK
Adversarial training: In image classification adversarial examples were first described by Szegedy itt al. (2014) even though earlier discussion of adversarial examples in email spam classification can be found Dalvi et al. (2004): Lowd Meek (2005). An early approach of adversarial training was the Fast Gradient Sign Method (FGSM) (Goodfellow et al. (015). which then was extended to a multi-step attack in Kurakin et al.) (2017). Nowadays, adversarial training as formulated in Madry et al. (2018) 8) as min-max optimization problem has been one of the few adversarial defenses which could not be broken by stronger attacks (Athalye et al., 2018). Other types of defenses use more so- phisticated techniques. typically preventing the direct optimization of the attack. However, adaptive attacks specifically designed for these defenses have often shown that these alternative techniques non-robust much less robust than claimed Tramer et al., 2020). Thus adversarial training remains the only general method ensuring adversarial robustness across architectures and datasets. Recent improvements have been achieved by using different objectives (Zhang et al. 2019). un- labeled data (Carmon et al.) 2019), adversarial weights perturbations (Wu et al. 22020) and wider networks /W et al. (2021) l). In Gowal al. (2020) several recent variants were sy systematically ex- plored and for a very large architecture, a WideResNet-70. 16, they obtain the most robust models for loo (radius 255 and l2 (radius 0.5) for CIFAR-10, which we use for our fine-tuning experiments.
Multiple norm robustness: It was early on discovered that adversarial robustness against a specific 1p threat model does typically not transfer to lg threat models for p * q (see Kang et al. (2019a); Tramer & Boneh (2019) for extensive studies). On the other hand to achieve really reliable ma- chine learning models (y-robustness wrt all p is necessary. The first approach to such a generally robust model was done in [Schott et al. (2019) which uses multiple variational auto-encoders for an analysis by synthesis (ABS) architecture. While this model restricted to MNIST, is robust against 4o. le and -ttackk.. However, it has been recently shown that with stronger black-box attack (Croce et al. (2020b) the lo lo-bouusss is significantly lower than originally claimed. [Trmmrr & Boneh 2019): Maini et al. (2020); Madaan al. (2021) variants of adversarial training to
2

---

Under review as a conference paper at ICLR 2022
achieve robustness in multiple norms. Since these are the most similar methods to ours, present them in detail below. |Mddaan et al. (202I) additionally proposes a meta-learning approach where one learns optimal noise augment the samples and uses consistency regularization to enforce sim- ilar predictions on clean, augmented and adversarial samples. Finally, |Stutz et al. (2020) combine adversarial training with a reject option for the classifier by down- wwin-weggtiigg the confidence of ad- versarial samples. Their models generalize to other threat models but the comparison to normal adversarially trained models is difficult as their model is non-robust without the reject option.
Provable robustness: In the area of provably robust defenses, Croce& & Hein (0200b) motivated a regularization approach based on the geometry of the 1p-balls which enforces multiple-nom robust- ness during training which allows then to derive provable guarantees for multiple- norm robustness in contrast to the empirical evaluation of adversarial training. However, their approach works only for small network architectures and relatively small radii of the lp-balls.
Fine-tuning of robust models: Fine-tuning of an existing neural network commonly used technique in deep learning (Goodfellow et al. 20166) to quickly adapt existing model to a different objective e.g. for language models (Howard & Ruder 2018). More recently, it has been shown the area of adversarial robustness that fine-tuning of pre-trained models, possibly using self-supervision yields better adversarial robustness (Hendrycks et al. 2019 Chen et al. 2020 Xu & Yang 2020). In Jeddi et al. 22020) it is shown that fine-tuming of non-robust models with 10 epochs can yield robust models with the caveat that their robustness evaluation is done using only a single run of PGD with 20 steps. We are not aware of any prior work discussing fine-tuning to transform an existing robust model wrt a single lp into a robust model wrt multiple threat models or wrt another 1y-threat model. In particular, we show that one can fine-tune an -obbuttt model so that it becomes rrobbsst even though the original model has not been 1 ro-rotutt at all.
Evaluation of adversarial robustness: Many white-box attacks for Loo (Madry et 2018 Gowal et al.,
2019).
lg (Madry et al., 2018 Carlini & Wagner, 2017) and 1 (Chen et al. 2018 Modas
al.
2019
Rony et al. [202T] have been proposed as well as several black box attacks (Brendel
et al.
2018
Liu et |0.9
et al.
2019 AI-Dujaili & O' O'Reilly 2020 Meunier et al. [019]]
Zhao et al.,
2019: Andriushchenko et al. (2020) for different threat models. It has recently been
shown that AutoAttack (Croce & Hein 2020c). parameter-free ensemble of the white-box attacks
FAB-attack (Croce & Hein 2020a) and the black-box Square-attack (Andriushchenko et al., 2020) is reliably evaluating adversarial robustness for l2 and loc. AutoAttack has recently been extended to 4 (Croce & Hein 202I) outperforming all existing state-of-the art attacks for 41- Croce & Hein (2020c) (022) showed that on models defended with adversarial training the two versions of APGD (with budget as in AutoAttack) already give an accu- rate robustness evaluation. As we have to evaluate very large models always for three threat models,
APGD for the cross-entropy and DLR-
we use those as a strong standard attack in our experiments.
2.1 ADVERSARIAL TRAINING FOR THE UNION OF 41, L2- AND Lsc -BALLS
Let us denote by So Rdd RK the classifier parameterized by 0 R., with input x E Rd and fo(x) E R² where K is the number of classes of the task. Let further D = {(x,, ()) be the training set, with y the correct label of x4, and C 12 R - E given loss function. The aim is to enforce adversarial robustness in all multiple Ip-balls simultaneously. i.e., defining Bp(cp) = fz E Rd: I|= S <p}. the threat model is the union of the individual ba-balls
A = (:) U Be(c) U (coo),
which is a non convex set, since in practice the Cp are chosen such that no ba-b contains any of the others. In adversarial training the worst case loss for input perturbation in the threat model, maxica [(fe(r; + o), y1),, is minimized. Efficiently maximizing the loss L in the union of threat models is non-trivial and different approaches to extend adversarial training to this setting have been proposed. They basically differ in the way how the inner maximization problem is tackled.
MAX: Tramer& & Bonch (2019) suggest to run the three attacks for each Bp(cp) for p (1,2, independently and then use the one which realizes the highest loss, that is
max [(fe(m, + 8), m) = max max L(Se(1, + 8). BEA
pE{1.2, 6EBp(tp)
This training optimizes directly the worst case in the union but comes at the price of being nearly three times as expensive as normal adversarial training wrt single balll
3

---

Under review as a conference paper at ICLR 2022
AVG: In their AVG alternative Tramer & Boneh (2019) suggest the three attacks for each (epc for p E {1,2, oo) independently but replace the inner maximization problem with
Σ
max [( + 6), 21).
6EBx(tp)
(1,2,,0)
and thus one basically averages the updates of all 1p bp-balls with the motivation of not "wasting" the computed attacks, in particular when the attained loss values are rather similar and thus the max is ambiguous. Again this costs three times as much normal adversarial training.
MSD: Maini et al. (2020) argue that the correct way to maximize the loss the union is to test during the PGD attacks all the three steepest ascent updates corresponding to the three norms (sign of the gradient for loc. normalized gradient for L2 and a smoothed 11-step by using the top-k components in magnitude of the gradient) and then take the step which yields the highest loss. Maini et al. (2020) report that MSD outperforms both AVG and MAX, also in terms of a more stable training. As all three updates (forward passes) are tested but only one backward pass needed (gradient is the same) this costs roughly two times as much as normal adversarial training.
SAT: Madaan al. 2021) introduce Stochastic Adversarial Training (SAT) which randomly sam- ples E [1, 3 for each batch and performs PGD only for the corresponding fy-norm. While SAT has the same cost as standard adversarial training. |Madaan et al. (202T) report that it does not perform very well.
FAST MULTIPLE-NORM ROBUSTNESS VIA EXTREME NORMS ADVERSARIAL TRAINING AND FINE-TUNING
All previous methods assume that for achieving robustness to multiple norms each of the threat models has to be used at training time. the following we first present an argument, using recent results from Croce & Hein 2020bb) showing that this is not the case. Based on this analysis we introduce our extreme norms adversarial training (E-AT) which achieves multiple norm robustness at the same price as training for a single norm threat model. Finally, show that only few epochs of fine-tuning with E-AT turns a model robust wrt a single norm into one which has competitive robustness wrt multiple norms.
3.1 GEOMETRY OF THE UNION OF -ALLS AND THEIR CONVEX HULI.
The main insight we use for E-AT is that a linear classifier which robust in both an 4 and an -all is also robust wrt the largest 4 ball for 1 S p
which fits into the convex hull of the union of the 11- and -bal. This ball significantly larger than the largest 4y-ball contained into the union of the 11- and -all (see Fig. 2p. Thus it is sufficient to be robust wrt the two "extreme" norms L and lox to ensure robustness. While this is exact for affine classifiers, we conjecture that for neural networks this will least hold approximately true (note that typical ReLU-networks yield piecewise affine classifiers (Arora et al. 2018)) and for the model it is the most efficient way in terms of capacity be 11- and To0 rrbust.
We now state the main results frome & Hein (2020b) which are the basis for our E-AT. We work in the non-trivial setting, where the balls are not included in each other, that is (ei)) g Boo (Exo) and Bx (foo) E (E1) as otherwise the problem of enforcing multiple norms (including I or (oo) robustness boils down again single norm robustness. has to hold E (Eoo ). For 6x 25g and dimension d = 3072 as in CIFAR-10 10 this yields an upper bound é < 96.38 which far higher than what has been used for 1tthee models (we use 12 for CIF.R-10). We denote by (1,, (00) = Bi (e1 ) U B (+oo) the union of the 41- and balls. One can then ask the obvious question: how much tp-robussness do I get for < p < oo
order to be in this non-trivial setting
&
from classifier robust in 01,00 6x)) This question is answered in the following result:
Proposition 3.1 CCroce& & Hein 2020b)) If cl 2 and el (Eoc, deco). then
(G1 - 6oo )"
min rERd (ERd (c),ca))
|I01|, = ch +
(1)
(d - 1) 1)p-1
4

---

Under review as a conference paper at ICLR 2022
Figure 2: Visualization of the b2-ball contained in the union resp. the convex hull of the union of 1 and ls- -alls in R?. First: co-centric 1,-bal (blue) and 1 -bal (black). Second: in red the largest 12-ball contained in the union of 1-- and -ball. Third: in green the convex hull of the union of the 1-- and -bal.. Fourth: the largest (2-ball (red) contained in the convex hull. The 12-ball contained in the convex hull is significantly larger than that in the union of 4 and Loe -bal.
Thus classifier which is robust for the union 01, Eso) has automatically a non-trivial robust- ness for all intermediate 1p-norms. For the case p is most interesting which given that ( > Coo can be tightly upper bounded as
min =ER2 < (E1,t))))
| 1
2 +
For the case of CIFAR-10 where d 3072 and = 12 and (2020) one gets c2 I 0.2188. As the radius of the 12-threat model in Maini et (2000) is chosen as C2 0.5, robustness in the union alone would not be sufficient to achieve the desired robustness wrt Ip for E {1, 2, oo). But the above point of view the worst case. In the following we see that if consider affine classifiers then guarantee for B (EI) and (exo) implies guarantee with respect to the convex hull C' of their union B1 (e1) Boo (e as an affine classifier generates half-space and thus only the extreme points of B₁ (e1) resp. (Exo) matter (see Figure 2 for illustrations of B1, B their union and their convex hull).
555 as chosen in Maini et al.
Theorem 3.1 (Croce & Hein (0020)) Let C' be the convex hull of Bi(Es) U Boo If 2 and E E (E ) then
min wERdYC
||0||, = (f1/emn - a + a9)
(3)
where a
61
and
1
As standard architectures using ReLU activation function yield piecewise affine classifier one can anticipate that this result gives at least rule of thumb on the expected Ip-robussness when one is 41- and 1 robust.. Again with the choice of 61, 6oo from above one gets for the radius of the ly-ball that fits into the convex hull of the union (e(e)) and (Exo):
C = min ||0|||
61
0.6178.
ZEROIC
Vaee~ - a
Thus for a desired 12- -oorouusss with radius less than 0.6178 sufficient for an affine classifier, and at least plausible for ReLU network, to enforce robussstsss with el 12 and -obuustess with co 255 This motivates our extreme norms adversarial training (E-AT) and fine-tuning.
3.2 EXTREME NORMS ADVERSARIAL TRAINING (E-AT)
In light of the geometrical argument presented in the previous section, we propose to train only on adversarial perturbations for the lx-- and litthea models if the l2-radius obtained from Theorem 3.1 is larger than the radius é the ly-threat model. this case sufficient to just train for the extremes 1 and loc in order to achieve robustness also to the intermediate Ip -tttccks with p E (1, oo). Since we seek a method as expensive as standard adversarial training, for each batch we do either the 11- the -ttack... For full training from random initialization simply alternating sampling
5

---

Under review as a conference paper at ICLR 2022
Table 1: CIFAR-10 Comparison of different full training schemes: We train WideResNNet28- 10 with TRADES-XENT loss (except for MNG-AC which we use as originally proposed) for each scheme (repeated for 3 seeds), and report the robust accuracy wrt So 12, 4 and the union of the threat models. Moreover, we show the clean accuracy and the time per epoch of training.
method
clean
Loc (6oz
(e2 = 0.5)
4 (c1 12)
union
time/epoch
AT
82.6 + 0.52
52,0 0.70
59.7 + 0.22
9.1 = 0.22
9.1 0.22
922
l--TT
88.2 + 0.37
35,9 - 0.17
70.9 + 0.39
36.1 = 0.25
31.3 - 0.17
928
4-AT
83.7 + 0.16
30.7 = 0.74
65.1 + 0.50
616 0.34
30.7 = 0.74
949
SAT
80.5 0.57
45.9 0.46
66.7 + 0.29
35.9 0.49
45.7 -: 0.62
925 s
MNG-AC
81.3 0.33
43.5
6.29
57.6 0.84
43.3
1500
AVG
+
4..4
08.0
55.0
45.1 = 1.06
2771
MAX
0.14
0.74
6.29
0.77
2479 8
MSD
80.6
0.19
6.36
46.9 1
1554 s
E-AT unif.
79.7 0.17
45.4
6.40
55.6 1 0.54
45.1 1 0.65
939
E-AT
79.9 0.69
46.6 + 0.24
66.2
56.0 1 0.37
46.4 0.28
921
uniformly at random from the 1 and Loc -attakk works already well. However, for very quick fine- tuning, e.g. just one epoch in the case of ImageNet, for multiple norm robustness from an existing classifier robust wrt single threat model, one has to take into account the existing robustness of the model. Thus we use an adaptive sampling strategy based on the running averages, reset at every epoch, of the robust training emrors rem, and re (note that these rumning averages just from averaging the robust error on the batches where the respective attack has been performed, thus extra attacks necessary), such that the probability for sampling the I,- 1y-threett model is
computed
remp
for p f1,
(S)
remr, + re
The motivation for this sampling scheme is that the robust error in the union A is mainly influenced by the worst threat model. We show the effect of the biased sampling E-AT fine-tuning in Sec. C..3 The next section shows that E-AT is very competitive baseline compared to the significantly more involved and up to three times more expensive training schemes discussed in Sec. 2.1
3.3 MULTIPLE NORM ROBUSTNESS FROM RANDOM INITIALIZATION
In this experiment on CIFAR-10 we evaluate the performance of the different adversarial training variants to achieve adversarial robustness wrt the union of the lo 12 and 11-threat models where we
use the standard radii c 12, 6 0.5 and 6o0 255 As baselines we report the 4, 1,-robussness of
models specifically trained for only single norm: ATT (2-AT and ATT We train Nete 28- (Zagoruyko
Komodakis 2016) with the TRADES- DEENT loss since Gowal et al. (2020) show that this yields slightly higher robustness than standard TRADES (Zhang al. 2019) without additional data (see Sec. A for more details). For AT, SAT and E-AT we use APGD (Croce &
Hein 2020c) for training, while for AVG, MAX and MSD we standard PGD for 12, and SLIDE (Trmer & Boneh (019) for 4-PGD, even though we have observed no strong difference APGD, However, note that for MSD do not use the 4,PGD variant suggested by Maini et al.) (200)) as this resulted in significantly worse performance (worse than all other methods). For MNG-AC (Madaan al. 2021) we use the original code (with the 61 we consider here and rescaled step size). Finally, we show also the performance of E-AT using a uniformly random sampling scheme (named E-AT unif.) for selecting the l, lp-attack to use for each batch rather than the biased one presented in Eq. (5)
for multiple norms training compared
In the results in Table we can see that the models we retrained using MSD and MAX perform best with 46.9% and 47.4% adversarial robustness in the union, but our E-AT attains very close results with 46.4% (within the range of standard deviation) with 1.7x and 2.7x faster runtime per epoch. Given that pure lo0 -taining yields 52.0% -obuusttss which turns out to be the most difficult norm, it is quite remarkable that multiple-nor loss. The other variants SAT, MNG-AT, AVG and E-AT unif. perform worse with higher similar computational cost to E-AT. Note that E-AT unif. performs similarly to SAT (which also samples uniformly the lttaktack for each batch) in every threat model without seeing lz-attacks at training
robustness can be achieved with a relatively minor

---

Under review as a conference paper at ICLR 2022
Table CIEAR-10- epochs of E-AT fine-tuning on 1y-robust models: We use E-AT to fine-tune models robust wrt a single 4y-norm for multiple-nomm robustness, and report the robust accuracy on 1000 test points for all threat models, and the difference compared to the initial classifier. indicates that additional data is used.
model
clean
1o0 (co
$ ) (62 = 0.5) (c1 12)
union
Fine-tuning Loc -robust models
RN-50 Loo
88.7
50,9
59.4
5.0
5.0
Engstrom et al.
2019 FT
86.2
-2.5
46.0
-4.9
70.1
10.7
49.2
44.2
43,4
38.4
WRN-34-22 Lx
87.2
56.6
63.7
8.5
8.5
Gowal et al
2020
FT
88.3
1.1
49.3
-7.3
71.8
8.1
51.2
42.7
46.2
37.7
WRN-28-10 10 La (*)
90.3
59.1
65.7
8.0
8.0
Carmon et al.
2019
FT
90.3
0.0
52.6
-6.5
74.7
9.0
54.0
46.0
48.7
40.7
WRN-28- 10
89.9
62.9
67.2
10.8
10.8
Gowal et al
2020
FT
91.2
1.3
53.9
-9.0
76.0
8.8
56.9
46.1
50.1
39.3
WRN-70-16 16
100 (*)
90.7
65.6
66.9
8.1
8.1
Gowal al.
2020
FT
91.6
0.9
54.3
-11.3
78.2
11.3
58.3
50.2
51.2
43.1
Fine-tuning 12-robust models
RN-50-
91.5
29.7
70.3
27.0
23.0
Engstrom et al.
2019 + FT
87.8
-3.7
43.1
13.4
70.8
0.5
50.2
23.2
41.7
18.7
RN-50 (*)
91.1
37.7
73.4
31.2
28.8
Augustin et al.
2020
FT
87.0
-4.1
47.2
9.5
70.4
-3.0
54.1
22.9
46.0
17.2
WRN-70-16 4
94.1
43.1
81.7
34.6
32.4
Gowal ct al.
2020
FT
91.2
-2.9
51.9
8.8
79.2
-2.5
58.8
24.2
49.7
17.3
Fine-tuning rrobutt models
RN-18-4
87.1
22.0
64.8
60.3
22.0
CCroe & Hein 2021
83.5
-3.6
40.3
18.3
68.1
3.3
55.7
-4.6
40.1
18.1
time. Thus E-AT provides very efficient and effective alternative MSD and MAX which allows to scale adversarial training for multiple norms to larger problems such as ImageNet. We repeat the comparison with the same setup but the smaller PreAct ResNet-18 18 (He et al.] (016)) in Sec. B.3 with standard adversarial training on the cross-entropy loss and obtain similar results. Thus, E-AT generalizes across architectures and training schemes. Finally, we observe how almost all methods for multiple norms lead to drop of clean accuracy compared to the single norm Lp-AT, showing that the threat model of the union significantly more challenging.
3.4 MULTIPLE-NORN ROBUSTNESS VIA FAST FINE-TUNING OF EXISTING ROBUST MODELS
In the previous section, we have shown that training for only the two extreme threat models is sufficient to achieve robustness wrt all three norms. Prior works (Tramèr & Boneh 2019 Kang al. 2019ab observed that models adversarially trained wrt loo give
trivial robustness la- attacks, although lower than what one gets directly training against such attacks, and vice versa. This is confirmed by our evaluation of the models in Table [ (top part), where we also notice that A-AT provides good robust accuracy in l2. On the other hand, training for Lo resp. l does not yield particular robustness to the dual norm, which is reasonable since the perturbations generated in the two threat models are very different, while l2 can be seen an intermediate case which can defend at least partially against ----- and ( -attacks. Therefore, we propose to use models trained for robustness wrt single norm good initializations to achieve, within small computational budget, multiple norms robustness. This done by fine-tume tine-tuning pretrained models with our E-AT for 3 epochs when using CIFAR-10 and 1 epoch with ImageNet- 1k, starting with learning rate 0.05 0.01, depending on the model. and decreasing by a factor of 10 at each epoch. We do 10 steps of APGD adversarial training for CIFAR-10, 10, while and 15 with loo and 1 respectively ImageNet since optimizing in

---

Under review as a conference paper at ICLR 2022
Table ImageNet- Results of one epoch of E-AT fine-tuning of existing robust models: We use existing models trained to be robust wrt single 1p-ball (either 160 or (2) from Engstrom et al. (2019) and fine-tune them for a single epoch with our E-AT scheme.
model
clean
1xc (c
1/2 (E2 = 2)
2 (E1
255)
union
755
RN-50 Lo0
62.9
29.8
17.7
0.0
0.0
(Engstrom al.
2019) FT
58.0
-4.9
27.3
-2.5
41.1
23.4
24.0
24.0
21.7
21.7
RN-50-
58.7
25.0
40.5
14.0
13.5
[Engstrom et al.
2019) +
56.7
-2.0
26.7
1.7
41.0
0.5
25.4
11.4
23.1
9.6
the b-lll requires more iterations in that case. When the model was originally trained with extra data beyond the training set on CIFAR-10, we use the 500k images introduced by Carmon et al. (219)) as additional data for fine-tuning (see also Sec. A
CIFAR-10: RobustBench (Croce et al. 2020a) provides a collection of the currently most robust classifiers. We took the most robust models, among those which do not use synthetic data, for 12- and nnom and the l-rrbutt one from Croce& & Hein (2021) (all are trained with the same radii fp as in our experiment). Each of them fine-tuned for epochs with E-AT where (*) denotes models trained using extra data, which also means that we used the extra data from Carmon al. (2019) for fine-tuning (Gowal et al. 2020) have created their own extra data but which is not available). We present in Table the results. First of all the fine-tuning works for all tested architectures and results in many cases in stronger robustness in the union than for the specifically trained WideResNet-255 models. In partieular, the most robust to -ooee from Gowal et al. (2020) with 65.6% L robustnesss and only 8.1% [ (i-robustness can be fine-tuned to a multiple-norm robust model with 51.2% robust- ness which is up our knowledge the best reported multiple-norm robustness. Very interesting is that the l2 fo-robusss of 8% is quite close to the 81.7% l2 robbustteess of the specifically L2 ly-trained model from Gowal et al. (2020). Moreover, the 4 roobuusssssss of 58.31 3% is close to the best reported one of 60.3% from Croce & Hein (2021) (however we improve this lot in the next section) and the model has even higher clean accuracy. Clearly, this comes at the price of a significant loss in loc but this is to be expected. Striking is that fine-tuning the 12 lo-obust model from Gowal et al. (2020) results in a very similar result. In a nutshell, E-AT fine-tuning of existing 1n-robust models yields very efficient and competitive baselines for future research in this area.
ImageNet: Similarly to CIFAR-10 we start with the specific 12-resp. -robust models from En- gstrom et al. (2019) which have been trained with 62 3 and 6 255 We use 2 for the experiments as the robust accuracy is still in a reasonable range of 40% and together with our choice of 255 and 6 255 the 12-radius from Theorem 3.1 1.1 is almost exactly 2. We fine-tune the two models only for single epoch and report the results in Table 3 Again, we note that our evaluation stronger than the original one (although this on a different number of data points) which effects in particular the robuustesss Note that the initial -odel is completely non-robust for 4 but achieves 24.0% ro-uutttt accuracy and also the 1o-rsstst accuracy improves from 17.1% to 41.1% and this at the price of relatively small loss in L robust and clean accuracy. For the 12-robsst model all robust accuracies improve as the original model was trained for e 3. It yields the best multiple-norm robust accuracy of 23.1%. Up to our knowledge no multiple-norm robustness has been reported before for ImageNet and thus these results are an important baseline.
4
Additional experiments: Sec. C the appendix contains further studies about E-AT fine-tuning: we show that fine-tuning a naturally trained model does not provide competitive robustness and leads to low clean accuracy. Moreover, we show the stability of the scheme over random seeds, and that increasing the number of epochs progressively improves the robustness in the union.
FINE-TUNING LAAINNG/-RRBUSSS -RBBUST MODELS TO BECOME 1, ROBUST FOR p + q
Motivated by our results for the multiple norm threat model we study to which extent fine- tune an .o-rroustt model to a I, robust model with p * q. Again the emphasis is on an extremely short fine-tuning time that this is much faster than full adversarial training.
8

---

Under review as a conference paper at ICLR 2022
Table 4: Fine-tuning 15-robust models to another threat model: For each norm we fine-tune the most robust models wrt the other ones for 3 epochs for CIFAR-I0 and epoch for ImageNet and report clean and robust accuracy for all threat models. Even for the threat models where the robustness of the original model is low, the fine-tuning is sufficient to yield robustness almost at the same level of the specialized models with same architecture. For each threat model (column) we highlight in blue the model trained for the specific norm, in orange those only fine-tuned in the target norm. The values of the thresholds e are the same used the multiple norms experiments.
CIFAR-10
ImageNet
clean
l2
4
clean
100
l2
WRN-70-16 16 (Gowal
2020
loc (*)
RN-50 (Engstrom et al.
2019) 160
original
90.7
65.6
66.9
8.1
original
62.9
29.8
17.7
0.0
+ FT wrt l2
92.8
47.4
80.0
34.0
+FT wrt l2
62.9
25.5
41,5
8.4
+ FT wrt 1
92.4
33.9
74.7
70.2
+FT wrt
57.7
18.0
37.6
27.4
WRN-70-16 (Goaal et al.
2020
la (*)
RN-50 Engstrom et al.
2019)
original
94.1
43.1
81.7
34.6
original
58.7
25.0
40.5
14.0
+ +FT wrt too
92.3
58.5
73.5
11.4
+ wrt 1o0
59.1
31.5
40.1
7.5
+ FT wrt 1
92.8
29.2
75.7
68.9
FT wrt 71
56.8
18.0
37.1
28.7
RN-18 (Croce& & Hein
2021 11
original
87.1
22.0
64.8
60.3
+ FT wrt 6o0
82.7
44.2
66.6
25.4
+ wrt
88.0
31.0
69.8
39.7
CIFAR-10:
We fine-tune for 3 epochs the most 1x. model at 6no 255 of Gowal et al. (0220)
with adversarial training wrt l2 and I with 62 = 0.5 and e = 12. Table shows that fine-uning for l d--robustness yields 70.2% ro-rbusst accuracy which 9.9% more than the previously most robust model. Also we get strikingly high (2-robust accuracy for the lg-fine-tuned model of 80.0% not far away from the 81.7% which ones gets by training for l2 from scratch. Surprisingly, fine-tuning the 12-robust model Gowal et al. (2020) wrt 7 does outperform the lj-robustness achieved by fine-tuning their -rousst model. Interestingly, fine-tuning the -robust PreAct ResNet- 18 for lp yields better Ig-robustness than l2-training from scratch, see Table Again this shows that fine- tuning of existing models is minimal effort and already provides strong baselines for adversarial robustness obtained by adversarial training from scratch and in some cases even outperforms them.
We fine-tune the l2- and loo -robutt models from Engstrom et al. (2019) to the other threat model respectively and wrt 41 The results are Table 4 The resulting classifiers are both more robust than the originally trained models from Engstrom et al. (2019): we get a 12-robust accuracy of 41.5% which is 0.6% higher than the model and too robust accuracy of 31.5% which is 1.7% better than the original model. For both models it is possible to achieve within one epoch of fine-tuning non-trivial robustness
ImageNet:
4, In particular, notice that the model for Loc. initially has 0.0% robustness wrt 41, while after fine-tuning it achieves 27.4% robust accuracy which is similar to what obtained by fine-tume tinettunig the l2 fo-robust model for I which yields 28.7%. Note that when we fine-tuned for multiple-nov norn robustness we got L rrobustt accuracy of 21.7% resp. 23.1% (Table 3). Thus by fine-tuning specifically for -rrobuussesss one gains 5.7% resp. 5.6% l robustt accuracy. Up knowledge ImageNet models are the first ones for which lo-bouusss is reported.
5 CONCLUSION
Based on the geometry of the -a-lalls we have introduced E-AT, a novel training scheme for mul- tiple norm robustness which achieves comparable adversarial robustness in the union while being significantly faster. We also show for the first time that fine-tuning can be used to transfer adver- sarial robustness from single 4,-ttheatt model to the multiple norms one, and that one can even obtain an robssttt classifier with a quick fine-tuning of an 4p-robust one with p + q. This yields strong baselines for future research. We have in this way generated models with SOTA performance for multiple-no multiple-norm and 4 rorbbuussssss on CIFAR-10 and the first models on ImageNet which show significant multiple-norm well as / robbbusss.s...

---

Under review as a conference paper at ICLR 2022
ETHICS STATEMENT
There are no conflicts of interest in this work. As we consider the efficient training and fine-tuning of adversarially robust models, we think that this work contributes more trustworthy Al systems and thus has rather positive implications.
REPRODUCIBILITY STATEMENT
We provide information about training in Sec. 2.1 and foo the main part, and add further details in Sec. A of the appendix. We provide in the supplementary material an implementation of our method in PyTorch. Note that most of the models used for fine-tuning are available in RobustBench, and the evaluation code is part of AutoAttack. We report for most of the experiments the mean and standard deviations over different random seeds, as well as the runtime and infrastructure used.
REFERENCES
Abdullah Al-Dujaili and Una-May O' O'Reilly. There are no bit parts for sign bits in black-box attacks. In ICLR, 2020.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square at- tack: query-efficient black-box adversarial attack via random search. In ECCV, 2020.
Raman Arora, Amitabh Basu, Poorya Mianjy, and Anirbit Mukherjee. Understanding deep neural networks with rectified linear unit. ICLR, 2018.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018.
Maximilian Augustin, Alexander Meinke, and Matthias Hein. Adversarial robustness in- and
out-distribution improves explainability. In ECCV, 2020.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision- based adversarial attacks: Reliable attacks against black-box machine learning models. In ICLR, 2018.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. In NeurlPS, pp. 11190-11201 111001110..111111111111111111111 2019.
Pin- Yu Chen, Yash Sharma, Huan Zhang, Jinfeng Yi, and Cho-Jui Hsich. Ead: Elastic-net attacks to deep neural networks via adversarial examples. In AAAI, 2018.
Tianlong Chen, Sijia Liu. Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From self-supervieed pre-training to fine-tuning. In CVPR, 2020.
Minhao Cheng, Thong Le, Pin-Yu Chen, Jinfeng Yi, Huan Zhang, and Cho-Jui Hsieh. Query- efficient hard-label black-box attack: An optimization-based approach. In ICLR, 2019.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In ICML, 2020a.
Francesco Croce and Matthias Hein. Provable robustness against all adversarial I, perturrbaatin for 1. In ICLR, 2020b.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free free attacks. In ICML, 2020c.
Francesco Croce and Matthias Hein. Mind the box: I lp-apgd for sparse adversarial attacks on image classifiers. In ICML., 2021.
10

---

Under review as a conference paper at ICLR 2022
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: a standardized adversarial robustness bench- mark. arXiv preprint a0X1v:2010.09670, 2020a.
Francesco Croce, Maksym Andriushchenko, Naman D. Singh, Nicolas Flammarion, and Matthias Hein. Sparse-rs: versatile framework for query-efficient sparse black-box adversarial attacks. arXiv preprini, arXiv:2006. 12834, 2020b.
Nilesh Dalvi, Pedro Domingos, Mausam, Sumit Sanghai, and Deepak Verma. Adversarial classifi- cation. In KDD, 2004.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Mma training: Direct input space margin maximization through adversarial training. In ICLR, 2020.
Logan Engstrom, Andrew Ilyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library), 2019. URL https: //t:://ggtttta com/ Madry robustness
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Sven Gowal, Krishnamurthy Dvijotham. Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Ue- sato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. preprint, arXiv: 1 188.11112222 12715v3, 2018.
Sven Gowal, Jonathan Uesato, Chongli Qin, Po-Sen Huang, Timothy Mann, and Pushmeet Kohli. An alternative surrogate loss for pgd-based adversarial testing. 2019.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.035932.. 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV, 2016.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. In ICLR, 2019.
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. Using pre-training can improve model robustness and uncertainty. ICMI., 2019.
Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. In ACL, 2018.
Ahmadreza Jeddi, Mohammad Javad Shafice, and Alexander Wong. A simple fine-tuning is all you need: Towards robust deep learning via adversarial fine-tuning. arXiv preprint, arXiv:2012. 2020.
Daniel Kang, Yi Sun, Tom Brown, Dan Hendryeks, and Jacob Steinhardt. Transfer of adversarial robustness between perturbation types. arXiv preprint, arXiv: 2019a.
Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardt. Testing robustness against unforeseen adversaries. arXiv preprint arXiv:1908.0800 2019b.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In ICLR Workshop, 2017.
Cassidy Laidlaw, Sahil Singla, and Soheil Feizi. Perceptual adversarial robustness: Defense against unseen threat models. In ICLR, 2021.
Sijia Liu, Pin-Yo Yu Chen, Xiangyi Chen, and Mingyi Hong. signSGD via zeroth-order oracle. In ICLR, 2019.
Daniel Lowd and Christopher Meek. Adversarial learning. In KDD, 2005.
11

---

Under review as a conference paper at ICLR 2022
Divyam Madaan, Jinwoo Shin, and Sung Ju Hwang. Learning to generate noise for multi-attack robustness. ICML, 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In ICLR, 2018.
Pratyush Maini, Eric Wong, and Zico Kolter. Adversarial robustness against the union of multiple perturbation models. In ICML, 2020.
Laurent Meunier, Jamal Atif, and Olivier Teytaud. Yet another but more efficient black-box box adver- sarial attack: tiling and evolution strategies. arXiv preprint, wrXiv:1910002244, 2019.
Apostolos Modas, Seyed-Mohsen Moosavi-Dezfoolly and Pascal Frossard. Sparsefool: few pixels make a big difference. In CVPR, 2019.
Marius Mosbach, Maksym Andriushchenko, Thomas Trost, Matthias Hein, and Dietrich Klakow. Logit pairing methods can fool gradient- based attacks, In NeurlPS 2018 Workshop on Security in Machine Learning. 2018.
Sukrut Rao, David Stutz, and Bernt Schiele. Adversarial training against location-optimized adver- sarial patches. In ECCV Workshop on the Dark and Bright Sides of Computer Vision: Challenges and Oppurtinities for Privacy and Security, 2020.
Jérôme Rony, Eric Granger, Marco Pedersoli, and Ismail Ben Ayed. Augmented Lagrangian adver- sarial attacks. ICCV, 2021.
Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the first adversarially robust neural network model on mnist. In ICLR, 2019.
David Stutz, Matthias Hein, and Bernt Schiele. Disentangling adversarial robustness and general- ization. CVPR, 2019.
David Stutz, Matthias Hein, and Bernt Schiele. Confidence-calibrate adversarial training: Gener- alizing unseen attacks. In ICML, 2020.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICLR, pp. 2503-250111 2014.
Florian Tramèr and Dan Boneh. Adversarial training and robustness for multiple perturbations. In NeurlPS, 2019.
Florian Tramèr, Nicholas Carlini, Wieland Brendel, and Aleksander Madry, On adaptive attacks to adversarial example defenses. In NeurIPS, 2020.
Eric Wong and Zico Kolter. Learning perturbation sets for robust machine learning. In ICLR, 2021.
Eric Wong, Frank Schmidt, Jan Hendrik Metzen, and Zico Kolter. Scaling provable adversarial defenses. In NewrlPS, 2018.
Boxi Wu, Jinghui Chen, Deng Cai, Xiaofei He, and Quanquan Gu. Do wider neural networks really help adversarial robustness? arXiv preprint arXiv:2010.0/22992. 2021.
Dongxian Wu, Shu tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. In NeurlPS, 2020.
Cong Xu and Min Yang. Adversarial momentum-contrastivv pre-training. arXiv preprint, arXiv:2012.1315442, 2020.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, pp. 87.1-87.12, 2016.
Michat Zajac, Konrad Zoina, Negar Rostamzadeh, and Pedro O Pinheiro. Adversarial framing for image and video elassification. In AAAI, pp. 10077-10078, 2019.
12

---

Under review as a conference paper at ICLR 2022
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In ICML, 2019.
Pu Zhao, Sijia Liu, Pin-Yu Chen, Nghia Hoang, Kaidi Xu, Bhavya Kailkhura, and Xue Lin. On the design of black-box adversarial examples by leveraging gradient-free optimization and operator splitting method, In ICCV, pp. 121-130, 2019.
13

---

Under review as a conference paper at ICLR 2022
A EXPERIMENTAL DETAILS
For the comparison of training schemes we use for multiple-norm robustness we train PreAct ResNet- 18 (He et al. 2016) with softplus activation function for 80 epochs with initial learning rate of 0.05 reduced by a factor of 10 after 70 epochs. When training WideResNet-28. 10 (Zagoruyko & Komodakis 2016) epochs. We use SGD optimizer with momentum of 0.9 and weight decay of 5 10- batch size of 128. We use random cropping and horizontal flipping as augmentation techniques. For adversarial training of models robust wrt a single norm and with SAT and our novel scheme E-AT we use APGD with default parameters, while for the retrained AVG, MAX, and MSD we use PGD for to (step size /4) and l2 (step size e/3), SLIDE Tramer & Boneh 2019) for 1 (standard parameters). For all methods we use 10 steps for the inner maximization problem in adversarial training (note that AVG and MAX repeat the attack for all threat models, and MSD tests multiple steps, thus they are more expensive). For schemes we select the best performing checkpoint for the comparison when using the piecewise schedule. the final checkpoint with the cyclic schedule. Moreover, the TRADES XENT loss (TRADES loss (Zhang et al.] (019)) with adversarial points maximizing the cross-entropy loss) since Gowal et al. (2020) show that this gives slightly better robustness on CIFAR-10 without additional data, while we use standard adversarial training (Madry et al.] (018) for PreAct ResNet-18. We train the classifiers of MNG-AC (Madaan al., 2021) with the original code, where we set = 12 and rescale the step size linearly. Finally, for the runtime comparison we run each method on a single Tesla V100 GPU.
use a cyclic schedule for the learning rate with maximum value 0.1 for 30
For fine-tuning on CIFAR-I0 we use 3 epochs and the same setup as for full training except for the learning rate schedule, since in this case we use as initial value the best performing one in {0.01, 0.05} (the larger value works best for the smaller networks) and reduce it by factor of 10 the beginning of each epoch. When the model was originally trained with extra data beyond the training set on CIFAR-10, 10, we use the 500k images introduced by Carmon et al. (219)) as additional data for fine-tuning, and each batch is splitted equally between standard and extra images, and we count I epoch when the whole standard training set has been used: note that in this way, using only 3 epochs not the whole pseudo-labelled dataset is exploited.
For fine-tuning every 1/3 of training steps. We follow the setup of Engstrom et al. 2019) for data augmentation and setting batch size to 256 and weight decay to 10 For adversarial traminn use APGD with 5 steps for Loo and 12, 15 steps for 1 since optimizing in the -ball intersected with the box constraints is more challenging, see Croce & Hein (202T)
ImageNet use 1 epoch, initial learning rate of 0.01, reduced by factor of 10
B ADDITIONAL ANALYSIS, EVALUATION AND EXPERIMENTS FOR E-AT
We here analyze in more details our E-AT scheme and expand the comparison to existing methods presented above.
B. ROBUSTNESS WRT 12 OF E-AT
To show the effect of E-AT 12-robustness plot in Fig. 3 the robust accuracy wrt computed with FAB (Croce & Hein 2020a), which minimizes the size of the perturbations, as a function of the threshold E2 for PreAct R8sNet-18 trained with 4--AT E2 0.5 and one using E-AT (see complete results for such models in Sec. B.3). Theorem 3.1 suggests that the extreme norms training provides robustness at 62 s 0.62, which confirmed by the plots. Although no lz-attack has been used during training by the E-AT model, it has robustness wrt 17 similar to that of the classifier specifically trained for such threat model.
B.2 CONFIRMATION THAT ADVERSARIAL TRAINING WITH THE EXTREME NORMS IS SUFFICIENT: ANALYSIS OF MAX AND MSD TRAINING
Both MAX and MSD schemes perform adversarial training considering all the threat models simul- taneously, and we analyze here their training procedure in more detail. Fig. 4 shows for the MAX strategy how many times, in percentage, for each epoch the points computed for each threat model realize the maximal loss over the three attacks (4, 12, 1o)), and then are subsequently used for the
14

---

Under review as a conference paper at ICLR 2022
Robustness to l2-attacks
1_ +/
80
robust robust accuracy
60
40
20:
E2
Figure 3: Iz-robustness curve of model trained with lz-adversarial training (AT) for 62 = 0.5 (orange) versus our E-AT (blue). which expected to yield robustness at e2 0.62. Although 12- attacks are not used for training, our extreme-adv. training scheme E-AT yields (2-robustness similar the one obtained with specific az-adversarial training.
MAX
MSD
80
60
40
20
20
0 30
0
10
20
30
40 epoch
50
60
70
80
0 10 20
40
50
60
70
80
epoch
Figure 4: CIFAR-10, ResNet18. Left: For MAX-training we show for each epoch during training the percentage of points attaining for the indicated 4--threat model (p E {1, oo)) the highest loss over the the three threat models. Right: For MSD-training we show the percentage of steps taken wrt each threat model over epochs (note that MSD does the steepest descent step for each th-threat model and then realizes the one yielding maximal loss).
update of the model. Similarly, for MSD, we show the frequency with which step wrt each -n-or taken when computing the adversarial points (average over all iterations and training points). In both cases the loc -threat model is the most used one with the 1i-threat model being used times less often. However, the /2-threat model almost never chosen. This empirically confirms the analysis from Theorem 3.1 which shows that training only wrt I and I is (at least for linear classifier) sufficient to achieve ly-robusssess for the chosen é2 and thus during training no extra up- dates wrt the l2-threat model are necessary. This is in line with the results reported in Fig. 3 which show that for different thresholds the In-robustness achieved by E-AT-training is similar to that of standard l er-training (for generating Fig. we FAB attack (Croce & Hein 2020a) to compute robust accuracy at varying 62).
B.3 E-AT FROM RANDOM INITIALIZATION WITH DIFFERENT ARCHITECTURE
We repeat the experiment from the main part about training for multiple norms robustness from random initialization with a smaller architecture like PreAct ResNet-18 (He et al. 2016). MAX- training yields the most robust model but E-AT is only 1.6% worse outperforming SAT and AVG but note that the standard deviation is rather high. Since it has the same architecture we additionally include the original MSD model of Maini et al. (2020), marked with (*), which obtains 41.4% robustness in the union, whereas with our reimplementation we get 43.9 43.9%, improving their results significantly. Note that they reported in their paper 47.0% robustness in the union, while APGD-
15

---

Under review as a conference paper at ICLR 2022
Table 5: CIFAR-10- Comparison of different full training schemes for multiple-norm robust- ness on PreAct ResNet-18: For each scheme we report the robust accuracy wrt loo. l2. ( and the worst case over the union the threat models. Moreover, we show the clean accuracy and the time epoch of training. MSD and MAX perform best in the union, but our E-AT achieves almost the same robustness in the union but better in 1 and Iz-robustness and requires only about about third or half of the training time. (*) indicates the original MSD model from Maini et al. (2020).
method
clean
(Eoc 255
(e2 0.5)
(61 12)
union
timelepoch
AT-
84.0 + 0.31
48.1 + 0.21
59.7 + 0.41
6.3 + 0.97
6.3 + 0.97
151 s
4-AT
88.9 + 0.57
27.3 + 1.79
68.7 + 0.09
25.3 1.60
20.9 + 1.80
153
4-AT
85.9 1.07
22.1 = 0.14
64.9 0.49
59.5 0.85
22.1 = 0.09
195
SAT
83.9 + 0.82
40.7 + 0.71
68.0 + 0.39
54.0 1.20
40.4 + 0.66
161
AVG
84.6 0.31
40.8 + 0.66
68.4 + 0.71
52.1
40.1 + 0.78
479
MAX
80.4 = 0.54
45.7 0.90
0.41
0.82
44.0 0.71
466
MSD (*)
82.1
43.1
64.5
46.5
41.4
MSD
81.1 = 1.14
44.9 + 0.63 0.63
65.9 0.64
1.1581111
43.9 0.76
306 s
E-AT unif.
82.2 = 1.84
42.7 + 0.74
6.45
53.6
42.4 0.60
163
E-AT
81.9 1.44
43.0 + 0.87
66.4 0.58
53.0 0.29
42.4 + 0.73
160 s
based evaluation reduces this 41.4% which shows that our robustness evaluation significantly stronger.
B.4 ADDITIONAL STATISTICS
Table 6: CIFAR-10 Comparison of different full training schemes: We repeat the results from Table and Table 5 with additionally the average robust accuracy over the three threat models (last column).
method
clean
1oc (6:0 255
(e2 0.5)
4 (c1 12)
union
average
WideResNet-28-100
AT
82.6 + 0.52
52.0 + 0.70
59.7 + 0.22
9.1 + 0.22
9.1 + 0.22
40.3 + 0.4
4-
88.2 + 0.37
35.9 0.17
70.9 + 0.39
36.1 + 0.25 0.25
31.3 + 0.17
47.6 0.2
h-AT
83.7 0.16
30.7 + 0.74
65.1 + 0.50
61.6
30.7 + 0.74
52.5
SAT
80.5 0.57
45.9 0.46
66.7 0.29
55.9 +
45.7 0.62
56.2 +
MNG-AC
81.3 0.33
43.5 + 0.66
66.9 0.22 0.22
57.6
43.3 + 0.70
56.0
AVG
82.5 + 0.41
45.4 1.11
+
5
45.1 + 1.06
56.1 +
MAX
1.14
48.4 1 0.74
65.3
50.2 1 0.59
47.4 0.77
54.6 1.61..5 0.5
MSD
80.6 0.33
48.0 1 0.19
65.6
$1.7
46.9
55.1 1.2
E-AT unif.
79.7 0.17
0.50
0.46
55.6 0.54
0.65
$5.7 0.4
E-AT
79.9 0.69
46.6 = 0.24
66.2 0.61
36.0
46.4 = 0.28
56.3 + 0.3
PreAct ResNet-18
AT
84.0 0.31
48.1
59.7 + 0.41
0.97
6.3
38.0 + 0.3
l-
88.9 0.57
27.3
68.7 + 0.09
1..31.660
20.9 1.80
40.5
4-AT
85.9 1.07
22.1 0.14
64.9 1 0.49
59.5
0.09
48.8 1.4
SAT
8..9
40.7
68.0
5..0
54.2 0.66
AVG
84.6
40.8 0.66
8.740.71
0.37
40.1 1 0.78
53.8 0.1
MAX
80.4 1 0.54
45.7 0.90
66.0 0.41
48.6
44.0 0.71
53.4 0.5
MSD
81.1 + 1.14
44.9 0.63
65.9 65.90..6
49.5 1.18
43.9 + 0.76
53.4 0.4
E-AT unif.
82.2 + 1.84
42.7 + 0.74
0.46 67.5 0.46
53.6
42.4 0.60
54.6
E-AT
81.9 1.44
43.0 0.87
66.4 + 0.58
5.20
4..4
54.2
Table 6 repeats the results from Table D and Table 5 with additionally the average robust accuracy the three threat models (last column), with standard deviation over 3 random seeds. On WRN-
16

---

Under review as a conference paper at ICLR 2022
28- 10, in this metric E-AT achieves the best results, and in particular significantly outperforms MAX while having the same clean accuracy. For the smaller RN-18, E-AT unif. attains the highest average robustness, with E-AT and SAT being slightly worse.
C FINE-TUNING WITH E-AT FOR MULTIPLE NORMS ROBUSTNESS
C.I FINE-TUNING ROBUST AND NATURAL MODELS
Table 7: CIFAR-10
epochs of fine-tuning with E-AT: We report the results of fine-tuning PreAct ResNet-18 18 models to become robust wrt the union of the threat models. Fine-tuning any 1p- robust model leads to competitive clean and robust accuracy to full training, differently from using naturally trained model.
model
clean
Loo (Coo
255
la (c2
0.5)
(c:
12)
union
RN-18 - standard
94,4
0.0
0.0
0.0
0.0
FT
66.6
-27.8
29.9
29.9
50.1
50.1
38.5
38.5
29.8
29.8
RN-18 8 100
83.7
48,1
59.8
7.7
7.7
FT
82.3
-1.4
43.4
-4.7
68.0
8.2
48.0
40.3
41.2
33.5
RN-18 12
88.2
29.8
68.6
27.5
23.1
FT
85.4
-2.8
40.6
10.8
69.8
1.2
48.7
21.2
39.1
16.0
RN-18 l
87.1
22.0
64.8
60.3
22.0
FT
83.5
-3.6
40.3
18.3
68.1
3.3
55.7
-4.6
40.1
18.1
We fine-tune for 3 epochs PreAct ResNet-18 either naturally trained or robust wrt single norm. Table shows clean and robust accuracy for each threat model for the initial classifier and after E-AT hine-tuning: while for all robust models the fine-tuning yields values competitive with the full training for multiple norms (see 5), starting from a standard model leads significantly lower both clean performance and robustness in the union of the three lp-balls.
C.2 RUNTIME WITH LARGE MODELS
We reported in Table and Table 3 the runtime per epoch of E-AT. For larger architectures the computational cost increases significantly, and adversarial training with the WideResNet-70-166 the largest one we consider, on CIFAR-10 takes, in our experiments, around 6100 s per epoch when training
using only the training set and over 10000 if the unlabelled data is used (since twice steps are effectively used). This shows how transfering robustness with fine-tuning might allow to obtain classifiers robust wrt different threat models fast and much lower computational cost.
C.3 RESULTS OVER RANDOM SEEDS AND EFFECT OF BIASED SAMPLING SCHEME
Table 8: CIFAR-10 Uniform biased sampling in E-AT for fine-tuning: We fine-tune with E- AT for 3 epochs the RN-18 robust wrt individual norms with either uniform (E-AT unif.) or biased (E-AT) sampling scheme (mean and standard deviation of the clean and robust accuracy over 5 seeds is reported). The biased sampling scheme is helpful when fine-tuning the la and 4 models which are not robust in the most challenging threat model, i.e. lsc.
model
clean
I.o (Exo 255
la (62 = 0.5)
1 (61 = 12)
wnion
RN-18 las uniform
82.6 + 0.62
44.4 44.3 0.63
68.0 0.26
48.5 + 0.96
42.2 + 0.34 42.2 0.78
RN-18 Lao biased
82.7 0.41
68.1 0.48
48.7 0.46
RN-18 la
uniform RN-18 lg biased
85.9 0.51
40.0 0.65
69.4 + 0.68
50.3 0.44
38.9 0.79
85.8 0.68
40.7 0.90
69.5 + 0.50
49.5 0.54
39.4 + 0.72
RN-18 uniform RN-18 biased
83.5 0.68
39.8 + 0.51
68.0 + 0.19
55.8 0.55
39.6 0.47
83.6 0.56
40.5 +
68.1 + 0.13
55.3 + 0.38
40.3 0.33
17

---

Under review as a conference paper at ICLR 2022
We study the effect of randomness in the training process when fine-tuning for robustness mul- tiple norms using E-AT. Moreover, we compare the biased sampling scheme introduced in Eq. (5) to uniform when to choose which 4,. with p {1, attack to for adversarial training for each batch. In Tab report the average results and corresponding standard deviation over runs with different seeds CIFAR-10 models originally robust wrt single
For all metrics and starting models different runs show similar performance (the largest standard deviation is 0.96%) showing the stability of the scheme. Note that in the previous experiments for all runs and methods, without selecting the best one in pool. Moreover, the biased sampling schemes yields slightly better results when fine-tuning the 12- and in particular the 4 rrobuttt model: we hypothesize that, since lxc. is the most challenging threat model, it is important to use it more often at training time when the initial model is non robust wrt 1oo.
used the same seed
C.4 RESULTS USING MORE EPOCHS
Table CIFAR-10- Fine-tuning for more epochs: We show the effect of fine-tuning for different
number of epochs (3 is the standard we use) the PreAct ResNet-18 18 (standard or robust wrt loo).
model
clean
Loc (E50
(e2 0.5)
(e1
12)
umion
RN-18 Lo0
83.7
48.1
59.8
7.7
7.7
epochs FT 82.3
-1.4
43.4
-4.7
68.0
8.2
48.0
40.3
41.2
33.5
epochs FT
83.0
-0.7
45.2
-2.9
68.8
9.0
50.1
42.4
43.1
35.4
epochs
83.1
-0.6
44.6
-3.5
68.7
8.9
50.4
42.7
42.6
34.9
+ 10 epochs FT
84.0
0.3
44.9
-3.2
69.2
9.4
51.0
43.3
42.8
35.1
+ 15 epochs FT
84.6
0.9
44.9
-3.2
69.5
9.7
52.1
44.4
43.2
35.5
RN-18 standard
94.4
0.0
0.0
0.0
0.0
+ 3 epochs FT
66.6
-27.8
29.9
29.9
50.1
50.1
38.5
38.5
29.8
29.8
+ epochs FT
70.6
-23.8
33.8
33.8
55.2
55.2
44.4
44.4
33.4
33.4
epochs FT
72.1
-22.3
36.1
36.1
58.9
58.9
45.9
45.9
35.6
35.6
+ 10 epochs
75.4
-19.0
37.1
37.1
61.0
61.0
47.9
47.9
36.9
36.9
+ 15 epochs FT
76.0
-18.4
40.2
40.2
61.6
61.6
49.2
49.2
40.0
40.0
Table shows the effect of our E-AT-finettuning for different numbers of epochs on either RN-18 robust wrt loo or naturally trained. For the robust model, with longer training the clean accuracy progressively improves, as well as the robustness in the union of the threat models. particular, the models fine-tuned for 15 epochs has robust accuracy similar to that achieved by the MAX-training (43.2% compared 13.3% see Table J.. while still being significantly faster even considering the training time of the initial model. When starting from a standard model, E-AT fine-tuning leads to a large drop clean accuracy while the robustness the union remains lower than what can be achieved with robust models, even when using 15 epochs.
ImageNet Fine-tuning for more epochs: We fine-tune the lg-robust model from Engstrom et al. (2019) for either I or epochs with our E-AT scheme.
Table 10:
model
clean
Loo (Ex
(62 2)
(61
255)
union
RN-50 1g Engstrom et al.
58.7
25.0
40.5
14.0
13.5
2019
epochs FT
56.7
-2.0
26.7
1.7
41.0
0.5
25.4
11.4
23.1
9.6
epochs
57.4
-1.3
27.8
2.8
41.6
1.1
26.7
12.7
23.7
10.2
Additionally we fine-tume tinettuneee for epochs, instead as done above, the 1o-rob model on ImageNet from Engstrom et al. (2019) with our E-AT. Table [0 shows that the longer fine-tuning improves all the performance metrics between 0.6% and 1.3%.
18

---

Under review as a conference paper at ICLR 2022
Table 11: CIFAR-10 Fine-tuning for 1 epoch: We show the effect of fine-tuning for a single (compared to the standard 3) models robust wIt a single norm.
model
clean
Loo (cx
(c2 0.5)
14 (G1
12)
union
R0-50
Engstrom al.
88.7
50.9
59.4
5.0
5.0
2019
epochs FT 84.8
-3.9
46.6
-4.3
68.3
8.9
47.2
42.2
42.8
37.8
epochs FT
86.2
-2.5
46.0
-4.9
70.1
10.7
49.2
44.2
43.4
38.4
RN- 50 Engstrom al.
91.5
29.7
70.3
27.0
23.0
2019
epochs FT 85.9
-5.6
41.8
12.1
69.6
-0.7
47.6
20.6
39.7
16.7
epochs FT
87.8
-3.7
43.1
13.4
70.8
0.5
50.2
23.2
41.7
18.7
RN-18 Crace & Hein
87.1
22.0
64.8
60.3
22.0
2021
epochs FT epochs FT
78.9
-8.2
37.7
15.7
62.9
-1.9
51.3
-9.0
37.6
15.6
83.5
-3.6
40.3
18.3
68.1
3.3
55.7
-4.6
40.1
18.1
C.5 RESULTS ON CIFAR-10 USING A SINGLE EPOCH
Since single epoch of fine-tuning on ImageNet, here test its effect on CIFAR-10. In Table TT we fine-tune with E-AT models adversarially trained wrt single norm for epoch: this is sufficient significantly increase the robustness in the union of the threat models, which gets close to that obtained with the standard epochs (differences are in the range 0.6% to 2.5%). In particular, the loo robust classifier is again the most suitable for the fine-tur one-tuning, since has been trained in the most challenging threat model.
C.6 FINE-TUNING WITH OTHER METHODS
Table CIFAR-10 Other methods vs E-AT for fine-tuning: We fine-tune with different meth-
ods for multiple norms for 3 epochs the RN-18 robust wrt individual norms (mean and standard deviation the clean and robust aceuracy over seeds reported). Additionally, we report E-AT with 6 epochs since it is at least two times faster than MAX and MSD.
model
clean
1 (Ex0
(62 0.5)
1 (61 12)
union
RN-18 loc SAT
83.5 + 0.23
43.5 + 0.16
68.0 + 0.43
47.4 = 0.48
41.0 + 0.26
RN-18 lxc -AX
82.2 + 0.33
45.2 0.39
67.0 0.68
46.1 0.44
42.2 + 0.56
RN-18 lac MSD
82.2 + 0.42
44.9
67.1 + 0.64
47.2 0.59
42.6 + 0.17
RN-18 lx E-AT unif.
82.6
0.37
68.0 0.26
48.5 0.96
42.2 + 0.34
RN-18 L.s E-AT
82.7 0.41
0.63
68.1 1 0.48
48.7 = 0.46
42.2 1 0.78
RN-18 lxc E-AT ep.
83.2 2 0.41
4.59
68.2 : 0.29
50.0 0.77
42.3 + 0.64
RN-18 12 SAT
86.8 + 0.33
3.35
69.6 + 0.69
49.1 = 0.62
37.2 1 0.41
RN-18 12 MAX
85.1 = 0.83
0.29
69.4 1 0.22
45.6 0.46
40.0 1 0.33
RN-18 MSD
85.3 0.42
0.70
69.2 0.23
44.0 0.45
39.0 1 0.50
RN-18 E-AT unif.
85.9 0.51
4.60
69.4 0.68
50.3 0.44
38.9 + 0.79
RN-18 12 E-AT
85.8 0.68
0.90
69.5 0.50
49.5 0.54
39.4 0.72
RN-18 la E-AT cp.
86.3 = 0.22
41.8 0.15
70.2 7 0.31
50.1 0.35
40.5 + 0.31
RN-18 4 SAT
85.1 = 0.32
3.64
68.3 + 0.40
0.40
55.0 0,73
38.2 + 0.64
RN-18 4 MAX
82.2 = 0.34
4.51
66.8 + 0.53
48.1 0.35
41.5 + 0.38
RN-18 1 MSD
81.8 + 0.72
42.7400.4
66.8 + 0.36
47.9 0.57
41.5 + 0.49
RN-18- E-AT unif
83.5 0.68
+
68.0 0.19
55.8 0.55
39.6 + 0.47
RN-18- 4 E-AT
83.6 + 0.56
40.5
0.38
+ 0.13
55.3 0.38
40.3 + 0.33
RN-18 4, E-AT cp.
84.2 + 0.36
41.2 + 0.47
68.7 + 0.48
55.9 0.33
40.9 + 0.57
We here explore the option of fine-tuning robust models with techniques other than E-AT, and report the results in Table 12 In particular, we test SAT, MAX and MSD for fine-tuning the RN-18 robust single norms done in Table 8 (the results of E-AT unif. and E-AT taken from there).
19

---

Under review as a conference paper at ICLR 2022
First, sees that SAT performs significantly worse than the other methods in this scenario, for all the fine-tuned classifiers. Second, we observe that E-AT achieves very similar results to MAX and MSD, especially when using the 1x -roust classifier which yields the highest robustness in the union. However, we highlight that MAX and MSD are 3x and 2x more expensive than E-AT (see Taalles and 5). Thus, include in Table T2 B-AT' with twice the budget, that is 6 epochs, which has comparable cost to MSD and closes the small gap (on average MAX and E-AT with 6 epochs perform equal and are slightly better than MSD). Since one of our goals is reduce the cost of getting multiple-non norm robust classifiers, we use E-AT as the main tool for fine-tuning
C.7 FINE-TUNING PERCEPTUALLY ROBUST MODELS
Table 13: CIFAR-10 3 epochs of E-AT fine-tuning on ly-robust models: We use E-AT fine- tune models robust wrt single lp -n-orrm for multiple-norn robustness, and report the robust accuracy on 1000 test points for all threat models, and the difference compared to the initial classifier.
model
clean
loc (cxo
8
12 (E2 = 0.5)
I (EI = 12)
umion
265
RN-50 Lo
88.7
50.9
59.4
5.0
5.0
[Engstrom al.
2019)
+
86.2
-2.5
46.0
-4.9
70.1
10.7
49.2
44.2
43.4
38.4
RN-50- la
91.5
29.7
70.3
27.0
23.0
Engstrom al.
2019)
+
87.8
-3.7
43.1
13.4
70.8
0.5
50.2
23.2
41.7
18.7
RN-50 PAT
82.6
31.1
62.4
33.6
27.7
[Liddlaw et al.
2021
83.7
1.1
43.7
12.6
68.5
6.1
50.7
17.1
42.3
14.6
We test the effect of E-AT fine-tuning on model trained to be robust to perturbations which are aligned with human perception. In particular, we use the classifier obtained with perceptual adver- sarial training (PAT), i.e. wrt the LPIPS metric, from Laidlaw et al. (202I). and compare it to two models with the same architecture (ResNet-50) adversarially trained wrt Too and 12. Table 13 shows the robustness in every threat model for the original models and those obtained with 3 epochs of E-AT fine-tuning, The PAT classifier has initially the highest robustness in the union, confirming the obersevation of Laidlaw et al. (02I)) that PAT provides some robustness to unseen attacks, After fine-tuning, all three models achieve similar worst-case robustness, with the classifier originally lo-- robust being slightly better. This shows that our E-AT fine-tuning is effective even when applied to models adversarially trained not wrt an ly norm.
D ROBUSTNESS AGAINST UNSEEN NON L,-- L,-BOUNDED ATTACKS
Table 14: CIFAR-10 Robustness against non 4, by-bounded attacks: We test the robustness of WRN-28-10 trained different threat models against different types of attacks. Moreover, we add the PAT model from Laidlaw et al. (2211), which uses RN-50 as architecture.
model
clean comm.
lo
patches frames
fog
snow
gabor
elastic
jpeg
union
avg.
NAT
94.4
71.6
0.1
2.6
47.3
3.9
35.0
0.0
0.0
12.2
lx--T
81.9
72.6
7.3
21.6
26.2
36.0
35.9
52.5
59.4
5.1
2.0
30.5
b-AT
87.8
79.2
13.2
25.0
17.7
44.9
22.1
43.5
56.6
14.0
4.5
29.6
4-AT
83.5
75.0
40.9
41.3
21.1
35.6
20.6
41.2
53.3
25.5
8.6
34.9
PAT
82.6
76.9
23.3
37.9
21.7
53.5
25.6
41.8
53.5
13.7
8.0
33.9
E-AT
79.1
71.3
39.5
37.7
30.5
34.8
33.4
50.2
58.6
38.7
15.9
40.4
We here want to study to which extent the robustness achieved with multiple norms training general- izes to unseen and possibly very different threat models on CIFAR-10. We select three sparse attacks ao- bounded, patches and frames) and five adversarial comruttion (fog, snow, Gabor noise, elastic,
20

---

multiple Under review as a conference paper at ICLR 2022
(P-)) from Kang al. 2019b). Additionally, compute the accuracy of the classifiers on the common corruptions. which are not udversarially optimized, of CIFAR- 10-C (eendrycks Diet- terich 2019). Table 14 shows the results against such attacks of WRN- 28- 10 adversarially trained wrt single norms and with E-AT, and the PAT model from Laidlaw et al. (021), which uses RN-50 architecture. We add also naturally trained model as further baseline. The model trained wrt multiple norms shows much higher robustness in both the union and average of these attacks, almost 2x higher in union than the best of the other models. Moreover, for almost all threat models, the E-AT model attains the highest robustness very close, while the best among the single 1 b--rouust models varies: this means that training for multiple norms robustness allows summarize different kinds of robustness in the same classifier. Finally, E-AT outperforms even the PAT model which is trained wrt LPIPS and aims at generalization to unseen attacks.
Experimental details: To test robustness against ln-attacks we use Sparse-RS (Croce et al. 2020b) with a budget 18 pixels and 10k queries. We adopt patches of size
pixels, optimized with the PGD-based from Rao et al. (2020) (without constraints the position of the patch on the images), and frames of width pixel (Zajac et al. 2019), again optimized with PGD: in both cases 10 random restarts of 100 iterations. For the adversarial corruptions use the original implementation (Kang et al.] (2019b) with 100 iterations and search for budget e for which the models show different levels of robustness (in details, for fog 128,
0.5, Gabor noise e = 60, elastic = 0.125, I -JPGG 0.25). Finally, we average the classification accuracy over the 5 severities of the common corruptions. All the statistics are on 1000 test points.
E EXPERIMENTS ON MNIST
Table 15: MNIST Comparison of full training schemes and fine-tuning with E-AT for mul- tiple norm robustness: We train classifier (architecture as in Maini et al. (2020) on MNIST with different training scheme. For SAT and E-AT report, together with the statistics random seeds, the results of the best run. Additionally, we show the results of fine-tuning the A-AT model with E-AT for different numbers of epochs, which achieves the best results. (*) AVG, MAX and MSD classifiers are those provided by Maini et al. (2020).
model
clean
L0 (6xx 0.3)
12 (c2 2)
4 (cI 10)
union
98.9 + 0.12
90.0 0.45
8.4 1.49
6.0 0.65
+ 0.73
l--AT
98.8 + 0.17 98.8 + 0.09
0.0 + 0.05
70.7 + 0.26
59.1 0,37
0.0 + 0.05
4-AT
0.0 + 0.00
45.8 + 0.42
77.2 + 0.14
0.0 + 0.00
SAT
98.6 + 0.17
62.0 0.86
65.7 1.69
61.3 + 1.29
53.9 + 1.25
AVG (*)
99.1
58.6
60.8
22.5
21.1
MAX (*)
98.6
39.4
59.9
25.6
20.3
MSD
98.2
63.7
66.6
51.0
48.7
E-AT unif.
98.8 + 0.12
67.1 3.03
50.2 + 4.93
62.0 + 4.59
45.9 + 4.43
E-AT
98.7 1 0.12
69.0 1 3.66
56.4 1.44:39.94 3.94
61.1 1 4.03
50.6 1 3.89
b-AT E-AT (3 ep.)
96.9 1 0.32
$7.5 1 0.92
67.8 1 0.58
62.0 1 1.16
54.6 1 0.59
AT E-AT (5 ep.)
97.4 0.21
60.6 2.19
65.9 1 0.56
63.8 1 0.93
56.2 1 1.16
best
SAT
98.8
62.7
67.2
62.6
55.3
E-AT unif.
98.9
67.4
56.3
63.6
51.6
E-AT
98.8
71.0
58.4
62.0
54.4
b-AT E-AT ep.)
97.3
58.0
67.9
62.9
55.7
b- E-AT (5 ep.
97.5
61.6
66.2
64.0
57.5
We further test the different techniques on the MNIST dataset. We use the same CNN of Maini
et al. 2020 as architecture and 6.c = 0.3, 62 = 2 and é = 10 as threshold at which evaluating
robustness, done by Maini al. 2020 We note that while it is an easier dataset, MNIST is challenging when it comes to adversarial training since presents unexpected phenomena: e.g.
Tramèr & Boneh 2019) noted that l adversaaraa training induces gradient obfuscation when using
21

---

Under review as a conference paper at ICLR 2022
attack wrt la and 11, and both Tramer & Boneh 2019); Maini et al. (2020) had use many PGD- steps (up to 100), and Tramer & Boneh (2019) even a ramp-up schedule for the e during training. While other modifications to the training setup might be beneficial for some or all the methods, just increased the number of APGD--teps to 50 for 1 (see more details below). In Table [5 we compare E-AT to SAT, AVG, MAX and MSD (for the last three we use the models provided by Maini et al. (2020). First, E-AT outperforms the available classifiers trained with AVG, MAX and MSD, meaning that even on MNIST it a strong baseline. However, in this case, SAT, which trains on all types of perturbations, achieves better results than E-AT on average: E-AT has higher variance over runs but the best run (over multiple seeds) is close the best one of SAT in terms of robustness in the union (55.3% vs 54.4%). Interestingly, SAT has much higher robustness wrt la compared to E-AT, but this somehow expected since Eq. S would "predict" robustness for E-AT at 62 R 1.7 while 62 2 is used for testing, and this is precise only for linear models. Thus the slightly worse performance of E-AT compared to SAT for the chosen radii of the threat models is to be expected from our geometric analysis.
Moreover, since we have shown that fine-tuning an fp-robust model with E-AT yields high multiple norms robustness, and given that E-AT from random initialization is weak mostly wrt 12 fine- tune the 12-AT classifier with E-AT. This, with just or epochs, significantly outperforms SAT (up +2.3% robustness in the union), while preserving I2-robustness. In total, we improve the previous SOTA for multiple-norm robustness for MNIST from 48.7% (MSD) to 57.5% (E-AT fine-tuning of an 12-robust model with 5 epochs) which is significant improvement. Note that in this case we increase the radii Eoo and E1 to 0.33 and 14 respectively to preserve the (g-robustness: in fact, with such values Eq. yields 62 2.16. However, training from random initialization, the standard setup, with the larger thresholds leads to worse robustness in the union. We hypothesize that this due the increased difficulty of the task to learn: it known that even single norm adversarial training is problematic when increasing the value of (Ding et al. 2020).
For training we use 30 epochs with cyclic learning rate (maximum value 0.05, also used for fine-tuning) and no data augmentation (other settings as for CIFAR-10). As mentioned, we use in adversarial training for multiple norms (SAT, E-AT unif. and E-AT) 50 steps of APGD for (,, and to reduce the training cost we decrease to those for loc. Moreover, for training, in 1--AGG we increase the parameter to control the initial sparsity of the updates to 0.1 (default is 0.05). For evaluation, and Square Attack (Andriushchenko et al. 2020) are at times stronger than PGD-based attacks, as shown in the original papers.
Experimental details:
use the full AutoAttack, since MNIST FAB (Croce & Hein 2020a)
22


Reviews:
[{"review_id": "hcUOKxQPGY1", "reviewer": "Reviewer_LQa3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper is generally well-written. The motivation for the problem and the E-AT method are clear, and it seems like the experiments were carefully run.\n\nStill, I have some concerns about whether the results support the proposed method. It seems like the main contribution proposed by the authors is the idea of training against only $\\ell_1$ and $\\ell_\\infty$ attacks to get robustness to $\\ell_p$ attacks for other $p$ values. However, when selecting between $\\ell_1$ and $\\ell_\\infty$ attacks randomly, the method performs nearly identically to (or maybe a bit worse than) SAT which randomly selects between $\\ell_1$, $\\ell_2$, and $\\ell_\\infty$ attacks at each iteration. The proposed method is only superior to SAT when using the proposed biased sampling scheme. Thus, it seems like it is actually the biased sampling scheme—not the use of $\\ell_1$ and $\\ell_\\infty$ threat models—that leads to the improvement. As far as I can tell, none of the other experiments in the paper clearly separate the effect of these two changes to SAT-type methods.\n\nIn general, the paper seems like a collection of three separate ideas:\n\n 1. restricting adversarial training against multiple $\\ell_p$ norms to only $\\ell_1$ and $\\ell_\\infty$, which is the subject of the theory part of the paper and the motivation for the name E-AT,\n 2. the biased sampling scheme,\n 3. and the idea of using fine-tuning to quickly impart robustness against a different $\\ell_q$ threat model to a model trained with an $\\ell_p$ threat model for $q \\neq p$.\n\nAs I mentioned above, the effects of 1 and 2 are currently difficult to disentangle, and 3 seems somewhat orthogonal to the other two contributions. In particular, in Table 4 it seems that adversarial fine-tuning (3) is effective even without 1 and 2. The three ideas are mostly just connected by the common problem they aim to solve—producing robustness against different and multiple $\\ell_p$ norms. Even the title of the submission seems split between the different ideas. Thus, it would be helpful if the authors could better motivate the combination of E-AT and fine-tuning in the same paper, or even consider splitting the paper into two submissions.\n", "summary_of_the_paper": "This paper proposes a method to produce image classifiers which are adversarially robust against multiple $\\ell_p$ threat models—in particular, against $\\ell_1$, $\\ell_2$, and $\\ell_\\infty$ attacks. The method involves training against $\\ell_1$ and $\\ell_\\infty$ attacks with the hypothesis that this will additionally give robustness for $\\ell_p$ threat models with $1 \\leq p \\leq \\infty$. This hypothesis is supported by prior results that proved that affine classifiers robust to $\\ell_1$ and $\\ell_\\infty$ threat models are also be robust to other $\\ell_p$ threat models. The authors test their method on CIFAR-10 and ImageNet for both training classifiers from scratch and for fine-tuning robust models trained on one $\\ell_p$ threat model to the other $\\ell_p$ threat models.", "summary_of_the_review": "Overall, the paper could use some more work to separate the effects of the components of E-AT and motivate the combination of it with the fine-tuning contribution, which seems orthogonal. Thus, I do not recommend acceptance, although I am open to raising my score during the discussion period.\n\n**After discussion period:** My overall position is that I would be more supportive of accepting the paper if it focused on fine-tuning over E-AT. Currently, the extensive focus on E-AT detracts from what seems like the more important contribution and could lead that contribution to be overlooked. However, I believe changing the focus would require a major revision and thus I maintain my weak reject score. Nonetheless, I respect that some of the other reviewers have different opinions on whether the paper should be published and I hope the AC weighs all our points to make the final decision.\n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "g_vIxK1k8g", "reviewer": "Reviewer_tpW1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n1. This paper makes a very strong contribution to the sub-field of multiple norm adversarial robustness by showing that one can achieve this goal by spending a small budget on any pre-trained robust model.\n2. I like the graphs in Figure 4 of the Appendix because they highlight how prior methods naturally do extreme norm sampling in the given setting of CIFAR-10. However, as a word of caution here -- in my understanding, you will not see the same trend in the case of MNIST -- this makes me speculate the performance of EAT on MNIST.\n3. The overall paper is written very comprehensively, and the authors have done an extensive study of various pre-trained models. The authors have already included most experiments that I would have been curious to know as a reader -- such as fine-tuning standard models, using EAT from scratch, performance on Imagenet, comparison with other prior works. I am very satisfied by the presentation.\n\n## Suggestions/ Weaknesses\n1. Can you also see the same results on MNIST? I suspect this is hard because the $\\ell_\\infty$ model is not robust to $\\ell_2$ attacks in MNIST. I believe that CIFAR-10 is just acting as a nice test bench for this kind of work because the problem was biased against $\\ell_2$ attacks (because of how perturbation budgets have been historically chosen in literature). There is no reason why the perturbation budget of interest should be 0.5, especially when you consider that the robustness of $\\ell_2$ models against $\\ell_2$ attacks is significantly more than that of any other $\\ell_p$ robust model against $\\ell_p$ attacks.\n2. Comparison with CCAT or other methods such as PAT would be helpful. Can you also fine-tune models beyond $\\ell_p$ robustness -- to say that this is a general property of adversarial robustness and not particularly hinged on geometry?\n\n\n## Questions\n1. Are you using AutoAttack (plus) or standard? I would suggest using the plus version to be comparable to other works.\n2. How is EAT-unif different from AVG (if $\\ell_2$ was not considered)? Noting that in most cases unif and biased sampling have nearly the same effect.\n3. Do we really need to use EAT to fine-tune models? What about using MAX or MSD as a fine-tuning step? Does this perform better? Since in most results where EAT is independently used, the performance is sub-optimal. I wonder if there is any reason why EAT should be optimal for fine-tuning or if it needed to be introduced in the first place?\n4. What is the impact of the additional data versus running 6 epochs of fine-tuning?\n\n## Post-author response\nI am voting for acceptance of this work. However, I also understand why the biggest contribution of this paper might be hard to grasp for the average reader. The authors spend too much time and paper space to highlight how EAT is novel. I do not think that is the case (it is a heuristic modification that works in practice!), nor do I think this paper needs that to be accepted. The remaining contribution is significant enough for me to vote for acceptance -- and in fact one of the most significant developments in multi robustness after the initial few papers. I would encourage the authors to spend more time to re-evaluate what is the most significant contribution in their opinion, and possibly rethink how to go about the prose.\n\n", "summary_of_the_paper": "This paper addresses the problem of multiple perturbation adversarial robustness for attacks subsumed within $\\ell_p$ regions for $p\\in{1,2,\\infty}$. The main contribution of this work is to show how a model robust to a particular attack type (typically $\\ell_\\infty$) can be fine-tuned (at low cost) to be robust against multiple (or alternate) perturbation types. The authors build on prior formalization about the geometry of $\\ell_p$ balls (by Croce et. al.) to empirically demonstrate its effect. The results are convincing and evaluated against AutoAttack which is", "summary_of_the_review": "The paper has a very strong contribution to the field of multi-norm adversarial robustness. However, I am concerned if this phenomenon generalizes to other datasets -- based on some observations noted by the authors. I would be happy to raise my scores if the authors can show results on MNIST as well.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "eAhCwQi-b1b", "reviewer": "Reviewer_XXNy", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n(1) The proposed E-AT method is computationally affordable since it doesn’t use all types $\\ell_{p}$ adversarial examples to do training, it only alternates between the two extreme norms. \n\n(2) E-AT could quickly fine-tune models that are only robust against one perturbation type into models that are robust against the union of multiple $\\ell_{p}$ adversarial perturbations, and the fine-tune results are shown better than simple $\\ell_{p}$ norm fine-tuning. \n\nWeaknesses:\n\n(1) Although the paper demonstrates its motivation in Sec 3.1, the reviewer is still confused about the geometry theory proposed to defend multiple $\\ell_{p}$ perturbations. It’s indeed correct for a simple one-layer linear classifier that model being robust in both $\\ell_{1}$ and $\\ell_{\\infty}$-ball is also robust w.r.t the largest $\\ell_{p}$-ball that fits into the convex hull of the union of the $\\ell_{1}$ and $\\ell_{\\infty}$-ball. However, when it comes to the deep neural network which is in the high-dimensional manifold, there is no guarantee for the above statement since there is no such model that is absolutely robust (i.e., robust accuracy is 100%) against certain norm types on CIFAR-10 or ImageNet dataset. Being partly robust on $\\ell_{1}$ and $\\ell_{\\infty}$ (e.g., robust accuracy is 65.88%) doesn’t mean being robust within the $\\ell_{p}$-ball that fits into the convex hull of the union of the $\\ell_{1}$ and $\\ell_{\\infty}$-ball. There should be more proof about the certified robustness of the proposed method. \n\n(2) In Fig 1, the paper shows the model adversarial robustness against multiple norm perturbations after E-AT fine-tuning is better than L1 fine-tuning. It may be unfair since the evaluation metric for union robustness is the sample-wise worst-case accuracy, and [Ref1] demonstrates that $\\ell_{1}$ and $\\ell_{\\infty}$ robustness are mutually exclusive so fine-tuning on $\\ell_{1}$ perturbation will largely decrease the robustness on $\\ell_{\\infty}$ perturbation, causing the bad results under the worst-case evaluation metric. More fine-tuning is needed for the baseline methods to balance the robust accuracy on all norm types. \n\nReferences\n\n[Ref1] Tramer F, Boneh D. Adversarial Training and Robustness for Multiple Perturbations[J]. Advances in Neural Information Processing Systems, 2019, 32: 5866-5876.\n", "summary_of_the_paper": "This paper mainly studies the problem of defending multiple norm adversarial perturbations. The authors propose extreme norms adversarial training (E-AT), which leverages different geometry of the $\\ell_{p}$-balls to conduct adversarial training by adaptively alternating between the $\\ell_{1}$ norm and $\\ell_{\\infty}$ norm. They also show that using E-AT fine-tune could turn $\\ell_{p}$ robust model into a model that is robust against the union of $\\ell_{p}$ adversarial perturbations. The authors also provide some theoretical proof for their method. ", "summary_of_the_review": "The motivation and the theoretical effect of the proposed defense should be clarified and demonstrated, and some experimental comparisons need to be improved. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "F8qz3OV9tse", "reviewer": "Reviewer_bnsW", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\nThe paper tackles an important problem of training models robust against multiple attacks efficiently. The overall paper is well written and easy to follow. The proposed evaluation compares state-of-the-art single and multi-perturbation robustness methods, and the conducted experiments on ImageNet are impressive.\n\n---\n\n### Weaknesses\n- The paper is limited in novelty. While the fine-tuning aspect is novel, but the similarity between the $\\ell_1$ and $\\ell_2$ perturbation was highlighted previously in Maini et al. (2020) [1], where it was shown that the first two principal components of $\\ell_1$ and $\\ell_2$ adversaries are largely overlapping. Therefore, I would recommend including a discussion about their conducted analysis to clarify the contributions of the proposed method. \n- Further, E-AT is simply a modification of SAT, where E-AT uses the proposed adaptive sampling instead of uniform sampling.  The gains by the proposed method are marginal in robustness and training time compared to SAT. While E-AT gains marginal improvement in union accuracy, SAT obtains higher clean accuracy. Therefore, the effectiveness of E-AT is limited from these experimental evaluations.\n- The theoretical and empirical analysis is restrictive to $\\ell_p$-norm.  The scalability to non-$\\ell_p$ attacks (e.g., spatial attacks, patch attacks, common corruptions, unforeseen adversaries) is unclear. While $\\ell_p$ attacks are a standard in the community, it is essential to show the evaluation to unseen non-$\\ell_p$ attacks to demonstrate the success of the proposed method in practical scenarios.\n\n***Other questions and comments*** \n- E-AT obtains lower accuracy on clean examples and across all the $\\ell_p$-norms in Table 1 compared to the other baselines trained with multiple attacks. Can the authors also report the average metric following Tramèr et al. (2020) and Madaan et al. (2021) to show if E-AT effectively mitigates the robustness tradeoff between multiple attacks compared to prior works?\n- How is the performance for fine-tuning on CIFAR-10 with a single epoch? \n- How does the performance vary on increasing the fine-tuning epochs further in Table 8? Can the authors also report the results for ImageNet?\n- The performance difference between uniform and biased sampling in E-AT is marginal (Table 5 and Table 7). Comment.\n- Adding the numbers for all the bars would enhance the readability of Figure 1.\n\n--- \n### Post Rebuttal\n\nThank you for the response and the additional experimental evaluation conducted during the rebuttal. In light of the clarifications and additional evaluations, I have increased my score to 6.\n\n---\n\n### References:\n[1] Maini et al. (2020). Perturbation Type Categorization for Multiple  Bounded Adversarial Robustness.\n", "summary_of_the_paper": "The paper tackles the problem of robustness against multiple perturbations and proposes extreme norms adversarial training (E-AT) that adaptively alternates between $\\ell_1$ and $\\ell_\\infty$-norm. Furthermore, the paper fine-tunes Gowal et al. (2020) to improve its multi-norm robustness. Finally, the experiments are conducted on CIFAR-10 and ImageNet with APGD for training, showing the proposed method's effectiveness.", "summary_of_the_review": "I think this paper tackles an important problem and conducts an exhaustive experimental evaluation. However, I would like to clarify the points raised in my review and would be happy to raise my score if the authors can successfully address my concerns in the discussion period.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper, authors study adversarial robustness against the union of Lp-threat models. Reviewers had some concerns about this work. They mentioned the paper is not well-organized and the explanations of the novel components should be clearer.  In particular, they suggested authors to study the effects of different components of E-AT and motivate their combinations with fine-tuning. The lack of novelty was another concern. I suggest authors to focus on the fine-tuning part in their revised draft which has more novelty. Given all, I think the paper needs a bit more work before being accepted.
Decision: Reject