Revisiting adapters with adversarial training
Authors: Sylvestre-Alvise Rebuffi
Francesco Croce
Sven Gowal
Venue: ICLR 2023 notable top 25% (2023)
Keywords: adapters
adversarial
robustness
soup

Abstract: While adversarial training is generally used as a defense mechanism, recent works show that it can also act as a regularizer. By co-training a neural network on clean and adversarial inputs, it is possible to improve classification accuracy on the clean, non-adversarial inputs. We demonstrate that, contrary to previous findings, it is not necessary to separate batch statistics when co-training on clean and adversarial inputs, and that it is sufficient to use adapters with few domain-specific parameters for each type of input. We establish that using the classification token of a Vision Transformer (ViT) as an adapter is enough to match the classification performance of dual normalization layers, while using significantly less additional parameters. First, we improve upon the top-1 accuracy of a non-adversarially trained ViT-B16 model by +1.12% on ImageNet (reaching 83.76% top-1 accuracy). Second, and more importantly, we show that training with adapters enables model soups through linear combinations of the clean and adversarial tokens. These model soups, which we call adversarial model soups, allow us to trade-off between clean and robust accuracy without sacrificing efficiency. Finally, we show that we can easily adapt the resulting models in the face of distribution shifts. Our ViT-B16 obtains top-1 accuracies on ImageNet variants that are on average +4.00% better than those obtained with Masked Autoencoders.

Published as a conference paper at ICLR 2023
REVISITING ADAPTERS WITH ADVERSARIAL TRAINING
Sylvestre-Alviss Rebuffi, Francesco Croce" & Sven Gowal DeepMind, London
(syl vestre, sgowall edeepminde nd. com
ABSTRACT
While adversarial training is generally used defense mechanism, recent works show that it can also act as a regularizer. By co-training a deep network on clean and adversarial inputs, it is possible to improve classification accuracy on the clean, non-adversarial inputs. We demonstrate that, contrary to previous findings, it is not necessary to separate batch statistics when co- co-training on clean and adversarial inputs, and that it is sufficient to use adapters with few domain -ppnain-ppecific parameters for each type of input. We establish that using the classification token of a Vision Transformer (VIT) as an adapter is enough to match the classification performance of dual normalization layers, while using significantly less additional parameters. First, we improve upon the top-1 accuracy of non-adversariall trained VIT-B 16 model by +1.12% on IMAGENET (reaching 83.76% top-1 accuracy). Second, and more importantly, we show that training with adapters enables model soups through linear combinations of the clean and adversarial tokens. These model soups, which we call adversarial model soups, allow trade-off between clean and robust accuracy without sacrificing efficiency. Finally, we show that we can easily adapt the resulting models in the face of distribution shifts. Our VIT-BI6 16 obtains top-1 accuracies on IMAGENET variants that are on average +4.00% better than those obtained with Masked Autoencoders.
INTRODUCTION
Deep networks are inherently susceptible to adversarial perturbations. Adversarial perturbations fool deep networks by adding an imperceptible amount of noise which leads to an incorrect prediction with high confidence (Carlini & Wagner, 2017; Goodfellow et al., 2015; Kurakin et al., 2016b; Szegedy al., 2014). There has been a lot of work on building defenses against adversarial perturbations (Papernot et al., 2016; Kannan et al., 2018); the most commonly used defense is adversarial training as proposed by Madry et al. (2018) and its variants (Zhang et al., 2019; Pang et al., 2020; Huang et al., 2020; Rice et al., 2020; Gowal et al., 2020), which use adversarially perturbed images at each training step as training data. Earlier studies (Kurakin et al., 2016a; Xic et al., 2019b) showed that using adversarial samples during training leads to performance degradation AdvProp (Xic et al., 2019a) challenged this observation by showing that adversarial training can regularizer, and therefore improve nominal accuracy, when using dual batch normalization (BatchNorm) layers (loffe & Szegedy, 2015) to disentangle the clean and adversarial distributions.
clean images. However,
We draw attention to the broad similarity between the AdvProp approach and the adapters literature (Rebuffi et al., 2017; Houlsby et al., 2019) where a single backbone network is trained on multiple domains by means of adapters, where few parameters specific each domain trained separately while the rest of the parameters are shared. In light of this comparison, we further develop the line of work introduced by AdvProp and analyze from an adapter perspective. In particular, we explore various adapters and aim to obtain the best classification performance with minimal additional parameters. Our contributions are as follows:
We show that, in order to benefit from co-training on clean and adversarial samples, it is not necessary to separate the batch statistics of clean and adversarial images in BatchNorm layers. We demonstrate empirically that it is enough to use domain specific trainable parameters to achieve similar results.
Work done during an intersship DeepMind

---

Published as a conference paper at ICLR 2023
Inspired by the adapters literature, we evaluate various adapters. We show that training separate classification tokens of a VIT for the clean and adversarial domains is enough to match the classification performance of dual normalization layers with 49 fewer domain specific parameters. This classification token acts as a conditioning token which can modify the behaviour of the network to be either in clean robust mode (Figure 1).
Unlike Xie et al. (2019a) and Herrmann et al. (2022), we also aim at preserving the robust performance of the network against adversarial attacks. We show that our conditional token can obtain SOTA nominal accuracy in the clean mode while at the same time achieving competitive le -o--uststss in the robust mode. As by-product of our study, we show that adversarial training of VrT-B 6 on IMAGENET leads to state-of-the robustness against foe bounded perturbations of size 4/255.
We empirically demonstrate that training with adapters enables model soups (Wortsman et al., 2022). This allow us to introduce adversarial model soups, models that trade-off between clean and robust accuracy through linear interpolation of the clean and adversarial adapters. To the best of our knowledge, our work is the first to study adversarial model soups. We also show that adversarial model soups perform better IMAGENET variants than the state-of-the-ar with masked auto-encoding (He et al., 2022).
2 RELATED WORK
Adversarial training. Although more recent approaches have been proposed, the most successful method to reduce the vulnerability of image classifiers to adversarial attacks is adversarial training, which generates on-the-fly fly adversarial counterparts for the training images and uses them to augment the training set (Croce et al., 2020). Goodfellow et al. (2015) used the single-step Fast Gradient Sign Method (FGSM) attack to craft such adversarial images. Later, Madry et al. (2018) found that using iterative Projected Gradient Descent (PGD) yields models robust to stronger attacks. Their scheme has been subsequently improved by several modifications, e.g. a different loss function (Zhang et al., 2019), unlabelled or synthetic data (Carmon et al., 2019; Uesato al., 2019; Gowal et al., 2021), model weight averaging (Gowal et al., 2020), adversarial weight perturbations (Wu et al., 2020), and better data augmentation (Rebuffi et al., 2021). While the main drawback of adversarial training is the degradation of performance of robust models on clean images (Tsipras et al., 2018), Xie et al. (2019a) showed that adversarial images can be leveraged as a strong regularizer to improve the clean accuracy of classifiers on IMAGENET. In particular, they propose AdvProp, which introduces separate BatchNorm layers specific to clean or adversarial inputs, with the remaining layers being shared. This approach and the role of normalization layers when training with both clean and adversarial points has been further studied by (Xie & Yuille, 2019; Walter et al., 2022). Recently, Wang et al. (2022) suggest removing BatchNorm layers from the standard RESNET architecture (He et al., 2016) to retain high clean accuracy with adversarial training, but this negatively affects the robustness against stronger attacks.' Finally, (Kireev et al., 2021; Hermmann et al., 2022) showed that carefully tuning the threat model in adversarial training might improve the performance on clean images and in the presence of distribution shifts, such as common corruptions (Hendrycks & Dietterich, 2018).
Adapters. In early work on deep networks, Caruana (1997) showed that sharing network parameters among tasks acts as a regularizer. Aiming a more efficient parameter sharing, (Rebuffi et al., 2017; Rosenfeld & Tsotsos, 2018) introduced adapters- - small training modules specific to each task which can be stitched all along the network. other lines of work, (Mallya et al., 2018; Mancini et al., 2018) adapt a model to new tasks using efficient weight masking and (Li et al., 2016; Maria Carlucci et al., 2017) perform domain adaptation by batch statistics modulation. While these approaches require having as many adapters tasks, Perez et al. (2018) propose an adapter layer whose weights are generated by a conditioning network. Besides computer vision, adapters are also used in natural language processing for efficient fine-tuning (Houlsby et al., 2019; Pfeiffer et al., 2020; Wang et al., 2020) and multi-task learning (Stickland & Murray, 2019).
Merging multiple models. While ensembles are a popular and successful way to combine multiple independently trained classifiers to improve on individual performance (Ovadia et al., 2019; Gontijo- Lopes et al., 2021), they increase the inference cost as they require a forward pass for each sub-network
See past / //git hob. con/ ams nor researc of normel fieer- Free- robu at tae mingg /f issuee /2.
2

---

Published as a conference paper at ICLR 2023
Transformer blocks
Transformer blocks
Transformer blocks
i
Embedder
Embedder
dbstr
Embedder
IMG
IMG
IMG
(a) Clean mode
(b) Robust mode
(c) Model soup
Figure Classification token adapter. The image is embedded into visual tokens (in pink). The behaviour of the model can set to clean mode, robust mode or model soup by respectively using the clean token (in blue), the adversarial token (in orange) linear combination these two tokens. parameters embedder, the transformer blocks and the classifier are shared between modes.
of the ensemble. An alternative approach taken by Wortsman et al. (2022) who propose to fine- tune fully trained model with different hyperparameter configurations and then average the entire set of weights of the various networks. The obtained model soups get better performance than each individual model and even ensembles. Model soups are in spirit similar to Stochastic Weight Averaging (lzmailov et al., 2018) which consists in averaging weights along an optimization trajectory rather than averaging over independent runs.
METHOD
3.1 CO-TRAINING WITH NOMINAL AND ADVERSARIAL TRAINING
Goodfellow et al. (2015) propose ad adversarial training as way to regularize standard training. They jointly optimize the model parameters 0 on clean and adversarial images with the co-training loss
[JJ(J(a) 0).9) y) + (1 oz) max Lff a + 6).).)) SE9
where pairs of associated examples and labels y are sampled from the training dataset, f(: 0) a model parametrized by 0, L defines the loss function (such as the cross- cross-entropy loss in the classification context), and S is the set of allowed perturbations. Setting training on clean images and setting a = 0 leads to adversarial training as defined by Madry et al. (2018). In our case, we consider foo boun-eounded perturbations of size 4/255, so have S == {ō || S E}, and we use untargeted attacks to generate the adversarial perturbations (see details in Section 4).
boils down to nominal
3.2 SEPARATING BATCH STATISTICS IS NOT NECESSARY
BatchNorm is widely used normalization layer shown improve performance and training stability of image classifiers (Ioffe & Szegedy, 2015). We recall that a BatchNorm layer, given a batch as input, first normalizes by subtracting the mean and dividing by the standard deviation computed over the entire batch, then it applies an affine transformation, with learnable scale and offset parameters. During training, accumulates these so-called batch statistics use during test time, so that the output of the classifier for each image is independent of the other images in the batch. The batch statistics can be seen an approximation of the statistics over the image distribution.
Xie et al. (2019a) show that optimizing the co-training loss in Eq. I can yield worse results on clean images than simple nominal training. This is especially the case when the network has a low capacity or the attack (i.e., the inner maximization) is too strong (such as using a large perturbation radius e). To solve this issue, they propose AdvProp, which consists in using distinct BatchNorm layers for clean and adversarial images. They argue that "maintaining in incorrect statistics estimation" which could be the reason for the performance degradation. We note that using two sets of BatchNorm layers for the clean and adversarial samples as in AdyProp
of [BatchNorm] statistics results
3

---

Published as a conference paper at ICLR 2023
80%
60%
60%
Accuracy
Dual BatchNorm
Accuracy
Dual BatchNorm
Shared BatchNorm
Parlmas BatchNorm
40%
40%
Shared GroupNorm mm
Dual GroupNorm
Shared LayerNorm
Dual LayerNorm
20%
20%
step
Step
Figure 2: Dual parameters are enough. We report the clean (solid lines) and robust accuracy (dashed lines) over training steps of RESNET-50 trained on IMAGENET with the co-training loss of Eq. (c = 4/255); for models with dual layers. Clean accuracy refers to the cleun mode and the robust accuracy the robust mode. Lefi panel. We compare models with different normalization layers with domain-specifie parameters (Shared BatchNorm, Shared LayerNorm, Shared GroupNorm) to Dual BatchNorm as proposed by Xie (2019a): regardless of the type normalization, the robustness classifiers without dual layers drops to (almost) zero at the end of training. Right panel. We use domain-speciff specific normalization layers (Dual BatchNorm, Dual LayerNorm, Dual GroupNorm) and model with BatchNorm with shared batch statistics but domain-specific scale and offset (DualParams BatchNorm): all models achieve high clean and robust accuracy.
creates two sets of batch statistics but also two sets of learnable scale and offset parameters. In the following we investigate whether having separate batch statistics is a necessary condition for successful co-training.
Figure shows the clean and robust accuracy of various model architectures training progresses. The left panel demonstrates that, if we share both batch statisties and scales/offsets (Shared Batch- Norm, orange curves), the robust accuracy (orange dashed line) quickly drops, far from the one obtained by AdvProp (Dual BatchNorm, blue curve) which is above 34%. However, if we use a single set of batch statistics but specific scales and offsets for clean and adversarial images, we can observe on the right panel of Figure 2 that the robust accuracy (DualParams BatchNorm, orange dashed line) matches the one (blue dashed line) obtained by AdvProp. This demonstrates that is possible to achieve nominal and robust classification results similar to those of AdvProp without separate batch statistics.
Furthermore, there exist normalization layers such LayerNorm (Ba et al., 2016) or GroupNorm (Wu & He, 2018) which do not use batch statistics, as their normalization step done per sample and not per batch. Hence, according to the hypothesis of Xie et al. (2019a), these types of normalization layer should not suffer from performance degradation. Nevertheless, the left panel of Figure 2 shows that their robust accuracy (green and red dashed lines) does not match the robust accuracy of Adv (Dual BatchNorm), and is unstable over training steps. However, by making the scales and offsets of LayerNorm and GroupNorm specific to clean and adversarial images, their robust accuracy matches that obtained with dual BatchNorm layers, as shown in the right panel of Figure 2. This suggests that key element make the co- co-training loss of Eq. I work for various normalization layers is have trainable parameters which are specific to the clean and adversarial images."
3.3 REVISITING ADAPTERS WITH ADVERSARIAL TRAINING
The last observation strongly relates this setting to the adapters literature where a single backbone architecture has some parameters, called adapters, which are specific to different domains while the rest of the parameters are shared among tasks. In our case, the clean images form one domain and the adversarial images constitute another domain. In this work, we go beyond having separate normalization layers for the clean and adversarial images and investigate other types of adapters.
2 Interesttinly, contrary to our observation that standard GroupNorm fails to retain robustness, Xie & Yuille (2019) report that GroupNorm matches Dual BatchNorm. We explain this difference as we use a stronger untargeted attack in this manuscript compared the targeted attack of Xie & Yuille (2019). Using stronger attack allows reveal failure modes that would have been hidden otherwise.
4

---

Published as a conference paper at ICLR 2023
Formally, the model parameters 0 can be decomposed into parameters 2b which are shared among domains and parameters Φ which are specific to a domain. We call Oetean the parameters used when training on clean images and @ad the parameters used when training on adversarial images. For example, in Section 3.2, when we used dual LayerNorm layers, the scales and offsets of these normalization layers are contained in &etean and Padv whereas the rest of the model parameters are in zb. Based on Eq. I, we optimize the following loss:
(f &b U Petcan), y) (1 a) max L[f(2 + 2b psdv). y)-
EES
Finally, we introduce some notation for models with adapters at inference time: we call f(: Ud the clean mode for prediction as we use the adapters Oetemn trained on the clean data. Conversely, call [(; up U @an.) the robust mode when using the adapters Dady trained on the perturbed data.
3.4 TRAINING WITH ADAPTERS ENABLES ADVERSARIAL MODEL SOUPS
Wortsman et al. (2022) propose model soups, which consist in averaging the weights of multiple models fine-tuned from the same pre-trained model. The resulting weight averaged model can benefit from the original models without incurring any extra compute and memory cost inference time. Currently, in our setting the user would have to know at test time if the network should be in clean or robust mode. A model soup, by its ability to merge models, a way bypass this issue. We formulate the hypothesis that training with adapters enables model soups. With this in mind, we observe that training with adapters means that most of the model parameters are already shared, so model souping would simply consist in linearly interpolating the weights of the adapters for the two modes. We call adversarial model soups, the model soups with model co-trained on clean and adversarial samples. We get the following parametrized model:
(3) where 9 is the weighting factor when averaging the adapters. If 3 = 1, the model soup boils down to the clean mode and conversely 3 = corresponds to the robust mode. In Section 5.2, we assess this hypothesis and show that forming model soups between independent nominal and robust models fails.
f(; &b U (3@clcan + (1 - 3)
4 EXPERIMENTAL SETUP
Architecture. We focus our study on the B16 variant of the Vision Transformer (VrT-B 16) introduced by Dosovitskiy et al. (2020). We adopt the modifications proposed by He et al. (2022): the linear classifier is applied on the mean of the final tokens except the classification token. We train this network by using supervised training from scratch as proposed in He et al. (2022) (see the appendix).
Attacks. We consider adversarial robustness against untargeted foo bbounded attacks with radius <= <= 4/255. This is the most common setup for IMAGENET models, and it is more challenging to defend against than the targeted threat model used by Xie & Yuille (2019). To generate the adversarial perturbations we use Projected Gradient Descent (Madry al., 2018) with 2 steps named PGD2 (see details in the appendix) at training time and with 40 steps for evaluation (PGD40).
Datasets. We focus our experimental evaluation the IMAGENET dataset (Russakovsky et al., 2015), with images at 224 x 224 resolution for both training and testing. as this is the standard large-scale benchmark for SOTA models and was used by Xie al. (2019a) for AdvProp. We report clean and adversarial accuracy on the whole validation set. Moreover, we test the robustness against distribution shifts via several IMAGENET variants: IMAGENET-C C (Hendrycks & Dietterich, 2018), IMAGENET-A (Hendrycks et al., 2019), IMAGENET-R (Hendrycks et al., 2020), IMAGENET-SKETCII (Wang et al., 2019), and Conflict Stimuli (Geithos et al., 2018).
EXPERIMENTAL RESULTS
Similarly to our observation in Section 3.2 for RESNET-50, a fully shared VIT-B16 16 trained with the co-training loss Eq. fails to retain any robustness. Therefore, we first investigate various adapters for VIT-B B 16 to find efficient training setting in Section 5.1. Then we study adversarial model soups with adapters in Section 5.2 and finally show that training with adapters generalizes to other datasets and threat models.
5

---

Published as a conference paper at ICLR 2023
5.1 FINDING AN EFFICIENT SETTING
Choice of adapter: Using adapters increases the number of parameters as the layers which we choose as adapters have twice as many parameters: one set of parameters for clean images and another for adversarial images, Hence, to avoid increasing the network memory footprint too heavily, we restrict our adapters study to layers with few parameters, thus excluding sell-attention (Vaswani et al., 2017) layers and MIP layers. This leaves the options of having dual embedder, positional embedding. normalization layers or classification token; among them, the classification token has by far the least amount of parameters, 49-770x fewer than the other candidates (see details in Table 1). We must still verify that so few parameters are enough to preserve the advantages of the AdvProp training scheme. Hence, we train model for each type of adapter and compare them with two models without adapters, one trained with nominal training and the other with adversarial training. We observe in Table 1 that by using two classification tokens adapters, which means only 768 extra parameters out of 86M, we reach 83.56% clean accuracy on IMAGENET, which is an improvement of +0.92% over standard training. Moreover, close to the robust accuracy given by adversarial training. Notably, we see that adapting other layers with more parameters such as all LayerNorm scales and offsets results in similar performances in both clean and robust modes. This indicates that (i) it is not necessary to split the normalization layers to reproduce the effect of AdvProp, and (ii) even a very small amount of dual parameters provide sufficient expressiveness to adapt the shared portion of the network to the two modes. Therefore, in the rest of the manuscript we focus on dual classification tokens as it requires the smallest number of extra parameters.
obtain robust accuracy of 49.87% in the robust mode, which is
Number of attack steps. As the results in Table 1 were obtained with PGD?, we check if we can reduce the number of attack steps to be more computationally efficient. In Table 2, we report the results for two one-step methods: N-FGSM by de Jorge et al. (2022) and FAST-AT by Wong et al. (2020). If we use the step sizes recommended in the corresponding papers, both methods suffer from catastrophic overfitting (Wong et al., 2020) (illustrated in Figure 6 in the appendix) and therefore have no robustness at all. In Table 2 we avoid such catastrophic overfitting by reducing the step sizes to and 0. 75c for ATST-AT and N-FGSM respectively and observe that both methods perform more than 1% worse in robust accuracy than PGD?. We also increase the number of attack steps to 5 with PGD We notice small improvement over PGD? of 0.4% in robust accuracy while the clean accuracy is on par. Hence, PGD" seems to be good compromise between efficiency and classification performance.
Table 1: Influence of the adapter type. We report the clean and ro- bust accuracies in clean and robust mode on IMAGENET of networks trained with different layer types as adapters. We compare them with networks without adapters trained with nominal training and adversarial training. We recall that VIT-B16 has 86M parameters.
Table 2: Influence of the number of attack steps. We report the clean (in clean mode) and robust accuracy (in mo- bust mode) on IMAGENET training with various number of attack steps.
SETUP
ADAPTER
CLEAN MODE
ROBUST MODE
ATTACK
steps
CLEAN
ROBUST
params
CLEAN ROBUST
CLEAN ROBUST
FAST-AT N-FGSM
83.62%
48.49%
BASELINES
83.54%
47.76%
Nominal Training
82.64%
0%
PGD
83.56%
49.87%
Adversarial Training
0
76.88%
56.19%
PGD
83.60%
50.27%
Co-training (Eq. 1)
82.37%
0%
AdvProp
38k
83.39%
0.02%
80.96%
32.97%
VARIOUS ADAPTERS
Embedder
591k
83.51%
0.01%
77.68%
50.69%
Positional embedding
151k
83.65%
0.02%
77.75%
39.99%
All LN's scales/ofsses
38k
83.63%
0%
77.65%
50.02%
Classification token
0.8k
83.56%
0.01%
77.69%
49,87%
Weighting the co-training loss. In the co-training loss in Eq. 1, the a hyperparameter controls how much the loss is weighted towards clean or adversarial samples. For example, setting 0 means we train solely on adversarial samples. In Figure 3, where we evaluate several values for a (dividing the range between 0 and 1 into intervals of length 0.1), notice that only the values between and a 0.4 form a Pareto front that strictly dominates the other intervals. Indeed, between a = 1 and 0.4, decreasing leads to better performance both in clean and robust modes. In fact,
0

---

Published as a conference paper at ICLR 2023
84
84
B=
Clean accuracy
Clean accuracy
82
0=
82
80
80
Cls token soup
78
Cls token ensemble
8 = 0
a
40 Robust accuracy
20
0 10
20 Robust accuracy
30
40
50
Figure 3: Co-training loss weighting, We report the clean accuracy (in clean mode) and robust accuracy (in robust mode) on IMAGENET when training with various weightings of the co-training loss with adapters Eq. 2. We recall that a = corresponds to pure nomi- nal training and 0 adversarial training.
Figure 4: Soup ensemble. We report the clean and robust accuracy (single mode for both) IMACENET various model soups and ensembles same net- work trained with classification token adapters.
setting a 0.4 leads to 83.76% clean accuracy (in clean mode) and 52.19% robust accuracy (in robust mode) which are both better than the values obtained in Table with a = 0.5. In Figure 7 (in the appendix), we visualize the filters of the embedder when training with various values of a. We observe that for = 0.2 and for a = 0.8 the filters look quite similar to the filters learned with adversarial training (a 0) and nominal training (a = 1), respectively. Interestingly, filters learned with = 0.4 and a 0.6 are not the simple combination of nominal and adversarial filters but rather new visually distinct filters. This indicates that co-training on clean and adversarial samples can lead to new hybrid representation for the shared layers compared to nominal and ad adversarial training.
Robustness to stronger attacks. For completeness we further test the robustness of subset of our models with mixture of AUTOATTACK (Croce & Hein, 2020) and MULTITARGETED (Gowal et al., 2019), denoted by AA+MT, Pure adversarial training, which obtains 66.19% robust accuracy against state-of-the-arr
PGD40 (Table 1), reaches 54. 15% robust accuracy against AA+MT. This is robust accuracy on IMAGENET improving by +6.55% over the 47.60% reported by Debenedetti et al. (2022). While Debenedetti et al. (2022) advocate for weak data augmentation for training robust VIT, our training procedure follows He et al. (2022) and contains heavy augmentations (see appendix): we conclude that large models still benefit from strong data augmentations even with adversarial training. Finally, the robust mode of the model co-trained with 0.4 in the previous paragraph reaches 49.55% robust accuracy against AA+ which still surpasses the prior art and preserves competitive robust performance.
5.2 EVALUATING MODEL SOUPS
Adapters enable adversarial model soups. One downside of using adapters is that one needs to know if for input image the network should be put in clean robust mode. This motivates adversarial model soups which allow to create a single model performing well both in clean and robust accuracy. First, if we independently train two VrT-B16, and then try to perform model soups on them, we notice in Table 9 (in the appendix) that both robust and clean accuracies drop immediately when the weighting factor 3 between parameters is not equal to 0 or 1. We evaluate various model soups with the models of Table 1. meaning that the parameters specific to the clean and robust domain are averaged with weight / obtain a single classifier. We notice in Figure 9 (in the appendix) that adversarial model soups work equally well with the various types of adapters, where sliding the value of 3 allows to smoothly trade-off clean accuracy for robustness. This validates our hypothesis that adapters enable model soups.
nominally and the other adversarially,
Soup or ensemble. In Figure 4 we compare the classification performance adversarial model soups and ensembles obtained by linear combination of the clean and robust modes at the probability prediction level. We notice that ensembling produces a better Pareto front than adversarial model soup but ensembles, with their two forward passes, require twice the compute of model soups. Hence,
7

---

Published as a conference paper at ICLR 2023
ImageNet
82.4
IN-Real
81.7
85.0
83
85.9
86.5
87.2
87.8
88.3
88.5
88.5
88.5
IN-A
13.4
14.0
15.1
17.0
19.6
23.6
28.6
33.7
36.8
38.2
38.4
IN-R
55.2
55.3
55.2
55.4
55.5
55.6
55.6
55.4
55.1
54.7
54.4
IN-Sketch
39.6
39.8
40.0
40.3
40.5
40.7
41.0
41.2
41.
41.2
41.1
Conflict Stimuli
56.5
55.7
54.4
53.4
51.1
49.1
46.9
44.4
41.6
40.0
39.8
IN-C
0.1
0.2
0.3
0.4
0.6
0.8
0.9
B
Figure 5: Soups for IMAGENET variants. We report the accuracy on IMAGENET variants for adversarial model soups. The x-axis corresponds to the interpolation weighting factor 3 where /3 = boils down to the clean mode and / 0 the robust mode, Red (blue) means better (worse) than the clean mode (in grey).
adversarial model soups allow to choose the trade-off between clean and robust accuracy with performance close to ensembling while only requiring the same compute as a single network.
Extrapolation. For the anecdote, we experiment with adversarial model soups for extrapolation with values of the weighting factor above and below O. Interestingly, we observe that setting = 1.05 leads to 83.81 % clean accuracy which is better than the 83.76% obtained in the clean mode. Similarly, setting = 0.05 leads to 52.26% robust accuracy which is slightly better than the 52. obtained in the robust mode. Hence, appears that adversarial model soups do not need to be restricted to interpolation.
Soups for IMAGENET variants. As adversarial model soups allow create models with chosen trade-off between clean and robust accuracy, we might expect that such models perform better than nominal ones when distribution shifts occur. For example, Kireev training can even help with common corruptions when specifically tuned for such task (note that they use smaller datasets than IMAGENET). We then compute the accuracy of adversarial model soups with varying 3 on IMAGENET variants (results in Figure 5): while half of the best performance are obtained with the clean classification token, for some variants such as IMAGENET-R. IMAGENET-C and MAAGENTTSSEFTT CH the best results are obtained with intermediate tokens. Hence, adversarial model soups can be used to reach compromise hetween IMAGE variants to get the best average performance. Here 3 0.9 yields the best mean accuracy 61 61.23%. In Table 3, notice that this adversarial model soup improves the mean accuracy by +4.00% over a fine-tuned Masked Autoencoder (MAE-B16) checkpoint from He et al. (2022) and by +2.37% over Pyramid-AT from Herrman et al. (2022). It also improves by +2.24% over the best performing ensemble of two networks trained independently with nominal and adversarial training respectively.
(2021) showed that adversarial
Table 3: IMAGENET variants results. We report the adversurial model soup achieving the best average accuracy over the eight variants and compare it against several baselines. All models are trained on IMAGENET and the same VIT-B B 16 architecture.
SETUP
COMPUTE IMAGENET IN-REAL IN-V2 IN-A
IN-R
IN-SKETCH CONFLICT IN-C
MEAN
BASELINES
Nominal maining
x1
82.64%
87.33%
71.42%
28.03%
47.94%
34.43%
30.47%
64.45%
55.84%
Advensarial training
76.88%%
83.91%
12.35%
59.45%
55.44%
Finc-tuned MAE-B16
X1
83.10%
88.025
72.80%
37.92%
49.30%
35.69%
27.81%
63.23%
57.23%
Pyzamid-Ae AT (fully shared network) x1
83.14%
87.82%
72.53%
32.72%
51.78%
38.60%
37.27%
67.01%
58. 58.86%
Independent networks ensemble
82.86%
87.78%
71.73%
25.99%
54.20%
37.3355
65.615
WITH ADApTERS
Clussiication token
x1
83,69%
88. 58%50%
73.48
38.23
51.74%
41.17%
10.00%
70.07%
61.23%
Classifiationn
73.36%
62.18%
8

---

Published as a conference paper at ICLR 2023
Table 4: Comparing threat models. We optimize the co-training loss with adversarial examples coming from various attacks. For each run, we select adversarial model soup achieving the best mean performance on IMAGENET variants, which we report in the table. We set the co co-training loss weighting factor a 0.5.
ATTACK
IMAGENET
IN RE-L
IN-V2
IN-A
IN-R
IN-SKE
CONFLICT
IN-C
MEAN
Nominal training
82.64%
87.33%
71.42%
28.03%
47.94%
34.43%
30.47%
64.45%
55.84%
fou (untargeted)
83.36%
88.31
72.73%
34.81%
54.15%
40.37%
41.25%
69.54%
60.56%
l (untargeted)
83.35%
38.12%
72.82%
35.17%
52.59%
39.34%
38.75%
68,99 68.99%
59.89%
fw (targeted)
83.21%
87.97%
72.95%
33.87%
52.90%
38.70%
39.06%
68.49%
59.6 19.64%
Pyramid-C
83.62%
88.20%
73.36%
35.15%
53.94%
39.40%
41.88%
68.39%
60.49%
Table 5: Evaluating on other datasets. We report the clean accuracy (in the clean mode) when training on various datasets with several perturbation radii. We compare with nominal training on the same architecture.
RADIUS
MNIST
CIFAR-I0
CIFAR-100
SVHN
SUN-397
RESISC-45
DMLAB
Nominal
99.48%
94.63%
79.21
97.88%
65.24%
94.03%
55.66%
1/255
99.56%
96.68%
80.88%
98.38%
67.08%
95.09%
58.94%
2/255
99.67%
97.07% %
81.02%
98.14%
67.27%
95.20%
57.99%
4/255
99.68%
97.00%
80.97%
97.83%
67.32%
94.86%
57.81%
8/255
99.71 %
96.34%
81.27%
96.18%
65.97%
94.12%
53.64%
5.3 EVALUATING ON OTHERR THREAT MODELS AND DATASETS
Evaluating other threat models. IMAGENET variants are also good benchmark to compare different types of adversarial attack to generate the perturbations for the co-training loss in Eq. 2: untargeted bounded perturbations with budget e = 4/255 (our standard setup), untargeted fz- bounded with E (1. 4, 8). targeted (random target class as in Xie et al., 2019a) -bundeed with E E {4/255, 8/255, 12/255]. and the Pyramid attack proposed by Herrmann et al. (2022). In Table 4, we select the best adversarial model soups after training with each method VIT-B16 with dual classification tokens, and report its results on all variants. We see that the clean accuracy on the IMAGENET validation set improves in all cases compared to standard training. Moreover, although the best performing attack varies across variants, we notice that the untargeted &oo attack achieves the best average accuracy.
Evaluating on other datasets. We further test the effect of using the co-training loss with the classification token as adapter on other datasets. In Table 5, we see that our training procedure provides consistent performance boost in clean accuracy compared to nominal training MNIST (LeCun et al., 2010), CIFAR-10, CIFAR- 100 (Krizhevsky et al., 2014), SVHN (Netzer et al., 2011), SUN- 397 (Xiao et al., 2010), RESISC-45 (Cheng et al., 2017) and DMLAB (Beattie et al., 2016). This shows that our method generalizes well across datasets and can help regularize Vision Transformers on these smaller datasets. where they are known to perform worse compared to CNNs without pre-training (Zhang et al., 2021). In Appendix C, we also demonstrate that models pre-trained with co-training IMAGENET yield significantly better classification results when fine-tuning nominally on small datasets compared to fine-tuning from nominally and adversarially pre-trained models.
6 CONCLUSION
In this work we have shown that adapters with few hundreds of domain specific parameters are sufficient to switch between models with radically different behaviors. In fact, just replacing the classification token of a VIT can turn a classifier with SOTA nominal accuracy and no adversarial robustness into another one with robust accuracy close to that achieved with standard adversarial training. Moreover, merging the adapters allows to smoothly transition between the two modes, finding classifiers (i.e. our adversarial model soups) with better performance on distribution shifts. These observations open up new interesting directions for future work to explore how to take advantage of the regularizing effect of adversarial training and whether is possible to combine via soups other types of models.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENTS
We are grateful to Evan Shelhamer for reviewing the drafts of the paper and his literature comments, Olivia Wiles, Florian Stimberg, Taylan Cemgil and others at DeepMind for helpful conversations and feedback on the project.
REFERENCES
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv: 064500 2016. 4
Charles Beattie, Joel Z. Leibo, Denis Teplyashin, Tom Ward, Mareus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Víctor Valdés, Amir Sadik, et al. Deepmind lab, arXiv preprint arXiv:1612.0380,, 2016. 9
Nicholas Carlini and David Wagner. Adversarial examples are not casily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop Artificial Intelligence and Security, pp. 3-14. ACM, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy Liang. Unlabeled data improves adversarial robustness. In NeurlPS, 2019. 2
Rich Caruana. Multitask learning. Machine learning, 28(1):41-755 1997. 2
Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10))1865-1883) 2017. 9
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with ensemble of diverse parameter- free attacks. arXiv preprint arXiv:2003.01699, 2020. 7
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag. Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.096700 2020. 2
Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le. Randaugment: Practical automated data augmentation with reduced search space. CVPR, 2020. 15
Pau de Jorge, Adel Bibi, Riccardo Volpi, Amartya Sanyal, Philip HS Torr, Grégory Rogez, and Puneet K Dokania. Make some noise: Reliable and efficient single-step adversarial training. arXiv preprint 22Xiv:2202.01 181, 2022. 6
Edoardo Debenedetti, Vikash Sehwag, and Prateck Mittal. A light recipe to train robust vision transformers. arXiv preprint arXiv:2209.07399, 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1 11999, 2020.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference Learning Representations, 2018. URL https:://// /pwp//peeeviiveee// httppss//openneeiieee
5
Raphael Gontijo-Lopes, Yann Dauphin, and Ekin D Cubuk. No one representation to rule them all:
Overlapping features of training methods, arXiv preprint arXiv:2 /2899, 2021. 2
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. ICLR, 2015. 2,
Sven Gowal, Jonathan Uesato, Chongli Qin, Po-Sen Huang, Timothy Mann, and Pushmeet Kohli. An Alternative Surrogate Loss for PGD-based Adversarial Testing, arXiv preprint 1900.099,0 2019." 7
10

---

Published as a conference paper at ICLR 2023
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03591, 2020.
wttp://gppppp......//t// 03593. 1,2
Sven Gowal, Sylvestre- Alvise Rebuffi, Olivia Wiles, Florian Stimberg, Dan Andrei Calian, and Timothy A Mann. Improving robustness using generated data. NeurIPS, 34:4218 4233, 2021. 2
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in hour. arXiv preprint (xXiv:/70600277) 2017. 15
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, 2016. 2
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000-16009, 2022. 2, 7, 15
Dan Hendrycks and Thomas Dietterich. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. In International Conference on Learning Representations, 2018.
URL https: httt::://oeenreveee.ettt/t/eeeee
HJ net http://wwww...ee///////eeee//aid 2,
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. arXiv preprint arXiv:1907.07174 2019. 5
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer, The Many Faces of Robustness: A Critical Analysis Out-of-Distribution Generalization, arXiv preprint arXiv:2006. /6241, 2020. URL https: arxiv
/206 6241. 5
Charles Herrmann, Kyle Sargent, Lu Jiang, Ramin Zabih, Huiwen Chang, Ce Liu, Dilip Krishnan, and Deqing Sun. Pyramid adversarial training improves vit performance. In CVPR, pp. 13419-13429, 2022. 2,
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In ICML, pp. 7799.2799. PMLR, 2019. 1.2
Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q Weinberger. Deep networks with stochastic depth. In ECCV, pp. 646-661. Springer, 2016. 15
Lang Huang, Chao Zhang, and Hongyang Zhang. Self-Adaptive Training: beyond Empirical Risk Minimization. arXiv preprint arXiv:2002.10319, 2020.
Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pp. 448-456. PMLR, 2015. 1,3
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging Weights Leads to Wider Optima and Better Generalization. Uncertainty Artificial Intelligence, 2018.
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial Logit Pairing. arXiv preprint arXivv1803.06333, 2018. 1
Klim Kireev, Maksym Andriushchenko, and Nicolas Flammarion. On the effectiveness adversarial training against common corruptions. arXiv preprint arXiv:2103.02335, 2021. URL https: // //arxiv. org/ wttp////pp/p//o/eee /210////////////////// 02325. 2
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung. Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. ECCV, pp. 491-507. Springer, 2020. 15
11

---

Published as a conference paper at ICLR 2023
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The CIFAR-10 dataset. 2014. URL http: //www. cs toronto o /ed//kii///eeec cifar. html. 9
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint orXiv:161..012566 2016a.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. ICLR workshop, 2016b.
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yanm. wtwp://.www....////////cccm wwt.ee/m////miissttttmmmmmmm 2, 2010. 9
Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. Revisiting batch normalization for practical domain adaptation. arXiv preprint arXiv:I603.0477 2016.
Shao-Yuse Lo and Vishal M Patel. Defending against multiple and unforeseen adversarial videos. IEEE Transactions on Image Processing, 31:962-973, 2021. 18
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint
arXiv:1711.05101, 2017. 15
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. ICLR, 2018. 1,2, 3,5
Arun Mallya, Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting a single network to multiple tasks by learning to mask weights. ECCV, pp. 67-82, 2018. 2
Massimiliano Mancini, Elisa Ricci, Barbara Caputo, and Samuel Rota Bulo. Adding new tasks to a single network with weight transformations using binary masks. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pp. 0-0, 2018. 2
Fabio Maria Carlucci, Lorenzo Porzi, Barbara Caputo. Elisa Ricci, and Samuel Rota Bulo. Autodial: Automatic domain alignment layers. In ICCV, pp. 5067-5075, 2017. 2
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011. 9
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. NeurlPS, 32, 2019. 2
Tianyu Pang, Xiao Yang. Yinpeng Dong, Kun Xu, Hang Su, and Jun Zhu. Boosting Adversarial Training with Hypersphere Embedding. NeurIPS, 2020. 1
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. IEEE Symposium on Security and Privacy, 2016.
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with general conditioning layer. In Proceedings of the AAAI Conference Artificial Intelligence, volume 32, 2018. 2
Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapter- fusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2005.002- 2020. 2
Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. Learning multiple visual domains with
residual adapters. NeurIPS, 30, 2017. 1,2 2
Sylvestre-Alvise Rebuffi Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy A Mann. Data augmentation can improve robustness. NeurIPS, 34:2993552999999 2021.
Leslie Rice, Eric Wong, and J. Zico Kolter. Overfitting in adversarially robust deep learning. ICML, 2020. 1
12

---

Published as a conference paper at ICLR 2023
Amir Rosenfeld and John K Tsotsos. Incremental learning through deep adaptation. JEEE TPAMI, 42(3):651-663, 2018.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International jourraa of computer vision, 2015.
Asa Cooper Stickland and lain Murray. Bert and pals: Projected attention layers for efficient adaptation in multi-task learning. In ICML, pp. 5986-5995, PMLR, 2019. 2
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. ICLR, 2014.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Tumer, and Aleksander Madry. Robustness may be at odds with accuracy. arXiv preprint 1805. 2018.
Jonathan Uesato, Yean-Baptiss Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, and Pushmeet Kohli. Are labels required for improving adversarial robustness? NeurIPS, 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. NeurlPS, 30, 2017. 6
Nils Philipp Walter, David Stutz, and Bernt Schiele. On fragile features and batch normalization in adversarial training. arXiv preprint arXiv:2204. 12..:393,, 2022. 2
Haohan Wang. songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems, pp. 10506-10518, 2019. 5
Haotao Wang, Aston Zhang, Shuai Zheng, Xingjian Shi, Mu Li, and Zhangyang Wang. Removing batch normalization boosts adversarial training. In International Conference on Machine Learning, pp. 23433-23444 PMLR, 2022. 2
Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Guihong Cao, Daxin Jiang, Ming Zhou, et al. K-adapter: Infusing knowledge into pre-trained models with adapters. arXiv preprint ar arXiv:2002.01806, 2020. 2
Ross Wightman. Pytorch image models.
com
getoorc--imagemood
2019. 15
Eric Wong, Leslie Rice, and J Zico Kolter. Fast is better than free: Revisiting adversarial training.
arXiv preprint 4xX19:2001.03994, 2020. 6
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roclofs, Raphael Gontijo-Lopss, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Korbllith, al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. ICML, pp. 23965-23998. PMLR, 2022. 2, 3.5
Dongxian Wu, Shu-tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust general- ization. NeurlPS, 2020. 2
Yuxin Wu and Kaiming He. Group normalization. ECCV, pp. 3-19, 2018.
Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pp. 3485-3492. IEEF, 2010. 9
Cihang Xie and Alan Yuille. Intriguing properties of adversarial training scale. arXiv preprint 1777009906603787, 2019, 2, 4.
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan Yuille, and Quoe V Le. Adversarial Examples Improve Image Recognition. arXiv preprint arXiv:1911.0966 2019a. URL https: // www.iiii.ggvee.ee 1911 09665. 2, 4, 5,9
13

---

Published as a conference paper at ICLR 2023
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. CVPR, 2019b.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy train strong classifiers with localizable features. ICCV, 2019. 15
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan.
Theoretically Principled Trade-off between Robustness and Accuracy. ICML. 2019. 1, 2
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. ICLR, 2018. 15
Zizhao Zhang, Han Zhang, Long Zhao, Ting Chen, and Tomas Pfister. Aggregating nested transform- ers. arXiv preprint arXiv:2105.12723, 2021. 9
14

---

Published as a conference paper at ICLR 2023
A MORE EXPERIMENTAL DETAILS
Training details. In this manuscript we train VIT-BIC 16 models using the training pipeline proposed He al. (2022). The model is optimized for 300 epochs using the Adam W optimizer (Loshchilov & Hutter, 2017) with momenta 31 0.9, 8g = 0.95, with weight decay of 0.3 and cosine learning rate decay with base learning rate 1c-4 and linear ramp-up of 20 epochs. The batch size is set to 4096 and scale the learning rates using the linear scaling rule of Goyal et al. (2017). We optimize the standard cross-entropy loss and we use a label smoothing of 0.1. We apply stochastic depth (Huang al., 2016) with base value 0.1 and with dropping probability linearly increasing with depth. Regarding data augmentation, we use random crops resized to 224 224 images, mixup (Zhang et al., 2018), CutMix (Yun al., 2019) and RandAugment (Cubuk et al., 2020) with two layers, magnitude 9 and random probability of 0.5. We note that our implementation of RandAugment is based on the version found in the timm library (Wightman, 2019). We also use exponential moving average with momentum 0.9999. For RESNET-50 we keep the same training scheme used for VIT-B 16, and the standard architecture, except for combining GroupNorm with Weight Standardization in all convolutional layers following Kolesnikov et al. (2020). For the DualParams BatchNorm version we fix the robust branch to always use the accumulated statistics rather then the batch ones.
Training on smaller datasets. When training from scratch on smaller datasets in Section 5.3, we optimize the smaller VrT-S with batch size of 1024 and base learning of 2e-4. For datasets with small image resolution such as CIFAR-10 10, we do not rescale the images to 224 x 224 but we use patch size of and stride of to get enough vision tokens.
Attack details. For PGD2 and PGD use a gradient descent update with fixed step size of 2.5/255 and 1/255 respectively. For PGI)40 we change the optimizer to Adam with step-size 0.1 decayed by 10 at steps 20 and 30. Regarding one step attacks, we use step size of 6/255 and initialization radius of 8/255 for N-FGSM and step size of 5/255 for Fast-AT.
B VISUALIZING FILTERS
Visualization procedure. We visualize the embedding layer by first standardizing the weights to have zero mean and unit variance. We then extract the first 28 principal components. Finally reshape them to 16 x 16 x 3 images and rescale them to have their values between 0 and 255 such as display these components as RGB images.
(a) 2GD but no adapters
(b) Fast-AT 1.25c step size
(c) N-FGSM 1.Sc step size
Figure 6: First 28 principal components of the embedding filters of VIT-B16. In panel (a), fully shared model (no adapters) is co-trained on clean samples and adversarial samples coming PGD2 In panels (b) and (c), models with classification token adapters are adversarially co-trained with adversarial samples coming from step attacks Fast-AT and N-FGSM when using their recommended step sizes. In these three cases, we observe catastrophic robust overfitting with no robustness the end of training. We observe that there are degenerate filters among the first principal components. For panels (b) and (c), robust overfitting be avoided by reducing their step sizes.
C TRANSFER LEARNING
Training details. For completeness we evaluate the transfer learning performance of the VIT- IMAGENET by co-training clean and adversarial samples. We choose the
B16 pre-trained
15

---

Published as a conference paper at ICLR 2023
0 (pure adversarial)
(b) 0.2
(c) 0.4
(d) = 0.6
(e) a == 0.8
(I) CK = (pure nominal)
Figure First 28 principal components of the embedding filters VIT-B16 with classification token adapter trained with various weightings a of the co- co- training loss Eq. 1. We recall that a = 1 corresponds to pure nominal training and a = to adversarial training. Filters learned with a = focus more on intensity variations (black and white patterns) whereas filters learned with 0 are more color based, Filters with intermediate values of exhibit circle and star patterns which
present for 0
model trained with classification token adapter and co- traiariiing coefficient 0.4, which fine- tune nominally on CIFAR- 10, CIFAR-100, SUN-397, RESISC-45 and DMLAB using SGD with momentum 0.9, batch size of 512, gradient clipping global norm 1 and no weight decay. We optimize the standard cross-entropy loss and we use a label smoothing of 0.1. For simplicity, we the same training schedule for all the datasets: total of 10k training steps and base learning rate of 0.01 attained after linear ramp-up of 500 steps followed by a cosine decay. Regarding data pre-processe pre-processing.
simply rescale the images 224 224 resolution without preserving aspect ratio and we apply random horizontal flipping as data augmentation. Finally, we use exponential moving average with momentum 0.999.
Fine-tuning results. As the network was pre-trained with classification token adapter, we have several possibilities for initializing the classification token before fine-tuning: adversarial token, clean token and model soups interpolating between these two tokens. For comparison, we also fine-tune two VIT-B 16 pre-trained on IMAGENET with nominal and adversarial training respectively. We report the results in Table 6 where we evaluate several fine-tuning strategies: fine-tuning (i) the classifier head, (ii) the classifier head and the classification token and (iii) all the weights. First, observe that fine-tuning both the classification token and the classifier head brings only a small improvement (from 79.27% 27% to 80.70% for the best average accuracy) Fine-tuning all the weights the best strategy as it reaches 88.40% average accuracy. Second, we observe that initializing the classification token with the adversarial token performs consistently better than with the clean token when fine-tuning all the weights. Finally, co-training as pre-training is significantly better than nominal and adversarial pre-training as fine-tuning from a co-trained model reaches 88. 88.40% average accuracy, +1. +1.05% improvement over nominal and adversarial pre-training.
fine-tuning the classifier head alone.
D ACCURACY LANDSCAPE
In our case, model soups are obtained by linear interpolation (or extrapolation) of the adver- sarial and clean tokens. We notice that the clean and adversarial tokens are almost orthogonal (cos( Dstemm (Pav)) = 0.14), so we can extend our study beyond model soups by taking linear com- binations of the two tokens 31 detean B2 By taking a sweep over the 31 and 32 coefficients, we obtain in Figure the clean and robust accuracy landscapes in the plane defined by the two tokens and where the diagonal corresponds to the model soups. We observe that the main direction of change for the clean and robust accuracies is the model soups diagonal (top left to bottom right). We can clearly the trade-off in clean/robust accuracy, but also there seems to be compromise
16

---

Published as a conference paper at ICLR 2023
Table 6: Co-training pre-training. We compare the transfer learning performance of model pre-trained using co-training to models pre-trained with nominal and adversarial training. evaluate various fine-tuning strategies on several datasets (headers in green) and we report the average over datasets the last rows (orange header). We also assess several initializations for the classification token hefore fine-tuning: adversarial token, clean token and model soups between these two tokens with various weightings 3. All models are pre-trained on IMAGENET and use the same VrT-Br (TT-B16 architecture during fine-tuning.
SRTUP
BASELINES
FROM CD TRAINED NET
Nominal
Adversarial
Robust mode
0.25
0.5 9 0.75
Clean mode
CIFAR-
Fine-tune head
96.07%
90.95%
90.28%
91.17%
93.61%
96.50%
97.15
Fine-tune head cls token
96.62%
92.76%
97.73%
97.70%
97.77%
97.82%
97.84%
Fine-tune
98.68%
98.96%
99.09%
99.03%
99.01%
99.05%
99.03%
CIFAR- 100
Fine-tune head
83.30%
73.80%
71.94%
73.52%
77.78%
83.99%
85.47 %
Fine-tune head cls token
84.59%
76.79%
87.26%
87.49%
87.55
87.45%
87.43%
Fine-tune all
91.18%
91.74%
92.37%
92.23%
92.32%
92.41 %
92.29%
SUN-397
Fine-tune head
72.70%
65.62%
65.93%
67.02%
70.19%
73.00%
73.47 %
Fine-tune head cls token
73.05%
67.21%
73.99%
74.14%
74.19%
74.12%
74.15%
Fine-tune all
76.48%
75.66%
77.87%
77.75%
77.74%
77.67%
77.72%
RESISC-45
Fine-tune head
91.69
86.70%
86.54%
87.37%
89.645 64.6%%%
90.58%
91.12%
Fine-tune head cls token
91.95%
87.52%
91.04%
91.07%
91.04%
91.49%
91.23%
Fine-tune all
96.78%
96.14%
97.07% %
96.72%
96.88%
97.07% %
96.80%
DMLAB
Fine-tune head
50.02%
50.11 %
48.58%
48.60%
49.08%
49.07%
49.16%
Fine-tune head cls token
50.91%
51.53%
50.81%
51.79%
52.47%
52.64% %
$2.41%
Fine-tune all
73.65%
73.93%
75.61%
75.66%
75.74% %
75.35%
75.58%
AVERAGE
Fine-tunc head
78.76%
73.44%
72.65%
73.54%
76.06%
78.63%
79.27 %
Fine-tune head cls token
79.42%
76.16%
80.17%
80.44%
80.60%
80.70%
80.61 %
Fine-tune all
87.35%
87.29%
88.40 %
88.28%
88.34%
88.31%
88.28%
43.
13
560
415
66
42
315
(a) Clean accuracy
(b) Robust accuracy (PGD²)
(c) Arithmetic mean
Figure Linear combination tokens. We report the clean accuracy (panel (a)) and robust accuracy against PGD² (panel (b)) on IMAGENET for various linear combinations of the clean and adversarial tokens. Model soups, which are linear interpolation (and extrapolation) between these two tokens, on the diagonal from top left bottom right. Panel (c) shows the arithmetic mean between the normalized (with min/max rescaling) clean robust accuracies (red means higher mean accuracy).
region between clean and robust accuracy as the other diagonal (from bottom left to top right) is visually distinct for clean and robust accuracy. In panel (c) of Figure 8, we plot the arithmetic mean between the normalized (with min/max rescaling) clean and robust accuracies. We observe that the best compromises between clean and robust accuracy have stronger adversarial token weight than the clean token weight.
17

---

Published as a conference paper at ICLR 2023
E LIMITATIONS AND FUTURE WORK
We have empirically shown that co-training a fully shared VIT does not retain any robustness whereas having two classification tokens specific the clean and adversarial images is enough get competitive performance both in clean and robust accuracy. However, we leave to future work the theoretical explanation on why this small architecture change (adding only 768 parameters) results in such gap in performance. Similarly, beyond our intuition that parameter sharing when using adapters makes model soups possible, we cannot support our empirical results with theory and leave it to future work. Another direction for future work is the automatic selection of the right soup for each sample which could be inspired by automatic selection modules like in Lo & Patel (2021).
F ADDITIONAL TABLES AND FIGURE
In the following we present additional tables and figures of results described in the main part but omitted above because of space limits.
Table 7: Model soups for several weightings 3. We report the accuracy for various model soups on IMAGENET variants interpolating between the clean and adversarial tokens with several weightings B. 8 boils down the clean mode and
robust mode.
IMAGENET
IN- REAL
IN- V2
IN-A
IN-R
IN-SKETCH
CONFLICT
IN-C
MEAN
78.25%
84.73%
66.04%
13.39%
55.23%
39.62%
56.48%
56.67%
56.30%
0.1
78.6 78.63%
85.03%
66.49%
14.00%
55.31%
39.79%
55.70%
57.26%
56.53%
0.2
79.14%
85.40%
67.13%
15.11%
55.23%
40.00%
54.37%
58.23%
56.83%
0.3
79.74%
85.89%
67.95%
16.96%
55.40%
40.26%
53.36%
59.73%
57.41%
0.4
80.50%
86.47%
68.91%
19.60%
55.55%
40.47%
51.09%
61.91%
58.06%
0.5
81.49%
87.18%
70.21%
23.56%
55.64%
40.70%
49.14%
64.74%
59.08%
0.6
82.38%
87.76%
71.43%
28.63%
55.58%
40.98%
46.88%
67.52%
60.15%
0.7
83.05%
88.26%
72.41
33.72%
55.43%
41.19%
44.38%
69.27%
60.96%
0.8 83.54%
88.46%
72.94%
36.79%
55.07%
41.22%
41.64%
69.97%
61.20%
0.9
83.69%
88.50%
73.48%
38.23%
54.74%
41.17%
40.00%
70.07 %
61.23%
83.76%
88.52%
73.53%
38.37%
54.43%
41.15%
39.77%
69.92%
61.18%
Table 8: Ensembles for several weightings 8. We report the accuracy on IMAGENET variants for various ensembles obtained by averaging the probability predictions of the clean and robust modes of a VrT-B 16 with classification token adapter. Here is the weight used average the probability predictions. boils down the clean mode and
to robust mode.
13
IMAGENET
IN- AN-REAL
IN-V2
IN-A
IN-R
IN-SKETCH
CONFLICT
IN-C
MEAN
0
78.25%
84.73%
66.04%
13.39%
55.23%
39.62%
56.48
56.67%
56.30%
0.1
79.97%
86%
68.33%
21.23%
56.26%
40.15%
56,41% %
64. 14%
59.07%
0.2
81.26%
87.11%
70.03%
23.13%
56.56%
40.63%
55.08%
67.09%
60.36%
0.3
82.15%
87.74%
71.24%
27.72%
56.63%
40.95%
53.67%
68.57%
61.08%
0.4
82.81%
88.16%
72.09%
29.96%
56.68%
41.11%
52.50%
69.36%
61.58%
0.5
33.24%
88.39%
72.88%
32.13%
56.65%
41.28%
51.41%
69.79%
61.97%
0.6
83.47%
88.51
73.28
33.64%
56.46%
41.35%
50.23%
69.98%
62. 12%
0.7
83.62%
88.58%
73.36%
35.05%
56.32%
41.36%
49.14%
70.04% %
62.18%
0.8
83.68%
88.58%
73.37%
36.27%
56.03%
41.35%
46.64%
70.04 %
61.99%
0.9
83.73%
88.57%
73.47%
37.25%
55.62%
41.28%
44.61%
69.99%
61.81%
1
83.76%
88.52%
73.53%
38.37%
54.43%
41.15%
39.77%
69.92%
61.18%
18

---

Published as a conference paper at ICLR 2023
Table 9: Model soups independent models on network co-trained with adapters. We report the clean and robust accuracy for model soups with various weightings of either (1) two independently trained VIT-B16, one nominally
other adversarially in the first group of columns or (2) a single co-trained model with classification token adapters in the last two columns. When souping the two independent models, g = 0 boils down to the model trained adversarially and 3 = 1 to the model trained nominally. We notice for these soups between the two independent models that both robust and clean accuracies drop immediately when Regarding
soups between clean and adversarial tokens (whose numbers correspond to the blue curve Figure 4), we notice the adversarial model soups achieve smooth transition hetween the clean and rohust mades.
weighting factor / between parameters not equal to
INDEPENDENT NETS
ADAPTERS
CLEAN
ROBUST
CLEAN
ROBUST
76.88%
56.19%
78.25%
52.19%
0.1
72%
8.56%
78.63%
51.73%
0.2
1.38%
0.06%
79.14%
50.79%
0.3
0.21% %
0.00%
79.74%
48.88%
0.16%
0.00%
80.50%
44.91%
0.14%
0.01%
81.49%
33.01%
0.10%
0.01 %
82.38%
11.65%
1.72%
0.09%
83.05%
2.16%
0.8
43.25%
0.14%
83.54%
0.34%
0.9
78.56%
0.01% %
83.69%
0.06%
82.64%
0.00%
83.76%
0.02%
Clean accuracy
82
Cls token
80
Embedder
LayerNorm
78
Pos Embed
B
10
20
30
40
50
Robust accuracy
Figure 9: Comparing soups for various adapters. We report the clean and robust accuracy (single mode for both) on IMAGENET for model soups of networks trained with various types of adapters.
Table 10: Robustness to stronger attacks. We report the clean accuracy (in clean mode) and robust accuracy (in robust mode) against PGD and against the stronger AA+MT IMAGENET for various co-training loss weightings a. We report the accuracies for ranging from corresponds the Pareto front when compromising between clean robust accuracy.
0.4 recall from Figure that
CLEAN
ROBUST 40)
ROBUST (AA+MT)
0
76.88%
56.19%
54.15%
0.1
82.38%
56.09%
53.98%
0.2
83.19%
55.46%
53.24%
0.3
83.46%
53.73%
51.37%
0.4
83.76%
52.19%
49.55%
19

---

Published as a conference paper at ICLR 2023
30%
0.5/255
1/255
Robust test accuracy
20%
4/255
10%
0%
Figure 10: Catastrophic robust overfitting when co-training without adapters. We report the robust accuracy against so perturbations of size 4 4/255 over training steps when co-training with various perturbation radii VrT-B16 without adapters. We observe that only 6oo 0.5/255 does suffer from a collapse in robust accuracy during training.
20


Reviews:
[{"review_id": "gXe2xJWEQh", "reviewer": "Reviewer_rT7Z", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a new finding that is contrary to previous works. That is, it is not necessary to separate batch statistics when co-training on clean and adversarial data. It shows that using the classification token of a Vision Transformer (ViT) as an adapter is sufficient to achieve the performance of the dual normalization layers proposed by previous works. This paper also introduces “adversarial model soups” that allow a smooth transition between the clean and robust modes. These observations provide new insights into the regularization effect of adversarial training.", "summary_of_the_review": "Based on the above comments, I suggest marginally below the acceptance threshold. The new finding is interesting, and the experiments are extensive. However, the novelty of adversarial model soups is not significant. Furthermore, there are some unexpected experimental results that need further explanation. My final rating will depend on the authors’ response.", "strength_and_weaknesses": "Strengths:\n\n1. AdvProp (Xie et al. 2019a) improves image recognition by adversarial training with separate batch statistics of clean and adversarial data, first showing that adversarial examples can benefit model accuracy. This paper views AdvProp from a novel adapter perspective. It demonstrates that separating batch statistics is not necessary and that using domain-specific trainable parameters can achieve similar performance. The motivation is clear, and the new finding is interesting.\n\n2. Compared to AdvProp that only consider clean performance, this paper takes both clean and robust performance into consideration. It introduces “adversarial model soups” that can trade off clean and robust accuracy via the adapter. This makes the models more flexible and applicable to more practical scenarios.\n\n3. In addition to adversarial examples, this paper also considers the robustness against distribution shifts, where multiple ImageNet variant datasets are used for benchmarking, such as ImageNet-C and ImageNet-A. It is always good to evaluate on broad datasets.\n\n4. Figure 2 is impressive. It first demonstrates the importance of the “dual” technique, then shows that AdvProp is not the only effective dual technique. Multiple normalization methods are evaluated. The results indicate that domain-specific trainable parameters are key, not batch statistics.\n\n\nWeaknesses / Questions / Suggestions:\n\n1. If my understanding is correct, the beta is a hyper-parameter of adversarial model soups that is manually adjusted. If that is true, it is less practical and less novel. At inference time, the given inputs would be any type (clean, adversarial, distribution shifts, etc). It is impossible to adjust the clean/adversarial modes or the beta value for each input sample. An automatic mechanism is needed. The model is expected to self-decide a proper mode or beta value for each input automatically with an end-to-end pipeline. In this case, the contribution of the “adversarial model soups” would be more significant. The author may refer to [r1], which combines an automatic selection module with separate batch normalization layers to achieve the idea. The author may figure out a way to automate the adversarial model soups. The current method is just a linear combination of domain-specific parameters, so the novelty is limited.\n\n2. Several experimental results are strange. In Table 1, Co-training (Eq. 1) gets 0% robust accuracy, which is unexpected. With alpha=0.5, Co-training should still get robustness to a certain extent (see Xie & Yuille, 2019). Similarly, in Table 2, it is also unexpected that Fast-AT gets 0% robust accuracy. According to (Wong et al. 2020), Fast-AT can achieve decent robustness with proper step size and random initialization. The authors are asked to explain these results, otherwise, the results would be less convincing.\n \n3. Several figures are not clear enough. Figure 1 is suggested to denote phi_clean, phi_adv, etc. (corresponds to Equations) on the figure to make it more understandable. Figure 4 (similarly, Figure 9 in the appendix) should provide the corresponding beta values like Figure 3 provides alpha values. Figure 5 should self-contain the x-axis tile (beta).\n\n4. The tables of experimental results can be more complete. Specifically, the section “Robustness to stronger attack” should have a table showing the numbers, which would be much more clear. Or expand columns in Table 1 for stronger attacks. Furthermore, Table 9 (in the appendix) should compare the numbers of adversarial model soups as well.\n\n[r1] S.-Y. Lo and V. M. Patel, “Defending Against Multiple and Unforeseen Adversarial Videos,” in IEEE Transactions on Image Processing, 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "dX7I7JSN_j", "reviewer": "Reviewer_VYKg", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper shows that separate batch statistics for co-training on clean and adversarial inputs are not necessary. An extremely lightweight adapter using the class token is enough to achieve comparable performance compared with the dual norm setting. It also enables model soup instead of model ensembling for faster model inference.", "summary_of_the_review": "The idea is simple yet effective. However, it should be compared with the AdvProp in detail. It could only be applied on Transformers with class token, further limiting its application.", "strength_and_weaknesses": "## Strength\n\n1. The method is simple yet effective, it significantly reduces the number of domain-specific parameters compared with AdvProp.\n1. The interpolation/extrapolation experiments in Figure 3 are interesting and show the benefit of using the class token as the adapter.\n\n## Weakness\n\n1. Even in the original AdvProp, the number of domain-specific parameters is marginally compared with the number of the whole model, which makes the benefit of further reducing the number of domain-specific parameters less useful.\n1. The interpolation, extrapolation, and model soup could also be applied to the AdvProp and the results would be interesting.\n1. Using the class token as the adapter so it doesn't work with CNN and transformer without class token, which limits the application of this method.\n1. Comparison with AdvProp is not thorough. AdvProp could be regarded as using dual norm as the adapters. The influence of different adapters should be studied.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "TRG7lxJLSE3", "reviewer": "Reviewer_Wvkr", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper works on the robustness of the neural networks and aims to boost the accuracy on clean ImageNet and different variants with the help of adversarial samples. Inspired by the AdvProp and model soups, the authors propose adversarial model soups, which are trained with adapters through linear combinations of the clean and adversarial tokens. Experiments show model soups can easily strike a balance between clean accuracy and dataset-shifting robustness.\n", "summary_of_the_review": "In a word, the paper does a good job for its interesting insights and innovative method. However, it suffers from several problems, like a lack of baselines and clarity. ", "strength_and_weaknesses": "Strength\n1. Model soups are drawing much attention due to their excellent performance. However, they require a great number of computing resources. The adversarial model soups alleviate the need for storage capacity and computing power and make them practical for mobile devices and independent researchers.\n2. Findings in Figure 2 are interesting, which shows that domain-specific normalization layers are enough to boost the robustness.\n3. The method is simple, and few hyper-parameters are required.\n \nWeakness\n1. AdvProp boosts both accuracy and robustness and doesn’t need more parameters. I think you had better add the results of AdvProp in Table 1.\n2. What’s the $\\alpha$ used in the baseline **Adversarial Training** in Table 1? Could you do an ablation study here providing the results of different $\\alpha$?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "BqUpjTsC9v", "reviewer": "Reviewer_NrHB", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes to treat clean and adversarial samples as different domains and co-train them by separate classification tokens of a VIT, which produces adversarial model soup to trade off between clean and robust accuracy by simple interpolation. The authors present a good alternative for advprop with fewer parameters and better robustness.", "summary_of_the_review": "The paper is very novel and clear with convincing results. The experiments are thorough and sound.", "strength_and_weaknesses": "Strength\n+ It is very novel and inspiring to separate tokens of VIT for domain-specific training in the adversarial setting, and implementing the well-known idea (clean and adversarial samples come from different domains) in the new tool (VIT) is interesting and inspiring.\n+ The authors also insightfully demonstrate that training with adapters enables model soup, which allows a simple trade-off between robustness and accuracy in adversarial model soup by interpolation between the clean and adversarial token. It frees the burden of retraining the whole model when the balance is to change.\n+ Extensive experiments show that the proposed method could achieve SOTA clean and robust performance in different modes. Besides, the results of the distribution shift are also encouraging.\n+ The authors also thoroughly analyze the influence of adapters, attack steps, the weighting hyperparameter, extrapolation, etc. The results in Table 9 convincingly necessitate the weight-sharing design.\n+ The paper is well-motivated, well-organized, and easy to follow.\n\nWeakness\n+ The authors should better illustrate how the adversarial examples are included in clean mode training.\n+ It would be better to report the adversarial model soup performance on other datasets, e.g., CIFAR-10, CIFAR-100, along with other models in the RobustBench.\n+ The authors should use \\citep instead of \\cite in some places and maintain a larger font size for tables.\n+ Section 3.2 does not seem necessary to be so long. In contrast, the results of the adversarial model soup, Figure 9, are more important to me.\n+ Would additional data further increase the performance as in [1,2]?\n\n\n[1] Improving robustness using generated data, NeurIPS 2021.\n\n[2] Data augmentation can improve robustness, NeurIPS 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%