When is Adversarial Robustness Transferable?
Authors: Anna-Kathrin Kopetzki
Aleksandar Bojchevski
Stephan Günnemann
Venue: Submitted to ICLR 2023 (2023)
Keywords: transfer learning
adversarial robustness

Abstract: Knowledge transfer is an effective tool for learning, especially when labeled data is scarce or when training from scratch is prohibitively costly. The overwhelming majority of transfer learning literature is focused on obtaining accurate models, neglecting the issue of adversarial robustness. Yet, robustness is essential, particularly when transferring to safety-critical domains. 
We analyze and compare how different training procedures on the source domain and different fine-tuning strategies on the target domain affect robustness. More precisely, we study 10 training schemes for source models and 3 for target models, including normal, adversarial, contrastive and Lipschitz constrained variants. We quantify model robustness via randomized smoothing and adversarial attacks. Our results show that improving model robustness on the source domain increases robustness on the target domain. Target retraining has a minor influence on target model robustness. These results indicate that model robustness is preserved during target retraining and transfered from the source domain to the target domain. 

Under review conference paper ICLR 2023
WHEN IS ADVERSARIAL ROBUSTNESS TRANSFERABLE?
Anonymous authors Paper under double-blind review
ABSTRACT
Knowledge transfer is an effective tool for learning, especially when labeled data is scarce or when training from scratch is prohibitively costly. The overwhelming majority of transfer learning literature is focused on obtaining accurate models, neglecting the issue of adversarial robustness. Yet, robustness is essential, partic- ularly when transferring to safety-critical domains. We analyze and compare how different training procedures the source domain and different fine-tuning strate- gies on the target domain affect robustness. More precisely, we study 10 training schemes for source models and 3 for target models, including normal, adversarial, contrastive and Lipschitz constrained variants. We quantify model robustness via randomized smoothing and adversarial attacks. Our results show that improving model robustness on the source domain increases robustness on the target domain. Target retraining has minor influence target model robustness. These results indicate that model robustness is preserved during target retraining and transfered from the source domain to the target domain.
INTRODUCTION
Since their proposal, neural networks are constantly evolving as they are being adapted for many diverse tasks. They have a tendency to become more complex and larger, since c.g. overparama- trization has proven to be highly beneficial. Training such large and complex neural networks usu- ally requires huge amount of (labeled) high-quality data. Since this amount of data is available all domains, transfer learning was proposed. The idea is to transfer the knowledge of trained model from the so called source domain to a similar, related task in target domain for which only small amount of data exists. Usually, the transfer is considered successful if the model achieves high accuracy on the target domain. However, accuracy is not the only desired property of neural networks. Adversarial robustness is often equally important, especially in safety- critical domains. Some techniques applied in transfer learning (Shafahi et al. 2020: Chen et al. 202T) claim that they improve robustness transfer learning. However, there is no study that directly compares these techniques to standard methods for improving robustness such as adversarial training or training with (local) Lipschitz constant. We fill this gap by answering the following questions:
1. Which training procedure results in the most robust source models?
2. Is robustness preserved during target retraining?
Does robust retraining on the target domain improve robustness?
4. Which training/target retraining provides models that are robust against distribution shifts?
Does transferability correlate with model robustness?
To answer these questions, we use popular transfer learning framework consisting of two parts (see Figure M] feature extractor which extracts representations from the inputs and is trained the source domain and a classifier à which maps extracted representations to predictions and is retrained on the target domain. We investigate and compare how different training procedures and target re- training techniques affect performance and robustness of this model. More specifically, we compare 10 training procedures that can be grouped in three categories. Category one consists of training methods that aim at achieving robustness by changing inputs, i.c. (1) training on clean inputs (ce), (2) randomly perturbed inputs (ceN) and (3) adversarially perturbed inputs (ceA), (4) supervised

---

Under review conference paper ICLR 2023
contrastive learning (con) (Khosla et al. 2020). (5) supervised contrastive learning based on (5) ran- domly perturbed inputs (conN) and (6) adversarially perturbed inputs (conA). The second category of training approaches consists of methods that change the latent space of the model to achieve ro- bustness, i.e. (7) latent adversarial training (feA) (Singh et 2019). (8) adversarial representation loss minimization (feD) (Chen et al. 2 and (9) a combination of supervised contrastive learning and adversarial representation loss minimization (conF), Our third category of methods uses con- straints on the whole model improve robustness. These constraints are realized by (10) training with a local Lipschitz constant (Ilc) (Huang et al. (202T) In order to analyze how target retraining affects model robustness we compare target retraining (a) clean (Ree), (b) randomly perturbed (ReeN) and (c) adversarially perturbed imputs (ReeA).
source training
(f (s)))
target retraining (hr)
hg
aptions
cleaa
advee
noioy
options
Contasstive Loss
hr
(e))()) - [(e + 8)
evaluation
+ poninalive #+0
randunized smoothing certificate
disimaatio shift
local Lipochitz constane
-1 option
transterablity meciic
Figure I: Transfer learning framework consisting of a feature extractor f. classifier hş on the source domain and by on the target domain. For input æ. f(x) * provides the features and hz(z) hz) (f the oulput. Source training procedures grouped in methods that change inputs (ce, ceN, ceA, con, conN, conA), methods that change the latent space (feD, feA, conF) and methods that constrain the whole model (Ilc).
To provide
complete picture of robustness consider robustness certification, performance against a variety of attacks, and performance under distribution shift. Namely, we employ (i) ran- domized smoothing based certification and (ii) Fast gradient sign method (FGSM), (iii) Project Gra- dient Descent (PGD) and (iv) DeepFool (DF) attacks the source domain and the target domain. terms of distribution shift, we determine source and target accuracy under different shifts based on random noise, changes of contrast, and Gaussian Blur shift. Next, we investigate whether there is correlation between transferability and model robustness. We compute transferability metric and analyze it together with model robustness and zero-shot performance. For transfcrability quantifica- tion we use the H-score, proposed by (Bao et al. 2019) to quantify the usability of representations learned source domain for learning target task.
This battery of robustness tests tell when is adversarial robustness transferable. As will show in Section 4 target models inherit robustness from the source models while target retraining has a minor impact. Our findings suggest that model robustness is transferable when source models are trained based on a procedure that enhances model robustness without being too focused on data- specific adversarial examples.
2 BACKGROUND AND RELATED WORK
Robustness is widely studied for standard tasks such as classification and regression, but there are few works that analyze how robustness properties can he transferred from the source to the target domain. There are different aspects of robustness. One aspect is the vulnerability to adversarial examples (Szegedy et al. 2014) small input perturbations that are carefully the predictions of a model (c.g. cause misclassification) Finding attacks for given model has been widely studied for different threat models. These attacks be used to compute an upper bound on the accuracy under adversarial perturbations. However, this bound can loose since properly evaluating adversarial robustness is challenging. While a model may be robust against particular attack there is usually guarantee that it will not fail against better and stronger attack. The lesson that seemingly robust models can be broken has been learned more than once (Carlini Wagner 2017 Athalye et al. 2018, Tramer et al. 2020).
to manipulate
A complementary strategy to evaluate adversarial robustness is via verification/ceetficatttot Robust- ness certificates provide guarantees that the prediction of model will not change for the specified

---

Under review as conference paper ICLR 2023
perturbation set. Since certificates are NP-hard to compute in general, verification uses tractable (but sound) relaxations to provide a lower bound on adversarial robutness. Verification methods can be grouped in different categories, such as methods based on smoothing (Cohen et al. 2019). Lipschitz bounds (Fazlyab et al. 2019), interval bound methods (Mirman et al. 2018), and opti- mization (Wong & Kolter 2018). Randomized smoothing (Cohen et al. 2019) is widely used due to its generality- it treats the model outputs. Since most of the other techniques do not scale to the models typically used in transfer learning and/or are only applicable to specific families of models we adopt randomized smoothing in our evaluations.
block box and only requires accesses to model inputs and
Another aspect is robustness distribution shift, which arise from (natural) variations in the data such as noise, changes contrast, lighting conditions, or object composition. For example, Hendrycks & Dictterich (2019) investigate robustness using synthetic distribution shifts by intro- ducing noise (Gaussian, shot noise), blurring, simulated weather conditions. contrast change, and corruptions from compression. To investigate this aspect we also adopt a similar procedure focusing on three types of representative shifts (noise, blurring and contrast).
The literature on robustness of transfer learning is scarce, especially relative to standard supervised learning. The few existing studies are disconnected, the findings are not comparable with cach other, and even lead to contradictory conclusions. While /Salman et al. (2020) shows that robust training improves the accuracy on the unperturbed target domain data, /Shafahi et al. (2020) shows the op- posite adversarial training increases robustness but decreases accuracy. This motivation for our study - we intend to provide a fair and comparable evaluation of the most promising techniques aiming at improving robustness.
One of the strongest empirical defences is adversarial training (Goodfellow al. 2013 Madry et al.) 2018 [T] Until now, transfer learning techniques mainly use adversarial training obtain fea- ture representations that generalize better. [Salman et al. (2020) and [Urera et al. (202T) show that adversarially trained/robust models indeed transfer better than their standard-rrained counterparts, especially the target domain has limited data. However, the primary goal of these works is to improve accuracy on unperturbed (rather than adversarial) target data. Similarly, Engstrom et al.] (2019), Hyas et al. (2019). and Allen & Li (2021) show that adversarial training improves fea- ture learning and results in representations that are more aligned with humans.
Goldblum et al. (2020) and /Vaishnavi et al.] (022) investigate whether robustness can transfer from an adversarially trained teacher to a student within the same domain via knowledge distillation. The goal for the student is match the model output (Goldblum al. 2020) or match the learned representations (Vaishnavi et al.) 2022). In contrast to that, we investigate robustness transfcrability across domains. Chan et al. (2020) argues that matching imput gradients is important for robustness transfer. (amaada & Otani (2022) investigate whether robustness transfers to downstream tasks such as object detection and semantic segmentation. They find that the fixed-feature setting robustness is partially preserved and opposed to previous findings, show that an adversarial prior does not help for robustness transfer. [Ner & Sharma (2022) investigate transfers from pre-trained models and theoretically shows that downstream robustness is bounded by the robustness of the underlying representation (irrespective of the pre- pre-training protocol).
(
Finally, most closely related to our work is the study by SShaaai et al. (2020) showing that adversarial training of the feature extractor coupled with oan--ayyr-llassieer improves robustness on the target domain. With a similar goal, [hen et al. (2021) proposes an adversarial training procedure that minimizes the distance between adversarial and unperturbed representations (i.e. the output of the feature extractor f') and proposes use classifier with fixed Lipschitz constant. In all above works the robustness (when evaluated) is considered only w.r.t. a small set of attacks. To provide a more complete picture, we additionally consider verification and robustness to distribution shift.
3 MODEL, TRAINING PROCEDURES & TARGET RETRAINING
A simple but popular transfer learning framework, that we use for this work consists of two parts: a so called feature extractor and a classifier h (see Figure []. The prediction for input is obtained as = h())))) The feature extractor is trained on the source domain and then frozen, ie. not
'The details are important since adversarial training against a weak attack produces a weak defense.

---

Under review as conference paper ICLR 2023
changed during target retraining. while the classifier is retrained on the target domain. The idea of this model is, that in related tasks similar features are important and thus the feature extractor can be transferred from source domain to the target domain without adaptions, while the classifier maps extracted representatioossfecuuuees to classes and must be adapted to the target domain.
We compare the following training schemes, to determine the hest way of obtaining robust source models and preserving robustness during transfer to the target domain (details see Appendix [A..).
1. Standard supervised learning (ce). As baseline, the whole model l e f is trained on clean input data using the cross-entropy (CE) as loss function.
Randomly perturbed inputs (ceN). We train the whole model o f on randomly perturbed inputs. The noisy inputs are obtained by randomly sampling the perturbation e from a Gaussian distribution (0,8 8 I) and adding it the clean input
3. Adversarially perturbed inputs (ceA). During adversarial training, an attack is used compute an adversarial perturbation 5 for each input x. The model o f is trained on the perturbed inputs x + 8. We use 10-step project gradient descent (PGD) attack to obtain ô during training.
4. Minimizing adversarial feature loss (feD). (Chen et al. 2021) proposed method explicitly aimed improving robustness of transfer learning models. This approach based on a loss function that linearly combines the cross entropy loss ceE with a so called representation distance loss LB [ - [(] where ||-1/2 is the L2-norm. The LR loss minimizes the distance between clean and adversarially perturbed inputs in representation space. The adversarial inputs aadv are obtained with the attack described in paragraph 3 (ceA). The final loss I, = LCE + DD In is a liner combination of the two, where E representation space.
[0; 1] is a
and DR is the dimensionality of the
5. Latent perturbations (feA). The previous training procedures focused on adversarial examples the input space and/or the representation space. However, non-robustness can be caused by any layer of the neural network that maps close points far from each other. To address this issue latent adversarial training
proposed (Singh et al. (019). The authors choose layer I and split the neural network n at that layer into two parts ni and na such that n = ng fi.. Then they use the fast gradient sign method (FGSM) on the sub-network n2, which results in and adversarial example in the input space of reg i.c. the latent space of the whole neural network n. For training, the authors additional compute an adversarial example in the input space of and combine the gradients of adversarial input and latent adversarial example to update the neural network parameters. (Singh et al. (219)) proposes to use this training procedure for fine-tuning after the training. Since standard adversarial training not used after (but during) training. we modify their approach to ensure a fair comparison. First, we use latent adversarial training from the beginning of the training. Second, for each training step randomly chose latent layer I for splitting the network and computing latent adversarial examples. Finally, we extended the method such that latent adversarial examples can be computed by any attack strategy. To ensure comparability we compute latent adversarial examples and adversarial inputs by using 10-step PGD attacks.
6. Local Lipschitz constant (Iic). Bounding the Lipschitz constant of neural network is known improve model robustness and can even be used to get guarantees. Since bounds on global Lipschitz constants are often loose and might lead to over-regularization we train neural network based on an upper bound local Lipschitz constant is obtained by taking interactions between weight matrices and activation functions into account. It can be proven that the obtained bound is tighter than the global Lipschitz constant. The details of this training approach are explained in (Huang et al. (02T).
trainable local Lipschitz constant as proposed by (Huang et al.) 2021). The
Supervised contrastive learning (con). Neural networks trained with fully-supervieed con- trastive learning (Khosla et al. 2020) consist of the same two parts as our transfer learning model: a feature extractor f that computes representation f(x:) for each input and classifier h that maps the representation to the output space hx)())) The idea of contrastive learning compute repre- sentations for a batch of samples and train the feature extractor by pulling representations that corre- spond to the same class (positive samples) together and pushing representations of different classes (negative samples) apart from each other. To ensure positive samples, on each input in a batch B we perform two different realizations of random data augmentations aug(.). such as random crops, random grey-scale changes, etc., which results in two new training batches: B1 (aug(a) Vo E B

---

Under review conference paper ICLR 2023
= Vx E BJ. We train on B₁ Ug. We include this training procedure since is directly based on a desirable property of the representations: Inputs of the same class should result in close representations. Enforcing this might affect source or target model robustness.
In each training epoch, we alternatingly update the parameters the feature encoder f and the clas- sifier h. First, the feature encoder is updated based on the representation computed for the inputs using the contrastive loss (that minimizes the distance between positive samples and maximizes the distance between negative samples). Second, the representations are propagated through the classifier and the classifier is updated using cross-entropy loss.
8. Supervised contrastive learning on randomly perturbed inputs (conN), This training proce- dure is exactly like supervised contrastive learning, except for the data augmentation of each batch. We use a clean version of the batch and a version that contains randomly perturbation samples: Bi = (aug(r) Va E B), and B2 = [x + Va E Bi]. The perturbation e is obtained by sampling from Gaussian distribution (0,0 I).
Supervised contrastive learning on adversarially perturbed inputs (conA). This training procedure is again exactly like supervised contrastive learning, except for the data augmentation. We use clean version of the batch and an adversarially perturbed version: B₁ = (aug(ar) V' E B] B2 (ar 5 Vo E B.]. The adversarial perturbation ō obtained by computing 10-step PGD attack on each imput x E Bi.
10. Fine-tuned contrastive learning (conF). We propose to combine supervised contrastive learn- ing described in paragraph 7 with fine-tuning based on minimizing the adversarial feature loss. Since the standard contrastive learning operates the representation space, but does not see any ad- versarial examples during training, we propose to add fine-tuning on the source dataset to increase robustness. After training, we retrain the whole model (on the source dataset) by minimizing the feature loss (see paragraph 4).
Target retraining. Since the feature extractor f is fixed during retraining on the target domain, techniques such as constrastive learning, minimizing adversarial feature loss or latent adversarial trining applicable to adapt the classifier to the target domain. We use and compare three different retraining procedures for the classifier: standard supervised leaning (R see paragraph I), training on randomly perturbed inputs (ReaN, see paragraph 2) and training on adversarially per- turbed inputs (Reas, paragraph 3).
4
WHEN IS ADVERSARIAL ROBUSTNESS TRANSFERABLE?
4.1 WHICH TRAINING PROCEDURE RESULTS THE MOST ROBUST SOURCE MODELS?
Our goal is to amalyze if and how robustness can be preserved during transfer from the source domain the target domain. Thus, we first need to obtain robust source models. To achieve this train models based on the procedures discussed in Section 3 All models are trained on three different datasets, i.c. SVHN, EMNIST and CIFARIO. Details on the experimental set up can be found in the appendix A.
We evaluate model robustness in two complementary ways by using attacks and verifica- tion/certification. More specifically we use attacks of different strength, i.c. fast gradient sign method (FGSM) as weak attack, and projected gradient descent (PGD) and DeepFool (DF) strong attacks. For certification we use randomized smoothing, since this verification technique treats the model as black box and thus is applicable on all model architectures and activation func- tions. Figure 2 illustrates the results of our robustness evaluation. The exact numbers and the verifiable radius can be found in Table [ [ablee[Appennn (Appendix A..2)
First, we observe that all training procedures, except ceN and IIc, result in comparable and high base accuracy (Abase rose/gray). Training on randomly perturbed inputs (ceN) can reduce the base accuracy by 0 - 5 % and training with local Lipschitz constant (Ile) by 20 % depending on the dataset.
The verifiable accuracy (Avera.), i.c the portion of points for which randomized smoothing could certify the prediction, varies between the different training procedures. Not surprisingly. models trained on randomly perturhed inputs (ceN and conN) have the highest verifiable accuracy. Using

---

Under review conference paper ICLR 2023
Veriication: SUIN MNST
Nerfforiam EMRUST KMASST
Verticalore CIARLO PRINST
(20 EHN
Takning Mathos
10 Nothos
Trshing Nithod
Attarking: DANK MAIST
Artacingg TNNIST. KYBUSTT
Attacking: CRARDD PIIMST
Acoint 080 SWA
Acarso DS0 en EWASST
Acarbay OFFE00
944
9127100
50 94
Mectod
1oriing MAcTFd
Figure 2: Accuracy on the clean test set (Abase. rose or gray), verifiable accuracy (Aeert, red) and accuracy decrease under FGSM (green). DeepFool (DF, blue) and PGD attacks (purple) of source models trained SVHN, EMNIST and CIFARIO.
robust training that includes attacks (ceA, feD, feA, conA, conF) local Lipschitz constant (Ile) result results in models with medium to high verifiable accuracy and using training without pertur- bations (ce, con) results in the least robust models according to randomized smoothing. Our attack analysis (Figure ( shows that using robust training (ceN, ceA, feD, feA, Ilc, conN, conA, conF) results in models that are significantly more robust than normally trained models (ce, con).
Thus, to obtain high accuracy and high robustness on the source dataset, we recommended to use a robust training procedure. The best performing training procedure w.r.t. to base accuracy. verifiable accuracy and robustness against adversarial attacks conF, i.e. supervised contrastive learning followed by a fine tuning step that uses adversarial attacks and a loss function that minimizes the distance between clean and ad representations in the feature space (see Table T
4.2 IS ROBUSTNESS PRESERVED DURING TARGET RETRAINING?
the previous section we analyzed how to get (the most) robust source models. Now we analyze robustness is preserved during transfer from the source domain to the target domain during tar- get retraining. Since focus inherited robustness properties in this chapter, do the target retraining clean inputs (Re.). Figure 3 shows the target robustness versus the source robustness of the models. Robustness is quantified by determining the verifiable accuracy using randomized smoothing (first row) and the accuracy under the most successful (FGSM, PGD, DeepFool) attack, i.e. the attack which results the largest accuracy decrease (second row).
source robustness would hundred percent preserved during transfer, the measurements would fall on the line = x. However, we observe a more complex and dataset dependent correlation between target robustness and source robustness. First, models with low source accuracy result in models with low target accuracy such as ce. Models with high source robustness have different capabilities of preserving this robustness during target retraining. The amount of robustness that can be preserved depends the source training procedure and the transfer learning task (i.e. source and target domain).
On SVHN MNIST we observe clear target robustness ranking of training procedures that result similarly robust source models. Using local Lipschitz constant (Ilc), contrastive learning on clean (con), randomly (conN) or adversarial perturbed inputs (conA) results in robust target models which are even more robust than the source models (measurements above = x line). On EMNIST KMNIST the highest target robustness is observed for llc and models trained with a loss function that minimizes representation distances corresponding models are less robust than source models. If models are transferred from CIFARIO to FMNIST, the highest target robustness transfer is achieved by Ilc. The other approaches are able to preserve (most of) the robustness during transfer.
clean and adversarial inputs (feD), but target

---

Under review conference paper ICLR 2023
100
Veriicalion: Sahh MAST
100
Venlication: EMMSTT RMRRIST
20
Verf CINARLD PMMMST
Doger Sub wetuettttt 196)
80
Sirec: tab wow.ccoocc (R0)
10
800
10
NA
Source Reoustres [S)
Source Repustress (1)
Sourcw Robusiten (76)
100
Aracting: S/IN - MM6T
100
Altaccng: EMAAST KINIST
Attecting: CIAR10 PHINET
Do Ratucnssss
80
LE4 HD
cent
cant
1ok
50
5
103
Source (st
Source Ropustress (4)
Sourso Nobusiness (S))
Figure 3: Target vs. source robustness quantified by verification or the most successful (FGSM, PGD, DeepFool) attack. Labels refer to the source training, target retraining is done on clean inputs.
Thus, even though they are not the most robust methods on the source domain, the most robust meth- ods on the target domain are the contrastive learning techniques and Ilc which uses local I Lipschitz constant. These training procedure achieve robustness without being too focused on adversarial ex- amples. While Ile aims at mapping close input to close outputs, the contrastive approaches minimize representation distances of inputs corresponding robustness by more general concepts that computing worst case perturbationns/adeersam examples which might be too data-specific to ensure that robustness is passed onto the target models.
the same class. Thus, both approaches achieve
4.3 DOES ROBUST RETRAINING ON THE TARGET DOMAIN IMPROVE ROBUSTNESS?
In the previous section we show that the training procedure significantly affect model robustness on the target domain. In this section we analyze the influence of target retraining on target robustness. To this end we compare different target retraining procedures, i.c. target retraining on clean inputs (Roe), on randomly perturbed inputs (ReeN) and on adversarial perturbed inputs (ReaA). Figure 4 compares accuracy, verifiable accuracy and accuracy decrease under attacks of these three target retraining schemes. Detailed numbers corresponding these plots (and further plots) can be found in Table 2 and Figure & in the Appendix(A.2
Rerficnee SVNIN 198ST
Wilfeconn EVREST OHNSS
We RerEoo OHARIO FIRSST
Acuurttyy on MusT
000/20/ SRIST
300
E
0
Taning Mectod
Taning
Strangen: wileeds 2V1N MEAST
Suunges alacct EMNST CHNIST
Sterges: Binck DIAB10 IIHNST
Accin on MRAST
16
New
PMAST
0 8
Motnod
Toniigg
Merrod
Tann mg Methos
Figure 4: Accuracy the clean test (Abase. bright colors), verifiable accuracy (Acert., dark colors) and accuracy decrease of the strongest (FGSM, PGD, DeepFool) attack of target retraining on clean (Re.), randomly perturbed (Reen) and adversarially perturbed (ReeA) inputs.
Comparing the three target retraining schemes, training on randomly perturbed inputs inputs (Reon) results in higher verifiable accuracy (first row, dark brown bars) but decreases the accuracy the

---

Under review conference paper ICLR 2023
clean dataset. Our attack analysis shows small differences in model robustness for the three target retraining schemes. Both analyses clearly illustrate that the training procedure on the source domain has major influence on target model robustness, while the target retraining has a minor effect on it. Thus, in order to obtain robust target models we require robust training of the source model.
4.4 WHICH TRAINING/TARGET RETRAINING PROVIDES MODELS THAT ARE ROBUST AGAINST DISTRIBUTION SHIFTS?
In the previous sections, we quantify and analyze adversarial robustness, i.e. robustness against small input perturbations that aim at fooling the model to make a wrong prediction. Another type of perturbations that occur in real-life transfer learning are distributions shifts. Distribution shifts are changes of the dataset such as random noise, changes in contrast or Gaussian blur. We analyze robustness of our source models (see Figure 5 and Table and target models (see Figure 5 Table 6
andd (
against these data shifts.
Diembation the on NNST
Didt butriallonaii shite INNSTT KSBIST
Datristion shite CEARLD TNNIST
Acouney on Ste
MOLOCC On ENNUS
5
Tatingg
Taaing
baning Mected
Figure 5: Accuracy the clean test set (Abase. gray) and accuracy under distribution shifts based on random noise, changes of the contrast and Gaussian blur of source models.
DSSt bution SNT SWNN M8851
Dicribution "OUNERAA 5VHN W98ST
Dlat bution "Slurt SIMIN HRAST Rux Paarhee
Accumity 51
5
Datebet
ENNIST KHNUST
Dhinturin EHAUST KMNNST
Accun On ONNST
10
Act/o 1it ONIST
160
ONVST
6
Toniing
Toining
DEA seii "Mose" CHAR1D
198:5T
DiSt Butiontiit "Controg"" CHAR20 19455
DS2 butipn StEt" CHAR20 1908ST
Accunty RINST
ACC ACC/ 0S3 RINIST
1aning
Toning
Mectod
Boning
Figure 6: Accuracy on the clean test set (Abase. bright colors) and accuracy under distribution shifts based on random noise, changes of the contrast and Gaussian blur of target models retrained on clean (Roe), randomly (ReaN) or adversarial perturbed (Reas) inputs.
Robustness of source models against distribution shifts depends on the dataset and the shift. The normally trained model (ce) is the least robust model. Models trained on randomly perturbed inputs (ceN, conN) are robust to noise shifts, while contrastive learning (con. conN, conA, conF) result the most robust models w.r.t. changes of the contrast (see Table 5). Robustness against Gaussian blur shifts be increased by robust training contrastive learning.

---

Under review as conference paper ICLR 2023
Target model robustness against distribution shifts mainly depends on the training procedure, while target retraining has a minor effect. Adversarial training and contrastive learning improve robustness against distribution shifts. On the SVHN - MNIST task training with a local Lipschitz constant (Ilc) and contrastive learning techniques without fine-tuning (con, conN, conA) result in the most robust target models against distribution shifts caused by noise, contrast changes or Gaussian blurring. On EMNIST KMNIST adversarial training with a loss that minimizes distance between representa- tions corresponding to clean and adversarial perturbed inputs (feD) yields the most robust models against all three analyzed distribution shifts. On CIFARIO FMNIST contrastive learning on ran- domly perturbed inputs (conN) and training with local Lipschitz constant result in the strongest performing models against noise, contrast change and Gaussian blur data distribution shifts. Thus, models that are robust against adversarial attacks are also more robust against distribution shifts compared to non-robust models.
4.5 DOBS HAANSERRABILIT CORRELATE WITH MODEL ROBUSTNESS?
One key requirement of transfer learning models is transferabilityy, i.e. the potential of a model to benefit the target task. We analyze if there is correlation between robustness and transferability of our source models (see Figure 7 Figure [3 Table 9 and 10 in the Appendix). In order to quantify transfcrability we use the H-score as proposed by |aoo et al. (2019) and determine the zero-shot per- formance of the source models. The zero-shot performance is the accuracy a source model achieves on the target dataset before any target retraining.
Vediicadion SWH.
Arackingg
shot porformance: SVh. NMSST
8.5
8,0
7.5
7.5
corl
06A
corl
2.00
(5))
CC
1c
5.0
6.00
cch
ce)
con
conN
3.5
3.5
TxD
5.0
co07
20
60
102
40
6D
100
02
Rosuerrex (S) (Veriicatoon)
Robuxthee [1S] (Mirecinng)
Source
SUN
Figure 7: Transferability (H-Score) versus robustness of the source models quantified as verifiable accuracy or accuracy under the strongest attack and zero-shot performance on SVHN - MNIST.
The absolute value of the H-score depends on the dataset. Considering the ranking, the contrastive learning approaches (con, ConN, conA, conF) have the highest H-scores all three transfer learning tasks. reason might be that these methods train the feature extractor by pulling together positive anchors (representations corresponding to the same class) and pushing apart negative anchors (rep- resentations corresponding different classes) in the feature space. Since con performs similar to the more robust contrastive approaches we could not find a correlation between rohustness and transferability. On the SVHN- - MNIST task source and target dataset share all ten classes and the transferability estimations are are consistent with the zero-shot performance. The contrastive ap- proaches have a high zero-shot accuracy of up to 66 %. On EMNIST KMNIST and CIFARIO FMNIST the source dataset and the target dataset contain different classes so zero-shot accuracy is similarly low for all methods as expected (see Figure 13 in the Appendix). Thus, if the source dataset and target dataset contain the same classes the supervised contrastive learning schemes achieve the highest transferability, zero-shot accuracy and can preserve robustness during transfer.
5 CONCLUSION
This work analyzes how different training procedures on the source domain and fine-tuning strate- gies on the target domain affect model robustness. We show that the training procedure on the source domain has major effect on target model robustness while target retraining has minor effect. Our results indicate that contrastive learning and training with a local Lipschitz constant best preserve robustness during target retraining. Furthermore. robustness to adversarial attacks also provides robustness against distribution shifts. Transfcrability and zero-shot performance depend on the re- latedness between the source and the target domain and on the source training process. The highest transferability and zero-shot performance is achieved by contrastive learning approaches, which are also among the strongest ones in preserving robustness during transfer.

---

Under review conference paper ICLR 2023
6 REPRODUCIBILITY STATEMENT
Detailed information about the datasets, the transfer learning tasks, models, attacks and randomized smoothing is provided in the subsections of section A.
REFERENCES
Zeyuan Allen-Zhteeeee and Yuanzhi Li. Feature purification: How adversarial training performs robust deep learning, 2021.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICMI. 2018, Stockholmsmüssan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 273-283. PMLR, 2018, press/v80/ at.ha yee h
mlr.
Yajie Bao, Yang Li, Shao-Lun Huang, Lin Zhang. Lizhong Zheng, Amir Zamir, and Leonidas Guibas. An cnermattionthhoorete approach to transferability in task transfer learning, 2019 IEEE International Conference Image Processing (ICIP), 2019.
Nicholas Carlini and David A. Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Bhavani Thuraisingham, Battista Biggio, David Mandell Freeman, Brad Miller, and Arunesh Sinha (eds.), Proceedings of the 10th ACM Workshop on Artificial Intelli- gence and Security, A/Sec CCS 2017, Dallas, TX, USA, November 3, 2017 pp. 3-14, ACM, 2017. doi: 10.1145//128572:3144444 URL https: org 10. 14411111333222 3140444
Alvin Chan, Yi Tay, and Y. Ong. What it thinks is important is important: Robustness transfers through input gradients. 2020 JEEE/CVF Conference on Computer Vision and Pattern Recogni- fion (CVPR), pp. 329-338, 2020.
Dian Chen, Hongxin Hu, Qian Wang, Yinli Li, Cong Wang, Chao Shen, and Qi Li, Cartl: Coopera- tive adversarially- transfer learning, 2021.
Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. Neural Information Processing Systems, Machine Learning for Creativity and Design Workshop, 2018.
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik. Emnist: Extending mnist to handwritten letters. 2017 International Joint Conference on Neural Networks (LJCNN), 2017. doi: 10.1 109/jjem.. www.mmm.......... 2017.7966211
Jeremy M Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via random- ized smoothing, 2019.
Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Brandon Tran, and Aleksander Madry. Adversarial robustness as prior for learned representations, 2019.
Mahyar Fazlyab, Alexander Robey, Hamed Hassani, Manfred Morari, and George J. Pappas. Ef- ficient and accurate estimation of lipschitz constants for deep neural networks. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' AAlché-Buc, Emily B. Fox, and Ro- man Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancou- ver, BC, Canada, 1 11423 11434. 2019. URL ht htt.:ppp:s//pooceeccngg.... newr ips.ccipp.coooo cc/ html
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation. In The Thirty-Fourtt AAAI Conference on Artificial Intelligence, AAAI 2020, The Phirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Sym- posium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pp. 3996 4003. AAAI Press, 2020. URL https: /ojs. aaai org/ /t///AAA//aaaa / http:///ttaaaaleeeee//////////
index. wwwpex.ppp/AAA/aaaaaea
10

---

Under review conference paper ICLR 2023
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceed- ings, 2015. URL http:/ /artp: wtwpllivwrg////o 1tt./1412.666 6572]
Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL https: 7/openzeview. net wwtp:/et///oueeee forum? 1d=HJ26t1Cqyxm
Yujia Huang, Huan Zhang. Yuanyuan Shi, J. Kolter, and Anima Anandkumar. Training certifiably robust neural networks with efficient local lipschitz bounds. arXiv, 11 2021.
Andrew Hyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features, 2019.
Prannay Khosla, Piotr Teterwak, Chen Wang. Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in Neural Information Processing Systems, 33:18661-18673 2020.
Alex Krizhevsky. Vinod Nair, and Geoffrey Hinton. Cifar-10. Canadian Institute for Advanced Research, 2009.
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. National Institute of Standards and Technology. 2010.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver. BC, Canada, April 30 May 3, 2018 Conference Track Proceedings, OpenReview.net, 2018. URL Dattps://spenrevien net/
forum? faduuniddrJJBfB....
Matthew Mirman, Timon Gehr, and Martin T. Vechev. Differentiable abstract interpretation for provably robust neural networks. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholnsmässoon Stockholm, Sweden, July 15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 3575- 3583. PMLR, 2018. htR.:ptp:///poooooo
mml...mmrr.......m...........mm www.rrpreasssee /vrres//80/// mi rmanl 1 8b. htm
Laura Fee Nem and Yash Sharma. How adversarial robustness transfers from pre-training to down- stream tasks. ArXiv, abs/2208.03835. 2022.
Yuval Netzer, Tao Wang. Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Read- ing digits in natural images with unsupervised feature learning. Neural Information Processing Systems, Deep Learning and Unsupervised Feature Learning Workshop, 201
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin. Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Ed- ward Yang, Zach Vitoto Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 2019.
Jonas Rauber, Roland Zimmermann, Matthias Bethge, and Wieland Brendel. Foolbox native: Fast adversarial attacks to benchmark the robustness of machine learning models in pytorch, tensor- flow, and jax. Journal of Open Source Sofiware, 5(53):2607, 2020.
Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adver- sarially robust imagenet models transfer better?, 2020.
Ali Shafahi, Parsa Saadatpanah, Chen Zhu, Amin Ghiasi, Christoph Studer, David Jacobs, and Tom Goldstein. Adversarially robust transfer learning, 2020.
Mayank Singh, Abhishek Sinha, Nupur Kumari, Harshitha Machiraju, Balaji Krishnamurthy, and Vineeth N Balasubramanian. Harnessing the vulnerability of latent layers in adversarially trained models. arxiv, 2019.

---

Under review conference paper ICLR 2023
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan J. Good- fellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http:
7/azxiv.org httgg/aba/11111111331 6199
Florian Tramèr, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. adaptive attacks adversarial example defenses. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria- Florina Balcan, and Hsuan-Tiae Lin (eds.), Advances in Neural Information Processing Systems Annual Conference on Neural Information Processing Systems 2020, NewrIPS 2020, De-
cember 6-12, 2020, virtual, 2020. URL https::/ppoceeeennnngeeg//////ee newriss.eerppppcoooo wtt.scccppppppe ccpppppperccr paper/ Abstract..tttttte......aaa html
2020 /ttp://1336c11111111444433333333
Francisco Utrera, Evan Kravitz, N. Benjamin Erichson, Rajiv Khanna, and Michael W. Mahoney. Adversarially-trainee deep nets transfer better: Illustration image classification, 2021.
Pratik Vaishnavi, Kevin Eykholt, and Amir Rahmati. Transferring adversarial robustness through robust representation matching. arXiv, 2022.
Eric Wong and J. Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th Inter- national Conference on Machine Learning, ICML 2018, Stockholmsmüssann Stockholm, Sweden, July 10-15, 2018 8, volume 80 of Proceedings of Machine Learning Research, pp. 5283-5292. PMLR, 2018. URL httt::/ppocceeinnng//e/
mlr. wwwpspessssveeee Sateess/vv0/woggg1 /wogg/onggllg ha.html
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmark- ing machine learning algorithms. Zalando SE, 2017.
Yutaro Yamada and Mayu Otani. Does robustness imagenet transfer to downstream tasks? ArXiv, abs/2204.0393 03934,, 2022.
12

---

Under review as conference paper ICLR 2023
A APPENDIX
A.I DETAILS OF THE EXPERIMENTAL SETUP
Models. All models have similar base architecture, proposed by (Huang et al. 202T). The encoder consists of 6 convolutional layers (kernel sizes: 3, 3, 4, 3, 3, 4), each one followed by an activation layer with ReLU or ReLUUx (Ilc) as activation function, a flatting layer to reshape (which is required for contrastive learning). followed by linear layer (size: 512) and another activation layer. We include the flatting layer for all models since it is essential for contrastive learning, but did not had an effect on normally trained classifiers and we wanted to keep the architecture of the approaches as close as possible. The classifier consists of one linear layer. Implementation is done in Pytorch (Paszke et al. 2019). For contrastive learning (con, conN, conA and conF) we use stochastic gradient descent as optimizer as proposed by (Khosla et al. 2020), all other models are optimized using Adam optimizer. As mixing coefficient for CE-llos and feature-distance loss (feD training) we chose x = 0.1 as proposed by (Chen et al.] 2021). Hyper-parameterss are determined by performing a grid-search. For training the contrastive models (con, conN, conA, Fcon) the grid search is done in [0.005; 0.1] and for the other models in [0.0001; 0.001]. All models are trained for 800 epochs and retrained for 200 epochs. The target retraining learning rate is search in [0.0001; 0.5] and we do are warm-start, i.e. we do not randomly internalize the weights of the classifier before target retraining. Model fine-tuning (conF) is done for 100 epochs and with a learning rate [0.0001; 0.001].
Datasets. We use six different datasets: The SVHN dataset (Netzer et al. 2011) contains images of street few housing numbers. The MNIST dataset (LeCun & Cortes) 20T0) consists of gray- scale images of handwritten digits. The CIFARIO dataset (Krizhevsky et al. 2009) contains 3 32 x 32 images of objects (airplane. bird, car, cat, deer, dog, horse, ship, truck and frog). The TashionMNNST/FMNNIST dataset (Xiao et al. (017) contains gray-scale images of clothes. The EMNIST dataset (Cohen et al.] (017)) consists the 26 character of the alphabet and the KMNIST (Clanuwat et al., 2018) dataset consists of gray-scale images of Japanese characters. Each dataset already consists of a predefined training set and a test set. We further split the training sets into training data (90%) and validation data (10%). The validation set is used during training/retrainingg to check the accuracy and determine the best model. The following data augmentations are used on all datasets and model: random horizontal filps, random corps, and random rotations ($15"). For contrastive models (con, conN, conA and conF) additional augmentations, i.c. random resize crop, color jitter and random gray-scale) is used.
Transfer Learning Tasks Based on the six datasets discussed above we create three transfer learn- ing tasks of different relatedness. We consider the following transfer tasks (source domain + target domain): SVHN MNIST (highly related), CIFARIO FMNIST (related), EMNIST KM- NIST (related).
Perturbations and Attacks. We use four different attack types: Noise attacks, Fast Gradient Sign Method (FGSM), Project Gradinent Descent (PGD) attacks and DeepFool attacks with attack radii of 0.1. We chose an attack radius of 0.1 since that is a popular perturbation size analyzed in randomized smoothing as well as for attacks. The perturbation is bounded by the L2-norm and applied to the input after data normalization. For adversarial training we use 10-step PGD attacks, while robustness analysis uses 1-step FGSM attacks, 100-step PGD attacks and 100-step DeepFool attacks based on the implementation provided by Rauber et al. (2020).
Randomized Smoothing. Randomized smoothing techniques (Cohen et al., 2019) draw samples x; ~ NNx,0) from the close neighborhood of input æ. propagate them through the neural network and aggregate the outputs to obtain smooth prediction. We use = 0.1, draw 500 samples for each input and bind the probability of returning an incorrect answer/prediction by a 10 44 If a prediction cannot be certified randomized smoothing abstains, i.c. returns -1 instead of label. Please note that the certified prediction mainly depends depends on the neighhorhood of the input sample x (i.e. the 500 drawn samples) and might he different than the prediction of the base model, which only depends on input
Distribution shifts. We generate different distribution shifts on each dataset based on random per- turbations (noise). changes of the contrast (contrast) and Gaussian blur (blur). More specifically, we use Gaussian noise shift (Noise), uniform noise shift (UNoise), contrast reduction shift (Contrast), contrast reduction shift based on binary search (ContrastBiin), contrast reduction based linear
13

---

Under review conference paper ICLR 2023
Maarhh(CntraatLinn) Gaussin blur (Blur) and salt and pepper (SaltPepper) shift. The perturbation or shift size bounded by the L2-norm and 5 as upper bound. To compute these perturbations we use the implementation provided by RRuber et al. (2020).
Transferability. In order to estimate transferability of source models we compute the H-score proposed by (Bao et al. 2019).

---

Under review conference paper ICLR 2023
A.2 ADDITIONAL EXPERIMENTAL RESULTS
Which training procedure results in the most robust source models?
Table shows how robust target model retrained on normal (ce), randomly perturbed (ceN) and ad- versarial perturbed (ceA) target inputs are. In order to quantify robustmess compute the verifiable accuracy using randomized smoothing and accuracy under FGSM, PGD and DeepFool attacks.
Table I: Accuracy on the clean test set (base), verifiable accuracy (cert.) and radius, accuracy under FGSM, PGD and DeepFool attacks of source models trained on SVHN, EMNIST and CIFARIO. The best values/highest accuracy are highlighted using bold letters.
SVHN MNIST
Train
Retrain
base [%]
Acert.
Roart.
AFGSM
APGD [%]
Deep [%]
94.03
78.41
0.1 1.
78.53
71.25
75.11
ceN
93.40
91.65
0.171
86.90
86.26
86.47
94.20
87.55
0.147
86.50
85.05
85.76
feD
93.69
88 $8.29
0.157
86.70
86.17
86.38
fcA
94,33
87.91
0.148
87.33
86.13
86.66
llc
77.60
76.00
0.165
72.66
72.71
71.50
93.32
69.91
0.096
79.35
66.73
75.06
conN
91.70
88
0.167
83.
83.00
83.30
conA
92.41
85.27
0.145
83.75
82.16
82,96
conF
95.28
86.55
0.147
88.54
87.20
88.06
EMNIST + KMNIST
Train
Retrain
Abase [%]
Abort. [%)]
Roar.
AFGSM [%]
APGD [%]]
Doeu [%]
ce
92. 12
17.16
0.029
85.31
46.36
39.68
ceN
92.57
92.42
0.202
91.43
91.41
91.41
ceA
92.40
53.11
0.138
90.97
90.82
90.89
feD
92.29
89.63
0.198
91.40
91.39
91.40
feA
92.91
52.91
0.131
91.63
91.55
91.58
llc
91.06
90.96
0.204
90.35
90.30
90.27
con
92.63
8.81
0.094
81.23
26.64
56.16
conN
92.39
92.13
0.199
90.83
90.40
90.39
conA
92.34
11.88
0.130
90.74
90.49
90.65
conF
93.26
81.78
0.169
91.97
91.88
91.91
CIFARIO FMNIST
Train
Retrain
Abase [%]
Agert. [%]
Reert.
AFGSM [%]
APGD [%]
DeeppFool [%]
ce
85.70
32.06
0.076
59.91
50.72
55.31
ceN
79,84
78 18.42
0.153
69.86
69.01
69.28
ceA
84,83
56.72
0.121
71.91
70.36
71.29
feD
83.46
64.58
0.127
72.66
71.49
72.14
TeA
84.35
60.59
0.126
73.67
73.12
65.75
62.44
0.172
61.34
61.41
60.79
con
87.26
31.43
0.083
61.14
$1.84
57.41
conN
83.59
79.41
0.153
70.45
69.10
69.28
conA
85.39
51.29
0.114
72.72
71.44
71.47
conF
89.91
45.46
0.114
78.17
76.47
77.38
15

---

Under review conference paper ICLR 2023
Does robust refraining on the target domain improve robustness?
Table 2 and Figure show how target retraining affects robustness. In order to quantify robust- ness we compute the verifiable accuracy using randomized smoothing and accuracy under FGSM, PGD and DeepFool attacks.
Table 2: Accuracy on the clean test set (base), verifiable accuracy (cert.) and raduis, accuracy under FGSM, PGD and DeepFool attacks of target models trained on SVHN and retrained on MNIST. The best values/highest accuracy highlighted using bold letters.
SVHN - MNIST
Train
Retrain
Abse [%]
Agert. [%]
Reert.
ArGSM [%]
APGD [%]
ADeepFool [%]
ce
Ree
71.07
8.09
0.002
23.97
1.34
1.07
ROWN
40.29
31.47
0.017
9.70
0.27
0.14
RecA
61.26
9.90
0.009
34.49
4.43
2.33
ceN
Rce
70.31
69.12
0.121
S7.67
53.13
53.08
RowN
71.02
71.46
0.127
59.38
55.08
54.70
RecA
69.72
68.60
0.121
57.33
52.75
52.79
ceA
Ree
60.40
36.28
0.054
45.88
22.55
34.80
ceA
RoeN
53,00
73.21
0.117
42.15
27.14
33,93
ccA
RecA
60.42
33,96
0.052
46.22
24.48
35.90
feD
Ree
78.92
75.20
0.141
71.53
67.48
69.38
ReeN
79.29
81.34
0.151
71.94
67.63
69.31
feD
RynA
79.15
75.38
0.141
72.01
67.90
69.73
feA
Ree
59.64
50,95
0.088
46.20
38.65
38.30
feA
ReeN
53.10
66.12
0.098
40.13
33.69
32.96
feA
R..
59.09
51.09
0.091
46.82
40.46
39.44
lle
Rec
96.40
96.13
0.196
95.83
95.73
95.72
ReeN
96.41
96.32
0,200
95.91
95.86
95.83
llc
Ry.A
96.43
96.15
0.197
95.83
95.75
95.70
con
Rec
95.01
93.59
0.180
90.78
89.17
89.41
con
ReeN
94.58
93.90
0.184
89.93
88.32
88.71
RoeA
94.82
93,63
0.183
91.03
89,49
89.81
ConN
Rec
94.58
94.12
0.193
92.33
91.85
91.91
conN
ReeN
94.45
93.86
0.193
92.09
91.55
91.59
conN
ReeA
94.65
94.05
0.192
92.38
91.85
91.83
conA
R.
93.88
92.86
0.190
91.17
90.74
90.76
conA
ReeN
93.49
93.09
0.191
90.96
90.63
90.60
conA
ReeA
93.60
92.94
0.190
91.07
90.75
90.81
conF
R.
85.08
84.86
0.145
75.75
71.95
71.96
conF
ReeN
83.05
84.81
0.147
72.32
68.16
68.59
conF
ReeA
86.14
86.04
0.144
76.10
71.94
72.18
16

---

Under review conference paper ICLR 2023
Table Accuracy on the clean test set (base). verifiable accuracy (cert.) and radius, accuracy under FGSM, PGD and DeepFool and attacks target models trained on EMNIST and retrained on KMNIST. The best values/highest accuracy highlighted using bold letters.
EMNIST KMNIST
Train
Retrain
Abase [%]
Agert. [%]]
Reert.
AFGSM [%]
APGD [%6]
ApeppFool [%]
cc
Rec
79.29
36.82
0.030
63.82
19.79
15.80
ce
ReeN
41.00
57.63
0.059
27.91
6.98
4.70
RoeA
78.08
38.97
0.039
65.77
29.14
23,06
ccN
Ree
81.96
81.57
0.188
79.36
79.21
79.14
ceN
ReeN
82.18
81.80
0.188
79.41
79.30
79.21
ReeA
82.19
81.71
0.188
79.45
79.31
79.26
ceA
Ree
80.71
44.83
0.123
76.30
75.86
75.96
ceA
ReeN
63.81
71.07
0.144
57.77
57.12
57.23
ceA
ReeA
80.65
44.73
0.124
76.40
75.87
76.04
feD
Ros
84.82
79.64
0.187
82.99
82.98
82.91
feD
ReeN
82.04
83.46
0.190
79.91
79.89
79.79
ReeA
84.76
0.187
83.05
83.04
82.97
feA
Ry
82.09
41.82
0.142
78.34
78.09
78.07
feA
ReeN
66.10
75.17
0.158
60.85
60.46
60.35
feA
ReeA
82.25
44.98
0.137
78.45
78.17
78.13
Ilc
Ro.
81.21
81.15
0.191
79.61
79.41
79.30
llc
ReeN
80.96
80.83
0.191
79.03
78.85
78.76
ReeA
81.26
81.19
0.191
79.72
79.50
79.36
con
Rye
69.69
13.67
0.066
46.34
7.24
8.65
con
ReON
18.45
39.21
0.050
12.47
2.12
2.50
RecA
62.56
17.70
0.039
47.16
11.48
12.80
conN
Ree
60.88
58.67
0.126
50.73
47.39
47.25
conN
ROON
60.39
58.64
0.128
49.90
47.26
47.13
conN
ReeA
58.98
$1.30
48.35
48.13
conA
Ree
61.80
12.26
0.168
52.30
49.85
50.17
conA
ROON
20.62
38.35
0.092
14.25
13.08
13.59
conA
RecA
60.93
11.41
0.178
51.28
49.32
49.45
conF
Ree
51.86
35.58
0.089
43.35
42.03
41.84
conF
RoWN
41.81
40.54
0.089
33.19
31.92
31.86
conF
RecA
51.81
35.84
0.094
43.75
42.46
42.22
17

---

Under review conference paper ICLR 2023
Table 4: Accuracy on the clean test (base), verifiable accuracy (cert.) and radius, accuracy under FGSM, PGD and DeepFool attacks of target models trained on CIFARIO and retrained on FMNIST. The best values/highest accuracy highlighted using bold letters.
CIFARIO FMNIST
Train
Retrain
Abase [%]
Agert. [%]]
Reert.
AFGSM [%]
APGD [%]]
DeepPFoo [%]
cc
Rec
71.11
26.25
0.039
47.72
28.13
31.91
ce
ReeN
35.64
64.96
0.084
16.98
6.92
8.37
RoeA
70.47
26.03
0.036
51.57
36.20
38.34
ccN
Ree
73.18
69.58
0.144
64.32
63.11
63.17
ceN
ReeN
72.48
71.33
0.150
64.49
63.31
63.53
ReeA
73.23
69.4
0.146
64.67
63.58
63.63
ceA
Ree
74.60
67.61
0.131
63.63
61.81
62.15
ceA
ReeN
65.87
72.10
0.155
56.58
54.81
55.13
ceA
ReeA
75.32
66.70
0.132
65.37
63.71
64.17
feD
Ros
71.52
57.24
0.118
62.19
60.41
60.96
feD
ReeN
59.31
70.05
0.159
50.89
49.25
49.66
ReeA
71.92
0.119
63.30
61.71
62.26
feA
Ry
69.83
61.53
0.108
59.44
57.08
57.56
feA
ReNN
64.08
70.39
0.137
54.99
52.89
53.28
feA
ReeA
69.82
61.12
0.113
60.29
58.38
58.69
Ilc
Ro.
80.07
79.70
0.180
76.08
75.80
75.64
llc
ReeN
79.02
79.28
0.184
75.93
75.70
75.59
ReeA
80.10
79.88
0.185
76.64
76.45
76.38
con
Rye
79.81
38.93
0.076
51.31
28.42
33.61
con
ReON
70.39
73.22
0.122
32.51
14.57
16.96
ReaA
78.13
48.20
0.092
57.73
41.66
44.26
conN
Ree
82.25
78.26
0.158
68.31
63.39
63.98
conN
ROON
80.94
78.42
0.163
67.43
62.74
63.44
conN
ReeA
82.04
78.35
0.165
70.27
66.87
67.15
conA
Ree
80.31
53.23
0.098
67.66
64.09
65.01
conA
ROON
73.11
73.07
0.132
61.69
58.02
58.99
conA
RecA
79.75
54.87
0.102
68.31
65.41
66.06
conF
Ree
75.97
61.55
0.097
60.67
56.97
58.09
conF
RoNN
62.25
70.72
0.120
42.03
38.47
39.68
conF
RecA
75.36
64.26
0.105
61.96
59.15
59.79

---

Under review conference paper ICLR 2023
FCSM staad: SXTEE PRIST
FOON sttak: EMNIST KSNEST
FOOM CITASD FRBIST
Accoory of
ACCUPSIC (8) CHAASS
PINES
P3 Metrod
Tg Merrod
Tah MeThod
FM
0814
SVIN VIEST
T5D atbei:: ENNAST CHUET
PSD actck: DITARL THEAST
Rccuncy 180 HM
5
Accuooyy on ORNST
123
FHEST
Meched
com m/d
00 100
cand m9
50 160
1910
Accooyy 180 Ma
5
Bogiaa Bechh
MOOICC 18) KOR
PHAIST
C 0
PMR
CP
22W
C6T
32
Merhed
CCN1 C21A C27
484
32
2011 C314 cCW
Figure 8: Accuracy on the clean test set (base, bright colors), verifiable accuracy (cert., dark colors). accuracy under FGSM (dark green), DeepFool (DF, dark blue) and PGD attacks (dark purple) of target models retrained on normal (Ree), randomly (Roex) or adversarially perturbed (Roos) target data.
Which training/target retraining provides models that are robust against distribution shifts?
Table 5 6 7 8 and Figure 9 TO
12 show accuracy of source models and target models under perturbations, changes in contrast and Gaussian blur. More specif- ically, we use Gaussian noise shift (Noise), uniform noise shift (UNoise), contrast reduction shift (Contrast), contrast reduction shift based on a binary search (ContrastBin), contrast reduction based on linear search(ContraslLinn), Gaussin blur (Blur) and salt and pepper (SaltPepper) shift.
distribution shifts based
19

---

Under review conference paper ICLR 2023
Table 5; Accuracy on the clean set (Abase) and under distribution shifts source models trained on SVHN (first row), EMNIST (second row) and CIFARI 10 (third row). The best values/highesi accuracy are highlighted using bold letters.
MNNST
Train Rotain
Ausue [5]
Apube [8]
Ausste [8]
Acuztrue [8]
Assutrattia [8]
ACostraciin [%]
Aptu [%]
AsuitPpppe [8]
cc
94.15
82.45
N1.91
N6.37
84.17
85.50
25
29.70
coN
93. 40
91.97
91.66
80.83
79.
78.65
26.87
36.62
14.
88-97
88. 7M
85,87
84.55
84.33
29,
29.25
69
50
89.57
85.16
83.67
83.49
28.54
38. 10.10
83.99
con
conN
91.70
81:10
81:01
26.30
coMA
92 41
87
87
86.07
85
84 06
26.86
25.15
conf
95,28
$3.06
88.63
88,15
86.81
86.70
29.51
38.49
EMNIST KMNIST
Train
Retrain
Apules [%]
Ausus
(%)
Acsuraatlla (%)
Apuu [%)
92.12
25
22.93
11.34
9.33
27.80
42.10
90.57
92.29
91.72
91.79
39.74
88.32
88.17
88.51
61.76
feA
92.91
92.82
92.83
92.45
91.71
91.03
83.14
41.47
lIc
91.06
91:07
91.00
90.27
89.29
89.27
85.39
86.87
92.63
15.50
16.54
92.54
92.08
91.34
83.53
4.59
conN
92.39
92:31
92.36
91.95
85.63
18.13
conA
FMNIST
Train
Retrain
Auue [%]
Abube [%6]
Ausste (%)
Acuztese (9)
Assntrattia [%]
Acontratii La [%]
Autu [%]
Asutteppee [%]
ce
85.70
14:11
43:27
75.41
72:25
71.72
16.61
26.34
cuN
79.84
81.24
80.08
60.61
56.
$5.50
25.37
33.40
84.83
64.77 64.70
64:05
74.72
71/04
70.54
20.89
31.79
71
70.54
24.96
29.78
84.35
67.93
67.77
34.55
conN
83.39
80.70
80.70
82.25
W1:22
80.12
45.54
41.80
conA
85.39
50.45
56.
84.36
82.67
82.63
44 45.96
41.47
conl
89.91
56.31
55 41
85.36
83.54
83.51
28.81
44.57
20

---

Under review conference paper ICLR 2023
Dianbution shil SVER MAST
Dtinbution shill:: Sahh MMST
Disiieut shille: SVER MINST
100
1N on SVAN
Spent
Alcuncy
Psning Vetrod
Tohing Metrod
Paiiing Method
Didrbution chi TNEEST-KO KURIST
Did betrbaiinncoo thite THMST KMB.ST
Elerbation thit THUST- 19BAST
Accurecy EMRUST
3
Accrooy Did EMRIST
123
10 16
Meched
Mectod
Accurte (80 Clok
Aco/SO 160 on C10R30
ACCUrSO (8) en C10N30
12 5
000 100
Merhod
00m 0aw 2210
32 190
Menhed
CONS can. co'
C0N caw com
Figure 9: Accuracy on the clean test set (Abares bright colors) and accuracy under distribution shifts based random noise, changes of the contrast and Gaussian blur of target model trained SVHN EMNIST or CIFARIO.
Table 6: Accuracy on the clean test set (Abase) and under distribution shifts of the target models trained SVHN and retrained on MNIST. The best values/highest accuracy are highlighted using bold letters.
MNIST
Rermin
Ause [%]
Aguse [%]]
Aprosw [5]
AQuntca [S]
Arstraaa [S]]
ACustractin [5]
Aguin [S]
Asua Pogper [%]
Rsx
71:07
ce
RaN
40.29
28.00
28
26.03
17.48
7.51
8,26
3.50
RaA
61.26
62.01
62.7
64.32
46 (6.20
27:11
29.88
16.19
ccN
Ra
70.31
61 34
60:72
57.16
46,62
38.70
40:16
23.31
ccN
71.02
67.94
67.5 30
64.99
$1.71
45.00
44.04
26.72
69.72
38.20
30.99
L668
RaN
53.00
81.83
8.42
ceA
Rack
00:42
38.24
39 12
40.04
25
14:35
20.25
1,50
Rs
78.92
43.52
41. 10
40.17
32.67
17.11
44.45
17.06
leD
Ras
79,29
55 34
52.87
46.16
38.54
25.91
$1.991
16.88
IcD
79.15
42.94
40 66
39:12
31
17.03
44.12
17.98
$9.64
38A1
Rack
59.09
35.12
lIc
B
96:10
96.57
96.56
96.60
95.97
95.96
91.79
92.51
IIc
RuN
96.41
46
96
96.53
96.10
96.09
93.37
93.12
IIc
Baa
96.43
96.51
96-17
96.68
96.06
96.05
92.03
92.58
Ra
05.01
94:17
94-47
95.34
94 10
94.20
91.21
24.79
94.58
94.38
91.73
93.55
90.59
29.13
Bis
conN
Rack
94.45
94:47
$4.65
93:45
90.27
50.4
conN
Back
94.65
94.09
94.79
94.96
93.72
95.63
90.43
47.82
conA
93.88
93.82
93.53
$4.08
93.25
93.19
89.68
31.67
conA
Bun
93.49
93.09
93.53
92.89
92.94
92.59
89.33
35.91
conA
93.60
93.35
93.62
91.05
93.06
93.02
$9.35
32.86
37.13
30.06
conF
BaN
conF
R.ck
86.14
31.82
30.66
28.35
26
16.42
27.15
21

---

Under review conference paper ICLR 2023
Table 7: Accuracy on the clean test (Abase) and under distribution shifts of the target models trained on EMNIST and retrained on KMNIST. The best values/highest accuracy are highlighted using bold letters.
EMNIST +s RMNIST
Train Rotmin
Ausu [9]
Apube [%]
Ausste [R]
Acuztzue [8]
Assutrattiia [8]
Acostractin [8]
Agtu [8]
AsuitPapper [8]
cc
Hoe
79.29
51:75
30.54
19.19
17788
16.14
36.07
37.19
cV
Ras
41.00
31.43
30 15
18.69
13.44
12.44
19.18
23.87
Rask
7X.08
54.1 14
12,82
19.61
18.27
16.64
40,84
43.26
ccN
Ba
$1.90 96
82.09
1.00
71.06
67.67
66.90
71.08
61.18
K2.18
Rue
28.14
02
24.63
ceA
RaN
63.81
71-12
61.13
47.05
43.16
20.37
coA
RaA
50.65
69.51
69 K8
27.90
24
23:23
61.33
25,25
RcD
Ba
$4.82
83.01
R2
79.86
71.21
76.96
79.13
$3.70
IeD
Rad
$2.04
83,32
83.39
83.59
76.85
78.66
75.52
$6.31
$4.76
22.96
83.12
79,86
77.26
77.05
79.11
53.78
BuN
29.25
Rack
$2.25
81,16
35
$0.80
78.92
70:56
36.85
Mcc
Rae
81.21
81
K1
79.91
76.91
76.87
73.01
71.50
Ilc
BaN
$0.96
80.50
80.81
79.66
76.73
76.61
72.11
71.25
llc
Rock
81.26
81
61:17
80.01
77/03
77.01
73.29
71.76
69.69
23.74
24.72
69.56
66.21
50.54
1,99
15:45
16:01
conN
B
60.88
60.99
60.85
42.91
2.26
conN
BuN
00.39
60.50
50.38
60.28
56.65
55.94
42.73
2.45
conN
Back
61.16
61.01
61.23
60.72
$7.30
$6.70
43.17
2.76
conA
Ba
61.80
23.08
24.00
62.60
59.05
58.67
41.60
1.90
conA
20.62
35:41
35.66
20.15
18.02
17.79
11.14
0.66
conA
00.93
22.83
58.22
2.18
conl
B.N
41.81
conl
Bock
31.81
47.63
48.10
51.42
47.28
05.36
34.90
4.45
Table 8: Accuracy the clean test set (Abase) and under distribution shifts of the target models trained on CIFARIO and retrained on FMNIST. The best values/highest accuracy are highlighted using bold letters.
CIFARIO FMNIST
Train Retrann
Arsnraatia [8)
Acstractio
[S]
AsuaPrppe
68
Bao
70:47
35
60
$1.00
21.38
21.0
ceN
B.
73.18
61.93
$0.56
41.54
37.68
35.90
33.92
21.92
ccN
72.48
15-19
65.33
44.60
40.87
38.15
35.95
21.23
ccN
Back
23.25
62.67
60.70
41.62
38.07
36.04
34.45
21.99
Bas
74.60
70.9
70.53
69.57
16.56
02.61
11.42
21.12
65.87
72.05
28.63
B.
71.52
13.69
feD
BaN
59.31
89.01
89.03
6L.87
52.33
50.82
36.03
16.15
RD
Back
71.92
61.23
61 63
56.76
53.
$0.00
37.21
15.03
feA
B..
09.83
47.54
47 50
49.23
45
41.89
25
14,85
61.08
59.07
$9.11
$2.91
09
43.52
30.91
20.98
09.82
$9.31
51.29
B.N
IIc
Bac
80,10
79.33
79.05
70.92
69 36
69.21
66.18
67.88
con
B..
99.81
28.53
49.73
79.71
76.19
75.86
$2.91
28.15
Bas
70.39
76.02
75,90
70.25
64,30
64.18
40.55
17.15
Back
78.13
58.4
77.29
74.91
74.77
$8.66
36.11
conN
$2.25
79.66
79.70
76.91
76.21
58.27
24.65
conN
90,00
75.06
Rock
conA
Rx
77.09
conA
Rad
73.11
75.74
75.35
71.64
69.01
68.03
51.17
14.17
<onA
Rack
70.75
64 68.68
65.01
77.26
74,50
74.61
60.82
12.35
conl
B.
75.97
62.19
61.79
68.66
65.13
62.87
41.23
27.31
codF
Raw
62.25
67
67.81
56.35
$1.13
47.13
34.86
14.25
66.01
22

---

Under review conference paper ICLR 2023
Datt bution ardt NN - 57857
Dldt bullen atdl "MNeba"t SS0R - VR1ST
Dutr butiot ahih "545% 5VHN HRUST
Accuncy (8Q MNIST
Aee
Acc/noo (80 MAIST
en VNEY
Method
Paning Methed
Taining Methoa
Dior Bution Controatt D01B 18AST
Eieraution aNe "Cntrrecttt DOI VRIST
Derrbuu "oommntt" 921% NMS5T
12
8 S
Michodd
Mestod
Voning Mictod
AcCoO// Of WAST
Mestod
Figure 10: Accuracy on the clean test set (Abases bright colors) and accuracy under distribution shifts based random noise, changes of the contrast and Gaussian blur of target model trained on SVHN and retrained on clean (Ros), randomly (RoaN) or adversarial perturbed (ReeA) ) inputs (MNIST).
Distriuution thit Tidee" RNNSST KMNSST
Distrbbrtion shi "Khoee EVNIST- GHEST
Distituurinn shitt "SPP EVRIST ENNIST
Accuntoyy KWN
Mono 150 KMR.ST
40
Plcctod
Mactod
Macted
DEsribucin
ACCiNN (NL on KONIST
MCDD (N) on KOWEST
KONEST
5
Veched
Mecttd
ADDU/DO//N 1N KTAUST
Method
Figure 11: Accuracy on the clean test set (Abase: bright colors) and accuracy under distribution shifts based random noise, changes of the contrast and Gaussian blur of target model trained on EMNIST and retrained on clean (Roo), randomly (Reex) or adversarial perturbed (Reca) inputs (KMNIST).
23

---

Under review conference paper ICLR 2023
100
Didnbbuiem and Maaser CHARID PABST
10
Decriutionn STE CEAEENCCCARL CEAREO 19957
Detrbuienanit SIP's CHARGO 1AMSST
Recurscy (i0 PANST
PAASST
Pining Method
Taningg Methed
Poiiing Method
183
Didrbbution ENd "Cattroof CIARR0 TWARST
103
Eleribation enp: CPARLD.
Disttbuttion diD Cortrndd CEARDS PHNEET
Recurtry (N0) FWAST
Accrocy 180 SWaST
Michod
Mestod
loning Mocood
169
ACCUSSC 180 PAAST
Mestod
Figure 12: Accuracy on the clean test set (Abase. bright colors) and accuracy under distribution shifts based random noise, changes of the contrast and Gaussian blur of target model trained on CIFARIO and retrained on clean (Roe), randomly (ReeN) adversarial perturbed (ReeN) inputs (FMNIST).
Does transferability correlate with model robustness?
Table shows the verifiable accuracy (Asmooth) accuracy under the strongest (FGSM. PGD, Deep- Fool) attack (Aatr. astack) and quantifies transferability using H-score. Table TT shows the source accuracy and the zero-shot target accuracy, i.c. the accuracy source model achieves on the target dataset before target retraining. Results of both tables are visulaized Figure
Table 9: Transferability, verifiable accuracy and accuracy under attack of source models versus robustness the source dataset. Transferability is measured by the H-Score, while robustness is quantified using verifiable accuracy and accuracy under the strongest attack.
SVHN MNIST
EMNIST KMNIST
CIBARID FMINST
Train
Aust
Agr. [%]
H
Aoct.
[%6
Aart. Autr,
[8]
6.81
17.16
18.23
32,06
5.44
ocA
6.84
53.11
90.82
18.76
$6.72
70.36
5.33
IeD
83. 70
16.17
6,69
30.63
91.30
19:25
64.58
71.49
5.16
$7.91
$6.13
6.81
52.91
91.55
18.82
60.59
72.56
5.28
Be
76.00
71.50
1.94
90.96
90.27
17.47
62.44
60.79
3.62
85.27
32.16
7.61
11.88
4.88
51.29
71.44
9.55
oxF
86.55
87.20
3.06
$1.78
91.88
98
45.45
76.47
7.39
24

---

Under review conference paper ICLR 2023
Table 10: Accuracy of source models (before target retraining) on the target dataset versus accuracy on the source dataset.
SVHIN
MNIST
EMNIST
KMNIST
CIFARIO FMNIST
Train
Agouree [%]
Target [%]
Agoures [%]
Aurget [%]]
Asaurea [%]]
Amrrggo [%)
94.03
19.81
92.12
1.56
85.70
5.44
ceN
93,40
23.13
92.57
2.01
79.84
5.96
ccA
94.21)
19.96
92.40
1,69
84.83
11,52
feD
93.69
16.79
92.29
1.87
83.46
6.39
feA
94.33
84.35
3.89
77.60
58.32
91.06
65.75
con
93.32
65.24
92.63
1.23
87.26
9.91
conN
91.70
65.78
92.39
0.95
83.59
9.77
conA
92.41
66.17
92.34
1.08
85.39
9.96
conF
95.28
25.10
93.26
1.68
89.91
10,20
Vediication SUIA MIIST
Attreiigg SPAK NNST
Zene-hott porfermance: SVE- NIIST
cah
8.3
2.5
cor7
conM
5.5
3.0
5.0
20
80
102
40
80
%
Accuracy
Voriiictionn ENNES KNNSS
Atracking: EWWS KVRIST
20.0
Zaro shst parfomaanc EMNST KORRST
25
25
37.5
20
20
10.0
7.5
2.0
2.5
102
MobusCHaS Wer
fotustresu (5) (Aliacking)
Source Accuracy
Ver foot on: CHAR10 19eaST
Aueding: CPARLD IMIIST
20.0
Zuro-shol performance CPARLO PINST
3775
col
AM
10.0
2.5
10
20 Robustness 181 [verFicatio1
20
50
Robucthess (Atbecknn))
Source Accurny 144 GRA310
Figure 13: Transferability quantified by the H-Score robustness of the source models quan- tified as verifiable accuracy (Verification) or accuracy under the strongest (FGSM, PGD, DecpFool) attack (Attacking) and zero-shot performance on SVHN MNIST, EMNIST KMNIST and CI- FARIO FMNIST.
25


Reviews:
[{"review_id": "xS7TR9DKYq3", "reviewer": "Reviewer_nHzp", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies how and when adversarial robustness to be preserved and transferred across domains. In experiments, the paper shows that 1) training procedures affect robustness, 2) contrastive learning and llc are more generic thus achieve better robustness during target retraining, 3) training procedure on the source domain has a major effect on target robustness than target retraining, 4) robustness to attacks infer robustness against distribution shifts, and 5) transferability depends on how related between the source and the target domain. However, these findings/conclusions are either trivial or from non-thorough analysis, which significantly reducing the paper contribution.", "summary_of_the_review": "This paper focuses on the transferability in adversarial robustness across domains. It gives very detailed experimental results.\n\nBut it is less likely to be accepted as an ICLR paper. There is no novel idea proposed, though a very detailed experiment process is presented. The evaluations are weak, and the discussions are trivial.", "strength_and_weaknesses": "Strengths: \n- The paper makes an experimental exploration on the transferability in adversarial robustness across domains. The topic itself is interesting.\n- This paper is well written and easy to follow.\n\nWeaknesses:\n- This is a paper with experimental studies and findings. My main concern is that the conclusions are trivial, highly expected with weak evaluations. Specifically\n   1. 'conF is the best training procedure.....' is expected as 'distance' loss improves robustness. The authors are suggested to include [1] TRADES as a training procedure. To demonstrate the effectiveness of contrastive method, a comparison between TRADES and TRADES+con is recommended.\n   2. 'contrastive learning and llc achieve better robustness during target retraining': The authors are suggested to study on harder datasets (i.e., ImageNet, CIFAR100), larger architectures (i.e., WRN), and stronger attacks (i.e., [2]AutoAttack). The weak evaluation may provide a false sense of robustness.\n   3. 'robustness to attacks infer robustness against distribution shifts': sec.4.4 only compares clean test accuracy vs. distribution shift. IMO, a direct comparison between robustness and distribution shift is better.\n   4. 'transferability depends on how related between tasks': this finding is trivial as it has been discovered by many previous works. \n- Most of contents are covered by [Shafahi et al. (2020)], which uses harder datasets (e.g., ImageNet->Cifar-100, Cifar-100 -> Cifar-10), larger architectures (ResNet50 and WRN32-10), and normal attack settings. Their conclusions are more thorough towards adversarial robustness. The contributions towards verification and robustness to distribution shift, as claimed by authors, are trivial.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "gK0V_1tXUR", "reviewer": "Reviewer_zUPp", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work analyzes how different training procedures on the source domain and fine-tuning strategies on the target domain affect model robustness. The authors show that the training procedure on the source domain has a major effect on target model robustness while target retraining has a minor effect. The authors indicate that contrastive learning and training with a local Lipschitz constant best preserve robustness during target retraining.", "summary_of_the_review": "The authors use a popular transfer learning framework consisting of two parts, a feature extractor f which extracts representations from the inputs and is trained on the source domain and a classifier h which maps extracted representations to predictions and is retrained on the target domain. And the authors investigate and compare how different training procedures and target retraining techniques affect performance and robustness of this model. If the experiments were properly configured, the results would help us understand how transferable is the adversarial robustness after fine-tuning. \n\nUnfortunately, the experiments miss a critical point: in many transfer learning tasks, the source model is likely to be pre-trained on a much larger dataset. This is especially true in the setting of contrastive learning since no human annotated labels are required in the source task. However, the author failed to consider this situation in their experiments, and in all tasks, the source dataset is not significantly larger than the target dataset. This significantly degrade the impact of their results. \n\nFurthermore, the authors of the paper “When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Fine Tuning” already showed that pre-training in contrastive learning can improve cross-task robustness transferability. The authors should better position their work and describe the value added. \n", "strength_and_weaknesses": "S1. The authors conduct a wide range of experiments to compare how different training procedures/target retraining techniques affect the adversarial robustness of a model.\nS2. The results showing that improving model robustness on the source domain increases robustness on the target domain, while target retraining has a minor influence on target model robustness, are intuitive yet important to the practitioners in the field.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "VDwxqzUFwc", "reviewer": "Reviewer_fomQ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors compare multiple training procedures in order to understand which training techniques improve target robustness.  They find that generally increasing robustness on the source increases robustness on the target.  They also find that techniques involving local lipschitz and contrastive learning generally transfer better.  The authors also find that target retraining doesn't impact target model robustness much.", "summary_of_the_review": "Overall, I find that this is a well-written paper with extensive experimentation.  However, I think that since the paper claims to analyze the transferability of adversarial robustness, the authors should consider both L2 and Linf attacks with perturbation sizes that align with adversarial ML literature.", "strength_and_weaknesses": "Strengths:\n- extensive experiments: the authors compare many different training techniques on a variety of datasets (SVHN, MNIST, CIFAR-10)\n- writing is clear\n\nWeaknesses:\n- When comparing verifiable robustness on the source, it would also be informative to provide results with all training techniques augmented with Gaussian noise.  If the base classifier is not robust to Gaussian noise, then it's not surprising that verified robustness via randomized smoothing would be low as randomized smoothing requires that the base classifier is robust to Gaussian noise, which makes the results in the top row of Figure 2 not very informative.\n- There are only results for L2 adversarial robustness with $\\epsilon= 0.1$.  Commonly in adversarial ML literature $\\epsilon=0.5$ with CIFAR-10 is used so I think some results should be provided for this larger radius as well.  Currently the standard trained source model does not drop to close to 0% accuracy with this small perturbation.  Additionally, it would be interesting to see some results for empirical Linf adversarial robustness as well.\n- It would also be interesting to see results with stronger attack methods (specifically using either fab-t or apgd-t from AutoAttack) to better measure empirical robustness.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "CqdxERO0a_", "reviewer": "Reviewer_3GPh", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents an empirical study on how different robust pretraining protocols affect the robustness of different downstream tasks. In this regard, many different pretraining strategies are compared on a few small-scale pretraining-finetuning pairs (e.g., SVHN-> MNIST or CIFAR10->FMNIST) in terms of the $\\ell_2$ adversarial robustness (at small $\\epsilon$) of the fine-tuned models. Based on this study, the authors claim that the best pretraining protocols are those that achieve a general-level of robustness without overfitting to a particular dataset, and that the fine-tuning protocol does not influence too much the final performance. All experiments are performed evaluating both empirical robustness (using diverse attacks) and certified robustness (based on randomized smoothing). Some results are also provided in terms of robustness to common corruptions.", "summary_of_the_review": "Unfortunately, I believe that the paper does not meet the bar for acceptance to ICLR. The overall research question of this empirical study is very interesting and relevant for the community, but the execution of the study is suboptimal and does not adress the right subquestions. For being accepted this paper would require to perform a much larger scale evaluation of robustness using larger pretraining tasks and multiple downstream targets. The evaluation of robustness should also be conducted at diverse robustness regimes since these have qualitatively different properties.", "strength_and_weaknesses": "# Strengths\n\n1. **Interesting scientific question**: I find, the main premise for this experimental study very relevant to the community. Indeed, understanding when and how does robustness transfer is a problem of great interest.\n2. **Evaluation of many robust pretraining baselines**: It is quite admirable that the authors tested and evaluated all their experiments on 10 different pretraining strategies both in terms of empirical accuracy and certifiable robustness.\n3. **Clear writing**: Overall, the paper is easy to read.\n\n# Weaknesses\n1. **Small-scale pretraining tasks**: Although I can understand that the computational complexity of repeating all the experiments presented in this work using larger-scale datasets for pretraining, such as ImageNet, would be much much higher, it is not very realistic to expect that the same phenomena observed when pretraining on EMNIST or CIFAR10 would be directly observed on ImageNet. This is clearly a very strong weakness of this paper, because even in the case where all the experiments had been flawlessly executed, they will probably show irrelevant results that do not capture the behaviors that are seen in practice (and that serve as motivation in the introduction).\n2. **Only very small values of $\\epsilon$ are studied and for the $\\ell_2$-norm**: Similarly, another important weakness of this work is that it only shows results with very small values of $\\epsilon$ (i.e., robustness budget) and only $\\ell_2$-norm. $\\epsilon=0.1$ is much smaller than the standard values used by the community to evaluate robustness on these datasets (e.g., $\\epsilon=0.5$ on CIFAR10 in RobustBench). This is not a small detail, but a very important factor that qualitatively affects the behaviour of the models. For example, at very small values of $\\epsilon$ (like the ones used in this paper) it is known that augmenting the training set with random noise can confer some non-trivial accuracy against those weak budgets, however this is clearly not the case at larger $\\epsilon$. In this regard, one big thing missing from all the experiments presented in this paper is an ablation study investigating the role of $\\epsilon$ and $\\ell_p$ norm on the results. And if a proper ablation study is not viable, at least all the experiments should have been evaluated in a much more challenging robustness regime (i.e., larger $\\epsilon$ and $\\ell_\\infty$).\n3. **Adversarial robustness is not necessarily connected to robustness to distribution shifts**: In an attempt to increase the breadth of the study, this paper also tries to address the problem of robustness against distribution shifts or common corruptions using the same experimental protocol (i.e., using adversarially pretrained models and fine-tuning on a different task). However, in general, there is still no consensus in the community on whether adversarial robustness and robustness to distribution shifts are actually connected (Yin et al. 2019, Kireev et al. 2022). In this regard, by evaluating only the differences in robustness to distribution shifts of models pretrained to be robust against adversarial perturbations, the paper is focusing again on an irrelevant problem. For this study to be relevant, the paper should compare the finetuning performance of models pretrained with methods that confer robustness to distribution shifts, and not necessarily only to adversarial perturbations.\n4. (Minor) **Strange transferability study**: I find the study about the role of representation transferability in downstream robustness a bit odd. Personally, when I first read the motivation of this research question in the introduction, I thought the main question was understanding how the pretraining task and its transferability downstream, influenced the robustness of the fine-tuned model. However, the study focuses only on studying differences in the quality of the representations, as all the experiments are performed for the same pretraining and downstream tasks.\n\n- Dong Yin, Raphael Gontijo Lopes, Jon Shlens, Ekin Dogus Cubuk, Justin Gilmer. A Fourier perspective on model robustness in computer vision. NeurIPS 2019.\n- Klim Kireev, Maksym Andriushchenko, Nicolas Flammarion. On the effectiveness of adversarial training against common corruptions. UAI2022.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject