On Transfer of Adversarial Robustness from Pretraining to Downstream Tasks
Authors: Laura Fee Nern
Harsh Raj
Maurice Georgi
Yash Sharma
Venue: NeurIPS 2023 poster (2023)
Keywords: Machine Learning Theory
Transfer Learning
Adversarial Robustness
TL;DR: We provide a theoretical bound on downstream robustness for linear probing by the robustness of its underlying representation function.

Abstract: As large-scale training regimes have gained popularity, the use of pretrained models for downstream tasks has become common practice in machine learning. While pretraining has been shown to enhance the performance of models in practice, the transfer of robustness properties from pretraining to downstream tasks remains poorly understood. In this study, we demonstrate that the robustness of a linear predictor on downstream tasks can be constrained by the robustness of its underlying representation, regardless of the protocol used for pretraining. We prove (i) a bound on the loss that holds independent of any downstream task, as well as (ii) a criterion for robust classification in particular. We validate our theoretical results in practical applications, show how our results can be used for calibrating expectations of downstream robustness, and when our results are useful for optimal transfer learning. Taken together, our results offer an initial step towards characterizing the requirements of the representation function for reliable post-adaptation performance.

On Transfer of Adversarial Robustness from Pretraining to Downstream Tasks
Laura F. Nern
Harsh Raj Delhi Technological University parsh777111raj0maaa com
Yahoo Research
laurafee waurafee..erneyaaalte...o.. nern@yahooinc.com
Maurice Georgi
Yash Sharma
Hyundai Mobis
University of Tübingen
georgimauricce@@mail.. com
y sharma@bettheelab.oo. org
Abstract
As large-scale training regimes have gained popularity, the use of pretrained models for downstream tasks has become common practice in machine learning. While pretraining has been shown to enhance the performance of models in practice, the transfer robustness properties from pretraining downstream tasks remains poorly understood. In this study, we demonstrate that the robustness of a linear predictor on downstream tasks can be constrained by the robustness of its under- lying representation, regardless of the protocol used for pretraining. We prove (i) bound on the loss that holds independent of any downstream task, as well as (ii) criterion for robust classification in particular. We validate our theoretical results practical applications, show how our results can be used for calibrating expectations of downstream robustness, and when our results are useful for optimal transfer learning. Taken together, our results offer initial step towards character- izing the requirements of the representation function for reliable post-adaptation performance
Introduction
Recently, there has been rise of models (BERT 31 GPT-3 M7, and CLIP trained on large-scale datasets which have found application to variety of downstream tasks [6] Such pretrained representations enable training a predictor. e.g., linear "head", for substantially increased performance in comparison to training from scratch [10 11, particularly when training data is scarce.
Generalizing cireumstances unseen in the training distribution is significant for real-world appli- cations 2 60) 29] and is crucial for the larger goal of robustness [27). where the aim is to build systems that can handle extreme, unusual, adversarial situations. In this context, adversarial attacks 55 i.c., inducing model failure via minimal perturbations to the input, provide a stress test for predictors, and have remained evaluation framework where models trained on large-scale data particularly when, in training, adversarials are not directly
have consistently struggled [53 21 22),
accounted for (23 40).
With that said, theoretical evidence [8] has been put forth that posits overparameterization is required for adversarial robustness, which is notable given its ubiquity in training large-scale datasets. Thus, given the costs associated with large-scale deep learning, we are motivated to study how adversarial robustness transfers from pretraining to downstream tasks.
http:://gtthubb.oo////ccccrrroot transfer
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

trinning
adversatial examples
rehan rpprcecation
linearly separated Clases
Figure I: Overview, We distinguish between robust (red) and non-robust (green) representation by its sensitivity to adversarial attacks, and use both for linear separation of the representation space, e.g., binary classification. This illustrates the dependence of the classifier on the representation's robustness, i.e., the predictor can only be robust if Fade and z can be linearly separated from a' in representation space.
For this, considering the representation function be the penultimate layer of the downstream predictor, we contribute theoretical results which bound the robustness of the downstream predictor
by the robustness of the representation function. Fig. [ and shows that linear classifier's robustness is dependent on the robustness of the pretrained representation. Our theoretical results presented in Sec. 3en Sec. 4 are empirically studied in Sec. [5]
iillustrates the intuition behind our theory,
where we validate the results in practice, and in Sec. 6 to show how the theory can be used for
calibrating expectations of the downstream robustness in self-supervised manner [4]. In all, our theoretical results show how the quality of the underlying representation function affects robustness
downstream, providing lens on what can be expected upon deployment. We present the following contributions:
we theoretically study the relationship between the robustness of the representation function, referred as adversarial sensitivity (AS) score, and the prediction head. In Thm. [ we prove a bound on the difference between clean and adversarial loss in terms of the AS score. In Thm. 2 we prove sufficient condition for robust classification again conditioned on the robustness of the representation function.
we evaluate the applicability and consistency of our theory in applications in Sec. 5.1 and Sec, 5.2 Furthermore, Sec. (6 we illustrate the utility of the AS score to predict expected robustness transfer via linear probing.
in Sec. 6.2 study how linear probing compares to alternatives, like full finetuning, in robustness transfer for robust performance downstream.
Related Work
Transfer Learning:
While empirical analyses of pretraining abound
33
12
62
number 38
of theoretical analyses have also been contributed [56] 52
19
14
49
35
The majority of
the theoretical studies aim to understand the benefit of transfer learning with respect to improved performance and data requirements compared to training from scratch. Therefore, the referenced studies include results on sample complexity [52 14 49 and generalization bounds (38 fine- tuning and linear probing in various scenarios. In this work, we primarily focus the linear probing approach and perform a theoretical analysis to investigate its efficiency in transferring adversarial robustness.
Robustness: There is direct connection between the mathematical formulation of adversarial robustness and the definition of local Lipschitz constant of the network [25], but computing this quantity intractable for general neural networks. However, there exists a line of work providing more
practical approaches in certifiable robustness [54] 44
15
37 36 46), which can provide guarantees on
the detection of adversarial examples. Relative to empirical approaches (23
certifiable methods have comparatively struggled to scale to datasets such as ImageNet 17 Furthermore, likely
8 48 35
due to computational expense, adversarial training has not yet been adopted by models trained on
large-scale datasets B1 7 43). Still, number of works have incorporated adversarial training into

---

pretraining, in particular, self-supervised contrastive learning [2 30 9 20].
With that said, our work focused on the analysis of robustness irrespective of the pretraining approach.
Regarding analysis, [59] studied the trade-off between accuracy and robustmess, and showed that, under set of separation conditions on the data, robust classifier is guaranteed exist. Their work focuses on proving the robustness of a network itself and therefore requires Lipschitzness of the model. Further, [8] showed that overparameterizztion is needed for learning such smooth (i.e. Lipschitz) functions, which is critical for robustness, On the other hand, we specifically study the relationship between the robustness of the representation function and the prediction head, scenario consistent with the practice of pretraining for transfer learning.
Transfer Learning& & Robustness: The question of how robustness is transferred from pretraining to downstream task has been studied empirically in the prior literature [50
Notably, while (50] observe that robust representations underly robust predictors, [3 also observe that non-
58
28
63
robust representations underly non-robus. predictors. An empirical analysis of adversarial robustness
transfer for the special case of linear probing included in .e.g., [50]. Our work not only derives theoretical results and concrete proof for the empirical phenomena reported in these studies but also provides further empirical investigation.
3 Robustness Bound
Our first theoretical result concerns the discrepancy between the standard and adversarial loss of a neural network f. The considered loss function e is evaluated on a dataset of input samples
X {xi,..., Fn X Rdd and labels Y (y Vn), where n; is sampled from X' with
probability distribution (.)).
For the network, we consider model f Rd R with a final linear layer, i.e. f(x) = fw.o(x) =
W (z)). The function go : Id + R2 can be considered as a representation function for the input parameterized by 0, and W E Reer is the weight matrix of the linear layer. Note that this composition in the literature.
is both standard practice 10 IT and has been extensively studied [56] 52 19
For given data pair (ze, V:), we are interested in the stability of the model's prediction of with respect to (w.rt.) perturbations of the input I; as our robustness measure. Specifically, for E Rd. find the maximum loss (J(; 6); v) under a specified constraint ||o]| < based on norm || ||- Note that the defined measure is commonplace in the literature on adversarial attacks [51 23
We introduce the following notation for the standard loss [(ww,)) and the adversarial loss Ladv (fw.e).
[(Sw,o) = (x)( (((Jw:o(x)) v)),
Caav(fw.d) == Ex(x)
max ((fw,o + 6).4)
[8]|cc
We aim to bound the effect of adversarial attacks in terms of their impact on the representation function, i.e. (zg) go (z, + 2) To obtain such a result for general loss function f. the variation in the loss must be bounded in relation to the proportional change in f(x)). This condition formalized by the concept of Lipschitzness,
Definition 1. A function& É: IRd + R is C-Lipschitz in the norm I on R c R4, if
((()) E(r')) < -lr--r'll..
for any r, r' E RR.
The following theorem is valid for any loss function that satisfies Defn. TwI.t. L1, Le or Loo norm.
Theorem We assume that the loss E(r, y) is -i-Lipcchitz in || for a E (1, 2, for E (Xx with Ci > O and bounded by C2 >> 0. i.e., 0 < E(r, S C2 Vr E S(X). Then, for a subset X independently drawn from (2)), ), the following holds with probability of at least p.
1 Σ
log(p/2)
(fwo) - C(fwo) S La La C
max g6 (2; + +5) (0(xx)ll2 + C2/
-2n
3

---

where
if lla
La (W)
=
Walk
ifl lla
max; (W
ifll
The proof of the theorem can be found in Appx. A.1 Note that the norm used to define the perturbation is arbitrary and only needs to be consistent on both sides of the bound.
First, we apply Hoeffding's s inequality, which requires fixed function fw.o, inde- pendently drawn samples {.i..... In), and an upper bound C2 on the loss function. Next, we bound the average losses utillizigg the Ci-Lipschitzeess in || lla of the loss function. Further, the Cauchy-Schwarz inequality is used to seperate the norms of W and the representation function go-
Proof sketch:
Intuition: Thm. shows that the effect an e-perturbation has on the representation function indeed upper bounds the difference between the standard and adversarial loss. Thus, we characterize the relationship between how vulnerable the linear predictor and the underlying representation is to adversarial examples. Further, Thm. formalizes the impact the weight matrix W has on robustness transfer and thereby provides theoretical argument for regularization in this context.
The statement of Thm. D is formulated in terms of the expectation of the losses particular, the second summand on the right hand-side (RHS) of the bound illustrates the finite- sample approximation error between the expectation and the average over n samples in X. The following lemma provides a version of the theorem for the average difference of the losses over fixed finite sample X. which can directly be concluded from the proof of Thm. iin use this lemma to study our theoretic result in the experiments in Sec. Sec.5511
P(X). In
Appx. Aappx.A.. A.1 We
Lemma (Fipite-sample version of Thm. D. We assume that the loss E(r, y) is C-Lipschitz in | for a E {1,2, 30). for / E f(x) with C 250. Then, the following holds for any dataset (X, Y).
1 th
max (ffre(( + 6), y.) ((fw.o(z.). <8
1g0( + (o()))l [6ll
max
(W) C
Lemma [ shows that the normalized difference between average clean and adversarial loss can be bound by the average robustness the representation function on the RHS, which we denote as the representation function's 's adversarial sensitivity (AS) score on the dataset X.
3.1 Valid Loss Functions
To elaborate on the applicability of the result, we discuss exemplary scenarios in which the assump- tions of our theory fulfilled in practice. As a first example, consider a classification task where we use the softmax cross- cross-entropy loss. While the cross-entropy loss itself is not Lipschitz continuous, since the logarithm is not, the softmax cross-entropy loss is indeed Lipschitz. We define the softmax cross-entropy loss as follows,
exp(f(z)g) Di=1 expl
(ce(f(x), = - log (py(x)) = log
where Py (a:) denotes the y-th entry of the softmax function applied to the final layer, f()), y is the y-th entry of the model output with f(x), = W yrr) and W,, being the y-th row of the matrix W. In Lemma 3 (in Appx. A), we prove that lcE is 2-Lipschitz in |loc.
Further, the theory also valid for regression tasks, where the mean square error (MSE)
(MSE(f(z).v) = [lf(x) - all2
used to measure the loss. This loss function is I-Lipschitz - 1|,, since, by the reverse triangle property, it holds that
[1 - vl12 (_())) - 2aا21 = (1f())) -f ((2))l2.
Additional losses which satisfy Defn. iiclude the logistic, hinge, Huber and quantile loss [3]

---

The graphic shows two classes of datapoints and linear classifier's decision boundary (black). The line My (green) is the optimal margin defining the boundary between a successful or failed adversarial attack. Line M (red) represents the margin derived from Thm. Thm.2wwhich which is an upper bound on Mo- Four datapoints from the blue class are highlighted: non-robust example, is robust but does not fulfill equation ① and 4 is robust classification fulfilling the bound (T).
Figure 2:
wrongly classified, is
Robustness Classification Criterion
In Sec. 3 we presented a theoretical result independent of any particular downstream task and valid for various loss functions. Here, we focus on classification tasks, and, in this context, we provide an additional statement that studies the robustness of linear classifier w.r.t. adversarial attacks. We refer to classifier as robust if there does not exist bounded input perturbation that changes the classifier's 's prediction.
In particular, consider the classifier
c(x) argmax; f(r)s.
where fxx W (z) is the model defined in Sec. The following theorem provides insights on how the robustness of the underlying representation function go impacts the robustness of linear classifier. Specifically, it formally describes how the robustness of the representation, i.e. the AS score on a datapoint æ, determines the set of weight matrices which yield a robust classifier.
Theorem 2. The classifier c(x) robust in x, i.e. c(x + 8) = c(x) for E Rd with ||ō|| < c, ffr E X with c(x) = y it holds that
[f()), - f(x),1
l1 + 5 - -90(x))2 I min
(1)
if
I|W W,ll2
The proof of the theorem can be found in Appx. A.2
Proof sketch: To prove robustness, need to show that f(x + 8)y, f(x 8); if f > f(x),- The proof consists of rewriting this condition and applying the Cauchy-Schwar inequality to separately bound in terms of the difference in W and go.
Intuition: Thm. shows that the effect e-perturbation has on the representation function ge determines the necessary class separation by the linear predictor for robust classification to be ensured. Specifically, the RHS Thm. 2 can be understood logits for I, where the classification margin refers to the difference between the classification score for the true class and maximal classification score from all false classes. Thus, the theorem states that if the margin is larger than the effect of an e-perturbation on the representation function, then classifier is robust. Note that if datapoint does not satisfy the bound, i.e. LHS is greater than the RHS, the robustness of the classifier is ambiguous. In this case, the RHS margin is not the exact separation, or the optimal margin, between robust and non-robust classification. Fig. 2 provides intuition for this interpretation of the result in binary classification scenario.
normalized margin between class
5

---

Empirical Study of Theory
The following experiments serve to validate the theoretical results in practical applications. For
the finite sample version of Thm. T Lemma 1 consider classification task in Sec. 1.I and
regression task in Appx. B.2 In Sec.
5.2
we study the robustness condition established in Thm.
In the following sections, we train a linear predictor on top of pretrained representation function for a specific downstream task, i.e., we perform transfer learning via linear probing. To study robustness transfer, we use robustly pretrained representation functions and examine if the transferred robustness consistent with the theoretical bounds. As our robust representation function, we use models robustly pretrained on CIFAR-I00 [1 (34), and ImageNet [47 [7]. Both models available RobustBench [16] and are adversarially trained using the softmax cross-entropy loss and -attackkkk.
For our downstream tasks, we consider CIFAR-10 [34], Fashion-MNIST [57] and Intel Image"
Further experimental details can be found in Appx. B.
5.1 Classification Cross-Entropy Loss
Here, we empirically evaluate Lemma [ the finite sample version of Thm. [ on classification tasks. As per Sec. 3.1] for softmax cross-entropy, we can apply Lemma | with Lipschitz constant C' - and mapping function Loc (W).
In Tab. 1 we present the left-hand side (LHS), the scaled difference between clean and robust cross-entropy (CE) loss. and the right-hand side (RHS), the representation function's 's AS score, of Lemmal] I Note that, for LHS, the attack on maximizes the CE loss, while for RHS, the attack on ge maximizes the impact the representation function, i.c. (1g)(x;)) ge (z, 8)1l2- See Appx. B.] for further details.
The results in Tab. T validate theoretical predictions in practice, the LHS is indeed always upper bounded by the RHS. Furthermore, we see signs of correlation between the LHS and RHS, i.e. significant changes on side of the bound accompanied by similar changes on the other. Tab, further highlights the connection between our result and weight norm regularization, as it confirms that smaller weight norm, i.e., smaller L,, (W), corresponds to smaller decrease in robustness, i.e., a smaller difference between the losses.
Table 1: For the CIFAR- 100 and ImageNet pretraining datasets, the LHS and RHS, i.e., the AS score, of Lemma π are computed for each of the CIFAR-10, Fashion-MNIST and Intel Image downstream tasks. Additionally, Too (W) and the absolute difference between clean and adversarial CE loss provided.
CIFAR-100
ImageNet
CIFAR-10
F-MNIST
Intel Image
CIFAR-10
F-MNIST
Intel Image
Diff. CE
0.74
0.8
0.47
4.14
9.4
0.14
Los (W)
4.4
4.6
3.1
1.9
2.0
1.4
LHS
0.08
0.09
0.08
1.11
2.33
0.05
AS score
0.98
2.11
1.05
15.7
53.2
2.75
Table The same datasets and metrics are considered in Tab. regularization (A 0.01) is employed for all downstream tasks.
Here, however, L2 weight matrix
CIFAR-100
ImageNet F-MNIST
10
F-MNIST
Intel Image
CIFAR-10
Intel Image
Diff. CE
0.46
0.51
0.38
3.57
7.3
0.14
Lx0 (W)
2.1
2.3
2.2
1.4
1.5
1.2
LHS
0.11
0.11
0.09
0.81
2.43
0.06
AS score
0.98
2.11
1.05
15.7
53.2
2.75

---

5.2 Classification Criterion
Here, we focus on empirically analysing Thm. 2 which specifies an upper bound on the represen- tation's s sensitivity against e-perturbations, where the condition ensures robustness of the classifier against local perturbations for each datapoint.
In Tab. 3. we observe that indeed: condition ① is fulfilled, the classification is always robust, as can be seen from "rob. fulfilled". With that said, we do observe instances of robust classification which do not fulfiil Eq. T which we proved ensures robustness, as can be seen from comparing "robust accuracy" "prop. fulfilled". As with Lemma D we see that tighter results would be necessary for identifying all transferred robustness. Furthermore. as discussed in Sec. since datapoints fulfilling our bound are even further away from the classification boundary than necessary, we also find that fulfilling the bound is predictive of robustness to stronger attacks, as can be seen in Appx. B.3
For the scenario of pretraining on ImageNet and transferring CIFAR-I0 Fashion- -NIST, almost no datapoint from the test set fulfills condition T This observation is accompanied by steep drop from clean accuracy robust accuracy, which suggests this is likely consequence of the large distribution shift between ImageNet and the lower-resolution images of CIFAR-10 (32 x 32) and Reshion-MNISSE (28 x 28) resized ImageNet resolution (256 256). Note that this distribution shift is evident regardless of how the images are scaled before applying the ImageNet representation function, see Appx. B.5 In Tab. T the correlation between distribution shift and loss of robustness is also captured prior to transfer learning by a large AS score, and Appx. B.5 provides further insight into this connection.
Table For the CIFAR- 100 and ImageNet pretraining datasets, we report clean & robust accuracy, the proportion of images fulfilling the theoretical bound, and, of the images fulfilling the theoretical bound, the proportion of images that are robust for each of the CIFAR-IO, Fashion-MNIST and Intel Image downstream tasks.
CIFAR-100
ImageNet
CIFAR-10 10
F-MNIST
Intel Image
CIFAR-10
F-MNIST
Intel Image
clean accuracy
70.9%
84%
78.5%
92.4%
87.2%
91.2%
robust accuracy
42.2%
56.2%
56.8%
15.8%
0%
85.3%
prop. fulfilled
13.4%
10.1% %
26.3%
0.1%
0%
56.2%
rob. fulfilled
100%
100%
100%
100%
100%
6 Adversarial Sensitivity Score
In the prior sections, we empirically validated that our theoretical results and required assumptions are realistic and hold in applications. Note that, in both of our results, the robustness of the representation function determines the robustness that transfers to downstream tasks. Specifically, Lemma 1 is stated using the adversarial sensitivity (AS) score of the representation function on a dataset X.
max 11 + 5) - (o()))ll2 [o
(2)
to bound the difference between clean and adversarial loss. Before specifying downstream task, we can compute the representation function's adversarial sensitivity score on set of unlabeled input images. In the following, we explore how to calibrate expectations on the robustness upon transfer of the representation function via linear probing to a partieular downstream task based on the AS score.
6.1 Robustness Transfer via Linear Probing
Here, we explore how the AS score can be used to provide valuable insights on the transferability of robustness with linear probing. Tab. we find that the AS score on the unlabeled pretraining and downstream datasets are similarly small, then it is reasonable to expect that robustness will transfer
via linear probing. However, if the score on the downstream task is much higher, it is reasonable to expect that robustness will not transfer via linear probing.
7

---

In Tab, 4 we compare the AS score to the relative difference between clean and robust CE loss on each dataset, i.e., (CE robust CE)CE. Indeed, the similar AS scores of CIFAR-100, CIFAR-10. and Intel Image upon pretraining on CIFAR- 100 correspond to a successful transfer of robustness, whereas the dissimilar adversarial sensitivity scores of ImageNet, Fashion-MNIST, and CIFAR-10 upon pretraining on ImageNet reflect the striking loss of robustness upon transfer. One explanation for a larger AS score is a shift in distribution between pretraining and downstream data. This connection confirmed in an additional empirical analysis in Appx. B.S
Table 4:
For the CIFAR-100 100 and ImageNet pretraining datasets, we report the AS score (Eq. (2). clean & robust cross-entropy loss, and the relative difference between said losses for the pretraining dataset, as well as the CIFAR-10, Fashion-MNIST and Intel Image downstream tasks.
Pretraining
Downstream
AS score
clean /robust CE
relative dif.
CIFAR-100
CIFAR-100
1.0
2.71 / 3.27
0.21
CIFAR-10
0.98
0,82 / 1.55
0.9
Fashion-MNIST
2.11
0.45 / 1 1.25
1.8
Intel Image
1.05
0.61 / 1.08
0.76
ImageNet
ImageNet
1.27
1.38 / 1.64
0.19
CIFAR-10
15.7
0,22 / 4.36
18.8
Fashion-MNIST
53.2
0.36 / 9.76
26.1
Intel Image
2.75
0.25 / 0.39
0.57
6.2 Beyond Linear Probing?
In the prior section, we observe a correlation between AS score and the difference between clean and robust CE, i.e., the AS score is predictive of robustness transfer via linear probing. However, linear probing is not the only transfer learning approach, what can we say about alternatives?
In finetuning, the representation function is changed, and thus also the AS score changes post- transfer. Thus, we can expect the AS score computed pre-transfer less predictive of the downstream robustness for finetuning than for linear probing. To gain better understanding, difference between clean/robust accuracy for linear probing (LP), full finetuning (FT) as well as linear probing then finetuning (LP-FT) (35], where LP-FT was proposed combine the benefits of the two approaches. The main results can be found in Tab. 5 and additional details about the experiment are provided in Appx. B.4
study the
Tab. [ shows the performance of each transfer learning method applied to the model robustly pretrained CIFAR-100 or ImageNet. On CIFAR-I00, LP preserves the most robust accuracy, whereas LP-FT performs better on ImageNet. Note that the AS score for LP is highlighted, as this the score downstream performance.
obtain prior to transfer learning, which allows
calibrate expectations
We find no correlation between IPP and FT performance, and do not observe predictive relationship between AS score pre-transfer and robustness transfer via FT. Instead, we observe successful robust- ness transfer via FT and L.-FT for cases with high AS score pre-transfer. This provides us with first indication that the adversarial sensitivity score may point to scenarios where FT preserves more robustness downstream than LP.
7 Discussion
Significance: If the representation does not change at all in response to e-perturbations, it may seem immediately clear that any predictor operating said representation is robust. However, if the representation function is variant to c-perturbations, how much robustness lost as result is dependent the nature of the predictor function under consideration and the distribution shift in the data from pretraining to target task. this work, we consider a linear predictor, and formally compute the exact effect this function can have the test-time robustness.
Comparison of the theorems: The theoretical statement in Thm. Dis closely related to the one in Thm. 2 as more robust predictor to adversarial attacks leads smaller difference in the loss

---

Table 5: Clean accuracy (Acc) and robust accuracy (RAcc) for the three transfer learning approaches using the model robustly pretrained on CIFAR-100.
CIFAR-100 Method
CIFAR-I0
Fashion-MNIST RAcc
Intel Image RAcc
Acc
RAcc
AS
Acc
AS
Acc
AS
FT 94.2%
3.9%
8.93
95.1 %
40%
6.68
90.1% %
7.5%
6.86
LP LP-FT
70.9%
42.2%
0.98
84%
56.2%
2.11
78.5%
56.8% %
1.05
93.9%
1.4%
6.44
95%
36%
4.27
89.2%
27.1%
3.02
ImageNet Method
CIFAR-10 RAcc
Fashion-MNIST RAcc
Intel Image RAcc
Acc
AS
Acc
AS
Acc
AS
FT 98.6%
38.4%
12.6
95.7%
63.5%
9.0
93.9%
85.8%
3.8
LP LP-FT
92.4%
15.8%
15.7
87.2%
0%
53.2
91.2%
85.3%
2.7
98.5%
44.5%
9.8
95.6%
73.9%
5.1
93.9%
85.9%
3.9
between clean and adversarial examples, and vice versa. Thm. T provides a bound on the robustness based on the difference of the loss function, while Thm. 2 only considers the specific form of the classifier and thus information about and restrictions on the used loss function is unnecessary. However, studying the loss function makes Thm. [ valid in much broader set of scenarios, and further provides reasoning on why using weight norm regularization during linear probing improves robustness transfer, see S.1.
The bound in Thm. 2 on the other hand, provides insights on how robustness of a classifier can be results derive robustness guarantees post-transfer from attacks on the representation function pre-transfer.
guaranteed. In particular,
could
Clarification: Note that consider model as a composition of linear layer and representation function, i.e., f(x) = W (ox),, but do not specify how the parameters of f are learned. As our pretraining task and subsequently W is trained w.r.t. downstream task. However, one could also approach the downstream task with the classical strategy of training the model end-to-end instead, i.e., optimizing W and simultaneously in fw.a W gğ- assuming the same architecture as before. The parameters 8, W and 6, W obtained from the two strategies may be very different. For example, the representation function ge may not be useful for linearly separating the dataset into classes 9ý. which optimized specifically for this task. This observation is descrihed as a task mismatch between pretraining and the supervised downstream task.
motivating example, the representation function go is optimized
Our contributed theoretical results demonstrate that we can study robustness by studying the learned functions themselves, irregardless of how said functions were optimized in the first place. With that said, this does not mean that the aforementioned task mismatch does not affect the model's robustness. Thm. Ms bound will loosen if, e.g., ge does not enable linear separation of the dataset into classes, and if the AS score of the representation function increases due to this task mismatch or distribution shift in the data. We provide study on the impact of the distribution shift the data on the AS score in Appx. [B.)
Finally, while the linear probing protocol is standard practice 10 alternative methodology exists for leveraging a representation function for a specified downstream task. For one, we could consider model f composition of nonlinear function h and representation function, i.e. f(z)= h o go (z))), scenario our theoretical statements do not account for. Furthermore, alternative approaches can include updating the parameters of the representation function ga. e.g., full finetuning. As suggested by the results in Sec. 6.2, while a high AS score may not be predictive of robustness transfer for finetuning, may instead indicate when finetuning is required to transfer robustness. Overall. the theoretical dependencies between the robustness of the representations and transfer learning alternatives linear probing present interesting opportunities for future analysis.
8 Conclusion
We prove two theorems which demonstrate that the robustness of a linear predictor on downstream tasks can be bound by the robustness of its underlying representation. We provide a result applicable to various loss functions and downstream tasks, as well as more direct result for classification. Our

---

theoretical insights can be used for analysis when representations are transferred between domains, and sets the stage for carefully studying the robustness of adapting pretrained models.
Acknowledgements
We thank the ICLR 2022 Diversity, Equity & Inclusion Co- Rosanne Liu and Krystal Maughan, for starting the CoSubmitinng Summer (CSS) program, through which this collaboration began. We thank the ML Collective community for the ongoing support and feedback of this research. The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting YS. This work supported by the German Federal Ministry of Education and Research (BMBF)): Tübingen AI Center, FKZ: 011S18039A
References
[1] Sravanti Addepalli, Samyak Jain, al. Efficient and effective augmentation strategy for adversarial training. Advances in Neural Information Processing Systems, 15:1488-150) 2022.
[2] Ehab A AlBadawy, Ashirbani Saha, and Maciej A Mazurowski. Deep learning for segmentation of brain tumors: Impact of iross-institutionall training and testing. Medical physics, 45(3): I 150-11111511 158., 2018
[3] Anders Andreassen, Yasaman Bahri, Behnam Neyshabur, and Rebecca Roelofs. The evolution of out-of- distribution robustness throughout tie-ttning. arXiv preprint S..i::200.118311, 2021.
[4] Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes. Gregoire Mialon, Yuandong Tian, et al. A cookbook of self-supervised learning. arXiv preprint arXiv:2304. 12210, 2023.
[5] Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šindić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning test time. In Joint European conference on machine learning and knowledge discovery in datobases, pages 387-402. Springer, 2013.
[6] Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jcannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, S. Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen A. Creel, Jared Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy. Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren E. Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho. Jenny Hong. Kyle Hsu, Jing Huang. Thomas F. Icard. Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling. Fereshte Khani, O. Khattb, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal I. Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayaman, Benjamin Newman, Allen Nie. Juan Carlos Niebles, Hamed Nilforoshan, J. F. Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou. Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Robert Reich, Hongyu Ren, Frieda Rong, Yusuf H. Roohani, Camilo Ruiz, Jack Ryan, Christopher R e'e, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishna Parasuram Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga. Jiaxuan You, Matci A. Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. On the opportunities and risks of foundation models. ArXiv, 2021.
[7] Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranay Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advancex in neural information processing systems, 13:1877-1901, 2020.
[8] Sébastien Bubeck and Mark Sellke. A universal law of robustness via isoperimetry. Advances in Neural Information Processing Systems, 34, 2021.
Tianlong Chen, Sijia Liu. Shiyu Chang. Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From supervipe pre-training to fine-tuning, In Proceedingy of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 699-708, 2020.
[10] Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning. pages 1597-1607. PMLR, 2020.
10

---

[11] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint Stt.:2003.04227,, 2020.
[12] Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In Proceedings of the IEEF/CVF International Conference Computer Vision, pages 9640-9649, 2021.
[13] Geoffrey Chinot, Guillaume Lecué, and Matthieu Lerasle. Robust statistical learning with lipschitz and convex loss functions. Probability Theory and related fields, 176(3-4):897 -900, 2020.
[14] Kurtland Chua, Qi Lei, and Jason D Lee. How fine-tuning allows for effective meta-learning. Advances in Neural Information Processing Systems, 34, 2021.
[15] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning, pages 1310-1320. PMLR, 2019.
[16] Francesco Croce, Maksym Andriushchenko, Vikash Sehwag. Edoardo Dehenedetti, Nicolas Flammarion, Mung Chiang. Prateek Mittal, and Matthias Hein. Robustbench: a standardized adversarial robustness benchmark. In Thirty-fitth Conference on Neural Information Processing Systems Datasets and Benchmarks Truck (Round 2), 2021.
[17] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. Ieee, 2009.
[18] Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Mma training: Direct input space margin maximization through adversarial training. In International Conference Learning Representations, 2020.
[19] Simon Shaolei Du, Wei Hu, Sham M. Kakade, Jason D. Lee. and Qi Lei. Few-shot learning via learning the representation, provably. In International Conference on Learning Representations, 2021.
[20] Lijie Fan, Sijia Liu, Pin-Yo Chen, Gaoyuan Zhang, and Chuang Gan. When does contrastive learning preserve adversarial robustness from pretraining to finetuning? In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems. 2021.
[21] Stanislav Fort. Adversarial examples for the openai clip in its zero-shot classification regime and their semantic generalization, Jan 2021.
[22] Stanislay Fort. Adversarial vulnerability of powerful near out-of-distribttion detection. arXiv preprint arXiv:2201.07012 2022.
[23] lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. International Conference on Learning Represenations, 2015.
[24] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026-1000 1034, 2015.
[25] Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. Advances in Neural Information Processing Systems, 30, 2017.
[26] Dan Hendryeks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019. URL https://
openreview www.:neeewnondddiieeeeett
[27] Dan Hendrycks. Nicholas Carlini, John Schulman, and Jacob Steinhardt. Unsolved problems in ml safety. arXiv preprint arXiv:2109.13916. 2021.
[28] De-An Huang, Zhiding Yu, and Anima Anandkumar. Transferable unsupervised robust representation learning, 2021.
[29] Neal Jean, Marshall Burke, Michael Xie, W Matthew Davis, David B Lobell, and Stefano Ermon. Combin- ing satellite imagery and machine learning to predict poverty. Science, 353(6301):790- 794, 2016
[30] Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. Advances in Neural Information Processing Systems, 33:16199-16210, 2020.
[31] Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, volume 1, page 2, 2019.
11

---

[32] Minseon Kim, Jihoon Tack, and Sung Ju Hwang. Adversarial self-supervised contrastive learning. Advances in Neural Information Processing Systems, 33:2983-2994, 2020.
[33] Simon Korbllihh, Jonathon Shlens, and Quoc V Le. Do better imagenet models transfer better? In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 2661-2671, 2019.
[34] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[35] Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang, Fine-tuning can distort pretrained features and underperform -ustro-distribution. In International Conference Learning Representations, 2022.
[36] Alexander Levine, Sahil Singla, and Soheil Feizi. Certifiably robust interpretation in deep learning. CoRR. abs/1905.12105, 2019. 2019.
[37] Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. Certified adversarial robustness with additive noise. Advances in neural information processing systems, 32, 2019.
[38] Dongyue Li and Hongyang Zhang. Improved regularization and robustness for fine-tuning in neural networks. Advances in Neural Information Processing Systems, 34:2249--27222, 2021.
[39] Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descont with warm restarts. In Interna- tional Conference on Learning Representations, 2017. URL httppp:://peenreveeeeeeetteeeeee Sxq89Scxx
[40] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018
[41] Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerehner. dsprites: Disentanglement testing sprites datasct.
httpghhttb.coooeceeiiidddsrtttse datasss/, 2017.
[42] Matthew E Peters, Sebastian Ruder, and Noah A Smith. To tune or not to tune? adapting pretrained representations to diverse tasks. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-201)) pages 7-1 2019.
[43] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 3748-8763. PMLR, 2021.
[44] Aditi Raghunathan, Jacob Steinhardt, and Percy Liang, Certified defenses against adversarial examples. In International Conference on Learning Representations. 2018.
[45] Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: python toolbox to benchmark the robustness of machine learning models. Reliable Machine Learning in the Wild Workshop. 34th International Conference on Machine Learning, 2017.
[46] Hadi Salman, Li, Ilya Razenshteyn, Pengchuan Zhang, Huan Zhang, Sehastien Buheck, and Greg Yang. Provably robust deep learning via adversarially trained smoothed classifiers. Advances in Neural Information Processing Systems, 32, 2019.
[47] Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Alcksander Madry. Do adversarially robust imagenet models transfer better? Advances in Neural Information Procexsing Systems, 33:3533- 3545, 2020.
[48] Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust imagenet models transfer better? Advances in Neural Information Processing Systems, 33:3533- 3545, 2020.
[49] Gal Shachaf, Alon Brutzkus, and Amir Globerson. A theoretical analysis of fine-tuning with linear teachers. Advances in Neural Information Processing Systems, 34, 2021.
[50] Ali Shafahi, Parsa Saadatpanah, Chen Zhu, Amin Ghiasi, Christoph Studer, David Jacobs, and Tom Goldstein. Adversarially robust transfer leaming. In International Conference on Learning Representations. 2020.
12

---

[51] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. International Conference Learning Representa- tions, 2014.
[52]
Nilesh Tripuraneni, Michael Jordan, and Chi Jin. On the theory of transfer learning: The importance of task diversity. Advances in Neural Information Processing Systems, 33:7852-7862,, 2020.
[53] Eric Wallace, Shi Feng. Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers attacking and analyzing nip. In Proceedings of the 2019 Conference on Empirical Methods Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019.
[54] Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning, pages 9286-5295, PMLR, 2018.
[55] Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations. 2020.
[56] Sen Wu, Hongyang R. Zhang, and Christopher Ré. Understanding and improving information transfer in multi-task learning. In International Conference on Learning Representations, 2020.
[57] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint urXiv:1708.07744 2017
[58] Yutaro Yamada and Mayu Otami. Does robustness on imagenet transfer to downstream tasks? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 92244, 2022.
[59] Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ R Salakhutdinov, and Kamalika Chaudhuri. A closer look at accuracy robustness. Advances in neural information processing systems, 33:8588 8601, 2020.
[60] Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian. Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. Bdd100k: diverse driving dataset heterogeneous multitask learning. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2636-2645, 2020.
[61] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference 2016. British Machine Vision Association, 2016.
[62] Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic. Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, et al. A large-scale study of representation learning with the visual task adaptation benchmark. arXiv preprint arXiv: 191000077 2019.
[63] Jiaming Zhang, Jitao Sang. Qi Yi, Yunfan Yang. Huiwen Dong, and Jian Yu. Imagenet pre-training also transfers non-robustness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 3436-34444 2023.
13

---

A Proofs and Auxiliary Results
We state two auxiliary results first and subsequently provide the full proofs for Thm. and Thm. 2 in Appx. 1.px..A.Jaandd Appx. [
Lemma 2. For a matrix W Rex and vector g R,, the following bounds hold,
1. [Walla - |Wllllll|..
2. [Wall1 s Ei= [Wlllll9|2.
3. [[Wg gloo I maxx|lW.llelllll
Proof. 1. |W I
(Way)2 < second step follows by Cauchy-Schwand Schwartz inequality.
where the ([Wallllll) ||
2. | gl1 = Eis [Wal & Eis w 1elloll2 = Eis |2lllll where the second step follows again by Souchy-Schwartz inequality.
3. [Wgloe max; Wigl max; Wallllll where the second step follows by Cauchy- Schwartz inequality.
The following Lemma proves a Lipschitz condition for the cross- ssotps loss. Lemma 3. We define the cross-entropy loss with softmax sunctinn for vector TR and class index y E ((1...,c), ie.,
exp(Sx}
{(f.y) = log (py(f)) llog
Ei expifii
Then, the following Lipschitz condition holds for {(f,y). y).
(((a, y) - ((b, v)1 S 2 ||a blloo,
for any vectors a, b E IR and arbitrary class index y cl,
Proof.
explay} 2ラ expla;}
exp(byt
(E(a, E(b, v)1 = log
log
Ejm1 j-n exp(b;}
2 j= explb;}
= log (exp(by - -ayl) log
Ej=1 expla;}
(byy - -ay) - -log Σ exp(a;}
j=l Σi=1 exp(ai) exp(b; -a,t)
< ((b a,) Σ (b(a((b, a,)l
where p)(a)= = exp(as)
Eis exp(ar] denotes the soft-max output of the vector The inequality in the last line follows by Jensen's 's inequality, since log is convex and the soft-max probabilities p, sum up to one.
1(6,- - an) (p - a,)l 16, ayl Eps(a)bb ag s 2 2lb alloor
since both summands in the middle can be bounded by the supremum norm.
14

---

A.1 Proof of Theorem
Proof of Theorem 1
We use Hoeffding's inequality on the independent random variables Di...., Dú, which are defined as
D, = max {(f(, + 6).8;) - {(f(r)). <e
based on independently drawn data points with probability distribution P(X)) where the observations are xt E X' with corresponding labels yt- We can use the upper bound to obtain that D; S C2 and conclude that
2t
Σ Dj mE[D) 21 2t|= 22.exp exp
(lg(pp2) 2n
Σ D. [-D] s <C2/
p.
Thus, with probability of at least 1 holds that
E
Ladv (f) c(5) BCC.,))
max {(f(x: + +),,)) {(f())))) [16|scc
log(p/2)
max E(fxz + 5), e(J(x;)) vi) <|
C22
-2n
We can further bound the first term the right-hand side, since the loss function ((r.y) is C1- Lipschitz || for r E f() and f(z) = (g)
max E(W go + 8), v:) E(W go (x:),, <e
n
((Woo(z; 5,), e(W (wwoooo(ss....mo v.)
1
< C1
W (0 + 6,) - W (1:(llll
where 81,...,5, 8n X with ||6.]| < chosen appropriately. In next step, we apply Lemma and the definition of La in the theorem statement,
G Σ W go + 6,) Wos(z:)lla s (W)
Σ 11ga (x; + +.) go (xx)))l2
The 5, 's might not be the same that maximize the distance between the representations but their effect can be upper bounded by the maximum. Combining the derived bounds and using that go(2; 5.) max go + 8), obtain our result.
ollcc
Laav (5) L(f) = [nav(() - c())
n Σ
max [1go(1; + +8) g0(xx)] C2p log(p/2) [|]| <e
< (W)
A.2 Proof of Theorem 2
Proof of Theorem 2 We want to proof that the classifier c(x) is robust r E X, i.e., holds that
efx + 5) = c(x: for all T&d with ||o|| < e. The classifier predicts class y, i.e., e(ar) = y. the y-th value of the model fz) is larger than all other entries, i.c., f(x), > f(z), for all j * We
15

---

denote the y-th row of the linear layer's matrix W by Wy, y. thus f(z), = W (z).
gg
For an input x with c(x) = y, we want to prove that c(x 5) y for any ó with |o]| < c. ie.
f(x 8)y f(x + o)gr if f f(x))-
holds that
f(z 8)g = f(x +0; + (g) go 6))) (Wggo())-Wjgool))) W)go(z)) + (Wo + + 6) go (x))))
Thus, for f(x + 8)y f(x 6) to be true, it is necessary that (Wy(go(1) -a(( + 6))) + (W (Wggo(>) Wyga + (Wyloo( + + (0)))9(z))) 2 0 Wogo(z) (zgo(1)= 2 (Wy- W5)(o (0 + 6)))
By Cauchy-Schwarce inequality follows that
(W, (go()( ga(x + 5)) = IW,, (go()( - ga + 6))1 - I,, -Will2 l190 + + 90 (z)2)l
Since further W (xx) W (a) = [(x), - [(z) the condition of equation holds for any
j * y if
((z)--((x)) - = 1f(x) - f(x), 2 W W,ll2 (1 + 5) go (z))l22
For c(x + +6) c(x) to be true, the above condition needs to be true for all j af y and this holds if
min [f(r; - f(x),]
max [1 + 6) -go(=))l2- [
ity IW, W,lls
B Additional Experimental Results
We provide additional information on the datasets used and the employed hyperparameters in Appx. B.I A further application of Thm. iis presented in Appx. B.2 where robustness transfer analysed for a regression task. In Appx. B.3 we extend the analysis from Sec. by evaluating
the robustness against stronger attacks. In Appx. B.4, an empirical study compares the robustness transfer depending on the chosen transfer learning method. Finally, in Appx. B.3 we investigate
the effect of shift in distribution between pretraining and downstream task data on the transferred robustness and the AS score.
B.1 Experimental Setup for Section 5
For our experiments, we consider two models robustly pretrained on CIFAR-100 [34] and ImageNet [17. CIFAR-100 consists of 32x32 RGB images with 100 different classes. ImageNet- consists of 256x256 RGB images of 1,000 different classes.
We investigate the performance of robustness transfer on CIFAR- 10 34], Fashion-MNIST [57 and Intel Image Classificationf] The CIFAR- 10 dataset consists of 32x32 RGB images in 10 classes. Intel Image Classification consists of 150x150 RGB images of natural scenes (buildings, forest, glacier, mountain, sea, and street). Fashion- is dataset of Zalando's 's article images, where each example is 28x28 grayscale image, associated with a label from 10 classes.
We use two models from RobustBench 16], namely robust models trained on CIFAR- 100 and ImageNet. For CIFAR-100, we use a WideRessNet-34110 (R:46 parameters) architecture [6] which is adversarially trained using combination of simple and complex augmentations with separate batch normalization layers []. For ImageNet, we use Wede (2x66M parameters)
architecture [61] which is is also adversarially trained [47 40).
For the model pretrained on CIFAR-100, run 20 epochs of linear probing (LP) using batch size of 128, and resize all input images to 32x32. For the model pretrained on ImageNet, we run 10 epochs of LP using batch size of 32, and resize all input images to 256x256. We reduce the batch size to compensate for the increase in input resolution. For LP, we set the learning rate using cosine
16

---

Layer
Layer Type
Activation and Normalization
Convolutional layer (c=6, k=5)
ReLu, Average Pooling, and Batch Norm
2
Convolutional layer (c=16, k=5)
ReLu, Average Pooling, and Batch Norm
3
Linear Layer (out=120)
ReLu and Batch Norm
4
Linear Layer (out=84)
ReLu and Batch Norm
5
Linear Layer (out=84)
ReLu and Batch Norm
"head"
Linear Layer (out=1)
Table 6: The table presents the model architecture used in experiments on MSE loss and the dSprites dataset. The convolutional layers are provided with number of output channels and kernel size, i.e., (c=channels, k=kernel size), where as the linear layers are listed together with the size of their output
vector.
annealing schedule (39], with an initial learning rate of 0.01, and use stochastic gradient descent with momentum of 0.9 on the cross-entropy loss.
To compute LHS and RHS, we must attack the cross-entropy loss on ff as well as the MSE on go. In both cases, we employ -PG attacks implemented in the foolbox package 45]. As hyperparameters for the PPG attack, we choose 20 steps and relative step size of 0.7, since this setting yields highest adversarial sensitivity (AS) scores over all the tasks. The attack strength is set to É = 8/ 5/255 for attacking the CIFAR-100 pretrained model and c = 1/255 for the ImageNet model. Thus, reduce the on ImageNet due to the constant pixel blocks introduced after resizing CIFAR-10 and FaShion-MNIST images from 32x32 and 28x28 to 256x256, respectively.
use the same
CIFAR- 100 used during the RobustBench pre-training and
The experiments were run on single GeForce RTX 3080 GPU and for each downstream task transfer learning or theory evaluation took between 30 minutes to hours.
B.2 Regression MSE
Here, empirically evaluate Lemma on regression tasks using the mean squared error (MSE) as the scoring criterion. As per Sec. 3.1 we can apply Lemma [ with Lipschitz constant C = 1 and
mapping funetion (WW).
We consider the dSprites [4 dataset, which consists of 64x64 grayscale images of 2D shapes procedurally generated from 6 factors of variation (FoVs) which fully specify each sprite image: color, shape, scale, orientation, x-position (posX) and y-position (posY). The dataset consists of 737,280 images, of which of decoding an FoV from an image, i.c., we optimise the network f to minimize MSE with the exact orientation of the sprite. For LP, we consider 4 downstream tasks, each corresponds to decoding unique FoV, while one matches the task used for pre-training.
an 80-20 train-test split. Here, pre-train for the regression task
The model architecture is summarized in Tab. 6 We consider 3 different representation functions, network before training (with default kaiming uniform initialization (24)., after training. and after adversarial training. For the robust model, we utilize Loe attacks implemented in the foolbox package 45 with 20 steps and relative step size of 1.0. The attack strength is set to e = 8/255. The model was trained for 10 epochs with a batch size of 512, learning rate of 1e-3 with cosine annealing schedule [39]. and stochastic gradient descent with a momentum of 0.9.
In Tab. 7. we present the left-hand side (LHS), the rescaled difference between clean and robust cross-entropy (CE) loss, and the right-hand side (RHS), the representation function's 's AS score, of Lemma To obtain the scores for LHS and RHS, we separately attack f as well as the MSE on go- where in both cases the objective uses the MSE. For these attacks, we again utilize -PGD, but also L2-PGD attacks, both based on the foolbox [45] implementation, with 50 steps and a relative step size of 1.0. The attack strength e is set to e = 16/255 for both the attack types. As can be seen, LHS is indeed always upper bounded by RHS. Note that, for each pretraining and attack type combination, the RHS the same over all latent factors, i.e. in each row, because the same images and the same representation function are used.
17

---

Pretraining
Attack Type
Scale
Orientation
posX
pos
Adversarial
-PG
LHS
0.10
0.09
0.02
0.02
RHS
2.29
2.29
2.29
2.29
Standard
-PGD
LHS
3.32
1.20
2.02
2.95
RHS
23.80
23.80
23.80
23.80
Random
LHS 2.33
2.29
2.43
1.84
RHS
29.25
29.25
29.25
29.25
Adversarial
L2-PGD
LHS
0.004
0.003
0.001
0.001
RHS 0.15
0.15
0.15
0.15
Standard
L2-PGD
LHS
0.33
0.22
0.17
0.19
RHS 3.15
3.15
3.15
3.15
Random
L2-PGD
LHS 0.19
0.21
0.14
0.12
RHS 1.69
1.69
1.69
1.69
Table 7: Left-hand side (LHS) and right-hand side (RHS) of Lemma using a model pretrained on the orientation latent factor of dSprites.
B.3 Classification Criterion Attack Strength
Here, we study the hypothesis stated in Sec. 5.2.5.2 that datapoints fulfilling the bound in Thm. are still robust against increased attack strength. We repeat the experiments from Sec. 5.2 for the model pretrained on CIFAR-100, where the attack strength e 8/255, since, for this case, subset of each dataset fulfilled the equation. Additionally, we compute the robust accuracy against stronger attacks employing double and three times the attack strength, i.e., using 2e 16/25 255 and 3e 24/255.
In Tab. [8] we observe that indeed most the datapoints fulfilling the bound in Eq. IT are still robust against the doubled attack strength, which is suggested by the robust accuracy for 2c being larger than the proportion that fulfilled the bound. Most of these datapoints, approximately 50%, are even robust against 3e. Further analysis is needed better understand the observed robust accuracy scores
and to derive a precise connection to the bound from Thm. 2
CIFAR-100
attack strength
CIFAR-10
Fashion-MNIST
Intel Image
clean accuracy
0
70.9%
84%
78.5%
robust accuracy
42.2%
56.2%
56.8%
prop. fulfilled robust accuracy robust accuracy
c
11.3%
4.6%
27.1%
2c
18.4%
28.5%
31.4%
3c
6.6%
13.1%
13.4%
Table 8: For the CIFAR-I00 pretraining dataset, we report clean accuracy, the proportion of images fulfilling the theoretical bound, and robust accuracy for several attack strengths for each of the CIFAR-10, Fashion-MNIST and Intel Image downstream tasks.
B.4 Comparison Study between LP, FT and LP-FT
Here, we compare robustness transfer on classification tasks for three transfer learning techniques, linear probing, full finetuning, and linear probing then finetuning [35). As explained before, linear probing (LP) refers to freezing all pretrained weights and only optimizing the linear classification head. In full finetuning (FT) all pretrained parameters updated during transfer learning. Linear probing then finetuning (LP-FT) combines LP and FT sequentially, as after LP is done, FT is applied to all parameters.
For the model pretrained on CIFAR-100, we choose batch size 128, and run 20 epochs FT, LP and 10 epochs of FT after LP for LP-FT. For the model pretrained on ImageNet, we choose batch size of to compensate for the increase in input resolution to 256x256. We run 10 epochs for FT and LP and 5 epochs for LP-FT. In both settings,
half as many epochs for finetuning
18

---

after linear probing in LP-FT, as in B5]. For all methods we apply a cosine annealing learning rate schedule [3), with an initial learning rate of 0.01 for LP on CIFAR-I00 100, and 0.001 for all other scenarios. We use stochastic gradient descent with momentum of 0.9 on the cross-entropy loss for all experiments.
We evaluate the adversarial robustness on the test set after training by employing the L PPGD attack provided by the foolbox package [45], with 20 steps and a relative step size of 0.7. As the attack strength, we choose 8/2 /55 for the model pretrained on CIFAR-100. Due to the increase in input resolution for the ImageNet model, we apply c 1/255.
Tab. [ in Sec. 6.2 shows the performance of each transfer learning method applied to the model robustly pretrained on CIFAR-10U For all datasets. LP preserves the most robust accuracy, while performing worst on natural examples. FT and LP-FT show similar natural and robust accuracy. Only for the Intel Image dataset do we observe significant difference (10%) in the robustness of L.P-TT and FT. In contrast, when pretraining on ImageNet, LP has the worst clean and robust accuracy. Instead, LP-FT has the highest robust aceuracy for all datasets, while FT achieves the highest clean accuracy, Although FT has the highest clean accuracy for all datasets, the difference between FT and LP-FT is marginal.
Additionally, Tab. provides confirmation that our theory indeed holds for the other transfer learning approaches as well. The difference is that for FT and LP-FT the AS score changes during transfer learning and thus the score computed prior transfer learning is no longer predictive of the transferred robustness. Still, it is an interesting observation that indeed the AS score shrinks during LP-FT when it improves robustness transfer compared to L.P.
CIFAR-100
CIFAR-10 Loo (W)
Fashion-MNIST
Intel Image Loo (W)
Method
ACE
AS
ACE
(W)
AS
ACE
AS
FT
8.7
1.31
8.93
4.49
1.26
6.68
5.62
1.1
6.86
LP
0.73
4.4
0.98
0.8
4.59
2.11
0.47
3.05
1.05
LP-FT
14.1
4.42
6.44
5.98
4.6
4.27
3.06
3.07
3.02
ImageNet Method
CIFAR-10
Fashion-MNIST Loo(W)
Intel Image
ACE
Loo (W)
AS
ACE
AS
ACE
Loo (W)
AS
FT 4.39
1.1
12.6
2.28
1.1
9.0
0.32
0.94
3.8
LP 4.14
1.86
15.7
9.4
2.02
53.2
0.14
1.39
2.7
LP-FT 3.18
1.9
9.8
0.92
2.06
5.1
0.37
1.0
3.9
Table 9: Absolute difference between clean and robust CE loss (ACE) ), To (W) from Thm. and the adversarial sensitivity score (AS) for the three transfer learning approaches using the model robustly pretrained on CIFAR-I00 or ImageNet.
B.5 Distribution Shift Experiments
In this section, we analyse the impact of distribution shift between pretraining and downstream data on the adversarial sensitivity score and thereby on robustness transfer via linear probing. First, consider the distribution shift between CIFAR-10 and ImageNet and, in particular, its dependence on how the smaller CIFAR-10 images are resized before applying the representation function. Second, we employ ideas from prior work [26 to create increasingly corrupted versions of CIFAR-100 to simulate varying shifts in distribution.
In our experiments in Sec. 5 we resize the 32x32 CIFAR-I0 input images to 256x256 for the ImageNet model, and observe a strong distribution shift by severely increased AS score. Here, we evaluate if increasing the image size before applying the ResNet model causes this shift, or it is inherent in the data. In Tab. [0 we repeat our linear probing experiment for the CIFAR- 10 to ImageNet case on differently rescaled input images. The results indicate that reducing the increase in the input image size improves robustness transfer, at the cost of drop in clean performance. However, even for a smaller input image size, the AS score remains increased indicating tangible distribution shift remains.
19

---

imput image
clean /robust CE
clean /robust acc
prop, fulfilled
LHS
RHS
256x256
0.22 4.36
92% 16%
0.1%
1.14
15.7
128x128
0.41 / 1.31
86% / 57%
0.2%
0.28
12.7
64x64
1.04 1.49
64% 48%
0.1%
0.14
8.2
Table 10:
The table contains clean and robust CE loss and accuracy, the proportion of images
fulfilling the theoretical bound in Eq. (T. LHS and RHS of Lemma [ when performing linear
probing from an ImageNet pre-trained model based on differently resized CIFAR-10 images.
In the following, we consider an increasingly corrupted version of CIFAR-100 to simulate distribution shift following prior work [26]. In particular, use the Gaussian noise (Tab. (2) and Zoom blur (Tab. [3) corruption types with increasing severity to simulate an increasing distribution shift. The experimental results from both distortion methods confirm that increasing shift between the data distributions leads to an increase in the AS score and poorer robustness transfer via linear probing.
The different corruption strength used as suggested in [26 and stated in Tab. 11
Approach
C1
C2
C5
Gaussian Noise
0.04.
0.06
0.10
Zoom Blur
[1. 1.06]
[1, 1.11]
[1. 1.26]
Table 11: Definition of the corruption strength considered in our experiments. The Gaussian noise corruption severity is defined by the scale, i.e. the variance of the Gaussian noise distribution. The Zoom blur defined via the interval of zoom factors. For each 0.01 increment in the interval, a zoomed-in image is created, and all zoomed-in in images summed to create blurred image.
Metric
CIFAR-100
CIFAR-100-C1
CIFAR-100-C2
CIFAR- 100-C5
Acc. Robust Acc.
67.3%
66.5%
65.3%
63.6%
46.8%
44.5%
43.3%
40.9%
CE
1.28
1.32
1.37
1.44
robust CE
2.1
2.21
2.3
2.44
difference CE
0.82
0.89
0.93
1.0
Tooo (W)
3.31
3.34
3.36
3.47
LHS (Lem AS score
0.124
0.133
0.138
0.144
0.835
0,892
0.915
0.949
Table 12: Various metrics for transfer learning from model robustly pretrained on CIFAR-10M to an increasingly corrupted CIFAR-100 dataset using Gaussian noise. C1 indicates severity level the corruption is applied.
20

---

Metric
CIFAR-100
CIFAR-100-CI
CIFAR-100-C2
LIFAR-100-CS
Acc.
67.3%
65.4%
65.7%
63.9%
Robust Acc.
46.8%
41.2%
40.3%
35.5%
CE
1.28
1.36
1.36
1.43
robust CE
2.1
2.39
2.42
2.73
difference CE
0.82
1.03
1.06
1.3
Loe (W)
3.31
3.36
3.35
3.42
LHS (Lem AS score
0.124
0.153
0.158
0.19
0.835
1.001
1.031
1.195
Table 13: Various metries for transfer learning from model robustly pretrained on CIFAR-100 to an increasingly corrupted CIFAR- 100 dataset using Zoom blur. C1 indicates severity level 1 of the corruption applied.
21


Reviews:
[{"review_id": "Hlzrk5IoHE", "reviewer": "Reviewer_9x4B", "summary": "The paper considers the problem of robustness transfer from pretraining to downstream finetuning. The authors focus on the setup of linear probing, where downstream tasks are learned with a linear model on top of frozen pretrained features. They however give some insights on how to go beyond linear probing and what happens when the model is fully finetuned. \n\nThe paper is composed of a theory part, followed by an empirical analysis to derive practical insights from the theory. In the first part, the authors prove a bound on the difference between the adversarial and clean loss that is independent of the downstream task. The only conditions required for the this bound to hold are the Lipschitz continuity of the loss with respect to its first argument and the loss boundedness. The authors further highlight a number of loss functions that are valid in the scope of their theory, including the softmax cross entropy loss and the MSE. The authors further provide a criterion for robust classification, providing a minimum margin value that is sufficient for robustness. A notable aspect of their theory is its independence on the pretraining approach.\n\nIn the second part, the authors provide insights on how to calibrate expectation of robustness for downstream tasks. They validate their results with models pretrained on CIFAR100 and ImageNet, adversarially trained with softmax cross entropy and $L_\\inf$ attacks, and consider CIFAR10, FashionMNIST and Intel Image as downstream tasks. ", "questions": "* Most of the settings explored in the paper consider fairly similar pretraining and downstream tasks. What happens when the distribution shift is more severe? (The author did mention this is a interesting direction for further research, but empirical insights could make the paper stronger, even without further theory at this point). \n\n* What happens when pretraining is done without explicit guard against adversarial attacks? \n\n* In table 3, wouldn't it be more interesting to consider correlation between the robustness score and the clean & robust cross-entropy loss instead of their relative difference? For more data points, one could consider different checkpoints during training, that will give different robustness levels. ", "limitations": "The authors did consider how they can improve their work and further developments, without explicitly stating the limitations. This seems reasonable to me given the nature of the work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper is clearly written and well structured. The related works are properly addressed and the papers brings interesting novel insights.  \n* The theoretical results are sound and make sense. The empirical analysis is well designed to validate the theory. (Both could be improved for higher impact, see next sections). \n* The contributions are significant and important to the community, especially in the context of the increasing interest in the pretraining -> fine tuning paradigm. \n* The theory is general enough, and applies in interesting practical settings. ", "weaknesses": "* While the theory is general and applies to different objectives and pretraining approaches, the empirical analysis seems narrow, focusing on image classification with adversarial pretraining. It would be interesting to show how the results generalise to other modalities, and in case where robustness upstream doesn't come from explicit guards against attacks, but from other properties, e.g. scale. \n\n* While the theoretical bound is interesting, there seems to be an opportunity to make it tighter with more (but still reasonable) conditions. Instead of the Hoeffding’s inequality, one can consider other more precise inequalities. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review and feedback. We would like to comment on the weaknesses and questions raised in your review.\n\n**Weaknesses:**\n1. While we agree that empirical validation in additional modalities and with large-scale models would be valuable, we have limited computational resources. With that said, we'd like to emphasize that the theory holds regardless of the source of the pre-trained model's robustness. Nonetheless, we’ll add these experiments as directions for future work in our revision to encourage further investigation by the community. In addition, we’d like to draw your attention to Appendix B.2, where we consider a regression task and evaluate our theory on models pre-trained using standard and adversarial training, as well as randomly initialized models. The results in Table 5 confirm that the theory holds regardless of the pre-training approach, and we also see that the robustness score (RHS) is expectedly much smaller if the model is adversarially trained.  \n2. A more precise inequality that we are aware of is Chernoff’s bound. This bound, however, requires the moment-generating function of the considered random variables and thus requires additional information/assumptions on the distribution of the difference between clean and adversarial loss. Given Hoeffding's inequality is valid regardless of the underlying distribution, we chose to use it for our proof. Please let us know if you had another inequality in mine that would be applicable.\nFinally, it’s worth noting that Hoeffding’s inequality is not the most critical factor prohibiting a tighter bound on the empirical robustness, note that the finite-sample version of Theorem 1 (Lemma 1), which was used for empirical validation, does not use Hoeffding’s inequality. What’s more critical is the Lipschitzness of the loss function and the bounds necessary for separating the effect of the linear probe’s weight matrix from the representation function.\n\n**Questions:**\n1. Thank you very much for bringing this to our attention. We agree that the distribution shift between pre-training and downstream tasks should be discussed in the paper in more detail and we will do so in the revised version. In our experiments, we already consider scenarios with a severe distribution shift, e.g., ImageNet$\\rightarrow$F-MNIST. In Table 3, we can observe that, as a result of the distribution shift, the robustness score increases.  We will make it clear that conducting a theoretical analysis to explain these empirical findings remains open to future work. \nAdditionally, for the revision, we will incorporate a more granular analysis of the effect of distribution shift by performing transfer learning from ImageNet to corrupted ImageNet (ImageNet-C) with varying corruption strengths to obtain a fine-grained measure of the impact distribution shift has on robustness score.\n2. A comparison of our bound with adversarially pre-trained, cleanly pre-trained, and randomly initialized networks is provided in Appendix B.2 in Table 5. The results confirm that the theory is valid independent of the pre-training approach and does not require the pre-trained model to be robust.\n3. Regarding Table 3, while we do provide clean and robust CE loss, we decided to focus on relative difference, since it allows us to compare between tasks with different loss magnitudes, e.g., ImageNet vs Intel Image. Note that our theory considers the absolute difference between clean and robust loss, but relative to the norm of the linear probe’s weight matrix.\nRegarding different checkpoints, while we do not have pre-training checkpoints available, we nevertheless had a closer look at how robustness changes during transfer learning. In Figure 1 in the added pdf, we can see that the norm of the weight matrix (here the maximum of row-wise L2 norms) and the difference between clean and robust CE loss increased during transfer learning, while the LHS of our bound from Lemma 1 remains quite stable, as the numerator and denominator increase almost proportionally. Regarding the RHS, the robustness score remains constant, as it only depends on the pre-trained model processing the downstream dataset. \n"}, {"review_id": "ee1hOOUjoF", "reviewer": "Reviewer_dV4u", "summary": "The paper studies the robust transfer of a pre-trained model, which demonstrate that the robustness of a linear predictor on downstream tasks can be constrained by the robustness of its underlying representation, and theoretically prove a bound on the difference between clean and adversarial loss in terms of the robustness score. The experiment further demonstrates the effectiveness of the ‘robustness score proposed’ in the paper.", "questions": "1 In Figure 1, How do define and distinguish between robust representation and non-robust representation?\n\n2. The experiments in the paper are all transferred from pre-trained models on larger-scale data to small-scale data, whether it is also possible to transfer from small-scale to large-scale data.\n\n3. How does the robustness of a pre-trained model affect the performance of the models on downstream tasks?\n", "limitations": "The pre-trained model is used as model initialization for different tasks. In the paper, it is only demonstrated on the classification task. It is not clear whether the proposed theory and method can be used for other tasks (e.g., object detection).", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written and the proposed theories are clearly formulated.\n\n2. The paper provides theoretical guarantees for the robust transfer of pre-trained models.\n\n3.  The experiment further demonstrates the effectiveness of the proposed theories.", "weaknesses": "See questions", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. In the following, we address all your questions and concerns.\n\n**Questions:**\n1. In Figure 1, we distinguish between robust and non-robust representations based on their response to adversarial perturbations. We consider two separate representation functions in green and red. Both functions can map the same input image to slightly different feature representations. The robust representation (red) is less affected by adversarial perturbations, meaning that the model's feature representation remains stable and close to the representation of the original image even in the presence of small perturbations to the input data. On the other hand, the non-robust representation (green) results in significant changes to the model's representation vector when subjected to adversarial perturbations, indicating vulnerability to such attacks. This difference in adversarial sensitivity between robust and non-robust representation functions affects how robust, or non-robust, a linear classifier trained upon this feature representation space is, which is the focus of our study. \n2. While we indeed only considered the large-scale to small-scale case, as this is the standard scenario where transfer learning is applied, our theory still applies to the small-scale to large-scale case. To validate this empirically, we provide additional results in Table 2 in the added pdf. Here, we study transferring from CIFAR-10 (small-scale dataset) to CIFAR-100 (a large-scale dataset). The results again confirm our theoretical bound.\n3. While we focused on robustness transfer in particular, we agree that the link between pre-trained robustness and performance is a relevant question as well. In Table 3, we can see that there can be a trade-off between robust and clean performance by comparing the downstream performance on the CIFAR-10 and FashionMNIST tasks for the two different pre-trained models. In Table 3 in the extra pdf, we provide an additional experiment comparing robustness transfer and performance depending on two adversarial pre-training schemes. From these results, one can also conclude that in some cases the robustness can also be improved without sacrificing performance. In all, our experiments suggest directions to explore, and we will highlight said directions in the revision. \n\n\n**Limitations:**\\\nIn AppendixB.2, we provide experiments similar to Section 5.1 but considering a regression task and the MSE as the loss function. \nFurther, we describe various loss functions for which our theory holds in Section 3.1. Thus, for all tasks which are fine-tuned using one of these loss functions our bound holds. Object detection is, indeed, an interesting special case as well. If considering a loss function that is a weighted sum of CE loss for the classification task and MSE for the localization task, given our empirical validation for classification and regression, our theory is also applicable to object detection applications. \n"}, {"review_id": "gPelDcEjML", "reviewer": "Reviewer_2ud3", "summary": "This paper study how the robustness properties of a given representation might be transferred to a linear probe which was trained on this representation.  Since the linear probe is a simple multiplication between the weights parameters and the representation vector, the authors were able to demonstrate that by using the representation we can bound the downstream linear probe robustness performances. To do so, they introduced a robustness score, which measure the maximal distance in the representation space that can be reached through adversarial perturbations. If this distance if lower than the linear classification margin, then we can expect the linear probe to be robust to these adversarial pertubations. ", "questions": "- In Table 2, you wrote that the proportion of fulfilled data points (condition 1) is zero for CIFAR10 and F-MNIST when pretrained on ImageNet. However, you wrote that the accuracy of fulfilled data points is 100%. I don't understand how the accuracy can be 100% if you have zero point in this category (If this is the case, it should be undefined, not 100%).\n- Why the results for LP in Table 8 are different from Table 2 ?\n", "limitations": "The authors correctly addressed the limitations of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper is well written and easy to follow.", "weaknesses": "- There isn't a discussion about how regularizing the weight matrix of the linear probe might impact the downstream robustness. It's only one hypothesis but if within a representation vector, there might be some dimensions that are changing much more than some others, applying a sparse matrix multiplication might removed those dimensions and thus offer better robustness downstream. \n\n- Some confusion about the name of robustness score. If this score is measuring how robust a representation is, then we could expect that a higher score will imply a better robustness while a lower score will offer weaker robustness.  However, in this instance, it doesn't seem to be the case, since a smaller robustness score imply better robustness. So I would advise the authors to rename their score of to change their formula such that higher robustness score imply better robustness.\n\n- Some issues and limitations with the experimental setup. One hypothesis given by the authors for the results in Table 2 for which the proportion of fulfilled data points (condition 1) is zero, is that the resolution gap between the pretrained data and dowstream data is too big. However, since the authors are using a convnet for their experiment (wide-resnet), they should be able to change the resolution. So instead of resizing to 256x256, the authors could have only rescale to 64x64 (or 128x128) and compute their results from here. So I would have expected a better empirical evaluation of this discrepancy in the results. I also would have expected some discussion and study around the role of hyper-parameters when training the linear probe on condition 1. Lastly, the authors use adversarial pretrained model for their experiments, however, I would have expected to see some results on baseline trained without adversarial training. \n\n- Inconsistency between some experiments. In Table 2, for the pretrained ImageNet model, the cifar10 accuracy is 92% while the robustness accuracy is 0%. However when looking at Table 8 the cifar10 accuracy is still 92% but the robustness accuracy is 48%. So if both table show linear probing accuracy, why is there those differences ?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough feedback. In the following, we would like to comment on the weaknesses and questions raised in your review.\n\n**Weaknesses:**\n1. Thank you for highlighting this topic. We agree that the impact of regularizing the weight matrix of the linear probe on downstream robustness should be discussed, and we will include it in the revision. According to our theoretical bound, the smaller the norm of the weight matrix, the more robustness transfer would be improved. For empirical affirmation, we provide new experimental results in Table 1 in the added pdf. Here, we repeat the experiments from Table 1 in our article, where CIFAR-100 is the pre-training set, but with weight matrix regularization during optimization (\\lambda=0.01). The results indicate that regularization leads to a reduced weight norm and a reduced difference between clean and robust loss, i.e., better robustness transfer. The robustness score is not affected by these changes, as it only depends on the pre-trained feature representation of the downstream dataset. Interestingly, the left-hand-side (LHS) of our Lemma 1 bound remains relatively stable as well. \n2. Thanks for highlighting this. Based on your feedback we will rename the score in the revised paper to “adversarial sensitivity score” (AS score) to make the name more in line with the concept. For this rebuttal, however, we will continue using the name robustness score to avoid confusion. \n3. As you have observed correctly, we suggested the resolution gap between the datasets, e.g., ImageNet and CIFAR-10, as a possible reason for the poor robustness transfer but did not study it empirically. We have conducted such an analysis by considering different input image sizes, which are provided in Table 4 of the added pdf. The results show that the robustness transfer improves (smaller LHS and RHS of Lemma 1) if the input images are only resized to 64x64 instead of 256x256. At the same time, the clean performance drops when lowering the input resolution.\nRegarding hyperparameters for linear probing, we tuned on the validation loss, where the considered loss function is the same as used during training, i.e., MSE or CE loss. To clarify, the linear probe was trained on the corresponding loss using standard training (no adversarial examples involved), and condition (1) is not involved during training. The point of condition (1) is to evaluate not just if the predictions on our test examples are robust but if they are sufficiently separated to fulfill the theoretical bound. If we missed the point of your comment, it would be great if you could clarify what you meant. \nRegarding a baseline trained without adversarial training, in Appendix B.2, we consider such a baseline comparison. We perform transfer learning for a regression task and compute LHS and RHS of our theorem for pre-trained models, which are obtained from standard training, adversarial training, or random initialization.\n4. The difference you have observed between the robust accuracy scores in Table 2 and Table 8, is due to different attack parameters in the two experiment setups. In both cases, we computed the adversarial examples using the Linf-PGD attack implemented in the foolbox package with attack strength ϵ = 8/255 and 20 steps. However, the relative step sizes vary between the two tables, 1/30 was used in Appendix B.4, while 0.7 was used in Section 5.2. This is because, for the experiments in Section 5.2, we performed a hyperparameter search on the attack parameters, but were not able to re-run the experiments in Appendix B.4 again. This is briefly mentioned in Appendix B.4. \nFor the revision, we will re-evaluate B.4 to provide consistent robustness results throughout the experiments. With that said, we still want to note that the conclusions drawn from the experiments are still valid.\n\n**Questions:**\n1. Thank you for bringing this to our attention, this is indeed a mistake on our side and we will correct it in the final version of the paper.\n2. Please refer to our response to weakness 4.\n"}, {"review_id": "CQfiDeLIDh", "reviewer": "Reviewer_MCBm", "summary": "This paper studies the question of how the robustness of pretrained representations transfers to downstream tasks. Under the linear probing setting (i.e., the downstream classifier $f$ is linearly related to the pretrained representations $g_{\\theta}$), they prove that the gap between the standard and robust risk of $f$ can be bounded by the summand of the \"perturbation on representations\" plus a constant (w.r.t. $g_{\\theta}$ and $x$). By rearranging terms, they obtain the *robustness score* of a dataset, which can be used to estimate how well the robustness of pretraining would transfer to downstream tasks. This paper also provides an upper bound for the robustness score. Experiments on several datasets are conducted to validate their theoretical results.", "questions": "**Section 2 (Related Work) is hard to understand.** Could the author please explain, in more detail, the relationship between the related works and yours? My major questions include:\n\n1. In Line 55-59, the author claims that ref. [54, 43, 15, 36, 35, 45], instead of ref. [25], provides practical guarantees on the detection of adversarial examples. By saying \"instead\", the author seems to be expressing the idea that ref. [25] theoretically guarantees the detection of adversarial examples. However, ref. [25] does not seem to provide such results.\n2. What is the relationship between adversarial training, self-supervised contrastive learning, and the present work?\n3. I don't think the present work is \"in contrast with\" ref. [59] and [8], as is claimed in Line 69-71. In my opinion, ref. [59], [8] and the present work are studying difference topics.\n4. As is mentioned in Line 75-76 and 47-48, linear probing seems to be closely related to the main contributions of this paper. In fact, the theoretical results in Sections 3 and 4 are all based on the linear probing (i.e., $f(x) = W g_{\\theta}(x)$). However, I have found no related work on linear probing in this paper. It would make this paper more comprehensible if the author briefly introduce the basic setting and related work about linear probing.\n", "limitations": "No.", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. This paper provides theoretical results that upper bounds the gap between robust and standard loss under the setting that they concern. Besider, the experimental results perfectly fit their theory. \n2. They propose the concept of robustness score, which is easy to calculate and is useful in estimating the robustness of the downstream classifiers.\n", "weaknesses": "+ The main theoretical result (i.e., Theorems 1) is essentially a robust risk bound for linear classifiers, which is, technically speaking, quite a simple result. Previous works have proved similar robust risk bounds for 2-layer ReLU networks and even more generally, for VC-class functions. I understand that Theorem 1 indeed provides a robust risk bound for downstream tasks under the linear probing setting, and the analyse for linear classifiers cannot be extended to other function classes due to linear probing. However, I still think the present paper's theoretical contribution is not enough. \n+ **The current version of this paper looks like a work in progress.** As is stated above, the theoretical contribution of this paper is not enough. Maybe the author should complete the future work that is mentioned in Section 7.\n+ The writing of this paper might need further polishing. For example, Figure 1 does not illustrates the intuition behind their theory (something related to your proofs is expected). The related work is also hard to understand, making the readers hard to evaluate the contribution of this paper.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We want to thank you for your feedback and in the following we address the weaknesses and questions raised in your review.\n\n**Weaknesses:**\n1. We are aware of results on robust risk bounds for linear classifiers or VC-class functions, which, roughly speaking, provide insights into the learnability of a robust classifier based on the function class. However, we are not aware of a theoretical result that provides similar insights into robustness transfer or could be directly applied to the scenario that we consider. Our result proves for any fixed function/network how robustness can be transferred through a linear layer and a loss function, which makes it directly applicable to many transfer learning use cases. In particular, our results are valid for several loss functions, i.e., several tasks can be considered, not just classification, and also different adversarial attacks are allowed, e.g. L_1, L_2, or L_inf attacks. \nIn all, we believe that the result is novel and provides a relevant contribution to studying robustness in transfer learning. In particular, if the result is seen as “simple”, that can be advantageous for facilitating the use of our bound in applications. With that said, if we are missing some relevant theoretical contributions to this topic, it would be great if you could share the corresponding references.\n2. Following your suggestion, we reviewed again the future work that we have suggested in section 7, and we agree that some of the mentioned topics should be included in the paper, and we will do so in the revision. In particular, we will incorporate the results and discussion mentioned below.\n    - **a)** Regarding “deriving conditions for optimal weights of the linear classifier based on the theorem’s bound”, according to our theoretical bound, regularizing the weight matrix of the linear classifier is optimal for robustness transfer, as the smaller the norm of the weight matrix, the more robustness transfer will be improved. For empirical affirmation, we provide new experimental results in Table 1 in the added pdf. In this new experiment, we repeat the experiments from Table 1 from our article, where CIFAR-100 is the pre-training set, but with regularization added during optimization ($\\lambda=0.01$). The results indicate that reducing the weight norm does indeed reduce the difference between clean and robust loss, i.e., better robustness transfer. \n    - **b)** Regarding “the impact of the distribution shift in the data on the robustness score”, we will add a discussion on our experiments that already include scenarios with a strong distribution shift. For example, there is a severe distribution shift when doing transfer learning from ImageNet to F-MNIST. In this case, we can observe that the robustness score increases compared to the case with no distribution shift, i.e., ImageNet -> ImageNet, see Table 3 for the robustness scores. Thus, our empirical results imply that a larger distribution shift between pre-training and downstream tasks does lead to a larger robustness score. Additionally, we will incorporate a more granular analysis of the effect of distribution shift by performing transfer learning from ImageNet to ImageNet corrupted with variable corruption strength to obtain a fine-grained measure of the effect distribution shift has on robustness score. With that said, a theoretical analysis of the connection between robustness transfer and distribution shift remains open. \n     - **c)** Regarding “theoretical dependencies between the robustness of the representations and transfer learning alternatives to linear probing”, studying the relation between distribution shift and robustness transfer theoretically may yield insights, given our aforementioned experiments in Section 5 do suggest a connection between distribution shift and the most robustness-preserving transfer learning method. \n3. While we agree that Figure 1 isn’t specific to our proofs, the illustration aims to show how the difference between robust and non-robust representation function, i.e., their sensitivity wrt. adversarial perturbation can affect the robustness of a linear classifier trained upon this feature representation space. The graphic intends to provide a general intuition regarding the overall subject of study than a precise insight into our theoretical contribution. While Figure 2 provides intuition for understanding one of our theoretical results, we will consider a more proof-related figure for the revision. \\\nThank you for your detailed feedback on our related work section. We answered your questions in the following and will incorporate clearer descriptions and more details on the relation to our work when revising this section.\n\n**Questions:**\n1. We correct the writing to make this comparison more clear. Ref. [25], in Theorem 2.1, provides formal robustness guarantees for a classifier f, by leveraging a connection to Lipschitzness, but it indeed is not directly comparable to the approaches in Ref. [54, 43, 15, 36, 35, 45].\n2. These sentences were introducing the landscape of adversarial robustness and were not intended to be directly linked to our contribution. However, we understand why a relationship was expected, and will consider this when revising this section.\n3. Thanks for bringing this to our attention, we will correct the language.\n4. Linear probing is studied empirically in [34] in the context of out-of-domain performance, and in [49] in the context of adversarial robustness. Our work provides theoretical justification for some of their empirical observations on linear probing. As you pointed out correctly, we should have stated this connection to our work more precisely and will provide a self-contained section to introduce linear probing to clarify where our contributions sit in that area of literature."}]

Author rebuttal:
We want to thank the reviewers for their thorough reviews and meaningful suggestions which will help us improve our paper.
We are also happy to hear that our article is considered “*well written and easy to follow*” (R-*2ud3*), that our contribution is “*significant and important to the community*” (R-*9x4B*) and the experiments demonstrate “*the effectiveness of the proposed theories*” (R-*dV4u*) and it provides an applicable “*robustness score, which is easy to calculate and is useful in estimating the robustness of the downstream classifiers*” (R-*MCBm*).

In the following, we state the primary takeaways from the reviews that will be incorporated into the revision:
- We would like to highlight here that we did also consider pretrained models that were not adversarially trained and scenarios different from classification. The corresponding regression experiments can be found in Appendix B.2.
- As briefly mentioned in Section 5.2, our experiments do consider the relationship between distribution shift and robustness transfer. A more detailed discussion on this topic with respect to the robustness score is relevant, as highlighted by reviewer *9x4B*, and will be added to the revision. This new discussion will include a more granular analysis of the effect of distribution shift by performing transfer learning from ImageNet to corrupted ImageNet (ImageNet-C) with varying corruption strengths.
- Following the suggestion by reviewer *2ud3*, we will add a separate discussion about how our theory provides insights into the effectiveness of weight norm regularization on robustness transfer. The empirical study in Table 1 of the added pdf confirms the theoretical statement that a smaller weight norm improves robustness transfer.

Meta-review: The recommendation is based on the reviewers' comments, the area chair's personal evaluation, and the post-rebuttal discussion. 

This paper studies the transferability of adversarial robustness via linear probing on pretrained models, which is a timely and important topic. All reviewers find the studied setting novel and the results provide new insights. The authors’ rebuttal has successfully addressed the major concerns of reviewers.

During the discussion, one reviewer pointed out that if this submission is positioned as a theory-oriented paper, the technical depth may need to be further sharpened; while if positioned as an experiment-driven paper, the empirical results may need to be expanded beyond supporting the theoretical statements. However, the reviewer also agrees that this submission has provided some new insights. Other reviewers are in favor of accepting this submission. Therefore, considering the joint contributions to new theoretical analysis and experimental results are sufficiently significant, I recommend acceptance of this submission. I also expect the authors to include the new results and suggested changes during the rebuttal phase to the final version.
Decision: Accept (poster)