Title: On Transfer of Adversarial Robustness from Pretraining to Downstream Tasks
Venue: NeurIPS 2023
Accepted: yes
Keywords: Machine Learning Theory

Transfer Learning

Adversarial Robustness
Abstract: As large-scale training regimes have gained popularity, the use of pretrained models for downstream tasks has become common practice in machine learning. While pretraining has been shown to enhance the performance of models in practice, the transfer of robustness properties from pretraining to downstream tasks remains poorly understood. In this study, we demonstrate that the robustness of a linear predictor on downstream tasks can be constrained by the robustness of its underlying representation, regardless of the protocol used for pretraining. We prove (i) a bound on the loss that holds independent of any downstream task, as well as (ii) a criterion for robust classification in particular. We validate our theoretical results in practical applications, show how our results can be used for calibrating expectations of downstream robustness, and when our results are useful for optimal transfer learning. Taken together, our results offer an initial step towards characterizing the requirements of the representation function for reliable post-adaptation performance.

On Transfer of Adversarial Robustness from Pretraining to Downstream Tasks

Abstract

As large-scale training regimes have gained popularity, the use of pretrained models for downstream tasks has become common practice in machine learning. While pretraining has been shown to enhance the performance of models in practice, the transfer of robustness properties from pretraining to downstream tasks remains poorly understood. In this study, we demonstrate that the robustness of a linear predictor on downstream tasks can be constrained by the robustness of its underlying representation, regardless of the protocol used for pretraining. We prove (i) a bound on the loss that holds independent of any downstream task, as well as (ii) a criterion for robust classification in particular. We validate our theoretical results in practical applications, show how our results can be used for calibrating expectations of downstream robustness, and when our results are useful for optimal transfer learning. Taken together, our results offer an initial step towards characterizing the requirements of the representation function for reliable post-adaptation performance. 1

1 Introduction

Recently, there has been a rise of models (BERT [31], GPT-3 [7], and CLIP [43]) trained on large-scale datasets which have found application to a variety of downstream tasks [6]. Such pretrained representations enable training a predictor, e.g., a linear 'head', for substantially increased performance in comparison to training from scratch [10, 11], particularly when training data is scarce.

Generalizing to circumstances unseen in the training distribution is significant for real-world applications [2, 60, 29], and is crucial for the larger goal of robustness [27], where the aim is to build systems that can handle extreme, unusual, or adversarial situations. In this context, adversarial attacks [51, 5], i.e., inducing model failure via minimal perturbations to the input, provide a stress test for predictors, and have remained an evaluation framework where models trained on large-scale data have consistently struggled [53, 21, 22], particularly when, in training, adversarials are not directly accounted for [23, 40].

With that said, theoretical evidence [8] has been put forth that posits overparameterization is required for adversarial robustness, which is notable given its ubiquity in training on large-scale datasets. Thus, given the costs associated with large-scale deep learning, we are motivated to study how adversarial robustness transfers from pretraining to downstream tasks.

training data

representationspace

adversarial examples

Tadv

+E·

robust representation function

standard representation function

linearly separated classes

For this, considering the representation function to be the penultimate layer of the downstream predictor, we contribute theoretical results which bound the robustness of the downstream predictor by the robustness of the representation function. Fig. 1 illustrates the intuition behind our theory, and shows that a linear classifier's robustness is dependent on the robustness of the pretrained representation. Our theoretical results presented in Sec. 3 and Sec. 4 are empirically studied in Sec. 5, where we validate the results in practice, and in Sec. 6 to show how the theory can be used for calibrating expectations of the downstream robustness in a self-supervised manner [4]. In all, our theoretical results show how the quality of the underlying representation function affects robustness downstream, providing a lens on what can be expected upon deployment. We present the following contributions:

we theoretically study the relationship between the robustness of the representation function, referred to as adversarial sensitivity (AS) score , and the prediction head. In Thm. 1, we prove a bound on the difference between clean and adversarial loss in terms of the AS score. In Thm. 2, we prove a sufficient condition for robust classification again conditioned on the robustness of the representation function.

we evaluate the applicability and consistency of our theory in applications in Sec. 5.1 and Sec. 5.2. Furthermore, in Sec. 6, we illustrate the utility of the AS score to predict expected robustness transfer via linear probing.

in Sec. 6.2, we study how linear probing compares to alternatives, like full finetuning, in robustness transfer for robust performance downstream.

2 Related Work

Transfer Learning: While empirical analyses of pretraining abound [33, 12, 62, 42, 3, 48], a number of theoretical analyses have also been contributed [56, 52, 19, 14, 49, 35, 38]. The majority of the theoretical studies aim to understand the benefit of transfer learning with respect to improved performance and data requirements compared to training from scratch. Therefore, the referenced studies include results on sample complexity [52, 19, 14, 49] and generalization bounds [38] for finetuning and linear probing in various scenarios. In this work, we primarily focus on the linear probing approach and perform a theoretical analysis to investigate its efficiency in transferring adversarial robustness.

Robustness: There is a direct connection between the mathematical formulation of adversarial robustness and the definition of a local Lipschitz constant of the network [25], but computing this quantity is intractable for general neural networks. However, there exists a line of work providing more practical approaches in certifiable robustness [54, 44, 15, 37, 36, 46], which can provide guarantees on the detection of adversarial examples. Relative to empirical approaches [23, 40, 18, 48, 55], certifiable methods have comparatively struggled to scale to datasets such as ImageNet [17]. Furthermore, likely due to computational expense, adversarial training has not yet been adopted by models trained on large-scale datasets [31, 7, 43]. Still, a number of works have incorporated adversarial training into

pretraining, in particular, self-supervised contrastive learning [32, 30, 9, 20]. With that said, our work is focused on the analysis of robustness irrespective of the pretraining approach.

Regarding analysis, [59] studied the trade-off between accuracy and robustness, and showed that, under a set of separation conditions on the data, a robust classifier is guaranteed to exist. Their work focuses on proving the robustness of a network itself and therefore requires Lipschitzness of the model. Further, [8] showed that overparameterization is needed for learning such smooth (i.e. Lipschitz) functions, which is critical for robustness. On the other hand, we specifically study the relationship between the robustness of the representation function and the prediction head, a scenario consistent with the practice of pretraining for transfer learning.

Transfer Learning & Robustness: The question of how robustness is transferred from pretraining to downstream task has been studied empirically in the prior literature [50, 58, 28, 63]. Notably, while [50] observe that robust representations underly robust predictors, [63] also observe that nonrobust representations underly non-robust predictors. An empirical analysis of adversarial robustness transfer for the special case of linear probing is included in ,e.g., [50]. Our work not only derives theoretical results and concrete proof for the empirical phenomena reported in these studies but also provides further empirical investigation.

3 Robustness Bound

Our first theoretical result concerns the discrepancy between the standard and adversarial loss of a neural network f . The considered loss function ℓ is evaluated on a dataset of input samples X = { x 1 , . . . , x n } ⊂ X ⊂ R d and labels Y = { y 1 , . . . , y n } , where x i is sampled from X with probability distribution P ( X ) .

For the network, we consider a model f : R d → R c with a final linear layer, i.e. f ( x ) = f W,θ ( x ) = Wg θ ( x ) . The function g θ : R d → R r can be considered as a representation function for the input parameterized by θ , and W ∈ R c × r is the weight matrix of the linear layer. Note that this composition is both standard practice [10, 11], and has been extensively studied [56, 52, 19] in the literature.

For a given data pair ( x i , y i ) , we are interested in the stability of the model's prediction of y i with respect to (w.r.t.) perturbations of the input x i as our robustness measure. Specifically, for δ ∈ R d , we find the maximum loss ℓ ( f ( x i + δ ) , y ) under a specified constraint ∥ δ ∥ < ϵ based on some norm ∥ · ∥ . Note that the defined measure is commonplace in the literature on adversarial attacks [51, 23].

Weintroduce the following notation for the standard loss L ( f W,θ ) and the adversarial loss L adv ( f W,θ ) ,

We aim to bound the effect of adversarial attacks in terms of their impact on the representation function, i.e. ∥ g θ ( x i ) -g θ ( x i + δ ) ∥ 2 . To obtain such a result for a general loss function ℓ , the variation in the loss must be bounded in relation to the proportional change in f ( x i ) . This condition is formalized by the concept of Lipschitzness,

Definition 1. A function ℓ : R d → R is C -Lipschitz in the norm ∥ · ∥ α on R⊂ R d , if

for any r, r ′ ∈ R .

The following theorem is valid for any loss function ℓ that satisfies Defn. 1 w.r.t. L 1 , L 2 or L ∞ norm.

Theorem 1. We assume that the loss ℓ ( r, y ) is C 1 -Lipschitz in ∥·∥ α , for α ∈ { 1 , 2 , ∞} , for r ∈ f ( X ) with C 1 > 0 and bounded by C 2 > 0 , i.e., 0 ≤ ℓ ( r, y ) ≤ C 2 ∀ r ∈ f ( X ) . Then, for a subset X independently drawn from P ( X ) , the following holds with probability of at least 1 -ρ ,

where

The proof of the theorem can be found in Appx. A.1. Note that the norm used to define the perturbation δ is arbitrary and only needs to be consistent on both sides of the bound.

Proof sketch: First, we apply Hoeffding's inequality, which requires a fixed function f W,θ , independently drawn samples { x 1 , . . . , x n } , and an upper bound C 2 on the loss function. Next, we bound the average losses utilizing the C 1 -Lipschitzness in ∥ · ∥ α of the loss function. Further, the Cauchy-Schwarz inequality is used to seperate the norms of W and the representation function g θ .

Intuition: Thm. 1 shows that the effect an ϵ -perturbation has on the representation function g θ indeed upper bounds the difference between the standard and adversarial loss. Thus, we characterize the relationship between how vulnerable the linear predictor and the underlying representation is to adversarial examples. Further, Thm. 1 formalizes the impact the weight matrix W has on robustness transfer and thereby provides a theoretical argument for regularization in this context.

The statement of Thm. 1 is formulated in terms of the expectation of the losses over P ( X ) . In particular, the second summand on the right hand-side (RHS) of the bound illustrates the finitesample approximation error between the expectation and the average over n samples in X . The following lemma provides a version of the theorem for the average difference of the losses over a fixed finite sample X , which can directly be concluded from the proof of Thm. 1 in Appx. A.1. We use this lemma to study our theoretic result in the experiments in Sec. 5.1.

Lemma 1 (Finite-sample version of Thm. 1) . We assume that the loss ℓ ( r, y ) is C -Lipschitz in ∥ · ∥ α , for α ∈ { 1 , 2 , ∞} , for r ∈ f ( X ) with C > 0 . Then, the following holds for any dataset ( X , Y ) ,

Lemma 1 shows that the normalized difference between average clean and adversarial loss can be bound by the average robustness of the representation function on the RHS, which we denote as the representation function's adversarial sensitivity (AS) score on the dataset X .

3.1 Valid Loss Functions

To elaborate on the applicability of the result, we discuss exemplary scenarios in which the assumptions of our theory are fulfilled in practice. As a first example, we consider a classification task where we use the softmax cross-entropy loss. While the cross-entropy loss itself is not Lipschitz continuous, since the logarithm is not, the softmax cross-entropy loss is indeed Lipschitz. We define the softmax cross-entropy loss as follows,

where p y ( x ) denotes the y-th entry of the softmax function applied to the final layer, f ( x ) y is the y-th entry of the model output with f ( x ) y = W y g ( x ) and W y being the y-th row of the matrix W . In Lemma 3 (in Appx. A), we prove that ℓ CE is 2-Lipschitz in ∥ · ∥ ∞ .

Further, the theory is also valid for regression tasks, where the mean square error (MSE)

is used to measure the loss. This loss function is 1-Lipschitz in ∥ · ∥ 2 , since, by the reverse triangle property, it holds that

Additional losses which satisfy Defn. 1 include the logistic, hinge, Huber and quantile loss [13].

M

M

0

3

1

4 Robustness Classification Criterion

In Sec. 3, we presented a theoretical result independent of any particular downstream task and valid for various loss functions. Here, we focus on classification tasks, and, in this context, we provide an additional statement that studies the robustness of a linear classifier w.r.t. adversarial attacks. We refer to a classifier as robust if there does not exist a bounded input perturbation that changes the classifier's prediction.

In particular, we consider the classifier

where f ( x ) = Wg θ ( x ) is the model as defined in Sec. 3. The following theorem provides insights on how the robustness of the underlying representation function g θ impacts the robustness of a linear classifier. Specifically, it formally describes how the robustness of the representation, i.e. the AS score on a datapoint x , determines the set of weight matrices which yield a robust classifier.

Theorem 2. The classifier c ( x ) is robust in x , i.e., c ( x + δ ) = c ( x ) for δ ∈ R d with ∥ δ ∥ < ϵ , if for x ∈ X with c ( x ) = y it holds that

̸

The proof of the theorem can be found in Appx. A.2.

Proof sketch: To prove robustness, we need to show that f ( x + δ ) y > f ( x + δ ) j if f ( x ) y > f ( x ) j . The proof consists of rewriting this condition and applying the Cauchy-Schwarz inequality to separately bound in terms of the difference in W and g θ .

Intuition: Thm. 2 shows that the effect an ϵ -perturbation has on the representation function g θ determines the necessary class separation by the linear predictor for robust classification to be ensured. Specifically, the RHS of Thm. 2 can be understood as a normalized margin between class logits for x , where the classification margin refers to the difference between the classification score for the true class and maximal classification score from all false classes. Thus, the theorem states that if the margin is larger than the effect of an ϵ -perturbation on the representation function, then the classifier is robust. Note that if a datapoint does not satisfy the bound, i.e. LHS is greater than the RHS, the robustness of the classifier is ambiguous. In this case, the RHS margin is not the exact separation, or the optimal margin, between robust and non-robust classification. Fig. 2 provides intuition for this interpretation of the result in a binary classification scenario.

5 Empirical Study of Theory

The following experiments serve to validate the theoretical results in practical applications. For the finite sample version of Thm. 1, Lemma 1, we consider a classification task in Sec. 5.1, and a regression task in Appx. B.2. In Sec. 5.2, we study the robustness condition established in Thm. 2.

In the following sections, we train a linear predictor on top of a pretrained representation function for a specific downstream task, i.e., we perform transfer learning via linear probing. To study robustness transfer, we use robustly pretrained representation functions and examine if the transferred robustness is consistent with the theoretical bounds. As our robust representation function, we use models robustly pretrained on CIFAR-100 [1, 34], and ImageNet [47, 17]. Both models are available on RobustBench [16] and are adversarially trained using the softmax cross-entropy loss and L ∞ -attacks. For our downstream tasks, we consider CIFAR-10 [34], Fashion-MNIST [57] and Intel Image 2 . Further experimental details can be found in Appx. B.1.

5.1 Classification - Cross-Entropy Loss

Here, we empirically evaluate Lemma 1, the finite sample version of Thm. 1, on classification tasks. As per Sec. 3.1, for softmax cross-entropy, we can apply Lemma 1 with Lipschitz constant C = 2 and mapping function L ∞ ( W ) .

In Tab. 1, we present the left-hand side (LHS), the scaled difference between clean and robust cross-entropy (CE) loss, and the right-hand side (RHS), the representation function's AS score, of Lemma 1. Note that, for LHS, the attack on f maximizes the CE loss, while for RHS, the attack on g θ maximizes the impact on the representation function, i.e. ∥ g θ ( x i ) -g θ ( x i + δ ) ∥ 2 . See Appx. B.1 for further details.

The results in Tab. 1 validate our theoretical predictions in practice, as the LHS is indeed always upper bounded by the RHS. Furthermore, we see signs of correlation between the LHS and RHS, i.e. significant changes on one side of the bound are accompanied by similar changes on the other. Tab. 2 further highlights the connection between our result and weight norm regularization, as it confirms that a smaller weight norm, i.e., smaller L α ( W ) , corresponds to a smaller decrease in robustness, i.e., a smaller difference between the losses.

Table 1: For the CIFAR-100 and ImageNet pretraining datasets, the LHS and RHS, i.e., the AS score, of Lemma 1 are computed for each of the CIFAR-10, Fashion-MNIST and Intel Image downstream tasks. Additionally, L ∞ ( W ) and the absolute difference between clean and adversarial CE loss are provided.

5.2 Classification Criterion

Here, we focus on empirically analysing Thm. 2, which specifies an upper bound on the representation's sensitivity against ϵ -perturbations, where the condition ensures robustness of the classifier against local perturbations for each datapoint.

In Tab. 3, we observe that indeed: if condition (1) is fulfilled, the classification is always robust, as can be seen from "rob. fulfilled". With that said, we do observe instances of robust classification which do not fulfill Eq. (1), which we proved ensures robustness, as can be seen from comparing "robust accuracy" to "prop. fulfilled". As with Lemma 1, we see that tighter results would be necessary for identifying all transferred robustness. Furthermore, as discussed in Sec. 4, since datapoints fulfilling our bound are even further away from the classification boundary than necessary, we also find that fulfilling the bound is predictive of robustness to stronger attacks, as can be seen in Appx. B.3.

For the scenario of pretraining on ImageNet and transferring to CIFAR-10 or Fashion-MNIST, almost no datapoint from the test set fulfills condition (1). This observation is accompanied by a steep drop from clean accuracy to robust accuracy, which suggests this is likely a consequence of the large distribution shift between ImageNet and the lower-resolution images of CIFAR-10 ( 32 × 32 ) and Fashion-MNIST ( 28 × 28 ) resized to ImageNet resolution ( 256 × 256 ). Note that this distribution shift is evident regardless of how the images are scaled before applying the ImageNet representation function, see Appx. B.5. In Tab. 1, the correlation between distribution shift and loss of robustness is also captured prior to transfer learning by a large AS score, and Appx. B.5 provides further insight into this connection.

6 Adversarial Sensitivity Score

In the prior sections, we empirically validated that our theoretical results and required assumptions are realistic and hold in applications. Note that, in both of our results, the robustness of the representation function determines the robustness that transfers to downstream tasks. Specifically, Lemma 1 is stated using the adversarial sensitivity (AS) score of the representation function on a dataset X ,

to bound the difference between clean and adversarial loss. Before specifying a downstream task, we can compute the representation function's adversarial sensitivity score on a set of unlabeled input images. In the following, we explore how to calibrate expectations on the robustness upon transfer of the representation function via linear probing to a particular downstream task based on the AS score.

6.1 Robustness Transfer via Linear Probing

Here, we explore how the AS score can be used to provide valuable insights on the transferability of robustness with linear probing. In Tab. 4, we find that if the AS score on the unlabeled pretraining and downstream datasets are similarly small, then it is reasonable to expect that robustness will transfer via linear probing. However, if the AS score on the downstream task is much higher, it is reasonable to expect that robustness will not transfer via linear probing.

In Tab. 4, we compare the AS score to the relative difference between clean and robust CE loss on each dataset, i.e., (CE - robust CE)/CE. Indeed, the similar AS scores of CIFAR-100, CIFAR-10, and Intel Image upon pretraining on CIFAR-100 correspond to a successful transfer of robustness, whereas the dissimilar adversarial sensitivity scores of ImageNet, Fashion-MNIST, and CIFAR-10 upon pretraining on ImageNet reflect the striking loss of robustness upon transfer. One explanation for a larger AS score is a shift in distribution between pretraining and downstream data. This connection is confirmed in an additional empirical analysis in Appx. B.5.

6.2 Beyond Linear Probing?

In the prior section, we observe a correlation between AS score and the difference between clean and robust CE, i.e., the AS score is predictive of robustness transfer via linear probing. However, linear probing is not the only transfer learning approach, what can we say about alternatives?

In finetuning, the representation function is changed, and thus also the AS score changes posttransfer. Thus, we can expect the AS score computed pre-transfer is less predictive of the downstream robustness for finetuning than for linear probing. To gain a better understanding, we study the difference between clean/robust accuracy for linear probing (LP), full finetuning (FT) as well as linear probing then finetuning (LP-FT) [35], where LP-FT was proposed to combine the benefits of the two approaches. The main results can be found in Tab. 5, and additional details about the experiment are provided in Appx. B.4.

Tab. 5 shows the performance of each transfer learning method applied to the model robustly pretrained on CIFAR-100 or ImageNet. On CIFAR-100, LP preserves the most robust accuracy, whereas LP-FT performs better on ImageNet. Note that the AS score for LP is highlighted, as this is the score we can obtain prior to transfer learning, which allows us to calibrate expectations on downstream performance.

We find no correlation between LP and FT performance, and do not observe a predictive relationship between AS score pre-transfer and robustness transfer via FT. Instead, we observe successful robustness transfer via FT and LP-FT for cases with high AS score pre-transfer. This provides us with a first indication that the adversarial sensitivity score may point to scenarios where FT preserves more robustness downstream than LP.

7 Discussion

Significance: If the representation does not change at all in response to ϵ -perturbations, it may seem immediately clear that any predictor operating on said representation is robust. However, if the representation function is variant to ϵ -perturbations, how much robustness is lost as a result is dependent on the nature of the predictor function under consideration and the distribution shift in the data from pretraining to target task. In this work, we consider a linear predictor, and formally compute the exact effect this function can have on the test-time robustness.

Comparison of the theorems: The theoretical statement in Thm. 1 is closely related to the one in Thm. 2, as a more robust predictor to adversarial attacks leads to a smaller difference in the loss

between clean and adversarial examples, and vice versa. Thm. 1 provides a bound on the robustness based on the difference of the loss function, while Thm. 2 only considers the specific form of the classifier and thus information about and restrictions on the used loss function is unnecessary. However, studying the loss function makes Thm. 1 valid in a much broader set of scenarios, and further provides reasoning on why using weight norm regularization during linear probing improves robustness transfer, see Sec. 5.1.

The bound in Thm. 2, on the other hand, provides insights on how robustness of a classifier can be guaranteed. In particular, one could use our results to derive robustness guarantees post-transfer from attacks on the representation function pre-transfer.

Clarification: Note that we consider a model f as a composition of a linear layer and a representation function, i.e., f ( x ) = Wg θ ( x ) , but do not specify how the parameters of f are learned. As our motivating example, the representation function g θ is optimized on a pretraining task and subsequently W is trained w.r.t. a downstream task. However, one could also approach the downstream task with the classical strategy of training the model end-to-end instead, i.e., optimizing ˜ W and ˜ θ simultaneously in f ˜ W, ˜ θ = ˜ Wg ˜ θ , assuming the same architecture as before. The parameters θ, W and ˜ θ, ˜ W obtained from the two strategies may be very different. For example, the representation function g θ may not be as useful for linearly separating the dataset into c classes as g ˜ θ , which was optimized specifically for this task. This observation is described as a task mismatch between pretraining and the supervised downstream task.

Our contributed theoretical results demonstrate that we can study robustness by studying the learned functions themselves, irregardless of how said functions were optimized in the first place. With that said, this does not mean that the aforementioned task mismatch does not affect the model's robustness. Thm. 1's bound will loosen if, e.g., g θ does not enable linear separation of the dataset into c classes, and if the AS score of the representation function increases due to this task mismatch or distribution shift in the data. We provide a study on the impact of the distribution shift in the data on the AS score in Appx. B.5.

Finally, while the linear probing protocol is standard practice [10, 11], alternative methodology exists for leveraging a representation function for a specified downstream task. For one, we could consider a model f as a composition of a nonlinear function h and a representation function, i.e. f ( x ) = h ◦ g θ ( x ) , a scenario our theoretical statements do not account for. Furthermore, alternative approaches can include updating the parameters of the representation function g θ , e.g., full finetuning. As suggested by the results in Sec. 6.2, while a high AS score may not be predictive of robustness transfer for finetuning, it may instead indicate when finetuning is required to transfer robustness. Overall, the theoretical dependencies between the robustness of the representations and transfer learning alternatives to linear probing present interesting opportunities for future analysis.

8 Conclusion

We prove two theorems which demonstrate that the robustness of a linear predictor on downstream tasks can be bound by the robustness of its underlying representation. We provide a result applicable to various loss functions and downstream tasks, as well as a more direct result for classification. Our

theoretical insights can be used for analysis when representations are transferred between domains, and sets the stage for carefully studying the robustness of adapting pretrained models.

References

Sravanti Addepalli, Samyak Jain, et al. Efficient and effective augmentation strategy for adversarial training. Advances in Neural Information Processing Systems , 35:1488-1501, 2022.

Ehab A AlBadawy, Ashirbani Saha, and Maciej A Mazurowski. Deep learning for segmentation of brain tumors: Impact of cross-institutional training and testing. Medical physics , 45(3):1150-1158, 2018.

Anders Andreassen, Yasaman Bahri, Behnam Neyshabur, and Rebecca Roelofs. The evolution of out-ofdistribution robustness throughout fine-tuning. arXiv preprint arXiv:2106.15831 , 2021.

Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, et al. A cookbook of self-supervised learning. arXiv preprint arXiv:2304.12210 , 2023.

Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndi´ c, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases , pages 387-402. Springer, 2013.

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, S. Buch, Dallas Card, Rodrigo Castellon, Niladri S. Chatterji, Annie S. Chen, Kathleen A. Creel, Jared Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren E. Gillespie, Karan Goel, Noah D. Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas F. Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, O. Khattab, Pang Wei Koh, Mark S. Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir P. Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Benjamin Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, J. F. Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Robert Reich, Hongyu Ren, Frieda Rong, Yusuf H. Roohani, Camilo Ruiz, Jack Ryan, Christopher R'e, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishna Parasuram Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei A. Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. On the opportunities and risks of foundation models. ArXiv , 2021.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020.

Sébastien Bubeck and Mark Sellke. A universal law of robustness via isoperimetry. Advances in Neural Information Processing Systems , 34, 2021.

Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From self-supervised pre-training to fine-tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 699-708, 2020.

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning , pages 1597-1607. PMLR, 2020.

Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297 , 2020.

Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9640-9649, 2021.

Geoffrey Chinot, Guillaume Lecué, and Matthieu Lerasle. Robust statistical learning with lipschitz and convex loss functions. Probability Theory and related fields , 176(3-4):897-940, 2020.

Kurtland Chua, Qi Lei, and Jason D Lee. How fine-tuning allows for effective meta-learning. Advances in Neural Information Processing Systems , 34, 2021.

Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In International Conference on Machine Learning , pages 1310-1320. PMLR, 2019.

Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: a standardized adversarial robustness benchmark. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) , 2021.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009.

Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. Mma training: Direct input space margin maximization through adversarial training. In International Conference on Learning Representations , 2020.

Simon Shaolei Du, Wei Hu, Sham M. Kakade, Jason D. Lee, and Qi Lei. Few-shot learning via learning the representation, provably. In International Conference on Learning Representations , 2021.

Lijie Fan, Sijia Liu, Pin-Yu Chen, Gaoyuan Zhang, and Chuang Gan. When does contrastive learning preserve adversarial robustness from pretraining to finetuning? In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems , 2021.

Stanislav Fort. Adversarial examples for the openai clip in its zero-shot classification regime and their semantic generalization, Jan 2021.

Stanislav Fort. Adversarial vulnerability of powerful near out-of-distribution detection. arXiv preprint arXiv:2201.07012 , 2022.

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. International Conference on Learning Representations , 2015.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision , pages 1026-1034, 2015.

Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. Advances in Neural Information Processing Systems , 30, 2017.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations , 2019. URL https:// openreview.net/forum?id=HJz6tiCqYm .

Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. Unsolved problems in ml safety. arXiv preprint arXiv:2109.13916 , 2021.

De-An Huang, Zhiding Yu, and Anima Anandkumar. Transferable unsupervised robust representation learning, 2021.

Neal Jean, Marshall Burke, Michael Xie, W Matthew Davis, David B Lobell, and Stefano Ermon. Combining satellite imagery and machine learning to predict poverty. Science , 353(6301):790-794, 2016.

Ziyu Jiang, Tianlong Chen, Ting Chen, and Zhangyang Wang. Robust pre-training by adversarial contrastive learning. Advances in Neural Information Processing Systems , 33:16199-16210, 2020.

Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT , volume 1, page 2, 2019.

Minseon Kim, Jihoon Tack, and Sung Ju Hwang. Adversarial self-supervised contrastive learning. Advances in Neural Information Processing Systems , 33:2983-2994, 2020.

Simon Kornblith, Jonathon Shlens, and Quoc V Le. Do better imagenet models transfer better? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2661-2671, 2019.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang. Fine-tuning can distort pretrained features and underperform out-of-distribution. In International Conference on Learning Representations , 2022.

Alexander Levine, Sahil Singla, and Soheil Feizi. Certifiably robust interpretation in deep learning. CoRR , abs/1905.12105, 2019.

Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. Certified adversarial robustness with additive noise. Advances in neural information processing systems , 32, 2019.

Dongyue Li and Hongyang Zhang. Improved regularization and robustness for fine-tuning in neural networks. Advances in Neural Information Processing Systems , 34:27249-27262, 2021.

Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations , 2017. URL https://openreview.net/forum?id= Skq89Scxx .

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations , 2018.

Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dsprites: Disentanglement testing sprites dataset. https://github.com/deepmind/dsprites-dataset/, 2017.

Matthew E Peters, Sebastian Ruder, and Noah A Smith. To tune or not to tune? adapting pretrained representations to diverse tasks. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019) , pages 7-14, 2019.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning , pages 8748-8763. PMLR, 2021.

Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial examples. In International Conference on Learning Representations , 2018.

Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: A python toolbox to benchmark the robustness of machine learning models. In Reliable Machine Learning in the Wild Workshop, 34th International Conference on Machine Learning , 2017.

Hadi Salman, Jerry Li, Ilya Razenshteyn, Pengchuan Zhang, Huan Zhang, Sebastien Bubeck, and Greg Yang. Provably robust deep learning via adversarially trained smoothed classifiers. Advances in Neural Information Processing Systems , 32, 2019.

Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust imagenet models transfer better? Advances in Neural Information Processing Systems , 33:35333545, 2020.

Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust imagenet models transfer better? Advances in Neural Information Processing Systems , 33:35333545, 2020.

Gal Shachaf, Alon Brutzkus, and Amir Globerson. A theoretical analysis of fine-tuning with linear teachers. Advances in Neural Information Processing Systems , 34, 2021.

Ali Shafahi, Parsa Saadatpanah, Chen Zhu, Amin Ghiasi, Christoph Studer, David Jacobs, and Tom Goldstein. Adversarially robust transfer learning. In International Conference on Learning Representations , 2020.

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. International Conference on Learning Representations , 2014.

Nilesh Tripuraneni, Michael Jordan, and Chi Jin. On the theory of transfer learning: The importance of task diversity. Advances in Neural Information Processing Systems , 33:7852-7862, 2020.

Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers for attacking and analyzing nlp. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , 2019.

Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning , pages 5286-5295. PMLR, 2018.

Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations , 2020.

Sen Wu, Hongyang R. Zhang, and Christopher Ré. Understanding and improving information transfer in multi-task learning. In International Conference on Learning Representations , 2020.

Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 , 2017.

Yutaro Yamada and Mayu Otani. Does robustness on imagenet transfer to downstream tasks? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9215-9224, 2022.

Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ R Salakhutdinov, and Kamalika Chaudhuri. A closer look at accuracy vs. robustness. Advances in neural information processing systems , 33:8588-8601, 2020.

Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2636-2645, 2020.

Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference 2016 . British Machine Vision Association, 2016.

Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, et al. A large-scale study of representation learning with the visual task adaptation benchmark. arXiv preprint arXiv:1910.04867 , 2019.

Jiaming Zhang, Jitao Sang, Qi Yi, Yunfan Yang, Huiwen Dong, and Jian Yu. Imagenet pre-training also transfers non-robustness. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 3436-3444, 2023.

A Proofs and Auxiliary Results

We state two auxiliary results first and subsequently provide the full proofs for Thm. 1 and Thm. 2 in Appx. A.1 and Appx. A.2.

Lemma 2. For a matrix W ∈ R c × r and a vector g ∈ R r , the following bounds hold,

∥ Wg ∥ 2 ≤ ∥ W ∥ 2 ∥ g ∥ 2 ,

∥ Wg ∥ 1 ≤ ∑ c i =1 ∥ W i ∥ 2 ∥ g ∥ 2 ,

∥ Wg ∥ ∞ ≤ max i ∥ W i ∥ 2 ∥ g ∥ 2 .

Proof. 1. ∥ Wg ∥ 2 = √∑ c i =1 ( W i g ) 2 ≤ √∑ c i =1 ( ∥ W i ∥ 2 ∥ g ∥ 2 ) 2 = ∥ W ∥ 2 ∥ g ∥ 2 , where the second step follows by Cauchy-Schwartz inequality.

∥ Wg ∥ 1 = ∑ c i =1 | W i g | ≤ ∑ c i =1 ∥ W i ∥ 2 ∥ g ∥ 2 = ∑ c i =1 ∥ W i ∥ 2 ∥ g ∥ 2 , where the second step follows again by Cauchy-Schwartz inequality.

∥ Wg ∥ ∞ = max i | W i g | ≤ max i ∥ W i ∥ 2 ∥ g ∥ 2 , where the second step follows by CauchySchwartz inequality.

The following Lemma proves a Lipschitz condition for the cross-entropy-softmax loss.

Lemma 3. We define the cross-entropy loss with softmax function for a vector f ∈ R c and a class index y ∈ { 1 , . . . , c } , i.e.,

Then, the following Lipschitz condition holds for ℓ ( f, y ) ,

for any vectors a, b ∈ R c and an arbitrary class index y ∈ { 1 , . . . , c } .

Proof.

where p j ( a ) := exp { a j } ∑ c l =1 exp { a l } denotes the soft-max output of the vector a . The inequality in the last line follows by Jensen's inequality, since -log is convex and the soft-max probabilities p j sum up to one.

since both summands in the middle can be bounded by the supremum norm.

A.1 Proof of Theorem 1

Proof of Theorem 1. We use Hoeffding's inequality on the independent random variables D 1 . . . . , D n , which are defined as

based on independently drawn data points with probability distribution P ( X ) , where the observations are x i ∈ X with corresponding labels y i . Wecan use the upper bound on ℓ to obtain that 0 ≤ D i ≤ C 2 and conclude that

Thus, with probability of at least 1 -ρ it holds that

We can further bound the first term on the right-hand side, since the loss function ℓ ( r, y ) is C 1 -Lipschitz in ∥ · ∥ α for r ∈ f ( X ) and f ( x ) = Wg θ ( x ) .

where δ 1 , . . . , δ n ∈ X with ∥ δ i ∥ < ϵ are chosen appropriately. In a next step, we apply Lemma 2 and the definition of L α in the theorem statement,

The δ i 's might not be the same that maximize the distance between the representations but their effect can be upper bounded by the maximum. Combining the derived bounds and using that g θ ( x i + δ i ) ≤ max ∥ δ ∥ <ϵ g θ ( x i + δ ) , we obtain our result.

A.2 Proof of Theorem 2

Proof of Theorem 2. We want to proof that the classifier c ( x ) is robust in x ∈ X , i.e., it holds that c ( x + δ ) = c ( x ) for all δ ∈ R d with ∥ δ ∥ < ϵ . The classifier predicts class y , i.e., c ( x ) = y , if the y-th value of the model f ( x ) is larger than all other entries, i.e., f ( x ) y > f ( x ) j for all j = y . We

̸

denote the y-th row of the linear layer's matrix W by W y , thus f ( x ) y = W y g θ ( x ) . For an input x with c ( x ) = y , we want to prove that c ( x + δ ) = y for any δ with ∥ δ ∥ < ϵ ., i.e.,

It holds that

Thus, for f ( x + δ ) y > f ( x + δ ) j to be true, it is necessary that

By Cauchy-Schwartz inequality follows that

̸

Since further W y g θ ( x ) -W j g θ ( x ) = f ( x ) y -f ( x ) j , the condition of equation (3) holds for any j = y if

̸

For c ( x + δ ) = c ( x ) to be true, the above condition needs to be true for all j = y and this holds if

̸

B Additional Experimental Results

We provide additional information on the datasets used and the employed hyperparameters in Appx. B.1. A further application of Thm. 1 is presented in Appx. B.2, where robustness transfer is analysed for a regression task. In Appx. B.3, we extend the analysis from Sec. 6 by evaluating the robustness against stronger attacks. In Appx. B.4, an empirical study compares the robustness transfer depending on the chosen transfer learning method. Finally, in Appx. B.5, we investigate the effect of a shift in distribution between pretraining and downstream task data on the transferred robustness and the AS score.

B.1 Experimental Setup for Section 5

For our experiments, we consider two models robustly pretrained on CIFAR-100 [34] and ImageNet [17]. CIFAR-100 consists of 32x32 RGB images with 100 different classes. ImageNet-1K consists of 256x256 RGB images of 1,000 different classes.

We investigate the performance of robustness transfer on CIFAR-10 [34], Fashion-MNIST [57] and Intel Image Classification 3 . The CIFAR-10 dataset consists of 32x32 RGB images in 10 classes. Intel Image Classification consists of 150x150 RGB images of 6 natural scenes (buildings, forest, glacier, mountain, sea, and street). Fashion-MNIST is a dataset of Zalando's article images, where each example is a 28x28 grayscale image, associated with a label from 10 classes.

We use two models from RobustBench [16], namely robust models trained on CIFAR-100 and ImageNet. For CIFAR-100, we use a WideResNet-34-10 ( ≈ 46M parameters) architecture [61] which is adversarially trained using a combination of simple and complex augmentations with separate batch normalization layers [1]. For ImageNet, we use a WideResNet-50-2 ( ≈ 66M parameters) architecture [61] which is is also adversarially trained [47, 40].

For the model pretrained on CIFAR-100, run 20 epochs of linear probing (LP) using a batch size of 128, and resize all input images to 32x32. For the model pretrained on ImageNet, we run 10 epochs of LP using a batch size of 32, and resize all input images to 256x256. We reduce the batch size to compensate for the increase in input resolution. For LP, we set the learning rate using a cosine

annealing schedule [39], with an initial learning rate of 0.01, and use stochastic gradient descent with a momentum of 0.9 on the cross-entropy loss.

To compute LHS and RHS, we must attack the cross-entropy loss on f as well as the MSE on g θ . In both cases, we employ L ∞ -PGD attacks implemented in the foolbox package [45]. As hyperparameters for the L ∞ -PGD attack, we choose 20 steps and a relative step size of 0.7, since this setting yields highest adversarial sensitivity (AS) scores over all the tasks. The attack strength ϵ is set to ϵ = 8 / 255 for attacking the CIFAR-100 pretrained model and ϵ = 1 / 255 for the ImageNet model. Thus, we use the same ϵ on CIFAR-100 as used during the RobustBench pre-training and reduce the ϵ on ImageNet due to the constant pixel blocks introduced after resizing CIFAR-10 and Fashion-MNIST images from 32x32 and 28x28 to 256x256, respectively.

The experiments were run on a single GeForce RTX 3080 GPU and for each downstream task transfer learning or theory evaluation took between 30 minutes to 3 hours.

B.2 Regression - MSE

Here, we empirically evaluate Lemma 1 on regression tasks using the mean squared error (MSE) as the scoring criterion. As per Sec. 3.1, we can apply Lemma 1 with Lipschitz constant C = 1 and mapping function L 2 ( W ) .

We consider the dSprites [41] dataset, which consists of 64x64 grayscale images of 2D shapes procedurally generated from 6 factors of variation (FoVs) which fully specify each sprite image: color, shape, scale, orientation, x-position (posX) and y-position (posY). The dataset consists of 737,280 images, of which we use an 80-20 train-test split. Here, we pre-train for the regression task of decoding an FoV from an image, i.e., we optimise the network f to minimize MSE with the exact orientation of the sprite. For LP, we consider 4 downstream tasks, each corresponds to decoding a unique FoV, while one matches the task used for pre-training.

The model architecture is summarized in Tab. 6. We consider 3 different representation functions, network before training (with default kaiming uniform initialization [24]), after training, and after adversarial training. For the robust model, we utilize L ∞ -PGD attacks implemented in the foolbox package [45] with 20 steps and a relative step size of 1.0. The attack strength ϵ is set to ϵ = 8 / 255 . The model was trained for 10 epochs with a batch size of 512, learning rate of 1e -3 with a cosine annealing schedule [39], and stochastic gradient descent with a momentum of 0.9.

In Tab. 7, we present the left-hand side (LHS), the rescaled difference between clean and robust cross-entropy (CE) loss, and the right-hand side (RHS), the representation function's AS score, of Lemma 1. To obtain the scores for LHS and RHS, we separately attack f as well as the MSE on g θ , where in both cases the objective uses the MSE. For these attacks, we again utilize L ∞ -PGD, but also L 2 -PGD attacks, both based on the foolbox [45] implementation, with 50 steps and a relative step size of 1.0. The attack strength ϵ is set to ϵ = 16 / 255 for both the attack types. As can be seen, LHS is indeed always upper bounded by RHS. Note that, for each pretraining and attack type combination, the RHS is the same over all latent factors, i.e. in each row, because the same images and the same representation function are used.

B.3 Classification Criterion - Attack Strength

Here, we study the hypothesis stated in Sec. 5.2 that datapoints fulfilling the bound in Thm. 2 are still robust against an increased attack strength. We repeat the experiments from Sec. 5.2 for the model pretrained on CIFAR-100, where the attack strength is ϵ = 8 / 255 , since, for this case, a subset of each dataset fulfilled the equation. Additionally, we compute the robust accuracy against stronger attacks employing double and three times the attack strength, i.e., using 2 ϵ = 16 / 255 and 3 ϵ = 24 / 255 .

In Tab. 8, we observe that indeed most of the datapoints fulfilling the bound in Eq. (1) are still robust against the doubled attack strength, which is suggested by the robust accuracy for 2 ϵ being larger than the proportion that fulfilled the bound. Most of these datapoints, approximately 50% , are even robust against 3 ϵ . Further analysis is needed to better understand the observed robust accuracy scores and to derive a precise connection to the bound from Thm. 2.

Table 8: For the CIFAR-100 pretraining dataset, we report clean accuracy, the proportion of images fulfilling the theoretical bound, and robust accuracy for several attack strengths for each of the CIFAR-10, Fashion-MNIST and Intel Image downstream tasks.

B.4 Comparison Study between LP, FT and LP-FT

Here, we compare robustness transfer on classification tasks for three transfer learning techniques, linear probing, full finetuning, and linear probing then finetuning [35]. As explained before, linear probing (LP) refers to freezing all pretrained weights and only optimizing the linear classification head. In full finetuning (FT) all pretrained parameters are updated during transfer learning. Linear probing then finetuning (LP-FT) combines LP and FT sequentially, as after LP is done, FT is applied to all parameters.

For the model pretrained on CIFAR-100, we choose a batch size of 128, and run 20 epochs for FT, LP and 10 epochs of FT after LP for LP-FT. For the model pretrained on ImageNet, we choose a batch size of 32 to compensate for the increase in input resolution to 256x256. We run 10 epochs for FT and LP and 5 epochs for LP-FT. In both settings, we use half as many epochs for finetuning

after linear probing in LP-FT, as in [35]. For all methods we apply a cosine annealing learning rate schedule [39], with an initial learning rate of 0.01 for LP on CIFAR-100, and 0.001 for all other scenarios. We use stochastic gradient descent with a momentum of 0.9 on the cross-entropy loss for all experiments.

We evaluate the adversarial robustness on the test set after training by employing the L ∞ -PGD attack provided by the foolbox package [45], with 20 steps and a relative step size of 0 . 7 . As the attack strength, we choose ϵ = 8 / 255 for the model pretrained on CIFAR-100. Due to the increase in input resolution for the ImageNet model, we apply ϵ = 1 / 255 .

Tab. 5 in Sec. 6.2 shows the performance of each transfer learning method applied to the model robustly pretrained on CIFAR-100. For all datasets, LP preserves the most robust accuracy, while performing worst on natural examples. FT and LP-FT show similar natural and robust accuracy. Only for the Intel Image dataset do we observe a significant difference (10%) in the robustness of LP-FT and FT. In contrast, when pretraining on ImageNet, LP has the worst clean and robust accuracy. Instead, LP-FT has the highest robust accuracy for all datasets, while FT achieves the highest clean accuracy. Although FT has the highest clean accuracy for all datasets, the difference between FT and LP-FT is marginal.

Additionally, Tab. 9 provides confirmation that our theory indeed holds for the other transfer learning approaches as well. The difference is that for FT and LP-FT the AS score changes during transfer learning and thus the score computed prior transfer learning is no longer predictive of the transferred robustness. Still, it is an interesting observation that indeed the AS score shrinks during LP-FT when it improves robustness transfer compared to LP.

Table 9: Absolute difference between clean and robust CE loss ( ∆ CE ), L ∞ ( W ) from Thm. 1 and the adversarial sensitivity score (AS) for the three transfer learning approaches using the model robustly pretrained on CIFAR-100 or ImageNet.

B.5 Distribution Shift Experiments

In this section, we analyse the impact of distribution shift between pretraining and downstream data on the adversarial sensitivity score and thereby on robustness transfer via linear probing. First, we consider the distribution shift between CIFAR-10 and ImageNet and, in particular, its dependence on how the smaller CIFAR-10 images are resized before applying the representation function. Second, we employ ideas from prior work [26] 4 to create increasingly corrupted versions of CIFAR-100 to simulate varying shifts in distribution.

In our experiments in Sec. 5, we resize the 32x32 CIFAR-10 input images to 256x256 for the ImageNet model, and observe a strong distribution shift by a severely increased AS score. Here, we evaluate if increasing the image size before applying the ResNet model causes this shift, or if it is inherent in the data. In Tab. 10, we repeat our linear probing experiment for the CIFAR-10 to ImageNet case on differently rescaled input images. The results indicate that reducing the increase in the input image size improves robustness transfer, at the cost of a drop in clean performance. However, even for a smaller input image size, the AS score remains increased indicating a tangible distribution shift remains.

In the following, we consider an increasingly corrupted version of CIFAR-100 to simulate distribution shift following prior work [26]. In particular, we use the Gaussian noise (Tab. 12) and Zoom blur (Tab. 13) corruption types with increasing severity to simulate an increasing distribution shift. The experimental results from both distortion methods confirm that an increasing shift between the data distributions leads to an increase in the AS score and poorer robustness transfer via linear probing. The different corruption strength are used as suggested in [26] and are stated in Tab. 11.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the question of how the robustness of pretrained representations transfers to downstream tasks. Under the linear probing setting (i.e., the downstream classifier $f$ is linearly related to the pretrained representations $g_{\\theta}$), they prove that the gap between the standard and robust risk of $f$ can be bounded by the summand of the \"perturbation on representations\" plus a constant (w.r.t. $g_{\\theta}$ and $x$). By rearranging terms, they obtain the *robustness score* of a dataset, which can be used to estimate how well the robustness of pretraining would transfer to downstream tasks. This paper also provides an upper bound for the robustness score. Experiments on several datasets are conducted to validate their theoretical results.\n\nSTRENGTHS:\n1. This paper provides theoretical results that upper bounds the gap between robust and standard loss under the setting that they concern. Besider, the experimental results perfectly fit their theory. \n2. They propose the concept of robustness score, which is easy to calculate and is useful in estimating the robustness of the downstream classifiers.\n\nWEAKNESSES:\n+ The main theoretical result (i.e., Theorems 1) is essentially a robust risk bound for linear classifiers, which is, technically speaking, quite a simple result. Previous works have proved similar robust risk bounds for 2-layer ReLU networks and even more generally, for VC-class functions. I understand that Theorem 1 indeed provides a robust risk bound for downstream tasks under the linear probing setting, and the analyse for linear classifiers cannot be extended to other function classes due to linear probing. However, I still think the present paper's theoretical contribution is not enough. \n+ **The current version of this paper looks like a work in progress.** As is stated above, the theoretical contribution of this paper is not enough. Maybe the author should complete the future work that is mentioned in Section 7.\n+ The writing of this paper might need further polishing. For example, Figure 1 does not illustrates the intuition behind their theory (something related to your proofs is expected). The related work is also hard to understand, making the readers hard to evaluate the contribution of this paper.\n\nQUESTIONS:\n**Section 2 (Related Work) is hard to understand.** Could the author please explain, in more detail, the relationship between the related works and yours? My major questions include:\n\n1. In Line 55-59, the author claims that ref. [54, 43, 15, 36, 35, 45], instead of ref. [25], provides practical guarantees on the detection of adversarial examples. By saying \"instead\", the author seems to be expressing the idea that ref. [25] theoretically guarantees the detection of adversarial examples. However, ref. [25] does not seem to provide such results.\n2. What is the relationship between adversarial training, self-supervised contrastive learning, and the present work?\n3. I don't think the present work is \"in contrast with\" ref. [59] and [8], as is claimed in Line 69-71. In my opinion, ref. [59], [8] and the present work are studying difference topics.\n4. As is mentioned in Line 75-76 and 47-48, linear probing seems to be closely related to the main contributions of this paper. In fact, the theoretical results in Sections 3 and 4 are all based on the linear probing (i.e., $f(x) = W g_{\\theta}(x)$). However, I have found no related work on linear probing in this paper. It would make this paper more comprehensible if the author briefly introduce the basic setting and related work about linear probing.\n\nLIMITATIONS:\nNo.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper study how the robustness properties of a given representation might be transferred to a linear probe which was trained on this representation.  Since the linear probe is a simple multiplication between the weights parameters and the representation vector, the authors were able to demonstrate that by using the representation we can bound the downstream linear probe robustness performances. To do so, they introduced a robustness score, which measure the maximal distance in the representation space that can be reached through adversarial perturbations. If this distance if lower than the linear classification margin, then we can expect the linear probe to be robust to these adversarial pertubations.\n\nSTRENGTHS:\nThe paper is well written and easy to follow.\n\nWEAKNESSES:\n- There isn't a discussion about how regularizing the weight matrix of the linear probe might impact the downstream robustness. It's only one hypothesis but if within a representation vector, there might be some dimensions that are changing much more than some others, applying a sparse matrix multiplication might removed those dimensions and thus offer better robustness downstream. \n\n- Some confusion about the name of robustness score. If this score is measuring how robust a representation is, then we could expect that a higher score will imply a better robustness while a lower score will offer weaker robustness.  However, in this instance, it doesn't seem to be the case, since a smaller robustness score imply better robustness. So I would advise the authors to rename their score of to change their formula such that higher robustness score imply better robustness.\n\n- Some issues and limitations with the experimental setup. One hypothesis given by the authors for the results in Table 2 for which the proportion of fulfilled data points (condition 1) is zero, is that the resolution gap between the pretrained data and dowstream data is too big. However, since the authors are using a convnet for their experiment (wide-resnet), they should be able to change the resolution. So instead of resizing to 256x256, the authors could have only rescale to 64x64 (or 128x128) and compute their results from here. So I would have expected a better empirical evaluation of this discrepancy in the results. I also would have expected some discussion and study around the role of hyper-parameters when training the linear probe on condition 1. Lastly, the authors use adversarial pretrained model for their experiments, however, I would have expected to see some results on baseline trained without adversarial training. \n\n- Inconsistency between some experiments. In Table 2, for the pretrained ImageNet model, the cifar10 accuracy is 92% while the robustness accuracy is 0%. However when looking at Table 8 the cifar10 accuracy is still 92% but the robustness accuracy is 48%. So if both table show linear probing accuracy, why is there those differences ?\n\nQUESTIONS:\n- In Table 2, you wrote that the proportion of fulfilled data points (condition 1) is zero for CIFAR10 and F-MNIST when pretrained on ImageNet. However, you wrote that the accuracy of fulfilled data points is 100%. I don't understand how the accuracy can be 100% if you have zero point in this category (If this is the case, it should be undefined, not 100%).\n- Why the results for LP in Table 8 are different from Table 2 ?\n\nLIMITATIONS:\nThe authors correctly addressed the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The paper studies the robust transfer of a pre-trained model, which demonstrate that the robustness of a linear predictor on downstream tasks can be constrained by the robustness of its underlying representation, and theoretically prove a bound on the difference between clean and adversarial loss in terms of the robustness score. The experiment further demonstrates the effectiveness of the ‘robustness score proposed’ in the paper.

STRENGTHS:
1. The paper is well-written and the proposed theories are clearly formulated.

2. The paper provides theoretical guarantees for the robust transfer of pre-trained models.

3.  The experiment further demonstrates the effectiveness of the proposed theories.

WEAKNESSES:
See questions

QUESTIONS:
1 In Figure 1, How do define and distinguish between robust representation and non-robust representation?

2. The experiments in the paper are all transferred from pre-trained models on larger-scale data to small-scale data, whether it is also possible to transfer from small-scale to large-scale data.

3. How does the robustness of a pre-trained model affect the performance of the models on downstream tasks?

LIMITATIONS:
The pre-trained model is used as model initialization for different tasks. In the paper, it is only demonstrated on the classification task. It is not clear whether the proposed theory and method can be used for other tasks (e.g., object detection).
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers the problem of robustness transfer from pretraining to downstream finetuning. The authors focus on the setup of linear probing, where downstream tasks are learned with a linear model on top of frozen pretrained features. They however give some insights on how to go beyond linear probing and what happens when the model is fully finetuned. \n\nThe paper is composed of a theory part, followed by an empirical analysis to derive practical insights from the theory. In the first part, the authors prove a bound on the difference between the adversarial and clean loss that is independent of the downstream task. The only conditions required for the this bound to hold are the Lipschitz continuity of the loss with respect to its first argument and the loss boundedness. The authors further highlight a number of loss functions that are valid in the scope of their theory, including the softmax cross entropy loss and the MSE. The authors further provide a criterion for robust classification, providing a minimum margin value that is sufficient for robustness. A notable aspect of their theory is its independence on the pretraining approach.\n\nIn the second part, the authors provide insights on how to calibrate expectation of robustness for downstream tasks. They validate their results with models pretrained on CIFAR100 and ImageNet, adversarially trained with softmax cross entropy and $L_\\inf$ attacks, and consider CIFAR10, FashionMNIST and Intel Image as downstream tasks.\n\nSTRENGTHS:\n* The paper is clearly written and well structured. The related works are properly addressed and the papers brings interesting novel insights.  \n* The theoretical results are sound and make sense. The empirical analysis is well designed to validate the theory. (Both could be improved for higher impact, see next sections). \n* The contributions are significant and important to the community, especially in the context of the increasing interest in the pretraining -> fine tuning paradigm. \n* The theory is general enough, and applies in interesting practical settings.\n\nWEAKNESSES:\n* While the theory is general and applies to different objectives and pretraining approaches, the empirical analysis seems narrow, focusing on image classification with adversarial pretraining. It would be interesting to show how the results generalise to other modalities, and in case where robustness upstream doesn't come from explicit guards against attacks, but from other properties, e.g. scale. \n\n* While the theoretical bound is interesting, there seems to be an opportunity to make it tighter with more (but still reasonable) conditions. Instead of the Hoeffding’s inequality, one can consider other more precise inequalities.\n\nQUESTIONS:\n* Most of the settings explored in the paper consider fairly similar pretraining and downstream tasks. What happens when the distribution shift is more severe? (The author did mention this is a interesting direction for further research, but empirical insights could make the paper stronger, even without further theory at this point). \n\n* What happens when pretraining is done without explicit guard against adversarial attacks? \n\n* In table 3, wouldn't it be more interesting to consider correlation between the robustness score and the clean & robust cross-entropy loss instead of their relative difference? For more data points, one could consider different checkpoints during training, that will give different robustness levels.\n\nLIMITATIONS:\nThe authors did consider how they can improve their work and further developments, without explicitly stating the limitations. This seems reasonable to me given the nature of the work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-27"}