SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We use two models from RobustBench [16], namely robust models trained on CIFAR-100 and ImageNet. For CIFAR-100, we use a WideResNet-34-10 ( $\approx$ 46M parameters) architecture [61] which is adversarially trained using a combination of simple and complex augmentations with separate batch normalization layers [1]. For ImageNet, we use a WideResNet-50-2 ( $\approx$ 66M parameters) architecture [61] which is also adversarially trained [47, 40].
For the model pretrained on CIFAR-100, run 20 epochs of linear probing (LP) using a batch size of 128, and resize all input images to  $32 \times 32$ . For the model pretrained on ImageNet, we run 10 epochs of LP using a batch size of 32, and resize all input images to  $256 \times 256$ . We reduce the batch size to compensate for the increase in input resolution. For LP, we set the learning rate using a cosine
<sup>3</sup>https://www.kaggle.com/datasets/puneet6060/intel-image-classification
Layer	Layer Type	Activation and Normalization
1	Convolutional layer (c=6, k=5)	ReLU, Average Pooling, and Batch Norm
2	Convolutional layer (c=16, k=5)	ReLU, Average Pooling, and Batch Norm
3	Linear Layer (out=120)	ReLU and Batch Norm
4	Linear Layer (out=84)	ReLU and Batch Norm
5	Linear Layer (out=84)	ReLU and Batch Norm
"head"	Linear Layer (out=1)	-
Table 6: The table presents the model architecture used in experiments on MSE loss and the dSprites dataset. The convolutional layers are provided with number of output channels and kernel size, i.e., (c=channels, k=kernel_size), where as the linear layers are listed together with the size of their output vector.
annealing schedule [39], with an initial learning rate of 0.01, and use stochastic gradient descent with a momentum of 0.9 on the cross-entropy loss.
To compute LHS and RHS, we must attack the cross-entropy loss on  $f$  as well as the MSE on  $g_{\theta}$ . In both cases, we employ  $L_{\infty}$ -PGD attacks implemented in the foolbox package [45]. As hyperparameters for the  $L_{\infty}$ -PGD attack, we choose 20 steps and a relative step size of 0.7, since this setting yields highest adversarial sensitivity (AS) scores over all the tasks. The attack strength  $\epsilon$  is set to  $\epsilon = 8 / 255$  for attacking the CIFAR-100 pretrained model and  $\epsilon = 1 / 255$  for the ImageNet model. Thus, we use the same  $\epsilon$  on CIFAR-100 as used during the RobustBench pre-training and reduce the  $\epsilon$  on ImageNet due to the constant pixel blocks introduced after resizing CIFAR-10 and Fashion-MNIST images from 32x32 and 28x28 to 256x256, respectively.
The experiments were run on a single GeForce RTX 3080 GPU and for each downstream task transfer learning or theory evaluation took between 30 minutes to 3 hours.
B.2 Regression - MSE
Here, we empirically evaluate Lemma 1 on regression tasks using the mean squared error (MSE) as the scoring criterion. As per Sec. 3.1, we can apply Lemma 1 with Lipschitz constant  $C = 1$  and mapping function  $L_{2}(W)$ .
We consider the dSprites [41] dataset, which consists of 64x64 grayscale images of 2D shapes procedurally generated from 6 factors of variation (FoVs) which fully specify each sprite image: color, shape, scale, orientation, x-position (posX) and y-position (posY). The dataset consists of 737,280 images, of which we use an 80-20 train-test split. Here, we pre-train for the regression task of decoding an FoV from an image, i.e., we optimise the network  $f$  to minimize MSE with the exact orientation of the sprite. For LP, we consider 4 downstream tasks, each corresponds to decoding a unique FoV, while one matches the task used for pre-training.
The model architecture is summarized in Tab. 6. We consider 3 different representation functions, network before training (with default kaiming uniform initialization [24]), after training, and after adversarial training. For the robust model, we utilize  $L_{\infty}$ -PGD attacks implemented in the foolbox package [45] with 20 steps and a relative step size of 1.0. The attack strength  $\epsilon$  is set to  $\epsilon = 8 / 255$ . The model was trained for 10 epochs with a batch size of 512, learning rate of 1e-3 with a cosine annealing schedule [39], and stochastic gradient descent with a momentum of 0.9.
In Tab. 7, we present the left-hand side (LHS), the rescaled difference between clean and robust cross-entropy (CE) loss, and the right-hand side (RHS), the representation function's AS score, of Lemma 1. To obtain the scores for LHS and RHS, we separately attack  $f$  as well as the MSE on  $g_{\theta}$  where in both cases the objective uses the MSE. For these attacks, we again utilize  $L_{\infty}$ -PGD, but also  $L_{2}$ -PGD attacks, both based on the foolbox [45] implementation, with 50 steps and a relative step size of 1.0. The attack strength  $\epsilon$  is set to  $\epsilon = 16 / 255$  for both the attack types. As can be seen, LHS is indeed always upper bounded by RHS. Note that, for each pretraining and attack type combination, the RHS is the same over all latent factors, i.e. in each row, because the same images and the same representation function are used.
Pretraining	Attack Type	Scale	Orientation	posX	posY
Adversarial	L∞-PGD	LHS	0.10	0.09	0.02	0.02
RHS	2.29	2.29	2.29	2.29
Standard	L∞-PGD	LHS	3.32	1.20	2.02	2.95
RHS	23.80	23.80	23.80	23.80
Random	L∞-PGD	LHS	2.33	2.29	2.43	1.84
RHS	29.25	29.25	29.25	29.25
Adversarial	L2-PGD	LHS	0.004	0.003	0.001	0.001
RHS	0.15	0.15	0.15	0.15
Standard	L2-PGD	LHS	0.33	0.22	0.17	0.19
RHS	3.15	3.15	3.15	3.15
Random	L2-PGD	LHS	0.19	0.21	0.14	0.12
RHS	1.69	1.69	1.69	1.69
B.3 Classification Criterion - Attack Strength
Here, we study the hypothesis stated in Sec. 5.2 that datapoints fulfilling the bound in Thm. 2 are still robust against an increased attack strength. We repeat the experiments from Sec. 5.2 for the model pretrained on CIFAR-100, where the attack strength is  $\epsilon = 8 / 255$ , since, for this case, a subset of each dataset fulfilled the equation. Additionally, we compute the robust accuracy against stronger attacks employing double and three times the attack strength, i.e., using  $2\epsilon = 16 / 255$  and  $3\epsilon = 24 / 255$ .
In Tab. 8, we observe that indeed most of the datapoints fulfilling the bound in Eq. (1) are still robust against the doubled attack strength, which is suggested by the robust accuracy for  $2\epsilon$  being larger than the proportion that fulfilled the bound. Most of these datapoints, approximately  $50\%$ , are even robust against  $3\epsilon$ . Further analysis is needed to better understand the observed robust accuracy scores and to derive a precise connection to the bound from Thm. 2.
Table 7: Left-hand side (LHS) and right-hand side (RHS) of Lemma 1 using a model pretrained on the orientation latent factor of dSprites.
attack strength	CIFAR-10
CIFAR-10	Fashion-MNIST	Intel Image
clean accuracy	0	70.9%	84%	78.5%
robust accuracy	€	42.2%	56.2%	56.8%
prop. fulfilled	€	11.3%	4.6%	27.1%
robust accuracy	2€	18.4%	28.5%	31.4%
robust accuracy	3€	6.6%	13.1%	13.4%
Table 8: For the CIFAR-100 pretraining dataset, we report clean accuracy, the proportion of images fulfilling the theoretical bound, and robust accuracy for several attack strengths for each of the CIFAR-10, Fashion-MNIST and Intel Image downstream tasks.
B.4 Comparison Study between LP, FT and LP-FT
Here, we compare robustness transfer on classification tasks for three transfer learning techniques, linear probing, full finetuning, and linear probing then finetuning [35]. As explained before, linear probing (LP) refers to freezing all pretrained weights and only optimizing the linear classification head. In full finetuning (FT) all pretrained parameters are updated during transfer learning. Linear probing then finetuning (LP-FT) combines LP and FT sequentially, as after LP is done, FT is applied to all parameters.
For the model pretrained on CIFAR-100, we choose a batch size of 128, and run 20 epochs for FT, LP and 10 epochs of FT after LP for LP-FT. For the model pretrained on ImageNet, we choose a batch size of 32 to compensate for the increase in input resolution to  $256 \times 256$ . We run 10 epochs for FT and LP and 5 epochs for LP-FT. In both settings, we use half as many epochs for finetuning
after linear probing in LP-FT, as in [35]. For all methods we apply a cosine annealing learning rate schedule [39], with an initial learning rate of 0.01 for LP on CIFAR-100, and 0.001 for all other scenarios. We use stochastic gradient descent with a momentum of 0.9 on the cross-entropy loss for all experiments.
We evaluate the adversarial robustness on the test set after training by employing the  $L_{\infty}$ -PGD attack provided by the foolbox package [45], with 20 steps and a relative step size of 0.7. As the attack strength, we choose  $\epsilon = 8 / 255$  for the model pretrained on CIFAR-100. Due to the increase in input resolution for the ImageNet model, we apply  $\epsilon = 1 / 255$ .
Tab. 5 in Sec. 6.2 shows the performance of each transfer learning method applied to the model robustly pretrained on CIFAR-100. For all datasets, LP preserves the most robust accuracy, while performing worst on natural examples. FT and LP-FT show similar natural and robust accuracy. Only for the Intel Image dataset do we observe a significant difference (10%) in the robustness of LP-FT and FT. In contrast, when pretraining on ImageNet, LP has the worst clean and robust accuracy. Instead, LP-FT has the highest robust accuracy for all datasets, while FT achieves the highest clean accuracy. Although FT has the highest clean accuracy for all datasets, the difference between FT and LP-FT is marginal.
Additionally, Tab. 9 provides confirmation that our theory indeed holds for the other transfer learning approaches as well. The difference is that for FT and LP-FT the AS score changes during transfer learning and thus the score computed prior transfer learning is no longer predictive of the transferred robustness. Still, it is an interesting observation that indeed the AS score shrinks during LP-FT when it improves robustness transfer compared to LP.
CIFAR-100
Method	ΔCE	CIFAR-10
L∞(W)	AS	ΔCE	Fashion-MNIST
L∞(W)	AS	ΔCE	Intel Image
L∞(W)	AS
FT	8.7	1.31	8.93	4.49	1.26	6.68	5.62	1.1	6.86
LP	0.73	4.4	0.98	0.8	4.59	2.11	0.47	3.05	1.05
LP-FT	14.1	4.42	6.44	5.98	4.6	4.27	3.06	3.07	3.02
ImageNet
Method	ΔCE	CIFAR-10
L∞(W)	AS	ΔCE	Fashion-MNIST
L∞(W)	AS	ΔCE	Intel Image
L∞(W)	AS
FT	4.39	1.1	12.6	2.28	1.1	9.0	0.32	0.94	3.8
LP	4.14	1.86	15.7	9.4	2.02	53.2	0.14	1.39	2.7
LP-FT	3.18	1.9	9.8	0.92	2.06	5.1	0.37	1.0	3.9
Table 9: Absolute difference between clean and robust CE loss  $(\Delta \mathrm{CE})$ $L_{\infty}(W)$  from Thm. 1 and the adversarial sensitivity score (AS) for the three transfer learning approaches using the model robustly pretrained on CIFAR-100 or ImageNet.
B.5 Distribution Shift Experiments
In this section, we analyse the impact of distribution shift between pretraining and downstream data on the adversarial sensitivity score and thereby on robustness transfer via linear probing. First, we consider the distribution shift between CIFAR-10 and ImageNet and, in particular, its dependence on how the smaller CIFAR-10 images are resized before applying the representation function. Second, we employ ideas from prior work [26] $^4$  to create increasingly corrupted versions of CIFAR-100 to simulate varying shifts in distribution.
In our experiments in Sec. 5, we resize the  $32 \times 32$  CIFAR-10 input images to  $256 \times 256$  for the ImageNet model, and observe a strong distribution shift by a severely increased AS score. Here, we evaluate if increasing the image size before applying the ResNet model causes this shift, or if it is inherent in the data. In Tab. 10, we repeat our linear probing experiment for the CIFAR-10 to ImageNet case on differently rescaled input images. The results indicate that reducing the increase in the input image size improves robustness transfer, at the cost of a drop in clean performance. However, even for a smaller input image size, the AS score remains increased indicating a tangible distribution shift remains.
<sup>4</sup>https://github.com/hendrycks/robustness/blob/master/ImageNet-C/create_c
input image	clean /robust CE	clean /robust acc	prop. fulfilled	LHS	RHS
256x256	0.22 / 4.36	92% / 16%	0.1%	1.14	15.7
128x128	0.41 / 1.31	86% / 57%	0.2%	0.28	12.7
64x64	1.04 / 1.49	64% / 48%	0.1%	0.14	8.2
Table 10: The table contains clean and robust CE loss and accuracy, the proportion of images fulfilling the theoretical bound in Eq. (1), LHS and RHS of Lemma 1, when performing linear probing from an ImageNet pre-trained model based on differently resized CIFAR-10 images.
In the following, we consider an increasingly corrupted version of CIFAR-100 to simulate distribution shift following prior work [26]. In particular, we use the Gaussian noise (Tab. 12) and Zoom blur (Tab. 13) corruption types with increasing severity to simulate an increasing distribution shift. The experimental results from both distortion methods confirm that an increasing shift between the data distributions leads to an increase in the AS score and poorer robustness transfer via linear probing. The different corruption strength are used as suggested in [26] and are stated in Tab. 11.
Approach	C1	C2	C5
Gaussian Noise	0.04,	0.06	0.10
Zoom Blur	[1, 1.06]	[1, 1.11]	[1, 1.26]
Table 11: Definition of the corruption strength considered in our experiments. The Gaussian noise corruption severity is defined by the scale, i.e. the variance of the Gaussian noise distribution. The Zoom blur is defined via the interval of zoom factors. For each 0.01 increment in the interval, a zoomed-in image is created, and all zoomed-in images are summed to create a blurred image.
Metric	CIFAR-100	CIFAR-100-C1	CIFAR-100-C2	CIFAR-100-C5
Acc.	67.3%	66.5%	65.3%	63.6%
Robust Acc.	46.8%	44.5%	43.3%	40.9%
CE	1.28	1.32	1.37	1.44
robust CE	2.1	2.21	2.3	2.44
difference CE	0.82	0.89	0.93	1.0
L∞(W)	3.31	3.34	3.36	3.47
LHS (Lem.1)	0.124	0.133	0.138	0.144
AS score	0.835	0.892	0.915	0.949
Table 12: Various metrics for transfer learning from a model robustly pretrained on CIFAR-100 to an increasingly corrupted CIFAR-100 dataset using Gaussian noise. C1 indicates severity level 1 of the corruption is applied.
Metric	CIFAR-100	CIFAR-100-C1	CIFAR-100-C2	CIFAR-100-C5
Acc.	67.3%	65.4%	65.7%	63.9%
Robust Acc.	46.8%	41.2%	40.3%	35.5%
CE	1.28	1.36	1.36	1.43
robust CE	2.1	2.39	2.42	2.73
difference CE	0.82	1.03	1.06	1.3
L∞(W)	3.31	3.36	3.35	3.42
LHS (Lem.1)	0.124	0.153	0.158	0.19
AS score	0.835	1.001	1.031	1.195
Table 13: Various metrics for transfer learning from a model robustly pretrained on CIFAR-100 to an increasingly corrupted CIFAR-100 dataset using Zoom blur. C1 indicates severity level 1 of the corruption is applied.