Understanding the Role of Invariance in Transfer Learning
Authors: Till Speicher
Vedant Nanda
Krishna P. Gummadi
Venue: Accepted by TMLR ()

Abstract: Transfer learning is a powerful technique for knowledge-sharing between different tasks. Recent work has found that the representations of models with certain invariances, such as to adversarial input perturbations, achieve higher performance on downstream tasks. These findings suggest that invariance may be an important property in the context of transfer learning. However, the relationship of invariance with transfer performance is not fully understood yet and a number of questions remain. For instance, how important is invariance compared to other factors of the pretraining task? How transferable is learned invariance? In this work, we systematically investigate the importance of representational invariance for transfer learning, as well as how it interacts with other parameters during pretraining. To do so, we introduce a family of synthetic datasets that allow us to precisely control factors of variation both in training and test data. Using these datasets, we a) show that for learning representations with high transfer performance, invariance to the right transformations is as, or often more, important than most other factors such as the number of training samples, the model architecture and the identity of the pretraining classes, b) show conditions under which invariance can harm the ability to transfer representations and c) explore how transferable invariance is between tasks.
The code is available [here](https://github.com/tillspeicher/representation-invariance-transfer).

Published in Transactions on Machine Learning Research (07/2024)
Understanding the Role of Invariance in Transfer Learning
Till Speicher MPI-SWS
tspeicher&mppiisssscccm
Vedant Nanda MPI-SWS
Krishna P. Gummadi MPI-SWS
gummadi@mpi-sun
Reviewed on OpenReview: https: // openreview. net/ forum? id= SpJTALSPIU
Abstract
Transfer learning a powerful technique for knowledge-sharing between different tasks. Recent work has found that the representations of models with certain invariances, such as to adversarial input perturbations, achieve higher performance on downstream tasks. These findings suggest that invariance may be an important property in the context of transfer learning. However, the relationship of invariance with transfer performance not fully understood yet and a number of questions remain. For instance, how important is invariance compared to other factors of the pretraining task? How transferable is learned invariance? In this work, we systematically investigate the importance of representational invariance for transfer learning, pretraining. To do so, we introduce a family of synthetic datasets that allow us to precisely control factors of variation both in training and test data. Using these datasets, we a) show that for learning representations with high transfer performance, invariance to the right transformations as, or often more, important than most other factors such as the number of training samples, the model architecture and the identity of the pretraining classcs, b) show conditions under which invariance can harm the ability to transfer representations and c) explore how transferable invariance is between tasks. The code is available https::
well how it interacts with other parameters during
// h/g.hhaw.otttaaaecceee//eeeeeetr
1 Introduction
Many learning problems are incrcasingly solved by adapting pretrained (foundation) models to downstream tasks (Bommasani et al. 2021). To decide when and how pretrained models can be transferred to new tasks, it is important to understand the factors that determine the transfer performance of their representations. The literature on transfer learning has proposed a number of factors that influence transfer performance. Among them are for instance the accuracy of model on its training dataset, the architecture and size of the model and the size of the training dataset (Kolesnikov et al. 2020 Huh et al. 2016 Kornblith et al. 2019). One surprising recent result that modcls robust adversarial attacks, i.c. invariant certain ball perturbations, exhibit higher downstream performance on transfer tasks than non-robust ones, despite achieving lower performance their training dataset (Salman et al.) 2020). Other invariances, such to textures, have also been found to boost performance (Geirhos et al. 2018a). These findings suggest invariance as another important factor that can influence transfer performance.
A model can be said to be invariant to some transformation its output or representations do not change in response applying the transformation to its input. Invariance has been recognized as an important property of models and their representations and consequently has received lot of attention (Cohen al. 2019 Bloem-Reddy & Teh 2020 Lyle et al. 2020 Fricsson al. 2021b). Most of this work, however,

---

Published in Transactions on Machine Learning Research (07/2024)
focuses the role that invariance plays for specific task. In the case of transfer learning, on the other hand, there are two tasks involved: the task on which the model is trained and the task to which it is transferred. The effect of invariance, both learned during pretraining and required by the downstream task, on the transfer performance of representations has reccived less attention and not fully understood yet.
One reason why the relationship between invariance and transfer performance has not been more thoroughly explored is that doing so challenging, especially when only using common real-world datasets. To investi- gate invariance at fine-grate mnn-grained level, it is necessary to know the different ways in which inputs to model differ, in order to determine how those differences relate to changes in representations. This, however, is not possible with typical real-world datasets such as CIFAR-10 (Krizhevsky et al. 2009), ImageNet (Russakovsky et al. 2015 or VTAB Zhai et al. 2020j. For example, the CIFAR-10 dataset contains images of cats, but using this dataset to assess whether a model is invariant to the position or pose of cats is not possible, since no position or other information available beyond class labels.
Therefore, to study invariance carefully, we introduce family of synthetic datasets, Transforms-2D, that allows us to precisely control the differences and similarities between inputs in model's training and test sets. Using these datasets, we explore the importance of invariance in achieving high transfer performance, as well as how transferable invariance is to new tasks. Concretely, we make the following contributions:
We introduce a family of synthetic datasets called Transforms-2D, which allows us to carefully control the transformations acting on inputs. By using these datasets, we are able to train models to exhibit specific invariances in their representations and to evaluate their performance on transfer tasks that require specific invariances. We also use them the basis for measuring invariance to input transformations.
We investigate the connection between invariance and downstream performance and compare it to other factors commonly studied in the transfer learning literature, such as the number of training samples, the model architecture, the relationship of training and target classes, and the relationship of training- and target-task performance. We find that while these other factors play a role in determining transfer performance, sharing the invariances of the target task is often as or more important. We further show how undesirable invariance can harm the transfer performance of representations.
We explore the transferability of invariance between tasks and find that in most cases, models can transfer high degree of learned invariance to out of distribution tasks, which might help explain the importance of invariance for transfer performance.
While our observations derived from experiments synthetic data, we validate them on real- world datasets and find that similar trends hold in these settings.
2 Related Work
Transfer learning is well studied topic in the machine learning community. Prior work has identified
number of factors that contribute to transfer performance, such as model size (Kolennkkvv et al.] 2020:
Abnar et al. 2021), size and characteristics of the pretraining dataset (Huh et al. 2016 Kornblith et al. 2019 Azizpour et al. 2015: Neyshabur et al.) 2020 Entezari et al. 2023 and adversarial robustness (Salman et al. 20200. In this work, we investigate the impact of invariance on transfer performance more broadly as another dimension and compare its effect to the aforementioned factors. In this context, prior work has found that DNNs can have difficulties generalizing certain invariances (Azulay & Weiss| 2018 Zhou et al., 2022). Our work also aims to understand this phenomenon better by investigating conditions under which invariance transfers more closely.
Invariance and equivariance have been studied in the context of representation learning with the goals of
better understanding their properties [Kondor & Trivedi 2018 Bloem- Bloem-Reddy & Teh 2020 Lyle et al. 2020 Cohen et al.] 2019 Von Kiigelgen et al.) 2021). ). measuring them (Goodfellow et al.] 2009 Fawzi & Frossard
2015 Gopinath et al.] 2019 Kvinge et al. 2022 Nanda et al. (002)), leveraging them for contrastive learn-
ing (Wang& & Isola] 2020 Ericsson et al. 2021a and building in- and equivariant models (Benton et al.
2

---

Published in Transactions on Machine Learning Research (07/2024)
Weiler & Cesa 2019 Our work is also attempting to under- stand the implications and benefits of invariant models better. However, most prior work on understanding invariance analyzes how invariance benefits a particular task or is focussed on a specific domain (Ai et al, 2023 , whereas we are interested in understanding the relationship of invariance between different tasks more broadly. Additionally, we complement. the theoretical perspective that many prior works are offering with an empirical analysis.
2020
Cohen & Welling 2016 Zhang 2019
Data augmentations have been studied tool to improve model performance (Perez & Wang 2017;
Shorten & Khoshgoftaar 2019 Cubuk al. 2018 and to imbue models with specific invariances (Ratner ct al. 2017). Their effects have also been thoroughly investigated (Perez & Wang 2017 Chen et al. 2020a;
Huang et al. 2022 Geiping et al. 2022 Balestriero et al. 2022). In our work we leverage data augmentations to both train models with specific invariances as well as to evaluate the degree and effect invariance in their representations.
In self-supervised learning (SSL), models are trained to be invariant to certain data augmentations in order to obtain pseudo labels (Doersch et al. 2015 Zhang et al. 2016 to introduce contrast between similar and dissimilar inputs (Chen et al.) 2020b Grill et al. 2020 Kim et al. 2020 Ericsson et al. 2021b). However, invariance is often treated in an ad hoc and opportunistic manner, i.e. data transformations are selected based on how much they boost the validation performance of models. Our findings complement work that uses invariance as a building block, by assessing the importance of invariance to data transformations, relative to other factors such as the model architecture.
The robustness literature has also studied invariance extensively in order to safeguard model performance
against adversarial perturbations (Szegedy et al.) 2013 Papernot et al. 2016), natural image corruptions (Geirhos et al. 2018b; Hendrycks & Dietterich, 2019 Taori et al.) 2020 or distribution shift (Recht et al. 2019). This line of work is similar in spirit to ours, as it also shows that invariance, or a lack thereof, can have a significant impact on model performance. However, robustness research is primarily interested in avoiding performance degradations when specific transformations are introduced to a dataset, rather than understanding how the ability transfer representations wetween datasets depends on their invariance the
to transformations. Some prior works have found that too much invariance can be detrimental robustness of models (Jacobscn al., 2018 Kamath et al. 2019: Singla et al. 2021 We also investigate this phenomenon and expose conditions under which representational invariance can harm transfer performance. Additionally, the field of invariant risk minimization has investigated robustness to changes in spurious
correlations between different domains (Muandet et al. 2013 Arjovsky et al. 2019
Synthetic datasets. There have been proposals for fully synthetic datasets that allow for more careful study of the properties of representations (Hermann & Lampinen 2020 Matthoy al. (017). Our work leverages previous work by Djolonga et al. (2021) and uses it to construct dataset that allows for precise control over variations in the data.
3 Controlling and Evaluating Invariance in Representations
We begin by describing our approach controlling for and evaluating the presence of invariance in repre- sentations.
3.1 Terminology
Notation. We operate in the standard supervised learning setting and denote by D { m datasct consisting of N examples E X' C R" and associated labels y E y = f1...., K}. The task is to find a function : X' + y that minimizes the empirical risk on D. g is a neural network that is trained by minimizing the categorical cross-entropy loss over its predictions. For our purpose it is convenient to write 9 as = 9ets (.)) where grep X' + Z maps inputs & to representations E Z C R and Iet : Z + y maps representations to predictions ÿ. We will refer to grep simply as g if the meaning is clear from the context.
3

---

Published in Transactions on Machine Learning Research (07/2024)
We primarily focus on representations at the penulatimate layer that are fixed after pretraining in this work. We study fixed representations, since retraining grep would change the invariance properties of the repre- sentations, and thus would not allow us to cleanly answer the question of how the invariance properties of representations affect their downstream performance. We focus on the penulatimate layer since its repre- sentations are most commonly used for transfer learning under the linear probing regime (Kornblith et al. 2019 Salman et al., 2020 Chen et al. 2020b
Invariance. We are intercsted in the invariance of representations to transformations in model's imput. In the context of this work, define invariance as follows: Given transformation t X + X, we say that model g is invariant to t if g(((x)) = g(z) for all x E X. We say that g is invariant to a set of transformations T if it is invariant to all transformations E T.
3.2
Constructing Invariant Representations
The main approaches to construct invariant representations are training with data augmentations (Cubuk et al. 2018: Antoniou et al. 2017; Benton et al. 2020 Chen et al. 2020a) and architectures that are equi- or invariant by construction (Cohen & Welling 2016 Zhang 2019). The data augmentation approach randomly applies certain transformations to the input data during training. Since the transformations are independent from the training data, models trained this way have to become invariant. to them in order to minimize their loss. Invariant architectures on the other hand build specific inductive biases into the model architecture, that make them equi- invariant to certain transformations, such rotation translation (Worrall et al. 2017).
In this work, we choose data augmentations - i.e. input transformations - as the method to construct invariant representations. Our choice is motivated by the fact that a) training with data transformations is the most commonly used method to construct invariant representations in the literature, b) is flexible and allows us to construct invariances to any type of transformation that we can define through code (as opposed to architectural invariance, which requires different model design for each type of invariance) and c) it allows to leverage the same mechanism we use to train invariant networks to also evaluate the invariance of representations. In Section 5 we show that training with input transformations indeed leads to representations that are invariant to those transformations.
3.3 Controlling Data Transformations via Synthetic Data
To both construct representations with known invariance properties and to evaluate them on tasks with known invariance requirements, we need to be able to control the transformations present in a model's training and test data. For instance, to determine whether the representations of a model are invariant to a particular transformation, is necossary to probe it with inputs that only differ by this transformation.
Using real-world datasets for this task is very challenging for two reasons. First, information about how inputs are transformed relative to each other, for example whether objects in images differ by certain translations or rotations, is typically not available in most real-world datasets, beyond coarso-grained information like the class that an input belongs to. Second, even if such annotations were available for real-world data, they would come with the risk of confounders between transformations and other data factors, such as the objects present in the data. For example, images of huskies might all have been taken on snowy background (Ribeiro) et al. 2016). To carefully control for such confounders, data would have to be sampled in a randomized manner in a lab setting, thus diminishing realism benefits.
Therefore, in order to properly study invariance in representations, we introduce a family of synthetic image datasets that allows us to precisely control which objects are present in images, well as the transformations acting on them. We call it the family of Transforms-21 datascts.
A Transforms-2D datasel D(O,T) is defined by a set of foreground objects O and a set of transformations T, with associated distributions Po and Pr over O and T, respectively. addition, there is a set B of background images with uniform distribution PB, which is the same across all datasets sample an image from D(O, T), we sample an object o ~ Po, transformation t ~ Pr, and a background b B, and then create an image as b(l(o)), where t(o) is the transformed object and b(.) denotes pasting
the family. To
4

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Rotate
Hue
Blur
[Transformss2D examples.] Example images sampled from the Transforms-2D dataset. Each row shows a different transformation being applied to one of the object prototypes.
Figure 1:
onto the background b. Each object o E O defines class in the dataset, with each sample having as its class. That means that each class is based on single object prototype E O, and different instances of the same class are created by applying transformations from T to the prototype. Sample images for different transformations are shown in Figure D
Foreground objects and background images. Each sample from a D(O,T) dataset consists of a trans- formed foreground object pasted onto a background image. Foreground objects are a subset of 61 pho- tographs of real-world objects, such as food and household items, vehicles, animals, etc. Each object image has transparency masks, such that it only contains the object and no background. Po samples objects uniformly at random from O, We use the same set B of background images for each D(O,T), which are photographs of nature scenes, chosen uniformly at random from a set of 867 candidates. The foreground and background images are based on the S1-score dataset (Djolonga et al. 2021). We subsample images to have a resolution of 32 x 32 pixels, since this size provides enough detail for models to be able to distinguish between different objects, even with transformations applied to them, while allowing for faster iteration times in terms of training and evaluation.
Transformations. A transformation is typically a combination of multiple transformations of different lypes, e.g. translation followed by rotation. Therefore, the set of transformations T is the Cartesian product of sets of different transformation types, í.e. T = T() X x... x T(k), and transformations t E T. {()) are concatenations of the transformations of each type z(0 E T'0 We denote the cardinality of T by the number of transformation types that it uses, f.e. if T- T(),,, T((), then [T] = k. To sample a transformation / Pr; we sample transformations z(0) Pros for each type and concatenate them to form t. We use three categories of transformation types that span a comprehensive set of image manipulations: i) geometric (translate, rotate, scale, vertical flip, horizontal flip, shear), ii) photometric (hue, brightness, grayscale, posterize, invert, sharpen), and iii) corruption (blur, noise, pixelate, elastic, erasing, contrast). Additional information on and examples of the transformations, including information about their distributions Pro. can be found in Appendix A.1
= ((1) o
In our experiments we use 50,000 training samples to train models with specific invariances, as well as 10,000 validation and 10,000 test samples, if not stated differently. These numbers mimic the size of the CIFAR datasets (Krizhevsky al., 2009).
3.4 Measuring Invariance Representations
In order to measure how invariant representations are to specific transformations T, we measure how much they change in response to applying transformations from T to the input, f.e. how sensitive representations are to transformations from T. Given model g, set of transformations T and objects O, we measure the sensitivity of to T based the L2-distance
sens(g[T.O) 1
E CaI,EDD((T,O)
[1((21)) (xx2)|12]
(1)
where 21.02 D)T,O) are pairs sampled from D)T,0) as ((() and x2 b(l2(o)), for o Po, t1 ~ Pr and b ~ PB, d.e. Iı and only differ in the transformation applied to them. C' is a normalization constant, which measures the average distance between any two random samples from D(T, 0). i.e. C D.T.)D(T.0) [l]g(>) ](x)) without any sampling constraints on x, Intuitively, sens(g]T,O)
measures the ratio of the distance between two samples that only differ in their transformation, relative to
5

---

Published in Transactions on Machine Learning Research (07/2024)
the average distance between any two samples from O),,O). The lower sens(g]T, O), the more invariant the iepsssentations are to the transformations in 7. In our experiments approximate each of the expectations as an average over 10,000 sample pairs.
4 How Important is Representational Invariance for Transfer Learning?
We want to understand the factors that determine whether representation trained on one dataset transfers, Le. performs well, on another datasct. In particular, we are intorested in understanding how important invariance for transfer performance, compared to other factors, and whether the wrong invariances can harm transfer performance.
4.1 How Important is Invariance Compared Other Factors?
To better understand how important representational invariance is compared to other factors, we leverage the Transforms-2DD dataset to create training and test tasks with known required invariances. In particular, we compare invariance against the following factors: dataset size, model architecture, and the number and identily of classes.
We set up experiments that mimic the typical transfer learning setting. In each experiment, we sample disjoint sets of training and evaluation objects O₂ and O₂ for the source and target task, respectively, as well disjoint sets of training and evaluation transformations Tr and T.. Using these sets we create datasets described below and train models on training task, freeze their weights and transfer their penultimate layer representations to a target task, where we train a new lincar output layer. Both the training and target lask are classification problems based on the Transforms-2D dataset, which differ in the set of objects that need to be classified. For our experiments, we set |Ot| = |O2l = 30 (roughly half the set of 61 available objects) and [T,] = [Tel = 3, with transformation types sampled uniformly among the 18 available types of transformations in Transforms-2D.
Effect of invariance: To measure the effect of invariance on transfer performance, train pairs of models two versions of the training dataset, D. D(O,, Te) and D, D;),, respectively, whose only difference is that D, uses the same transformations Te as the target dataset, while Da uses the disjoint set of training transformations T,. This setup provides us with a "same-transforatiinss and a "different- transformations" model 9s and Jd, respectively, which only differ in the transformations in their training dataset and thus the invariances in their representations. Comparing the performance these two models on the target dataset D. D(,,,Te) after fine-tuning allows us to quantify how important having the right invariances is for the target task.
For example, the larget lask might transform objects by rolating, blurring and posterizing them (T.). In order to perform well on this task (De), model needs to learn representations that are invariant to these transformations. Models Is that are pretrained on source datasets D, with the same transformations To acquire these invariances, whereas models ga that are trained on datasets Da with disjoint transformations Tr, e.g. on dataset where objects are translated, scaled and color inverted, would not acquire the invariances required for the target task.
To compare the effect of invariance to that of the other factors, (c.g. the number of training samples) we train multiple such pairs of models ys and ga- Each pair is trained datasets D, and Da that both use a different value of the factor that we want to compare to. Comparing the within-pair with the belween-pair performance differences allows us to quantify how imporlant invariance is compared to these other factors. Details on the training and evaluation procedure can be found in Appendix B
Effect of other factors:
Dataset size: The size of the training dataset is typically considered an important factor in deter- mining how useful representations are for downstream tasks. We compare effect to invariance by training each pair of models with a different number of training samples that are drawn from D, and Dæ. We use n E (1000, 10000, 50000, 100000, 500000).
6

---

Published in Transactions on Machine Learning Research (07 (07/2024)
1.00
1.00
1.00
0.95
0.95
0.90
Accuracy
0.90
Arcuracy
0.90
0.85
0.80
disjoint, transfer disjoint, same, training
0.85
0.75
0.80
same, transfer
0.75
subset
disjoirt:
same
superset
ResNet18 ResNet50
DN121 Architectre
VGG11
VIT
1000
10000
5,0000
100000
5000D0
Class
PT Samples
(a) Invariance class relationship.
(b) Invariance architecture.
Invariance number of training samples.
Figure 2: [Impact of invariance vs other factors on transfer performance in Transforms- 2D.] Training (dotted lines) and transfer performance (solid lines) for models trained with different factors of variation and different invariances on the Transforms-21 dataset. Models trained to be invariant the same transformations as the target tasks (blue) transfer significantly better than models trained to be invariant different transformations (orange). This effect very strong compared the effect of other factors, such the number of training samples, the model architecture the relationship between the training and target classes. The reported numbers are aggregated over 10 runs.
Model architecture: The architecture and capacity of models is important for their performance, with larger models typically performing better. To compare the effect of architecture and model size, we train multiple pairs of models gs. 9d, each with a different architecture. Here, we use ResNet-18, ResNet-50, Densenet-121, VGG-11 and Vision Transformer (ViT). For more details, see Appendix B.I
Number and identity of classes: transfer Icarning, the objects in the training and target domain typically differ. To understand how much impact the difference of training and target objects has compared to invariance, are related in different ways to the target objects Oe: a subset Osub C Oz, |0aubl _/0,1| 10, disjoint set with the same cardinality Odiaj n n., 0 and [Oassyl = [O,l 30, the same O Osame O2
construct four pairs of training datasets whose objects O
and superset Osup C O₂ of O with |0zupl = 2 * 1Orl = 60.
Validation real-world data:
The experiments on Transforms-25 2D data allow us to cleanly disentangle the effect of invariance from the other factors, but come with the risk of being limited to synthetic data. To test whether our observations hold up under conditions closer to real-world scttings, we perform similar experiments as those on Transforms-2D on the CIFAR-10 and CIFAR-100 datasets. We cannot control transformations directly these datasets, but we can approximate the setup described above by applying the transformations from Transforms-21] as data augmentations. As before, we pretrain models on two versions of the CIFAR datasets, and the other with disjoint set of transformations. We use subset of 5 classes for CIFAR-10 and 50 classes for CIFAR-101 100 for pretraining and study transfer performance on the other half of the classes. Again, we compare the effect of using the same vs a disjoint set of invariances as the target task to the effect of varying the number training samples, the model architecture and the class relationship.
with the same transformations/
augmentations the target task
Results: Figure 2 compares the effect of invariance on transfer accuracy with that of the other factors (number of training samples, architecture, class relationship), on the Transforms-2D dataset. The accuracy of all models on their training tasks is very similar. However, when transferred to the target task, the models that were trained with the same transformations as the target task (i.e. to have the invariances required by the target task) outperform the models that were trained with a disjoint set of transformations by a significant. margin in all cases. We show analogous results for the CIFAR-10 and CIFAR-100 datasets in Appendix C.I in Figures and
Figure 3 quantifies the difference in transfer performance caused by invariance and compares the differences caused by the other factors for the Transforms-2D, well as the CIFAR-10 and CIFAR-100

---

Published in Transactions on Machine Learning Research (07 22244
1.05
1,00
0.12
0.05
0.12
0.19
0.8
0.19
0.7
same
0.95
0.15
0.07
Accuracs
0.7
0.17
0.19
Accurery
0.5
0.19
0.27
0.85
0.03
0.6
0 13
0.4
0.14
0.5
0:13
0.80
0.4
Class Rel.
Architecture
PT Samples
Class
Architecture
Samples
Class Rel.
Architecture
PT Samples
(a) Transforms-21
(b) CIFAR-10
(c) CIFAR-100
Figure 3: [Diference in transfer performance due to invariance other factors.] We compare the differences in transfer performance caused by representational invariance with the differences caused by changes to other factors the Transforms-2D and the CIFAR-10 and CIFAR-100 datasets (with data augmentations). Orange (blue) bars show the span of transfer performance for models Id (same-transfrrmation models 9x); for each comparison factor (class relationship, architecture, number of samples). Black bars show the difference between transfer performance means across factor values for ga and 9s, i.e. the difference in performance caused by having the same vs different invariances as the target task. Across all datasets, the difference in transfer performance due to representational invariance is comparable and often larger than the difference due to varying the other factors.
datasets. For Transforms-2D, the difference altributable to invariance is larger than that attributable to all other factors, with the exception of the different-trante tranerfttrranooootiionn model ga for clas--rlatioosship, where it is comparable. For CIFAR-10 and CIFAR-100, the difference in transfer performance due to invariance is comparable to the difference due to class relationship and architecture. It is similar or lower than the difference due to the number of training samples, but still substantial.
In the latter case of sample counts, is worth noting that the CIFAR datasets, classes consist of a diverse set of images that can be seen as different transformations of the same object (e.g. cars or birds that differ in their model or species, color, camera angle, background, etc.). With more samples per class, more diverse set of transformations of the same object and thus might be able to
the models likely become more invariant to irrelevant differences in object appearance. Therefore, comparing invariance due to data augmentations with the number of training samples, might implicitly compare explicit and implicit roprosentational invariance. This relationship highlights the difficulties in studying the effect of invariance in uncontrolled real-world settings.
In Appendix C.2 we investigate how full-model fine-tuning affects the invariance of representations, and whether fine-tuning the whole model can change the invariances learned during pretraining. We find that with small amount of fine-tuning data, representations that were pretrained with the right invariances still outperform ones that were pretrained with different: invariances. The more data models are fine-tuned on, f.e. the more fine-tuning approximates re-training the model on the target dataset, the more the the advantage of pretraining with the right invariances diminishes.
Taken together, when transferring representations between tasks, our results show that invariance as im- portant or more important than other commonly studied factors in most cases. Our findings have important implications for model pretraining and pretraining dataset selection, domains that are especially relevant in where the usage of pretrained foundation models is becoming very prevalent (Bommasani et al, 2021). Practitioners should pay special attention to the variations present in pretraining data, as well as to data augmentations, since both can strongly influence representational invariance and thus downstream performance. For instance, the high importance of invariance compared to the number and type of classes in the pretraining data shown in Figure Paland and Figure 3 means that when creating pretraining datasets, it may be beneficial allocate more effort towards obtaining larger diversity of object transformations compared to sampling more objects.
8

---

Published in Transactions on Machine Learning Research (07/)024)
4.2 Can Invariance be Exploited to Harm Transfer Performance?
The previous results highlight that having the right representational invariances can significantly benefit transfer performance. On the flip-side, they also show that the wrong invariances harm transfer performance. Prior work has demonstrated similar detrimental effects for certain types of excessive invariance (Jacobson et al. 2018). An interesting question therefore is: Could an adversary exploit invariance to harm transfer performance on specific downstream tasks?
To answer this question, combine our Transforms-2D data with the CIFAR-10 dataset (Krizhevsky al. 2009), such that either the information in the Transforms-2D) data or in CIFAR-10 is irrelevant for the target task. Concretely, we augment the CIFAR-10 dataset by pasting small versions of the Transforms-21 objects described in section onto the images in a completely random manner, i.e. uncorrelated with the CIFAR-10 labels.
Notation. We denote by X the features available in the input and by Y the category of labels that the model is trained to predict. C stands for CIFAR-10 and O for objects from Transforms-2D. X C means that the imput data is CIFAR backgrounds, and Y = C means that the task is to classify images based on the CIFAR classos. X = C + O, D.Y= = C means that the inputs are CIFAR backgrounds with objects pasted on them, with the task of classifying the inputs based on their CIFAR classes.
In total, we use four datasets which differ in their combinations of features X and labels Y: the standard CIFAR-10 dataset (X = C, = C*) the CIFAR-10 datasct with random objects pasted on it with the task of predicting CIFAR-10 classes (X (X= -X=++O0 0,Y = C), the same augmented CIFAR-10 10 dataset with the task of predicting the category of the pasted objects (X = C 0,Y = O) and a dataset with only objects pasted a black background, with the task of predicting the object category (X = O,Y O). We use 10 object prototypes that are scaled down and pasted at a random position onto the CIFAR-10 images. Example images from these datasets can be found in Appendix A.2 We train models on each of the datasets, freeze their representations, and then fine-tune and evaluate each model's last layer on each of the other datasets. We use Xp, Yp to refer to a model's I's pretraining dataset and objective and X,, Y. to refer to the dataset that its representations are transferred to.
Accuracy
Sensitivity C + C
X
C
C
O
C
0
Y
C
0
C
0.98
0.89
0.56
1.00
0.99
0.21
Pre-
C
O
0.98
0.97
0.15
1.00_a.00
1.00
0.08
training
++0 O
0.26
0.13
1.00
0.98
0. 0.13
0.99
Dataset
0
0.29
0.28
0.25
1.00_zo.00
1.00
0.11
[The impact of irrelevant features on downstream performance and invariance.] Rows show pre-training tasks, with X (i.e. Xp) demoting the features available in the training dataset and Y (i.e. Yp) the category of labels that the model was trained to predict. The accuracy columns denote the transfer accuracy after finc-tuning on the respective dataset (X,) at predicting the respective label (Yz). The sensitivity values show how sensitive resp. invariant the models' representations are according to the sens- metric defined in Section 3.4on the C- O data when changing the CIFAR background images (Y; = C) while keeping the pasted foreground objects fixed, and while changing the pasted object (Y, = O) while keeping the CIFAR background fixed. Lower values mean higher invariance. Observations: The representations of models pretrained with access to features that are irrelevant for their target task (objects O for X, = C+ O,Y, = C' and CIFAR backgrounds C' for X, C' Y,Y, = =) transfer worse to tasks where those features are important than their counterparts that did not have access to those features, i.e. X, = C,Y, C and X,, = O, Y, = O. The sensitivity scores show that the difference in performance is due to representations bccoming invariant fcatures that are not relevant to the pre-training objective, í.e. low sensitivity resp. high invariance for Xp- = C + O models towards the category Y, # Y, they were not trained on, compared to high sensitivity towards the category Y, = Yp they were trained on. Note that all models achieve 100% accuracy on the X; O, Yt O task, since they just have to separate 10 distinct object images.
Table 1:
9

---

Published in Transactions on Machine Learning Research (07 (204))
Model Type
Transformation Relationship
Synthetic Datasets
Real-
World Datasets
In- In-Distribution
Mild OOD
Strong OOD
CIFAR-10
CIFAR-100
Same
0.14
0.15 10.07
0.57 0. 10.16
0.37 10.21
0.3511..18
Image
Other
0.40+a.15
0.39
0.69
0.50=0.20
0.49
None
0.
0.42+tu.15
0.68
0.44
0.45
Same
0.12 1.1220.08
0.44 1.116
0.24 0.100 0.110
0.39 10.22
0.36 10.20
Random
Other
0.64
0.68 16
0.33 11
0.46-0.21
0.45+0.20
None
0.65
0.694u.17
0.35+0.12
0.50
0.49
Table 2: [Invariance transfer under distribution shift.] Each cell shows the average sens-score (lower is more invariant), for models trained on image and random data, under distribution shift. "Transformation Relationship" refers to the relationship between the training and test transformations of the models. Rows labeled as "Same" show how invariant models are lo their training transformations on each of the dalasets (e.g. translation-traaied model evaluated on translated data), whereas rows labeled as "Other" show how invariant they are on average to transformations other than the ones they were trained on (e.g. a iissslationttriinee model on rotations). "None" rows show the baseline invariance of models trained without transformations, i.e. simply to classify untransformed objects. The most invariant models in each category are shown in bold. Models are significantly more invariant to the transformations they were trained on than to other transformations, and this relationship persists on mild and strong OOD, as well as real-world data, indicating that a medium to high degree of representational invariance is preserved under distribution shifts.
Results: Table I shows the transfer accuracies of each model on each of the datasets. The main takeaway is that the transfer performance of the models differs significantly, depending on what information they have access to during pretraining, í.e. what features were available, and how relevant they are for the pretraining task. The impact that the relevance of input information has on transfer performance can be seen by comparing the two models trained on the augmented CIFAR images (X, C'+ O). The model pretrained to predict CIFAR labels (Y, = C) performs well on this task (Y, = Y, = C), but poorly at predicting objects (Y = O), whereas the inverse relationship holds for its counterpart pretrained to predict objects (Y₁ O). The sensitivity scores (based on the sens-metric) show that that the representations of both models during pretraining become invariant to the irrelevant information in their inputs (objects or CIFAR backgrounds, respectively) and thus their representations cannot be used anymore to predict the corresponding classes (Y, Yp). Additionally, models that had access to irrelevant features during pretraining (X, = C++) achieve lower transfer performance at predicting the category of classes they were not pretrained for (Y, * Y)) than models pretrained to predict the same category Y,, but without access the irrelevant features (X, C and X, =0 O).
The results show that during pretraining, the representations of models become invariant to information that is available in the input but irrelevant for the pretraining task. This effect can be exploited to harm transfer performance by introducing invariances to features that are relevant for downstream tasks into the representations. We further examine the relationship of relevance and availability with transfer performance in Appendix CR We find that more relevance gradually leads to less invariant representations, but that even if irrelevant objects are only available in few inputs, models already become significantly more invariant to them.
In summary, our results show that invariance significantly affects how well representations transfer to downstream tasks. Its effect on transfer performance can be both positive, when representations share the right invariances with the target task, and negative, when they lack the right invariances or - even worse possess invariance towards important information in the inputs.
5 How Transferable is Invariance?
So far, we have shown that sharing invariances between training and target tasks is quite important for the transfer performance of representations. But why does invariance have such an outsized influence? Our hypothesis is that invariances transfer well under distribution shift. If invariances learned on pretraining
10

---

Published in Transactions on Machine Learning Research (07/2024)
task are largely preserved in different domains, that would make them more robust than specific features that might change from domain to domain, and it might help to more robustly detect features that are present across domains.
5.1 Invariance Transfer Under Distribution Shift
To test how well invariance in pretrained representations transfers under distribution shift, we train models to be invariant to specific transformations using the Transforms-2D dataset and evaluate their invariance on out of distribution tasks. We create two categories of out of distribution (OOD) datasets: a mild OOD category with only small differences from the training dataset and a strong OOD category that is very different. Concretely, we create the mild OOD datasets by sampling different set of image objocts than the ones used for training, For the strong OOD datasets we use structured random data, by sampling a specific random pattern with pixel values distributed uniformly random background patterns, also sampled uniformly at random. Note that we can apply transformations to the random objects in the same way as to the image objects in Transforms-2D. Using this setup we train a different model for each of the 18 transformations in Transforms-21 i.e. such that each of the models is invariant to a different transformation, and evaluate its invariance on each of the transformations, for each datasel calegory. We use ResNet-18 models here but report similar results for other archilectures in Appendix D.2 To measure the invariance resp. sensitivity of model's representation a particular transformation, use the sens-metric defined in Section 3.4
random for each class and then pasting it
We also study the reverse direction of invariance transfer, by training models on the random strong OOD data described above, and evaluating their performance on OOD data analogously but in reverse, t.e mild OOD data uses a different set of random objects and strong OOD data is the Transforms-2D data for these models. In addition to the synthetic datasets, we also evaluate how well invariance transfers to real- world datasets, i.e. to CTFAR-10 and CIFAR-100, by using the Transforms-2D) transformations as data augmentations. Additional details can be found in Appendix [.I]
Results: Table 2 shows the invariance of the two types of models on each of the dataset categories. We see that the invariance of models to the transformations that they were trained on is consistently higher than to other transformations on all datasets. Models trained to be invariant to the target transformations are also consistently more invariant than the baseline models trained without transformations across all the datasets. This shows that models do, in fact, acquire a significant degree of invariance to their training transformations, and are subsequently able to transfer it to other distributions. However, it scems to be more difficult to transfer invariance to random data (strong OOD for the image model and mild OOD for the random model) than to image data. It is also interesting to note that the image and random models achieve very similar the real-world datasets. This suggests that even training structured random
degrees of invariance data can be a good prior for learning invariant representations, provided that the necessary transformations can be expressed on this type of data. Additional results and breakdowns of invariance over individual transformations can be found in Appendix D.2
5.2 Invariance Mismatch between Training and Target Tasks
A different type of distribution shift happens when there is a mismatch in the transformations present in training compared to the target task, which we suspect often happens in real-world settings. To better un- derstand these cases, we investigate the effect of training models on sub- and supersets of the transformations required by the target task. We create nested sets of transformations T, i (1,...,,8) ,8, such that T, C Tj for j j and [T]| = i. For each Tú, we train model (as described in Section 4.1 and then evaluate its transfer performance on data for each of the Ti'..
Results in Figure 4 show that being invariant to more than the necessary transformations does not nega- lively impact performance (as long as those invariances do not conflict with the larget task as in Section 1.2 However, if any required invariances are missing in the representations, a models' performance quickly de- creases. This can help to explain why pretraining often successful in practice: the invariances learned during pretraining do not need perfectly match those required by the target task, as long as they cover superset of required invariances. We hypothesize that commonly used pretraining datasets (such as Ima-
11

---

Published in Transactions on Machine Learning Research (07/202) 2024)
1.0
0.9
Tansfer accuracy
0.8
0.7
0.6
0.4
0.3
0.2
3
5
Transfer Transfomss
Figure 4: [ResNet-18 models trained on nested sets of transformations and evaluated on datasets with super- and subsets of those transformations. Models trained on data with the same set or a superset transformations as the target dataset consistently achieve almost 100% accuracy. However, models trained with only subset of the transformations show considcrably lower performance that decrcases the smaller the subset of training transformations compared the larget task. The results show that learning superset of required invariances does not harm transfer performance but that missing required invariances degrades transfer performance.
geNet), together with data augmentations, induce large set of broadly useful invariances. We show results for ResNet-18 models here and report very similar results for other architectures in Appendix D.3
6 Conclusion
We study the importance of invariance in transfer learning by using a family of synthetic datasets, Transforms-2D, that allows us to precisely control differences between imput points. By leveraging this method, we are able to show that invariance a crucial factor transfer learning and often as or more important than other factors such the number of training samples, the model architecture and the class relationship of the training and target task. Sharing the right invariances with the target task positively impacts transfer performance, while lack of the right invariance even invariance towards important input features harms transfer performance. We further investigate the transferability of invariance under distribution shift and find that in most cases, models can transfer a high degree of invariance to new settings. Overall, our findings show that for transfer learning to be successful, the training and target tasks need to share important invariances.
Limitations. Since achieving precise control over data transformations is not possible with real-world data, our experiments heavily rely synthetic data (see the discussion in Section 3.3). However, results derived from synthetic data might not generalize exactly to practical settings. To address this limitation we include validation experiments on real-world datasets in Sections 4.1 and 5.1 that show that the observations made using synthetic data can be largely extrapolated to more realistic settings as well.
Broader Impact. Our work primarily investigates invariance at a foundational level, and does not directly propose specific applications. However, we demonstrate in Soction 4.2 that invariance can be exploited harm transfer performance. An adversary might techniques based on this insight to create models that do not easily transfer to particular downstream tasks. On the other hand, the same insight could also be used by model developers for alignment purposes, i.e. to make models less adaptable to certain harmful downstream applications.
12

---

Published in Transactions on Machine Learning Research (07 (024))
References
Samira Abnar, Mostafa Dehghani, Behnam Neyshabur, and Hanie Sodghi. Exploring the limits of large scale pre-training, orXiv preprint arXi::2100. 02095, 2021.
Bo Ai, Zhanxin Wu, and David Hsu. Invariance is key to generalization: Examining the role of representation in sim- sim-to-real transfer for visual navigation. arXiv preprint arXiv:2310.15020 2023.
Antreas Antoniou, Amos Storkey, and Harrison Edwards. Data augmentation generative adversarial net- works. arXiv preprint wrXiv:1111.04444 2017.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint xr:: 02803, 2019.
Hossein Azizpour, Ali Sharif Razavian, Josephine Sullivan, Atsuto Maki, and Stefan Carlsson. Factors of transferability for a generic convnet representation. IEEE transactions on pattern analysis and machine intelligence, (8(9):1900-180)) 2015.
Aharon Azulay and Yair Weiss. Why do deep convolutional networks generalize poorly to small image transformations? arXiv preprint arXiv: 12177, 2018.
Randall Balestricro, Ishan Misra, and Yann LeCun. A data-augmentation is worth thousand samples: Exact quantification from analytical augmented sample moments. arXiv preprinl arxiv:2202. 08325, 2022.
Gregory Benton, Mare Finzi, Pavel Ixmailov, and Andrew G Wilson. Learning invariances in neural networks from training data. Advances in neural information processing systems, 33:17605 11617665--1666,, 2020.
Benjamin Blddm and Yee Whye Teh. Probabilistic symmetries and invariant neural networks. The Journal of Machine Learning Research, (((1):333-3350) 2020.
Rishi Bommasani, Drew Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, al. On the opportunities and risks of foundation models. Xiv preprint arXic:2108.0720 2021.
Shuxiao Chen, Edgar Dobriban, and Jane H Lee. A group-theoretic framework for data augmentation. The Journal of Machine Learning Research, 21(1):9885-995) 2020a.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive lcarning of visual reprosentations. arXiv preprint arXiv:2002.05700 2020b.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference on machine learning, pp. 2990 2999. PMLR, 2016.
Taco Cohen, Mario Geiger, and Maurice Weiler. A general theory of equivariant cnns on homogeneous spaces. Advances in neural information processing systems, 32, 2019.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv: 09501, 2018.
Josip Djolonga, Jessica Yung, Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D' D'mor Dan Moldovan, Sylvain Gelly, Neil Houlsby, Xiaohua Zhai, and Mario Lucic. On robustness and transferability of convolutional neural networks. In 2021 JEFE/CVE Conference on Compuler Vision and Pollern Recognation (CVPR), pp. 16453-16463, 2021. doi:
Carl Docrsch, Abhinav Gupta, and Alexci A. Efros. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), December 2015.
13

---

Published in Transactions on Machine Learning Research (07/2024)
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa. Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words:
Transformers for image recognition at scale. arXiv preprint arXic:2010. 1e02110..19999 11929, 2020.
Rahim Entezari, Mitchell Wortsman, Olga Saukh, M Moein Shariatnia, Hanie Sedghi, and Ludwig Schmidt. The role of pre-training data in transfer learning. arXiv preprint wriiv2332.13602 2023.
Linus Ericsson, Henry Gouk, and Timothy M Hospedales. Why do self-supervised models transfer? investi- gating the impact of invariance downstream tasks. arXiv preprinl arXiv:2111.11390 2021a.
Linus Fricsson, Henry Gouk, and Timothy M Hospedales. How well do sels-supervised models transfer? Tn Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5414 5423, 2021b.
William Falcon and The PyTorch Lightning team. PyTorch Lightning, 3 2019. URL https://github... com/
Lightning- AT/lightning
Alhussein Fawzi and Pascal Frossard. Manitest: Are classifiers really invariant? CoRR, abs/ 15070507. 2015. URL
/tsp///bb/15500065555 06535
Jonas Geiping, Micah Goldblum, Gowthami Somepalli, Ravid Shwartz-Ziv, Tom Goldstein, and Andrew Gor- don Wilson. How much data are augmentations worth? investigation into scaling laws, invariance, and implicit regularization. XrXiv preprinl arXiv:2210.0644 2022.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trainee cnns are biased towards texture; increasing shape bias improves accuracy and robustness. Xiv preprint ar 1rxi::18111 12231, 2018a.
Robert Geirhos, Carlos R. M. Temme, Jonas Rauber, Heiko H. Schütt, Matthias Bethge, and Felix A. Wichmann. Generalisation in humans and deep neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Aduances in Newral Information Processing Systems. volume 31. Curran Associates, Inc., 2018b. URL
09115886444055565ss1ff6ddddasaa Pap.eeaaeeedaaaaaa....ppp. pdf
lan J. Goodfellow, Quoc V. Le, Andrew M. Saxe, Honglak Lee, and Andrew Y. Ng. Measuring invariances in deep networks. In Proceedings of the 22nd International Conference Newral Information Processing Systems, NIPS'09, pp. 646-654, Red Hook, NY, USA, 2009. Curran Associates Inc. ISBN 9781615679199.
Divya Gopinath, Hayes Converse, Corina S. Pasareanu, and Ankur Taly, Property inference for deep neural networks, 2019. URL https: /tttt:://arttiiiii httppss///rxxv.ooo/ hrrg/abs/1904.. 13215
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33:21271 21284, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 770-778, 2016.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness common corrup- tions and perturbations. In International Conference Learning Representations, 2019. URL https:
///oenreeiee..nt///orom wtt.://ooorummidddoottat.aoo..
Katherine Hermann and Andrew Lampinen. What shapes feature representations? exploring datasets, architectures, and training. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (ods.), Advances in Neural Information Processing Systems, volume 33, pp. 9995- 10006. Curran Associates, Inc., 2020. URL 719
14

---

Published in Transactions on Machine Learning Research (07/2024)
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolu- tional networks. In Proceedings of the TEEE conference on computer vision and pollern recognilion, pp. 2017.
Kevin H Huang, Peter Orbanz, and Morgane Austern. Quantifying the effects of data augmentation. arXiv preprint arXim:2202.0913 09134, 2022.
Minyoung Huh, Pulkit Agrawal, and Alexei A Efros. What makes imagenet good for transfer learning? arXiv preprint arXio: 08614, 2016.
Jörn-Henrik Jacobsen, Jens Behrmann, Richard Zemel, and Matthias Bethge. Excessive invariance causes adversarial vulnerability. arXiv preprint arXiv:1811000401, 2018.
Sandesh Kamath, Amit Deshpande, and KV Subrahmanyam. Invariance robustness of neural networks. 2020. In URL https://openreviie. net/forum, 2019.
Minseon Kim, Jihoon Tack, and Sung Ju Hwang. Adversarial self-suporvised contrastive learning. arXiv preprinl ar xixv:2006.00555 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ar preprint 141iv:1412. 6980, 2014.
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. In Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 28-28, 2020, Proceedings, Port V 16, pp. 491-507. Springer, 2020.
Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In International Conference on Machine Learning, pp. 2747 2755. PMLR, 2018.
Simon Kornblith, Jonathon Shiens, and Quoc Lc. Do better imagenet models transfer better? In Proceedings of the IEEE/CVF conference computer vision and poltern recognition, pp. 2661-2671, 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ananya Kumar, Aditi Raghunathan, Robbic Jones, Tengyu Ma, and Percy Liang. Fine-tuning can distort pretrained features and underperform out-of-distribution. arXio preprind arXiw:2202. 10054, 2022.
Henry Kvinge, Tegan Emerson, Grayson Jorgenson, Scott Vasquez, Timothy Doster, and Jesse Lew. In what ways are deep neural networks invariant and how should we measure this? In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
URL https::/ wtt.:t/oooruniiisSS000nn3
Πya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint SEE:::6088039883 2016.
Clare Lyle, Mark van der Wilk, Marta Kwiatkowska, Yarin Gal, and Benjamin Bloem-Reddy. On the benefits 00178
invariance in neural networks, 2020. URL hattps://arxvv.orgg
Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dspritte: Disentanglement testing http://www..o//edemmm///////d//d
sprites dataset.
2017.
Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In International conference on machine learning, pp. 10-18. PMLR, 2013.
Vedant Nanda, Till Speicher, Camilla Kolling, John Dickerson, Krishna P. Gummadi, and Adrian Weller. Measuring representational robustness of neural networks through shared invariances. In ICML, 2022.
15

---

Published in Transactions on Machine Learning Research (07/2024)
Behnam Neyshabur, Hanic Sedghi, and Chiyuan Zhang. What is being transferred in transfer learning?
Advances in neural information processing systems, 33:512-505 523, 2020.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami.
The limitations of deep learning in adversarial settings. In 2016 IEEE European symposium on securily and privacy (EuroS8P), pp. 372 387. IEEE, 2016.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High- Performance Deep Learning Library.. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché Buc, E. Fox, and R. Garnett (eds.), Advances in Neurol Information Processing Systems 32, pp. 8035. Curran Associates, Inc., 2019. URL http::/ www.meee.ucppapppeeeeee wrttrorddraammmeerrtteeessss-rro
deepaaaeeedde---eiinnneeeeee-e -ibrrrygggibrr....o pdf
Luis Perez and Jason Wang. The effectiveness of data augmentation in image classification using deep learning, 2017. URL http:://arxxx..or///////// 1btp://.abb/12.04161111
Alexander J Ratner, Henry Ehrenberg, Zeshan Hussain, Jared Dunnmon, and Christopher Ré. Learning to compose domain-speciffe transformations for data augmentation. Advances in neural information process-
ing systems, 30, 2017.
Benjamin Recht. Rebecca Roelofs, Ludwig Schmidt,, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pp. 5389 5400. PMLR, 2019.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin." "wyy should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211-22) 252,, 2015. doi: 10.1007 /s1 11111-1111--0166
Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversarially robust imagenet models transfer better? Advances in Neural Information Processing Systems, 33:3533- 3545, 2020.
Connor Shorten and Taghi M. Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data, 6(1):60, 2019. doi: 10.1886/440337.000.01177... URL https://doi. 840537-019-0199-0
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arxim:
1 1499. 1556, 2014.
Vasu Singla, Songwei Ge, Basri Ronen, and David Jacobs. Shift invariance can reduce adversarial robustness. Advances in Neural Information Processing Systems, 34:1858 -17551871,, 2021.
Christian Szegedy, Wojciech Zaremba, Ilya Sulskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.66 6199, 2013.
Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Measuring robustness to natural distribution shifts in image classification. In Advances in Neural Information Processing Systems, 2020. URL https://prroceedings wwwpeeeddinnnn.uuuss..o.m www.ine.../.paaaeeeeee htt.rpp..c/ppppp////0///111. pdf.
Julius Von Kügglgen, Yash Sharma, Luigi Grescle, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style. Advances in neural information processing systems, 34:16451-1644 16467, 2021.
16

---

Published in Transactions on Machine Learning Research (07/2024)
Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning, pp. 9929 29999999. PMT.R, 2020.
Maurice Weiler and Gabriele Cesa. General (2)-equivariant, steerable cnns. Advances in Neural Information Processing Systems, 32, 2019.
Daniel E Worrall, Stephan J Garbin, Daniyar Turmukhammbeov,, and Gabriel J Brostow. Harmonic networks: Deep translation and rotation equivariance. Proceedings of the IEEE Conference on Computer Vision and Pallern Recognition, pp. 5028 5037, 2017.
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikow, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, Lucas Beyer, Olivier Bachem, Michael Tschannen, Marcin Michalski, Olivier Bousquet, Sylvain Gelly, and Neil Houlsby. The visual task adaptation benchmark, 2020. URL httppp//pppauaeeeeeeeeeee/rrrrdd
Richard Zhang. Making convolutional networks shift-invarianl again. In International conference on machine learning, pp. 7324-7334. PMLR, 2019.
Richard Zhang, Phillip Isola, and Alexei A. Efros. Colorful image colorization. In Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (eds.), Computer Vision - ECCV 2016, pp. 649-666, Cham, 2016. Springer International Publishing. ISBN 978-3-319-46487-9.
Allan Zhou, Fahim Tajwar, Alexander Robey, Tom Knowles, George J. Pappas, Hamed Hassani, and Chelsea Finn Do deep networks transfer invariances across classes? In International Conference Learning Representations, 2022. URL
17

---

Published in Transactions on Machine Learning Research (07 (07/2024) 2024)
A Dataset Details
A.1 Transforms-2D Dataset Details
Transforms-2D datasets D)O,T) are paramcterized by a set of foreground objects O and a set of transfor- mations of those objects T. An image is sampled by choosing a foreground object uniformly from set of objects O, sampling a transformation from T and pasting the transformed object t(o) onto a background image chosen uniformly at random.
Foreground objects and background images. Foreground and background images are based on the ST-Score dataset Djolonga et al. (2021), which was introduced to study the connection between out-of- distribution and transfer performance. The creators of the S1-Score dataset have curated 61 foreground cat- egories (such banana, jeans, sock, etc.) with total of 614 object images, and 867 background images of nature scenes. The dataset is available under the following URL: ttttps://githu: comeuu..oommgge www.m//ooooooeeerrrreeoe It uses the Apache 2.0 license.
8i si-score
There are several images for each foreground category, e.g. several images of bananas, and each image has transparency mask such that images only contain the object and no background. To be able to precisely control the variation between different images and ensure that any changes in object appearance are only due to the transformations that applied to them, we only use one image per foreground calegory through- out the analysis. In practice, we simply choose the image whose file name has the lowest lexicographical rank. For cach paramcterization D(O,T) of the Transforms-2D datasct, we always use the same set of 867 background images (hence they do not appear as a parameter), but choose a subset O out of all available 61 objects categories.
Transformations. To cover a reasonably large variety of transformations and to ensure that our results are not overly dependent on the specific choice of transformations, we use three categories of 2D transformations: geometric, photometric and corruption transformations. Geometric transformations tions of the object geometry, photometric transformations change the color of the object and corruption transformations, inspired by Hendrycks & Dietterich (2019) degrade the quality the object. Each category consists of six transformation types, e.g. rotation is part of the geometric transformations, and for each transformation type, there can be potentially a large or infinite number of actual transformations (e.g. there are infinitely many rotations, one for each possible angle). Transformations are mainly implemented via standard PyTorch (Paszke et al., 2019) data augmentations. An overview over the transformations, their parameters and samples for each of them is shown in Figure
affine transforma-
We scale all images to resolution of 32 32, which mimics the resolution of the CIFAR-10 and CIFAR-100 datasets Krizhevsky et al. (2009). Resolution is configurable though, and it is also possible to sample images with considerably higher resolution. We do not use additional data augmentations to train models, since that would interfere with the transformations present in the datasets. For most experiments, we use 50,000 training, 10,000 validation and 10,000 test samples, again mimicking the CTFAR datasets.
A.2 Additional Information the CIFAR- + Transforms-20 2D Dataset
F Figure e showw example images for the augmented CIFAR-10 images that in the analysis in section
4.2
investigate the effects of irrelevant information learned invariances.
The original paper
Djolonga al. 2021) mentions categories, but publicly available dataset only contains categories.
18

---

Published in Transactions on Machine Learning Research (07 (27/))
Category
Transformation
Parameters used
Samples
Translate
and position by [0%, 50%] image size
Geometric
Rotate
by [0,360] degrees
Scale
to [40%, 100%] size
Shear
and y by [-50, 50) degrees
Vertical flip
with 50% probability
Horizontal flip
with 50% probability
Hue
deviation in [-0.5.0.5]
Photometric
Brightness
change in [-1,1]
Grayscale
with 50% probability
Posterize
with 50% probability to bit
Invert
with 50% probability
Sharpen
with probabilly 50% sharp- ness factor
Gaussian blur
kernel size pixels, [0.1,155
Corruption
Gaussian noise
with / = 0, = 1, probability 50%
Pixelate
to half resolution, probability 50%
Elastic distortion
a 150, probability 50%
Erasing
square with 14 x pixels size at random position, probability 50%
Contrast
change in [-1,1]
Figure 5: [Transforms-2D transformations.] Categories, transformation types, transformation parame- ters and samples generated using the transformations for the Transforms- 2D dataset.
19

---

Published in Transactions on Machine Learning Research (07 (07/202) /024)
CIFAR-1O-only
CIFAR-10 with Transforms-2D objects
Transforms-2D-only
Figure 6: [Examples of test images in the irrelevant feature analysis]. From top to bottom: CIFAR-
only (X C), CIFAR-10 with pasted objects (X = + O), objects only (X = O)
B
Additional details on the training and evaluation setup
B.1 Architectures
For most of the experiments in the paper we use ResNet-18 models (He et al. 2016). They are generally sufficient for the 32 x 32 input size that we use for the Transforms-2D datasel and achieve close to 100% accuracy on their training distributions.
We also compare the importance of invariance with other pretraining factors, including architectures in Sec-
tion 4.1 and in Appendix C.1 For this, we additionally use ResNet-50 (He et al. 2016). DenseNeet-ll 121 Huang
al. [2017), VGG-11 [Simonyan & Zisserman (2014) and Vision Transformer (ViT) Dosovitskiy al. (020
models. All models are implemented in Py (Paszke et al. 2019). For all CNN models use implemen-
tations adapted for CIFAR datasets available here /https://iite
cow//kuanggluu/pytoorrrr cifar For
ViTs, we use an implementation from this repository /tttps///iithu/ /ttp//gttttttt/tt////g//eg com
(achiev-
ing more than 80% accuracy on CIFAR-10) with a patch size of 8, 7 layers, 384 hidden and MLP units, 8 heads and no Dropout.
B.2 Training
We train models using Pytorch Lightning (Falcon & The PyTorch Lightning team 2019 on the Transforms- 2D dataset for 50 epochs and fine-tune their output layer (while keeping the rest of the network frozen) for 200 epochs. We find that 50 training epochs are sufficient for models reach close to 100% accuracy. For fine-tuning we choose larger nunber of 200 epochs, because models that have been pretrained with invariances different from those required by the target task typically converge only slowly. Models that have been trained with the same invariances the other hand converge much faster.
All CNN models are trained and fine-tuned using the Adam optimizer (Kingma & Ba 2014 with a learning rate of 0.001. For ViTs, we use cosine learning rate scheduler [Loshchilov & Hutter 2016 with a learning that is decayed from 0.001 0.00001 over the duration of training, and a weight decay of 0.00001, with warmup epochs. We keep the checkpoint that achieves the highest validation accuracy during training and fine-tuning.
20

---

Published in Transactions on Machine Learning Research (07 (07/202)) /024)
0.8
0.8
0.8
0.7
Accuracy
0.7
0.6
same, transfer disjoint, disjoint, transfer
0.4
same, training
0.4
subset
diseinnt
same
superset
ResNet28 ResNet50 DN1 VGG11
VIT
50
200
1000
4500
Class Rel.
Architecture
PT Samples
(a) Invariance class relationship.
(b) Invariance architecture.
(c) Invariance number of training samples per class.
Figure 7: [Invariance to data augmentations vs the importance of other factors on CIFAR- 10.] Training and transfer performance for models trained with different factors of variation and different transformations on the CIFAR-10 dataset.
0.8
0.7
0.7
Arcumey
0.5
0.5
0.5
04
disjoint, trainig
disjoint, transfer
training
03
0.2 ResNett8 ResNe:50
subset
discinn
DN121 Architacture
VGG11
100
250
Rel.
Samples
(a) Invariance vs class relationship.
(b) Invariance vs architecture.
(c) Iuvariance vs number of training samples per class.
Figure 8: [Invariance to data augmentations the importance of other factors CIFAR- 100.] Training and transfer performance for models trained with different factors of variation and different transformations on the CIFAR-100 dataset.
B.3 Experiments
We repeat each experiment times and report mean and variance values. Shaded regions the plots indicate 95% confidence intervals, and annotations in the tables indicate one standard deviation. During each run, we randomize the configuration of the Transforms-2D dataset and the sampling process. For the configuratio,, this means that sample different sets of objects O and different sets of transformation types in T. For the sampling process, we randomize the order of sampling foreground objects, transformations of each transformation type and background images.
C Additional Results on the Importance of Invariance for Transfer Performance
C.1 Real-world Experiments on Augmented CIFAR data
We conduct experiments analogous to those described in Section 4.1 real-world CIFAR-10 and CIFAR-100 data. We observe results similar to those reported in Figure for CIFAR-10 in Figure 7 and for CIFAR-100 in Figure S In all cases, the model trained with the same transformations as the target task significantly oulperforms its counterpart pretrained with a disjoint set of transformations (in most cases between 10% and 20%). One difference to the results reported in Section 4.1 is that in Figure 2 even the worst performing model using the same transformations as the target task outperforms the best-performing model using a disjoint set of transformations. This is no longer the case with the CIFAR models, for example the model with disjoint transformations trained with 4500 samples per class outperforms the model with the
21

---

Published in Transactions on Machine Learning Research (07 (07/2024)
1.0
1.00
1.0
0.9
0.9
Accurscy
0.90
0.0
0.8
0.85
0.7
dijjonnt transfer training
0.80
0.6
0.7
same, training
0.75
same, trandfer
0.70
0.6
subset
disjaint
same
superset
ResNet19 ResNet50
DN121
VGG11
VIT
1000
10000
50000
100000
500000
Class
Architecture
PT Samples
(a) Invariance class relationship.
(b) Invariance architecture.
(c) Invariance vs number of pretrain- ing samples.
Figure 9: [Invariance to data augmentations vs the importance of other factors on Transforms- 2D with full finc-tuning on 200 samples.] Training and transfer performance for models trained with different factors of variation and different transformations on the Transforms-2D dataset, fine-tuned on 200 samples after linear probing.
transformations trained with 200 samples per class (but not the one with 1000 samples per class). This is somewhat expected, though, as more samples presumably help the models to better learn the invariances inherent in the different classes in the datasot, opposed to transformation-indueed invariances.
Takeaways: The results on real-world CIFAR data largely confirm our findings made using the synthetic Transforms-2D dataset. Invariance to the right transformations significantly improves the transfer perfor- mance of models and a very important factor compared other protraining dimensions such as the number of samples, the architecture and the class relationship. E.g. on CIFAR-10, you need roughly 20 times more samples to compensate for a mismatch in invariance.
C.2 How Does Fine-tuning the Whole Model Impact Invariance?
We want to better contextualize our findings and understand how difficult it change the invariance properties of pretrained representations. Therefore, apply additional fine-tuning pass to the models in Scction 4.1 and Appendix C.1 on the target dataset after their last layer has been tuned top of frozen feature extractor, mimicking the linear probing, then full fine-tuning approach discussed in Kumar et al. (2022).
We finc-tune models after they have been trained with lincar probing on the target task for and additional 50 epochs with a learning rate of 10-3 3 (chosen based on a grid search over the values 10-1,10-2 30-3,10-4, 10005) and a linearly decreasing learning rate schedule. For Transforns-2DD, we ex- plore two fine-tuning dataset sizes: low-data sctting with 200 and a high-data sctting with 2000 fino-tuning samples. For the augmented CIFAR-10 and CIFAR-100 datasets, use 1% and 10% of the original training data for the low- and high-data settings, respectively. Again, results are averages over 10 runs.
Figure g shows the results for Transforms-2D with 200 samples, Figure 10 for CIFAR-10 with 1% of the full training samples and Figure 11 for CIFAR-100 with 1% the full training samples. Figure 12 and Figure 13 show the comparison of the differences transfer performance due to invariance vs other factors, for the low- and high-data fine-tuning scenarios, respectively. The results show that full fine-tuning can change the invariance properties of representations such that even representations not trained with the right set of invariances on the amount of available fine-tuning data. Especially for Transforms-2D in the low-data setting, the representations trained with the right invariances still perform significantly better than the ones trained with disjoint transformations. In the high-data setting the difference diminishes, since the training starts approximate full re-training on the target task. However, the representations trained with the same transformations as the target task still perform least well those trained with different transformations, in all cases.
adapt to the target task. However, the degree to which this is possible depends
22

---

Published in Transactions on Machine Learning Research (07 2204)
0.9
0.9
0.8
0.8
0.8
0.7
0.7
0.7
0.6
0.6
same, trardfer
0.5
0.5
disjaint, training
0.5
0.4
disjoint, transfer
same, training
0.4
0.3
subset
disjaint
same
superset
ResNet18 ResNet50
DN121
VGG11
VIT
10
50
200
1000
4500
Class
Architecture
Samples
(a) Invariance class relationship.
(b) Invariance vs architecture.
(c) Invariance vs number of pretrain- ing samples.
Figure 10: [Invariance to data augmentations vs the importance of other factors on augmented CIFAR-10 with full fine-tuning on 1% of the full training samplcs.] Training and transfer perfor- mance for models trained with different factors of variation and different transformations on the CIFAR-10 dataset, tne-tuned on 1% of the full training set samples after linear probing.
0.9
0.8
0.8
0.7
Acciaca
0.6
same, transfer
0.6
Accuracy
0.6
0.5
disjoint, training
disjaint, transfer
0.4
0.4
same, training
0.4
0.3
0.2
0.2
0.2
subset
disjaint
same
superset
ResNetl8 ResMee50
DN121
VGd11
VIT
10
50
100
250
500
Class
Architeture
Samples
(a) Invariance vs class relationship.
(b) Invariance vs architecture.
(c) Invariance vs number of pretrain- ing samples.
Figure 11: [Invariance to data augmentations vs the importance of other factors on augmented CIFAR-100 with full fine-tuning on 1% of the full training samples.] Training and transfer perfor- mance for models trained with different factors of variation and different transformations on the CIFAR-100 dataset, fine-tuned on 1% of the full training set samples after linear probing.
C.3 How Does Relevance and Availability of Input Information Impact Invariance?
In Section 4.2 we have shown that the relevance and availability of input information impacts the invariances learned by model. In this Section, we make two observations.
First, we show that the two models trained on CIFAR-10 images with objects pasted on them (X, C+ perform very differently after fine-tuning at predicting CIFAR-10 classes (Y, = C) and object categories (Y, = O) respectively, depending on whether they were pre-trained to predict the CIFAR-10 classes (Y, = C) or object categories (Yp = O), even though they saw exactly the same input data. This result shows that the relevance of a feature for target task is a key driver in determining which input features a model becomes invariant to.
Second, the models trained only CIFAR-10 images to predict CIFAR-10 classes (X,= C.Y, C) and only object images to predict object categories (X, O, Y, = O) show better transfer performance the respective other task (Y; F Yp) than their counterparts that were trained on CIFAR-10 images with pasted objects (X, C + O). These two models did not have access to the object- and CIFAR-feature respectively and could therefore not develop invariances towards them, as did the models that saw them during training. This mcans that another important property in determining the invariances Iearned by a model is the availability of features during training. Next, we investigate the effects of relevance and availability in more detail.
23

---

Published in Transactions on Machine Learning Research (07 (0722
0.30
0.30
0.25 03
0.9
0.17
0.04
0.31
0.14
Accuracy
0.7
6:26
0.14
0.33
Accurwey
0.
0.8
0.6
0.120.07
0:13
0.10
0.7
0.04
0.2
0.4
0.6
0.3
Class Rel.
Architecture
PT Samples
Class
Architecture
Samples
Class Rel.
Architecture
PT Samples
(a) Transforms-21
(b) CIFAR-10
(c) CIFAR-100
Figure 12: [Difference in transfer performance due to invariance other factors, for full fine- tuning, for 200 samples Transforms-2D and 1% of samples the CIFAR datasets.] We compare the differences in transfer performance caused by representational invariance with the differences caused by changes to other factors on the Transforms- 2D and the CIFAR-10 and CIFAR-100 datasets (with data augmentations). Orange (blue) bars show the span of transfer performance for tlfferentttrranformattio models Id (same-trransorrmation models 9a), for each comparison factor (class relationship, architecture, number of samples). Black bars show the difference belween transfer performance means across factor values for ga and Is. i.e. the difference in performance caused by having the same different invariances the target task.
1.050
1.025
0.02
0.0
0.06
0.04
0.07
Rocurscc
0.7
0.220.10
0.0g0.06
0.25
0.33
Accurary
0.4
0.5
0.16
0.30
0.08
0.
0.950
0.6
0.925
0.5
0.2
0.4
0.1
Class Rel.
Architecture
Samples
Class Rel.
Architecture
Samples
Class Rel.
Architecture
PT Samples
(a) Transforms-22
(b) CIFAR-10
(c) CIFAR-100
Figure 13: [Difference in transfer performance due to invariance vs other factors, for full fine- tuning, for 2000 samples on Transforms-2D and 10% of samples on the CIFAR datasets.] We compare the differences in transfer performance caused by representational invariance with the differences caused by changes to other factors on the Transforms-22 2D and the CIFAR-10 and CIFAR-100 datasets (with data augmentations). Orange (blue) bars show the span of transfer performance for differeenttraansfommaiio models 9d csame-trasformattion models ga), for each comparison factor (class relationship, architecture, number of samples). Black bars show the difference between transfer performance means across factor values for and Is. i.e. the difference in performance caused by having the same vs different invariances as the target task.
Relevance. To understand how the invariance to features changes with their relevance for the target task, we train models on the dataset described in Section 4.2 and Appendix A.2. However, we now introduce correlation of different strengths between the objects and the CIFAR labels. In particular, we train models Jo on datasets D., where a E (0,0.2, 0.4, 0.6, 0.8, 0.85, 0.9, 0.95, 1.0) the correlation strength between CIFAR labels and object categories. Input images in Da CIFAR-10 class (out of a set of 10 total objects) on the CIFAR-10 training images o-fraction of the time and pasting random object otherwise. Then, we fine-tune the gas' last layers and evaluate them on the augmented CIFAR-10 images, both for predicting the CIFAR-10 class (X, = + O, Y, =C) as well as the object category (X, = C + +0,Y
constructed by pasting one specific object per
Availability. To investigate the effect of availability, we train models on datasets Da. 3 E {0, 0.2, 0.4, 0.6, 0.8, 1.0) that equivalent to the X C -.Y C dataset, iec. objects pasted
24

---

Published in Transactions on Machine Learning Research (07 (024))
CITRS Accuracy
Oojects Acourscy
CIFAS Accuracy
Objects Acourcyy
0.8
mised CIFAR labels
0.6
CiRAR-only
0.6
mixed object labels
0.6
0.4-
objecs only
0.4
0.4
0.2
0.25
0.50
1.00
0.25
0.50
0.75
1.00
0.25
0.50
0.75 1.00
0.25
0.50
0.75
1.00
CIFAR-Object Correlation a
CIFR-Objjct Correlation
a
Object Availability B
Object Availability 8
(a) Correlation-dependenn(()
Correlation-dependent()) transfer performance to object labels
Availablityydepeedeent() transfer performance to CIFAR-10 labels
Availabiiity-dependeet transfer performance to object labels
transfer performance to CIFAR-10 labels
Figure 14: [The effect of feature relevance and availability on learned invariances]
Left plots: Transfer accuracy the X; C+O dataset for models trained with different correlation strengths of pasted object calegories with CIFAR labels. As the correlation resp. relevance of the pasted objects for the target task increases, models become increasingly better at predicting them and their representations become more sensitive to their presence.
Right plots: Transfer accuracy on the = O dataset for models trained with different availability of pasted object categories. As soon as feature becomes available the input but is irrelevant for the target task, models start become invariant it.
randomly onto CIFAR backgrounds, but with the difference that objects are only pasted 8-fraction of the time for dataset Dg- We again fine-tune and evaluate the last. layer of the models trained on Dg on the same datasets as for the relevance analysis. In both cases we paste the object in the upper right corner of the image.
Figure 14a and Figure 14b show the transfer accuracies of models trained to predict CIFAR-10 classes and object categorics, respectively. The blue curve correspond to the models pretrained on X, = C O datasets where the objects were pasted with different correlations with the CIFAR labels. Dashed lines show the performance of reference models. As the correlation of the pasted object categories with the CIFAR-10 classes increases, CIFAR-10 transfer accuracy mostly remains constant, whereas object category accuracy increases steadily. Only when pasted objects and CIFAR labels become perfectly correlated, CIFAR-10 accuracy drops and object detection accuracy reaches 100%. This trend shows that as the pasted objects become more relevant for the target task, the models' representations gradually become less invariant them and allow for incrcasingly better prediction of the object category. CIFAR-10 accuracy on the other hand remains constant, up to the point where the pasted object can reliably replace the CIFAR background.
The CIFAR performance does not depend on the availability of the pasted objects, i.e. whether or not an irrelevant object is present in the training data has no impact on it. Object category prediction performance on the other hand quickly drops as soon as the model has access to the irrelevant object features, showing that it immediately develops an invariance towards the objects. In summary, relevance and availability both play role in determining which invariances a model learns, but relevance seemss to be the more important property.
D Additional Details and Results on Invariance Transfer
D.1 Details on the invariance transfer experiments
Data. For the synthetic image datasets we use the Transforms-2D dataset and generate samples as de- scribed in Appendix A.1. We create out of distribution variants by using a new set of image objects, that the models have not seen during training. We create the synthetic random datasets by sampling pixel-values for foregrounds and backgrounds uniformly at random. Classes are created by using a different fixed random foreground pattern for each class, í.e. all samples for a class are transformed versions of the same random pattern on different random backgrounds. Random patterns are squaro-shaped, and have a length of 70% of the image size along each dimension. Examples of random images be seen in Figure 15a For both
25

---

Published in Transactions on Machine Learning Research (07 /024)
Elastic
Pixelate
(a) Examples of images from the synthetic random dataset.
Translate
Hue
(b) Examples of augmented CIFAR-10 images.
Figure 15: [Examples of images used for the OOD analysis.]. The top part shows examples of images from the synthetic random datasct, while the bottom part shows examples of augmented CIFAR-10 images. Each row shows the examples for single transformation.
Model
Transformation
Synthetic Datasets
Real-World Datasets
Type
Relationship
In- nn-Diitribution
Mild OOD
Strong OOD
CIFAR-10
CIFAR-100
Same
0.1220.06
0.19
0.66
0.34
0.33_o.18
Image
Other
0.39
0.39
0.75
0.50
0.49
None
0.38
0.39
0.78
1.
0.49
Same
0.12:
0.41
0.20
0.33
0.30
Random
Other
0.64 13
0.71
0.29
0.44
0.43
None
0.65
0.73
0.26
0.41
0.40
Table 3: [Invariance transfer under distribution shift for VGG-11 models.]
synthetic image and random datasets, use 30 classes and single type of transformation per dataset. Images have a size of 32x32 pixcls.
For the real world datasets (CIFAR-10 and CIFAR-100 Krizhevsky et al. 2009 ), we apply the transfor- mations from the Transforms- 2D dataset as data augmentations. We use the same transformations as the synthetic datasets, with the exception of the translation transformation, which here translates images by up 30% of the image size along each axis, instead of positioning objects at random position in the image. This change is necessary since with the CIFAR images occupying the entire image, translations would otherwise have effect. Examples of augmented CIFAR-10 images can be seen in Figure 15b
Mcasuring invariance. For the experiments here, report the average sens-score defined in Sec- tion 3.4 i.e. the ratio of L2-distance between representations at the penultimate layer for inputs that only differ in their transformations, to the L2-distance between any two inputs from the respective dataset (lower values mean more invariance). For example, when computing the sens-score for the translation transforma- tion, we compute the average L2-distances between instances of the same object on the same background in different positions, divided by the average L.2-distances of different objects on different backgrounds, different positions. We use the penultimate layer representations here, since those are the representations most relevant for transfer learning.
D.2 Additional results on invariance transfer
Invariance transfer results for additional architectures. Tables 3 and 5 show the results for the experiments on invariance transfer under distribution shift for VGG-11, DenseNet-121 and ViT models, respectively. Each cell shows the average sens-score (lower values mean more invariance), for models trained on image and random data. "Transformation Relationship" refers to the relationship between the training and test transformations of the models. Rows labeled as "Same" show how invariant models their
26

---

Published in Transactions on Machine Learning Research (07 (07/2024)
Model
Transformation
Synthetic Datasets
Real-
Woold Datasets
Type
Relationship
In- In-Distribution
Mild OOD
Strong OOD
CIFAR-10
CIFAR-100
Same
0.16
0.24
#..7
0 20
0.40=o.18
Image
Other
0.39
0.40
0.73+v.14
0.50+a.11
0.48
None
0.39
0.41
0.74
0.50
0.48
Same
0.17:0011
0.50
0.24
0.44 22
0.42 0..21
Random
Other
0, 0.66
0.72
0.28
0.4644.22
0.44
None
0.69
0.75#0.11
0.30
0.4640.22
0.46+0.22
Table [Invariance transfer under distribution shift for DenseNet-121 models]
Model Type
Transformation Relationship
Synthetic Datasets
Real-World Datasets
Tn- Dn-Distribution
Mild OOD
Strong OOD
CIFAR-10
CTFAR-100
Same
0.22
0.26+a.11
0.
0.36
0.33_o.18
Image
Other
0.41
0.41:0.16
0.64:0.19
0.4840.17 I7
0.4640.17
None
0.43
0.42+0.17
0.65
0.48
0.47
Same
0.28
1.
0.23
0.31 14
0.30
Random
Other
0.61
0.64
0.34g0.14
0.45
0.43
None
0.63
0.67
0.34
0.46
0.45
Table 5: [Invariance transfer under distribution shift for VIT models.]
training transformations on each of the datasets (e.g. traunsltiion-trained model evaluated on translated data), whereas rows labeled as "Other" show how invariant they are to transformations other than the ones they were trained on (e.g. translation-trainee model rotations). "None" rows show the baseline invariance of models trained without transformations, f.e. simply to classify untransformed objects. The most invariant models in each category are shown in bold. Results are computed over 10 runs that randomize the image and random objects, backgrounds, the way transformation values are sampled and the weight initialization of the models. The subscript numbers indicate the standard deviation of the sens-score over the 10 runs.
For all architectures, observe very similar trends, which indicates that observations about how models transfer invariance are robust. First, training models to be invariant to specific transformations indeed makes them more invariant to primarily those transformations. The invariance on the training transformations ("Same" rows) is significantly higher than for other transformations ("Other" rows), and also higher than the invariance of the baseline models trained without transformations (*None") rows.
Second, models can transfer a medium to high degree of invariance diitrii-iitiiittioon data. The invariance of models trained on image data is similar to that other image data with different forcground objects ("Mild OOD"), and the same is roughly true for models trained random data when transferring to image data ("Strong OOD"). Both types of models are less invariant on (unseen) random data ("Strong OOD" for the image models, "Mild OOD" for the random models), but they are still more invariant to their training transformations than to other transformations, and also more invariant than the baseline models trained without transformations. Both types of models achieve high invariance on the real-world CIFAR-10 and CIFAR-100 datascts. These results indicate that models can retain a high degree of invariance, even when the data that they are evaluated on substantially different from the data they were trained on. This helps to explain why invariance is such important property for transfer learning: it can be transferred well to out-of-distribution data.
Third, the models trained on image and random data achieve similar invariance on the real-world CIFAR datasets. This indicates that for learning representations that are invariant to certain transformations, the specific fcatures of the data do not matter as much. The caveat that this approach only works if the features in the dataset are suitable express the desired transformations. The pixel-level transformations in the Transforms-2D datatset can be applied to random data, but for higher-level transformations, e.g. transfor-
27

---

Published in Transactions on Machine Learning Research (07/2024)
mations of the pose of animal, the dataset needs to contain features that can express the transformation, e.g. animals with certain poses.
none
0.57
0.61
0.34
0.28
0.56
0.56
0.24
0.53
0.14
0.38
0.54
0.16
0.17
0.45
0.26
0.45
0.34
0.45
contrast
0.68
0.85
0.45
0.42
0.83
0.86
0.11
0.19
0.05
0.21
0.40
0.21
0.24
0.41
0.41
0.61
0.39
0.13
0.8
erasing
0.68
0.84
0.45
0.42
0.75
0.81
0.17
0.44
0.09
0.34
0.51
0.11
0.27
0.39
0.39
0.57
0.21
0.36
elastic
0.33
0.21
0.25
0.20
0.41
0.28
0.47
0.63
0.29
0.47
0.52
0.28
0.16
0.44
0.17
0.17
0.31
0.55
0.7
pixelate
0.51
0.58
0.29
0.23
0.54
0.51
0.31
0.61
0.17
0.42
0.51
0.19
0.10
0.44
0.08
0.40
0.36
0.51
noise
0.54
0.67
0.36
0.30
0.63
0.62
0.19
0.55
0.11
0.41
0.49
0.17
0.16
0.15
0.23
0.47
0.33
0.46
Training Transformations
0.6
blur 0.53
0.62
0.32
0.26
0.58
0.56
0.30
0.60
0.17
0.41
0.51
0.22
0.07
0.42
0.36
0.50
sharpen
0.60
0.71
0.38
0.33
0.66
0.67
0.22
0.50
0.12
0.34
0.53
0.07
0.19
0.44
0.31
0.51
0.34
0.41
invert
0.57
0.67
0.34
0.30
0.63
0.63
0.21
0.56
0.12
0.38
0.09
0.26
0.19
0.48
0.26
0.47
0.37
0.49
0.5
posterize
0.61
0.73
0.38
0.33
0.68
0.71
0.22
0.42
0.12
0.10
0.53
0.16
0.21
0.47
0.32
0.52
0.35
0.38
grayscale
0.60
0.72
0.39
0.34
0.68
0.69
0.13
0.52
0.03
0.36
0.51
0.19
0.19
0.45
0.31
0.52
0.36
0.43
0.4
brightness
0.68
0.86
0.45
0.44
0.84
0.86
0.12
0.16
0.06
0.17
0.46
0.16
0.27
0.40
0.42
0.62
0.38
0.15
hue
0.57
0.68
0.37
0.32
0.65
0.65
0.05
0.53
0.07
0.36
0.49
0.20
0.19
0.43
0.29
0.49
0.35
0.45
0.3
shear
0.20
0.17
0.41
0.24
0.47
0,28
0.29
0.35
0.56
rotate
0.39
0.35
0.07
0.06
0.16
0.26
0.46
0.68
0.28
0.47
0.50
0.33
0.20
0.49
0.18
0.25
0.35
0.60
0.2
h_flip
0.49
0.54
0.29
0.05
0.47
0.47
0.33
0.59
0.19
0.41
0.50
0.21
0.16
0.42
0.21
0.39
0.34
0.50
v_ v_flip
0.54
0.59
0.07
0.26
0.51
0.51
0.30
0.60
0.17
0.43
0.52
0.21
0.18
0.45
0.23
0.42
0.36
0.51
0.1
translate
0.40
0.18
0.28
0.24
0.46
0.49
0.37
0.62
0.24
0.44
0.50
0.30
0,22
0.47
0.24
0.25
0.36
0.55
scale
0.18
0.67
0.35
0.30
0.63
0.54
0.32
0.60
0.18
0.42
0.51
0.26
0.16
0.48
0.23
0.44
0.44
0.52
scale
translate
V_flip
h_flip
rotate
shear
hue
Brightnend
Seyscaa
posterize
invert
sharpen
blur
pixelate
elastic
erasing
contrast
Target Transformations
Figure [Invariance transfer between transformations, for models trained on image data.] Rows show how invariant models trained be invariant transformations. Columns show how invariant each model is to the specific target transformation. Values are computed using the sens-metric (see Section 3.4/. Lower numbers indicate higher invariance. Models trained with specific transformation in their training data become more invariant to this transformation. We show the invariance of each model on the test set of its training dataset type ("In-Distribution")
specific transformation are to each of the
Invariance Transfer Results Between Different Transformations. We show breakdown of the invariance transfer of the image and random models for ResNet-18 architectures in Figures 16 and 17 respectively. For each transformation, we show the invariance of models trained to be invariant to that transformation, é.c. trained to classify objects modified by that transformation, for each of the other transformations. The results show that models indeed become primarily invariant to their training transformations (low values on the diagonal). However, we can also observe some "spillovers", e.g. training models be rotation- and shear- invariant also increases the invariance to horizontal and vertical flips (but not vice versa), and training for hue- and grayscale-invariance increases the invariance to the respective other transformation.
nn-ttrnsfformation basis, the training distributions (objects and backgrounds)
28

---

Published in Transactions on Machine Learning Research (07 (07/2024)
none
0.81
0.97
0.55
0.55
0.96
0.95
0.78
0.78
0.49
0.54
0.60
0.48
0.64
0.55
0.55
0.64
0.34
0.62
contrast
0.79
0.98
0.50
0.50
0.97
0.97
0.78
0.39
0.39
0.60
0.59
0.56
0.28
0.49
0.43
0.69
0.38
0.09
0.9
erasing
0.83
0.98
0.53
0.53
0.97
0.96
0.75
0.76
0.49
0.58
0.61
0.55
0.67
0.43
0.56
0.64
0.13
0.59
elastic
0.33
0.16
0.39
0.40
0.76
0.44
0.78
0.98
0.52
0.63
0.63
0.62
0.17
0.63
0.22
0.20
0.38
0.96
0.8
pixelate
0.69
0.94
0.43
0.45
0.92
0.89
0.76
0.93
0.40
0.61
0.61
0.58
0.24
0.56
0.06
0.63
0.38
0.84
noise
0.83
0.98
0.53
0.52
0.94
0.94
0.76
0.64
0.49
0.58
0.64
0.35
0.69
0.26
0.57
0.64
0.24
0.57
0.7
Training Transformation
blur
0.46
0.94
0.90
0.40
0.62
0.60
0.63
0.37
0.72
sharpen
0.79
0.97
0.54
0.55
0.95
0.94
0.78
0.65
0.45
0.44
0.60
0.04
0.60
0.52
0.54
0.64
0.33
0.48
0.6
invert
0.83
0.97
0.54
0.55
0.96
0.96
0.79
0.77
0.50
0.57
0.06
0.55
0.65
0.60
0.55
0.65
0.40
0.62
posterize
0.80
0.97
0.54
0.55
0.96
0.95
0.79
0.70
0.47
0.04
0.60
0.43
0.62
0.54
0.54
0.64
0.35
0.58
0.5
grayscale
0.81 0.98
0.54
0.54
0.96
0,95
0.49
0.87
0.04
0.59
0.60
0.57
0.55
0.56
0.52
0.65
0.36
0.75
brightness
0.81
0.97
0.52
0.51
0.97
0.97
0.79
0.22
0.44
0.31
0.59
0.58
0.40
0.55
0.47
0.69
0.48
0.23
0.4
hue
0.82
0.98
0.52
0.53
0.96
0.96
0.10
0.89
0.16
0.60
0.61
0.59
0.58
0.56
0.53
0.66
0.38
0.78
shear
0.63
0.25
0.34
0.70
0.28
0.75
0.64
0.65
0.75
0.64
0.78
0.65
0.47
0.97
0.3
rotate
0.76
0.13
0.13
0.21
0.56
0.67
0.64
0.75
0.61
0.70
0.54
0.97
h_flip
0.53
0.05
0.95
0.95
0.79
0.58
0.59
0.56
0.63
0.73
0.2
v_flip
0.83
0.97
0.04
0.55
0.95
0.94
0.80
0.84
0.49
0.58
0.59
0.56
0.63
0.58
0.54
0.64
0.39
0.73
translate
0.67
0.10
0.33
0.45
0.73
0.64
0.82 0.99 0.66 0.62
0.72 0.61 0.78 0.64
0.72
0.66
0.48
0.97
0.1
scale 0.14
0.88
0.45
0.46
0.89
0.84
0.77 0.92 0.43 0.59 0.63
0.57 0.35 0.59
0.41
0.60
0.41
0.86
scale
translate
V. f.flip
h flp
rotate
shear
hue
brightnes
grayscale
posterize
invert
sharpen
blur
noise
pixelate
elastic
erasing
contrast
Target Transformations
Figure 17: [Invariance transfer between transformations, for models trained on random data.] Rows show how invariant models trained to be invariant to specific transformation are to each of the transformations. Columns show how invariant each model is to the specific target transformation. Values are computed using the sens-metric (sce Section 3.4). Lower numbers indicate higher invariance. Models trained with specific transformation in their training data become more invariant to this transformation. We show the invariance of each model on the test set of its training dataset type ("In-Distribution))
D.3 Additional Results for Invariance Mismatch
shows the results analogous those in Section 5.2 for invariance mismatch between training and transfer tasks, for addtional model families: VGG-11, DenseNet-121 and ViT models. We observe the pattern in Section 5.2 d.e. models trained to be invariant to the same set or superset of transformations those in the target dataset consistently achieve almost 100% accuracy, but models that are missing invariance to transformations in the target dataset achieve significantly lower performance.
Figure 18
29

---

Published in Transactions on Machine Learning Research (07/202) /024)
1.0
1.0
1.0
0.8
Triring
0.8
Training
Transfor accurssy
Traking
6.7
Thansle
0.6
0.5
0.3
0.3
0.2
Transter Transfoms
Thanster Transfomss
# Transfer Transforrr
(a) VGG-11
(b) DenseNet-121
(c) VIT
Figure 18: [Models trained on nested sets of transformations and evaluated on datasets with super- and subsets of those transfornations.] Models trained on data with the same set or a superset of transformations as the target dataset consistently achieve almost 100% accuracy. However, models trained with only a subset of the transformations show considerably lower performance that decreases the smaller the subset of training transformations is compared to the target task. The results show that learning a superset of required invariances does not harm transfer performance but that missing required invariances degrades transfer performance.
30


Reviews:
[{"review_id": "fb5kECO4yg", "reviewer": "Reviewer_QUU6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n1. As far as I am aware, this work is the first to attempt a systematic study of the role of invariance in transfer learning. The authors provide a fairly carefully constructed set up using synthetic data to come to the findings described above in the summary. \n\n2. The results with respect to importance of invariance as contrasted with other factors for transfer learning (Figure 2) were interesting. I also thought Table 1 was insightful, showing that exposure to target task features a priori could actually be detrimental for transfer, since the model learns to ignore them.\n\n## Weaknesses\n\n1. The study is fairly limited in that most of the investigations are performed via a highly synthetic construction (i.e. the Transforms-2D dataset). The type of invariance studied is specifically of the empirical approximate nature, i.e., invariance is introduced via data augmentations, not of the exact theoretical nature studied by e.g. Cohen et al. (2021). The synthetic datasets have classes that are derived from augmentations of a single given object, i.e., each specific object defines a class, which is not realistic. However, the authors are upfront about this fact in their \"Limitations\" discussion and they do reproduce a subset of their results for CIFAR-10 and CIFAR-100 augmentations, which is a reasonable extension of their current synthetic results. \n\n2. Some of the metrics, particularly for measuring invariance in representations, are not rigorously presented. In particular, I am referring to the sens metric in Section 3.4 and the computation of average distance between any two random samples. It should be stated that these are approximations of expectations, and depending on the nature of T and O, one may need to sample many pairs $(x_i, x_i') \\sim \\mathcal{D}(T,O)$ to obtain a good enough estimate. I do not believe the authors provided commentary on how many samples needed were drawn for their own approximations/sens-metric computations.\n\n## Verdict\n\nThis work aims to present a systematic study of the role of invariance in transfer learning. The results make use of carefully constructed synthetic datasets, and are fairly novel, as far as I am aware. Although the conclusions may be somewhat limited by the synthetic nature of the constructions, the authors attempt an extension to CIFAR-10 and CIFAR-100 for a subset of their results, and a number of the results are fairly interesting and insightful. I believe the claims made in the submission are well supported, and the findings of the paper would be interesting to some portion of TMLR's audience, specifically, those working on transfer learning. As such, both criteria for acceptance to TMLR are satisfied; ergo, I recommend acceptance of this paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper aims to study the role of invariance in the context of transfer learning. In service of attaining this end, the authors introduce a family of synthetic datasets they call \"Transforms-2D.\" By carefully synthesizing these datasets the authors are able to train models to exhibit specific invariances in their representation and to evaluate their performance on transfer tasks that require specific invariances. The authors further investigate the connection between invariance and downstream performance and compare it to other factors studied in the transfer learning literature (for example number of training samples, model architecture, etc.). The findings are that invariance of the target task is often as or more important than these other factors. They also explore the transferability of invariance across tasks and find that in many cases, models can transfer a high degree of learned invariance to out-of-distribution tasks. The paper also validates some of these findings on their synthesized datasets on the real-world datasets CIFAR-10 and CIFAR-100.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The writing in this paper is for the most part quite good. Ideas are clearly presented and figures and tables are well set up to illustrate the main points of the paper. I would like for the presentation of the sens-metric to be made more rigorous to address Weakness 2 above; in particular, I would like the authors to include an explicit discussion (perhaps in the appendix) of how well the sens-metric computations were approximated/how many samples were drawn in making this approximation. Additionally, I have compiled a non-exhaustive list of minor corrections below:\n\n- Abstract: \"is as or often more important\" -> \"is as, or often more, important\"\n- Introduction, first paragraph: Please use \\citep for parenthetical citations. For example, \"...on their training dataset Salman et al. (2020).\" should clearly be \"... on their training dataset (Salman et al., 2020).\" The same mistake is made in the following sentence. There are several instances of this mistake in the paper (e.g. first sentence of Section 3.2); please be sure to use parenthetical citations instead of in-text citations, where appropriate.\n- Introduction, second paragraph: \"In the case of transfer learning on the other hand,\" -> \"In the case of transfer learning, on the other hand,\"\n- Section 4.2, page 7, final paragraph: \"Figure 2 compares we effect of invariance on transfer accuracy\" -> \"Figure 2 compares the effect of invariance on transfer accuracy\"\n- Section 4.2, final paragraph: \"Practitioners should pay special attention to the transformations present\" -> \"Practitioners should pay special attention to the variations present\"", "broader_impact_concerns": "This paper briefly mentions exploiting invariance to harm transfer performance, in Section 4.2. This is a fairly speculative section, but I believe it may be worth adding a high-level discussion about this to a separate broader impact statement (since one is currently not present).", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6p4r6WlJfs", "reviewer": "Reviewer_Hw8K", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1. The paper is well written and organized.\n2. The paper’s key contribution: providing a controlled environment for testing invariance learning in image classification setup, is an interesting contribution in my opinion. The paper’s dataset, if made publicly available, can help the community in studying further aspects of this problem.\n3. Thorough experiments and ablations. Understanding the role of learning invariance for transfer learning is an important question, this paper is a good starting point.\n\n**Weaknesses**\n\n**Question 1: Penultimate layer being the most important for transfer learning**\n\n> While the methodology in this paper is applicable to representations at any layer, we focus on the representations at the penultimate layer, since they are most relevant in the context of transfer learning. \n\nCould the authors justify this claim? Prior work [1, 2] has shown that the choice of layer for transfer learning depends on the particular distribution shift between training and test distributions, sometimes earlier layers are better, sometimes middle and sometimes last. This claim needs to be justified or modified in the paper.\n\n**Question 2: Restricting the analysis to “class-preserving” or nuisance transformation**\n\nI think the definition and handling of transformations in the formal description 3.1, while defining invariance, need to be clarified. Invariance is not always a good thing, sometimes transformations are not “class-preserving” or nuisance, and if the model is invariant to these transformations, it can be bad. Imagine the scenario of detecting digits (0 to 9) from images, and the transformation set $T$ be all possible rotations. Then being invariant to $T$ means the model’s representation for 6 and 9 has to be the same, which is obviously bad. Similar things can happen in real life scenarios as well (image two types of snakes which can only be differentiated by their color/habibat (i.e., background) and invariance here can be a bad thing). The analysis needs to be restricted to “class-preserving” transformations and this need to be formally defined in this section.\n\n**Question 3: Set of transformations T form a group**\n\nDoes the set of transformations, $T$, form a group?\n\n**Question 4: Transfer learning scenario studied is very simplistic**\n\nThe transfer learning scenario tested on this paper is the following:\nTrain a neural network on source domain\nFreeze the model weights, train a linear classifier on the embeddings obtained on the target domain.\n\nThis is the linear probing setup described in [3]. It is a very primitive setup, with already known drawbacks. Refer to table 1 and 2 of [3]. Some additional transfer learning setup, such as full fine-tuning (FT), linear probing then full fine-tuning (LP-FT), surgical fine-tuning [1] would make the paper more compelling. How invariant do the representations remain in each setup? How important is invariance as a factor for all of these different transfer learning algorithms?\n\n**Question 5: Plot 2**\n\nCould the authors use 4 different colors instead of 2 colors and dot/dash separation? Current plot is a little bit hard to read.\n\n**Question 6: Dataset diversity**\n\n>  For example, for supervised learning it may be beneficial to spend more effort in obtaining a more diverse set of samples per class, rather than smaller sets of samples for more classes.\n\nI have questions regarding this point. \nDo the authors mean unsupervised pre-training or supervised training where the model is trained to learn a function from image x to label y?\nWould this model be fine-tuned for a different classification task down the line, or used as is? If the latter, then the number of classes is fixed by the application. If the former, the relative benefit between having more classes present on the pretraining dataset helping the model to learn discriminative features between many different types of objects, vs more diverse samples from the same class is not clear to me. Could the authors point to the exact experimental insight that point to this?\n\n**Question 7: Discussing Prior Work**\n\nCould [4] be discussed by the authors?\n\n**References**\n\n[1] Surgical Fine-Tuning Improves Adaptation to Distribution Shifts, https://arxiv.org/abs/2210.11466\n\n[2] Less is More: Selective Layer Finetuning with SubTuning, https://arxiv.org/abs/2302.06354\n\n[3] Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution, https://arxiv.org/abs/2202.10054\n\n[4] Invariance is Key to Generalization: Examining the Role of Representation in Sim-to-Real Transfer for Visual Navigation, https://arxiv.org/abs/2310.15020", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the role of invariance in transfer learning. Transfer learning is the problem where one learns a problem structure on one setup, and tries to transfer that knowledge to a different setup. The most common use today of transfer learning comes from the rise of foundation models, where models trained on huge pretraining datasets are then fine-tuned for specific downstream tasks, and often achieve SOTA performance.\n\nOn the other hand, good models should be invariant to nuisance or “class-preserving” transformations: a dog slightly shifted in the image to the left still remains a dog and the model should be able to identify it as such.\n\nThis paper studies the relationship between these two questions. The key contributions are:\n1. Proposing a simple benchmark of synthetic images, Transforms-2D, that precisely controls the objects and transformations in particular images, to be able to study invariance to transformations in a controlled manner.\n2. Studying the importance of invariance in transfer learning.\n3. Providing an adversarial example where the model being invariant to certain transformations can actually harm its performance in the downstream task.\n4. Studying invariance transfer under distribution shift.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In general, I like this paper: I would vote for acceptance if all my questions above are answered to my satisfaction.", "broader_impact_concerns": "No broader impact concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KlMCCrRI44", "reviewer": "Reviewer_pq9a", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The topic of understanding the role of invariance in transfer learning is quite meaningful and important. The construction of the synthetic dataset is interesting. However, I feel that the main message and the presentation are not good enough.\n\n1. The title is \"understanding the role of invariance\". However, this paper shows that invariance can either benefit or harm transfer performance (see Section 4.2). This is confusing as it suggests that the role of invariance is uncertain.\n\n2. The target of invariance is vague (i.e., invariance on the source task's input or the target task's input). In the synthetic dataset, transformations are conducted on both the source task's input and the target task's input. The transformation sets can be the same or disjoint. I am not sure what is the specific meaning of invariance in this setup.\n\n3. The conclusion (\"invariance is more important than others\") is questionable. How do you measure the importance? For example, is there a precise mathematical definition of the importance of the number of training samples that can be quantitatively calculated?\n\n4. Notations in Section 4.2 are confusing. For example, in the second paragraph of Section 4.2, I don't know the meaning of $(X=C,Y=C)$. What is $C$? How can the features $X$ and the labels $Y$ equal to the same thing?\n\n5. For the synthetic dataset Transforms-2D, why do you need a random background? I guess the transformation is only on the foreground object?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper designs experiments to study the effect of transformations (such as rotate, hue, blur) on images to transfer learning. The authors claim that invariance to the transformations is important to transfer learning. To investigate the invariance carefully, the authors design a family of synthetic datasets called Transforms-2D which consists of images with a set of foreground objects, a random background, and a set of transformations. With experimental results on the designed dataset, CIFAR-10, and CIFAR-100, the authors claim that invariance is often as or more important than other factors such as the number of training samples, the model architecture, and the class relationship of the training and target task.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clarify the main conclusion on what is the role of invariance.\n\n2. Be more specific/rigorous on certain concepts, such as comparing the importance.\n\n3. Some notations need a better explanation (e.g., ones in Section 4.2).", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper looks into the role of invariance in transfer learning. The authors look into penultimate layer and do extensive experiments for a introduced synthetic dataset called Transforms-2D dataset. They added limited experiments on cifar 10 and 100 during rebuttal process. The authors addressed vague parts and gaps in the presentation during the rebuttal process.  The provided answers by the authors were clear and satisfactory.
Decision: Accept (poster)