Visual Representation Learning Does Not Generalize Strongly Within the Same Domain
Authors: Lukas Schott
Julius Von Kügelgen
Frederik Träuble
Peter Vincent Gehler
Chris Russell
Matthias Bethge
Bernhard Schölkopf
Francesco Locatello
Wieland Brendel
Venue: ICLR 2022 Poster (2022)
Keywords: Generalization
Composition
Out of distribution
Disentanglement

Abstract: An important component for generalization in machine learning is to uncover underlying latent factors of variation as well as the mechanism through which each factor acts in the world.
In this paper, we test whether 17 unsupervised, weakly supervised, and fully supervised representation learning approaches correctly infer the generative factors of variation in simple datasets (dSprites, Shapes3D, MPI3D) from controlled environments, and on our contributed CelebGlow dataset. 
In contrast to prior robustness work that introduces novel factors of variation during test time, such as blur or other (un)structured noise, we here recompose, interpolate, or extrapolate only existing factors of variation from the training data set (e.g., small and medium-sized objects during training and large objects during testing). Models that learn the correct mechanism should be able to generalize to this benchmark.
In total, we train and test 2000+ models and observe that all of them struggle to learn the underlying mechanism regardless of supervision signal and architectural bias. Moreover, the generalization capabilities of all tested models drop significantly as we move from artificial datasets towards more realistic real-world datasets.
Despite their inability to identify the correct mechanism, the models are quite modular as their ability to infer other in-distribution factors remains fairly stable, providing only a single factor is out-of-distribution. These results point to an important yet understudied problem of learning mechanistic models of observations that can facilitate generalization.

Published as a conference paper at ICLR 2022
VISUAL REPRESENTATION LEARNING DOES NOT GENERALIZE STRONGLY WITHIN THE SAME DOMAIN
Lukas Schotth Julius von Kügelgen Frederik Träuble Peter Gehler*, Chris Russell', Matthias B Bernhard Schölkopf Francesco Locatello* 1, Wieland Brendel!. 'University of Tübingen, Max Planck Institute for Intelligent Systems, Tübingen Uniieesitty of Cambridge, 4 AAmazon Web Services
Joonnt senior authors, #ook done during an internship at Amazon 1 lukas 8 wcwotttttetcccccctcoccccococc. @beettttttehhheee.e la. org
ABSTRACT
An important component for generalization in machine learning is to uncover un- derlying latent factors of variation as well as the mechanism through which each factor acts in the world. In this paper, we test whether 17 unsupervised, weakly su- pervised, and fully supervised representation learning approaches correctly infer the generative factors of variation in simple datasets (dSprites, Shapes3D, MP13D) from controlled environments. and on our contributed CelebGlow dataset. In con- trast to prior robustness work that introduces novel factors of variation during test time, such as blur or other (un)structured noise, we here recompose, inter- polate, or extrapolate only existing factors of variation from the training data set (e.g., small and medium-sized objects during training and large objects during testing). Models that learn the correct mechanism should be able to generalize to this benchmark. In total, we train and test 2000+ models and observe that all of them struggle to learn the underlying mechanism regardless of supervision sig- nal and architectural bias. Moreover, the generalization capabilities of all tested models drop significantly as we move from artificial datasets towards more realis- tie real-world datasets. Despite their inability identify the correct mechanism, the models are quite modular as their ability to infer other in-distribution factors remains fairly stable, providing only single factor is out-of-distribution. These results point to an important yet understudied problem of learning mechanistic models of observations that can facilitate generalization.
INTRODUCTION
Humans excel at learning underlying physical mechanisms or inner workings of system from ob- servations (Funke et al., 2021; Barrett et al., 2018; Santoro et al., 2017: Villalobos et al., 2020; Spelke, 1990), which helps them generalize quickly to new situations and to learn efficiently from little data (Battaglia et al., 2013; Dehaene, 2020; Lake et al., 2017: Téglás et al., 2011). In con- trast, machine learning systems typically require large amounts of curated data and still mostly fail to generalize to out-of-distributio (OOD) scenarios (Schölkopf et al., 2021; Hendrycks & Diet- terich, 2019; Karahan et al., 2016; Michaelis et al., 2019; Roy et al., 2018; Azulay & Weiss, 2019; Barbu al., 2019). It has been hypothesized that this failure of machine learning systems is due to shortcut learning (Kilbertus* et al., 2018; Ilyas et al., 2019; Geirhos et al., 2020; Schölkopf et al., 2021). In essence, machines seemingly learn to solve the tasks they have been trained on using auxiliary and spurious statistical relationships in the data, rather than true mechanistic relationships. Pragmatically, models relying on statistical relationships tend to fail if tested outside their train- ing distribution, while models relying on (approximately) the true underlying mechanisms tend to generalize well novel scenarios (Barrett al., 2018; Funke et al., 2021; Wu et al., 2019; Zhang et al., 2018; 18 Parascandolo et al., 2018; Schölkopf et al., 2021; Locatello et al., 2020a;b). To learn effective statistical relationships, the training data needs to cover most combinations of factors of variation (like shape, size, color, viewpoint, etc.). Unfortunately, the number of combinations scales exponentially with the number of factors. In contrast, learning the underlying mechanisms behind the factors of variation should greatly reduce the need for training data and scale more gently with the number of factors (Schölkopf et al., 2021; Peters et al., 2017; Besserve et al., 2021).
Our goal is to quantify how well machine learning models already learn the mecha- nisms underlying a data generative process. To this end, we consider four image data sets where each image is described by small number of independently controllable factors of variation such
Benchmark:

---

Published as a conference paper at ICLR 2022
as scale, color, or size. We split the training and test data such that models that learned the under- lying mechanisms should generalize to the test data. More precisely, we propose several systematic out-of-distribution (OOD) test splits like composition (c.g., train small hearts, large squares + rest= = small squares, large hearts), interpolation (e.g. small hearts, large hearts - medium hearts) and extrapolation (e.g., small hearts, medium hearts -> large hearts). While the factors of variation are independently controllable (c.g., there may exist large and small hearts), the observations may exhibit spurious statistical dependencies (e.g., observed hearts are typically small, but size may not be predictive at test time). Based on this setup, we benchmark 17 representation learning approaches and study their inductive biases. The considered approaches stem from un-/weakly supervised dis- entanglement, supervised learning, and the transfer learning literature.
Our benchmark results indicate that the tested models mostly struggle to learn the under- lying mechanisms regardless of supervision signal and architecture. As soon as factor of variation outside the training distribution, models consistently tend to predict value in the previously ob- served range. On the other hand, these models can be fairly modular in the sense that predictions of in-distribution factors remain accurate, which is in part against common criticisms of deep neural networks (Greff et al., 2020; Csordás et al., 2021; Marcus, 2018; Lake & Baroni, 2018).
Results:
Previous datasets with independent controllable factors such as dSprites, Shapes3D, and MPI3D (Matthey et al., 2017; Kim & Mnih, 2018; Gondal et al., 2019) stem from highly struc- tured environments. For these datasets, common factors of variations are scaling, rotation and simple geometrical shapes. We introduce dataset derived from celebrity faces, named CelebGlow, with factors of variations such as smiling, age and hair-color. It also contains all possible factor com- binations. It based on latent traversals of a pretrained Glow network provided by Kingma et al. (Kingma & Dhariwal, 2018) and the Celeb-HQ dataset (Liu et al., 2015).
New Dataset:
We hope that this benchmark can guide future efforts to find machine learning models capable of understanding the true underlying mechanisms in the data. To this end, all data sets and evaluation scripts are released alongside leaderboard on GitHub.
y11
2 PROBLEM SETTING
Assume that we render each observation or image x E Rd using a "computer graphic model" which takes as input a set of indepen- dently controllable factors of variation (FoVs) y E Rn like size or color. More formally, we assume a generative process of the form g(y), where g : R" + Rd is injective and smooth function. In the standard independently and identically distributed (IID) setting. would generate the training and test data in the same way, i.e., we would draw from the same prior distribution p(y) and then gener- ate the corresponding images x according to g(-). Instead, we here consider an OOD setting where the prior distribution (r) during training different from the prior distribution (y) during testing.
X
8
Figure 1: Assumed graphical model connecting the factors of variations y = (y)1 ..., yn to observations x = g(y). The selection variable 8 E (tr, te} leads different train and test splits (y)),, thereby inducing correlation between the FoVs.
In fact, in all settings of our benchmark, the training and test distributions are completely disjoint, meaning that each point can only have non non-zero probability mass in either Per or (y). Cru- cially, however, the function y which maps between FoV' and observations is shared between train- ing and testing, which is why we refer to as an invariant mechanism. As shown in the causal graphical model in Fig. 1, the factors of variations y are independently controllable to begin with, but the binary split variable s introduces spurious correlations between the FoVs that are different result of selection bias (Storkey, 2009; Bareinboim & Pearl, 2012). In
at training and test time particular, we consider Random, Composition, Interpolation, and Extrapolation splits as illustrated in Fig. 2. We refer to $4.2 for details on the implementation of these splits.
The task for our machine learning models f is to estimate the factors of variations y that generated the sample on both the training and test data. In other words, want that (ideally) The main challenge is that, during training, we only observe data from Per but wish to general- ize to Pte. Hence, the learned function f should not only invert g locally on the training domain supp(per(y) (y)) S R" but idcally globally. In practice, let Dre = { (xfx)*)) xx)} be the test data with yk drawn from (y) and let f Rd + RM be the model. Now, the goal is design and optimize the
g 1
hhtpp::/ thw.tttthhhhoo com cwt../coo///eeeeeee
cot.ub..co/bbbtthheeeeeeee bge 1 ab/ Iniema nnGnnerll zaa onHenchmark
2

---

Published as a conference paper at ICLR 2022
Random
Composition
Interpolation
Extrapolation
Train
Test
Figure 2: Systematic test and train splits for two factors of variation. Black dots correspond to the training
and red dots the test distribution. Examples of the corresponding observations are shown the right.
model on the training set Der such that it achieves a minimal R-squared distance between y and fxx") on the test set Dee.
During training, models are allowed sample the data from all non-zero probability regions (y)p(p..(y)) in whatever way is optimal for its learning algorithm. This general formulation cov- ers different scenarios and learning methods that could prove valuable for learning independent mechanisms. For example, supervised methods will sample an IID data set Drr {(y", x*)} with yd ~ (y) while self-supervised methods might sample a data set of unlabeled image pairs Drr [(x", }) We aim understand what inductive biases help these OOD settings and how to best leverage the training data to learn representations that generalize.
INDUCTIVE BIASES FOR GENERALIZATION IN VISUAL REPRESENTATION LEARNING
We now explore different types of assumptions, or inductive biases, on the representational for- mat ($3.1), architecture ($3.2), and dataset ($3.3) which have been proposed and used in the past to facilitate generalization. Inductive inference and the generalization of empirical findings is fundamental problem of science that has a long-standing history in many disciplines. Notable ex- amples include Occam's 's razor, Solomonoff's s inductive inference (Solomonoff, 1964), Kolmogorov complexity (Kolmogorov, 1998), the bias-variance-tradedd (Kohavi et al., 1996; Von Luxburg & Schölkopf, 2011), and the no free lunch theorem (Wolpert, 1996; Wolpert & Macready, 1997). In the context of statistical learning, Vapnik and Chervonenkis (Vapnik & Chervonenkis, 1982; Vapnik, 1995) showed that generalizing from a sample its population (i.e., IID generalization) requires re- stricting the capacity of the class of candidate functions-a type of inductive bias. Since shifts between train and test distributions violate the IID assumption, however, statistical learning theory does not directly apply to our types of OOD generalization.
OOD generalization across different (e.g., observational and experimental) conditions also bears connections to causal inference (Pearl, 2009; Peters et al., 2017; Hernán & Robins, 2020). Typically, a causal graph encodes assumptions about the relation between different distributions and is used to decide how to "transport" learned model (Pearl & Bareinboim, 201 1; Pearl et al., 2014; Bareinboim & Pearl, 2016; von Kügelgen et al., 2019). Other approaches aim to learn a model which leads to invariant prediction across multiple environments (Schölkopf et al., 2012; Peters et al., 2016; Heinze- Deml et al., 2018; 8; Rojas-Carulla et al., 2018; Arjovsky et al., 2019; Lu et al., 2021). However, these methods either consider a small number of causally meaningful variables in combination with domain knowledge, or assume access to data from multiple environments. In our setting, on the other hand, we aim to learn from higher-dimensional observations and generalize from a single training set to different test environment.
Our work focuses on OOD generalization in the context of visual representation learning. where deep learning has excelled over traditional learning approaches (Krizhevsky et al., 2012; LeCun al., 2015; Schmidhuber, 2015; Goodfellow al., 2016). the following, we therefore concentrate on inductive biases specific to deep neural networks (Goyal & Bengio, 2020) on visual data. For details regarding specific objective functions, architectures, and training, we refer to the supplement.
3.1 INDUCTIVE BIAS REPRESENTATIONAL FORMAT
Learning useful representations of high-dimensional data is clearly important for the downstream performance of machine learning models (Bengio et al., 2013). The first type inductive bias we consider therefore the representational format. A common approach to representation learning to postulate independent latent variables which give rise to the data, and try to infer these in unsupervised fashion. This is the idea behind independent component analysis (ICA) (Comon,
3

---

Published as a conference paper at ICLR 2022
1994; Hyvärimen & Oja, 2000) and has also been studied under the term disentanglement (Bengio et al., 2013). Most recent approaches learn a deep generative model based on the variational auto- encoder (VAE) framework (Kingma & Welling, 2013; Rezende et al., 2014), typically by adding regularization terms to the objective which further encourage independence between latents (Higgins et al., 2017; Kim & Mnih, 2018; Chen et al., 2018; Kumar al., 2018; Burgess et al., 2018).
is well known that ICA/disentanglement is theoretically non-identifiable without additional as- sumptions or supervision (Hyvärinen & Pajunen, 1999; Locatello et al., 2018). Recent work has thus focused on weakly supervised approaches which can provably identify the true independent la- tent factors (Hyvärinen & Morioka, 2016; Hyvarinen & Morioka, 2017; Shu al., 2019; Locatello et al., 2020a; Klindt et al., 2020; Khemakhem et al., 2020; Roeder et al., 2020). The general idea is to leverage additional information in the form of paired observations (xt, x²) where x is typically an auxiliary variable (e.g., an environment indicator or time-stamp) or a second view, i.e., X g(8*) with y' ~ p(yly"), where y' are the Vs of x² and p(y ly) depends on the method. We remark that such identifiability guarantees only hold for the training distribution (and given infinite data), and thus may break down once we move to a different distribution for testing. In practice, however, we hope that the identifiability of the representation translates to learning mechanisms that generalize.
In our study, we consider the popular 3- VAE (Higgins et al., 2017) as an unsupervised approach, as well as Ada-GVAE (Locatello et al., 2020a), Blow-VAE (Klindt et al., 2020) and PCL (Hyvarinen & Morioka, 2017) as weakly supervised disentanglement methods. First, we learn a representation given only (pairs of) observations (i.e., without access the Fo using encoder fenc R + g.. We then freeze the encoder (and thus the learned representation z) and train a multi-layer perceptron (MLP) ÍMLP R" + Rº predict the Fo y from z in a supervised way. The learned inverse mechanism f in this case is thus given by f = SMLP o Senne
E Th.h
3.2 INDUCTIVE BIAS 2: ARCHITECTURAL (SUPERVISED LEARNING)
The physical world is governed by symmetries (Nother, 1915), and enforcing appropriate task- dependent symmetries in our function class may facilitate more efficient learning and generalization. The second type of inductive bias we consider thus regards properties of the learned regression func- tion, which we refer to architectural bias. Of central importance are the concepts of invariance (changes in input should not lead to changes in output) and equivariance (changes in input should lead to proportional changes in output). In vision tasks, for example, object localizution exhibits equivariance to translation, whereas object classification exhibits invariance to translation. E.g., translating object in input image should lead (cquivariance), but should not affect the predicted object class (invariance).
equal shift in the predicted bounding box
A famous example is the convolution operation which yields translation equivariance and forms the basis of convolutional neural networks (CNNs) (Le Cun et al., 1989; LeCun et al., 1989). Combined with set operation such as pooling, CNNs then achieve translation invariance. More recently, the idea of building equivariance properties into neural architectures has also been successfully applied to more general transformations such rotation and scale (Cohen & Welling, 2016; Cohen al., 2019; Weiler & Cesa, 2019) or (coordinate) permutations (Zhang et al., 2019; Achlioptas et al., 2018). Other approaches consider affine transformations (Jaderberg et al., 2015), allow trade off invariance vs dependence on coordinates (Liu et al., 2018), or use residual blocks and skip connections to promote feature re-use and facilitate more efficient gradient computation (He et al., 2016; Huang et al., 2017 7). While powerful in principle, a key challenge is that relevant equivariances for a given problem may be unknown a priori or hard to enforce architecturally. E.g., 3D rotational equivariance is not easily captured for 2D-projected images, as for the MPI3D data set.
In our study, we consider the following architectures: standard MLPs and CNNs, CoordConv (Liu et al., 2018) and coordinate- based (Sitzmann et al., 2020) nets, Intationally-Eqqivaaitt
(Rotation- EQ) CNNs (Cohen & Welling, 2016), Spatial Transformers (STN) (Jaderberg et al., 2015), ResNet (RN) 50 and 101 (He et al., 2016), and DenseNet (Huang al., 2017). All networks f are trained to directly predict the FoVs y x f(x) in a purely supervised fashion.
3.3 INDUCTIVE BIAS 3: LEVERAGING ADDITIONAL. DATA (TRANSFER LEARNING)
The physical world is modular: many patterns and structures reoccur across variety of settings. Thus, the third and final type of inductive bias we consider is leveraging additional data through transfer learning. Especially in vision, it has been found that low-level features such as edges or
4

---

Published as a conference paper at ICLR 2022
simple textures are consistently learned in the first layers of neural networks, which suggests their usefulness across a wide range of tasks (Sun et al., 2017). ofate-of-the-art approaches therefore enormous image corpora prior to fine-tuning on data from the target
often rely pre-training task (Kolesnikov et al., 2020; Mahajan et al., 2018; Xie et al., 2020). The guiding intuition that additional data helps to learn common features and symmetries and thus enables more efficient use of the (typically small amount of) labeled training data. Leveraging additional data as an inductive bias connected the representational format $3.1 they are often combined during pre-training.
In our study, we consider three pre-trained models: RN-50 and RN-101 pretrained on ImageNet- 21k (Deng al., 2009; Kolesnikov et al., 2020) and a DenseNet pretrained on ImageNet- Ik (ILSVRC) (Russakovsky et al., 2015). We replace the last layer with randomly initialized readout layer chosen to match the dimension of the Vs of a given dataset and fine-tune the whole network for 50,000 iterations on the respective train splits.
EXPERIMENTAL SETUP
4.1 DATASETS
We consider datasets with images generated from set of discrete Fac- tors of Variation (FoVs) following a deterministic generative model. All selected datasets are designed such that all possible combinations of fac- tors of variation are realized in a corresponding image. dSprites (Matthey et al., 2017), composed of low resolution binary images of basic shapes with 5 FoVs; shape, scale, orientation, x-position, and y-position. Next, Shapes3D (Kim & Mnih, 2018), a popular dataset with 3D shapes in a room with 6 FoVs: floor, color, wall color, object color, object size, object type, and camera azimuth. Furthermore, with CelebGlow we introduce a novel dataset that has more natural factors of variations such as smiling, hair-color and age. For more details and samples, we refer to Appendix B. Lastly, we consider the challenging and realistic MPI3D (Gondal et al., 2019), which contains real images of physical 3D objects attached to a robotic finger generated with 7 FoVs: color, shape, size, height, background color, x-axss, and y-axis. For more details, we refer to Appendix H.I.
Random dataset samples from dSprites (Ist), Shapes3D (2nd), CelebGlow (3rd), and MP13D-real (4th).
Figure 3:
4.2 SPLITS
For each of the above datasets, denoted by DD we create disjoint splits of train sets Dir and test sets Die. We systematically construct the splits according to the underlying factors to evaluate different modalities of generalization, which we refer to as composition, interpolation, extrapolation, and random. See Fig. 2 for visual presentation of such splits regarding two factors.
We exclude all images from the train split factors are located in a particular corner of the FoV hyper cube given by all Vs.s. This means certain systematic combinations of FoVs are never seen during training even though the value of each factor is individually present in the train set. The related test split then represents images of which at least two factors resemble such an unseen composition of factor values, thus testing generalization w.r.t. composition.
Composition:
Interpolation: Within the range of values of each FoV, we periodically exclude values from the train split. The corresponding test split then represents images of which at least one factor takes one of the unseen factor values in between, thus testing generalization w.r.t. interpolation.
Extrapolation: We exclude all combinations having factors with values above a certain label thresh- old from the train split. The corresponding test split then represents images with one or more ex- trapolated factor values, thus testing generalization w.r.t. extrapolation.
Random: Lastly, as baseline to test our models performances across the full dataset in distri- bution, cover the case of an IID sampled train and test set split from D. Compared to inter- and extrapolation where factors are systematically excluded, here it is very likely that all individual factor values have been observed in some combination.
We further control all considered splits and datasets such that ~ 30% of the available data is in the training Dtr and the remaining ~ 70% belong to the test set Die- Lastly, do not split along factors of variation if no intuitive order exists. Therefore, we do not split along the categorical variable shape and along the axis of factors where only two values available.
5

---

Published as a conference paper at ICLR 2022
Un-/weaks supvervised
Fully supervised
Transfer
BetaVAE
MLP
RNSU
RN50 (Imag-Net-214)
Ada-GVAE
CNN
RN101
RN101 (ImageNes-2144 21k)
SlowVAE
CoordCony
DenseNe:
DenseNet (ImageNet-Ikk
PCL
Coordinate Based
Rotation EQ
Rotation EQ big
1.0
Spatial Transformer
0.5
0.0
random
interpolation (a) -coor on MPI3D-Real test splits.
composition
extrapolation
1.0
0.5-
random
composition
interpolation
extrapolation
(b) Rt² -score on CelebGlow test splits.
Figure 4: R² -core on various test-train splits. Compared to the in-distrihution random splits, on the out-of- distribution (OOD) splits composition, interpolation, and extrapolation, observe large drops in performance.
4.3 EVALUATION
To benehmark the generalization capabilities, compute the R2-score the coefficient of determi- nation, on the respective test set. We define the R2-scooee based on the MSE score per FoV yj
R2 =
MSE; of
with MSE, = E(xx))ED.. [(> [1y-
(1)
where of is the variance per factor defined on the full dataset D. Under this score, R2 can be interpreted as perfect regression and prediction under the respective test set whereas R² = indicates random guessing with the MSE being identical the variance per factor. For visualization purposes, we clip the Rº to O if is negative. We provide all unclipped values in the Appendix.
EXPERIMENTS AND RESULTS
Our goal is to investigate how different visual representation models perform on our proposed sys- tematic out-of- -iitt---ittiitttinn (OOD) test sets. We consider un-/weakly supervised, fully supervised, and transfer learning models. We focus our conclusions on MPI3D-Real as it is the most realistic dataset. Further results on dSprites and Shapes3D are, however, mostly consistent.
In the first subsection, 1,.1, we investigate the overall model OOD performance. In Sections 5.2 and 5.3, we focus on more in-depth eтror analysis by controlling the splits s.t. only a single fac- tor OOD during testing. Lastly, in $5.4, we investigate the connection between the degree of disentanglement and downstream performance.
5.1 MODEL PERFORMANCE DECREASES ON OOD TEST SPLITS
In Fig. and Appendix Fig. 11, we plot the performance of each model across different gener- alization settings. Compared to the in-distribution (ID) setting (random), we observe large drops in performance when evaluating our OOD test prominent on MPI3D-RD Real. Here, we further see that, on average, the performances seem to increase as we increase the supervision signal (comparing RN50, RN101, DenseNet with and without addi- tional data on MPI3D). On CelebGlow, models also struggle to extrapolate. However, the results on composition and interpolation only drop slightly compared to the random split.
all considered datasets. This effect is most
For Shapes3D (shown in the Appendix E), the OOD generalization is partially successful, especially the composition and interpolation settings. We hypothesize that this is due to the dataset specific, fixed spatial composition of the images. For instance, with the object-centric positioning, the floor, wall and other factors are mostly at the same position within the images. Thus, they can reliably be inferred by only looking at certain fixed spot in the image. In contrast, for MPI3D this is more difficult as, e.g., the robot finger has to be found to infer its tip color. Furthermore. the factors of variation in Shapes3D mostly consist of colors which encoded within the same input dimensions,

---

Published as a conference paper at ICLR 2022
Random
ID factors
0OD factors
1.0
0.5
0.0
BetaVAE
Ada GVAE
Slo
PCL
MLP
CNN
CoardCorv
Courdirat Baad
Ratation-E
Rotation EQ-big
STN
Rteo
RA101
RI5o (Imogefeer 214)
Densufer
RN101 (Imagetet
DemeNer (Imagefett 1k)
MP13D- Real extrapolation
1.0
0.5
0.0
(b) CelebGlow extrapolation
Figure 5: Extrapolation and modularity, R2-score on subsets. In the extrapolation setting, we further differentiate between factors that have been observed during training (ID factors) and extrapolated values (OOD factors) and measure the performances separately. As reference. we compare to random split. A model is considered modular, if it still infers ID factors correctly despite other factors being OOD.
and not across pixels as, for instance, x-translation in MPI3D. For this color interpolation, the ReLU activation function might be good inductive bias for generalization. However, it is not sufficient to achieve extrapolations, as we still observe a large drop in performance here.
Conclusion: The performance generally decreases when factors are OOD regardless of the super- vision signal and architecture. However, we also observed exceptions in Shapes3ID where OOD generalization was largely successful except for extrapolation.
5.2 ERRORS STEM FROM INFERRING OOD FACTORS
While in the previous section we observed a general decrease in R² score for the interpolation and extrapolation splits, our evaluation does not yet show how errors are distributed among individual factors that in- and distiibbbitiiionn.
In contrast to the previous section where multiple factors could be OOD distribution simultancously, here, control data splits (Fig. 2) interpolation, extrapolation s.t. only single factor is OOD. Now, we also estimate the Re-score separately per factor, depending on whether they have individually been observed during training (II) factor)
exclusively in the test set (OOD factor). For instance, if we only have images of a heart with varying scale and position, we query the model with hearts larger scales than observed during training (OOD factor), but at a previously observed position (ID factor). For a formal description, see Appendix Appendix H.2. This controlled setup enables us to investigate the modularity of the tested models, as we can separately measure the performance on OOD and ID factors. As reference for an approximate upper bound, we additionally report the performance of the model on a random train/test split.
In Figs. 5 and 14, we observe significant drops in performance for the OOD factors compared to random test-train split. contrast, for the IDD factors, we see that the models still perform close to the random split, although with much larger variance. For the interpolation setting (Appendix Fig. 14), this drop is also observed for MPI3D and dSprites but not for Shapes3D. Here, OOD and ID are almost par with the random split. Note that notion of modularity is based on systematic splits of individual factors and the resulting outputs. Other works focus on the inner behavior of a model by. c.g., investigating the clustering of neurons within the network (Filan et al., 2021). Preliminary experiments showed no correlations between the different notions of modularity.
Conclusion:
The tested models can be fairly modular, in the sense that the predictions of ID factors remain accurate. The low OOD performances mainly stem from incorrectly extrapolated or interpo- lated factors. Given the low inter-/extrapollatin (i.e., OOD) performances MPI3D and dSprites, evidently no model learned to invert the ground-truth generative mechanism.
5.3 MODELS EXTRAPOLATE SIMILARLY AND TOWARDS THE MRAN
In the previous sections, we observed that our tested models specifically extrapolate poorly on OOD factors. Here, we focus on quantifying the behavior of how different models extrapolate.
To check whether different models make similar errors, compare the extrapolation behavior across architectures and seeds by measuring the similarity of model predictions for the OOD fac- tors described in the previous section. No model is compared to itself if it has the same random
7

---

Published as a conference paper at ICLR 2022
25000
5000
5000
5000
0.0 0.5
0.0
0.5
1.0
1.5
0.0
0.5
1.0
1.5
0.0
0.5
1.0
1.5
Distance to mean ratio
(a) MPI3D-Ren Real
CelebGlow
(c) Shapes3D
(d) dSprites
Figure 6: Extrapolation towards the mean. We calculate (2) on the extrapolated OOD factors measure the closeness towards the mean compared to ground-truth. Here, the values are mostly in [0, 1]. Thus, models tend to predict values in previously observed ranges.
seed. On MPISD, Shapes3D and dSprites, all models strongly correlate with each other (Pearson p 2 0.57) but anti-correlate compared to the ground-truth prediction (Pearson p S -0.48), the over- all similarity matrix is shown in Appendix Fig. 17. One notable exception is on CelebGlow. Here, some models show low but positive correlations with the ground truth generative model (Pearson 2 0.57). However, visually the models are still quite off as shown for the model with the highest correlation in Fig. 18. most cases, the highest similarity is along the diagonal, which demon- strates the influence of the architectural bias. This result hints at all models making similar mistakes extrapolating a factor of variation.
We find that models collectively tend towards predicting the mean for each factor in the training distribution when extrapolating. To show this, we estimate the following ratio of distances
r 1(x'), V, lyj -5sl.
where y, = = E y is the mean of FoV yr. If values of (2) are E [0, 1]. models predict values which are closer to the mean than the corresponding ground-truth. We show a histogram over all supervised and transfer-based models for each dataset in Fig. 6. Models tend towards predicting the mean as only few values are >= This is shown qualitatively in Appendix Figs. 15 and 16.
Overall, we observe only small differences in how the tested models extrapolate, but a strong difference compared to the ground-truth. Instead of extrapolating, all models regress OOD factor towards the mean in the training set. We hope that this observation can be considered to develop more diverse future models.
Conclusion:
5.4 ON THE RELATION BETWEEN DISENTANGLEMENT AND DOWNSTREAM PERFORMANCE
Previous works have focused on the connection between disentanglement and OOD downstream performance (Träuble et al., 2020: Dittadi al., 2020; Montero al., 2021). Similarly, for our systematic splits, we measure the degree of disentanglement using the DCI-Disentanglement (East- wood & Williams, 2018) score on the latent representation of the embedded test and train data. Subsequently, we correlate it with the -fe-performance of a supervised readout model which we report $5.1. Note that the simplicity of the readout function depends on the degree of disen- tanglement, e.g., for perfect disentanglement up to permutation and sign flips this would just be an assignment problem. For the disentanglement models, we consider the un-/ weakly supervised models B- B-VAE(Higgsns al., 2017), Slow VAE (Klindt et al., 2020), Ada-GVAIELLocteell et al., 2020a) and PCL (Hyvarinen & Morioka, 2017).
We find that the degree of downstream performance correlates positively with the degree of disen- tanglement (Pearson p = 0.63, Spearman = 0.67). However, the correlations vary per dataset and split (see Appendix Fig. 7). Moreover, the overall performance of the disentanglement models followed by a supervised readout on the OOD split is lower compared to the supervised models (see e.g. Fig. 4). In ablation study with an oracle embedding that disentangles the test data up to permutations and sign flips, we found perfect generalization capabilities (R2 2 0.99).
Disentanglement models show improved performance in OOD generalization. Nev- ertheless, we observe a mostly positive correlation between the degree of disentanglement and the downstream performance.
Conclusion:
6 OTHER RELATED BENCHMARK STUDIES
In this section, we focus on related benchmarks and their conclusions. For related work in the context of inductive biases, we refer to $3.
Corruption benchmarks: Other current benchmarks focus on the performance of models when adding common corruptions (denoted by -C) such as noise snow to current dataset test sets,
8

---

Published as a conference paper at ICLR 2022
resulting in ImageNet-C. CIFAR- 10-C, Pascal-C, Coco-C, Cityscapes- and MNIST-C (Hendrycks & Dietterich, 2019; Michaelis et al., 2019; Mu & Gilmer, 2019). In contrast, in our benchmark, we assure that the factors of variations are present in the training set and merely have to be generalized correctly. In addition, our focus lies on identifying the ground truth generative process and its underlying factors. Depending on the task, the requirements for model are very different. E.g., the ImageNet-C classification benchmark requires spatial invariance, whereas regressing factors such as, e.g., shift and shape of an object, requires in- and equivariance.
A Abstract reasoning: Model performances on OOD generalizations are also intensively studied from the perspective of abstract reasoning, visual and relational reasoning tasks (Barrett et al., 2018: Wu et al., 2019; Santoro et al., 2017: Villalobos et al., 2020; Zhang et al., 2016; Yan & Zhou, 2017; Funke et al.,, 2021; Zhang et al., 2018). Most related, (Barrett et al., 2018; Wu al., 2019) also study similar interpolation and extrapolation regimes. Despite using notably different tasks such as abstract or spatial reasoning. they arrive at similar conclusions: They also observe drops in performance in the generalization regime and that interpolation is, in general, easier than extrapolation, and also hint at the modularity of models using distractor symbols (Barrett et al., 2018). Lastly, posing the concept of using correct generalization as necessary condition to check whether underlying mechanism has been learned has also been proposed in (Wu et al., 2019; Zhang et al., 2018; Funke et al., 2021).
Disentangled representation learning: Close to work, Montero et al. (Montero et al., 2021) also study generalization in the context of extrapolation, interpolation and a weak form of composi- tion on dSprites and Shapes3D, but not the more difficult MPT3D--attset. They focus reconstruc- tions of unsupervised disentanglement algorithms and thus the decoder, a task known to be theoret- ically impossible(Locatelee
et al., 2018). In their setup, they show that OOD generalization is lim- ited. From their work, it remains unclear whether the generalization along known factors is general problem in visual representation learning. and how neural networks fail to generalize. We try to fill these gaps. Moreover, we focus on representation learning approaches and thus on the encoder and consider a broader variety of models, including theoretically identifiable approaches (Ada-GAVE, VAF, PCI.), and provide a thorough in-depth analysis of how networks generalize.
Previously, Träuble et al. (2020) studied the behavior of unsupervised disentanglement models on correlated training data. They find that despite disentanglement objectives, the learned latent spaces mirror this correlation structure. In line with our work, the results of their supervised post-hoc regression models Shapes3D suggest similar generalization performances spective disentanglement models in Figs. 4 and 11. OOD generalization w.r.t. extrapolation of one single FoV is also analyzed in (Dittadi et al., 2020). Our experimental setup in $5.4 is similar to their 'OOD2' scenario. Here, our results are in accordance, as we both find that the degree of disentanglement is lightly correlated with the downstream performance.
we see in
Others: To demonstrate shortcuts in neural networks, Eulig et al. (2021) introduce a benchmark with factors of variations such color on MNIST that correlate with a specified task but control for those correlations during test-time. In the context of reinforcement learning, Packer et al. (2018) assess models on systematic test-train splits similar to our netraplextraoooaaiionn and show that current models cannot solve this problem. For generative adversarial networks (GANs), it has also been shown that their learned representations do not extrapolate beyond the training data (Jahanian et al., 2019).
7 DISCUSSION AND CONCLUSION
In this paper, we highlight the importance of learning the independent underlying mechanisms be- hind the factors of variation present in the data to achieve generalization. However, we empirically show that among a large variety of models, no tested model succeeds in generalizing to all our pro- posed OOD settings (extrapolation, interpolation. composition). We conclude that the models are limited in learning the underlying mechanism behind the data and rather rely on strategies that do not generalize well. We further observe that while one factor out-of-distribution, most other in- distribution factors inferred correctly. In this sense, the tested models are surprisingly modular.
To further foster research on this intuitively simple, yet unsolved problem, we release our code as a benchmark. This benchmark, which allows various supervision types and systematic controls, should promote more principled approaches and can be seen as a more tractable intermediate mile- stone towards solving more general OOD benchmarks. In the future, a theoretical treatment identi- fying further inductive biases of the model and the necessary requirements of the data to solve our proposed benchmark should be further investigated.
9

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGEMENTS
The authors thank Steffen Schneider, Matthias Tangemann and Thomas Brox for their valuable feedback and fruitful discussions. The authors would also like thank David Klindt, Judy Borowski, Dylan Paiton, Milton Montero and Sudhanshu Mittal for their constructive criticism of the manuscript. The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting FT and LS. We acknowledge support from the German Federal Ministry of Education and Research (BMBF) through the Competence Center for Machine Learn- ing (TUE.AI, FKZ (839)) and the Bernstein Computational Neuroscience Program Tübingen (FKZ: 000Q))02). WB acknowledges support via his Emmy Noether Research Group funded by the German Science Foundation (DFG) under grant no. BR 6382/1-1 as well as support by Open Phi- lantropy and the Good Ventures Foundation. MB and WB acknowledge funding from the MICrONS program of the Advanced Research Projects Activity (IARPA) via Department of Interior/Interror Business Center (DoI/IBC) contract number D16PC00003.
10

---

Published as a conference paper at ICLR 2022
REFERENCES
Panos Achlioptas. Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning representations and generative models for 3d point clouds. In International conference on machine learning, pp. 40-49. PMLR, 2018.
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.00 02893, 2019.
Aharon Azulay and Yair Weiss. Why do deep convolutional networks generalize poorly to small image transformation?? Journal of Machine Learning Research, 20(184):1-22 2019.
Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. Objectnet: A large-scale bias-controlled dataset for pushing limits object recognition models. In Advances in Neural Information Processing Systems, pp. 9448 9458, 2019.
Elias Bareinboim and Judea Pearl. Controlling selection bias in causal inference. Artificial Intelligence and Statistics, pp. 100-108. PMLR, 2012.
Elias Bareinhoim and Judea Pearl. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, 113(27):335--735) 2016.
David Barrett, Felix Hill, Adam Santoro, Ari Morcos, and Timothy Lillicrap. Mcasuring abstract reasoning in neural networks. In International conference on machine learning, pp. 511-1111155 511-520. PMLR, 2018
Peter W Battaglia, Jessica B Hamrick, and Joshua Tenenbaum. Simulation as an engine of physical scene understanding. Proceedings of the National Academy of Sciences, 110045):18327-18322 2013.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives.
IEEE transactions on pattern analysis and machine intelligence, 35(8))199--1822 2013.
M. Besserve, R. Sun, D. Janzing, and B. Schölkopf. A theory of independent mechanisms for extrapolation in generative models. In 35th AAAI Conference on Artificial Intelligence: A Virtual Conference, 2021.
Christopher P Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Desjardins, and Alexander Lerchner. Understanding disentangling in 3-VAF. arXiv preprint arXiv: (004:800403333 2018.
Ricky TQ Chen, Xuechen Li, Roger Grosse, and David Duvenaud. Isolating sources of discntanglement in In Proceedings of the 32nd International Conference Neural Information Processing Systems, pp. 2615-2625, 2018.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference on machine learning, pp. 2990-2999, PMLR, 2016.
Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge equivariant convolutional networks and the icosahedral cnn. In International Conference on Machine Learning, pp. 1321-1330. PMLR, 2019.
Pierre Comon.
Independent component analysis, new concept? Signal processing, 36(3):887-114, 1994.
Róbert Csordás, Sjoerd Steenkiste, and Jürgen Schmidhuber. Are neural nets modular? inspecting func- tional modularity through differentiable weight masks. International Conference on Learning Represen- tations, 2021. URL https: owwnreveeteeeeeee..ee ex net foooooormmi/ 7u7Vup gMD>.
Alexander Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D Hoffmann, et al. Underspecification presents challenges for credibility in modern machine learning. arXiv preprint 2xx::2011.0339) 2020.
Stanislas Dehaene. How We Learn: Why Brains Learn Better Than Any Machine... for Now. Penguin, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Ieee, 2009.
Andrea Dittadi, Frederik Träuble, Francesco Locatello, Manuel Wüthrich, Vaibhav Agrawal, Ole Winther, Stefan Bauer, and Bernhard Schölkopf. On the transfer of disentangled representations in realistic settings. arXiv preprint arXiv:2010. 14407, 2020.
Cian Fastwood and Christopher KI Williams. A framework for the quantitative evaluation of disentangled representations. In In International Conference Learning Representations, 2018.
11

---

Published as a conference paper at ICLR 2022
Elias Eulig, Piyapat Saranrittichai, Chaithanya Kumar Mummadi, Kilian Rambach, William Beluch, Xiahan Shi, and Volker Fischer. Diagvib-6: diagnostic benchmark suite for vision models in the presence of shortcut and generalization opportunities. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10655-100000 10664, 2021.
Muhammad Fahad, Arsalan Shahid. Ravi Reddy Manumachu, and Alexey Lastovetsky. A comparative study of methods for measurement of energy computing. Energies, 12(11):2204, 2019.
Daniel Filan, Stephen Casper, Shlomi Hod, Cody Wild, Andrew Critch, and Stuart Russell. Clusterability in neural networks. arXiv preprint (t:::200300386 2021.
C. M. Funke, J. Borowski, K. Stosio, W. Brendel, T. S. A. Wallis, and M. Bethge. Five points to check when comparing visual perception in humans and machines. Journal of Vision, Mar 2021. URL https: // jjvv arve journal 1 s. orgg http://rrric....s.aaattt..eeaaa.
Robert Geirhos, Hennie Jacobsen, Claudio Michaclis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11): 665-673, 2020.
Muhammad Waleed Gondal, Manuel Wuthrich, Djordje Miladinovic, Francesco Locatello, Martin Breidt, Valentin Volchkov, Joel Akpo, Olivier Bachem, Bernhard Schölkopf, and Stefan Bauer. On the transfer of inductive bias from simulation to the real world: a new disentanglement dataset. Advances Neural Information Processing Systems, pp. 15714-15725, 2019.
lan Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT press Cambridge, 2016.
Anirudh Goyal and Yoshua Bengio. Inductive biases for deep learning of higher-level cognition. arXiv preprint arXiv:2011.1509) 2020.
Klaus Greff, Sjoerd van Steenkiste, and Jürgen Schmidhuber. On the binding problem in artificial neural networks. arXiv preprint arXiv:20/2.05208. 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Christina Heinze Heinze-Deml, Jonas Peters, and Nicolai Meinshausen. Invariant causal prediction for nonlinear mod- cls. Journal of Causal Inference, 6(2), 2018.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. Proceedings of the International Conference on Learning Representations, 2019.
Miguel A Hernán and James M Robins. Causal inference: what Boca Raton: Chapman & Hal//RC, 2020.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mo- hamed, and Alexander Lerehner. beta-VAE: Learning basic visual concepts with constrained variational framework. In Internattional Conference on Learning Representations, 2017.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings the IEEE conference on computer vision and pattern recognition, pp. 4700- 4708, 2017.
Aapo Hyvärinen and Hiroshi Morioka. Unsupervised feature extraction by linear ica. In Proceedings of the 30th International Conference Neural Information Processing Systems, pp. 3772-3780 3777- 2016.
learning and non-
Aapo Hyvarinen and Hiroshi Morioka. Nonlinear ica of temporally dependent stationary sources. In Artificial Intelligence and Statistics, pp. 460 469, PMLR, 2017.
Aapo Hyvärinen and Erkki Oja. Independent component analysis: algorithms and applications. Neural net- works, 13(4-5):411-444 1--4300 2000.
Aapo Hyvärinen and Pettrri Pajunen, Nonlinear independent component analysis: Existence and uniqueness results. Neural networks, 12(3):49--39, 1999.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bun Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, vol- ume 32. Curran Associates, Inc., 2019. URL https: //pp///eoeee/ee ngs. iig.nng...n. neurinn fi File//224220998d8 cee f2ec1 9b37151 Paper .ddf.
ieerrppp.. pa cctpppprr///119/
12

---

Published as a conference paper at ICLR 2022
Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial transformer networks. arXiv preprint 150.::05006002255. 2015.
Ali Jahanian, Lucy Chai, and Phillip Isola. On the" steerability" of generative adversarial networks. arXiv preprint orXiv:1907.07110 2019.
Samil Karahan, Merve Kiline Yildirum, Kadir Kirtac, Ferhat Sukru Rende, Gultekin Butun, and Hazim Kemal Ekenel. How image degradations affect deep enn-hased face recognition? In 2016 International Conference of the Biometrics Special Interest Group (BIOSIG), pp. 1-5. IEFE, 2016.
Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autocncoders and nonlinear iea: A unifying framework. In International Conference on Artificial Intelligence and Statistics, pp. 2207-2217. PMLR, 2020.
N. Kilbertus*, G. Parascandolo*, B. Schölkopf*. Generalization anti-causal learning. In NewrIPS 2018 Workshop on Critiquing and Correcting Trends in Machine Learning, 2018. URL https:// mi- md-critique-corzett
git.ub.............. io/. *authors are listed in alphabetical order.
Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In International Conference Machine Learn- ing, pp. 2649-2658. PMLR. 2018
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6111 2013.
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gamett (eds.), Advances in Neural Infor- mation Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https //tpccedeiggg...... dd 5aw666222200022cccooooooo wtt.://////////9//////// 31: .pd..
neurips. cc wwtpppppppee////ee/
David Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhaninov, Wieland Brendel, Matthias Bethge, and Dylan Paiton. Towards nonlinear disentanglement in natural data with temporal sparse coding, arXiv preprint arXiv:2007.10930, 2020.
Ron Kohavi, David H Wolpert, et al. Bias plus variance decomposition for zero-one loss functions. In ICML, volume 96, pp. 275 83, 1996.
Alexander Kolesnikov, Lucas Beyer, Xiaohua 7hai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 16, pp. 491-507. Springer, 2020.
Andrei N. Kolmogorov. On tables of random numbers (reprinted from "sankhya: The indian journal of statistics", series vol. 25 part 1963). Theor. Comput. Sci., 207(2)387-395, 1998. URL http: //dblp. cami-trier.deo del db/ journals ttts/tt///tte /cccs/ccs20.cc. httm Mol::mlloollooooooovv
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25:1097-1105, 2012.
Abhishek Kumar, Prasanna Sattigeri, and Avinash Balakrishnan. Variational inference of disentangled latent concepts from unlabeled observations. International Conference on Learning Representations, 2018.
Brenden Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of
sequence-to toqueunncee recurrent networks. In International Conference on Machine Learning, pp. 2873- 2882. PMLR, 2018.
Brenden Lake, Tomer Ullman, Joshua Tenenbaum, Samuel Gershman. Building machines that learn and think like people. Behavioral and brain sciences, 40, 2017.
Yann Le Cun, Bernhard Boser, John S Denker, Donnic Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Handwritten digit recognition with back propagation network. In Proceedings of the 2nd International Conference on Neural Information Processing Systems, pp. 396-404, 1989.
Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, (4):541-551, 1989.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436-444 444, 2015.
13

---

Published as a conference paper at ICLR 2022
Rosanne Liu, Joel Lehman, Piero Molino, Felipe Petroski Such, Eric Frank, Alex Sergeev, and Jason Yosinski. intriguing failing convolutional neural networks the coordconv solution. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Pro- cessing Systems, volume 31. Curran Associates, Inc., 2018, URL https: httpttp:/pprccce.iiii ppw.oodiooo..g...ooooopppp nourips.. co/paper/ papper/2011 201///1//////// 8/ faleeil////// e 601 0688 688868888888 97779777711 7 eel 5dd 7bc9a83d1 Paper Paf...pff..
Ziwei Liu, Ping Luo, Xiaogang Wang. and Xiaoou Tang. Deep learning face attributes in the wild. In Proceed- ings International Conference Computer Vision (ICCV), December 2015.
Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions the unsupervised learning disentangled repre- sentations, arXiv preprint 181 2018
Francesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf. Olivier Bachem, and Michael Tschan- nen. Weakly-supervised disentanglement without compromises. arXiv preprint arXiv:2002.02885 02886, 2020a.
Francesco Locatello, Dirk Weissenbom, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention. In Advances in Neural Information Processing Systems, 2020b.
Chaochao Lu, Yuhuai Wu, Jose Miguel Hemández-Lobart and Berhaard Schölkopf. Nonlinear invariant risk minimization: A causal approach. arXiv preprint arXiv:2102.1233 2021.
Dhruv Kumar Mahajan, Ross B. Girshick. Vignesh Ramanathan, Kaiming He. Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten. Exploring the limits of weakly supervised pretraining. In ECCV, 2018.
Gary Marcus. Deep learning: critical appraisal. arXiv preprint arXiv:1801.006331 2018.
Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dsprites: Disentanglement testing httpe/huw.oommdeepiiidddddiiiiie
sprites dataset.
2017.
Claudio Michaclis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander Ecker, Matthias Bethge, and Wieland Brendel, Benchmarking robustness in object detection: Autonomous driving when winter is coming. arXiv preprint 1907.07484 2019.
Milton leraa Montero, Casimir JH I Ludwig, Rui Ponte Costa, Gaurav Malhotra, and Jeffrey Bowers. The role of disentanglement generalisation. In International Conference Learning Representations, 2021. URL https: wpwnrevieeeeeeeeeeeeeeeeeeeeeeee n forum? 1d=qbB977 4 JKUVy.
Norman Mu Justin Gilmer. Mnist-c: A robustness benchmark for computer vision. arXiv preprint arXiv: 02337, 2019.
David Mytton. Assessing the suitability of the greenhouse gas protocol for calculation of emissions from public cloud computing workloads. Journal of Cloud Computing, 91)::-111, 2020.
Emmy Nother. The finiteness theorem for invariants of finite groups In Mathematische Annalen, 77, 89-92, 1915.
Charles Packer, Katelyn Gao, Jemej Kos, Philipp Krähenbühh, Vladlen Koltun, and Dawn Song. Assessing generalization in deep reinforcement learning. arXiv preprint 1810v 12282, 2018.
G. Parascandolo, N. Kilbertus, M. Rojas-Carulla, and B. Schölkopf. Learning independent causal mechanisms. In Proceedings of the 35th International Conference Machine Learning (ICML), volume 80 of Proceed- ings of Machine Learning Research, pp. 4033 4041. PMLR, July 2018. URL http:/ htttt::://prcceedigge. mir. press/ /v0 parascandolollooocccc.. cwroooooooaaa... html.
Judea Pearl. Causality. Cambridge university press, 2009.
Judea Pearl and Elias Bareinboim. Transportability of causal and statistical relations: formal approach. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 25, 2011.
Judea Pearl, Elias Bareinboim, et al. External validity: From do-calculus to transportability across populations. Statistical Science, 29(4):579-595 2014.
Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference by using invariant prediction: identi- fication and confidence intervals. Journal of the Royal Statistical Society. Series B (Statistical Methodology), pp. 947-1012, 2016.
14

---

Published as a conference paper at ICLR 2022
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements causal inference: foundations and learn- ing algorithms. The MIT Press, 2017.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning, pp. 1278-1286.
PMLR, 2014.
Geoffrey Roeder, Luke Metz, and Diederik P Kingma. On linear identifiability leamned representations. arXiv preprint arXiv:2007.008/ 2020.
Mateo
Berhhard Schölkopf, Richard Tumeer and Jonas Peters, Invariant models for causal transfer learning. The Journal Machine Learning Research, 19(1):309-1342, 2018.
Prasun Roy. Subhankar Ghosh, Saumik Bhattacharya, and Umapada Pal. Effects of degradations on deep neural network architectures. arXiv preprint 1807.10106 2018
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Joumal of Computer Vision (IJCV), 115(3)211-252, 2015. doi: 10. 1007/s 126--01--0816-y.
Adam Santoro, David Raposo, David Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. simple neural network module for relational reasoning. I. Guyon, U. V. Luxburg. S. Bengio, H. Wallach, R. Fergus, Vishwanathan, and R. Garnett (eds.), Advances in Neural Informa- tion Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: /ppppeeeceenng.... neuri ps. wwtpcc/ppppp2/11111 /200//2111222 fi e/efaccff 4b0f
5938aa Maassallf-Paaeer .pff.
Jürgen Schmidhuber: Deep learning in neural networks: An overview. Neural networks, 61:85-117, 2015.
B. Schölkopf. D. Janzing, Peters, E. Sgouritsa, Zhang, and J. M. Mooij. On causal and anticausal learning. In Proceedings of the 29th International Conference on Machine Learning (ICML), pp. 1255-1262, 2012.
Berhhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IPFE, 2021.
Rui Shu, Yining Chen, Abhishek Kumar, Stefano Ermon, and Ben Poole. Weakly supervised disentanglement with guarantees. In International Conference on Learning Representations, 2019.
Vincent Sitzmann, Julien NP Martel, Alexander W Bergman, David B Lindell, and Gordon Wetzstein. Implicit ncural representations with periodic activation functions, arXiv preprint arXiv:2006. 09661, 2020.
R. Solomonoff. A formal theory of inductive inference. Information and Control, Part II, 7(2):224-254, 1964.
Elizabeth S Spelke. Principles of object perception. Cognitive science, 14(1):29-56, 1990.
Amos Storkey. When training and test machine learning, 30:3-28, 2009.
different: characterizing learning transfer. Dataset shift
Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and policy considerations for deep learning in nlp, 2019.
Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta. Revisiting unreasonable effectiveness of data in deep learning era. In Proceedings of the IEEE international conference on computer vision, pp. 843-852 003-852,200 2017.
Emö Téglás, Edward Vul, Vittorio Girotto, Michel Gonzalez, Joshua B Tenenhaum, and Luca I. Bonatti, Pure reasoning in 12-month-old infants as probabilistic inference. Science, 332(6033) 2011.
Frederik Träuble, Elliot Creager. Niki Kilbertus, Francesco Locatello, Andrea Dittadi, Anirudh Goyal, Bern- hard Schölkopf, and Stefan Bauer. disentangled representations learned from correlated data. arXiv preprint arXiv:2006.0 07886, 2020.
Vladimir N Vapnik. The nature of statistical learning theory. Springer International Publishing, 1995.
Vladimir N Vapnik and A Ya Chervonenkis. Necessary and sufficient conditions for the uniform convergence of means to their expectations. Theory of Probability & Its Applications, 26(3):532-553, 1982.
Kimberly Villalobos. Vilim Stih, Amineh Ahmadinejad, Shobhita Sundaram, Jamell Dozier, Andrew Francl, Frederico Azevedo, Tomotake Sasaki, and Xavier Boix. Do neural networks for segmentation understand insideness? Technical report, Center for Brains, Minds and Machines (CBMM), 2020.
15

---

Published as a conference paper at ICLR 2022
Julius von Kügelgen, Alexander Mey, and Marco Loog, Semi-generative modelling: Covariate-shift adaptation with cause and effect features. The 22nd International Conference on Artificial Intelligence and Staristics, pp. 3561-1369 PMLR, 2019.
Ulrike Von Luxburg and Bernhard Schölkopf. Statistical learning theory: Models, concepts, and results. In Handbook of the History of Logic, volume 10, pp. 651-706. Elsevier, 2011.
Maurice Weiler and Gabriele Cesa. General Equivarrit Steerable CNNs. Conference on Neural Information Processing Systems (NewrlPS), 2019.
David Н. Wolpert. The lack of a priori distinctions between learning algorithms. Neural Comput. 8(7): 1341-1390, October 1996. ISSN 0899-7667. doi: 00.162/neco.1996. 1996....99........ 8.713.17 URL https: org/10. /2/ /ww.. 1996.99.8. 7. 7.131.
D.H. Wolpert and W.G. Macready. No free lunch theorems for optimization. IEEE Transactions on Evolution- ary Computation, 1(1):67-82 82, 1997. doi: 10.1 019942355588892
Xiaolin Wu, Xi Zhang, and Jun Du. Challenge of spatial cognition for deep learning. CoRR, abs/ (et//1908009999 /1abs/9998. 04396.
2019. URL http: /
Qizhe Xie. Minh- Thang Luong, Eduard Hovy, and Quoe V Le. Self-training with noisy student improves ima- genet classification. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recogni- tion, pp. 10687-10698, 2020.
Z. Yan and X. Zhou. How intelligent are convolutional neural networks? ArXiv, abs//1099006 0020990612666 2017.
Renqiao Zhang. Jiajun Wu, Chengkai Zhang, William T. Freeman, and Joshua B. Tenenbaum. A comparative evaluation of approximate probabilistic simulation and deep neural networks as accounts of human physical scene understanding. CoRR, abs/1605.00 100080 2016. URL http: aawviv.vvvv..///iia/..a.aaaaaiva org abs / 605. 01138.
Xinhua Zhang. Yijing Watkins, and Garrett T Kenyon. Can deep learning learn the principle of closed contour detection? In International Symposium Visual Computing, pp. 455-460. Springer. 2018
Yan Zhang, Jonathon Hare, and Adam Prugel-Bennett Deep set prediction networks. In H. Wallach, H. Larochelle, A. Beygelzimer, E. d'Alché-Bur E. Fox, and R. Garnett (eds.), Advances in Neural Infor- mation Processing Systems, volume 32, Curran Associates, Inc., 2019. URL tpsp:: / /ttp//ppepp.gg....///e//// neuri
e 9ed0 5bae22 75 4e2 5beeeee54e4333333333 Paper, Paper....
16

---

Published as a conference paper at ICLR 2022
ETHICS STATEMENT
Our current study focuses on basic research and has no direct application or societal impact. Never- theless, think that the broader topic of generalization should be treated with great care. Especially oversimplified generalization and automation without human in the loop could have drastic conse- quences in safety critical environments court rulings.
Large-scale studies require a lot of compute due to multiple random seeds and exponentially growing sets of possible hyperparameter combinations. Following claims by Strubel et (Strubell al., 2019), we tried to avoid redundant computations by orienting ourselves current common values the literature and by relying on systematic test runs. In a naive attempt, we tried to estimate the power consumption and greenhouse gas impact based on the used cloud compute instance. However, too many factors such as external thermal conditions, actual workload, type of power used and others involved (Mytton, 2020; Fahad et al., 2019). In the future, especially with the trend towards larger network architectures, compute clusters should be required to enable options which report the estimated environmental impact. However, it should be noted that cloud vendors already among the largest purchasers renewable electricity (Mytton, 2020).
For impact statement for the broader field of representation learning, refer to Klindt et al. (2020).
REPRODUCIBILIIY STATEMENT
Our code is attached, and all important details to reproduce results are repeated in Appendix H.
17

---

Published as a conference paper at ICLR 2022
1.0
dSprites 0.53
0.73
0.39
0.31
0.76
0.5
Shapes3D
0.39
0.41
0.12
-0.38
0.23
0.0
-0.5
MPB3D
0.95
0.95
0.53
0.73
0.45
1.0
none
random
compoition
interpolation
extrpolation
Figure 7: Spearman Correlation of degree of disentanglement with downstream performances. We measure the DCClDisentangleemet metric on the 10-dimensional representation for 8-VAE, PCL, SlowVAE Ada-GVAE corresponding I?2 -core on the downstream performance. All p-values are below 0.01 except for composition on Shapes3D which has p-value=0.14 Note that, we here provide Spearman's rank correlation instead Pearson as the p-values are slightly lower.
Data set
Modification
R-squared Test
Modification
R-squared Test
dSprites dSprites
random
1.000
random + sign-flpp
1.000
composition
1,000
composition sign-flip
1.000
dSprites
interpolation
1.000
interpolation + sign-flip
1.000
dSprites
extrapolation
1.000
extrapolation sign-flip
0.999
Shapes3D
random
1.000
random + sign-flip
1.000
Sh
composition
1.000
composition sign-flin flip
1.000
Shapes3D
interpolation
1.000
interpolation + sign-flip
1,000
Shapes3I
extrapolation
1.000
extrapolation + sign-flip
1.000
MPI3D-Real
random
1.000
random + sign-flpp
1.000
MPI3D-Real
composition
1,000
composition + sign-flip
0.996
MPI3D-Real
interpolation
1,000
interpolation sign-flip
1.000
MPI3D-Real
extrapolation
0.999
extrapolation + sign-flip
0.997
Table 1: Performances of the readout-ML.P the ground-truth.
A CONNECTION BETWEEN READOUT PERFORMANCE AND DISENTANGLEMENT OF THE REPRESENTATION
Here, narrow down the root cause of the limited extrapolation performance of disentanglement models in the OOD settings as observed in Figs. and 11. More precisely, we investigate how the readout-MLP would perform our readout MLP directly on the ground-trute factors of variation for all possible test-train splits described in Fig. 2 and measured the -c-oe test error for each split. Here, the MLP only has to learn the identity function. In slightly more evolved setting, termed sign-flip, we switched the sign input to train the readout-MI.P on mapping from -ground-truth to ground-truth. This mimies the identifiability guarantees of models like Slow VAE which are up to permutation and sign flips under certain assumptions. The R-squared for all settings in Table are > .99, therefore the readout model should not be the limitation for OOD generalization in our setting if the representation is identified up to permutation and sign flips. Note that this experiment does not cover disentanglement up to point-wise wise nonlinearities linear/ affine transformations required by other models.
perfectly disentangled representation. Therefore, train
B CELEBGLOW DATASET
The current disentanglement datasets such as dSprites, Shapes3I,, MP13D, and others are con- structed based on highly controlled environments (Matthey et al., 2017: Kim Mnih, 2018; Gondal et al., 2019). Here, common factors of variations are rotations scaling of simple geometric ob- jects, such as a square. For a more intuitive investigation of other factors, we created the CelebGlow dataset. Here, the factors of variations Note that we rely on the Glow model instead of taking a real-world dataset, as this allows for a gradual control of individual factors of variation.
smiling. blondness and age. Samples shown in Fig. 8.
18

---

Published as a conference paper at ICLR 2022
Blondness
Smiling
Smiling
(a) Age 1/6 (young)
(b) Age 6/6 (old)
Figure 8: CelebGlow Dataset
The CelebGlow dataset is created based on the invertible generative neural network of Kingma et al. (Kingma & Dhariwal, 2018). We used their provided network that is pretrained on the Celeb-HQ dataset, and has labelled directions in the model-latent space that correspond to specific attributes of the dataset. Based on this latent space, created the dataset follows:
1. In the latent space of the model, we sample from a high dimensional Gaussian with zero mean and a low standard deviation of 0.3 to avoid too much variability.
2. Next, we perform latent walk into the directions that correspond to "Smiling". "Age" and "Blondness" in image space. To estimate the spacing, we rely on the function ma mipulate range". We perform 6 steps along each axis and all combinations (6x6x6 cube). As a scale parameter to the function, we use 0.8. Those factors were chosen s.t. the images differ significantly, but also to stay in the valid range of the model based on visual inspection.
3. We pass all latent coordinates through the glow network in the generative direction.
4. We further somp-sample the images from 256x256x3 to 64x64x3 to match the resolution of common disentanglement datasets.
5. Finally, store each image and the corresponding factor combination.
This procedure repeated for 1000 samples to get 6 6 1000 = 216000 samples in total, which around the same size other common datasets.
C HYPERPARAMETER TUNING ABLATION
As described in the implementation details, use common values from the literature to train the proposed models. Here, we investigate effects of such hyperparameters on the CNN architecture. Due to the combinatorial complexity, perparameters, we varied the number or training iterations (3 different numbers of iterations), we introduced 5 different strengths of regularization, different depths for the CNN architecture [6 layers, layers] and ran multiple random seeds for each combination.
do not perform search for other architectures. As hy-
The results on the extrapolation test MPI3D are shown in Fig. Given this hyperparameter search, we find no improvement over our reported numbers for the CNN.
The network can he found at: Attps:///g wwwpphhww...........eeete /g' ow/bl wt/masttr//eeeeeeeee/eesees demo/ script.. s.. 1.24
https:: / thub. www.apenaiioooooooooooooooo.aa.a /gl cw/b1 http://ll..mmmttteeeeeeeeee .py 1.219
19

---

Published as a conference paper at ICLR 2022
3005 2H
2005 24
-1.0
Le t2 Weight decoy
De 12
1.c
Weight decay
1.00 00
5.G++00
Maximem number itcrations
15c.
1e4000
3.cc 0o
00-04
Lowto
5.G-404
Maximam number iterations
05
1.0400
5.00 100
Lo-04
(a) 6 layer CNN
(b) 9 layer CNN
Figure 9: Hyperparameter search on MPI3D for a CNN.
MPI3D Toy
Un nne/wwakly supvervised
Fully supervised
Transfer
BetaVAE
MLP
RN50
RN50 (ImageNet-21k)
Ada-GVAE
CNN
RN101
RN101 (ImageNet-21k)
SlowVAE
CoordConv
DenseNet
DenseNet (ImageNet-1k)
PCL
Coordinate Based
Rotation-EQ
Rotation EQ big
Spatial Transformer
1.0
0.5
0.0
random
composition
interpolation
extrapolation
Figure 10: -s-oe MPI3D-Synthetic.
D REAL VERSUS SYNTHETIC DATASET
To narrow down the question "why the generalization capabilities drop real-world dataset MPI3D?", we run a comparison on MPI3D dataset with real and synthetic images.
The results the MPI3D dataset with synthetic images is shown in Fig. 10 and Table 10. Comparing this with R-squared performances to MPI3D with real-world images (Fig. and Table 9), we observe that the results do not change significantly (most results are in 1-2sigma range). We conclude that the larger drops in performance on MP13D compared to Shapes3D or dSprites, are not due to the real images opposed to synthetic images. Instead, hypothesize that is due to the more realistic setup of the MPI3D dataset itself. For instance, contains complex factors like rotation in 3D projected 2D. Here, occluded parts of objects have be guessed based certain symmetry assumptions.
E ABLATION ON NON-AMBIGUOUS DSPRITES
The setup of dSprites non-injective, as different rotations map to the same image. E.g., the square at rotation 90 is identical to the rotated by 180º and therefore ambiguous. Thus, the training process is noisy. an ablation study, we controlled for this by constraining the rotations to lie in [0, 90). We again all proposed models and report the R² in Fig. 11b.
20

---

Published as a conference paper at ICLR 2022
Un. Un-/weakly supvervised
Fully supervised
Transfer
BetaVAE
MLP
RN50
RN50 (ImageNet-21kk)
Ada-GVAE
CNN
RN101
RN101 (ImageNet-21k)
SlowVAE
CoordConv
DenseNet
DenseNet (ImageNet 1k)
PCL
Coordinate Based
Rotation-EQ
Rotation-EQ-big
1.0
Spatial Transformer
0.5
0.0
random
composition
interpolation
extrapolation
(a) -c-oe dSprites.
1.0
0.5
0.0
random
composition
interpolation
extrapolation
(b) -core dSprites rotation [0, 90).
1.0
Iir
0.5
0.0
random
interpolation (c) R -core Shapes3D test splits.
composition
extrapolation
Figure 11: Rº-score on various splits.
Comparing the new results with the original dSprites results shows: First, for the random test- train split, resolving the rotational ambiguity leads to almost perfect performance (close to 100% R-squared scores for most models). In the previous dSprites setup with rotational ambiguity, top curacies are around 70-95% R-squared scores for most models. Sccond, large drops in performance still be observed when move towards the systematic nut-of-distribution splits (composition, interpolation, and extrapolation). Also, our insights on how models extrapolate remain the same. Lastly, for the random split, the Rotation-EQ model shows non-perfect performance. Tracing this error to individual factors, it turns out this is due to limited capabilities in predicting the x, y posi- tions. We hypothesize that this due limitations of convolutions propagating spatial positions, as discussed in (Liu et al., 2018). The DenseNet performs perfectly on the train set and might be overfitting.
We conclude that the rotational ambiguity explains the drops on the random split. However, the clear drops performance on the systematic splits remain nonetheless. Thus, the analysis we perform in the paper and the conclusions we draw remain the same.
F DATA AUGMENTATIONS
investigate effects data augmentation during training time on the generalization perfor- mance in the extrapolation setting of our proposed benchmark.
As data augmentations, we applied random erasing, Gaussian Noise, small shearings, and blurring- Note that could not arbitrary augmentations. For instance, shift augmentations would lead to ambiguitics with the "shift" factor dSprites. Next, we trained CNNs with and without data augmentations all four datasets (dSprites, Shapes3D, MPI3D, CelebGlow) the extrapolation splits with multiple random seeds.
The results visualized in Fig. 12. For the mean performance, observe significant improve- ment by adding augmentations. However, the overall spread of the scores seems to decrease given augmentations some datasets. We explain this by the fact that the augmentations enforce
21

---

Published as a conference paper at ICLR 2022
dsor 042
shopes.3d
calob: glow
SSSuune Stan
Charn
Augtrortattrs
Ocat
Oloan
Aug Sggnnnaarrra
Ouan
Figure 12: R -core on data augmenations. We depict the performance on the extrapolation setting with and without data augmentations for CNN network with various random seeds considered datasets.
tain invariances, narrowing the solution space of optimal training solutions by providing a further specification (specification in the sense of D' Amour et al. (2020)).
G PERFORMANCE WITH RESPECT TO INDIVIDUAL FACTORS
We here try attribute the performance losses individual OOD factors (see $5.2). Thus, on the extrapolation setting, we modify the test-splits such that only a single factor is out-of-distribution. Next, we measure the overall performance across models (all fully supervised and transfer models) to demonstrate the effect of this factor. The results are depicted for all models in Fig. 13. Overall, factors like "height" on MPI3D that control the viewing of the camera and, subsequently, change attributes like the absolute position in the image of other factors (e.g., the tip of the robot arm) have a high effect.
H
IMPLEMENTATION DETAILS
H.1 DATA SETS
Each dataset consists of multiple factors of variation and every possible combination of factors generates a corresponding image. Here, we list all datasets and their corresponding factor ranges. Note, to estimate the reported Rt--ccore, we normalize the factors by dividing each factor y; by ly -y ie., all factors are in the range [0, 1]. dSprites (Matthey et al., 2017), represents some low resolution binary images of basic shapes with the FoVs shape [0. 1, 21, scale [0, ..., 41, orientation" [0. ... 39), x-position [0. ... 31}, and y-position [0. ... 31}. Next, Shapes31) (Kim & Mnih, 2018) which is similarly popular dataset with 3D shapes in a room scenes defined by the 6 FoVs floor color [0. .... 9). wall color [0. ... 9I. object color [0. .... 9I. object size [0, .... 7]. object type (0, ...., 3] and azimuth (0, .... 14J. Lastly, we consider the challenging and more realistic dataset MP13D (Gondal et al., 2019) containing real images of physical 3D objects attached to a robotic finger generated by FoVs color [0. .... 5]. shape [0. .... 5V. size [0, 1). height [0. 1, 23.
background color [0. I, 2J, x-axis (0, 39 and y-axis [0. .... 39J.
H.2 DATA SET SPLITS
Each dataset is complete in the sense that contains all possible combinations of factors of varia- tion. Thus, the interpolation and extrapolation test-train splits are fully defined by specifying which factors are exclusively in the test set. Starting from all possible combinations, if a given factor value
*Note that this dataset contains a non-injective generative model square and ellipses have multiple rota- tional symmetries.
Jappsti
1122
caleb glaw
Overal squared x
Overal Square
Sing bacior CDU aptratrta
Factor OXU (estrapo
single nodar (ecrrppootoooooo...
bingle hador U2D (catrapctaton
Figure 13: R score in individual factors. Extrapolation performance across models when only single factor (x-axis) OOD.
22

---

Published as a conference paper at ICLR 2022
is defined to be exclusively in the test set, the corresponding image is part of the test set. E.g. for the extrapolation case in dSprites, all images containing X x-positions 24 are part of the test set and the train set its respective complement D' Drest Composition can be defined equivalently to extrapola- tion but with interchanged test and train sets. The details of the splits are provided in table Tables and 3. The resulting train vs. test sample number ratios roughly 30 70. See Table 4. We will release the test and train splits to allow for fair comparison and benchmarking for future work.
For the setting where only a single factor is OOD, we formally define this
Dune-ood [(y*, x*) Dee Eli E N s.t. v + of V(y xt) E Du).
(3)
Here, we used the superscript indices to refer to sample and the subscript to denote the factor. Note that the defined set only nonempty in the interpolation and extrapolation settings.
H.3 TRAINING
All models are implemented using PyTorch 1.7. If not specified otherwise, the hyperparameters correspond to the default library values.
Un-/ weakly supervised For the un-/weakly supervised models, we consider 10 random seeds per hyperparameter setup. As hyperparameters, we optimize one parameter of the learning objective per model similar to Table 2 from Locatello al. (Locatello et al., 2020a). For the Slow VAE, we took the optimal values from Klindt et al. (Klindt et al., 2020) and tuned for E {1, 5, 10. 15, 20, 25). The PCL model itself does not have any hyperparameters (Hyvarinen & Morioka, 2017). For simplicity, we determine the optimal setup in a supervised manner by measuring the DCT-Disentanglement score (Fastwood & Williams, 2018) on the training split. The PCL. and SlowVAF models are trained on pairs of images that only differ sparsely in their underlying factors of variation following Laplace transition distribution, the details correspond to the implementation" $ of Klindt et al. (Klindt et al., 2020). The Ada-GVAE models are trained on pairs of images that differ uniformly in single, randomly selected factor. Other factors are kept fixed. This matches the strongest model from Lo- catello et al. (Locatello et al., 2020a) implemented on GitHub All 3-VAE models trained in an unsupervised manner. All un- and weakly supervised models are trained with the Adam optimizer with a learning rate of 0.0001. We train each model for 500, 000 iterations with batch size of 64, which for the weakly supervised models, corresponds to 64 pairs. Lastly, we train a supervised readout model on top of the latents for 8 epochs with the Adam optimizer on the full correspond- ing training dataset and observe convergence on the training and test datasets no overfitting was observed.
Fully supervised: All fully supervised models are trained with the same training scheme. We use the Adam optimizer with a learning rate of 0.0005. The only exception is DenseNet, which is trained with a learning rate of 0.0001, as we observe divergences on the training loss with the higher learning rate. We train each model with three random seeds for 500, 000 iterations with a batch size of 64. As a loss function, we consider the mean squared error MSE- = Σラ= ly, - (x) per mini-batch.
Transfer learning: The pre-trained models are fine-tuned with the same loss as the fully super- vised models. We train for 50, 000 iterations and with a lower learning rate of 0.0001. We fine-tune all model weights. As an ablation, we also tried only training the last layer while freezing the other weights. In this setting, we consistently observed worse results and, therefore, do not include them in this paper.
H.4 MODEL IMPLEMENTATIONS
Here, shortly describe the implementation details required to reproduce our model implementa- tion. We denote code from Python libraries in grey. If not specified otherwise, the default parameters and nomenclature correspond the PyTorch 1.7 library.
http:://wh.........o.......aataa dataset. py
com
'nttpe://github.ccm http://wwww..ooooooooeoo//e////a htt.://wccccctlaaaoooo.eeeeeeaee httpa/mettn_ttbbbbbbbmbbrerrter htt.:/acttn...leeeetttoss//////a
and https://github. con/ sentangi w dentaang 1eentanggemeet_]]1 ammet_1ib/ methods/veak/weakk
23

---

Published as a conference paper at ICLR 2022
dataset
split
name
exclusive test factors
dSprites
interpolation
shape
{}
1
dSprites
interpolation
scale
[1,4)
dSprites
interpolation
orientation
(32, 27, 12, 17, 22, 27)
3
dSprites
interpolation
x-position
(2, 71, 15, 20, 24, 29]
4
dSprites
interpolation
y-position
11, 20, 24, 29)
dSprites
extrapolation
shape
{}
6
dSprites
extrapolation
scale
(4,5)
dSprites
extrapolation
orientation
(32, 34, 35, 36, 37, 38, 39)
8
dSprites
extrapolation
x-position y-position floor color
(25, 26, 27, 28, 29, 30, 31)
dSprites
extrapolation
(25, 26, 27, 28, 29, 30, 31)
10
Shapes3I)
interpolation
(2,
11
Shapes3D
interpolation
wall color object color
(2.
12
Shapes3D
interpolation
(2
13
Shapes3D
interpolation
object size object type
(2.
14
Shapes3D
interpolation
{}
15
Shapes3D
interpolation
azimuth
(2, 12. 7)
16
Shapes3D
extrapolation floor color wall color object color
(8,
17
Shapes3I)
extrapolation
[8.9]
18
Shapes3D
extrapolation
(8.
19
Shapes3D
extrapolation
object size
(6.
Shapes3I
extrapolation
object type
()
21
S
extrapolation
azimuth
[12, 13, 14)
MP13D
interpolation
color
(3)
23
MP[3D
interpolation
shape
()
MPI3D
interpolation
size
()
25
MPI3D
interpolation
height
(1)
MP13D
interpolation
background color
(1)
27
MPI3D
interpolation
x-axis
34, 15)
MPI3D
interpolation
y-axis
(2, 5, 15)
29
MPI3D
extrapolation
color
(5)
30
MPI3D
extrapolation
shape
{
MPI3D
extrapolation
size
{}
32
MPI3D
extrapolation
height
(2)
33
MPI3D
extrapolation
background color
(2)
34
MPI3D
extrapolation
x-axis
38, 39)
MPI3D
extrapolation
y-axis
(36,37,38)
36
CelebGlow
interpolation
person
{}
CelebGlow
interpolation
smile
(1,
38
CelebGlow
interpolation
blond
(1,4)
39
CelebGlow
interpolation
age
(1,4)
40
CelebGlow
extrapolation
person
()
41
CelebGlow
extrapolation smile
(4,
42
CelebGlow
extrapolation blond
(4,
43
CelebGlow
extrapolation
(4.
Table Interpolation and extrapolation splits.
24

---

Published as a conference paper at ICLR 2022
dataset
split
name
exclusive train factors
0
dSprites
composition
shape
()
dSprites
composition
scale
()
2
dSprites
composition
orientation x- x-position y-position composition
[0, 1,
3
dSprites
composition
{0, 1,
4
dSprites
{0, 1,
5
Shapes3D
composition
floor color
[0]
6
Shapes3D
composition
wall color object color
[0]
7
Shapes3D
composition
[0]
8
Shapes3D
composition
object size
()
9
Shapes3D
composition
object type
[]
10
Shapes3D
composition
azimuth
[0]
11
MP[3D
composition
color
()
12
MPI3D
composition
shape
(
13
MPI3D
composition
size
()
14
MPI3D
composition
height
0
15
MPI3D
composition
background color x-axis
()
16
MPI3D
composition
[0, 5,
17
MPI3D
composition
y-axis
3,,,4,,,,,55
Table 3: Composition splits.
dataset
split
% test
train
Total samples
dSprites
random
32.6
67.4
737280
dSprites
composition
26.1
73.9
737280
dSprites
interpolation
32.6
67.4
737280
3
dSprites
extrapolation
32.6
67.4
737280
Shapes3D
random
30.7
69.3
480000
5
Shapes3D
composition
32.0
68.0
480000
Shapes3D
interpolation
30.7
69.3
480000
Shapes3D
extrapolation
30.7
69.3
480000
8
MPI3D
random
30.0
70.0
1036800
MPI3D
composition
27.8
72.2
1036800
10
MPI3D
interpolation
30.0
70.0
1036800
MPI3D
extrapolation
30.0
70.0
1036800
Table 4: Test train ratio.
25

---

Published as a conference paper at ICLR 2022
The un- and weakly supervised models 3- VAE, Ada-GVAE and SlowVAE all use the same encoder- decoder architecture as Locatello et al. (Locatello et al., 2020a). The PCL model uses the same architecture as the encoder as well and with the same readout structure for the contrastive loss as used by Hyvärinen et al. (Hyvarinen & Morioka, 2017). For the supervised readout MIP, we use the sequential model [Linear (1C, 40), ReLU (), Linear (40, 40), ReLU (40, 40), Linear (40, 40), ReLU (), Linear (40, number-factonn
The MLP model consists of [Linear
90), ReLU (), Lin 104444-mbbbraaaaaee
ear (90, 90), ReLJ (), Linear (90, 90), (eLU(), Linear (90, 90) ReLU () Lin ear (90, 45), ReLU (), Linear (22, number factoraccssss] The architecture is chosen such that it has roughly the same number parameters and layers as the CNN.
The CNN architecture corresponds the one used by Locatello et al. (Locatello et al., 2020a). We only adjust the number of outputs to match the corresponding datasets.
The CoordConv consists of CoordCom:2D layer following the PyTorch implementation' with 16 output channels. It is followed by 5 ReLU-Conv layers with 16 in- and output channels each and a MaxPool2D layer. The final readout consists of [Linear (32, 32), ReLU (), Linear (32,
number-factors:
The SetEncoder concatenates each input pixel with its i pixel coordinates normalized to |o, 1]. All concatenated pixels (i. pixel-value) are subsequently processed with the same network which con- sists [Linear (2 + number channels) ReLU () Linear (40, 40), ReLUO, Lin
ear (40, 20), RetU 0 ]. This is followed by a mean pooling operation per image which guar- antees an invariance over the order of the inputs, i.c. one could shuffle all inputs and the output
would remain the same. As a readout, it follows a sequential fully connected network consisting of
[Linear (20, 20), ReLU (), Linear (20, 20)- ReLU (), Linear (20, number fac cors)
The rotationally equivariant network RotEQ is similar to the architecture from Locatello et al. (Lo- catello et al., 2020a). One difference is that it uses the R2Conv module* from Weiler et al. (Weiler & Cesa, 2019) instead of the PyTorch Conv2d with an 8-fold rotational symmetry. We thus decrease the number of feature maps by factor of 8, which roughly corresponds to the same computational complexity as the CNN. We provide a second version which does not decrease the number of feature maps and, thus, has the same number of trainable parameters as the CNN but higher computational complexity. We refer to this version as RotEQ-big.
To implement the spatial transformer (STN) (Wu al., 2019), we follow the PyTorch tutorial implementation" 2 which consists of two steps. In the first step, we estimate the parameters of (2, 3)-shaped affine matrix using sequential neural network with the following architecture
[Conv2d (number_ (number_channelae channeehannn, 8, sernel_sizer ReLU() Conv2d (8, 10, kernel_size= siz===)) MaxPool2d (2, stride=2), ReLU () ze=6), MaxPoo12d (2, stri de=i), ReLU (), Lin ear (10 33, 31), ReLU () Linear (32, 3+2)] In the second step, the input image is transformed by the estimated affine matrix and subsequently processed by CNN which has the same architecture as the CNN described above.
MaxPool2d stride-20
Conv2d (10, 10, kernel
For the transfer learning models ResNet50 (RN50) and ResNet101 (RN101) pretrained on ImageNet-21k (IN-21k), we use the big-transfer (Kolesnikov et al., 2020) implementation" 10 For the RN50, we download the weights with the tag "BiT-M" MELT-M-R50x11 " and for the RN101, we use the tag
" For the DenseNet trained on ImageNet-Ik (IN-IK), we used the weights from densenet For all transfer learning methods, we replace the last layer of the pre-trained models with a randomly initialized linear layer which matches the number of outputs to the number
httpper//gittuu..cm/////aaaaaaa tutorial.htme
htt.ppp///cahhh.............ooop /ranssrtr // //storagge.ooollepppl http:tsscooo..ttoteett/.e htaar/mmmmilaattttaarraaan
ipynb and for the weights https:
26

---

Published as a conference paper at ICLR 2022
Random
ID factors
OOD factors
1.0
0.5
0.0
BetaVAE
Ada -VAE
SlowVAE
PCL
MLP
CNN
CoardConv
Coontinate Baard
Rotation-EQ
Rotation EQ-big
STN
RN50
RN101
RNSO (1magsNet-214
DenseNet
RN101 (ImagaNee-22l)
DenseNet (ImageNat-l 1k)
(a) MPI3D-Real interpolation
1.0
0.5
0.0
(b) CelebGlow interpolation
1.0
0.5
0.0
(c) Shapes3D interpolation
1.0
0.5
0.0
(d) dSprites interpolation
1.0
0.5
0.0
(e) dSprites extrapolation
1.0
0.5
0.0
(f) Shapes3D extrapolation
Figure 14: Interpolation Extrapolation and modularity.
of factors in each dataset. As ablation, we also provide randomly initialized version for each transfer learning model.
H.S COMPUTE
All models are run on the NVIDIA T4 Tensor Core GPUs on the AWS g4dn.4xlarge instances with an approximate total compute of 000 GPUh. To save computational cost, we gradually increased the number of seeds until we achieved acceptable p-values of < 0.05. In the end, we have 3 random seeds per supervised model and 10 random seeds per hyperparameter setting for the un and weakly supervised models.
I ADDITIONAL RESULTS
27

---

Published as a conference paper at ICLR 2022
modification
random
composition
interpolation
extrapolation
models
BetaVAF
78.2+ 2.1
0.14 6.5
64.0 5.1
18.3# 12.3
Ada-GVAR
86.6#
-1.4+ 5.5
73.4
51.24
SlowVAE
3..72
2.3
82.8# 1.4
66.7+ 1.7
PCL
87.0#
2.51
86.42
63.0 2.5
MLP
81.1
-10.6# 1.3
53.3#
2,.7
CNN
0.9
33.1
8..1
57.72
CoordConv
9.67
5.1
7..1
5..3#
Coordinate Based
73.3+ 0.3
2.84
49.3 1.3
8.8
Rotation-FQ
53.6# 0.1
7.1.94
28.3+ 1.0
-23.8
Rotation- Ei-io
33.7 1.3
-8.6 1.6
32.72 0.5
-25.9 1.2
Spatial Transformer
89.6 2.4
9.6
84.9 1.3
6.08
RN50
9.1
1.45
0.1
5..923.9
RN101
92.6* 0.1
5..8
81.7
5..1
DenseNet
92.24 0.2
6..4+
84.3
6.74
RN50 (ImageNet-21kk)
93.6 0.2
49.7 2.9
8..
62.0+ 0.8
RN101 (ImageNet-21k)
9.5
43.0# 4.8
83.5#
58.3+ 1.6
DenseNet (ImageNet-Ik Ik)
6..5+
60.8 5.7
57.34 17.7
38.4 19.8
Table 5: -s-score dSprites
modification
random
composition
interpolation
extrapolation
models
BetaVAE
94.3+- 0.3
4.3+ 1.5
94.8+- 0.5
29.5+ 8.4
Ada-GVA
99.8+- 0.0
9.3+-4.3
92.9+ 3.8
74.4+ 0.6
SlowVAB
99.8+ 0.0
58.9+-3.7
99.5+
77.1+ 2.7
PCL
99.7+ 0.0
45.3+-
99.6+-0.0
77.5+ 1.1
MLP
98.1+ 0.1
-8.8+0.2
76.7+-0.5
43.5+
CNN
100.0+-00 0.0
53.3+ 10.4
99.7+-0.0
0.5
CoordConv
100.0+-00 0.0
09.1+-0.3
99.0+-0.6
7722+
Coordinate Based
83.6+ 1.5
12.1+-377
1.
-11,5+-1566
Rotation-FO
53.0+ 0.3
14.6+-
71.3+
-6.9+
Rotation-EQ-biig
43.9+ 0.4
26.0+-
71.5+-0.0
-1.
Spatial Transformer
100.0+-00 0.0
39.0+-4.0
99.4+-
02.5+133.1
RN50
100.0+0..0
81.0+-
98.7+ 0.4
68.6+
RN101
100.0+-000
74.0+-
98.0+ 0.4
69.9+
DenseNet
100.0+- 0.0
0.5
99.6+-0.2
73.8+-
RN50 (ImageNet-21k)
100.0+- 0.0
55.8+
98.3+-0.3
68.2+-3.1
RN101 (ImageNet-21kk)
100.0 0.0
82.0+ 10.7
99.6+ 0.1
77.9+ 0.4
DenseNet (ImageNet-Ik)
87.0+ 18.2
97.6+ 0.5
99.8+- 0.1
-42.5 504.1
Table 6: RR-score dSprites rotation (0,90)
28

---

Published as a conference paper at ICLR 2022
modification
random
composition
interpolation
extrapolation
models
BAI
99.9+ 0.1
99.6* 0.2
90.8+ 8.0
34.4 13.5
Ada-GVAR
99.9 0.0
99.44 0.4
91.1 9.2
37.6# 21.6
SAE
99.8* 0.1
97.2= 1.5
5.74
32.8 7.2
PCL
99.9 0.0
93.6* 1.6
98.5+
29.84 29.1
MLP
100.0 0.0
99,8 0.2
98.5+
40.3 9,9
CNN
100.0 0.0
100.0 0.0
9.13#
48.9 23.2
CoordConv
100.0 0.0
99.3* 1.2
96.7+
46.2+ 4.3
Coordinate Based
100.0= 0.0
64.0= 3.7
9..6#
24.74 7.0
Rotation-FQ
100.0 0.0
98.5+ 2.2
9.79
57.2+ 11.7
Cetation-EQ-bip
100.0 0.0
100.0 0.0
98.8
52.74 1.5
Spatial Transformer
100.0* 0.0
100.0+
9.1
52.7+ 13.9
RN50
100.04 0.0
100.0
98.8+
62.8 3.7
RN101
100.0# 0.0
100.0#
99.1
67.8 1.7
DenseNet
100.0 0.0
99.3 1.2
9..
48.5 3.0
RN5O (ImageNet-2Ikk)
100.0# 0.0
100.0 0.0
9.13+
46.1 14.8
RN101 (ImageNet-21k)
100.0# 0.0
100.0 0.0
99.4#
34.8 8.3
DenseNet (ImageNet-Ikg
97.1 3.8
100.0# 0.0
86.84 17.7
53.2+ 22.3
Table 7: -sooe Shapes3D
modification
random
composition
interpolation
extrapolation
models
BetaVAE
68.4+ 0.6
43.7+ 3.1
66.2+- 0.1
43.5+ 0.3
Ada da-GVVA
69.8+ 0.4
48.6+ 2.7
68.4+ 0.6
44.5+
Slow
70.3+ 0.6
53.9+
69.5+-00
43.7+ 1.0
PCL
46.0+-0.1
-7.0+-
19.7+-0.1
2.8.4+-2.6
MLP
93.3+ 1.7
75.3+-
82.0+-066
50.0+
CNN
9..4+-0.0
75.9+
1.
57.9+
CoordConv
81.5++3.0
57.7+
74.4+
33.5+
Coordinate Based
74.8+-
58.0+-3.9
6..8+-0.5
22.7+-1.8
Rotation-EO
85.7+
59.9+-29
82,0+-2.1
44.1+-5.1
Rotation-EQ-big
9..
81.7+
96.1+-.11
6,.4+-0.3
Spatial Transformer
9..8+-0.1
76.8+-0.1
88.0+-099
57.7+-0.5
RN50
0.3
79.5+
89.3+-02
48,6+
RN101
99.3+-0.1
81.5+-200
88.6+-0.1
49.1+-0.7
DenseNet
9..
83.9+-066
89.1+-0.7
49.8+-2.1
RN50 (ImageNet-21k)
97.7+-1.3
76.9+-0.5
86,8+-0.7
50.1+-0.8
RN101 (ImageNet-21kk)
98.3+ 0.0
78.9+
0..8+00.7
49.3+ 1.0
DenseNet (ImageNet-Ik)
96.8+ 3.1
81.7+ 2.0
90.1+ 1.3
59.6+ 1.3
Table 8: sco-e CelebGlow
29

---

Published as a conference paper at ICLR 2022
modification
random
composition
interpolation
extrapolation
models
Beta VAB
79.4+ 1.1
-6.2+
10.9 8.9
-9.9 6.3
Ada-GVAE
64.5+ 0.8
3.0
9.54
-3.62 9.2
SlowVAE
89.0# 1.9
-16.62 11.3
-10.9 8.5
-31.54 15.2
PCL
95.8 0.7
10.7# 10.2
7.5
4.1 10.3
MLP
97.0# 0.5
3.5+4.0
7.5
-37.5# 12.1
CNN
99.8+
3..7
11.3
18.3 10.0
CoordConv
98.6+
27.7* 19.3
18.5# 19.0
15.3227.5
Coordinate Based
93.5+
5.3
-50.5 48.0
421.1 286.5
Rotation-FQ
0.6
23.6* 0.8
12.3# 12.1
44.3+ 15.3
Rotation-EQ-Bee
99.9
45.9 1.0
45,8 3.6
1.
Spatial Transformer
99.8+ 0.0
16.1
31.5# 12.2
9.0 8.8
RN50
100.0 0.0
1.5
29.4 5.8
22.0#
RN101
100.0 0.0
26.1+4.6
23.3+ 15.7
20.7 4.4
DenseNet
100.0# 0.0
1.0
54.6 3.3
7.0t 11.2
RN50 (ImageNet-21k)
99.8+ 0.0
23.8+
43.6 6.5
54.1 1.9
RN101 (ImageNet-21kk)
99.8# 0.1
37.0# 3.4
35.4 15.4
41.6# 8.5
DenseNet (ImageNet-Ikg
44.0# 79.0
49.0# 0.7
72.24 3.4
38.9# 1.9
Table 9: R² -core MPI3D
modification
random
composition
interpolation
extrapolation
models
Beta Bet
79.9+ 0.4
7.1+ 6.1
6.9+ 2.7
1.1+ 11.4
Ada-GVAE
57.6+ 1.2
1.2+ 4.0
-22.4+- 2.5
5.0+-
BlowVAD
71.8+ 1.3
-5.6+- 1.7
5.9+-
-5.0+-
PCL
97.4+
4.3+- 1.8
12.7+-2.1
-19.3+
MLP
90.5+
3.0
-33.2+
31.1+ 4.4
CNN
99.5+-0.0
23.0+-0.7
18,0+-700
-2..0+
CoordConv
1.84+1188
41.2+-4.4
30.9+-3..4
28.8
Coordinate Based
91.7+-
9.6+ 0.1
-113.1+-45.4
-76.7+225. 25.2
Rotation-FO
94.8+-
22.6+-4.9
15.5+-6.6
938.1+
Rotation-EQ-big
99.9+ 0.0
47,6+ 1.4
45.7+-4.8
4.5+- 13.2
Spatial Transformer
99.6+-
23,6+-6.4
37.7+ 18.4
-.6
RN50
99.9+ 0.1
23,4+ 1.5
40.3+ 1.3
14.9+ 10.2
RN101
99.9+ 0.1
19.7+-
37.8+
9.8+
DenseNet
99.9+ 0.0
33.2+-0.5
51.4+-3.0
4.0+-6.5
RN50 (lmageNet-21k
99.6+
27.7+-
48.7+-1.2
11.2+
RN101 (ImageNet-21k)
99.8+ 0.1
39.2+-
39.2+ 7.6
5.6+ 10.5
DenseNet (ImageNet-Ik)
62.9+ 51.9
41.8+- 2.6
73.7+
34.3+- 0.5
Table 10: -core MPI3D-Toy
30

---

Published as a conference paper at ICLR 2022
floor color
wall color
object color
object size
object type
azimuth
prediction
ground-truth
(a) Scatter scale OOD
prediction
ground-trut
(b) Wall color OOD
prediction
gound-truth
(c) Object color
prediction
gound-truth
(d) Scale
prediction
ground-truth
+++ Groundtruth
factors
factors
(e) Orientation
Figure 15: Shapes3D extrapolation. We show the qualitative extrapolation of a CNN model. category is excluded because no order is clear.
The shape
31

---

Published as a conference paper at ICLR 2022
color
shape
size
height
bg color
X-axis
y-axis
prediction
ground-truth
(a) Color
prediction
ground-truth
(b) Camera height
prediction
ground-truth
(c) Background color
prediction
ground-truth
(d) X-axis
prediction
ground-truth
+++ Groundtruth
factors
(e) Y-axis
Figure 16: MPI3D-Real extrapolation. We show the qualitative extrapolation of a CNN model. The shape category is excluded because no order is clear. Size is excluded hecause only two values are available.
32

---

Published as a conference paper at ICLR 2022
00050 (1--200 (M-20-)
(4-214)
Denseltet (M6.MM)
10450 (17-224 201)
10110 (0K 211)
De Dersolte (1N-3K)
Gound
Swift Saffrcadee
Denellee
CouneCoone
Deseelive
MLP
PaxF.O
204 101
MLP
CNN
RuiFQ
STN
MUP
6.0 8as 000 aa
MLP
CNN
CNN
CoundComy
ConrdConr
SetEnooder
SctEnooder
RodEQ
4.33
RocEQ
RatFQhig
313
521
6.25
RetFQuhg fig
STM
R00 5m 5.44
035 037
5.1
0.00
005
STN
035
0.00
RNO0
3:0
FON50
Denscheet
076 322
Donsclet
RX50 (IN-21k) 555 sas EM
6:77 6.30
RASI (84-214) a:l
670
002
072
02)
0:0
RX00 (N4-11))
RN101 (N4-214)
DaneNiet (IN-1k)
DonsNNtt (1S-14)
1000
(a) dSprites
(b) Shapes3D
Donaclle Hhrashh (s))AA))
Densthes (16a3)
ve si (trogphhae.cc 201g
loly Mhaaghee 200)
aodio (1N 2010
soaeO 00 2012
Gordratt Bood
Sh on Eo kgg
Spocil Toeellrn
ConesCore
RoxE--iig
Devalllet
CoodCoo
E3
Dunofles
MLP
CNN
RoFQ
STN
RNUCI
ON
10.50
anh 101
MLP
684 023 0.7 8.79 8.88 802 008 As 4.4
MLP Ju000e AI1-A4
Aut 032
CNN
CVK
028
Cooeffonn
CoordConno
075.008.00
4.244.4
Coodratt Beucz
0.70
RocEQ
371
2. 0.00 020
EG
RotEQ big
1
035 013099994
STN
0.70
Spatil Transfomer
RN1M 000 20.000000000000
B850 017
DessoNet 0.7 0.22 2.5
RN100
2.22
833
0.35
8.77 5.0
8:00
15
Desuchit
1M10 (Iragetee311)) aow 013 oET 808 0.3
611 oz
RNSC (N 214) Ruo 306
8.79
682
008
0. am
0.00
855
033
0.20
-0.30
RN120 (raageNee2234
RN10L (N 214)
DenieNict (ImaacNos-3))
636
6:3 ous 001--00
-175
Demafiet (1N-14)
-0.75
6440/04
8:0 3/54
1.00
(c) CelebGlow
(d) MPI3D-Real
Figure 17: Model similarity on extrapolation errors.
33

---

Published as a conference paper at ICLR 2022
person
smile
blond
age
prediction
grunnd-tuut
(a) Scatter scale OOD
prediction
ground-truth
(b) Wall color OOD
prediction
ground-truth
+++ Groundtruth
ID factors
OOD factors
(c) Object color
Figure 18: CelebGlow extrapolation. We show the qualitative extrapolation of DenseNet (ImageNet 1-K) model. This corresponds, to the model with the highest correlation with the ground truth Fig. on extrapolated factors (OOd factors). The person category is not extrapolated and used to measure correlations hecause no order is apparent.
34


Reviews:
[{"review_id": "ykt8u2x5Ok", "reviewer": "Reviewer_Yd4H", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a very well-written and clear paper with sufficient implementation details for reproducibility.\nGeneralization in deep learning models remains an interesting question to study. As this study demonstrates, the problem is far from solved. \nThe paper surveys a wide range of 17 approaches on four datasets with increasing complexity of visual stimuli. However, I have concerns with the following issues, which I believe, limit the contribution of this paper.\n\n1. At several places noted by the authors, also from Fig 4 and 5 (CelebGlow is better than MPI3D), there seems to exist variances in the generalization ability of models across different datasets or even within individual factors within each dataset. In addition to reporting the overall empirical results in entire datasets, can the authors provide explanations about why models perform better in certain factors/datasets and not the rest? And why would different variation factors within the same dataset result in different generalization ability?\n\n2. In some cases, it is fair to say that deep models fail at some generalization tests. However, have the authors considered designing human experiments and quantitatively verifying human generalization abilities in similar problem settings? Human performance can serve as an essential benchmark to compare with the generalization ability of computational models. This is not a requirement (and certainly not the focus) of the current study. But it is hard to come up with an upper bound or even some sense of what level of generalization one should aspire to achieve.\n\n3. The conclusions drawn from the paper are interesting and informative. However, there is a lack of explanations or insights about why these models fail to generalize in certain conditions. And if so, what are potential solutions to improve generalization ability.\n\n4. The authors have focused on deep learning-based approaches. It is unclear that models without learning via backprop would be able to generalize. In other words, would other learning algorithms using Hebbian rules have better generalization ability (e.g. Boltzmann machines)?\n\n5. The variation factors include translation, rotation, scaling, color, age, and so on. Have authors considered other dimensions? These factors are predefined and manually picked. The models might generalize better in some of these factors compared with the rest of factors. Have the authors considered ways to automatically exploit other factors that the models would show impairment in generalization abilities?\n", "summary_of_the_paper": "The paper tests 17 unsupervised, weakly supervised, and fully supervised representation learning approaches to infer the generative factors of variation across three simple datasets in well-controlled conditions. In addition, the authors introduce a CelebGlow dataset, which is more complex. The generalization abilities are characterized as composition, interpolation, and extrapolation. The conclusions from these empirical observations on the experimental results are interesting and suggest that most networks fail to generalize.\n\n", "summary_of_the_review": "The study poignantly points out failures of current models in generalization through a large number of empirical studies. The work is well written and clear. It is not particularly novel to argue that models fail to generalize, but the current study provides quantitative and extensive benchmarks in well-controlled settings. Beyond pointing to limitations of current models, the study does not offer new paths towards new algorithms or solutions to those problems. However, sometimes, understanding what the challenges are is a good step towards generating momentum to find better solutions. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "wHDXIFrLwvE", "reviewer": "Reviewer_ZKCQ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I think this is a very good paper. The writing is exceptionally clear and the problem framing and literature review will be an especially valuable resource for future work. The experiments are well designed and thorough, and the findings are valuable contributions to an important area.\n\nThe main weakness I see is that I'm not sure there is a lot of information gain from this paper. The results are more or less what I think most readers will expect. The findings seem in line with past work on generalization and with conventional wisdom -- ID works, OOD works worse, inductive biases can help on OOD, etc. That's not to say the present work is not valuable -- there are open questions here and the present paper is one of the most extensive studies I have seen on them. Just that the paper doesn't provide entirely unexpected answers. Because of this, I think the most valuable contribution of the paper may be the benchmark it provides, on top of which future studies may find something really new.\n\nTo elaborate further, the related work covers numerous papers that have come to roughly similar conclusions. Two more papers come to mind that also have similar conclusion but were not discussed:\n* Packer et al., “Assessing Generalization in Deep Reinforcement Learning”, 2019\n-- This paper also studied ID vs OOD generalization but in the context of reinforcement learning. Figure 1 from Packer et al. shows their train/test setup, which is quite related to the current paper's settings in Figure 2. The conclusions are similar to the current paper: 1) extrapolation is harder than interpolation, 2) SOTA algorithms that are supposed to \"solve\" this problem fail.\n* Jahanian et al., \"On the 'Steerability' of Generative Adversarial Networks\", 2020\n-- This paper studied the ability of GANs to extrapolate in their latent space. The finding is that they struggle to generate transformations that extend beyond the distribution seen during training. These results are similar to the findings in the current paper on the failure of VAE latent representations to extrapolate beyond the training data.\n\nThis is all to say I think there is ample prior literature that make the present conclusions unsurprising. But thorough work on this topic, and new benchmarks, is still valuable and that's what the current paper provides. I should also note that the finding about modularity is something I hadn't seen before, and I think that's a valuable contribution as well.\n\nAside from this, I think the paper is very solid. A few minor comments follow:\n1. Using CelebGlow feels a bit awkward since it is a generative model fit to data, and then you are again fitting samples from this model with another generative model. I wonder if there could be some bias where the samples are easier to model with a VAE since they were generated with a related model (Glow)... It's probably all fine but some commentary on this could be useful.\n2. Repeated reference to Hendrycks and Diettrich 2019.\n3. I would say inductive biases 1 and 3 overlap: inductive bias 1, as it is implemented in the paper, could be considered a special case of transfer learning where the pretraining is done with VAEs. This could be clarified to avoid implying that these are independent inductive biases.\n4. “if factors are located in a particular edge of the FoV hyper cube given by all FoVs” — “edge” —> “corner”?\n5. “Here, we further see that, on average, the performances seem to increase as we increase the supervision signal.” — This is a bit vague I don’t see it fully reflected in the figure. This point could be made more precise. What does “increase the supervision signal” refer to?\n6. “We find that the degree of downstream performance correlates weakly but positively with the degree of disentanglement (Pearson ρ = 0.63, Spearman ρ = 0.67)” — I’m not sure I would call these weak correlations. In many fields I believe this would be considered a strong correlation. \n7. “Existing notions of disentanglement models with a readout MLP do not help to facilitate the learning of the underlying mechanisms in the tested datasets.” — I don’t understand this conclusion. The correlations are substantial. My understanding of the results is that greater disentanglement _does_ correlate with better ability to identify the underlying mechanisms. Appendix Fig 7 seems to support this as well, with all but one of the correlations being positive.\n", "summary_of_the_paper": "This paper presents an empirical study of generalization in visual representation learning. The paper compares in-distribution (ID) generalization to out-of-distribution (OOD) generalization of three types -- interpolation, extrapolation, and composition. Datasets are constructed with several factors of variation where the training split exhibits some factors and the test split exhibits others, constructed to test the ID and OOD settings. A variety of different representation learning models with different kinds of inductive bias are tested. The main findings are that: 1) all models perform better ID than OOD, indicating that all methods fail to find the true underlying generative mechanisms that created the data, 2) when some factors are ID and others are OOD the ID factors are modeled well despite that the OOD factors may be modeled poorly, demonstrating a kind of modularity between the learning of different factors. The paper will be accompanied by a benchmark for others to improve on this task.", "summary_of_the_review": "This is a solid paper that contributes new empirical findings and a new benchmark on an important topic. I believe this work will stimulate future work on representation learning, disentanglement, and OOD generalization. I don't see any major errors and recommend accepting.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "p-vs77z-rA", "reviewer": "Reviewer_Cjpj", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Positives:\n- The paper is extremely well motivated and addresses a very important problem in the field of generalization in NNs\n- The dataset also looks very promising\n- The experiments are extremely well done and thorough\n- The idea of having \"conclusion\" section in each subsection in the experiments is a very nice idea.\n\nNegatives:\n- I would prefer if the figures had appropriate captions - all tables and figure captions should be self-sufficient. There should be sufficient details in them such that I am able to understand what the paper is doing.\n- I am slightly concerned about what \"new\" information this adds to the field - a lot of the conclusions in this paper were well known in the community, to my knowledge. This does not discredit the work at all, since it is always good to have experimental support related to \"intuitions\" that people have had, but I would be keen on the authors to answer what \"previously unconfirmed\" things they really discovered in this paper.\n- the paper states \"Instead of extrapolating, all models regress the OOD factor towards the mean in the training set.\" which should be clarified as I am not sure what that means. The ratio of distances idea makes sense, but I am not sure what the authors are trying to conclude with that statement.\n- Is there a reason why CelebGlow outperforms MPI3d and shapes3d considerably in Figure 4 and 5 on the \"extrapolation\" benchmark?\n\nLimitation:\n- The paper adaquately points out where and how the NNs fail to adapt, without giving a clear direction of what are possible ways to remediate the current gap. It would be a useful discussion point in the paper. ", "summary_of_the_paper": "The paper introduces a new dataset: CelebGLOW which is a controllable environment generation dataset, which can be used in the same form as significantly less complex datasets, such as sprites. The paper evaluates over 3 key inductive biases using the dataset: \"representational format\" (using images in this paper), neural network architectural variants (MLPs/CNNs/ transformers, etc), and ability to perform transfer learning. The paper evalutes the models on 4 given modes of generalization: interpolation, extrapolation, compositional learning and \"random\", and finds extremely fascinating conclusions through extensive experimentation. ", "summary_of_the_review": "It is a well-written paper with great analysis and a very useful study to have. I have small concerns about the novelty of the conclusions, but the paper is still a great addition to the community. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "imo4d-E44hM", "reviewer": "Reviewer_g7qs", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The direction of this work is nice and the problem tackled by this paper is indeed important. The obtained results are nice and there is indeed some potential value in this work. However, I have a few concerns with the paper:\n\ni) To test generalizability of neural networks, test datasets are constructed using four methods: interpolation, extrapolation, random, and composition. However, previous works uses these approaches as data augmentation to improve the generalizability of neural networks.  For example, multi-scale inputs during training in YOLOv2 allows it to predict well across different input dimensions. Another example is random erasing, which also improves generalization ability of neural networks. A discussion about these methods to improve generalization ability is missing. Also, what happens to the generalizability of neural networks when they are trained with these previous approaches?\n\nYOLOv2: https://arxiv.org/pdf/1612.08242.pdf\n\nRandom erasing: https://arxiv.org/pdf/1708.04896.pdf\n\nCutMix: https://arxiv.org/abs/1905.04899\n\nRandom Augmentation: https://arxiv.org/abs/1909.13719\n\nii) The generalization performance of neural networks drops when we move from artificial datasets to real-world datasets. However, the datasets studied in this paper are simple and limited, and far from real-world datasets. They do not consider many factors that are natural in real-world datasets, including non-rigid deformations and intraclass variability (e.g., people wear different types of glasses).\n\niii) Previous works have shown that larger models generalize better than smaller models. I do not see any argument about the model capacity and generalizability in the paper. Can authors comment on that?\n\nhttps://arxiv.org/pdf/1802.08760.pdf\n\n=========\n\nClarification: \n\nIn page 4, it is stated that \"Combined with a set operation such as pooling, CNNs then achieve translation invariance.\" To my understanding, translation invariance is one of the fundamental properties of convolutions. I think operations like pooling forces CNNs to learn multi-scale representations\n\nhttps://www.ijcai.org/Proceedings/83-2/Papers/091.pdf\n", "summary_of_the_paper": "This paper studies the generalization ability of neural networks in out-of-distribution (OOD) settings on simple datasets (such as Shapes3D) and shows that generalization performance drops when real-world datasets are used in comparison to artificial datasets.", "summary_of_the_review": "This paper studies generalizability of neural networks using four factors (interpolation, extrapolation, random, and composition). Previous works on better data augmentation policies uses similar factors to improve the generalization of visual recognition networks. So, the results of this paper are not surprising.\n\n\n=================\nPost rebuttal: I am convinced with author's response and I am leaning towards acceptance. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a through study of generalization in visual representation learning. It compares in distribution generalization to out of distribution generalization using a comprehensive benchmark. The paper received very positive reviews from all reviewers. Reviewers agreed that the paper has several strengths: It is very well written, the presented benchmark is very useful and the analysis is thorough. One concern that was brought up by the reviewers was that a majority of the presented findings are expected and in a sense, known to the community. The authors have addressed this concern by pointing out that their findings are more fine grained than past works and that their proposed benchmark is a stepping stone towards measuring general robustness. I must note that in spite of this concern, all reviewers have maintained their strong acceptance scores. I agree with the reviewers. This paper makes a strong contribution to this important problem via its benchmark and analysis, which future works can build off of, and hence I recommend acceptance.
Decision: Accept (Poster)