The role of Disentanglement in Generalisation
Authors: Milton Llera Montero
Casimir JH Ludwig
Rui Ponte Costa
Gaurav Malhotra
Jeffrey Bowers
Venue: ICLR 2021 Poster (2021)
Keywords: disentanglement
compositionality
compositional generalization
generalisation
generative models
variational autoencoders

Abstract: Combinatorial generalisation — the ability to understand and produce novel combinations of familiar elements — is a core capacity of human intelligence that current AI systems struggle with. Recently, it has been suggested that learning disentangled representations may help address this problem. It is claimed that such representations should be able to capture the compositional structure of the world which can then be combined to support combinatorial generalisation. In this study, we systematically tested how the degree of disentanglement affects various forms of generalisation, including two forms of combinatorial generalisation that varied in difficulty. We trained three classes of variational autoencoders (VAEs) on two datasets on an unsupervised task by excluding combinations of generative factors during training. At test time we ask the models to reconstruct the missing combinations in order to measure generalisation performance. Irrespective of the degree of disentanglement, we found that the models supported only weak combinatorial generalisation. We obtained the same outcome when we directly input perfectly disentangled representations as the latents, and when we tested a model on a more complex task that explicitly required independent generative factors to be controlled. While learning disentangled representations does improve interpretability and sample efficiency in some downstream tasks, our results suggest that they are not sufficient for supporting more difficult forms of generalisation.

Published as a conference paper at ICLR 2021
THE ROLE OF DiSENTANGLEMENT IN GENERALISATION
Milton L. Montero'?, 1.2 Casimir J.H. Ludwig 1, Rui Ponte Costa?, Gauray Malhotra' & JJeffre S. Bowers'
1. School of Psychological Science
2. Computational Neuroscience Unit, Department of Computer Science
University of Bristol Bristol, United Kingdom
{m. Terammmmttotr c. Luw..,,,.....ttt..aurraaaaannnn www.aaaaaaaaaaaaaaaaarraaa wtt.://lloooot.....ooe/////ooo.o
rui cwsta, gatrav,
ABSTRACT
Combinatorial generalisation - the ability to understand and produce novel com- binations of familiar elements is core capacity of human intelligence that current Al systems struggle with. Recently, it has been suggested that learning disentangled representations may help address this problem. It is claimed that such representations should be able to capture the compositional structure of the world which can then be combined to support combinatorial generalisation. In this study, we systematically tested how the degree of disentanglemen affects various forms of generalisation, including two forms of combinatorial generalisation that varied in difficulty. We trained three classes of variational autoencoders (VAEs) on two datasets on an unsupervised task by excluding combinations generative factors during training. At test time we ask the models to reconstruct the missing combinations in order measure generalisation performance. Irrespective of the degree of disentanglement, we found that the models supported only weak com- binatorial generalisation. We obtained the same outcome when we directly input perfectly disentangled representations as the latents, and when we tested model on a more complex task that explicitly required independent generative factors to be controlled. While learning disentangled representations does improve inter- pretability and sample efficiency in some downstream tasks, our results suggest that they are not sufficient for supporting more difficult forms of generalisation.
INTRODUCTION
Generalisation to unseen data has been key challenge for neural networks since the early days of connectionism, with considerable debate about whether these models can emulate the kinds of be- haviours that are present in humans (McClelland et al., 1986 Fodor & Pylyshyn 1988 Smolensky 1987 1988 Fodor McLaughlin 1990j. While the modern successes of Deep Learning do indeed point to impressive gains in this regard, human level generalisation still remains elusive (Lake & Ba- roni 2018) Marcus 2018). One explanation for this is that humans encode stimuli in a compositional manner, with a small set of independent and more primitive features (e.g., separate representations of size, position, line orientation, etc.) being used to build more complex representation (e.g., a square of a given size and position). The meaning of the more complex representation comes from the meaning of it's parts. Critically, compositional representations afford the ability to recombine primitives in novel ways: if person has learnt recognize squares and circles in context where all squares are blue and all circles are red, they can nevertheless also recognise red squares, even though they have never seen these in the training data. This ability to perform combinatorial generalisa- tion based on compositional representations is thought to be hallmark of human level intelligence (Fodor& & Pylyshyn 1988) (See McClelland et al. (1986) for diverging opinion).
Recently it has been proposed that generalisation in neural networks can be improved by extracting disentangled representations (Higgins et al. 2017) from data using (variational) generative models (Kingma& & Welling 2013)] Rezende et al. 2014). In this view, disentangled representations capture the compositional structure of the world (Higgins al.] 2018a; Duan al.) (020),, separating the generative factors present in the stimuli into separate components of the internal representation (Higgins et al. 2017, Burgess et al.] 2018). It has been argued that these representations allow

---

Published as a conference paper at ICLR 2021
downstream models to perform better due to the structured nature of the representations (Higgins cet al] 2017: 2018b) and to share information across related tasks (Bengio et al.) 2014). Here we interested in the question of whether networks can support combinatorial generalisation and extrapolation by exploiting these disentangled representations.
In this study systematically tested whether and how disentangled representations support three forms of generalisation: two forms of combinatorial generalisation that varied in difficulty as well as extrapolation, detailed below. We explored this issue by assessing how well models could render images when we varied (1) the image datasets (dSprites and 3DShape), (2) the models used to reconstruct these images (3- VAEs and FactorVAF VAES with different disentanglement pressures, and decoder models in which we dropped the encoders and directly input perfectly disentangled latents), and (3) the tasks that varied in their combinatorial requirements (image reconstruction vs. image transformation). Across all conditions we found that models only supported the simplest versions of combinatorial generalisation and the degree of disentanglement had no impact on the degree of generalisation. These findings suggest that models with entangled and disentangled representations are both generalising on the basis of overall similarity of the trained and test images (interpolation), and that combinatorial generalisation requires more than learning disentangled representations.
1.1 PREVIOUS WORK
Recent work on learning disentangled representations in unsupervised generative models has indeed shown some promise in improving the performance of downstream tasks (Higgins al., [018b] van Steenkiste et al., 2019) but this benefit is mainly related to sample efficiency rather than general- isation. Indeed, we are only aware of two studies that have considered the importance of learned disentanglement for combinatorial generalisation and they have used different network architectures and have reached opposite conclusions. Boowers et al. (2016) showed that a recurrent model of short- term memory tested on lists of words that required some degree of combinatorial generalisation (re- calling a sequence of words when one or more of words at test were novel) only succeeded when it had learned highly selective (disentangled) representations ("grandmother cell" units for letters). By contrast, Chaabouni et al.] (2020) found that models with disentangled representations do not confer significant improvements in generalisation over entangled ones language modeling set- ting, with both entangled and disentangled representations supporting combinatorial generalisation as long as the training set was rich enough. At the same time, they found that languages generated through compositional representations were easier to learn, suggesting this as a pressure to learn disentangled representations.
A number of recent papers have reported that VAEs can support some degree of combinatorial gen- eralisation, but there is no clear understanding of whether and how disentangled representations played any role in supporting this performance. [Esmaeili et al.] (219)) showed that a model trained on the MNIST dataset could reconstruct images even when some particular combination of factors were removed during training, such as a thick number 7 or narrow 0. The authors also showed that the model had learned disentangled representations and concluded that the disentangled representa- tions played role in the successful performance. However, the authors did not vary the degree of disentanglement in their models and, accordingly, it is possible that a VAE that learned entangled Sinilarly.Higggins representations would do just well. Similarly, [Higgins et al.) (2018c &c) have highlighted how VAEs
that learn disentangled representations can support some forms of combinatorial generalisation when generating images from text. For example, their model could render room with white walls, pink floor and blue ceiling even though it was never shown that combination in the training set. This an impressive form of combinatorial generalisation but, as we show below, truly compositional representations should be able to support several other forms of combinatorial generalisations that were not tested in this study. Moreover, it is not clear what role disentanglement played this suc- cessful instance of generalisation. Finally, /Zha et al. (2018) assessed VAE performance on range of combinatorial generalisation tasks that varied in difficulty, and found that the model performed well in the simplest settings but struggled in more difficult ones. But again, they did not consider whether learning disentangled representations was relevant to generalisation performance.
Another work that has significant relation to ours is Locatello et al. (2019), who examine how hard it is to learn disentangled representations and their relation to sampling efficiency for downstream tasks. We are interested in a related, but different question: even if model learns disentangled representation in an intermediate layer, does this enable models to achieve combinatorial generali-
2

---

Published as conference paper at ICLR 2021
Recombination to Range
Rotation
training
test
recombination to range element
m
Shape
Wu Un Uch UW
extrap
Translation
Extrapolation
Recombination to Element
Figure 1: Testing generalisation in image reconstruction. (a) An illustration of different tests of combinatorial generalisation for the three-dimensional case (i.e., three generative factors). The blank cells represent combinations that the model is trained Coloured cells represent novel test combinations that probe different forms of generalisation: Recombination-to--BGee Recombination-to--Ral
(red), (green) and Extrapolation (blue) see main text for details. (b) Fach row shows an example of training and test stimuli for testing a form generalisation. training set excludes ellipses the bottom-right corner at less than 120 though they are present at the bottom-right corner at other rotations. In the middle row, training set excludes squares in the right side of the image though other shapes and rotations are present at this location and squares are seen at all other combinations of rotations and translations. In the bottom row, training set excludes all shapes on the right side of the image.
the top row,
sation? So while Locatello et al. (2019) train their models on complete datasets to investigate the degree of disentanglement and sampling efficiency, we systematically exclude generative factors from training in order to test for combinatorial generalisation (see Methods and Results).
2 METHODS AND RESULTS
We assessed combinatorial generalisation on two different datasets. The dSprites image dataset (Matthey al. 2017) contains 2D images in black and white that vary along five generative factors: shape, scale, orientation, position-x and position-y and focuses on ma- nipulations of single objects. The 3D Shapes dataset (Burgess Kim 2018) contains 3D im- ages in colour that vary along six generative factors: floor- wal fall-hue, object e bejject-oiinttaten
contrast dSprites, the images are more realistic, which has been shown to aid reconstruction performance (Locatello et al.) 2019). To test combinatorial generalisation, we systematically excluded some combinations of these gener- ative factors from the training data and tested reconstruction these unseen values. Test cases can be divided into three broad categories based on the number of combinations excluded from training.
object-shape, object-scale
Recombination-to-Flee toemmmet (red squares Figure [T]: The model has never been trained on one combination of all of the generative factors. In dSprites, an example of this case would be excluding the combination: Ishape-ellipse, scale=1, orientation<: 120", position-x 8.5.5, position large ellipse at 120" in the
y>0.5] from the training set i.c. the model has never bottom-right corner, though it has seen all other combinations.
Recombination-ton Rannge (green squares in Figure []: The model has never been trained on all combinations of some of the factors (i.e. a subset of generative factors). For example, in the 3D Shapes dataset, combinations with [object- hue=1, shape=sphere] have been left out of the training set - i.e. none of the training images contain blue sphere. This condition is more complex than as an entire range of combinations [floor-hue--..... wall-hue-o...1 ob ject-hue-1, shape-sphere, scale-0...1, orientation-0...1] have been left out (here bold text indicates the range of values excluded). When the number of generative factors is larger than three, "Recombination-to-Ree -onnaag is, in fact, set of conditions that vary in difficulty, depending upon how many generative factors have been excluded. Another example would be excluding all com- binations where (floor-huess bue=1, object hhesl, shape=1, scale=1]. Here smaller range of
Recombinationnto--Beeee
3

---

Published as a conference paper at ICLR 2021
combinations [floor-hue=1, wall-hue=1, object-hue=l, shape=1, scale=1, orlentation:o...1 have been excluded.
Extrapolation (blue squares Figure D: This the most challenging form of generalisation where models are tested on values of generative factors that are beyond the range of values ob- served in the training dataset. For example, in the dSprites dataset, all combinations where [posi tion-x> 0.5] have never been seen.
Each of these conditions is interesting for different reasons. A model that learns compositional representations should be able to combine observed values of shape (ellipses), lation (bottom- right) and rotation (09 to 120%) to generalise all unseen combination of factors. The simplest case is the recombination-to--lllnee been trained, but a model that learns entangled representations might also succeed based its training on highly similar patterns (generalisation by interpolation). A more challenging case is accombination-tt-ranne condition given that more combinations have been excluded, making gen- eralisation by similarity (interpolation) more difficult. The final condition is not form of combi- natorial generalisation as the model cannot combine observed values of generative factors to render images. Indeed compositional representations may be inadequate for this form of generalisation.
condition in which all combinations but one have
2.1 IMAGE RECONSTRUCTION WITH DSPRITES DATASET
In the dSprites dataset, for testing the Recombination-to-llenee
case, we split each range of values
of a generative factor into three bins, so that we had 3 bins for all five generative factors. We then remove one of these 243 combinations during training,
x 3 x 3 such combinations of
namely those that satisfied [shape-ellipsis, position-x >= 0.6, position-y >= 0.6, 120" <= rotation 240", scale 0. In other words ellipses in the bottom right corner with those given rotations, which is a relatively small number of combinations that are all very similar to each other.
For the Recombination-to-nnn to-rrange case, we tested three different variants. First, we excluded all com- binations where [shape=square, position x>-.5] The model sees other shapes at those positions during training and it sees squares the left-hand side of the screen. Thus the models experiences both generative factor values independently and has to recombine them to produce a novel image at test time. In the second case, we excluded all combinations where [shape-square, scale>0.5] 3). In the third case, we excluded all combinations where [shape=square, rotation> 96" We observed very similar results for all three cases and below we report the results for the first variant.
Finally, for the Extrapolation case, we excluded all combinations of generative factors where Ipo sition-x a1. We chose set of different values for x: x 0.16, 0.25, 0.50, 0.75, where æ is normalised in the range [0, 1] (results shown in Figure 2 for x = 0.50). At test time the model needed to reconstruct images where translation along the x-axis, I, was greater than the cutoff
value.
We tested three classes of models on all three types of generalisation: standard Variational Au-
toencoder (VAEs, Kingma& & Welling (2013); Rezende et al. (2014)) 3-VAE (Higgins et al.) 2017
Burgess et al. 2018) with 3 8 and 3 12. FactorVAE (Kim & Mnih 2019) with = 20, 7 50
and y 100. The architectures are the ones found in Higgins et al. 2017). Burgess et al. (0118)
and Kim & Mnih (219)) (Details in the Appendix). We used a batch size of 64 and learning rate of 5e - 4 for the Adam optimizer (Kingma & Ba (017). In each case, we simulated three seeds and we report results for runs where we obtained largest disentanglement.
As shown by Locatello et al. 2019),, none of the models trained end-to-end in an unsupervised man- ner produce perfectly disentangled representations. Since we were interested in studying the effect of disentanglement on generalisation, we compared our results with model where we removed the encoder and directly gave disentangled latents as inputs to the decoder. We call this model the ground-truth decoder (GT Decoder from here on). This decoder uses the same MLP architecture as the one used in Higgins et al. (0177). We tested deeper decoders with convolutions and batch norm as well, but found no benefit a decrease in performance.
We measured the level of disentanglement using the framework introduced in Eastwood & Williams 2018). The procedure consists of using the latent representations generated for each image to predict the true generative factors using regression model (in our case, Lasso regression; see Ap-
4

---

Published as conference paper at ICLR 2021
Recombination element
Recombination to range
Extrapalation
Oliginal
latents
factors
scale
orientacion
prax
pos't
shape
scale
posx
shape
posk
shape
orientation
poa'l
train
test
train
test
train
test
Figure 2: Image reconstruction and disentanglement for the dSprites dataset (a) Top row shows examples of input images and the four rows below show reconstructions by four different models. Three pairs columns show reconstructions in training and test conditions. Left) Recombination- to-Element condition where the models did not [shape ellipae, scale orientation 120%, po sition-x 0.5, position-y 0.5], Middle) Recombination-tor sisiiiaattiott--annne Range condition where models did not [shape aguare, position-x > 0,51, Right) Extrapolation condition where models did not [posi tion-x 0.5] (b) Visualisation disentanglement. In each panel, columns show latent variables and show the generative factors. The size of the square represents the relative importance of the latent variable for predicting the generative factor. Sparse matrices indicate higher disentanglemen [Eastwood Williams) 2018). Each disentanglement matrix corresponds the model on that row in (a) in the Reconstructiontto-Ran Ro-naeeg condition. The visualisation of the entire set of models and all conditions shown in Appendix
pendix A
The level of disentanglement is quantified by their 'Overall disentanglement metric', which we call D-score here.
Figure 2 shows examples of model reconstructions for each the conditions which help assess the reconstruction success qualitatively (more examples are shown in Appendix C). A more quantitative assessment of the models can be made by examining the negative-log-lkeee likelihood of reconstructions for different conditions, plotted Figure 3 The amount disentanglement achieved by the models trained end-to-end varied over broad range and was function of model architecture and the hyper- parameter (3 and y) values. In general, reconstruction accuracy was better for smaller values of both during training and testing. This has been observed before and is a known issue encountered when increasing the value of 3 parameter (Hoffman Johnson, (016).. We found that models were able to perform the Recombination-to--lkee
generalisation but failed in the Recombination-to. Range and Extrapolation cases. In these cases, models either showed really poor reconstruction of the critical element substituted one of the excluded combination with combination that had been observed during training (see reconstructions for test cases in Figure aa)). Moreover, the amount of generalisation did not depend on the degree of disentanglement Indeed, the GT Decoder using perfectly disentangled representations was no better than the end-to-end models. Even though this model achieved lower NLL score, examining the image reconstructions showed that it failed to reconstruct the essential combination excluded from the training data (see Appendix(B)
The Recombination-tn to--ange condition shows another interesting qualitative difference between the entangled and disentangled models. All models failed to generalise, but in different ways. Entangled models tended to put blob in the correct location, which allows them to minimise loss pixel space over large set of test examples. In contrast, the models with higher level of disentanglement fell back the most similar shape (in pixel space) that they had seen at that location.

---

Published as a conference paper at ICLR 2021
Recombination to element
Recombination to range
Extrapolation
dSprites
200
2000
Modals
VAE
50
FactorVAEX
FastorvAEyy-50)
20080
FactorvAEN
Shapes30
15000
Data
Decodar
Train
Test
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.5
1.0
Disentanglement
Disentanglement
Disentangloment
Figure 3: Disentanglement vs reconstruction NLL. The relation between the level of disentan- glement and the performance of the model. Performance of the training data plotted along with performance in the test (generalisation) data. Disentanglement does not provide any help in perfor- mance for the end-to-end models. The ground truth decoder (GTD) is less affected, yet it is still the case that it fails to generalize (see Figure and Figure 4
condition was solved by all the models, regardless of disen- tanglement score. In fact, the entangled models tended achieve better reconstructions evidenced by the disentangled models with 3=12 which had hard time reconstructing ellipses at small scales and tended just produce a circle instead.
Finally, the Recombinatioontt--lleeeee
The second panel in Figure shows the coefficients computed by the disentanglement metric for the Reconstruction-too Range condition. The size of each square denotes the relative importance of latent (column) in predicting the corresponding generative factor (row). The higher the disentan- glement, the sparser the matrices. An examination of these matrices revealed that different models achieved large range of disentanglement though none of the end-to-end models achieved perfect disentanglement.
2.2 IMAGE RECONSTRUCTION WITH 3D SHAPES DATASET
The procedure for testing on the 3D Shapes dataset parallels the dSprites dataset above. The 3D Shapes dataset has six generative factors: floor-hue, wall-hue, object-hue, abject-shape, object-scale condition, we excluded one combination from training: huu 0.5, wall-hue
-riecttorientation. For the Recombination-to-Eleenns
object- hue 0.3, object-shape-gee Linder, object-scale=1, objeettordettation=)) For the Recombination-to-Ree tange condition, we excluded all combinations where hobject-hue >= 0.5 (eyan), sbject-shape oblong] and trained other combinations. This means that models saw several combinations where object-hut
>= 0,5 and where obj abject-shan was oblong but never the combination to- gether. For the Extrapolation condition, we excluded all combinations where [floor-hue >= 0.5].
We trained the same set of six end-to-end models above as well as the GT Decoder. All end- to-end models were trained for 65 epochs (around 500000 iterations as in the original articles), while the GT Decoder in Appendix C and clearly show that the models were able to learn the task. The results for the test conditions are shown in Figure 4 As it was the case with the dSprites dataset, we observed that the level of disentanglement varied across models, with VAE showing low D-score and Factor-VAL showing a high Dscore. We also tested the perfectly disentangled model where a decoder learns to construct images from disentangled latents.
trained for 1000 epochs. Reconstructions for the training set shown
shown in Figure 3 (bottom row) and some examples of typical reconstructions
All models managed to reconstruct the held-out combination in the Recombination- twaabiinnttioot--lleeeeeeee con- dition. However, none of the models succeeded in correctly reconstructing the held-out combi- nations in the Recombination-to--ral
or Extrapolation conditions. In both cases, we observed large reconstruction error either due to poor overall reconstruction (Extrapolation case) or because the critical combination, Lobject-hue, object-shape] training. And again, we did not see any correlation between disentanglement and the extent of com- binatorial generalisation. Even though the perfectly disentangled model had lower NLL score (sec
replaced with combination observed during

---

Published as a conference paper at ICLR 2021
Recombination element
Recombination range
Extrapolation
b
VAS Original
latents
factors
scale
shape
orientation
Hoor muse
Bete--AEgg-
wall hue
ooject huc
scale
shape
orientation
Fection FectorinEle
ooject
scale
shape
orientation
floor hue
GT Decoderr
wall hue
ooject hue
scale
oriontation
train
test
test
train
test
10
Figure 4: Image reconstructions and disentanglement for the Shapes3D dataset. We the same layout as in Figure a) Reconstruction examples for each of the three generalisation condi- tions. For the first condition, the model has not magenta floors with purple cylinders, yet it is able to reconstruct them properly. For the second condition, it has not seen magenta oblong shapes, yet it has seen it in other colors and it has condition magenta floors have never been seen during training. b) Example Hinton diagrams of the coefficients used to compute disentanglement. Diagram in each row corresponds the model in the same row in (a). Sparse matrices are better and the perfect one (up to permutation) is shown at the bottom.
magenta other shapes. Finally, the in the third
Figure bottom row), like other models it failed reconstruct the critical [object-hue, object-shape) combination that was left out in the training data (see example images of reconstruction in Figure and Appendix C
2.3 IMAGE COMPOSITION EXPERIMENTS
The limited combinatorial generalisation in the experiments above could be because of the limita- tions of the task rather than the models or their internal representations. Even though the models learned disentangled representations sentations, it could be that the simple reconstruction task does not provide enough impetus for the decoder learn how combine these disentangled representations enable generalisation. There- fore, in the final set of experiments we designed a variation of the standard unsupervised task that requires combining generative factors in order to solve the task using the dSprites dataset.
extent,
provided perfectly disentangled repre-
This new task is illustrated in Figure 5(a). The input consists of two images and an action. The goal of the task is to take the first (reference) image and modify it so that it matches the second (transform) image along the dimension specified by the action. This action is coded using a one- vector. This design based on the question answering task in [Santoro al.) (2017) and the compositional one in Higgins et al. (2018c). We produced training and test sets for each condition by sampling exference-rrasffoon
pairs along with an action uniformly from the generative factors. We ensured that this sampling respected the experiment restriction, so that the transformed image is not outside the current set.
The standard VAE is inadequate to solve this task. Therefore, constructed model with the architecture shown in Figure (].). This model first applics an encoder to both images, obtaining low-dimensional latent representations of each image. It then combines these latent representations with the action to obtain a transformed internal representation. There are several ways in which the input representations of the two images could combined with the action. We tried three different

---

Published as a conference paper at ICLR 2021
b
Output image the but shape from the transform image
Encode reference image
Encode transform Image
Decoder
change shape
Reference
image
Tansform
trage
action
Teesformed
Image
Figure 5: Image composition task. (a) An example of the composition task. In this case, the shape of the output must match the transform and rest of the values must match the reference. (b) The general architecture used, based the standard VAE. The model uses the same encoder both images. Then a transform takes latent representation samples and combines them to produce a transformed representation. This used produce the transformed image.
Table 1: Model performance in the second set of experiments.
Experiment
D-score
NLL (training)
NLL (testing)
Extrapolation 0.82
31.73
19138.82
Recomb to range
0.71
50.10
346.10
Recomb element
0.96
36.57
13.74
methods: (i) using standard MLP, (ii) element-wise interpolation between the two representations, with the interpolation coefficients determined by the action, and (iii) concatenating each input rep- resentations with the actions and linearly combining the resultant vectors. We obtained qualitatively similar results with all three methods, but found that the method (iii) gave the best results, providing well as highest levels of disentanglement. Therefore, in the
greatest accuracy of reconstruction rest of the manuscript we describe the results obtained using this method. Once this transformed internal representation has been generated, it is decoded to obtain output image. We the same encoding and decoding modules as the ones used by Burgess et al. (2018). The results for this model shown in Table I The model managed to solve the task. In doing it also comes to rely representations with a high level disentanglement (see Figure(o(b) even though the 3 parameter is set 1. Models with higher values could not solve the task altogether, presumably because the constraint is too strong. However, as was the case in the previous experiment, models failed to solve the challenging generalisation tasks.
In Figure 5 we show some examples of the model's behaviour and its internal representations. The model failed in similar ways the disentangled models in the previous experiment, confus- ing shapes when presented with unseen combinations. Even the Recombination to element case showed some failures (like in the example shown in Figure ((a)) though the models were, in gen- eral, successful in this condition, as can be inferred by comparing the negative log- log-likelihoods for the training and test trials for this condition in Table
DISCUSSION
is frequently assumed that disentangled representations are implicitly compositional (Higgins et al. ()) This raises the question as whether disentangled representations support com- binatorial generalisation, a key feature of compositional representations (Fodor Pylyshyn 1988). However, found evidence for this. Indeed representations that varied from highly entangled to perfectly disentangled were equally successful at cooomiinaiion-oooleemeen
generalisation, and both failed recombination--o..o -o-range and extrapolation. This was the even when trained VAE on an explicitly combinatorial task that led models to learn highly disentangled representations that were better generalisation.
Our findings might seem to contradict previous reports showing success in combinatorial general- isation tasks. In Eslami al, 2018), some success reported when rendering novel 3D shapes
8

---

Published as a conference paper at ICLR 2021
training
test
training
test
training
test
b
lanent
factors
shape
Rocombratt to altemma aine
roriantation
posk
pusY
shape
orientation
position
orientation
shape
orientation
shape
transformes transform
Recombinal zo angge
scale
orientation
pock
posy
Extrapolation
shape
orientation
posk
Recombination to element
Recombination to range
Extrapolation
posY
Figure Image composition generalisation results (a) Each column shows an example trial which consists of reference image, action, transform image and the transformed (output) image. We show examples of both training and test trials. Each of the training trials results the correct (expected) transformed image, while each of the test trials shows failure. (b) Visualisation of the degree of disentanglement for three conditions by the model. The sparse representation reflects the high level of disentanglement achieved by these models in this task.
with colours that had been previously seen on other shapes. And in Higgins et al. (2018c) was reported that using disentangled representation allowed the model recombine observed shapes and colours in novel ways. However, it is not clear what sorts of combinatorial generalisation was tested. For example, consider the SCAN model (Higgins al. (2118) that could render a room with [wite suitcase, blue walls, magenta floor], even though was never shown this combination during training (see Figure 4 in HHggins et al. (20T8c)). But, unlike our training set, not clear what exactly was excluded while training this model, and they may have been testing generalisation in condition similar to our Recombination-to--leee limited to the on the basis of overall similarity (interpolation) rather than exploiting disentangled representations in order support Pylyshyn 1988).
condition. Our finding that generalisation was condition suggests that models simply generalising
powerful form of compositional generalisation described by [Fodor &
effective in supporting This raises the question why disentangled representations representations are necessary but combinatorial generalisation. One possibility is that disentangled sufficient to support the principle of compositionality. On this view, model must also include mechanism for binding these representations in way that maintains their independence. This point has previously been made in the context of connectionist representations by Hummel in Hummel (2000). Another possibility is that a model may be able to perform combinatorial generalisation without needing disentangled indeed compositional representations if the training environment
rich enough (Chaabouni al. 2020 Hill et al. 2020 Lampinen & McClelland] 2020).
important goal for future research is to develop networks that support the more difficult forms of combinatorial generalisation and extrapolation. In fact there is already an active range of research this direction that include networks with specialized modules (Santoro et al. 2017), mechanisms (Mitchell & Bowers 2020; Hummel & Biederman 1992), structured representations (Higgins al.) 2018c] Watters et al. 2019), or learning objectives Vankov Bowers 2020) that may show greater success. It will be interesting to how these and other approaches fare in the more difficult generalisation settings we have identified here, and the role of disentanglement in any solutions.
ACKNOWLEDGGMMENT
We would like to thank Chris Summerfield, Irina Higgins, Ben Evans and Jeff Mitchell for useful discussions and feedback during the development of this research.
This research was supported by ERC Advanced Grant (Generalization Mind and Machine, #741134)

---

Published as a conference paper at ICLR 2021
REFERENCES
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation A Review and New Perspectives. arXiv:1206.55333 [cs], April 2014. htLLitttp///xxxv.rrr//aa/////// 5538 arXiv: 1206.5538.
Jeffrey S. Bowers, Ivan I. Vankov, Markus F. Damian, and Colin J. Davis. Why do some neurons in cortex respond to information in a selective manner? Insights from artificial neural networks. Cognition, 148:47-63, March 2016. ISSN 0010-0277. doi: 00.1016/f.coomniiion22000 12.000
URL http://wt /www.//wwww/
Chris Burgess and Hyunjik Kim. 3d shapes dataset. http://gghhaoooonddeeppim///asss 2018.
Christopher Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Desjardins, and Alexander Lerchner. Understanding disentangling in S\beta VAF. arXXiv/804.000 0959903999)) [cs, star), April 2018. http::/:://arxi/ http://www..rr/b/////be/// 03599
arXiv: 1804.03599.
Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni. Compositionality and generalization in emergent languages. arXiv preprint arXiv:2004.09124,
2020.
Sunny Duan, Loic Matthey, Andre Saraīva, Nicholas Watters, Christopher P. Burgess, Alexander Lerchner, and Irina Higgins. Unsupervised Model Selection for Variational Disentangled Rep- resentation Learning. arXiv: /905. 12614 [cs, stat], February 2020. URL http://arxxi.oorrg Abs/1905.12613
arXiv: 12614.
Cian Eastwood and Christopher KI Williams. A framework for the quantitative evaluation of dis- entangled representations. In International Conference Learning Representations, pp. 15, 2018.
S. M. Ali Eslami, Danilo Jimenez Rezende, Frederic Besse, Fabio Viola, Ari S. Morcos, Marta Garnelo, Avraham Ruderman, Andrei A. Rusu, Ivo Danihelka, Karol Gregor, David Reichert, Lars Buesing, Theophane Weber, Oriol Vinyals, Dan Rosenbaum, Neil Rabinowitz, Helen King, Chloe Hillier. Matt Botvinick, Daan Wierstra. Koray Kavukcuoglu, and Demis Hassabis. Neural scene representation and rendering. Science, 36016394):104412110 June 2018. ISSN 0036- 8075, 1095-9203. doi: 10. wtt//ccinne..aar611 URthhttss: /tetps::/sciencee sciencemag. 0eettt/606669444
Publisher: American Association for the Advancement of Science Section: Research Article.
Babak Esmacili, Hao Wu, Sarthak Jain, Alican Bozkurt, N. Siddharth, Brooks Paige, Dana H. Brooks, Jennifer Dy, and Jan-Willem Meent. Structured Disentangled Representations. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 2534-2533. PMLR, April 2019. URL http://proccedings.... wtt..r...eeeeee/eeeeeeeeeeee 2640-3498.
html ISSN:
Jerry Fodor and Brian P. McLaughlin. Connectionism and the problem of systematicity: Why Smolensky's solution doesn't work. Cognition, 35(2):183 May 1990. ISSN 0010-0277. doi: 101101100010002777999000044
URL http://wwww.cceeeeeeeee...cc/coa
arricle/pp1/00100279000444
Jerry A. Fodor and Zenon W. Pylyshyn. Connectionism and cognitive architecttur::: A critical analy-
sis. Cognition, 3-11)33-11 March 1988. ISSN 0010-0277. doi: 10. 10.100160011000277789900013 htL.htttp///ww.scieeeeeell http://cee........ooooc...cceeea
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick,
Shakir Mohamed, and Alexander Lerchner. 3-VAE: Learning basic visual concepts with a con- strained variational framework. pp. 13, 2017.
Irina Higgins, David Amos, David Pfau, Sebastien Racaniere, Loic Matthey, Danilo Rezende, and Alexander Lerchner. Towards Definition of Disentangled Representations. 1PX.::1812.0225 [cs, stat], December 2018a. URL phttt:://aaxee wwtp////////.......///// 02230
/
arXiv: 1812.02230.
10

---

Published as a conference paper at ICLR 2021
Irina Higgins, Arka Pal, Andrei A. Rusu, Loic Matthey, Christopher P. Burgess, Alexander Pritzel, Matthew Botvinick, Charles Blundell, and Alexander Lerchner. DARLA: Improving Zero-Shot Transfer in Reinforcement Learning. 1707 08475 [cs, stat], June 2018b. URL http:
//arxiv.org/abs/1107. 1784/77777700775 08475
arXiv: 1707.08475.
Irina Higgins, Nicolas Sonnerat, Loic Matthey, Arka Pal, Christopher P. Burgess, Matko Bosnjak, Murray Shanahan, Matthew Botvinick, Demis Hassabis, and Alexander Lerchner, SCAN; Learn- ing Hierarchical Compositional Visual Concepts. arXiv: Seii:1707.03389 [cs, stat], June 2018c. URL
http://arxtw.org//bb//100 .76817038999999
arXiv: 1707. 000.00339.
Felix Hill, Andrew Lampinen, Rosalia Schneider, Stephen Clark, Matthew Botvinick, James L. McClelland, and Adam Santoro. Environmental drivers of systematicity and generalization in situated agent.
htap/ttop//arxx.ooor///////1 org/abs/1910..
[cs], February 2020. //tpp/t///aaxxii
00571
arXiv: 1910.00571.
Matthew D Hoffman and Matthew J Johnson. Elbo surgery: yet another way carve up the varia- tional evidence lower bound. In Workshop in Advances in Approximate Bayesian Inference, NIPS, volume pp. 2, 2016.
E. Hummel. Localism as a first step toward symbolic representation. Behaviorul and Brain Sci- encey, 2344)4480-411 December 2000. ISSN 0140-525X. doi: 10.
John E Hummel and Irving Biederman. Dynamic binding in neural network for shape recognition. Psychological review, 99(3):480, 1992.
Hyunjik Kim and Andriy Mnih. Disentangling by Factorising. arXiv:1802.05983 [cs, stat), July arXiv: 1802.05983.
2019.
Diederik Kingma and Jimmy Ba. Adam: A Method Stochastic Optimization. 14Xiiv [cs], January 2017. URL htttp://aaaaai..rg///a//1111111 6980
arXiv: 1412.6980.
Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes. arXiv:1312.6114 [cs, stat], December 2013. URL URL_/ttp.//arrii.rr///b////2
arXiv: 1312.6114.
Klaus Greff, Aaron Klein, Martin Chovanec, Frank Hutter, and Jürgen Schmidhuber. The Sacred Infrastructure for Computational Research. In Katy Huff, David Lippa, Dillon Niederhut, and M Pacer (eds.), Proceedings of the 16th Python in Science Conference, pp. 49 56, 2017. doi:
10.25080/shinma- 7f4iiiinnnanaa767----- 7f4c6e7-000
Brenden Lake and Marco Baroni. Generalization without Systematicity: On the Compositional Skills of seruencc-to-Sequennee Recurrent Networks. In International Conference Machine Learning. pp. 2873 2-82.. PMLR. July 2018. URL http::/prooceedings ml Lr. wwwpp/eesssseee v80/
lakel8a [ ISSN: 2640-3498.
Andrew K Lampinen and James L McClelland, Transforming task representations to allow deep learning models to perform novel tasks. arXiv preprint arXiv:2005.04311 2020.
Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging Common Assumptions the Unsupervised Learn- ing of Disentangled Representations. pp. 11, 2019.
Gary Mareus. Deep learning: A critical appraisal. arXiv preprint arXiv:1801.00631/ 2018.
Emile Mathieu, Tom Rainforth, N. Siddharth, and Yee Whye Teh. Disentangling Disentanglement in Variational Autoencoders. arXiv: /812.0233 [es, stat], June 2019. URL. http:://raxx.. org/ 1812. 02833 arXiv: 1812.02833.
Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dSprites: Disentanglement lesting Sprites dataset. 2017. htRL:htss:///iiii http://gittttoocccm//////mmaa//s dataset/
James L McClelland, David E Rumelhart, PDP Research Group, and others. Parallel distributed processing. Explorations in the Microstructure of Cognition, 2:216-271, 1986. Publisher: MIT Press Cambridge, Ma.
11

---

Published as a conference paper at ICLR 2021
Jeff Mitchell and Jeffrey S. Bowers. Harnessing the Symmetry of Convolutions for Systematic Gen- eralisation. 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1-8, Glas- gow. United Kingdom, July 2020. IEEE. ISBN 1--2816--922-22 doi: 10. 10//1IIN/N 2020.9207183. URL htttttts///eeepplleeee ieee httgg//ccmmennt22/78e//
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga. Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. dAlché-Buc, E. Fox, and R. Garnett (eds.). Advances in Neural Information Processing Systems 32, pp. 8024-8035, Curran Associates, Inc., 2019. URL httt:://papppp 9015- pytscch an- wwmpeeatieeestyyleee high- performance deep learning- Library. pdf
heurips.cc/pappee
Pedregosa, G. Varoquaux, Gramfort, V. Michel, Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 2011.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic Backpropagation and Approximate Inference in Deep Generative Models. 1401:440014082 [cs, stat], January 2014. URL http:/t:t/a/xi..rrg//////// abs/ /401. 4082
arXiv: 1401.4082.
Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski, Razvan Pascanu, Pe- ter Battaglia, and Timothy Lillicrap. A simple neural network module for relational rea- soning, In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 4967-4976. Curran Associates, Inc., 2017. URL /ttp://papers nips. CC/ paper/ 7682 a simple Le- neural metprrok-ontttwwo-mmooooo module- for re relationa reasoning pdf
Paul Smolensky. The constituent structure of connectionist mental states: A reply Fodor and Pylyshyn. Southern Journal of Philosophy, 26(Supplement):137716.. 1987. Publisher: Citeseer.
Paul Smolensky. Connectionism, constituency, and the language of thought. University of Colorado at Boulder, 1988.
S. Desroziers J. Kriss V. Fomin, Anmol and A. Tejani High-level library to help with training neural networks in pytorch. https://github. comppytorcchiigiiteet 2020.
Sjoerd van Steenkiste, Jürgen Schmidhuber, Francesco Locatello, and Olivier Bachem. Are Disen- tangled Representations Helpful for Abstract Visual Reasoning? pp. 14, 2019.
Ivan Vankov and Jeffrey S. Bowers. Training neural networks to encode symbols enables combinatorial generalization. Philosophical Transactions of the Royal Society B: Biological Sciences, 37511791) February 2020. doi: 10. (t09///st..009..000..
URL Intpps: Publisher: Royal So-
//royalsccitttoooolliinnnnnoogoo http://0...99///tt..bb 0309
ciety.
Nicholas Watters, Loic Matthey, Christopher P. Burgess, and Alexander Lerchner. Spatial Broad- cast Decoder: A Simple Architecture for Learning Disentangled Representations in VABs. 1rXiv:1901.07000 fcs, stat), August 2019. URI. http://arriv.orr//a///0000000000 arXiv: 1901.07017.
Shengjia Zhao, Hongyu Ren, Arianna Yuan, Jiaming Song, Noah Goodman, and Stefano Ermon. Bias and Generalization in Deep Generative Models: An Empirical Study. 18/1.0 fes, stat], November 2018. URL htR./http///axxi.orrr/aaaas
03259
arXiv: 1811 R1.00955..
12

---

Published as a conference paper at ICLR 2021
A MODELS AND TRAINING
For our experiments on the standard unsupervised task we used two different VAE architectures. The first one is the same found in Higgins et al. (2017) and uses a 2-layer MLP an encoder with 1200 units and ReLU non-linearity. The decoder is a 3-layer with the same number of units and the Tanh non- non-linearity. The second architecture the one found in Burgess et al. (2018) and consists of a 3-layer CNN with 32x4x2x I convolutions and max pooling, followed by a 2 2-layer MLP with 256 units in each layer. The decoder is defined to be the transpose of this architecture. ReLU non-linearity where applied after each layer of the CNN and the MLP for both the encoder and the decoder. Both models used a Gaussian stochastic layer with 10 units as in the original papers.
We also tested two variants of this last architecture, one found in Mathieu et al. (2019) which changes the shape of the convolution and another with batch normalisation. Neither variant exhibited any improvements to disentanglement or reconstruction the full dSprite data and so were not included in the rest of the experiments.
For the image composition task we used same in Burgess et al. (2018) that we described above. The latent transformation layer was parameterized as:
htransformed = WW,caffe action] + Wccat [2; action]
where 2, and 2, are the samples from the stochastic layer for reference and transform image, cat is the concatenation operation performed along the column dimension. The output is another 10- dimensional vector with the transformed latent code.
Alternatively we also tried a 3 layer MLP with 100 hidden units, but saw no benefit in performance and decreased disentanglement when trainined on the full dataset.
Training on the unsupervised tasks ran for 100 epochs for dSprites and 65 epochs for Shapes3D, even though models converged before the end. The learning rate was fixed at 1e and the batch size at 64. 3 values used were 1, 8,8, 12, 16 on the full dSprite dataset. 3 = 4 and / = 16 where not included in the rest of the experiments since the former offered very little disentanglement and the latter very large reconstruction error. For the FactorVAE we used In the composition task the models where trained for 100 epochs with 3 = 1. Using 3 higher than 1 interfered with the model's ability to solve the task they where not used.
20, 50, 100 throughout.
For the ground-truth -tuth decoders (GT Decoder) we used the same MLP decoder of Higgins et al. (217)) mentioned above. Using deeper decoders with convolutions with/without batch norm after each layer was also tested, but did not provide significan benefits and also decreased the performance on some of the conditions.
All the models where implemented in PyTorch (Paszke et al. 2019 and the experiments where performed using the Ignite and Sacred frameworks (V. Fomin & Tejani 2020; Klaus Greff et al., 2017].
To measure disentanglement we used the framework proposed by Eastwood & Williams (0118) with a slight modification. The approach consists of predicting each generative factor value, given the latent representations of the training images using a non-linear model. In our case we used the LassoCV regression found in the Sklear library (Pedregosa et al. (011) with an a coefficient of 0.01 and cross-validation partitions. Deviating from the original proposal, we do not normalize the inputs to the regression model since we found that this tends to give lot of weight to dead units (when measured by their KL divergence). This is likely due to the model "killing" these units during training after they start with high KL value, which might not completely erase the information they carry about given generative factor.
Working code for rumning these experiments and analyses can be downloaded at uthttps://githti com/mmrl/disen -aweeentt-ndd-d-- and-gen
13

---

Published as a conference paper at ICLR 2021
B EXTRA PLOTS FOR DSPRITES DATASET
modelss
PactorVABy--00 Facteronddy PactoroADy-1000
Figure Disentanglement scores for dSprites. The disentanglement analysis results for the dSprites dataset. The scores for each of the metries evaluated by the DCI framework: disentan- glement (left), overall disentanglement (middle) and completeness (right) for each the conditions.
VAE
B B-VAEIB-B
B- B-VAEBB-12
FactorVAEdy-20
FactorVAEly=01
FactorVAEly-100
Iotent
Iseent
latert
faczOr
factor
factor
faczor
factor
factor
Figure 8: Hinton diagrams for dSprites dataset. The matrices of coefficients computed by the framework plotted as Hinton diagrams. These are used to obtain the quantitative scores the panel above. They offer qualitative view of how the model is disentangling. On the left is how perfect disentanglement looks in this framework.
14

---

Published as a conference paper at ICLR 2021
Train
Test
Recombination element
Recombination range
Extrapolation
Figure 9: Reconstructions for the dSprites dataset. For each condition and model, these are some reconstruction examples for both training and testing. There is general success and failure in the Recombination Element (top) and Extrapolation (bottom) conditions, respectively. For this last condition, the models seem to reproduce the closest instance they have seen, which tranlated to middle of image. For the Recombination to range (middle), the models tend to resort to generating blob at the right location, minimising their pixel-level error.
C EXTRA PLOTS FOR THE 3D SHAPES DATASET
15

---

Published as a conference paper at ICLR 2021
mideles
VAE Bert vald wdd
Bets-vARg--B
Factonisehy-221
Figure 10: Disentanglement analysis for Shapes3D. The disentanglement analysis results for the 3D Shapes dataset. The scores for each the metries evaluated by the DCI framework (Eastwood & Williams 2018): disentanglement (left), overall disentanglement (middle) and completeness (right) for each of the conditions.
VAE
Beta-VAElBaa 8)
Deta-VAEB=121
FactorVAEYy=00 FactorvAEK
FactorvAENy=100
Iatent
Iatent
Iatert
factor
factor
factor
factor
factor
Figure 11: Hinton diagrams for 3DShapes dataset. The matrices of coefficients computed by the framework plotted as Hinton Diagrams. As discussed in the main text, these matrices offer
a qualitative view of how the model disentangling. In general, sparse matrices indicate higher disentanglement. It is clear from these diagrams that the degree of disentanglement varies over broad range for the tested models.
16

---

Published as a conference paper at ICLR 2021
Train
Test
Recombination
Recombinat incombination range
Extrapalation
Figure 12: Reconstructions for the Shapes3D dataset. For each condition and model, these some reconstruction examples for both training (left) and testing (right). In each case, the input image shown the left-most column and each subsequent column shows reconstruction by different model. The test images always show a combination that was left out during training. All training images are successfully reproduced. However, reconstruction for images only succeeds consistently for the Recombinntiiontto-lleeee Fo-emmenn condition (top). All reconstructions fail in the Ex- trapolation condition (bottom) while most of them fail for the Recombination-H to---agee condition (middle). There are occasional instances in Recombination-to Ceaattioontt--aagee Range condition that seem to cor- rectly reconstruct the input image. This seems to happen when the novel combination for color and shape is closest to the ones the model has experienced during training. For example, models are better when the oblong shape is paired with cyan (which is close to green, which it has seen) and worse magenta.
17


Reviews:
[{"review_id": "Lv1HquEsfUu", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Post-revision update\n------------------------\n\nThanks to the authors, I think that the revision provided by the authors makes the paper substantially stronger. The inclusion of the more complex Shapes3D dataset substantially improves the experiments, and I think the discussion has improved. I have revised my rating to a clear accept in accordance.\n\nOriginal Review\n----------------------\n\nThis paper evaluates the role of disentanglement in generalization. The authors begin by articulating a useful distinction between different kinds of generalization by interpolation, recombination or extrapolation. They then train VAEs (and variations thereof) on a controlled, synthetic dataset, using two different training paradigms. They show that the models only generalize well to one of the most elementary types (recombination to element), but do not extrapolate well to the more difficult kinds of generalization. They also show that disentanglement does not seem to correlate with better generalization.\n\nI find this paper to be marginally above the acceptance threshold, but it has room for improvement (see below).\n\nStrengths:\n\n* Generalization and the role of compositionality and disentanglement therein are very important issues.\n\n* I like the articulation of the different kinds of generalization, and the clear illustration thereof.\n\nAreas for improvement:\n\n* The relationship of these results to Locatello et al. (2019) would be worth discussing further. They also showed that disentanglement did not necessarily lead to better generalization, with a wider range of experiments (though therefore perhaps less deep in evaluating different types of generalization).\n\n* The experiments are very narrow. The paper uses a single dataset (though with two different tasks), and it is very toy. As noted by Locatello et al. (2019), the inferences drawn from a single dataset may be very biased. While simple synthetic datasets can be useful for allowing more carefully controlled experiments, it would be useful to explore the same experiment\ns using different datasets with different features (e.g. color and texture). It would also be useful to understand generaliza\ntion in richer, more realistic datasets (see below).\n\n* It would be useful to show some of the major claims in a less opaque way. For example, to show the (non)relationship between disentanglement and generalization, the authors could make a plot with D-score on the x-axis, and generalization performance on the y-axis (with different plot panels for the different types of generalization, perhaps). \n\n* One reason that exploring other datasets is important is that the particular inductive biases of the models may facilitate generalization along certain feature dimensions. For example, the fact that convolutions are generally (relatively) translation-invariant but not rotation-invariant mean that the model might more easily extrapolate to unseen translations. In order to draw broad conclusions, it would be useful to both explore more diverse datasets and quantitatively analyze in more detail t\nhe generalization along different dimensions.\n\n* Increasing realism can produce qualitative improvements in compositional generalization in some settings. For example, Hill et al. (2020) showed that e.g. generalization was better in a 3D setting than 2D setting, and that an RL agent showed 100% compositional generalization in a setting where a classifier only showed ~80%, for instance (although this generalization was recombination, not extrapolation). Thus, the poverty of the stimuli may alter the paper's conclusions. Even if your study is useful, it's worth discussing this limitation more explicitly.\n\n* The paper raises the question of why disentangled representations are not more effective in supporting compositional generalization, but it's worth asking why we assume that they would. Disentanglement $\\neq$ compositional representations $\\neq$ systematic generalization. Fodor & Pylyshyn suggest that compositional representations are necessary for systematic generalization, but they don't certainly don't provide an empirical definition of how to evaluate compositionality of representations. Indeed, it's hard to define such a notion: \"The question of whether a model [generalizes] according to compositional task structure is distinct from the question of whether the model’s representations exhibit compositional structure. Because the mapping from [...] representations to behavior is highly non-linear, it is difficult to craft a definition of compositional representations that is either necessary or sufficient for generalization\" (Lampinen & McClelland, 2020). Your results, along with those of Locatello et al (2019) and others, lend support for this argument. Disentanglement in some middle layer of the network does not seem to show a causal role in generalization, presumably in part because the processes intervening between that representation and the output are nonlinear, because that nonlinear decoder is also capable of failing for some combinations of latent representations even if the representations themselves are compositional, and/or because disentanglement is not a sufficient notion of compositionality. Given these difficulties, I think you could potentially extrapolate further than you do, to ask whether we should be worrying about representations at all, rather than just evaluating (and improving) behavioral performance on the different types of generalization you articulate.\n\n* However, even empirical evaluation is challenging in more naturalistic settings. The notion of \"disentanglement\" or \"composition\" may be harder to define in realistic datasets. It's not clear what the appropriate decomposition of a complex naturalistic image is — objects are a natural place to start, which is why Higgins and others have focused on this type of decomposition. But what counts as disentangled in a visual scene of e.g. a forest? Is each leaf an object that must be disentangled in its own right? Should the color of each piece of bark on each tree be represented by its own dimension, since *in principle* it could vary independently? This seems unreasonable, which is perhaps why disentanglement is usually demonstrated on very simplistic datasets. Yet a human can of course *attend* to any particular aspect of the scene to disentangle that dimension as needed. In the real world, human-like performance might require the ability to construct *new decompositions on the fly,* because the appropriate decompositions may change as the task or data shifts. That is, the idea of seeking a priori disentanglement with respect to fixed dimensions might not be the right way to go about achieving human-like generalization, especially if we want that generalization to extrapolate to new data and new tasks. (C.f. Lampinen & McClelland, 2020 for some other related discussion.)\n \n* It also seems likely that the processes that allow humans to exhibit strong generalization may require extended or additional processing, rather than a single feed-forward pass as in a VAE. This would be necessary to allow the sort of attentive disentanglement described in the previous point. For example, the Stroop effect in cognition seems to me to illustrate feature entanglement, which requires higher level control processes to resolve the appropriate response. The paper does discuss the idea that other mechanisms or architectures might be involved in the discussion, but it seems it bears more elaboration, especially w.r.t. the above points about the definability of disentanglement with real world data.\n\n\n\nReferences\n-----------\n\nHill, Felix, et al. \"Environmental drivers of systematicity and generalization in a situated agent.\" International Conference on Learning Representations. 2020.\n\nLampinen, Andrew K., and James L. McClelland. \"Transforming task representations to allow deep learning models to perform novel tasks.\" arXiv preprint arXiv:2005.04318 2020.\n\nLocatello, Francesco, et al. \"Challenging common assumptions in the unsupervised learning of disentangled representations.\" international conference on machine learning. 2019.                                     ", "extra_scores": {}, "extra_text": {}}, {"review_id": "GDzfQutLtLm", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper studies the performance of models producing disentangled representations in the downstream task of combinatorial generalization. The experiments suggest that models producing disentangled representations do not generalize well enough. \n\nPros:\n- The paper is well-written and easy to follow. \n- The authors propose four novel benchmarks to systematically study the ability of a model to generalize. \n\nConcerns:\n- The key concern is that the paper does not present enough experiments to support the authors' claims. The study was conducted only for one dataset; I would suggest to include several other datasets in your study, e.g., MPI 3D, Shapes 3D, Cars 3D datasets. Also, the results would be stronger if the paper presented the assessment of other disentanglement specific metrics; see, for example, MIG [1], Modularity [2], etc.  \n\nComments/questions:\n- The combinatorial generalization task looks similar to the abstract reasoning task; it was shown that disentangled representations help in this downstream task [3]. How do you think, why it does not hold as well for combinatorial generalization?\n- Perhaps it would be interesting to vary random seeds in addition to $\\beta$ values; it was shown in Locatello [4] that random seeds sometimes have a stronger influence on disentanglement scores than model hyperparameters.\n\nMinor comments:\n- In some places, you write \"generalization\", in other -- \"generalisation\".\n\nUPD: The authors addressed my concerns and added additional experiments. The paper is improved, therefore, I increase the rating.\n\nReferences:\n\n[1] Ricky TQ Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. Isolating sources of disen- tanglement in variational autoencoders. In Advances in Neural Information Processing Systems, pp. 2610–2620, 2018.\n\n[2] Karl Ridgeway and Michael C Mozer. Learning deep disentangled embeddings with the f-statistic loss. In Advances in Neural Information Processing Systems, pp. 185–194, 2018.\n\n[3] van Steenkiste, Sjoerd, et al. \"Are Disentangled Representations Helpful for Abstract Visual Reasoning?.\" Advances in Neural Information Processing Systems. 2019.\n\n[4] Locatello, Francesco, et al. \"Challenging common assumptions in the unsupervised learning of disentangled representations.\" international conference on machine learning. 2019.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "ArZgdLjTWGt", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary\n\nLearning disentangled representation is often considered an important step to achieve human-like generalization. This paper studies how the degree of disentanglement affects various forms of generalization.  Variational autoencoders (VAEs) is trained with different levels of disentanglement on an unsupervised task by excluding combinations of generative factors during training. At test time the models are used to reconstruct the missing combinations in order to measure generalization performance. The paper shows that the models support only weak combinatorial generalization. The paper also tests the models in a more complex task which explicitly required independent generative factors to be controlled. The paper concludes that learning disentanglement representation is not sufficient for supporting more difficult forms of generalization.\n\nStrengths\n\nThe paper studies 4 types of generalization, interpolation, recombination to element, Recombination to range, extrapolation. \n\nIt shows beta-VAE can achieve reasonable generalization by interpolation, not the other three types. \n\nWeaknesses\n\nThe paper's study is limited to beta-VAE and dSprites dataset. However, it makes broad claims on the role of disentanglement in generalization.\n\nBeta-VAE has limitations in disentanglement. It is not clear other disentanglement approaches such as Wasserstein auto-encoder, InfoGAN-CR (ICML'20) would not generalize much better. \n\nThe study is on unsupervised disentanglement. Unsupervised disentanglement has inherent limitations, see \"Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations\" ICML'19.\n\nThe paper should conduct experimental studies on other datasets, e.g. those in the above reference.\n\nFor image composition tasks, it states \"concatenating input representations with the actions and linearly combining the resultant vectors\". It will be great to explain the insights.\n\nDecision\n\nThe paper has some interesting results on the role of disentanglement in generalization. However, the paper's study is very limited to specific model and a single dataset. Therefore, it is below acceptance threshold.\n \n----Post-revision update---\n\nThe authors have provided results on a second dataset 3DShapes and two more models – Factor-VAE and a perfectly disentangled model. However, the construction results of the GT decoder is much worse that other models, see Figure 2; it does not reconstruct the details of the \"heart\" shape even for training and the edges of the \"square\" are not straight. This begs the question how good the GT decoder is. The open question is, what is the generalization capability of a GT decoder that can both reconstruct and disentangle perfectly?\n\nWasserstein auto-encoder has been shown to disentangle better and the regularization term is on the aggregate posterior instead of individual samples. Without results on WAE, the paper should refrain from making broad claims on disentanglement.  Furthermore, it would be interesting to investigate GAN based approach such as InfoGAN-CR as well.\n\nFor an experimentation paper, it should be more thorough and go beyond just two shape datasets.\n\nI applaud the additional results the authors provided. I still think the paper is borderline (more toward 6 now). If it fixes the aforementioned weaknesses, I would recommend accept. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "8ZoH8MCEUWt", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nSummary\n---\nA large body of work creates disentangled representations to improve\ncombinatorial generalization. This paper distinguishes between 4 types of\ngeneralization and shows that existing unsupervised disentanglement approaches\ngeneralize worse to some and better to others.\n\n(introduction)\nThere are 3 types of combinatorial generalization. Each requires a learner to\ngeneralize to a set of instances where 0, 1, or 2 dimensions have been completely held out.\nPrevious work has not distinguished between these kinds of generalization when\ntesting how disentangled representations generalize. This work does that to\nunderstand the relationship between disentanglement and combinatorial\ngeneralization in a more fine grained manner.\n\n(approach)\nThroughout this paper, beta-VAE and a recent variant are trained with varrying levels of\ndisentanglement (controlled by beta) to reconstruct d-sprites images.\nThese images contain simple shapes and are generated using 5 ground truth\nlatent factors. The ground truth latent factors allow disentanglement to be\nmeasured (using Eastwood and Williams 2018), essentially by checking whether\nthe ground truth latent factors are linearly separable in the learned latent space.\n\n(experiment - plain reconstruction)\n* reconstruction error differs for different types of combinatorial generalization (holding out fewer dimensions is easier)\n* reconstruction error is not highly correlated with disentanglement\n\n(experiment - compositional reconstruction)\nInstead of reconstructing the input, a version of the input with one attribute changed is generated.\n* generation error differs for different types of combinatorial generalization (holding out fewer dimensions is easier)\n\n(conclusion)\nUsually disentanglement is encouraged to achieve combinatorial generalization, but this paper presents a simple experiment where it doesn't do that.\n\n\n\nStrengths\n---\n\nThe central claim of the paper may help clarify the disentanglement literature.\n\nIt seems very useful to taxonomize generalization in this way.\n\nThe writing and motivation is generally very clear. The figures are easy to understand and help demonstrate the narrative.\n\nThis paper aims to characterize an existing line of work in detail rather than proposing a new approach/dataset/etc. I like work of this nature and would like to see more like it.\n\n\n\nWeaknesses\n---\n\n\n1. The relationship between disentanglement and generalization is clearly or quantitatively demonstrated:\n\nThe most interesting claim in this paper is that disentanglement is not necessarily correlated with combinatorial generalization, but this claim is not clearly supported by the data.\n\n* The main support comes from table 1. Here higher D-score does not necessarily mean lower test NLL. This observation should be made quantitative, probably just by measuring correllation between D-score and test NLL.\n\n* Table 2 seems to contradict this claim. In that case higher D-score does mean lower test NLL.\n\n\n2. The taxonomy of generalization is a bit too specific to be useful and a bit incoherent:\n\nThe difference between \"Interpolation\" and \"Recombination to element\" generalization\nis not clear to me. Each of the purple and red cubes in figure 1a represents\na combinations of rotation, shape, and translation factors.\nIt may be that it makes a difference when some dimensions are categorial\nand others are continuous, as in the Interpolation example, but this doesn't\nseem to really solve the factor because continuous latent variables\nare still latent variables. I see some vague intuition behind this distinction,\nbut the paper does correctly identify the precise distinction.\n\nFurthermore, this taxonomy of generalization seems limited to me.\nIt seems like \"Recombination to element\", \"Recombination to range\", and \"Extrapolation\"\njust hold out a different number of dimensions (e.g., \"none\", \"rotation\", and \"shape and rotation\", respectively).\nThis begs the question of what happens when there are 4 generative dimensions?\nIs generalization when 3 of those are held out also called \"Extrapolation\"?\n\nI think more work needs to be done to create a taxonomy which precisely and clearly generalizes\nto N latent factors and creates a more coherent distinction between combinatorial and\nnon-combinatorial generalization.\nHowever, I think it's possible to create a better taxonomy and that it\nwill probably be very useful to do so.\n\n\n3. The paper should test the idea more thoroughly, on more datasets and on more disentanglement approaches. For example, it could include other datasets or tasks with different ground truth factors of variation (e.g., 3D chairs [1]). It could also include more disentanglement approaches like [2].\n\n\n[1]: M. Aubry, D. Maturana, A. Efros, B. Russell, and J. Sivic. Seeing 3d chairs: exemplar part-based 2d-3d\nalignment using a large dataset of cad models. In CVPR, 2014.\n[2]: Esmaeili, B. et al. “Structured Disentangled Representations.” AISTATS (2019).\n\n\n\n\n\nComments / Suggestions\n---\n\nDescribe the disentanglement metric in more detail. From the beginning disentanglement is treated differently from combinatorial generalization. It's not immediately clear what disentanglement is that makes it different and why that's interesting to study. For example, initially one might think that beta-VAE is inherently disentangled.\n\nCan this taxonomy of generalization be generalized to continuous domains? For example, can it be generalized to any (typically continuous) hidden layer a neural net learns?\n\n\n\nPreliminary Evaluation\n---\n\nClarity - The presentation is quite clear.\nQuality - The claims are not quite well enough supported. The experiments that were run don't support a clear conclusion and more experiments should have been run to support a more general conclusion.\nNovelty - I don't think anyone has catalogued the performance of disentanglement methods in terms of a generalization taxonomy.\nSignificance - This paper might help clarify the disentanglement literature and more broadly help people think about combinatorial generalization.\n\nI like this paper because of its clarity, novelty, and significance. However, I think the quality concerns are significant enough that it shouldn't be accepted at this stage.\n\nFinal Evaluation (Post Rebuttal)\n---\nThe author response and accompanying paper revision clearly and effectively addressed each of the 3 main weaknesses I pointed out, so I raised my rating.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper seeks to empirically study and highlight how disentanglement of latent representations relates to combinatorial generalization. In particular, the main argument is to show that models fail to perform combinatorial generalization or extrapolation while succeeding in other ways. This is a borderline paper. For empirical studies it is also less agreed upon in general where one should draw the line about sufficient coverage of experiments, i.e., the burden of proof for primarily empirically derived insights. The initial submission clearly did not meet the necessary standard as the analysis was based on a single dataset and studied only two methods (VAE and beta-VAE). The revised version of the manuscript now includes additional experiments (an additional dataset and two new methods), still offering largely consistent pattern of observations, raising the paper to its current borderline status. Some questions remain about the new results (esp the decoder). 
Decision: Accept (Poster)