Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper studies the generalization ability of neural networks in out-of-distribution (OOD) settings on simple datasets (such as Shapes3D) and shows that generalization performance drops when real-world datasets are used in comparison to artificial datasets.
main_review: The direction of this work is nice and the problem tackled by this paper is indeed important. The obtained results are nice and there is indeed some potential value in this work. However, I have a few concerns with the paper:

i) To test generalizability of neural networks, test datasets are constructed using four methods: interpolation, extrapolation, random, and composition. However, previous works uses these approaches as data augmentation to improve the generalizability of neural networks.  For example, multi-scale inputs during training in YOLOv2 allows it to predict well across different input dimensions. Another example is random erasing, which also improves generalization ability of neural networks. A discussion about these methods to improve generalization ability is missing. Also, what happens to the generalizability of neural networks when they are trained with these previous approaches?

YOLOv2: https://arxiv.org/pdf/1612.08242.pdf

Random erasing: https://arxiv.org/pdf/1708.04896.pdf

CutMix: https://arxiv.org/abs/1905.04899

Random Augmentation: https://arxiv.org/abs/1909.13719

ii) The generalization performance of neural networks drops when we move from artificial datasets to real-world datasets. However, the datasets studied in this paper are simple and limited, and far from real-world datasets. They do not consider many factors that are natural in real-world datasets, including non-rigid deformations and intraclass variability (e.g., people wear different types of glasses).

iii) Previous works have shown that larger models generalize better than smaller models. I do not see any argument about the model capacity and generalizability in the paper. Can authors comment on that?

https://arxiv.org/pdf/1802.08760.pdf

=========

Clarification: 

In page 4, it is stated that "Combined with a set operation such as pooling, CNNs then achieve translation invariance." To my understanding, translation invariance is one of the fundamental properties of convolutions. I think operations like pooling forces CNNs to learn multi-scale representations

https://www.ijcai.org/Proceedings/83-2/Papers/091.pdf

summary_of_the_review: This paper studies generalizability of neural networks using four factors (interpolation, extrapolation, random, and composition). Previous works on better data augmentation policies uses similar factors to improve the generalization of visual recognition networks. So, the results of this paper are not surprising.


=================
Post rebuttal: I am convinced with author's response and I am leaning towards acceptance.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents an empirical study of generalization in visual representation learning. The paper compares in-distribution (ID) generalization to out-of-distribution (OOD) generalization of three types -- interpolation, extrapolation, and composition. Datasets are constructed with several factors of variation where the training split exhibits some factors and the test split exhibits others, constructed to test the ID and OOD settings. A variety of different representation learning models with different kinds of inductive bias are tested. The main findings are that: 1) all models perform better ID than OOD, indicating that all methods fail to find the true underlying generative mechanisms that created the data, 2) when some factors are ID and others are OOD the ID factors are modeled well despite that the OOD factors may be modeled poorly, demonstrating a kind of modularity between the learning of different factors. The paper will be accompanied by a benchmark for others to improve on this task.
main_review: I think this is a very good paper. The writing is exceptionally clear and the problem framing and literature review will be an especially valuable resource for future work. The experiments are well designed and thorough, and the findings are valuable contributions to an important area.

The main weakness I see is that I'm not sure there is a lot of information gain from this paper. The results are more or less what I think most readers will expect. The findings seem in line with past work on generalization and with conventional wisdom -- ID works, OOD works worse, inductive biases can help on OOD, etc. That's not to say the present work is not valuable -- there are open questions here and the present paper is one of the most extensive studies I have seen on them. Just that the paper doesn't provide entirely unexpected answers. Because of this, I think the most valuable contribution of the paper may be the benchmark it provides, on top of which future studies may find something really new.

To elaborate further, the related work covers numerous papers that have come to roughly similar conclusions. Two more papers come to mind that also have similar conclusion but were not discussed:
* Packer et al., “Assessing Generalization in Deep Reinforcement Learning”, 2019
-- This paper also studied ID vs OOD generalization but in the context of reinforcement learning. Figure 1 from Packer et al. shows their train/test setup, which is quite related to the current paper's settings in Figure 2. The conclusions are similar to the current paper: 1) extrapolation is harder than interpolation, 2) SOTA algorithms that are supposed to "solve" this problem fail.
* Jahanian et al., "On the 'Steerability' of Generative Adversarial Networks", 2020
-- This paper studied the ability of GANs to extrapolate in their latent space. The finding is that they struggle to generate transformations that extend beyond the distribution seen during training. These results are similar to the findings in the current paper on the failure of VAE latent representations to extrapolate beyond the training data.

This is all to say I think there is ample prior literature that make the present conclusions unsurprising. But thorough work on this topic, and new benchmarks, is still valuable and that's what the current paper provides. I should also note that the finding about modularity is something I hadn't seen before, and I think that's a valuable contribution as well.

Aside from this, I think the paper is very solid. A few minor comments follow:
1. Using CelebGlow feels a bit awkward since it is a generative model fit to data, and then you are again fitting samples from this model with another generative model. I wonder if there could be some bias where the samples are easier to model with a VAE since they were generated with a related model (Glow)... It's probably all fine but some commentary on this could be useful.
2. Repeated reference to Hendrycks and Diettrich 2019.
3. I would say inductive biases 1 and 3 overlap: inductive bias 1, as it is implemented in the paper, could be considered a special case of transfer learning where the pretraining is done with VAEs. This could be clarified to avoid implying that these are independent inductive biases.
4. “if factors are located in a particular edge of the FoV hyper cube given by all FoVs” — “edge” —> “corner”?
5. “Here, we further see that, on average, the performances seem to increase as we increase the supervision signal.” — This is a bit vague I don’t see it fully reflected in the figure. This point could be made more precise. What does “increase the supervision signal” refer to?
6. “We find that the degree of downstream performance correlates weakly but positively with the degree of disentanglement (Pearson ρ = 0.63, Spearman ρ = 0.67)” — I’m not sure I would call these weak correlations. In many fields I believe this would be considered a strong correlation. 
7. “Existing notions of disentanglement models with a readout MLP do not help to facilitate the learning of the underlying mechanisms in the tested datasets.” — I don’t understand this conclusion. The correlations are substantial. My understanding of the results is that greater disentanglement _does_ correlate with better ability to identify the underlying mechanisms. Appendix Fig 7 seems to support this as well, with all but one of the correlations being positive.

summary_of_the_review: This is a solid paper that contributes new empirical findings and a new benchmark on an important topic. I believe this work will stimulate future work on representation learning, disentanglement, and OOD generalization. I don't see any major errors and recommend accepting.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper introduces a new dataset: CelebGLOW which is a controllable environment generation dataset, which can be used in the same form as significantly less complex datasets, such as sprites. The paper evaluates over 3 key inductive biases using the dataset: "representational format" (using images in this paper), neural network architectural variants (MLPs/CNNs/ transformers, etc), and ability to perform transfer learning. The paper evalutes the models on 4 given modes of generalization: interpolation, extrapolation, compositional learning and "random", and finds extremely fascinating conclusions through extensive experimentation. 
main_review: Positives:
- The paper is extremely well motivated and addresses a very important problem in the field of generalization in NNs
- The dataset also looks very promising
- The experiments are extremely well done and thorough
- The idea of having "conclusion" section in each subsection in the experiments is a very nice idea.

Negatives:
- I would prefer if the figures had appropriate captions - all tables and figure captions should be self-sufficient. There should be sufficient details in them such that I am able to understand what the paper is doing.
- I am slightly concerned about what "new" information this adds to the field - a lot of the conclusions in this paper were well known in the community, to my knowledge. This does not discredit the work at all, since it is always good to have experimental support related to "intuitions" that people have had, but I would be keen on the authors to answer what "previously unconfirmed" things they really discovered in this paper.
- the paper states "Instead of extrapolating, all models regress the OOD factor towards the mean in the training set." which should be clarified as I am not sure what that means. The ratio of distances idea makes sense, but I am not sure what the authors are trying to conclude with that statement.
- Is there a reason why CelebGlow outperforms MPI3d and shapes3d considerably in Figure 4 and 5 on the "extrapolation" benchmark?

Limitation:
- The paper adaquately points out where and how the NNs fail to adapt, without giving a clear direction of what are possible ways to remediate the current gap. It would be a useful discussion point in the paper. 
summary_of_the_review: It is a well-written paper with great analysis and a very useful study to have. I have small concerns about the novelty of the conclusions, but the paper is still a great addition to the community.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper tests 17 unsupervised, weakly supervised, and fully supervised representation learning approaches to infer the generative factors of variation across three simple datasets in well-controlled conditions. In addition, the authors introduce a CelebGlow dataset, which is more complex. The generalization abilities are characterized as composition, interpolation, and extrapolation. The conclusions from these empirical observations on the experimental results are interesting and suggest that most networks fail to generalize.


main_review: This is a very well-written and clear paper with sufficient implementation details for reproducibility.
Generalization in deep learning models remains an interesting question to study. As this study demonstrates, the problem is far from solved. 
The paper surveys a wide range of 17 approaches on four datasets with increasing complexity of visual stimuli. However, I have concerns with the following issues, which I believe, limit the contribution of this paper.

1. At several places noted by the authors, also from Fig 4 and 5 (CelebGlow is better than MPI3D), there seems to exist variances in the generalization ability of models across different datasets or even within individual factors within each dataset. In addition to reporting the overall empirical results in entire datasets, can the authors provide explanations about why models perform better in certain factors/datasets and not the rest? And why would different variation factors within the same dataset result in different generalization ability?

2. In some cases, it is fair to say that deep models fail at some generalization tests. However, have the authors considered designing human experiments and quantitatively verifying human generalization abilities in similar problem settings? Human performance can serve as an essential benchmark to compare with the generalization ability of computational models. This is not a requirement (and certainly not the focus) of the current study. But it is hard to come up with an upper bound or even some sense of what level of generalization one should aspire to achieve.

3. The conclusions drawn from the paper are interesting and informative. However, there is a lack of explanations or insights about why these models fail to generalize in certain conditions. And if so, what are potential solutions to improve generalization ability.

4. The authors have focused on deep learning-based approaches. It is unclear that models without learning via backprop would be able to generalize. In other words, would other learning algorithms using Hebbian rules have better generalization ability (e.g. Boltzmann machines)?

5. The variation factors include translation, rotation, scaling, color, age, and so on. Have authors considered other dimensions? These factors are predefined and manually picked. The models might generalize better in some of these factors compared with the rest of factors. Have the authors considered ways to automatically exploit other factors that the models would show impairment in generalization abilities?

summary_of_the_review: The study poignantly points out failures of current models in generalization through a large number of empirical studies. The work is well written and clear. It is not particularly novel to argue that models fail to generalize, but the current study provides quantitative and extensive benchmarks in well-controlled settings. Beyond pointing to limitations of current models, the study does not offer new paths towards new algorithms or solutions to those problems. However, sometimes, understanding what the challenges are is a good step towards generating momentum to find better solutions.
review_title: null

Meta-review: This paper presents a through study of generalization in visual representation learning. It compares in distribution generalization to out of distribution generalization using a comprehensive benchmark. The paper received very positive reviews from all reviewers. Reviewers agreed that the paper has several strengths: It is very well written, the presented benchmark is very useful and the analysis is thorough. One concern that was brought up by the reviewers was that a majority of the presented findings are expected and in a sense, known to the community. The authors have addressed this concern by pointing out that their findings are more fine grained than past works and that their proposed benchmark is a stepping stone towards measuring general robustness. I must note that in spite of this concern, all reviewers have maintained their strong acceptance scores. I agree with the reviewers. This paper makes a strong contribution to this important problem via its benchmark and analysis, which future works can build off of, and hence I recommend acceptance.
Decision: Accept (Poster)