Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: This paper extends recent work by Khemakhem et al on nonlinear ICA to allow for unknown number of generative factors. This is tackling an important problem in the field of generative modeling, where one would like to extract the generative factors of a dataset that independently control its features (i.e. disentanglement).

The paper is very clearly written, does a great job at motivating the problem and presenting the recent results from Khemakhem et al, before extending them with some simple theorems and demonstrating their application on toy data + EMNIST.
I think that this research direction is extremely promising, and obtaining a theoretical understanding of when/why disentangling could work would be particularly valuable to the field, and I would lean towards acceptance so that this work gets more attention.

It is slightly surprising however that their empirical results seem to indicate that these theoretical conditions do not seem to be necessary, which should be investigated further (and might help illustrate the dichotomy in some claims and results obtained in the disentangling literature recently).

1.	It was unclear to me how one should/would decide what to use for $u$ or what to leave out to be factorized by the method. 
	This may be out of scope for this current work, but one way to answer that would be to leverage datasets with more fully labelled factorised data (e.g. dSprites [1]) and present how one should leverage these with $u$, in order to identify the original generative factors.
2.	Related, using EMNIST was interesting, but given the lack of “accepted” generative factors to be recovered, it is hard to tell if the 22 variables found are “correct” or more similar to using a generative model which would entangled the data more (and hence would falsely introduce extra latent variables).
3.	The toy dataset, with its random RealNVP network to produce the data, was less “mixed” than I expected, looking at Figure 2B. In its projected view, the clusters are still rather easy to identify by eye, which surprised me somehow? Could you comment a bit more on how difficult is the task, or present a VAE baseline that would fail to explain that data?
4.	I did not see how the threshold for selecting 22 latent variables on EMNIST was set? Was the 23rd latent variable significantly less informative? The spectrum on Figure 3A was not precise enough to assess this fairly and the single mention of “measured by the standard deviations of test data transformed to the latent space” was too vague to reverse-engineer the decision.
5.	It was interesting to read about the observations of when this method should fail. It would be interesting if a dataset with explicit “gaps” would be constructed to analyze this case.
6.	Figure 4 might benefit from mentioning “what” each variable controls for directly in their individual caption / on top of them, instead of having to read this through in the full figure caption.
7.	The current model in the end is modeling the latent state using a mixture of gaussians (although these now have a theoretical connection to the true generative factors). How much does this differ to existing generative models using VAEs with a mixture of gaussian prior [2, 3]?


References:
[1] dSprites dataset: https://github.com/deepmind/dsprites-dataset 
[2] https://arxiv.org/abs/1611.02648
[3] https://arxiv.org/abs/1902.03717

review_title: Official Blind Review #4

final_score: rating: 6: Weak Accept
review_content: Based on a recent work on identifying the joint distribution over observed and latent variables, the paper presents an extension of it, where the dimension of the latent space is not assumed to be known. Moreover, the authors propose a new neural network architecture based on previous propositions (RealNVP and NICE) to perform this identification.

The paper's presentation is relatively clear, although all the theoretical results are relegated to the appendix. The extension to unknown latent space dimension seems to be quite straightforward, given the recent work that this paper is based on. However, the experimental results performed on EMNIST are quite convincing and the results are interesting. 
Overall, I think this paper would be interesting to the ICLR audience.

It seems currently that there is a need to choose a dimension to be large enough (i.e., larger than the true dimension of the latent space). I would have appreciated some discussion (and some experiments) if this dimension is chosen too small. 
Also, when the dimension is chosen large enough, the non-informative learned latent variables are determined by looking at their standard deviations. Do they always have to be small compared to the informative latent variables?
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper builds upon the recent theoretical framework on nonlinear ICA, put forward in recent work Khemakhem et al. (2019) that draw a lot of attention. The latter work provides an extension of the basic nonlinear ICA that is closely related to a VAE with a conditional factorized prior, essentially introducing side information (with assumed extra observables u) to resolve non-identifiability issues. 

This paper proposes an invertible architecture related to RealNVP and NICE, coined as  GIN: the General Incompressible-flow network. 

On the positive side,

A key feature of the proposed methodology is model selection (such as determining the model order) that is in general a hard problem even in linear latent variable models. 
The performance is illustrated on the EMNIST dataset, as well as carefully constructed synthetic experiments. The semantic descriptions of each captured dimension, as detailed in the appendix, is particularly interesting. 
The experimental section is quite extensive. 

On the negative side, 

The paper is largely based on the results of a recent technical report (Khemakhem et al. (2019)) and is not self contained, hence rather hard to digest. Even the proofs in the appendix and the intuition requires the reading of this longer technical report. 

It is also not at all clear where the side information (variables u) is coming from. On EMNIST a natural candidate is the digit labels (and this turns out to be is indeed the case in the experimental section) but it is not very clear what conditions need to be satisfied. What prevents us selecting simply a subset of observations x as u? 

Lacking an explicit motivation, the exercise of writing the canonical parameters of a multivariate Gaussian in (2) - (4) is not very informative. This needs to be better motivated.

The prior structure resembles quite closely the general hierarchical priors SVAE proposed in https://arxiv.org/abs/1603.06277. It would be also informative to discuss the links with this approach.

Arguably, the key contribution of the current paper is in 3.1. and 3.2, but these are rather hastily written and quite short. Overall, the balance of known results and new contributions 

review_title: Official Blind Review #1

Meta-review: This paper builds on the recent theoretical work by Khemakhem et al. (2019) to propose a novel flow-based method for performing non-linear ICA. The paper is well written, includes theoretical justifications for the proposed approach and convincing experimental results. Many of the initial minor concerns raised by the reviewers were addressed during the discussion stage, and all of the reviewers agree that this paper is an important contribution to the field and hence should be accepted. Hence, I am happy to recommend the acceptance of this paper as an oral. 
Decision: Accept (Spotlight)