Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: ## Overview
The paper tackles the identifiability problem in generative modeling, i.e., recovering the true latent representations from which the observed data originates. The paper argues that identifiable variational autoencoder (iVAE) suffers from intractability issue which leads to suboptimal solutions. The paper instead proposes an identifiable normalizing flows (iFlow) method as an alternative. The proposed iFlow outperforms iVAE in experiments using synthetic data. The paper is very well motivated and well supports its claim.

## Summary of the contributions
1. The paper proposes iFlow, an identifiable normalizing flow method which allows recovery of true latent space from observed data.
2. The paper shows iFlow outperforms iVAE in synthetic experiments.
3. The paper provides theoretical justification on the identifiability of the proposed iFlow method.

## Overall feedback
I find the paper well written and well organized. Easy to follow the content even though I am not expert on this matter. The paper provides both theoretical justification and empirical experimental validation which show the superior performance of proposed iFlow method. So I am leaning towards accepting the paper. The theory seems correct but I did not check all the equations and proofs in depth so I am not very confident about my rating.

## Suggestions
1. Please make sure all equations are properly punctuated.
2. The comparison with iVAE seems tricky since they use different types of architectures. Can you also report the hyper parameters used for iVAE?
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper is about learning an identifiable generative model, iFlow, that builds upon a recent result on nonlinear ICA. The key idea is providing side information to identify the latent representation, i.e., essentially a prior conditioned on extra information such as labels and restricting the mapping to flows for being able to compute the likelihood. As the loglikelihood of a flow model is readily available, a direct approach can be used for learning that optimizes both the prior and the observation model.	

The paper is very clear and very easy to follow. The idea is quite clear, and the direct approach is really attractive. Unfortunately, the experimental section is quite limited and does not fully study representational aspects. There is only an illustrative simulation on synthetic data, that in a sense verifies the theory. I was also not able to see the additional insight that the identifiability theory in 4.2 provides additional to Khemakhem et al. (2019). Please clarify.

My main concern is that this model is actually a supervised model that learns a mapping from u to x via z. Hence, the theoretical ‘identifiability’ guarantee needs to be stated with some care as this depends on the choice of an arbitrary u. For example if we set x = u, will learning take place? Please comment.  

Overall, I like the approach but I am uncertain about the level of novelty. For such a paper, one should expect a much more involved computational study. In this respect, I feel that the paper could be accepted but it certainly feels as if it needs more computational results as otherwise the original contribution would be too incremental for ICLR standards. I am giving a provisional reject in the hope that the authors will provide convincing arguments about their original contributions for clarification.

review_title: Official Blind Review #1

Meta-review: Main content:

Blind review #1 summarizes it well:

This paper is about learning an identifiable generative model, iFlow, that builds upon a recent result on nonlinear ICA. The key idea is providing side information to identify the latent representation, i.e., essentially a prior conditioned on extra information such as labels and restricting the mapping to flows for being able to compute the likelihood. As the loglikelihood of a flow model is readily available, a direct approach can be used for learning that optimizes both the prior and the observation model.	

--

Discussion:

Reviewer questions were mostly about clarification, which the authors addressed during the rebuttal period.

--

Recommendation and justification:

All reviewers agree the paper is a weak accept based on degree of depth, novelty, and impact.
Decision: Accept (Poster)