Venue: ICLR 2018 conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: This paper proposes to modify how noise factors are treated when developing VAE models.  For example, the original VAE work from (Kingma and Welling, 2013) applies a deep network to learn a diagonal approximation to the covariance on the decoder side.  Subsequent follow-up papers have often simplified this covariance to sigma^2*I, where sigma^2 is assumed to be known or manually tuned.  In contrast, this submission suggests either treating sigma^2 as a trainable parameter, or else introducing a more flexible zero-mean mixture-of-Gaussians (MoG) model for the decoder noise.  These modeling adaptations are then analyzed using various performance indicators and empirical studies.

The primary issues I have with this work are threefold:  (i) The paper is not suitably organized/condensed for an ICLR submission, (ii) the presentation quality is quite low, to the extent that clarity and proper understanding are jeopardized, and (iii) the novelty is limited.  Consequently my overall impression is that this work is not yet ready for acceptance to ICLR.

First, regarding the organization, this submission is 19 pages long (*excluding* references and appendices), despite the clear suggestion in the call for papers to limit the length to 8 pages: "There is no strict limit on paper length. However, we strongly recommend keeping the paper at 8 pages, plus 1 page for the references and as many pages as needed in an appendix section (all in a single pdf). The appropriateness of using additional pages over the recommended length will be judged by reviewers."  In the present submission, the first 8+ pages contain minimal new material, just various background topics and modified VAE update rules to account for learning noise parameters via basic EM algorithm techniques.  There is almost no novelty here.  In my mind, this type of well-known content is in no way appropriate justification for such a long paper submission, and it is unreasonable to expect reviewers to wade through it all during a short review cycle.

Secondly, the presentation quality is simply too low for acceptance at a top-tier international conference (e.g., it is full of strange sentences like "Such amelioration facilitates the VAE capable of always reducing the artificial intervention due to more proper guiding of noise learning."  While I am sympathetic to the difficulties of technical writing, and realize that at times sufficiently good ideas can transcend local grammatical hiccups, my feeling is that, at least for now, another serious pass of editing is seriously needed.  This is especially true given that it can be challenging to digest so many pages of text if the presentation is not relatively smooth.

Third and finally, I do not feel that there is sufficient novelty to overcome the issues already raised above.  Simply adapting the VAE decoder noise factors via either a trainable noise parameter or an MoG model represents an incremental contribution as similar techniques are exceedingly common.  Of course, the paper also invents some new evaluation metrics and then applies them on benchmark datasets, but this content only appears much later in the paper (well after the soft 8 page limit) and I admittedly did not read it all carefully.  But on a superficial level, I do not believe these contributions are sufficient to salvage the paper (although I remain open to hearing arguments to the contrary).
review_title: Because of the excessive length, poor presentation quality, and limited novelty, this paper is below the bar for ICLR at this time.

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper studies the importance of the noise modelling in Gaussian VAE. The original Gaussian VAE proposes to use the inference network for the noise that takes latent variables as inputs and outputs the variances, but most of the existing works on Gaussian VAE just use fixed noise probably because the inference network is hard to train. In this paper, instead of using the fixed noise or inference network for the noise, the authors proposed to train the noise using Empirical-Bayes like fashion. The algorithm to train noise level for the single Gaussian decoder and mixture of Gaussian decoder is presented, and the experiments show that fitting the noise actually improves the ELBO and enhances the ability to disentangle latent factors.
 
I appreciate the importance of noise modeling, but not sure if the presented algorithm is a right way to do it. The proposed algorithm assumes the Gaussian likelihood with homoscedastic noise, but this is not the case for many real-world data (MNIST and Color images are usually modelled with Bernoulli likelihood). The update equations for noises rely on the simple model structure, and this may not hold for the arbitrary complex likelihood (or implicit likelihood case). In my personal opinion, making the inference network for the noise to be trainable would be more principled way of solving the problem.
 
The paper is too long (30 pages) and dense, so it is very hard to read and understand the whole stuff. Remember that the ‘recommended’ page limit is 8 pages. The proposed algorithm was not compared to the generative models other than the basic VAE or beta-VAE.
review_title: An important problem has been tackled, but not in a satisfactory direction. 

final_score: rating: 2: Strong rejection
review_content: This paper attempts to improve the beta-VAE (Higgins et al, 2017) by removing the trade-off between the quality of disentanglement in the latent representation and the quality of the reconstruction. The authors suggest doing so by explicitly modelling the noise of the reconstructed image Gaussian p(x|z). The authors assume that VAEs typically model the data using a Guassian distribution with a fixed noise. This, however, is not the case. Since the authors are trying to address a problem that does not actually exist, I am not sure what the contributions of the paper are. 

Apart from the major issue outlined above, the paper also makes other errors. For example, it suggests using D_KL(q(z)||p(z)) as a measure of disentanglement, with lower values being indicative of better disentanglement. This, however, is incorrect, since one can have tiny D_KL by encoding all the information into a single latent z_i. Such a representation would be highly entangled while still satisfying all of the conditions the authors propose for a disentangled representation. 

Given the points outlined above and the fact that the paper is hard to read and is excessively long, I do not believe it should be accepted.
review_title: Substantial work needed

Meta-review: The reviewers agreed that the paper was too long (more than twice the recommended page limit not counting the appendix) and difficult to follow. They also pointed out that its central idea of learning the noise distribution in a VAE was not novel. While the shortened version uploaded by the authors looks like a step in the right direction, it was not sufficient to convince the reviewers.
Decision: Reject