Venue: ICLR 2018 Workshop

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper addresses the problem of learning representations that disentangle style from label information. A simple but important observation is made: the loss function used for training VAEs does not promote disentanglement, despite the obtained results. The authors empirically show the importance of the inductive bias provided by the shared parameters in the decoder or decoder for obtaining these results.

The paper is well written and provides several convincing experiments on a simple dataset.

Please provide details of the architectures used. Are the encoders/decoders used convolutional or MLPs? How would that affect the results?
review_title: Review

final_score: rating: 7: Good paper, accept
review_content: - This paper shows an interesting set of experiments investigating the causal relationship between disentangling and (SGD, loss). The claim is that regularization due to SGD is the main factor for disentanglement

- If i am not mistaken, similar types of conclusion were shown in the beta VAE work (Higgins et al), where they show that disentangling is mostly due to the optimization process. It's good to see more studies of it and the paper should better reflect such prior works. 

- If its the optimization or inference is causing this, then training the neural net with other algorithms such as SPSA or other evolutionary methods is a good control test

review_title: this paper studies the relationship between disentangling, SGD, loss and bias in architectures

final_score: rating: 7: Good paper, accept
review_content: The authors explore the question why semi-supervised variational autoencoders that combine a discrete variable y with continuous variables z are able to disentangle style and content. They show that the variational objective admits solutions that have the same ELBO, but do not disentangle style and content. They look at architectures in contain separate encoders/decoders for each value y=k as a means of investigating which component of the model induces disentanglement. They additionally find some evidence to suggest that gradient descent converges more quickly towards an disentangled solution than an entangled one. 

This is overall a very reasonable workshop contribution. My main comment would be that there are some (very recent) references missing from the discussion. These references point to the fact that we can decompose the ELBO into a number of terms, one of which takes the form of a total correlation (a higher-dimensional generalization of the mutual information) between latent variables under the marginal distribution 

    q(y, z) = 1/N Σ_n q(y,z | x^n)

Based on this, a number of papers have emerged in which authors point out that we can successfully disentangle representations by emphasizing this total correlation term in the objective. This suggests, that, contrary to what the authors write in the abstract, the VAE objective *does* in fact contain a component that induces disentangled. More precisely put, in these models the prior  p(y, z) = p(y) p(z) factorizes. The VAE objective implicitly minimizes KL(q(y, z) || p(y) p(z)), which means that we should in principle learn a marginal distribution q(y, z) = q(y) q(z) that also factorizes, which implies the invariance conditions that the authors pose in section 1. 

This aside, I see nothing wrong with this submission, and I am happy for it to appear, as long as the authors incorporate the references below, and perhaps include some discussion thereof.

References

1. Hoffman, M. D. & Johnson, M. J. Elbo surgery: yet another way to carve up the variational evidence lower bound. in Workshop in Advances in Approximate Bayesian Inference, NIPS (2016).

2. Kim, H. & Mnih, A. Disentangling by factorising. arXiv preprint arXiv:1802.05983 (2018).

3. Chen, T. Q., Li, X., Grosse, R. & Duvenaud, D. Isolating Sources of Disentanglement in Variational Autoencoders. arXiv:1802.04942 [cs, stat] (2018).

4. Gao, S., Brekelmans, R., Steeg, G. V. & Galstyan, A. Auto-Encoding Total Correlation Explanation. arXiv:1802.05822 [cs, stat] (2018).
review_title: A submission that provides some insights, but lacks some (very recent) references

Meta-review: Congratulations, your paper was accepted to the ICLR workshop.
Decision: Accept