Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposes a novel autoencoder algorithm, named Spherical AutoEncoder (SAE). In this paper, the authors argue that the sphere structure has good properties in high-dimensional. To leverage the properties, proposed algorithm centerizes latent variables and projects them onto unit sphere. To show the empirical performance of the proposed approach, the authors perform image reconstruction and generation using FFHQ dataset and MNIST dataset.

Comments:
I think the proposed approach, using spherical latent space, is interesting and make sense.

- As mentioned in section 3.2, the proposed algorithm is reduced to standard autoencoder since it is free from posterior inference. Then, to clarify the algorithm, it seems necessary to provide the formulation of objective functions.
- Is the objective still valid or reasonable even it is derived from the equation (10) without posterior inference?
- How does the objective change when centerization and spherization are applied to the GAN?
- Compared with using von Mises-Fisher distribution in the vanilla VAE, the advantage of the proposed method is not clear. To my understanding, the main difference seems to be whether using lower bound with posterior inference or deterministic framework without such approximation. However, there are no theoretical or empirical results to show the benefit of the proposed method. If theoretical or empirical results with reasonable intuition is provided, it will make the proposed algorithm more valuable.

Questions:
- Compare to ProGAN and StyleGAN, is the contribution of the paper to applying centerization to GAN and centerization and spherization to autoencoder?
- What dimension do you use as latent dimension in the experiments?
- Does the choice of prior distribution affect the experimental results? If so, is there any compatible reason with the intuition of SAE?

Typo:
Under equation (10) in page 5: \tilde{z} should be \hat{z}.

review_title: Official Blind Review #2

final_score: rating: 1: Reject
review_content: Summary

This paper considers the L2 normalization of samples “z” from a given prior p(z) in Generative Adversarial Netowks (GAN) and autoencoders. The L2 normalization corresponds to projecting samples onto the surface of a unit-hypersphere. Hence, to attempt to justify this normalization, the authors rely on some already established results regarding high dimensional hyperspheres. In particular, the focus is on the fact that, the Euclidean distance between any given point on a hypersphere and another randomly sampled point on the hypersphere tends to a constant, when the number of dimensions goes to infinity. This result is then used to show that the Wasserstein distance between two arbitrary distributions on a hypersphere converges to a constant when the number of dimensions grows. Based on this result, the authors claim that projecting the latent samples onto the surface of a hypersphere would make GAN less sensitive to the choice of the prior distribution. Moreover, they claim that such normalization would also benefits inference, and that it addresses the issue of variational inference in VAE.

Main comments. 

This paper is hard to follow and requires substantial improvements in terms of writing, owing to several grammatical and semantic issues. Moreover, there is a lack rigor; some important claims are supported neither by experiments nor by theoretical analysis. Experiments in the main paper are also weak. I can therefore not recommend acceptance. My detailed comments are below.

- An important claim in this paper is that the proposed approach “alleviates variational inference in VAE”. However, this requires clarification as well as theoretical/empirical justifications.
- In the introduction, it is stated that generated samples from VAE may deviate from real data samples, because “the posterior q(z|x) cannot match  the prior p(z) perfectly”. However, in VAE we do not expect the posterior to match the prior perfectly, as this would result in useless data representations or inference. Generation issues in VAE may rather be explained by the fact that, in this context we optimize a lower bound on the KL-divergence between the empirical data distribution and the model distribution. The latter objective does not penalize the model distribution if it puts some of its mass in regions where the empirical data distribution is very low or even zero.
- Theorem 2 (on the convergence of the Wasserstein distance (W2) on high dimensional hyperspheres) does not seem to hold if, for instance, P and P’ are empirical distributions with overlapping supports. Further, even when the above Theorem holds, the W2 distance may be relatively high since it is proportional to the square root of the number of samples.
- Moreover, why and how would Theorem 2 justify improved inference when projecting latent samples onto a hypersphere?
- Please consider revising the following statement in the introduction: “The encoder f in VAE approximates the posterior q(z|x)”. The encoder “f” in VAE parametrizes the variational posterior.
- Some typos,
	- Abstract, “… by sampling and inference tasks”  -- “on sampling …”
	- Introduction second paragraph after eq 2. “… it also causes the new problems” – “ … causes new problems”
	- Section 2.1, “For convenient analysis …” – “For a convenient …” 
	- Second paragraph after Theorem 1. “… perform probabilistic optimizations … ” – “… optimization …” 
	- Section 5.2, second paragraph. Is it Figure 9?

The main recommendations I would make are as follows.
- Consider revising the paper to improve its writing.
- Provide rigorous theoretical analysis and discussions to support the main claims. 
- Improve experiments by including more datasets and baselines (e.g., hyperspherical VAE [1]), as well conduct more targeted experiments to give more insights regarding the effect of the L2 normalization on inference and generation. 

[1] Davidson, Tim R., et al. "Hyperspherical variational auto-encoders." UAI, 2018.

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper proposed an interesting idea that by regularizing the structure of the latent space into a sphere, we can free VAE from variantional inference framework.

However, here are several concerns about this paper:
1. is this a variant of the Wasserstein auto-encoder?
2. the image quality of VAE (CelebA) is not that bad in other VAE papers, maybe tuning the \beta-VAE can also achieve the same quantitative and qualitative results.
3. Can you visualize the latent space (z) for the CelebA dataset, also comparing with the results from VAE?
review_title: Official Blind Review #1

Meta-review: This paper proposes to improve VAE/GAN by performing variational inference with a constraint that the latent variables lie on a sphere. The reviewers find some technical issues with the paper (R3's comment regarding theorem 3). They also found that the method is not motivated well, and the paper is not convincing. Based on this feedback, I recommend to reject the paper.
Decision: Reject