Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: In the paper the authors analyse the latent space generated by the variational autoencoder (VAE). They show that this latent space is imbued by a Riemannian metric and that this metric can be easily computed in terms of mean and variance functions of the corresponding VAE. They also argue that the current variance estimates are poor in regions without data and propose a meaningful variance function instead. In the experiments section the authors evaluate the quality and meaningfulness of the induced Riemannian metric.

There are minor grammatical errors and the paper would benefit from proofreading.

In the introduction the authors argue that points from different classes being close to each other is a misinterpretation of the latent space. An argument against would be that such a visualisation is simply bad. A better visualisation would explain the data structure without the need for an additional visualisation of the metric of the latent space.

In section 2, the multiplication symbol (circle with dot inside) is not defined.

It is not clear from the paper what the purpose of eq. 7 is, as well as most of the section 3. Only in appendix C, it is mentioned that eq. 7 is solved numerically to compute Riemannian distances, though it is still not clear how exactly this is achieved. I think this point should be emphasized and clarified in section 3.

In section 4, it says proof for theorem 1 is in appendix B. Appendix B says it proves theorem 2. Unfortunately, it is not clear how good the approximation in eq. 9 is.

Is theorem 1 an original result by the authors? Please emphasize.

In Fig. 6, why was 7-NN used, instead of k-means, to colour the background?

I think that the result from the theorem 1 is very important, since the estimation of the Riemannian metric is usually very slow. In this regard, it would be very interesting to know what the total computational complexity of the proposed approach is.

review_title: good paper

final_score: rating: 3: Clear rejection
review_content: The paper investigates the geometry of deep generative models. In particular, it describes the geometry of the latent space when giving it the (stochastic) Riemannian geometry inherited from the embedding in the input space described by the generator function. The authors describe the geometric setting, how distances in the latent space can be interpreted with the non-Euclidean geometry, and how interpolation, probability distributions and random walks can be constructed.

While the paper makes a decent presentation of the geometry of the generative setting, it is not novel. It is well known that (under certain conditions) the mapping described by the generator function is a submanifold of the input space. The latent space geometry is nothing but the submanifold geometry the image f(Z) inherits from the Euclidean geometry of X. Here f is the generator mapping f:Z->X. The latent space distances and geodesics corresponds to distances and geodesics on the submanifold f(Z) of X. Except for f being stochastic, the geometry is completely standard. It is not surprising that distances inherited from X are natural since they correspond to the Euclidean length of minimal curves in the input space (and thus the data representation) when restricting to f(Z).

I cannot identify a clear contribution or novelty in the paper which is the basis for my recommendation of rejection of the paper.
review_title: Decent outline of standard Riemannian geometry in generative models but no novelty

final_score: rating: 7: Good paper, accept
review_content: The paper makes an important observation: the generating function of a generative model (deep or not) induces a (stochastic) Riemannian metric tensor on the latent space. This metric might be the correct way to measure distances in the latent space, as opposed to the Euclidean distance.

While this seems obvious, I had actually always thought of the latent space as "unfolding" the data manifold as it exists in the output space. The authors propose a different view which is intriguing; however, they do not, to the best of my understand, give a definitive theoretical reason why the induced Riemannian metric is the correct choice over the Euclidean metric.

The paper correctly identifies an important problem with the way most deep generative models evaluate variance. However the solution proposed seems ad-hoc and not particularly related to the other parts of the paper. While the proposed variance estimation (using RBF networks) might work in some cases, I would love to see (perhaps in future work) a much more rigorous treatment of the subject.

Pros:
1. Interesting observation and mathematical development of a Riemannian metric on the latent space.

2. Good observation about the different roles of the mean and the variance in determining the geodesics: they tend to avoid areas of high variance.

3. Intriguing experiments and a good effort at visualizing and explaining them. I especially appreciate the interpolation and random walk experiments. These are hard to evaluate objectively, but the results to hint at the phenomena the authors describe when comparing Euclidean to Riemannian metrics in the latent space.

Cons:
1. The part of the paper proposing new variance estimators is ad-hoc and is not experimented with rigorously, comparing it to other methods in terms of calibration for example. 

Specific comments:
1. To the best of my understanding eq. (2) does not imply that the natural distance in Z is locally adaptive. I think of eq (2) as *defining* a type of distance on Z, that may or may not be natural. One could equally argue that the Euclidean distance on z is natural, and that this distance is then pushed forward by f to some induced distance over X. 

2. In the definition of paths \gamma, shouldn't they be parametrized by arc-length (also known as unit-speed)? How should we think of the curve \gamma(t^2) for example?

3. In Theorem 2, is the term "input dimension" appropriate? Perhaps "data dimension" is better?

4. I did not fully understand the role of the LAND model. Is this a model fit AFTER fitting the generative model, and is used to cluster Z like a GMM ? I would appreciate a clarification about the context of this model.
review_title: Interesting view on the induced Riemannian geometry of latent space in deep generative models. Several important applications are explored, but not in depth

Meta-review: This paper characterizes the induced geometry of the latent space of deep generative models. The motivation is established well, such that the paper convincingly discusses the usefulness derived from these insights. For example, the results uncover issues with the currently used methods for variance estimation in deep generative models. The technique invoked to mitigate this issue does feel somehow ad-hoc, but at least it is well motivated.

One of the reviewers correctly pointed out that there is limited novelty in the theoretical/methodological aspect. However, I agree with the authors’ rebuttal in that characterizing geometries on stochastic manifolds is much less studied and demonstrated, especially in the deep learning community. Therefore, I believe that this paper will be found useful by readers of the ICLR community, and will stimulate future research. 
Decision: Accept (Poster)