Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: 1. Define/explain data manifold: in the abstract, ``data moments" suggests the data is treated as random variables; the authors could explain more on how they make the connection between random variables and manifolds 
2. introduction: include an example of ``models having different representation space" 
3. introduction: elaborate more on ``unaligned data manifolds" 
4. introduction: the distinction between ``intrinsic" and ``extrinsic" can be made more explicit; as currently written, the difference between ``extrinsic"/``multi-scale"/``utilizing higher moments" isn't clear 
5. the authors mentioned the hypothesis that high-dimensional data lies on a low-dimensional manifold, but in (3.1) they only considered $\mathbb{R}^d$ without justifying this restriction 
6. (3.1) ``$u(t, \cdot) \to \delta(\cdot -x)$" change the dot to a variable 
7. (3.1) ``$u(t, x) = 0 \forall x \in \partial X$ and for all $t$" change to ``for all $t$ and $x \in \partial X$ 
8. equation 1, write ``we restrict our exposition to Euclidean spaces $X = \mathbb{R}^d$" and remove ``$\forall x, x' \in \mathbb{R}^d, t \in \mathbb{R}^+$" 
9.  (3.1) the phrase ``a compact X including $\mathbb{R}^d$" is confusing
    as $\mathbb{R}^d$ is not compact under Euclidean topology 
10.  (3.1) ``for a local domain with Dirichlet condition $D$" change to ``for a local domain $D$ with Dirichlet condition" 
11.  (3.1) ``Definition 1" is more of a proposition/theorem. 
12.  (3.1) notation: in the body paragraph the authors use $k(x, x', t)$ but in ``definition 1" they used $k(x, y, t)$ 
13. (3.1) ``$c$ disconnected components" to ``$c$ connected components" 
14. (3.1) the authors could elaborate more about how HKT contains all the information in the graph's spectrum. Currently, they only stated the limit equal to the number of connected components. 
15. Github code is helpful but it would be better if they can include an algorithm section
16. A figure of the architecture could be included under the ``putting IMD together" section 
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The paper propose a novel way to measure similarity between datasets, which e.g. is useful to determine if samples from a generative model resembles a test dataset. The approach is presented as an efficient numerical algorithm for working with diffusion on data manifolds.

I am very much in doubt about this paper as there are too many aspects of the work I do not (currently) understand. I suspect that the underlying idea is solid, but in its current form I cannot recommend publication of the paper.

Detailed questions and comments:

*) It seems that the focus is on GANs and related models that do not come with a likelihood. In my reading, the practical problem that the paper address is model comparison in models that do not have a likelihood. Is that correct? If so, I am left to wonder why models with a likelihood a not discussed, and why the such models aren't included in the experiments? Wouldn't the likelihood be a sensible baseline?

*) The term "multi-scale" is not defined until page 3. I'd recommend defining this term much more early or avoid its use until its defined.

*) I found the geometric exposition to be rather confusing. In Sec. 3.1 emphasis seem to initially be on Euclidean diffusion, and at some point emphasis changes to diffusion on graphs. Sometimes the paper seem focused on diffusion on general manifolds. I found this exposition to be rather chaotic, and I suspect that this is the root cause of me not understanding many aspects of the work.

*) In Sec. 3.2 the notion of diffusion *on* manifolds is discussed. At some point (not quite clear to me when) the results of this discussion is applied to diffusion *between* manifolds. I don't quite understand what it means to diffuse between manifolds. I would have appreciated a more explicit exposition here.

*) In the concluding remarks it is stated that IMD provides guarantees. Which guarantees are that?

review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: IMD is a lower bound to Gromov-Wasserstein distance. Which implies that when IMD is small, this does not guarantee that GW will be small. It is interesting how large can be that gap (and when typically the gap increases).
The term e{−2(t+t^−1)} in the definitions of GW and IMD, obviously imposes some "scale" for both manifolds M and N, so it is hard to call IMD multi-scale.

Experiments with language affinities are not convincing to me. How is armenian and hebrew are close, or hungarian and turkish (if "el" is greek, then greek, hungarian and turkish are close somehow?).

review_title: Official Blind Review #3

Meta-review: This paper introduces a way to measure dataset similarities. Reviewers all agree that this method is novel and interesting. A few questions initially raised by reviewers regarding models with and without likelihood, geometric exposition, and guarantees around GW, are promptly answered by authors, which raised the score to all weak accept. 

Decision: Accept (Poster)