Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors consider a problem of modeling distribution evolution given samples for different discrete time moments.  

The paper consists of several parts.

First, the authors describe some concepts on how to construct an autoencoder that preserves distances over a Riemannian manifold on which the data is located. They define the diffusion ground distance on the manifold. And then describe an approximation of the diffusion geodesic distance. Based on this approximation, we can estimate the diffusion ground distance and train an autoencoder such that the L2 distance in the latent space approximates these estimates.

Second, the authors introduce a definition of OT based on a time-evolving vector field and on a diffusion formulation (see (3) and (4)). In Th. 3 they repeat a theorem from [35] that OT between the measures mu and nu can be represented as an optimization w.r.t. the vector field of an integral term plus some regularization.

Third, the authors model the vector field with a neural ODE and train the parameters of a neural ODE so that between each consecutive step the neural ODE solves the OT problem, formulated based on the diffusion formulation.

Finally, the authors applied the proposed algorithm to model some trajectories of cells.
strengths_and_weaknesses: Strengths
- The topic of the paper is about modeling stochastic dynamics, which is important

Weaknesses
- The text of the paper is of low quality and clarity, and contains errors and misprints, see comments below.
- The paper is a combination of several moving pieces, and it is unclear why exactly such choices were made.
questions: - 18: “However, most have these works have focused on generative modeling” - bad wording

- TrajectoryNet, and other papers, mentioned in the introduction, are not the only approaches to model stochastic flows. For example, what about the approaches from papers
1) Large-Scale Wasserstein Gradient Flows
Petr Mokrov, Alexander Korotin, Lingxiao Li, Aude Genevay, Justin Solomon, Evgeny Burnaev
Neurips, 2021
2) Charlie Frogner and Tomaso Poggio. Approximate inference with Wasserstein gradient flows. AISTATS, 2020

- The authors use a geodesic autoencoder. Any particular motivation why for using such an autoencoder? Why not some standard autoencoder? Any comparison between using the geodesic autoencoder and the standard one? How does this choice influences the experimental results?

- 66: “an initial state [15, 38] in can be combined to infer the cell lineage…” - in can be? Looks like a misprint.

- 91: Here, d1 and d2 are distances. Earlier, d was used as a dimension. Moreover, again in 104 d(x,y) denotes a distance. I propose to use some consistent notations. 

- Why consider a Riemannian manifold? What if the manifold is not a Riemannian one? What is the motivation? How does this choice influences the experimental results?

- The authors use the heat kernel and estimate the diffusion ground distance using the Gaussian kernel function. What if we use some other kernel function? Will the choice significantly influence the accuracy of the approximation and final results? How does this choice influences the experimental results?

- How does a value of alpha influence the experimental results?

- It is unclear to what extent the diffusion geodesic distance approximates anything accurately. Any comments on the accuracy of approximation in Th. 2? Also, the approximation here is stochastic. So the formulation should be either w.r.t. probability or a.s. Any comments about this?

- The geodesic distance is introduced in 139, but there is no exact definition.

- 153: mu is an actual distribution on M. Is it connected somehow with mu in (3)?

- How does the choice of beta in 153 influence the experimental results?

- Why did the authors use Neural ODE? Why not standard RNN? Any rationale behind the choice?

- Fig. 1: “With the combination of noise, a neural network learns to predict the derivative x′(t0), and an ODE Solver produces the predicted x(t1).” - x(t) is a stochastic process. Which derivative do the authors mean? Also, nothing was told about using the derivatives w.r.t the trajectories in the paper's main text.

- Fig. 1: “Geodesic Autoencoder learns the latent space of representation of data is learnt such that is preserves a diffusion geodesic distance. ” - is this crucial for the considered applications?

- 182: “With a slight abuse of notation, a similar distance between distributions (rather than points) is constructed in [37] where it was applied to approximate the Earth Mover’s Distance with a geodesic ground cost.” What is the novelty of this approach?

- 186: instead of = we should place \approx here.

- 189: “we use it to compute distances between predicted and ground truth observations.” Which ground truth observations?

- 191: “Empirically, we observed that G fails to capture meaningful distances for a large number
 of observations. ” - if it fails, then what is the point of using it?

- 202: “to understand how specific genes evolved over time, by decoding in the gene space.” - I do not understand the paper's main topic then. Do the authors propose a new method? Or is this about the modeling of genes? or what?

- (6): W_d -> W_2

- Th. 3. Which D should we consider? Is any D OK?

- 224: too many indexes i are used in the same formula. I guess a summation should be done w.r.t. to some other index.

- (7): How do the authors differentiate L_m w.r.t. parameters of the Neural ODE?

- 238: T -> $T$

- 260: “trajectories on the manifold”. Why are they located on any manifold? How should the reader analyze Fig. 2 and conclude that the results for the proposed method are better? What is the ground truth? Any accuracy characteristics to compare methods?

- Table 1: Are the numbers in the table big or small? We should compare this with some simple baseline approaches. Possibly all methods provide some bad results?

- 293: “[39], Other …” -> “[39]. Other …”

- Appendix, 519: “Replacing the continuity equation by the path dynamic, we have…” Why can we make such a replacement? Any justification that their proof is valid for any dissimilarity?

- Appendix, 520-521: Here, the authors wrote that the autoencoder approximates the geodesic distance for sample points. First, this is not completely true, as in Th. 2 we also have a parameter alpha. Also, if these quantities are equal for sample points, it does not follow that for any pair of points from some distribution, the autoencoder will have good generalization ability so that inequalities in 522 are valid.
limitations: The potential negative societal impact of the work is not presented in the paper, although the authors did not explicitly state this.

Some limitations of the paper are addressed.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a geodesic auto encoder to map points on a manifold to latent space whilst preserving geodesic distance on the data manifold in Euclidean distance in the latent space.

The geodesic auto encoder is then used to simulate trajectories in latent space, using diffusions and Neural ODEs. One may decode back to the data space. The trajectories are trained through an OT loss to samples. 

The authors show a number of theoretical results to justify the preservation of geodesic loss in latent space, and how Wasserstein in latent space with L2 ground cost is equivalent to Wasserstein on the decoded space with geodesic ground cost.

Experiments are carried out on biological data to justify the approach, and method seem to work well and results are convincing.
strengths_and_weaknesses: Strengths
- Training a geodesic auto encoder appears novel, has a lot of applications, and allows standard machine learning tools in Euclidean space to be applied to data living on a manifold. The method presented is simple and easy to understand.
- The geodesic auto encoder + trajectory with Neural ODE/ SDE approach presented seems quite flexible and able to be modified to intermediate distributions, unlike the Schordinger bridge, which has practical importance
- The method seem to work well and results are convincing
- Interesting theoretical results connecting intermediate trajectories to OT

Weaknesses
- It is not clear how well the decoder generates data close to the manifold in question. Comparison to manifolds with analytic form such as a sphere/ hypersphere may be helpful to compare to a ground truth.
- [1] appears highly relevant and should perhaps be used as a baseline, especially given the similarities using W2 loss to map cross sectional samples of a time evolving population
- The diagrams are not clear to me. The coloured points are labelled "samples", are these samples from the training data or the model? How come Figure 2a and Figure2c have grey lines but Figure 2b does not? The legend states ground truth is labelled by a cross and the predicted by a filled in dot, then how come the ground truth varies so much by method used to interpolate?
- The link to code does not work: https://anonymous.4open.science/r/bananarama-FADD/README.md
- It is not clear why the geodesic auto encoder does not work well for a large N, as mentioned in line 191
- A written algorithm would be helpful to understand the training procedure

- Experiments
    - It appears prediction at time 3,4 in the Appendices Table 2 for leave one out prediction are quite different to the results in Table 1 in the main text. It may give a clearer, more balanced view to include all the results. Also why was W1 chosen over W2?
    - Number of time steps is relatively small
  

Minor:
- "sinkhorn" is a name and should be capitalised in line 79 
- line 310, should be "compared"

[1] Proximal Optimal Transport Modeling of Population Dynamics, Bunne, 2021
questions: - In Table 1, what does runtime refer to? Is it train time or runtime at deployment?
- DSB interpolates via reverse diffusions and approximates the OT cost between initial and terminal distributions (no intermediary distributions) in Euclidean space where the ground cost is determined by the initial forward diffusion (reference diffusion). Which reference diffusion is used here?
- It is stated in the appendices that DSB is used with "default" parameters, what are these default parameters? What network was used? How many diffusion steps?
- The diagrams are not clear to me. Are the colours points labelled "samples" samples from the training data or the model? How come Figure 2 a and c have grey lines but Figure 2b does not? The legend states ground truth is labelled by a cross and the predicted by a filled in dot, then how come the ground truth varies so much by the method used to interpolate?
- What layer widths were used for the Petal and Dyngen experiments and what was the latent dimension?
limitations: The authors acknowledge training ODE may be challenging in some settings and that they use <10 time steps.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes a computationally efficient dynamic optimal transport framework to track samples overtime.


 
strengths_and_weaknesses: Overall the paper is well written.

Some elements are unclear however
e.g.
- Fig 1 caption "Geodesic Autoencoder learns the latent space of representation of data is learnt such that is preserves a diffusion geodesic distance."  (?)

- the notation "d" is used for dimension and also for distances.

- some element of section 3 are well known and may not need to be written (e.g. wasserstein distance equation), 

- section 3.3 is unclear. 

- the connection between section 3  and section 4 is not clear.
 



questions: - in Theorem 1: what is $d^{2\alpha}_{\mathcal{M}}$ ? 
- what is the meaning of the sentence preceding Theorem 1 mentioning "Thm.2" ?


limitations: The limitations are well addressed. The proposed method works best with uniform sampling in the time dimension and also consider balanced optimal transport (equal mass of density functions whatever time t).
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a new architecture, called Manifold Interpolating Optimal-Transport Flow (MIOFlow), to learn the dynamics of snapshot data like single-cell data. It consists of two major parts: 1. An autoencoder that embeds the data into a latent space, conserving a geodesic distance. 2. The MIOFlow that is based on neural ordinary differential equations (nODE), which is trained to model the stochastic trajectories of snapshot data. To this end the embedding of the autoencoder is combined with noise as the input to the dynamics function of the nODE.  
They evaluate their model on artificial data as well as cell real word single-cell data, showing the advantages of their method. 

strengths_and_weaknesses: Strengths:
- To my knowledge, the MIOFlow architecture is a novel idea. 
- The manuscript is clearly and well written.
- They compare MIOFlow with other models and show superior performance on several datasets including single-cell data. 

Weaknesses:
- Most results are qualitative rather than quantitative
questions: Minor comment: Some sentences have too many/wrong words. Examples include:
- line 18
- Caption of Figure one
limitations: Yes
review_title: null

Meta-review: metareview: Even though several concerns have been raised by multiple reviewers, the reviewers largely agreed on the significance and novelty of the contributions. The authors provided quite detailed responses and given all the data, overall I believe the strengths of the paper outweigh its weaknesses. Hence I am recommending an acceptance. 
confidence: Certain
Decision: Accept