Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper provides theoretical and experimental support to the following hypothesis: maps induced by denoising diffusion processes are optimal transport maps between the base and target distributions. For the case of multidimensional Gaussians, the paper provides a complete analytical result. For the general case, it shows this directly for synthetic experiments with mixtures of Gaussian and indirectly, through consequences such as texture preservation, for image data.
strength_and_weaknesses: ### Strengths
* Theoretical understanding of denoising diffusion processes is important to bring their impressive qualitative performance with images to other applications. As such, the characterization put forth by this paper can give a good starting point to understanding the nature of the resulting maps.
* The paper is written well, with good overview of the background material, a clear motivation, and directly appreciable results.

### Weaknesses
* The analytical result is very limited, as it pertains to mapping from Gaussians to Gaussians. At some level it is also unsurprising, considering that Gauss-Markov processes do have energy-optimality properties. This doesn’t mean that the result is trivial, but it begs for additional complementation with empirical results to support the hypothesis.
* There is a little bit of ambiguity when moving from the discrete processes to their corresponding continuous versions. For example, the difference equation (4) is a backward process, but the corresponding differential equation (6) appears to be a forward one. It is worth saying something about this to ease reading for those who are not immersed in this area.
* The indirect evidence based on texture similarity in the image data experiments is a very compelling approach. However, it is worth discussing why other reasonable explanations may not work. For example, if the neural network is regularized in all cases to be a Lipschitz map, then the texture similarity could be simply by virtue of that limited distortive ability instead of some global form of transport optimality (though, admittedly, these two concepts could be related.)
* The numerical results paragraph in the image data experiments (p.9) is unclear. To have the analogue of Section 3.4, I would have expected computing the cost of the Monge map for these images, but it doesn’t seem that’s what is done. A clarification here is necessary.

clarity,_quality,_novelty_and_reproducibility: The paper is written very clearly. The basis of the work, namely recasting diffusion models into continuous SDEs and ODEs and relating flows to transport, is not novel. However, placing everything into this coherent hypothesis and providing clear evidence in support of it is novel, to the best of my knowledge and it is done well. The results appear to be easily reproducible.
summary_of_the_review: The paper gives compelling evidence that links denoising diffusion processes to optimal transport maps from base to target distributions. This is done clearly and methodically. This may form the basis of a better understanding of the remarkable performance of this approach to generating natural images and of furthering its use in novel applications. Apart from a couple of clarifications, the paper seems worthwhile to share with the community.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work hypothesizes that the limit maps produced by   DDPMs  coincide with the optimal transport maps, or they are very close to each other with high similarity. The theoretical proofs are given for the cases of the normal source distributions, and experimental results are given for more general distributions, such as the mixtures of uniform distributions. The experiments on real image data sets show that the images sharing the same latent code obtained by diffusion have the similar high level texture features, postures and semantics. Also, the L2 distance seems not suboptimal for visual data. The work bridges the optimal transport theory and the diffusion model. 

The contributions are:

1. theoretically verify that for the case of multivariate normal distributions the Monge optimal transport map coincides with the DDPM encoder map.

2. Numerically verification of the similarity between the optimal transport maps and the DDPM encoder map by solving the Fokker-Planck equation on more general synthetic distributions.

3. Additional qualitative empirical evidence supporting the hypothesis on real image datasets are provided.
strength_and_weaknesses: The work has many strengths:

1. It raised an interesting hypothesis which bridges DDPM and optimal transport theory, the conjecture is sensible, elegant and inspirational.
2. Solid theoretical proofs are given for multivariate Gaussian distribution as the source input. Numerical experiments on synthetic data and empirical validations on real images are supporting.
3. The writing is clear and easy to follow, the mathematical formulations are rigorous and succinct.

The weaknesses are 
1. The theoretic proofs can be further improved, in appendix between equation (19) and (20), the authors claims the solution is exaction the optimal transport map, the detailed deduction for this claim is missing. I expected to see more details to show this solution satisfies the Brenier Monge-Ampere equation.

2. In the numerical experiment design, the authors simply used the POT library, which algorithm was applied ? Some algorithms are intrinsically approximation and will introduce approximation errors. This need to be carefully analyzed as well to ensure the precision of the experiments.

3. The hypothesis is kind of vague, the DDPM solution and the optimal transport map are close with small deviation. More accurate quantitively estimate will be more convincing. 
clarity,_quality,_novelty_and_reproducibility: The work is novel to discuss the intrinsic relation between the diffusion model, which has fundamental importance. The manuscript is very well written, the motivations, inspirations are clearly introduced, the major concepts, mathematical formulations, proofs are clean and easy to follow, the experimental design, and algorithmic descriptions are explained thoroughly. The numerical experiments are clearly analyzed. The work is easy to reproduce. 
summary_of_the_review: This work hypothesizes that the limit maps produced by   DDPMs  coincide with the optimal transport maps, or they are very close to each other with high similarity. The work theoretically verifies the hypothesis for the case of multivariate normal distributions the Monge optimal transport map coincides with the DDPM encoder map, gives the numerical verification of the similarity between the optimal transport maps and the DDPM encoder map by solving the Fokker-Planck equation on more general synthetic distributions, and offers additional qualitative empirical evidence supporting the hypothesis on real image datasets are provided.

The idea of bridging optimal transport theory and DDPM is inspiring and important. The mathematical proof is rigorous and elegant, the empirical experimental results are encouraging. The hypothesis can be further formulated to be more quantitively  precise. The mathematical proofs can be directly generalized to other well known distributions. More complicated synthesized distributions can be tested for the hypothesis.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper claims that diffusion-based models actually implement transportation plans between target and Gaussian distributions. It comes with a very pedagogical introduction to both DDPM and optimal transport, and then the rest of the paper is spent doing experiments to show how this claim is verified on many examples. 
strength_and_weaknesses: As a definite strength, the paper is very readable and a pleasure to read. Its claim is clearly stated, although I am not 100% sure it is actually novel. Still, the experiments are nice and quite convincing.

As a weakness, maybe I could say that the paper is a bit weak on the theory, which is not a no-go for me. What I feel could be done in a clearer way is to frame the paper as clearly stating an "hypothesis" or a "conjecture". This could help better understand its contribution, which I believe is pretty exciting and simple to grasp.
clarity,_quality,_novelty_and_reproducibility: The paper is very clear, its experiments are both very convincing and funny, and it looks like the code will be made reproducible. My guess is: this paper may have a nice impact.
summary_of_the_review: The paper is a nice read, but I think it is a bit borderline, although I like it.
I believe it could be improved in the following ways:
* more references about DDPM and OT and their connections. (For instance you could mention diffusion based OT systems based on Wasserstein flows). 
* This improved state of the art may be done at the expense of putting the details of the  "method" part in appendix.
* a much clearer (title + text) presentation of this paper as stating a conjecture or an hypothesis that DDPM actually construct transportation maps. (plans ?)
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work presents a hypothesis that the DDPM encoder map coincides with the optimal transport map for common distributions, and it is shown for Gaussians this is indeed the case. Empirical evaluations are done to support this hypothesis.

strength_and_weaknesses: ## Strengths:
* Finding meaningful ways to interpret popular diffusion models like DDPM is a meaningful research direction
* The paper presents a rigorous proof of the hypothesis for the Gaussian case
* Experiments were done to compute the exact error between transport costs for low-dimensional synthetic examples as well as qualitatively for image generation tasks.


## Weaknesses:
* I found the novelty of this paper lacking as it does not introduce a new method or significantly new theoretical results.
* For the Gaussian case, the proof becomes a lot simpler because all intermediate densities are also normal. And it looks like the authors further assume $a(0) = 0$ and I have not found a proof of a more general case in the supplementary materials contrary to what is mentioned in the appendix proof.
* In Section 3.4, when computing the error between the optimal transport and the one induced by solving the Fokker-Planck equation numerically, I think it might be more useful to compare the difference of transport maps (that is, $\sum_i ||T(x_i)-T^*(x_i)||^2$ where $T$ is the ODE induced map, and $T^*$ is the optimal transport map). It is possible that the error is very small but the transport maps can still be different.
* I found the rationale of Section 3.5 a bit questionable, namely we can invert the encoder and just compare the inverted latent codes visually. To me, such evaluation only makes sense if the transport map is just translated by a constant vector. Have the authors tried with different random seeds (and hence different randomization of DDPM networks) when training models for a different dataset? Will the results still be consistent?
* It was very briefly mentioned that recently Lavenant & Santambrogio found a counterexample to the central hypothesis. In my opinion, the paper needs to do a lot more in explaining whether such a counterexample is a singular case or not, other than mentioning the two maps disagree at a single point (how can two maps disagree at only one point if both maps are continuous enough?) and somehow "finite number of samples in higher dimensions" this is not a problem.
* The numerical solver for the Fokker-Planck equation uses a low-rank TT format which I think might be foreign to the machine-learning community and deserves more explanations, at least in the appendix. For instance, the authors ought to elaborate on "the standard second-order operator splitting technique" that involves some exponentiation in time-stepping that I do not follow.

clarity,_quality,_novelty_and_reproducibility: The clarity of the paper is very good, and I saw the source code in the supplementary.

summary_of_the_review: Overall I think the paper lacks novelty and there are some evaluation choices that could be improved. Hence I'm leaning toward rejection.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: 
This paper analyzes one bridge between optimal transport and diffusion model.
Its main claim is that diffusion-based models implement transportation plans between target and Gaussian distributions.
While some theoretical support is provided (for some specific situations), the paper supports the
claim throung experimental analysis.

Most reviewers concur on the fact that the paper provides some significant contributions and can be accepted
for publication. However, I urge the authors to improve the final version based on the reviewers ( eg qZHA and R2Hy)
comments.



justification_for_why_not_higher_score: The theoretical results are a bit limited as it applies only on Gaussian measures. While it  is a more difficult problem, extending those results to other measures (eg mixture of Gaussians) would have make the  theoretical part far stronger.

justification_for_why_not_lower_score: The paper provides useful and novel insights on diffusion model and their codes.
Decision: Accept: poster