Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper proposes a novel training method to solve Schrödinger bridge (SB) problems by performing (approximate) iterative proportional fittings (IPFs). 
- SB problems is to find a diffusion process defined in a finite time range $t \\in [0, T]$ for given marginals at the two ends in time $t = 0, t=T$. Specifically, the paper focuses on cases when diffusion processes have state-independent and time-homogeneous diffusion terms. Note that SB problems can have variant conditions; for example, one gives a reference diffusion instead of the two marginals. 
- Classical IPFs solve SB problems when a reference diffusion is provided instead of the marginals at two time ends. IPFs iteratively solve optimization problems. With the given initial distribution at one end, each optimization problem finds a diffusion in one time direction by minimizing the reverse KL between it and a target. Here, the target is the previous step's solution (a diffusion runs reversed in time). Each IPF step alternates the time direction by setting initial distributions correspondingly. The first target is set to the reference distribution of the SB problem. For some limited cases, it has been proven that IPFs solve SB problems by repeating the optimizations.

In order to motivate the proposed method, the paper first shows that for discrete-time IPFs, the optimal solution $q_{k|k+1}^{n}$ of $n$-th IPF iteration (of one time direction) is the posterior $p_{k|k+1}^{n}$ of the previous step's solution $p_{k+1|k}^{n}$: the result also holds for reversed direction. Instead of minimizing the reverse KL on the entire path in time, the authors emphasize that one can minimize per-time-step loss between a current estimate and the previous optimal.

The paper then shows that there are two properties for discrete-time IPFs, when diffusion terms are state-independent and time-homogeneous diffusions. First, the optimal solution $q_{k|k+1}^{n}$ per each time step must follows the well-known reserve diffusion formula, which includes the target's $k$+$1$-th marginal score function $\nabla \log p_{k+1}^{n}$ (Song et al, 2021). Next, instead of using score functions, the optimal $q_{k|k+1}^{n}$'s mean can be written as the conditional expected value of $X_{k+1} + F_{k}^n(X_k) - F_{k}^n(X_{k+1})$ under $p_{k+1|k}^{n}$, where $F_k^n(\cdot)$ is the mean function of $p_{k+1|k}^{n}$. Intuitively, this can be understood as representing marginal score as conditional expectation of finite difference.

Finally, acknowledging these two properties, the paper proposes a novel (approximate) IPF method named Diffusion Schrödinger Bridge (DSB). Assume we have marginals at the two ends in time $t = 0, t=T$. Define two discrete-time Markov chain models: one runs in one time direction, and the other runs in reverse-time, combined with their corresponding initial distributions. For each $n$-th DSB step, by fixing one chain, e.g. $p_{k+1|k}^{n}$, the other chain $q_{k|k+1}^{n}$ learns their optimal mean by minimizing mean-squared loss, whose target is the conditional expected value of $X_{k+1} + F_{k}^n(X_k) - F_{k}^n(X_{k+1})$ under $p_{k+1|k}^{n}$. By alternating time directions of these mean matchings, both Markov chains learn to map from one distribution at one end to another. While the proposed method indirectly solves one IPF step, the authors demonstrate that the proposed methods converge in practice. The authors also emphasize that the proposed method works as long as sampling from the marginal distributions at two time-ends is available.

Furthermore, the paper improves theoretical analysis in previous studies on IPFs to solve SB problems and score matching. In particular, the paper shows the convergence of both the discrete-time and continuous-time IPFs under more general assumptions. Another example includes the convergence of learning continuous-time diffusion-based generative models via score matchings for state-independent and time-homogeneous diffusion.

In the experiments, the paper demonstrates that the proposed DSBs can be used in generative modeling and modeling interpolations between two distributions.
main_review: (Originality & Significance)  
In my understanding, the paper's contributions are clear, and I also consider that the results are essential for several reasons:  
First, it introduces a novel learning model to attack SB problems, learning diffusions between two distributions, which is previously known to be challenging for high-dimensional settings.  

Second, the proposed method can be applied to various application cases, including generative models and learning random processes that transform one distribution to another. The proposed method can also be understood as one type of generalization of now-popular diffusion-based generative models. Moreover, the proposed method can be extended to more general cases beyond state-independent and time-homogeneous diffusions.  

Third, the paper provides thorough theoretical discussions to motivate the proposed method.  

Finally, the paper demonstrates the effectiveness of the proposed method by a series of experiments.  


(Quality & Clarity)  
In general, the paper has a well-organized structure to motivate the proposed method and provide thorough discussions on relevant topics. However, the following aspects of the paper can be improved.

First, discussions on the limitations of the proposed method can be improved. I found that the main text only emphasizes the pros of the proposed method without discussing any of its critical drawbacks. For example, the paper only states that the proposed method improves generation qualities for a given number of discretization steps compared to previous diffusion-based models. However, the main text doesn't correctly mention that the improvement is achieved by increasing the training time. Moreover, I can only find many non-trivial implementation tricks of DSB at the very end of the supplementary materials. Without properly mentioning such characteristics of the proposed method, the main text sounds misleading to me.

Second, the connection between Theorem 1 and the proposed method can be improved. I initially thought that Theorem 1 is introduced to prove the convergence of the approximate IPS (= DSB) (maybe I'm wrong). However, I couldn't find if Theorem 1 is used to discuss the convergence of the proposed method.

(Typos)  
- Lines 609-610, $p_{k|k+1}$ → $p^n_{k|k+1}$.
limitations_and_societal_impact: (Limitations)  
I strongly believe that the paper is excellent. However, I'm concerned that the limitations of the proposed method are understated. Improving such discussion will enormously improve the quality of the paper.

(Societal Impact)
N/A
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: Leveraging recent advance in score-based generative modeling with SDEs, this paper proposes to solve high dimensional Schrodinger bridge problems by iteratively training score-based models. The method is the first to effectively solve Schrodinger bridge problems in high dimensional spaces, with demonstrations on distributions of real-world images, and can be viewed as a generalization to the previous SDE approach for score-based generative modeling.
main_review: Score-based generative models/ diffusion models have attracted increasing attention for theorists and practitioners in machine learning. This work shows that the basic ideas of them can be generalized to solve high dimensional Schrodinger bridge problems, all while enabling new capabilities and applications. Here are several important advantages of this submission:

1. Clear writing. Although fully understand everything in this submission definitely requires some background on stochastic analysis, authors have done a good job on conveying the intuition and basic concepts, which is not an easy task. The discussion on discrete cases makes it much easier for readers to intuitively comprehend the continuous case (Schrodinger bridges are defined as a continuous case) without much background.

2. Solid theoretical analysis. Theorem 1 bounds the deviation of distributions with imperfect score estimations, and provides one of the first proofs of the convergence of SGMs. Section 3.4 proves the convergence of iterative proportional fitting in non-compact settings. Both are very solid contributions.

3. The proposed framework generalizes score-based generative modeling with SDEs. The prior distribution is no longer required to be tractable, as long as samples can be obtained. The training procedure of prior work can be viewed as one step of iterative proportional fitting. As a result, the proposed method can be leveraged to fine-tune existing score-based models for better matching the prior, and allows additional applications like dataset interpolation.

Below are some drawbacks and questions for the authors:

1. Shorter time intervals doesn't necessarily mean smaller number of iterations for sampling. The time scales of SDEs can be rescaled arbitrarily, so we can have arbitrarily small time horizon without having any practical difference. From this perspective, I'm not convinced that Schrodinger bridges can speed up sampling of score-based models in the general case.

2. In section 3.3, authors claim that using score matching to approximate $\\{ \nabla \log p_{k+1}^i (x) \\}_{i=0}^n$ is prohibitively costly. However, if I understand it correctly, the mean-matching method proposed in Proposition 3 is **exactly** the same as using denoising score matching to estimate the score functions individually. 

3. Experiments are relatively weak as results are not competitive with previous score-based models on SDEs, and image resolutions are small.
limitations_and_societal_impact: No obvious concerns.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors present a generative model based on the Iterative Proportional Fitting of the Schrödinger Bridge problem. They provide convergence analysis of their proposed algorithm, and show the performance of their algorithm with image generation and datasets interpolation.
main_review: # Novelty: 
This paper introduces a novel score-based generative model based on the Iterative Proportional Fitting of the Schrödinger Bridge problem. The proposed algorithm can be used as optimal transport between any distributions, which is also new, compared to existing score-based models. 

# Major concern:
The originality claimed in the paper is overestimated, and related works are not properly cited. In line 314, the authors said they have provided the first convergence result for SGM methods. As far as I know, the convergence of SGM has been studied by Block et al. 2020 and Wang et al. 2021. Also in line 283, the authors claim that their method DSB “is the first practical algorithm for approximating the solution to the SB problem in high dimension”. They should clearly discuss the connection and difference between their proposal and the method in Wang et al. 2021 which also utilizes SB for generative modeling. 

Overall, the paper is well-written and theoretically sound, and the authors show the image generation and interpolation results on image datasets and the reproducibility is good. This work has worthy contribution to the SGM methods.

Reference:
Block A, Mroueh Y, Rakhlin A. Generative modeling with denoising auto-encoders and Langevin sampling[J]. arXiv preprint arXiv:2002.00107, 2020.
Wang G, Jiao Y, Xu Q, et al. Deep Generative Learning via Schr\"{o} dinger Bridge[J]. In ICML 2021, arXiv preprint arXiv:2106.10410, 2021.


limitations_and_societal_impact: No
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper introduces a new method to iteratively refine a diffusion-based generative model, that is motivated by the so-called Schrödinger-bridge. The method amounts to iteratively train a forward and a backward model, in a manner that is simpler than previous work.
main_review: This paper is definitely a great read, although it is a bit hard to follow at some crucial parts. The contribution is very interesting and will definitely prove very inspiring to many researchers, I recommend acceptance.

Here are my comments on the fly:


* Figure 1 is not really informative and should be extended to better represent what happens at each step
* I guess approximation (4) is classical ? It would be better to write "The following approximation is usually made" instead of using "we", that would make it clearer it's not a contribution
* is the other formulation for \theta^\star that uses all the \log p_{k|k-1}(X_k|X_{k-1}) instead of p_{k|0} equivalent and also principled theoretically, or is it some kind of approximation ?
* I think that your short discussion after th.1 is very interesting, but it's hard for me to understand why the second term for the bound is related to the increase of \alpha. Could you please be a bit more precise on that point ?

* I don't think your notations on the "static Schrödinger bridge problem" are easy to follow. What is \mu_{0,N} exactly ? Is it the restriction of \mu on 0 and N ? I guess that's the case, but the notation is a bit confusing. Maybe writing something like \mu_{0:N}=\mu_{0,N}\mu_{1:N-1\mid0,N} or something in this vein would be a bit clearer

* As it is worded, I don't understand whether the result from Mikami holds only in the case of the gaussian transition probability P_{k+1\mid k}.

* From the point at which you introduced the SB problem, I think you should provide some wordy explanation regarding what you have and what you want to achieve. If I understand correctly, the objective is to progressively refine the transportation through iteratively apply this static SB problem ? There could be a figure about this giving the big picture, or something of the like. I must tell that the paper starts to be a bit hard to follow here.

* The proposition 2 appears as very nice. However, I would once again gladly use a support figure to make the ideas clearer, that could very advantageously replace your current figure 1 that is not informative at all.

* Proposition 3 seems like the central part of the paper, that holds the core contribution. However, I don't manage to understand where equations (12)-(13) come from, they are not sufficiently explained in my opinion, especially considering the space you take to explain other less important things. Please take the time to better explain the reason for these.

* regarding figure 1 and proposition 3: it would be way more effective to have a figure where your actual networks appear (the B(k,x) and F(k,x)).

* Is it important to train \beta^0 using previous SGM methods, or would you propose an alternative easier way ?

* Reading the text, it looks like we'll have to store all the N networks, but actually it looks like only the last one may be stored, so that we actually discard the other ones on the way, right ?






limitations_and_societal_impact: I don't think there are any particular societal impact for this work that would require special treatment.

Regarding the limitations, it is true that it could be interesting to have their opinion on what are actually the main weaknesses of their method to know which parts could benefit from further research.
review_title: null

Meta-review: All reviewers recommended accepting this paper, and even if there were a number of concerns, the consensus among the reviewers was clear. This was an enjoyable paper to read and I recommend that you address the issues raised by the reviewers in the camera-ready, and especially pay attention to the comments regarding improving clarity and facilitating understanding.
Decision: Accept (Spotlight)