SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Hence, for the IMF sequence  $(\mathbb{P}^n)_{n\in \mathbb{N}}$ , the Markov path measures  $(\mathbb{P}^{2n + 1})_{n\in \mathbb{N}}$  are getting closer to the reciprocal class, while the reciprocal path measures  $(\mathbb{P}^{2n + 2})_{n\in \mathbb{N}}$  are getting closer to the set of Markov measures. Proposition 7 should be compared with (Ruschendorf, 1995, Proposition 2.1, Equation (2.16)) which shows that, for the IPF sequence  $(\tilde{\mathbb{P}}^n)_{n\in \mathbb{N}}$ , we have  $\lim_{n\to +\infty}\mathrm{KL}(\tilde{\mathbb{P}}^{n + 1}|\tilde{\mathbb{P}}^n) = 0$ . This result is similar to Proposition 7 but for the forward Kullback-Leibler divergence.
Using Proposition 7, we finally prove the convergence of the IMF sequence  $(\mathbb{P}^n)_{n\in \mathbb{N}}$  to the Schrödinger Bridge. This result was first shown in the concurrent work (Peluchetti, 2023, Theorem 2). We present a simpler proof in Appendix C.6.
Theorem 8. Under mild assumptions, the IMF sequence  $(\mathbb{P}^n)_{n\in \mathbb{N}}$  admits a unique fixed point  $\mathbb{P}^{\star} = \mathbb{P}^{\mathrm{SB}}$ , and  $\lim_{n\to +\infty}\mathrm{KL}(\mathbb{P}^n|\mathbb{P}^{\star}) = 0$ .
4 Diffusion Schrödinger Bridge Matching
In this section, we present Diffusion Schrödinger Bridge Matching (DSBM), a practical algorithm for solving the SB problem obtained by combining the IMF procedure with Bridge Matching.
Iterative Markovian Fitting in practice. IMF alternatively projects on the Markov class  $\mathcal{M}$  and the reciprocal class  $\mathcal{R}(\mathbb{Q})$ . We denote  $\mathbb{M}^{n + 1} = \mathbb{P}^{2n + 1}\in \mathcal{M}$  and  $\Pi^n = \mathbb{P}^{2n}\in \mathcal{R}(\mathbb{Q})$ . Assuming we know how to sample from the bridge  $\mathbb{Q}_{|0,T}$  given the initial and terminal conditions, sampling from the reciprocal projection  $\mathrm{proj}_{\mathcal{R}(\mathbb{Q})}(\mathbb{M})$  is simple: First, sample  $(\mathbf{X}_0,\mathbf{X}_T)$  from the joint distribution  $\mathbb{M}_{0,T}$ . Then, sample from the bridge  $\mathbb{Q}_{|0,T}(\cdot |\mathbf{X}_0,\mathbf{X}_T)$ . The bottleneck of IMF is in the computation of Markovian projections. By Definition 1,  $\mathbb{M}^{\star} = \mathrm{proj}_{\mathcal{M}}(\Pi)$  is associated with the process
$$
\mathrm {d} \mathbf {X} _ {t} = \left\{f _ {t} (\mathbf {X} _ {t}) + \sigma_ {t} ^ {2} \mathbb {E} _ {\Pi_ {T | t}} [ \nabla \log \mathbb {Q} _ {T | t} (\mathbf {X} _ {T} | \mathbf {X} _ {t}) \mid \mathbf {X} _ {t} ] \right\} \mathrm {d} t + \sigma_ {t} \mathrm {d} \mathbf {B} _ {t}, \quad \mathbf {X} _ {0} \sim \pi_ {0}.
$$
By Proposition 2, we can learn  $\mathbb{M}^{\star}$  using  $\mathbb{M}^{\theta^*}$  given by
$$
\mathrm {d} \mathbf {X} _ {t} = \left\{f _ {t} \left(\mathbf {X} _ {t}\right) + v _ {\theta^ {*}} (t, \mathbf {X} _ {t}) \right\} \mathrm {d} t + \sigma_ {t} \mathrm {d} \mathbf {B} _ {t}, \quad \mathbf {X} _ {0} \sim \pi_ {0}, \tag {9}
$$
$$
\theta^ {\star} = \operatorname {a r g m i n} _ {\theta} \left\{\int_ {0} ^ {T} \mathbb {E} _ {\Pi_ {t, T}} \left[ \left\| \sigma_ {t} ^ {2} \nabla \log \mathbb {Q} _ {T | t} \left(\mathbf {X} _ {T} \mid \mathbf {X} _ {t}\right) - v _ {\theta} (t, \mathbf {X} _ {t}) \right\| ^ {2} \right] / \sigma_ {t} ^ {2} \mathrm {d} t: \theta \in \Theta \right\}, \tag {10}
$$
In practice, we sample the SDE associated with  $\mathbb{M}$  and save a batch of joint samples  $(\mathbf{X}_0,\mathbf{X}_T)$ . This is similar to the trajectory caching procedure in De Bortoli et al. (2021), but we only retain initial and final samples.
where  $\{v_{\theta} : \theta \in \Theta\}$  is a parametric family of functions, usually given by a neural network. The optimal  $v_{\theta^{\star}}(t,x_{t}) = \sigma_{t}^{2}\mathbb{E}_{\Pi_{T|t}}[\nabla \log \mathbb{Q}_{T|t}(\mathbf{X}_{T}|\mathbf{X}_{t})\mid \mathbf{X}_{t} = x_{t}]$  for any  $t \in [0,T]$  and  $x_{t} \in \mathbb{R}^{d}$ .
With the above two procedures for computing  $\mathrm{proj}_{\mathcal{R}(\mathbb{Q})}(\mathbb{M})$  and  $\mathrm{proj}_{\mathcal{M}}(\Pi)$ , we can now describe a numerical method implementing IMF (8). Let  $\Pi^0 = \Pi_{0,T}^0\mathbb{Q}_{|0,T}$  where  $\Pi_0^0 = \pi_0$ ,  $\Pi_T^0 = \pi_T$ . Learn  $\mathbb{M}^1\approx \mathrm{proj}_{\mathcal{M}}(\Pi^0)$  given by (9) with  $v_{\theta^{\star}}$  given by (10). Next, sample from  $\Pi^1 = \mathrm{proj}_{\mathcal{R}(\mathbb{Q})}(\mathbb{M}^1) = \mathbb{M}_{0,T}^1\mathbb{Q}_{|0,T}$  by sampling from  $\mathbb{M}_{0,T}^1$  and reconstructing the bridge  $\mathbb{Q}_{|0,T}$ . We iterate the process to obtain a sequence  $(\Pi^n,\mathbb{M}^{n + 1})_{n\in \mathbb{N}}$ . In practice, this algorithm performs poorly (see Figure 3), since the approximate minimization (10) for computing  $\mathbb{M}^{n + 1}$  may not admit  $\mathbb{M}_T^{n + 1} = \pi_T$  exactly as in Proposition 2. Instead, we incur a bias between  $\mathbb{M}_T^{n + 1}$  and  $\pi_T$  which accumulates for each  $n\in \mathbb{N}$ .
To mitigate this problem, we alternate between a forward Markovian projection and a backward Markovian projection. This procedure is justified by the following proposition.
Proposition 9. Assume that  $\Pi = \Pi_{0,T}\mathbb{Q}_{|0,T}$  with  $\mathbb{Q}$  associated with  $\mathrm{d}\mathbf{X}_t = f_t(\mathbf{X}_t)\mathrm{d}t + \sigma_t\mathrm{d}\mathbf{B}_t$ . Under mild conditions, the Markovian projection  $\mathbb{M}^{\star} = \operatorname{proj}_{\mathcal{M}}(\Pi)$  is associated with both
$$
\mathrm {d} \mathbf {X} _ {t} = \left\{f _ {t} \left(\mathbf {X} _ {t}\right) + \sigma_ {t} ^ {2} \mathbb {E} _ {\Pi_ {T | t}} \left[ \nabla \log \mathbb {Q} _ {T | t} \left(\mathbf {X} _ {T} \mid \mathbf {X} _ {t}\right) \mid \mathbf {X} _ {t} \right] \right\} \mathrm {d} t + \sigma_ {t} \mathrm {d} \mathbf {B} _ {t}, \quad \mathbf {X} _ {0} \sim \Pi_ {0}, \tag {11}
$$
$$
\mathrm {d} \mathbf {Y} _ {t} = \left\{- f _ {T - t} \left(\mathbf {Y} _ {t}\right) + \sigma_ {T - t} ^ {2} \mathbb {E} _ {\Pi_ {0 | T - t}} [ \nabla \log \mathbb {Q} _ {T - t | 0} \left(\mathbf {Y} _ {t} \mid \mathbf {Y} _ {T}\right) \mid \mathbf {Y} _ {t} ] \right\} \mathrm {d} t + \sigma_ {T - t} \mathrm {d} \mathbf {B} _ {t}, \mathbf {Y} _ {0} \sim \Pi_ {T}. \tag {12}
$$
In Proposition 9, (11) is the definition of the Markovian projection, see Definition 1. However, (12) is an equivalent representation as a time-reversal. In practice,  $(\mathbf{Y}_t)_{t\in [0,T]}$  is approximated with
$$
\mathrm {d} \mathbf {Y} _ {t} = \left\{- f _ {T - t} \left(\mathbf {Y} _ {t}\right) + v _ {\phi^ {*}} (T - t, \mathbf {Y} _ {t}) \right\} \mathrm {d} t + \sigma_ {T - t} \mathrm {d} \mathbf {B} _ {t}, \quad \mathbf {Y} _ {0} \sim \pi_ {T}, \tag {13}
$$
$$
\phi^ {\star} = \operatorname {a r g m i n} _ {\phi} \left\{\int_ {0} ^ {T} \mathbb {E} _ {\Pi_ {0, t}} \left[ \| \sigma_ {t} ^ {2} \nabla \log \mathbb {Q} _ {t | 0} (\mathbf {X} _ {t} | \mathbf {X} _ {0}) - v _ {\phi} (t, \mathbf {X} _ {t}) \| ^ {2} \right] / \sigma_ {t} ^ {2} \mathrm {d} t: \phi \in \Phi \right\}. \tag {14}
$$
The optimal  $v_{\phi^*}(t,x_t) = \sigma_t^2\mathbb{E}_{\Pi_{0|t}}[\nabla \log \mathbb{Q}_{t|0}(\mathbf{X}_t|\mathbf{X}_0)\mid \mathbf{X}_t = x_t]$  for any  $t\in [0,T]$  and  $x_{t}\in \mathbb{R}^{d}$ .
Algorithm 1 Diffusion Schrödinger Bridge Matching
1: Input: Joint distribution  $\Pi_{0,T}^{0}$ , tractable bridge  $\mathbb{Q}_{|0,T}$ , number of outer iterations  $N\in \mathbb{N}$ .
2: Let  $\Pi^0 = \Pi_{0,T}^0\mathbb{Q}_{|0,T}$
3: for  $n \in \{0, \dots, N - 1\}$  do
4: Learn  $v_{\phi^{\star}}$  using (14) with  $\Pi = \Pi^{2n}$ .
5: Let  $\mathbb{M}^{2n + 1}$  be given by (13).
6: Let  $\Pi^{2n + 1} = \mathbb{M}_{0,T}^{2n + 1}\mathbb{Q}_{|0,T}$ .
7: Learn  $v_{\theta^{\star}}$  using (10) with  $\Pi = \Pi^{2n + 1}$ .
8: Let  $\mathbb{M}^{2n + 2}$  be given by (9).
9: Let  $\Pi^{2n + 2} = \mathbb{M}_{0,T}^{2n + 2}\mathbb{Q}_{|0,T}$ .
10: end for
11: Output:  $v_{\theta \star}, v_{\phi \star}$
Note that  $\mathbf{X}_0\sim \pi_0$  in the forward projection, while  $\mathbf{Y}_0\sim \pi_T$  in the backward projection. Therefore, using the backward projection removes the bias on  $\pi_T$  accumulated from the forward projection. Leveraging the time-symmetry of the Markovian projection and alternating between (13) and (9) yields the DSBM methodology summarized in Algorithm 1.
It is also possible to learn both the forward and backward processes at each step, and enforce that the backward and forward processes match. We explore this in Appendix G.
Initialization coupling. We now relate Algorithm 1 to the classical IPF and practical algorithms such as DSB (De Bortoli et al., 2021). Instead of initializing DSBM with  $\Pi_{0,T}^{0}$  given by a coupling between  $\pi_0,\pi_T$ , if we initialize it by  $\Pi_{0,T}^{0} = \mathbb{Q}_{0,T}$  where  $\mathbb{Q}_0 = \pi_0$  and  $\mathbb{Q}_{T|0}$  is given by the reference process defined in (1), then DSBM also recovers the IPF iterates used in DSB.
Proposition 10. Suppose the families of functions  $\{v_{\theta}:\theta \in \Theta \}$  and  $\{v_{\phi}:\phi \in \Phi \}$  are rich enough so that they can model the optimal vector fields. Let  $(\Pi^n,\mathbb{M}^{n + 1})_{n\in \mathbb{N}}$  be the optimal DSBM sequence in Algorithm 1 initialized with  $\Pi_{0,T}^0 = \mathbb{Q}_{0,T}$ , and let  $(\tilde{\mathbb{P}}^n)_{n\in \mathbb{N}}$  be the optimal DSB sequence given by the IPF iterates in (7). Then for any  $n\in \mathbb{N},n\geq 1$ , we have  $\mathbb{M}^n = \tilde{\mathbb{P}}^n$ .
We will thus call DSBM-IPF, the DSBM algorithm initialized with the joint distribution given by the forward reference process  $\Pi_{0,T}^{0} = \mathbb{Q}_{0,T}$ ; and DSBM-IMF, the DSBM algorithm initialized with an independent coupling  $\Pi_{0,T}^{0} = \pi_{0} \otimes \pi_{T}$ . However, the training procedure of DSBM-IPF is very different from the one of (De Bortoli et al., 2021; Chen et al., 2022). In existing works,  $\tilde{\mathbb{P}}^{n+1}$  is
obtained as the time-reversal of  $\tilde{\mathbb{P}}^n$  which requires full trajectories from  $\tilde{\mathbb{P}}^n$ , see e.g. (De Bortoli et al., 2021, Proposition 6). In contrast, in Algorithm 1 we only use the coupling  $\mathbb{M}_{0,T}^n$  to create the bridge measure  $\Pi^n = \mathbb{M}_{0,T}^n\mathbb{Q}_{|0,T}$ . By doing so, (i) the losses (10) and (14) can be easily evaluated at any time  $t\in [0,T]$ ; (ii) the trajectory caching procedure in DSBM is more computationally and memory efficient; (iii) while every IPF iteration  $\tilde{\mathbb{P}}^n$  is also supposed to be in  $\mathcal{R}(\mathbb{Q})$ , in practice one can observe a forgetting of the bridge  $\mathbb{Q}_{|0,T}$  (Fernandes et al., 2021). In DSBM, this effect is countered by explicit projections on the reciprocal class. See Appendix F for more details.
Probability flow ODE. At equilibrium of DSBM, we have that  $(\mathbf{Y}_t)_{t\in [0,T]}$  given by (13) is the time reversal of  $(\mathbf{X}_t)_{t\in [0,T]}$  given by (9) and are both associated with the optimal Schrödinger Bridge path measure  $\mathbb{P}^{\star}$ . As a result, we have that  $v_{\phi^{\star}}(t,x) = -v_{\theta^{\star}}(t,x) + \sigma_t^2\nabla \log \mathbb{P}_t^{\star}(x)$ . Hence, a probability flow  $(\mathbf{Z}_t^\star)_{t\in [0,T]}$  such that  $\mathrm{Law}(\mathbf{Z}_t^\star) = \mathbb{P}_t^\star$  for any  $t\in [0,T]$  is given by
$$
\mathrm {d} \mathbf {Z} _ {t} ^ {\star} = \left\{f _ {t} (\mathbf {Z} _ {t} ^ {\star}) + \frac {1}{2} \left[ v _ {\theta^ {\star}} (t, \mathbf {Z} _ {t} ^ {\star}) - v _ {\phi^ {\star}} (t, \mathbf {Z} _ {t} ^ {\star}) \right] \right\} \mathrm {d} t, \quad \mathbf {Z} _ {0} ^ {\star} \sim \pi_ {0}.
$$
See also De Bortoli et al. (2021); Chen et al. (2022) for derivation of this result. Note however that the path measure induced by  $(\mathbf{Z}_t^\star)_{t\in [0,T]}$  does not correspond to  $\mathbb{P}^{\star}$ ; in particular,  $(\mathbf{Z}_0^\star ,\mathbf{Z}_T^\star)$  is not an entropic OT plan. However, since for any  $t\in [0,T]$ ,  $\mathbf{Z}_t^\star$  has marginal distribution  $\mathbb{P}_t^\star$ , we can compute the log-likelihood of the model (Song et al., 2021b; Huang et al., 2021).
Markovian projection and Bridge Matching. The concept of Markovian projection has been rediscovered multiple times (Krylov, 1984; Gyöngy, 1986; Dupire, 1994). In the machine learning context, this was first proposed by Peluchetti (2021) to define Bridge Matching models. More recently, Liu et al. (2022b) derived theoretical properties of the Markovian projection in Proposition 2, first part of Lemma 6, and applied Bridge Matching for learning data on discrete and constrained domains.
Bridge and Flow Matching. Flow Matching corresponds to deterministic bridges with deterministic samplers (ODEs) and has been under active study (Liu et al., 2023b; Liu, 2022; Lipman et al., 2023; Albergo and Vanden-Eijnden, 2023; Heitz et al., 2023; Pooladian et al., 2023; Tong et al., 2023). Denoising Diffusion Implicit Models (DDIM) (Song et al., 2021a) can also be formulated as a discrete-time version of Flow Matching, see Liu et al. (2023b). These models have been extended to the Riemannian setting by Chen and Lipman (2023). Recently, Albergo et al. (2023) studied the influence of stochasticity in the bridge, through the concept of stochastic interpolants. Liu et al. (2023a); Delbracio and Milanfar (2023) used Bridge Matching to perform image restoration tasks and noted benefits of stochasticity empirically. Closely related to our work is the Rectified Flow algorithm of Liu et al. (2023b), which corresponds to an iterative Flow Matching procedure in order to improve the straightness of the flow and thus eases its simulation. An iterative rectifying procedure using stochastic interpolants is also proposed in (Albergo et al., 2023, Section 3.5). Our proposed DSBM-IMF algorithm is closest to Rectified Flow, which can be seen as the deterministic limiting case of DSBM-IMF as  $\sigma \rightarrow 0$ . However, there are a few important theoretical and practical differences. Most notably, we adopt the SDE approach which is crucial for the validity of Proposition 5 as well as for the empirical performance of DSBM. We discuss further distinctions between DSBM and Rectified Flow in Appendix A.3.
Diffusion Schrödinger Bridge. Schrödinger Bridges (Schrödinger, 1932) are ubiquitous in probability theory (Léonard, 2014b) and stochastic control (Dai Pra, 1991; Chen et al., 2021). More recently, they have been used for generative modeling: De Bortoli et al. (2021) introduced the DSB algorithm and Vargas et al. (2021); Chen et al. (2022) introduced similar algorithms. The case of Dirac delta terminal distribution was investigated by Wang et al. (2021). These methods were later extended to solve conditional simulation and more general control problems (Shi et al., 2022; Thornton et al., 2022; Liu et al., 2022a; Chen et al., 2023; Tamir et al., 2023). In Somnath et al. (2023), SBs are learned using one Bridge Matching iteration, assuming access to the true Schrödinger static coupling. Our proposed method DSBM-IPF is closest to DSB, but with improved continuous-time training and projections on the reciprocal class which mitigate two limitations of DSB. Concurrently with our work, Peluchetti (2023) independently introduced the DSBM-IMF approach (named IDBM therein).
2D Experiments. We first show our proposed methods can generate correct samples and learn lower kinetic energy transport maps in some 2D examples. We compare our method DSBM with flow-based methods including Flow Matching (FM) (Lipman et al., 2023), Conditional Flow Matching (CFM), OT-CFM (Tong et al., 2023), and Rectified Flow (RF) (Liu et al., 2023b); and other SB methods including DSB (De Bortoli et al., 2021) and SB-CFM (Tong et al., 2023). OT-CFM and SB-CFM utilizes sample-based mini-batch OT or EOT solvers (Fatras et al., 2021; Flamary et al., 2021) to define an approximate OT or SB static coupling  $\tilde{\Pi}_{0,T}^{\mathrm{OT}}$  or  $\tilde{\Pi}_{0,T}^{\mathrm{SB}}$ , see also Pooladian et al. (2023); Stromme (2023). We can also utilize this idea in the DSBM-IMF framework, which corresponds to using the initialization coupling  $\Pi_{0,T}^{0} = \tilde{\Pi}_{0,T}^{\mathrm{SB}}$  in Algorithm 1. This approximate SB coupling  $\tilde{\Pi}_{0,T}^{\mathrm{SB}}$  also satisfies  $\tilde{\Pi}_{0}^{\mathrm{SB}} = \pi_{0}$  and  $\tilde{\Pi}_{T}^{\mathrm{SB}} = \pi_{T}$  but can provide a better initialization than the independent coupling  $\Pi_{0,T}^{0} = \pi_{0} \otimes \pi_{T}$ . We name this approach DSBM-IMF+. The rest of the methods do not use OT solvers. DSB and DSBM directly learn the EOT map as the solution of the diffusion process.
In Table 2, we show the 2-Wasserstein distance between the true and generated samples, as well as the integrated path energy defined as  $\mathbb{E}\left[\int_0^T ||v(t,\mathbf{Z}_t)||^2\mathrm{d}t\right]$  where  $v$  is the learned drift along the ODE trajectory  $\mathbf{Z}_t$ . For direct comparability, we report for DSBM using its probability flow ODE. Lower path energies represent shorter (and potentially easier to integrate) trajectories. We find that in this low dimensional setting, OT-CFM performs the best by utilizing OT solvers, but DSBM outperforms FM and CFM when OT solvers are not used. Further, DSBM outperforms DSB on all datasets, suggesting DSBM solves the SB problem with higher accuracy. The results also show that among SB methods, DSBM-IMF+ can achieve lower sampling error than DSBM-IPF and DSBM-IMF. It also performs better than SB-CFM on 3 of the datasets and achieve lower path energy on all datasets. Finally, we find Rectified Flow achieves lower sampling error than DSBM except for the moons-8gaussians task, for which DSBM is significantly more accurate. Since RF can be informally seen as DSBM in the case  $\sigma \rightarrow 0$ , this suggests the optimal  $\sigma$  varies for each task and between generative and general transfer tasks. Figure 2 visualizes how  $\sigma$  affects the straightness and sample quality of learned transport maps between two mixture distributions.
High-Dimensional Gaussian Experiment. We next perform the Gaussian transport experiment in De Bortoli et al. (2021) with dimension  $d = 50$  to verify the scalability of our proposed approach. The true SB can be computed analytically in this case (Bunne et al., 2023). In Figure 3, we plot the convergence of the learned mean  $\mathbb{E}[\mathbf{X}_0]$ , variance  $\mathrm{Var}(\mathbf{X}_0)$ , and covariance  $\mathrm{Cov}(\mathbf{X}_0,\mathbf{X}_T)$  between times  $0,T$ . We also consider RF and a related baseline IMF-b, which performs IMF numerically but only in the backward direction. All methods converge approximately to the correct mean. However,
Dataset	2-Wasserstein (Euler 20 steps)	Path energy
moons	scurve	8gaussians	moons-8gaussians	moons	scurve	8gaussians	moons-8gaussians
DSBM-IPF	0.140±0.006	0.140±0.024	0.315±0.079	0.812±0.092	1.598±0.034	2.110±0.059	14.91±0.310	42.16±1.026
DSBM-IMF	0.144±0.024	0.145±0.037	0.338±0.091	0.838±0.098	1.580±0.036	2.092±0.053	14.81±0.255	41.00±1.495
DSBM-IMF+	0.123±0.014	0.130±0.025	0.276±0.030	0.802±0.172	1.594±0.043	2.116±0.018	14.88±0.252	41.09±1.206
DSB	0.190±0.049	0.272±0.065	0.411±0.084	0.987±0.324	-	-	-	-
SB-CFM	0.129±0.024	0.136±0.030	0.238±0.044	0.843±0.079	1.649±0.035	2.144±0.044	15.08±0.209	45.69±0.661
FM	0.212±0.025	0.161±0.033	0.351±0.066	-	2.227±0.056	2.950±0.074	18.12±0.416	-
CFM	0.215±0.028	0.171±0.023	0.370±0.049	1.285±0.314	2.391±0.043	3.071±0.026	18.00±0.090	116.5±2.633
RF	0.129±0.022	0.126±0.019	0.267±0.041	1.522±0.304	1.185±0.052	1.633±0.074	14.84±0.441	37.61±3.906
OT-CFM	0.111±0.005	0.102±0.013	0.253±0.040	0.716±0.187	1.178±0.020	1.577±0.036	15.10±0.215	30.50±0.626
Table 2: Sampling quality as measured by 2-Wasserstein distance and path energy for the 2D experiments.  $\pm 1$  SD over 5 seeds. Best values are in bold and second best are italicized.
$\sigma = 0.0$
$\sigma = 0.1$
$\sigma = 0.3$
$\sigma = 1.0$
$\sigma = 3.0$
Figure 2: Learned SB probability flow between two mixtures of Gaussians (green  $\rightarrow$  yellow).
Figure 3: Convergence of Gaussian experiment in  $d = 50$ .
KL ×10-3	d=5	d=20	d=50
DSB	3.26±1.60	13.0±3.49	32.8±1.28
SB-CFM	1.45±0.73	12.3±1.47	49.4±3.91
DSBM-IPF	1.23±0.23	4.42±0.76	8.75±0.87
DSBM-IMF	1.34±0.51	5.05±0.95	9.76±1.67
Table 3: Average  $\mathrm{KL}(\mathbb{P}_t|\mathbb{P}_t^{\mathrm{SB}})$  at 21 uniformly spaced  $t$
(a) OT-CFM
(b) DSB
(c) DSBM-IPF
Figure 4: Samples of MNIST digits transferred from letters.
Figure 5: FID vs iteration.
the variance estimates become inaccurate for RF and IMF-b. Among SB methods, DSB and IMF-b also gave inaccurate SB covariance estimates as the number of iteration increases. On the other hand, DSBM does not suffer from this issue. In Table 3, we further quantify the accuracy and compare with SB-CFM (Tong et al., 2023) by computing the KL divergence between the marginal distributions of the learned process  $\mathbb{P}_t$  and the true SB  $\mathbb{P}_t^{\mathrm{SB}}$ . Our proposed methods achieve similar KL divergence as SB-CFM in dimension  $d = 5$ , but are much more accurate in higher dimensions.
MNIST, EMNIST transfer. We test our method for domain transfer between MNIST digits and EMNIST letters as in De Bortoli et al. (2021). We compare DSBM as a direct substitute of DSB, and also with Bridge Matching (BM) (Peluchetti, 2021; Liu et al., 2022b), CFM, OT-CFM and RF. We plot some output samples from different algorithms in Figure 4 and the convergence of FID score in Figure 5. We find that OT-CFM becomes less applicable in higher dimensions and produces samples of worse quality (Figure 4a). On the other hand, image quality deteriorates during training of DSB and RF. DSBM achieves higher quality samples visually, and does not suffer from deterioration. It is also about  $30\%$  more efficient than DSB in terms of runtime.
CelebA transfer. Next, we evaluate and perform some ablations of our method on a transfer task on the CelebA  $64 \times 64$  dataset. We consider the images given by the tokens male/old and female/young. In Figures 6 and 7, we show that as  $\sigma$  increases, the quality of the images (as measured by the FID score) increases until  $\sigma$  is too high, but the alignment (as measured by LPIPS) between the generated image and the original sample decreases. Additionally, we investigate the dependency between  $\sigma$  and image dimension in Figure 8. In particular, for the same  $\sigma = 1$ , the outputs of DSBM for CelebA  $128 \times 128$  are better aligned with the original data than for CelebA  $64 \times 64$ . This is in agreement with the observations of Chen (2023); Hoogeboom et al. (2023) that the noise schedule in diffusion models should scale with the resolution.
AFHQ transfer. We demonstrate the scalability of our method on an additional transfer experiment on the AFHQ  $512 \times 512$  dataset between the classes cat and wiId. The results are shown in Figure 9. On this higher-dimensional problem, we observe that DSBM can also generate realistic samples which are similar to the input.
Unpaired Fluid Flows Downscaling. Finally, we apply DSBM to perform downscaling of geophysical fluid dynamics, i.e. super-resolution of low resolution spatial data. We use the dataset in (Bischoff and Deck, 2023), which consists of unpaired low  $(64\times 64)$  and high  $(512\times 512)$  resolution fields. As shown in Figure 10, DSBM is able to learn high resolution reconstructions by only slightly noising the low resolution input. In contrast, Bischoff and Deck (2023) use two diffusion models in forward and backward directions (Diffusion-fb) based on Meng et al. (2022), which improves over the Random baseline. Figure 11 shows that DSBM-IPF and DSBM-IMF achieve much lower  $\ell_2$
Figure 6: Left to right: initial and generated samples  $(64\times 64)$  obtained after 20 DSBM-IMF iterations for  $\sigma^2\in \{0.01,0.1,1,10\}$ .
Figure 7: FID (blue) and LPIPS (red) scores (lower is better for both) as we vary  $\sigma^2$ .
Figure 8: Top to bottom: DSBM  $(\sigma = 1)$ $64\times 64$ $128\times 128$  original images.
(a) Transfer between classes cat and wild.
(b) Transfer between classes wild and cat.