SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yuyang Shi*
Valentin De Bortoli*
ENS ULM
Arnaud Doucet
Solving transport problems, i.e. finding a map transporting one given distribution to another, has numerous applications in machine learning. Novel mass transport methods motivated by generative modeling have recently been proposed, e.g. Denoising Diffusion Models (DDMs) and Flow Matching Models (FMMs) implement such a transport through a Stochastic Differential Equation (SDE) or an Ordinary Differential Equation (ODE). However, while it is desirable in many applications to approximate the deterministic dynamic Optimal Transport (OT) map which admits attractive properties, DDMs and FMMs are not guaranteed to provide transports close to the OT map. In contrast, Schrödinger bridges (SBs) compute stochastic dynamic mappings which recover entropy-regularized versions of OT. Unfortunately, existing numerical methods approximating SBs either scale poorly with dimension or accumulate errors across iterations. In this work, we introduce Iterative Markovian Fitting (IMF), a new methodology for solving SB problems, and Diffusion Schrödinger Bridge Matching (DSBM), a novel numerical algorithm for computing IMF iterates. DSBM significantly improves over previous SB numerics and recovers as special/limiting cases various recent transport methods. We demonstrate the performance of DSBM on a variety of problems.
Mass transport problems are ubiquitous in machine learning (Peyre and Cuturi, 2019). For discrete measures, the Optimal Transport (OT) map can be computed exactly but is computationally intensive. In a landmark paper, Cuturi (2013) showed that an entropy-regularized version of OT can be computed more efficiently using the Sinkhorn algorithm (Sinkhorn, 1967). This has enabled the use of OT techniques in a variety of applications ranging from biology (Bunne et al., 2022) to shape correspondence (Feydy et al., 2017). However, applications involving high-dimensional continuous distributions and/or large datasets remain challenging for these techniques.
One of such data-rich applications is generative modeling, a central transport problem in machine learning which requires designing a deterministic or stochastic mapping transporting a reference "noise" distribution to the data distribution. For example, Generative Adversarial Networks (Goodfellow et al., 2014) define a static, deterministic transport map, while Denoising Diffusion Models (DDMs) (Song et al., 2021b; Ho et al., 2020) build a dynamic, stochastic transport map by simulating a Stochastic Differential Equation (SDE), whose drift is learned using score matching (Hyvärinen, 2005; Vincent, 2011). The excellent performances of DDMs have motivated recent developments of Bridge Matching and Flow Matching models, which are dynamic transport maps using SDEs (Song et al., 2021a; Peluchetti, 2021; Liu, 2022; Albergo et al., 2023) or ODEs (Albergo and Vanden-Eijnden, 2023; Heitz et al., 2023; Lipman et al., 2023; Liu et al., 2023b). Compared to DDMs, Bridge and Flow Matching methods do not rely on a forward "noising" diffusion converging to the reference distribution in infinite time, and are also more generally applicable as they can approximate transport maps between two general distributions based on their samples. Nonetheless, these transport maps
Figure 1: Relationship between DSBM and existing methods.
Sets for alternating projections	Preserved properties
IPF	P0=π0; PT=πT	M, R(Q)
IMF	M; R(Q)	P0=π0, PT=πT
Table 1: Comparison between Iterative Markovian Fitting (IMF) and Iterative Proportional Fitting (IPF). The Schrödinger Bridge is the unique  $\mathbb{P}$  s.t.  $\mathbb{P}_0 = \pi_0$ ,  $\mathbb{P}_T = \pi_T$ ,  $\mathbb{P} \in \mathcal{M}$ ,  $\mathbb{P} \in \mathcal{R}(\mathbb{Q})$  simultaneously by Proposition 5.  $\mathcal{M}$  is the space of (regular) Markov measures and  $\mathcal{R}(\mathbb{Q})$  the space of reciprocal measures of  $\mathbb{Q}$ .
are not necessarily close to the OT map minimizing the Wasserstein-2 metric, which is appealing for its many attractive properties (Peyre and Cuturi, 2019; Villani, 2009).
In contrast, the Schrödinger Bridge (SB) problem is a dynamic version of entropy-regularized OT (EOT) (Föllmer, 1988; Léonard, 2014b). The SB is the finite-time diffusion which admits as initial and terminal distributions the two distributions of interest and is the closest in Kullback-Leibler divergence to a reference diffusion. Numerous methods to approximate SBs numerically have been proposed, see e.g. (Bernton et al., 2019; Chen et al., 2016; Finlay et al., 2020; Caluya and Halder, 2021; Pavon et al., 2021), but these techniques tend to be restricted to low-dimensional settings. Recently, novel techniques using diffusion-based ideas have been proposed in (De Bortoli et al., 2021; Vargas et al., 2021; Chen et al., 2022) based on Iterative Proportional Fitting (IPF) (Fortet, 1940; Kullback, 1968; Rüschendorf and Thomsen, 1993), a continuous state-space extension of the Sinkhorn algorithm (Essid and Pavon, 2019). These approaches have been shown to scale better empirically, but numerical errors tend to accumulate over iterations (Fernandes et al., 2021).
In this paper, our contributions are three-fold. First, we introduce Iterative Markovian Fitting (IMF), a new procedure to compute SBs which alternates between projecting on the space of Markov processes and on the reciprocal class, i.e. the measures which have the same bridge as the reference measure of SB (Léonard et al., 2014). We establish various theoretical results for IMF. Contrary to IPF, the IMF iterates always preserve the initial and terminal distributions. The differences between IPF and IMF are presented in Table 1. Second, we propose Diffusion Schrödinger Bridge Matching (DSBM), a novel algorithm approximating numerically the SB solution derived from IMF. DSBM requires at each iteration solving a simple regression problem in the spirit of Bridge and Flow Matching, and does not suffer from the time-discretization and "forgetting" issues of previous DSB techniques (De Bortoli et al., 2021; Vargas et al., 2021; Chen et al., 2022). Finally, we demonstrate the performance of DSBM on a variety of transport tasks. $^{2}$
Notations. We denote by  $\mathcal{P}(\mathcal{C})$ , the space of path measures, i.e.  $\mathcal{P}(\mathcal{C}) = \mathcal{P}(\mathrm{C}([0,T],\mathbb{R}^d))$  where  $T > 0$ . The subset of Markov path measures associated with an SDE of the form  $\mathrm{d}\mathbf{X}_t = v_t(\mathbf{X}_t)\mathrm{d}t + \sigma_t\mathrm{d}\mathbf{B}_t$ , with  $\sigma, v$  locally Lipschitz, is denoted  $\mathcal{M}$ . For any  $\mathbb{Q} \in \mathcal{M}$ , the reciprocal class of  $\mathbb{Q}$  is denoted  $\mathcal{R}(\mathbb{Q})$ , see Definition 3. We also denote  $\mathbb{Q}_t$  its marginal distribution at time  $t$ ,  $\mathbb{Q}_{s,t}$  the joint distribution at times  $s$  and  $t$ ,  $\mathbb{Q}_{s|t}$  the conditional distribution at time  $s$  given state at time  $t$ , and  $\mathbb{Q}_{|0,T} \in \mathcal{P}(\mathcal{C})$  its diffusion bridge. Unless specified otherwise, all gradient operators  $\nabla$  are w.r.t. the variable  $x_{t}$  with time index  $t$ . Let  $(\mathsf{X},\mathcal{X})$  and  $(\Upsilon,\mathcal{Y})$  be probability spaces. Given a Markov kernel  $\mathrm{K}: \mathsf{X} \times \mathcal{Y} \to [0,1]$  and a probability measure  $\mu$  defined on  $\mathcal{X}$ , we write  $\mu\mathrm{K}$  the probability measure on  $\mathcal{Y}$  such that for any  $\mathrm{A} \in \mathcal{Y}$  we have  $\mu\mathrm{K}(\mathrm{A}) = \int_{\mathsf{X}}\mathrm{K}(x,\mathrm{A})\mathrm{d}\mu(x)$ . In particular, for any joint distribution  $\Pi_{0,T}$  over  $\mathbb{R}^d \times \mathbb{R}^d$ , we denote the mixture of bridges measure as  $\Pi = \Pi_{0,T}\mathbb{Q}_{|0,T} \in \mathcal{P}(\mathcal{C})$ , which is short for  $\Pi(\cdot) = \int_{\mathbb{R}^d \times \mathbb{R}^d}\mathbb{Q}_{|0,T}(\cdot|x_0,x_T)\Pi_{0,T}(\mathrm{d}x_0,\mathrm{d}x_T)$ .
2 Dynamic Mass Transport Techniques
2.1 Denoising Diffusion and Bridge Matching Models
Denoising Diffusion Models (Song et al., 2021b; Ho et al., 2020) are a popular class of generative models. They define a forward noisng process  $\mathbb{Q}\in \mathcal{M}$  using the SDE  $\mathrm{d}\mathbf{X}_t = -\frac{1}{2}\mathbf{X}_t\mathrm{d}t + \mathrm{d}\mathbf{B}_t$  on the time-interval  $[0,T]$ , where  $\mathbf{X}_0\in \mathbb{R}^d$  is drawn from the data distribution  $\pi_0$  and  $(\mathbf{B}_t)_{t\in [0,T]}$  is a  $d-$
$^{2}$ Code can be found at https://github.com/yuyang-shi/dsbm-pytorch.
dimensional Brownian motion. This diffusion<sup>3</sup> converges towards the standard Gaussian distribution  $\mathrm{N}(0,\mathrm{Id})$  as  $T\to \infty$ . A generative model is given by its time-reversal  $(\mathbf{Y}_t)_{t\in [0,T]} = (\mathbf{X}_{T - t})_{t\in [0,T]}$ , where  $\mathbf{Y}_0\sim \mathbb{Q}_T$  and  $\mathrm{d}\mathbf{Y}_t = \{\frac{1}{2}\mathbf{Y}_t + \nabla \log \mathbb{Q}_{T - t}(\mathbf{Y}_t)\} \mathrm{d}t + \mathrm{d}\mathbf{B}_t$  (Anderson, 1982; Haussmann and Pardoux, 1986). In practice,  $(\mathbf{Y}_t)_{t\in [0,T]}$  is initialized with  $\mathbf{Y}_0\sim \pi_T = \mathrm{N}(0,\mathrm{Id})$ , and the Stein score  $\nabla \log \mathbb{Q}_t(x_t) = \mathbb{E}_{\mathbb{Q}_{0|t}}[\nabla \log \mathbb{Q}_{t|0}(\mathbf{X}_t|\mathbf{X}_0)\mid \mathbf{X}_t = x_t]$  is approximated using a neural network  $s_\theta (t,x_t)$  minimizing the denoising score matching loss  $\mathbb{E}_{\mathbb{Q}_{0,t}}[||\nabla \log \mathbb{Q}_{t|0}(\mathbf{X}_t|\mathbf{X}_0) - s_\theta (t,\mathbf{X}_t)||^2 ]$ .
An alternative to considering the time-reversal of a forward noising process is to "build bridges" between the two distributions and learn a mimicking diffusion process. This approach generalizes DDMs and allows for more flexible choices of sampling processes. We call this framework Bridge Matching and adopt a presentation similar to Peluchetti (2021); Liu et al. (2022b), where  $\pi_T$  is the data distribution.4 We denote  $\mathbb{Q} \in \mathcal{M}$  the path measure associated with the following process
$$
\mathrm {d} \mathbf {X} _ {t} = f _ {t} \left(\mathbf {X} _ {t}\right) \mathrm {d} t + \sigma_ {t} \mathrm {d} \mathbf {B} _ {t}, \quad \mathbf {X} _ {0} \sim \mathbb {Q} _ {0}. \tag {1}
$$
Consider now the distribution of this process pinned down at an initial and terminal point  $x_0, x_T$ , denoted  $\mathbb{Q}_{|0,T}(\cdot | x_0, x_T)$ . Under mild assumptions, the pinned process  $\mathbb{Q}_{|0,T}(\cdot | x_0, x_T)$  is a diffusion bridge and is given by
$$
\mathrm {d} \mathbf {X} _ {t} ^ {0, T} = \left\{f _ {t} \left(\mathbf {X} _ {t} ^ {0, T}\right) + \sigma_ {t} ^ {2} \nabla \log \mathbb {Q} _ {T | t} \left(x _ {T} \mid \mathbf {X} _ {t} ^ {0, T}\right) \right\} \mathrm {d} t + \sigma_ {t} \mathrm {d} \mathbf {B} _ {t}, \quad \mathbf {X} _ {0} ^ {0, T} = x _ {0}, \tag {2}
$$
which satisfies  $\mathbf{X}_T^{0,T} = x_T$  using Doob  $h$ -transform theory (Rogers and Williams, 2000). Next, we define an independent coupling  $\Pi_{0,T} = \pi_0 \otimes \pi_T$ , and let  $\Pi = \Pi_{0,T}\mathbb{Q}_{|0,T}$ . This path measure  $\Pi$  is a mixture of bridges. We aim to find a Markov diffusion  $\mathrm{d}\mathbf{Y}_t = \{f_t(\mathbf{Y}_t) + v_t(\mathbf{Y}_t)\} \mathrm{d}t + \sigma_t\mathrm{d}\mathbf{B}_t$  on  $[0,T]$  which admits the same marginals as  $\Pi$ ; i.e. for any  $t \in [0,T]$ ,  $\mathbf{Y}_t \sim \Pi_t$ , so  $\mathbf{Y}_T \sim \pi_T$ . For such  $v_t$ , a generative model for sampling data distribution  $\pi_T$  is obtained by simulating  $(\mathbf{Y}_t)_{t \in [0,T]}$ . It can be verified that indeed  $\mathbf{Y}_t \sim \Pi_t$  for  $v_t^\star(x_t) = \sigma_t^2\mathbb{E}_{\Pi_{T|t}}[\nabla \log \mathbb{Q}_{T|t}(\mathbf{X}_T|\mathbf{X}_t) | \mathbf{X}_t = x_t]$ . We present the theory behind this idea more formally using Markovian projections in Section 3.1. In practice, we do not have access to  $v_t^\star$  and it is learned using neural networks with regression loss
$$
\mathbb {E} _ {\Pi_ {t, T}} \left[ \left\| \sigma_ {t} ^ {2} \nabla \log \mathbb {Q} _ {T | t} \left(\mathbf {X} _ {T} \mid \mathbf {X} _ {t}\right) - v _ {\theta} (t, \mathbf {X} _ {t}) \right\| ^ {2} \right]. \tag {3}
$$
For  $f_{t} = 0$  and  $\sigma_{t} = \sigma$ ,  $\mathbb{Q}_{|0,T}$  is a Brownian Bridge and we have
$$
\mathbf {X} _ {t} ^ {0, T} = \frac {t}{T} x _ {T} + \left(1 - \frac {t}{T}\right) x _ {0} + \sigma_ {t} \left(\mathbf {B} _ {t} - \frac {t}{T} \mathbf {B} _ {T}\right), \quad \mathrm {d} \mathbf {X} _ {t} ^ {0, T} = \left\{\left(x _ {T} - \mathbf {X} _ {t} ^ {0, T}\right) / (T - t) \right\} \mathrm {d} t + \sigma_ {t} \mathrm {d} \mathbf {B} _ {t}, \tag {4}
$$
with  $(\mathbf{B}_t - \frac{t}{T}\mathbf{B}_T)\sim \mathrm{N}(0,t(1 - \frac{t}{T})\mathrm{Id})$ . The regression loss (3) associated with (4) is given by
$$
\mathbb {E} _ {\Pi_ {t, T}} \left[ \left\| \left(\mathbf {X} _ {T} - \mathbf {X} _ {t}\right) / (T - t) - v _ {\theta} (t, \mathbf {X} _ {t}) \right\| ^ {2} \right]. \tag {5}
$$
Letting  $\sigma \rightarrow 0$ , we recover Flow Matching models (see Appendix A.1 for further details).
2.2 Schrödinger Bridges and Optimal Transport
The Schrödinger Bridge (SB) problem (Schrödinger, 1932) consists in finding a path measure  $\mathbb{P}^{\mathrm{SB}}\in \mathcal{P}(\mathcal{C})$  such that
$$
\mathbb {P} ^ {\mathrm {S B}} = \operatorname {a r g m i n} _ {\mathbb {P}} \left\{\mathrm {K L} (\mathbb {P} | \mathbb {Q}): \mathbb {P} _ {0} = \pi_ {0}, \mathbb {P} _ {T} = \pi_ {T} \right\}, \tag {6}
$$
where  $\mathbb{Q} \in \mathcal{P}(\mathcal{C})$  is a reference path measure. In what follows, we consider  $\mathbb{Q}$  defined by the diffusion process (1) which is Markov, and without loss of generality, we assume  $\mathbb{Q}_0 = \pi_0$ . Hence  $\mathbb{P}^{\mathrm{SB}}$  is the path measure closest to  $\mathbb{Q}$  in terms of Kullback-Leibler divergence which satisfies the initial and terminal constraints  $\mathbb{P}_0^{\mathrm{SB}} = \pi_0$  and  $\mathbb{P}_T^{\mathrm{SB}} = \pi_T$ .
Another crucial property of  $\mathbb{P}^{\mathrm{SB}}$  is that it can also be defined as a mixture of bridges  $\mathbb{P}^{\mathrm{SB}} = \Pi_{0,T}^{\mathrm{SB}}\mathbb{Q}_{|0,T}$ , where  $\Pi_{0,T}^{\mathrm{SB}} = \operatorname*{argmin}_{\Pi_{0,T}}\{\mathrm{KL}(\Pi_{0,T}|\mathbb{Q}_{0,T}) : \Pi_0 = \pi_0, \Pi_T = \pi_T\}$  is the solution of the static SB problem (Léonard, 2014b). In particular, for  $\mathbb{Q}$  associated with  $(\sigma \mathbf{B}_t)_{t\in [0,T]}$  we have
$$
\Pi_ {0, T} ^ {\mathrm {S B}} = \operatorname * {a r g m i n} _ {\Pi_ {0, T}} \{\mathbb {E} _ {\Pi_ {0, T}} [ | | \mathbf {X} _ {0} - \mathbf {X} _ {T} | | ^ {2} - 2 \sigma^ {2} T \mathrm {H} (\Pi_ {0, T}): \Pi_ {0} = \pi_ {0}, \Pi_ {T} = \pi_ {T} \},
$$
This is known as the Ornstein-Uhlenbeck (OU) process or VPSDE (Song et al., 2021b).
To keep notations consistent with existing works,  $\pi_0$  is the data distribution in the context of DDM and SB, whereas  $\pi_T$  is the data distribution in Bridge Matching. However, both SB and Bridge Matching methods allow transfer between arbitrary distributions  $\pi_0, \pi_T$ , so this distinction is not important.
where  $\mathrm{H}(\mu)$  denotes the entropy, i.e.  $\Pi_{0,T}^{\mathrm{SB}}$  is the solution of the entropy-regularized OT problem. In this case, the SB can also be obtained theoretically by solving the following problem (Dai Pra, 1991)
$$
v _ {\mathrm {S B}} = \operatorname * {a r g m i n} _ {v} \left\{\int_ {0} ^ {T} \mathbb {E} _ {\mathbb {P} _ {t}} [ | | v (t, \mathbf {X} _ {t}) | | ^ {2} ] \mathrm {d} t: \mathrm {d} \mathbf {X} _ {t} = v (t, \mathbf {X} _ {t}) \mathrm {d} t + \sigma \mathrm {d} \mathbf {B} _ {t}, \mathbb {P} _ {0} = \pi_ {0}, \mathbb {P} _ {T} = \pi_ {T} \right\}.
$$
Then  $\mathbb{P}^{\mathrm{SB}}$  is given by the SDE with drift  $v_{\mathrm{SB}}$  initialized with  $\mathbf{X}_0 \sim \pi_0$ . For  $\sigma = 0$ , we recover the classical OT problem and the Benamou-Brenier formula (Benamou and Brenier, 2000).
A common approach to solve (6) is the Iterative Proportional Fitting (IPF) method (Fortet, 1940; Kullback, 1968; Ruschendorf, 1995) defining a sequence of path measures  $(\tilde{\mathbb{P}}^n)_{n\in \mathbb{N}}$  where
$$
\tilde {\mathbb {P}} ^ {2 n + 1} = \operatorname * {a r g m i n} _ {\tilde {\mathbb {P}}} \{\mathrm {K L} (\tilde {\mathbb {P}} | \tilde {\mathbb {P}} ^ {2 n}): \tilde {\mathbb {P}} _ {T} = \pi_ {T} \}, \tilde {\mathbb {P}} ^ {2 n + 2} = \operatorname * {a r g m i n} _ {\tilde {\mathbb {P}}} \{\mathrm {K L} (\tilde {\mathbb {P}} | \tilde {\mathbb {P}} ^ {2 n + 1}): \tilde {\mathbb {P}} _ {0} = \pi_ {0} \}, (7)
$$
with initialization  $\tilde{\mathbb{P}}^0 = \mathbb{Q}$ . This procedure alternates between projections on the set of path measures with given initial distribution  $\pi_0$  and terminal distribution  $\pi_T$ . It can be shown (De Bortoli et al., 2021) that  $(\tilde{\mathbb{P}}^n)_{n\in \mathbb{N}}$  are associated with diffusions and that for any  $n\in \mathbb{N}$ ,  $\tilde{\mathbb{P}}^{2n + 1}$  is the time-reversal of  $\tilde{\mathbb{P}}^{2n}$  with initialization  $\pi_T$ , and  $\tilde{\mathbb{P}}^{2n + 2}$  is the time-reversal of  $\tilde{\mathbb{P}}^{2n + 1}$  with initialization  $\pi_0$ . Leveraging this property, De Bortoli et al. (2021) proposed Diffusion Schrödinger Bridge (DSB), an algorithm which learns the time-reversals iteratively. In particular, DDMs can be seen as the first iteration of DSB.
3 Iterative Markovian Fitting
3.1 Markovian Projection and Reciprocal Projection
Markovian Projection. Projecting on Markov measures is a key ingredient in our methodology and in the Bridge Matching framework. This concept was introduced multiple times in the literature (Gyöngy, 1986; Peluchetti, 2021; Liu et al., 2022b). In particular, we focus on Markovian projection of path measures given by a mixture of bridges  $\Pi = \Pi_{0,T} \mathbb{Q}_{|0,T} \in \mathcal{P}(\mathcal{C})$ .
Definition 1. Assume that  $\mathbb{Q}$  is given by (1) and that for any  $(x_0,x_T)\in \mathbb{R}^d$ $\mathbb{Q}_{|0,T}(\cdot |x_0,x_T)$  is associated with  $(\mathbf{X}_t^{0,T})_{t\in [0,T]}$  given by  $\mathrm{d}\mathbf{X}_t^{0,T} = \{f_t(\mathbf{X}_t^{0,T}) + \sigma_t^2\nabla \log \mathbb{Q}_{T|t}(x_T|\mathbf{X}_t^{0,T})\} \mathrm{d}t + \sigma_t\mathrm{d}\mathbf{B}_t$  with  $\sigma :[0,T]\to (0, + \infty)$ . Then, when it is well-defined, we introduce the Markovian projection of  $\Pi$ ,  $\mathbb{M}^{\star} = \operatorname {proj}_{\mathcal{M}}(\Pi)\in \mathcal{M}$ , which is associated with the SDE
$$
\mathrm {d} \mathbf {X} _ {t} ^ {\star} = \{f _ {t} (\mathbf {X} _ {t} ^ {\star}) + v _ {t} ^ {\star} (\mathbf {X} _ {t} ^ {\star}) \} \mathrm {d} t + \sigma_ {t} \mathrm {d} \mathbf {B} _ {t}, \qquad v _ {t} ^ {\star} (x _ {t}) = \sigma_ {t} ^ {2} \mathbb {E} _ {\Pi_ {T | t}} [ \nabla \log \mathbb {Q} _ {T | t} (\mathbf {X} _ {T} | \mathbf {X} _ {t}) | \mathbf {X} _ {t} = x _ {t} ].
$$
Note that in our definition  $\sigma_t > 0$  so  $\nabla \log \mathbb{Q}_{T|t}(x_T|x_t)$  is well-defined, but Flow Matching can be recovered as the deterministic case in the limit  $\sigma_t = \sigma \to 0$ . In the following proposition, we show that the Markovian projection is indeed a projection for the reverse Kullback-Leibler divergence, and that it preserves marginals of  $\Pi_t$ .
Proposition 2. Assume that  $\sigma_t > 0$ . Let  $\mathbb{M}^{\star} = \mathrm{proj}_{\mathcal{M}}(\Pi)$ . Then, under mild assumptions, we have
$$
\mathbb {M} ^ {\star} = \operatorname {a r g m i n} _ {\mathbb {M}} \left\{\mathrm {K L} (\Pi | \mathbb {M}): \mathbb {M} \in \mathcal {M} \right\},
$$
$$
\mathrm {K L} (\Pi | \mathbb {M} ^ {\star}) = \frac {1}{2} \int_ {0} ^ {T} \mathbb {E} _ {\Pi_ {0, t}} [ \| \sigma_ {t} ^ {2} \mathbb {E} _ {\Pi_ {T | 0, t}} [ \nabla \log \mathbb {Q} _ {T | t} (\mathbf {X} _ {T} | \mathbf {X} _ {t}) \mid \mathbf {X} _ {0}, \mathbf {X} _ {t} ] - v _ {t} ^ {\star} (\mathbf {X} _ {t}) \| ^ {2} ] / \sigma_ {t} ^ {2} \mathrm {d} t.
$$
In addition, we have that for any  $t \in [0, T]$ ,  $\mathbb{M}_t^\star = \Pi_t$ . In particular,  $\mathbb{M}_T^\star = \Pi_T$ .
Reciprocal Projection. While the Markovian projection ensures that the obtained measure is Markov, the associated bridge measure is not preserved in general, i.e.  $\mathrm{proj}_{\mathcal{M}}(\Pi)_{|0,T} \neq \Pi_{|0,T} = \mathbb{Q}_{|0,T}$ . Measures with same bridge as  $\mathbb{Q}$  are said to be in its reciprocal class (Léonard et al., 2014).
Definition 3.  $\Pi \in \mathcal{P}(\mathcal{C})$  is in the reciprocal class  $\mathcal{R}(\mathbb{Q})$  of  $\mathbb{Q}\in \mathcal{M}$  if  $\Pi = \Pi_{0,T}\mathbb{Q}_{|0,T}$ . We define the reciprocal projection of  $\mathbb{P}\in \mathcal{P}(\mathcal{C})$  as  $\Pi^{\star} = \mathrm{proj}_{\mathcal{R}(\mathbb{Q})}(\mathbb{P}) = \mathbb{P}_{0,T}\mathbb{Q}_{|0,T}$ .
Similarly to Proposition 2, we have the following result, which justifies the term reciprocal projection.
$$
\textbf {P r o p o s i t i o n 4 .} L e t \mathbb {P} \in \mathcal {P} (\mathcal {C}), \Pi^ {\star} = \operatorname {p r o j} _ {\mathcal {R} (\mathbb {Q})} (\mathbb {P}). T h e n, \Pi^ {\star} = \operatorname {a r g m i n} _ {\Pi} \{\mathrm {K L} (\mathbb {P} | \Pi): \Pi \in \mathcal {R} (\mathbb {Q}) \}.
$$
The reciprocal projection  $\Pi^*$  of a Markov path measure  $\mathbb{M}$  does not preserve the Markov property in general. In fact, the Schrödinger Bridge is the unique path measure which satisfies the initial and terminal conditions, is Markov and is in the reciprocal class of  $\mathbb{Q}$ , see (Léonard, 2014b).
Proposition 5. Let  $\mathbb{P}$  be a Markov measure in the reciprocal class of  $\mathbb{Q}$  such that  $\mathbb{P}_0 = \pi_0$ ,  $\mathbb{P}_T = \pi_T$ . Then, under assumptions on  $\mathbb{Q}$ ,  $\pi_0$  and  $\pi_T$ ,  $\mathbb{P}$  is unique and is equal to the Schrödinger Bridge  $\mathbb{P}^{\mathrm{SB}}$ .
3.2 Iterative Markovian Fitting
Based on Proposition 5, we propose a novel methodology called Iterative Markovian Fitting (IMF) to solve Schrödinger Bridges. We consider a sequence  $(\mathbb{P}^n)_{n\in \mathbb{N}}$  such that
$$
\mathbb {P} ^ {2 n + 1} = \operatorname {p r o j} _ {\mathcal {M}} \left(\mathbb {P} ^ {2 n}\right), \quad \mathbb {P} ^ {2 n + 2} = \operatorname {p r o j} _ {\mathcal {R} (\mathbb {Q})} \left(\mathbb {P} ^ {2 n + 1}\right), \tag {8}
$$
with  $\mathbb{P}^0$  such that  $\mathbb{P}_0^0 = \pi_0$ ,  $\mathbb{P}_T^0 = \pi_T$  and  $\mathbb{P}^0 \in \mathcal{R}(\mathbb{Q})$ . These updates correspond to alternatively performing Markovian projections and reciprocal projections.
Combining Proposition 2 and Definition 3, we get that for any  $n \in \mathbb{N}$ ,  $\mathbb{P}_0^n = \pi_0$  and  $\mathbb{P}_T^n = \pi_T$ . This property is in contrast to the IPF algorithm (7) for which the marginals at the initial and final times are not preserved. We highlight this duality between IPF (7) and IMF (8) in Table 1.
We conclude this section with a theoretical analysis of IMF. First, we start by showing a Pythagorean theorem for both the Markovian projection and the reciprocal projection.
Lemma 6. Under mild assumptions, if  $\mathbb{M} \in \mathcal{M}$ ,  $\Pi \in \mathcal{R}(\mathbb{Q})$  and  $\mathrm{KL}(\Pi|\mathbb{M}) < +\infty$ , we have
$$
\mathrm {K L} (\Pi | \mathbb {M}) = \mathrm {K L} (\Pi | \operatorname {p r o j} _ {\mathcal {M}} (\Pi)) + \mathrm {K L} (\operatorname {p r o j} _ {\mathcal {M}} (\Pi) | \mathbb {M}).
$$
If  $\mathrm{KL}(\mathbb{M}\Pi) < +\infty$ , we have
$$
\mathrm {K L} (\mathbb {M} | \Pi) = \mathrm {K L} (\mathbb {M} | \operatorname {p r o j} _ {\mathcal {R} (\mathbb {Q})} (\mathbb {M})) + \mathrm {K L} (\operatorname {p r o j} _ {\mathcal {R} (\mathbb {Q})} (\mathbb {M}) | \Pi).
$$
Using Lemma 6, we have the following proposition.
Proposition 7. Under mild assumptions, we have  $\mathrm{KL}(\mathbb{P}^{n + 1}|\mathbb{P}^{\mathrm{SB}})\leq \mathrm{KL}(\mathbb{P}^n |\mathbb{P}^{\mathrm{SB}}) < \infty$ , and  $\lim_{n\to +\infty}\mathrm{KL}(\mathbb{P}^n |\mathbb{P}^{n + 1}) = 0$ .