Original: N ON -D ENOISING F ORWARD -T IME D IFFUSIONS
Revised: N ON -D ENOISING F ORWARD -T IME D IFFUSIONS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The scope of this paper is generative modeling through diffusion processes.
Revised: The scope of this paper is generative modeling through diffusion processes.

Original: An approach falling within this paradigm is the work of Song et al.
Revised: An approach falling within this paradigm is the work of Song et al.

Original: (2021), which relies on a time-reversal argument to construct a diffusion process targeting the desired data distribution.
Revised: (2021), which relies on a time-reversal argument to construct a diffusion process targeting the desired data distribution.

Original: We show that the time-reversal argument, common to all denoising diffusion probabilistic modeling proposals, is not necessary.
Revised: We show that the time-reversal argument, common to all denoising diffusion probabilistic modeling proposals, is not necessary.

Original: We obtain diffusion processes targeting the desired data distribution by taking appropriate mixtures of diffusion bridges.
Revised: We obtain diffusion processes targeting the desired data distribution by taking appropriate mixtures of diffusion bridges.

Original: The resulting transport is exact by construction, allows for greater ﬂexibility in choosing the dynamics of the underlying diffusion, and can be approximated by means of a neural network via novel training objectives.
Revised: The resulting transport is exact by construction, allows for greater ﬂexibility in choosing the dynamics of the underlying diffusion, and can be approximated by means of a neural network via novel training objectives.

Original: We develop an unifying view of the drift adjustments corresponding to our and to time-reversal approaches and make use of this representation to inspect the inner workings of diffusion-based generative models.
Revised: We develop a unifying view of the drift adjustments corresponding to our and to time-reversal approaches and make use of this representation to inspect the inner workings of diffusion-based generative models.

Original: Finally, we leverage on scalable simulation and inference techniques common in spatial statistics to move beyond fully factorial distributions in the underlying diffusion dynamics.
Revised: Finally, we leverage on scalable simulation and inference techniques common in spatial statistics to move beyond fully factorial distributions in the underlying diffusion dynamics.

Original: The methodological advances contained in this work contribute toward establishing a general framework for generative modeling based on diffusion processes.
Revised: The methodological advances contained in this work contribute toward establishing a general framework for generative modeling based on diffusion processes.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Denoising diffusion probabilistic modeling (DDPM)
Revised: Denoising diffusion probabilistic modeling (DDPM)

Original: (Sohl-Dickstein et al., 2015; Ho et al., 2020;
Revised: (Sohl-Dickstein et al., 2015; Ho et al., 2020;

Original: Song et al., 2021) is a recent generative modeling paradigm exhibiting strong empirical performance.
Revised: Song et al., 2021) is a recent generative modeling paradigm exhibiting strong empirical performance.

Original: Consider a dataset of N samples D
Revised: Consider a dataset of N samples D

Original: = { x ( n ) } Nn =1 with empirical distribution P D .
Revised: = { x ( n ) } Nn =1 with empirical distribution P D .

Original: The unifying key steps underlying DDPM approaches are: (i) the deﬁnition of a stochastic processX on [0 , τ ] , with initial distribution P D ,  progressively transforming P D toward a simple data-independent distribution P Z ; (ii) the derivation of the reverse-time dynamicsof X progressively transforming P Z toward P D ; (iii) the approximation of the reverse-time transitions by means of a neural network.
Revised: The unifying key steps underlying DDPM approaches are: (i) the deﬁnition of a stochastic process with initial distribution P D , whose forward-time (noising) dynamics progressively transform P D toward a simple data-independent distribution P Z ; (ii) the derivation of the backward-time (denoising / sampling) dynamics transforming P Z toward P D ; (iii) the approximation of the backward-time transitions by means of a neural network.

Original: Following the training step (iii), a sampleY , whose distribution approximates P D, is drawn by (iv) simulating from the approximated reverse-time transitions starting with a sample from P Z .
Revised: Following the training step (iii), a sample whose distribution approximates P D is drawn by (iv) simulating from the approximated backward-time transitions starting with a sample from P Z .

Original: Both discrete-time (Ho et al., 2020) and continuous-time (Song et al., 2021) formulations of DDPM have been pursued and this work focuses on the latter case.
Revised: Both discrete-time (Ho et al., 2020) and continuous-time (Song et al., 2021) formulations of DDPM have been pursued. This work focuses on the latter case, to which we refer as diffusion time-reversal transport (DTRT).

Original: As in (Song et al., 2021), we specify the continuous-time dynamics  through diffusion processes, i.e. solutions to stochastic differential equations (SDE) with associated drift f ( · ) and diffusion g ( · ) coefﬁcients.
Revised: As in DTRT, dynamics are speciﬁed through diffusion processes, i.e. solutions to stochastic differential equations (SDE) with associated drift f ( · ) and diffusion g ( · ) coefﬁcients.

Original: A number of approximations are involved in the aforementioned steps.
Revised: A number of approximations are involved in the aforementioned steps.

Original: Firstly, as the process X evolves on a ﬁnite time interval [0 , τ ] , its (forward-time) terminal distribution Q τ retains a dependency on the initial distribution P D .
Revised: Firstly, as the dynamics are deﬁned on a ﬁnite time interval, a dependency from P D is retained through the noising process.

Original: The exact reverse-time simulation of X requires to start with a sample from Q τ , and P Z is only an approximation for Q τ .
Revised: Hence, starting with a sample from the data-independent distribution P Z in (iv) introduces an approximation.

Original: Secondly, while the reverse-time dynamics of (ii) are directly available for diffusions, they are approximated by means of a neural network in (iii).
Revised: Secondly, while the backward-time dynamics of (ii) are directly available for diffusions, they are approximated by means of a neural network in (iii).

Original: Thirdly, samplingfrom Y in (iv) is achieved through a discretizationof the reverse-time transitions of X on a time-grid, which introduces a discretization error.
Revised: Thirdly, sampling in (iv) is achieved through a discretization on a time-grid, which introduces a discretization error.

Original: De Bortoli et al.
Revised: De Bortoli et al.

Original: (2021, Theorem 1) links thesethree approximations to the total variation distance between the distribution of Y and P D .
Revised: (2021, Theorem 1) links these approximations to the total variation distance between the distribution of the generated samples from (iv) and P D .

Original: In the ﬁrst methodological contributionof this paper we introduce a methodology for constructing diffusion processes targeting P D without relying on time-reversal arguments.
Revised: In our ﬁrst methodological contribution we develop a procedure for constructing diffusion processes targeting P D without relying on time-reversal arguments.

Original: The desired transport (coupling) betweenan initial distribution P Z anda terminal distribution P D is achieved by: (1) specifying a diffusion process X on [0 , τ ] starting from  x 0 ; (2) conditioning X on hitting  x τ at time τ , thus obtaining a diffusion bridge; (3) taking a bivariate mixture  of diffusion bridges over ( x 0 , x τ )  resulting in a diffusion with initial distribution P Z and terminal distribution P D .
Revised: The proposed transport (coupling) between P Z and P D is achieved by: (1) specifying a diffusion process X on [0 , τ ] starting from a generic x 0 ; (2) conditioning X on hitting a generic x τ at time τ , thus obtaining a diffusion bridge; (3) taking a bivariate mixture Π 0 ,τ of diffusion bridges over ( x 0 , x τ ) with marginals Π 0 = P Z and Π τ = P D , obtaining a mixture process M ; (4) matching the marginal distribution of M over [0 , τ ] with a diffusion process, resulting in a diffusion with initial distribution P Z and terminal distribution P D .

Original: The realized diffusion bridges mixture transport (DBMT) between P Z and P D is exact by construction.
Revised: The realized diffusion bridge mixture transport (DBMT) between P Z and P D is exact by construction.

Original: We thus sidestep the approximation common to all DDPM approaches due to the discrepancy between Q τ and its approximating P Z .
Revised: We thus sidestep the approximation common to all DDPM approaches due to the dependency from P D retained through the noising process.

Original: Moreover, the DBMT can be realized for almost arbitrary, and independently chosen, P Z , f ( · ) and g ( · ) .
Revised: Moreover, the DBMT can be realized for almost arbitrary P Z , f ( · ) and g ( · ) .

Original: This represents a departure from the diffusion time-reversal transport (DTRT) of Song et al. (2021), whereQ τ depends on P D thought the integration time τ , which can be considered ﬁxed (Section 4.1), and through f ( · ) and g ( · ). The need to minimize the dependency of Q τ from P D , i.e. to achieve Q τ ≈ P Z for some simple  P Z, places heavy constraints on the speciﬁcations of f ( · ) and g ( · ) .
Revised: This increased ﬂexibility is a departure from the DTRT where f ( · ) and g ( · ) need to be chosen to obtain convergence toward a simple distribution P Z .

Original: Similarly to the DTRT, achieving the DBMT requires the computation of a drift adjustment term which depends on D .
Revised: Similarly to the DTRT, achieving the DBMT requires the computation of a drift adjustment term which depends on D .

Original: For a SDE class of interest, we develop an uniﬁed and interpretable representation of DTRT and DBMT drift adjustments as simple transformations of conditional expectations over
Revised: For a SDE class of interest, we develop a uniﬁed and interpretable representation of DTRT and DBMT drift adjustments as simple transformations of conditional expectations over

Original: D .
Revised: D .

Original: This novel result provides insights on the target mapping that we aim to approximate and on the quality of approximation achieved by the trained score models of Song et al.
Revised: This novel result provides insights on the target mapping that we aim to approximate and on the quality of approximation achieved by the trained score models of Song et al.

Original: Having deﬁned for the DBMT a Fisher divergence objective similarly to Song et al.
Revised: Having deﬁned for the DBMT a Fisher divergence objective similarly to Song et al.

Original: (2021), we leverage on this uniﬁed representation to deﬁne two additional training objectives featuring appealing properties.
Revised: (2021), we leverage on this uniﬁed representation to deﬁne two additional training objectives featuring appealing properties.

Original: In our last methodological contribution we extend the class of SDEs that can be realistically employed in computer vision applications.
Revised: In our last methodological contribution we extend the class of SDEs that can be realistically employed in computer vision applications.

Original: Speciﬁcally, computational considerations have so far restricted the transitions of the stochastic processes employed in DDPM to be fully factorials.
Revised: Speciﬁcally, computational considerations have so far restricted the transitions of the stochastic processes employed in DDPM to be fully factorials.

Original: We view images at a given resolution as taking values over a 2D lattice which discretizes the continuous coordinate system [0 , 1] 2 representing heights and widths.
Revised: We view images at a given resolution as taking values over a 2D lattice which discretizes the continuous coordinate system [0 , 1] 2 representing heights and widths.

Original: Diffusion processes are viewed as spatio-temporal processes with spatial support [0 , 1] 2 .
Revised: Diffusion processes are viewed as spatio-temporal processes with spatial support [0 , 1] 2 .

Original: Doing so, it is possible to leverage on scalable simulation and inference techniques from spatial statistics and consider more realistic diffusion transitions.
Revised: Doing so, it is possible to leverage on scalable simulation and inference techniques from spatial statistics and consider more realistic diffusion transitions.

Original: This paper is structured as follows.
Revised: This paper is structured as follows.

Original: In Section 2 we review the DTRT of Song et al.
Revised: In Section 2 we review the DTRT of Song et al.

Original: (2021) and in
Revised: (2021) and in

Original: Section 3 we introduce the DBMT.
Revised: Section 3 we introduce the DBMT.

Original: In order to implement the DTRT and the DBMT it is necessary to specify the underlying SDE, i.e the coefﬁcients f ( · ) and g ( · ) .
Revised: In order to implement the DTRT and the DBMT it is necessary to specify the underlying SDE, i.e. the coefﬁcients f ( · ) and g ( · ) .

Original: We study a class of interest in
Revised: We study a class of interest in

Original: Section 4.
Revised: Section 4.

Original: The uniﬁed view of drift adjustments is introduced in Section 5, while Section 6 develops  training objectives for the DTRT and for the DBMT.
Revised: The uniﬁed view of drift adjustments is introduced in Section 5. Section 6 develops the training objectives and Section 7 reviews the obtained results and ﬁnalizes the DBMT construction.

Original: In Section 7 we establish the connection with spatio-temporal processes.
Revised: In Section 8 we establish the connection with spatio-temporal processes.

Original: We conclude in Section 8.
Revised: We conclude in Section 9.

Original: Appendices A to D contain the theoretical framework, assumptions, proofs,analytical formulas for the SDE class of Section 4 and additional material.
Revised: Appendices A to D contain the theoretical framework, assumptions, proofs, and additional material.

Original: Notation and conventions: we use uppercase letters for probability distributions (measures, laws) and lowercase letters for densities;  random elements are denoted by uppercaseletters (an exception is made for times, always lowercase for typographical reasons); if P is the distribution of a stochastic process, we use subscript notation to refer to its ﬁnite dimensional distributions (densities with p ), conditional or not, for some collection of times; for example p t (cid:48) | t denotes a transition density, which is understood to be a function of four arguments p t (cid:48) | t ( y | x ) =
Revised: Notation and conventions: we use uppercase notation for probability distributions (measures, laws) and lowercase notation for densities; each probability distribution, and corresponding density, is uniquely identiﬁed by its associated letter not by its arguments (which are muted); for example P ( dx ) is a distribution, p ( x ) is its corresponding density; random elements are always uppercase (an exception is made for times, always lowercase for typographical reasons); if P is the distribution of a stochastic process, we use subscript notation to refer to its ﬁnite dimensional distributions (densities with p ), conditional or not, for some collection of times; for example p t (cid:48) | t denotes a transition density, which is understood to be a function of four arguments p t (cid:48) | t ( y | x ) =

Original: f ( t, t (cid:48) , x, y ) ;  we refer directly to a given SDE instead of referring to the diffusion process satisfying such SDE when no ambiguity arises; we use [ a ] i and [ A ] i,j for vector and matrix indexing, A (cid:62) for matrix transposition.
Revised: f ( t, t (cid:48) , x, y ) ; δ x is the delta distribution at x and ⊗ is used for product distributions; we refer directly to a given SDE instead of referring to the diffusion process satisfying such SDE when no ambiguity arises; we use [ a ] i and [ A ] i,j for vector and matrix indexing, A (cid:62) for matrix transposition.

Original:  D IFFUSION T IME -R EVERSAL T RANSPORT
Revised:  D IFFUSION T IME -R EVERSAL T RANSPORT

Original: The starting point of Song et al.
Revised: The starting point of Song et al.

Original: (2021) is a diffusion process X satisfying a generic D -dimensional time-inhomogenous SDE with initial distribution X 0 ∼ P D
Revised: (2021) is a diffusion process Y satisfying a generic D -dimensional time-inhomogenous SDE with initial distribution Y 0 ∼ P D

Original: [Equation]
Revised: [Equation]

Original: (1) over noising time u ∈ [0 , τ ] .
Revised: (1) over noising time r ∈ [0 , τ ] .

Original: Thorough this paper we denote with Q the law of the diffusion solving (1) and with q the corresponding densities.
Revised: Thorough this paper we denote with Q the law of the diffusion solving (1) and with q the corresponding densities.

Original: Thus, let q u (cid:48) | u ( y | x ) , 0 ≤ u < u (cid:48) ≤ τ , be the transition density of (1), and let q u ( x ) , 0 < u ≤ τ , be the marginal density of (1).
Revised: Thus, let q r (cid:48) | r ( y | x ) , 0 ≤ r < r (cid:48) ≤ τ , be the transition density of (1), and let q r ( y ) , 0 < r ≤ τ , be the marginal density of (1).

Original: As X 0 ∼ P D , we have
Revised: As Y 0 ∼ P D , we have

Original: [Equation]
Revised: [Equation]

Original: The dynamics of (1) over the reversed, i.e. sampling, time t = τ − u , t ∈ [0 , τ ] , are given by (Anderson, 1982; Haussmann & Pardoux, 1986; Millet et al., 1989)
Revised: The dynamics of (1) over the reversed, i.e. sampling, time t = τ − r , t ∈ [0 , τ ] , are given by (Anderson, 1982; Haussmann & Pardoux, 1986; Millet et al., 1989)

Original: [Equation]
Revised: [Equation]

Original: where u = u ( t ) = τ − t , G ( x, u ) =
Revised: where r = τ − t is the remaining sampling time, G ( x, r ) =

Original: g ( x, u ) g ( x, u ) (cid:62) and the D -dimensional vector ∇ · G ( x, u ) is deﬁned by [ ∇ · G ( x, u )]
Revised: g ( x, r ) g ( x, r ) (cid:62) and the D -dimensional vector ∇ · G ( x, r ) is deﬁned by [ ∇ · G ( x, r )]

Original: i = (cid:80)
Revised: i = (cid:80)

Original: Dj =1 ∇ x j [ G ( x, u )] i,j .
Revised: Dj =1 ∇ x j [ G ( x, r )] i,j .

Revised: That is the processes X t and Y r

Original:  Approximating the terminal distribution Q τ of (1), i.e. the initial distribution of (3), with P Z , X 0 is sampled from P Z and (3) is discretized and integrated over t to produce a sample
Revised: = Y τ − t have the same distribution. Approximating the terminal distribution Q τ of (1), i.e. the initial distribution of (3), with P Z , X 0 is sampled from P Z and (3) is discretized and integrated over t to produce a sample

Original: X τ approximately distributed as P D .
Revised: X τ approximately distributed as P D .

Original: The computation of the multiplicative drift adjustment ∇ x ln q u ( x ) entering (3), i.e. the score of the marginal density (2), requires in principle O ( N ) operations.
Revised: The computation of the multiplicative drift adjustment ∇ y ln q r ( y ) entering (3), i.e. the score of the marginal density (2), requires in principle O ( N ) operations.

Original: Let s φ ( x, u ) be a neural network for which we would like s φ ( x, u ) ≈ ∇ x ln q u ( x ) .
Revised: Let s φ ( y, r ) be a neural network for which we would like s φ ( y, r ) ≈ ∇ y ln q r ( y ) .

Original: It remains to ﬁnd a suitable training objective for which unbiased gradients with respect to φ can be obtained at O (1) cost with respect to the dataset size N .
Revised: It remains to ﬁnd a suitable training objective for which unbiased gradients with respect to φ can be obtained at O (1) cost with respect to the dataset size N .

Original: As q u ( x ) has a mixture representation, the identity of Vincent (2011) for Fisher divergences provides us with the desired objective for a ﬁxed u ∈ (0 , τ ]
Revised: As q r ( y ) has a mixture representation, the identity of Vincent (2011) for Fisher divergences provides us with the desired objective for a ﬁxed r ∈ (0 , τ ]

Original: [Equation]
Revised: [Equation]

Original: The key point is that an unbiased, O (1) with respect to N , mini-batch Monte Carlo (MC) estimator for the expectation (4) can be trivially obtained by sampling a batch X 0 ∼ P D , X u ∼ Q u | 0 ( dX u | X 0 ) , and evaluating the average loss over the bach.
Revised: The key point is that an unbiased, O (1) with respect to N , mini-batch Monte Carlo (MC) estimator for the expectation (4) can be trivially obtained by sampling a batch Y 0 ∼ P D , Y r ∼ Q r | 0 ( dy r | Y 0 ) , and evaluating the average loss over the batch.

Original: In order to achieve a global approximation over the whole time interval (0 , τ ] , Song et al.
Revised: In order to achieve a global approximation over the whole time interval (0 , τ ] , Song et al.

Original: (2021) proposes uniform sampling of timeu
Revised: (2021) proposes uniform sampling of time

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: where  ru = E
Revised: where R r = E

Original: [ (cid:107)∇ X u ln q u | 

Revised: [ (cid:107)∇ Y r ln q r |

Original: ( X u | X 0 ) (cid:107) 2 ] − 1 is a regularization term.
Revised: ( Y r | Y 0 ) (cid:107) 2 ] − 1 is a regularization term.

Original: A MC estimator for (5) is constructed by augmenting the MC estimator for (4) with the additional sampling step u ∼ U (0 , τ ] .
Revised: A MC estimator for (5) is constructed by augmenting the MC estimator for (4) with the additional sampling step r ∼ U (0 , τ ] .

Original:  D IFFUSION B RIDGES M IXTURE T RANSPORT
Revised:  D IFFUSION B RIDGE M IXTURE T RANSPORT

Original: Our starting point is a generic D -dimensional time-inhomogenous SDE which, in contrast to Song et al.
Revised: Our starting point is a generic D -dimensional time-inhomogenous SDE which, in contrast to Song et al.

Original: (2021), is directly deﬁned on the sampling time t ∈ [0 , τ ]
Revised: (2021), is directly deﬁned on the sampling time t ∈ [0 , τ ]

Original: [Equation]
Revised: [Equation]

Original: We reserve P ·| 0 ( ·| x 0 ) to denote the law of the diffusion solving (6) for a given starting value x 0 and p ·| 0 ( ·| x 0 ) to denote the corresponding densities.
Revised: We reserve P ·| 0 ( ·| x 0 ) to denote the law of the diffusion solving (6) for a given starting value x 0 and p ·| 0 ( ·| x 0 ) to denote the corresponding densities.

Original: 3. D IFFUSION B RIDGES
Revised: 3. D IFFUSION B RIDGES

Original: Diffusion bridges are central to the proposed methodology, in this Section we cover their basic theory.
Revised: Diffusion bridges are central to the proposed methodology, in this Section we cover their basic theory.

Original: A diffusion bridge is a diffusion process starting from a given value which is conditioned on hitting a terminal value.
Revised: A diffusion bridge is a diffusion process starting from a given value which is conditioned on hitting a terminal value.

Original: It is a deep result, and consequence of Doob h -transforms (Särkkä & Solin (2019, Chapter 7.9), Rogers & Williams (2000, Chapter IV.6.39)), that a diffusions processes pinned down on both ends is still a diffusion process.
Revised: It is a deep result, and consequence of Doob h -transforms (Särkkä & Solin (2019, Chapter 7.9), Rogers & Williams (2000, Chapter IV.6.39)), that a diffusion processes pinned down on both ends is still a diffusion process.

Original: In particular the Markov property is preserved.
Revised: In particular the Markov property is preserved.

Original: More precisely, (6) with initial value x 0 conditioned on hitting a terminal value x τ at time τ is characterized the following SDE on [0 , τ ] with initial value x 
Revised: More precisely, (6) with initial value x 0 conditioned on hitting a terminal value x τ at time τ is characterized the following SDE on [0 , τ ] with initial value x 0 (Särkkä & Solin, 2019, Theorem 7.11)

Original: [Equation]
Revised: [Equation]

Original: + g ( X t , t ) dW t , (7) where G ( x, t ) =
Revised: where G ( x, t ) =

Original: g ( x, t ) g ( x, t ) (cid:62) .
Revised: g ( x, t ) g ( x, t ) (cid:62) .

Original: The multiplicative adjustment factor ∇ x t ln p τ | t ( x τ | x t ) forces the process to hit x τ at time τ and the diffusion process solving (7) is known as the diffusion bridge from ( x 0 , 0) to ( x τ , τ ) .
Revised: The multiplicative adjustment factor ∇ x t ln p τ | t ( x τ | x t ) forces the process to hit x τ at time τ and the diffusion process solving (7) is known as the diffusion bridge from ( x 0 , 0) to ( x τ , τ ) .

Original: As previously noted, p τ | t ( x τ | x t ) in (7) refers to the transition density of (6).
Revised: As previously noted, p τ | t ( x τ | x t ) in (7) refers to the transition density of (6).

Original: 3. D IFFUSION M IXTURES
Revised: 3. D IFFUSION M IXTURES

Original: The proposed transport construction relies on a  resultestablishing SDE representations for diffusion mixtures.
Revised: The proposed transport construction relies on a representation result for diffusion mixtures.

Original: We present here an informal version and report the precise statement, the required assumptions, and the proof in Appendix A.
Revised: We present here an informal version and report the precise statement, the required assumptions, and the proof in Appendix A.

Original: Theorem 1 (Mixture of diffusions — informal) .
Revised: Theorem 1 (Diffusion mixture representation — informal) .

Original: Let { X λ } , λ ∈ Λ be a collection of diffusions with associated SDEs { dX λt } and marginal densities { π λt } .
Revised: Let { X λ } , λ ∈ Λ be a collection of diffusions with associated SDEs { dX λt } and marginal densities { π λt } .

Original: Let L be a mixing distribution on Λ , π  be the L -mixture of { π λt } .
Revised: Let L be a mixing distribution on Λ , π t be the L -mixture of { π λt } .

Original: Then there exists a diffusion process X with marginal π  .
Revised: Then there exists a diffusion process X with marginal π t .

Original: Moreover X follows a SDE whose drift and diffusion coefﬁcients are weighted averages of the corresponding coefﬁcients in { dX λt } , where the weights are proportional to { π λt } and to the mixing density.
Revised: X follows a SDE whose drift and diffusion coefﬁcients are weighted averages of the corresponding coefﬁcients in { dX λt } , where the weights are proportional to { π λt } and to the mixing density.

Original: Theorem 1 is ﬁrst established in Brigo (2008, Corollary 1.3) limitedly to ﬁnite mixtures and 1dimensional diffusions.
Revised: Theorem 1 is ﬁrst established in Brigo (2002, Corollary 1.3) limitedly to ﬁnite mixtures and 1dimensional diffusions.

Original: The proof of Theorem 1 in Appendix A is more direct and extends the result to the required multivariate setting.
Revised: The proof of Theorem 1 in Appendix A is more direct and extends the result to the required multivariate setting.

Original: In Section 3.1 we introduced diffusion bridges mapping arbitrary initial values X 0 to arbitrary ﬁnal values X τ .
Revised: In Section 3.1 we introduced diffusion bridges mapping arbitrary initial values x 0 to arbitrary ﬁnal values x τ .

Original:  We now proceed to apply Theorem 1 to mix diffusion bridges and obtain a diffusion process for which ( X 0 , X τ ) is distributed according to a generic bivariate distribution Π 0 ,τ .
Revised: Let Π 0 ,τ denote a generic bivariate distribution on R D × R D with marginals Π 0 , Π τ . We deﬁne the diffusion mixture M as the mixture of diffusion bridges corresponding to ( X 0 , X τ ) ∼ Π 0 ,τ .

Original: That is, we apply Theorem 1 to the collection of diffusion bridges (7) indexed by their initial and terminal values, λ = ( x 0 , x τ ) , Λ = R D × R D , with mixing distribution 
Revised: That is, we apply Theorem 1 to the collection of diffusion bridges (7) indexed by their initial and terminal values, λ = ( x 0 , x τ ) , Λ = R D × R D , with mixing distribution L ( dλ )

Original: L ( dλ ) =
Revised: =

Original: Π 0 ,τ ( dx 0 , dx τ ) .
Revised: Π 0 ,τ ( dx 0 , dx τ ) .

Original: Then Theorem 1yields the following SDE on [0 , τ ] with initial-terminal distribution Π 0 ,τ
Revised: By Theorem 1 the following SDE on [0 , τ ] with initial distribution Π 0 has the same marginal distribution as M , in particular its terminal distribution is Π τ [Equation]

Original: [Equation]
Revised: [Equation]

Original: In (8), A ( x t , t ) gives the multiplicative drift adjustment factor for (6).
Revised: In (8), A ( x t , t ) gives the multiplicative drift adjustment factor for (6).

Original: While our most general construction allows for dependent X 0 and X τ , we can achieve an independent transport via a factorial distribution Π 0 ,τ = Π 0 ⊗ Π τ .

Original: Another case of particular interest occurs when Π 0 puts all the mass on a single value x 0 .
Revised: A case of particular interest occurs when Π 0 puts all the mass on a single value x 0 .

Original: In the following we refer to A ( x t , t, x 0 ) in stance of A ( x t , t ) , and to π t | 0 ( x t | x 0 ) in stance of π t ( x t ) , when it is necessary to distinguish this speciﬁc case.
Revised: In the following we refer to A ( x t , t, x 0 ) in stance of A ( x t , t ) , and to π t | 0 ( x t | x 0 ) in stance of π t ( x t ) , when it is necessary to distinguish this speciﬁc case.

Original: Having deﬁned in (8) a processes with initial-terminal distribution Π 0 ,τ we extend the scope of Π to indicate the law of this process.
Revised: We also extend the scope of Π to indicate the law of M . Indeed, we already denoted with Π 0 ,τ its initial-terminal distribution, and with π t its marginal density. Accordingly,

Original: Indeed, we have already denoted with π t in (8) its marginal density.
Revised: A ( x t , t ) = E X τ ∼ Π( dx τ | x t ) [ ∇ x t ln p τ | t ( X τ | x t )] .

Original: The transportof interest from P Z to P D is  achieved by Π τ = P D and P Z
Revised: The transport from P Z to P D is then achieved by Π τ = P D and P Z

Original: = Π 0 ( P Z can be arbitrarily deﬁned).
Revised: = Π 0 ( P Z can be arbitrarily deﬁned).

Original: As P D is an empirical distribution, the integral in (8) with respect to x τ reduces to averages over D .
Revised: As P D is an empirical distribution, the integral in (8) with respect to x τ reduces to averages over D .

Revised: In summary, the diffusion X solution of (8) realizes the proposed transport from P Z to P D by matching the marginal distribution of M .

Original:  SDE C LASS
Revised:  SDE C LASS

Original: The starting point of the proposed transport is the unconstrained SDE (6).
Revised: The starting point of the proposed transport is the unconstrained SDE (6).

Original: In this Section we deﬁne
Revised: In this Section we deﬁne

Original: SDEs which are realized through a time-change of simpler SDEs and which are general enough to subsume the SDEs introduced in Song et al.
Revised: SDEs which are realized through a time-change of simpler SDEs and which are general enough to subsume the SDEs introduced in Song et al.

Original: Consider the D -dimensional SDEs
Revised: Consider the D -dimensional SDEs

Original: [Equation]
Revised: [Equation]

Original:  where α t (cid:54) = 0 is a scalar function and G ( X t , t ) =
Revised: (10) where α t (cid:54) = 0 is a scalar function and G ( X t , t ) =

Original: Γ introduces an arbitrary covariance structure.
Revised: Γ introduces an arbitrary covariance structure.

Original: (9) is the SDE of a correlated and scaled Brownian motion and (10) is the SDE of an Ornstein-Uhlenbeck process driven by a correlated and scaled Brownian motion.
Revised: (9) is the SDE of a correlated and scaled Brownian motion and (10) is the SDE of an Ornstein-Uhlenbeck process driven by a correlated and scaled Brownian motion.

Original: The transition densities of (9) and (10) are Gaussian (Appendix B).
Revised: The transition densities of (9) and (10) are Gaussian (Appendix B).

Original: We denote both with (cid:101) p t (cid:48) | t , informally (9) is a special case of (10) with α t = 0 .
Revised: We denote both with (cid:101) p t (cid:48) | t , informally (9) is a special case of (10) with α t = 0 .

Original: SDE (10) in the time-homogenous case α t = − 1 / 2 has stationary distribution N D (0 , Γ) .
Revised: SDE (10) in the time-homogenous case α t = − 1 / 2 has stationary distribution N D (0 , Γ) .

Original: We now introduce the time-change.
Revised: We now introduce the time-change.

Original: Let β t > 0 be a continuous function on [0 , τ ] .
Revised: Let β t > 0 be a continuous function on [0 , τ ] .

Original: Then b t = (cid:82) t 

Revised: Then b t = (cid:82) t

Original: β u du deﬁnes a monotonically (strictly) increasing function b t : [0 , τ ] → [0 , b τ ] .
Revised: β u du deﬁnes a monotonically (strictly) increasing function b t : [0 , τ ] → [0 , b τ ] .

Original: The following SDEs on [0 , τ ] represent the class of dynamics for (1) and (6) on which we focus in the rest of this paper 
Revised: The following SDEs on [0 , τ ] represent the class of dynamics for (1) and (6) on which we focus on the rest of this paper [Equation]

Original: [Equation]
Revised: [Equation]

Original: and G ( x, t ) =
Revised: and G ( x, t ) =

Original: β t Γ .
Revised: β t Γ .

Original: The standard time-change result for diffusions (Øksendal, 2003, Theorem 8.5.1) establishes that the processes X t respectively from (11) and (12) are equivalent in law to their timescaled counterparts Z b t from (9) and (10).
Revised: The standard time-change result for diffusions (Øksendal, 2003, Theorem 8.5.1) establishes that the processes X t respectively from (11) and (12) are equivalent in law to their timescaled counterparts Z b t from (9) and (10).

Original: That is, SDEs (11) and (12) correspond to the evolution of the simpler SDEs (9) and (10) under a non-linear time wrapping where time ﬂows with instantaneous intensity β t .
Revised: That is, SDEs (11) and (12) correspond to the evolution of the simpler SDEs (9) and (10) under a non-linear time wrapping where time ﬂows with instantaneous intensity β t .

Original: For both (11) and (12) the time-change argument yields p τ | t ( y | x ) =
Revised: For both (11) and (12) the time-change argument yields p τ | t ( y | x ) =

Original: (cid:101) p b
Revised: (cid:101) p b

Original: τ | b t ( y | x ) for the transition density of (6), and equivalently for the transition density q τ | t of (1).
Revised: τ | b t ( y | x ) for the transition density of (6), and equivalently for the transition density q τ | t of (1).

Original: We thus obtain
Revised: We thus obtain

Original: [Equation]
Revised: [Equation]

Original: 13) for appropriate scalar functions a ( t, τ ) , v ( t, τ ) with v ( t, τ ) > 0 (Appendix B).
Revised: [Equation] for appropriate scalar functions a ( t, τ ) , v ( t, τ ) with v ( t, τ ) > 0 (Appendix B).

Original: By direct computation
Revised: By direct computation

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: From Bayes theorem and the Markov property we have
Revised: From Bayes theorem and the Markov property we have

Original: [Equation]
Revised: [Equation]

Original: where once again a br (0 , t, τ ) , a br (0 , t, τ ) and v br (0 , t, τ ) > 0 are scalar functions given in Appendix B. Finally,once again by direct computation
Revised: where once again a br (0 , t, τ ) , a br (0 , t, τ ) and v br (0 , t, τ ) > 0 are scalar functions given in Appendix B. Finally, by direct computation

Original: [Equation]
Revised: [Equation]

Original: These results provide all the analytical formulas required for the computation of the adjustment factors
Revised: These results provide all the analytical formulas required for the computation of the adjustment factors

Original: A ( x t , t ) , A ( x t , t, x 0 ) and of the training objectives used to approximate them (Section 6).
Revised: A ( x t , t ) , A ( x t , t, x 0 ) and of the training objectives used to approximate them (Section 6).

Original: 4. I NTERPRETATION OF D ENOISING T IME -R EVERSED SDE S
Revised: 4. I NTERPRETATION OF D ENOISING T IME -R EVERSED SDE S

Original: Song et al.
Revised: Song et al.

Original: (2021) introduces two speciﬁcations of (1), named VESDE and VPSDE, which are respectively given by
Revised: (2021) introduces two speciﬁcations of (1), named VESDE and VPSDE, which are respectively given by

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: See Appendix B for the functional form of β ve ,u and β vp ,u .
Revised: See Appendix B for the functional form of β ve ,r and β vp ,r .

Original: We thus recover (18) and (19) from (11) and (12) with Γ =
Revised: We thus recover (18) and (19) from (11) and (12) with Γ =

Original: I and α t = − 1 / 2 .
Revised: I and α t = − 1 / 2 .

Original: That is, VESDE and VPSDE correspond to a time change of the much simpler SDEs for the standard Brownian motion and for the standard Langevin SDE
Revised: That is, VESDE and VPSDE correspond to a time change of the much simpler SDEs for the standard Brownian motion and for the standard Langevin SDE

Original: [Equation]
Revised: [Equation]

Original:  U NIFIED V IEW OF D RIFT A DJUSTMENTS
Revised:  U NIFIED V IEW OF D RIFT A DJUSTMENTS

Original: The linearity of SDEs (11) and (12), underlying our and Song et al.
Revised: The linearity of SDEs (11) and (12), underlying our and Song et al.

Original: (2021) work, has the important consequence that (14) and (15) are linear in x t .
Revised: (2021) works, has the important consequence that (14) and (15) are linear in x t .

Original: This in turn allow us to derive an alternative representation for the drift adjustment in (8).
Revised: This in turn allow us to derive an alternative representation for the drift adjustment in (8).

Original: Indeed, substituting (14) in (8) gives (Appendix A)
Revised: Indeed, substituting (14) in (8) gives (Appendix A)

Original: [Equation]
Revised: [Equation]

Original: Similarly, for the time-reversal drift adjustment term in (3) we have (Appendix A)
Revised: Similarly, for the time-reversal drift adjustment term in (3) we have (Appendix A)

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: The relations (20) and (21) provide an uniﬁed view of the inner workings of the DTRT and of the
Revised: The relations (20) and (21) provide a uniﬁed view of the inner workings of the DTRT and of the

Original: DBMT targeting P D .
Revised: DBMT targeting P D .

Original: In the following we always refer to sampling time t .
Revised: In the following we always refer to sampling time t .

Original: Let ∆ t = τ − t = u represent the remaining sampling time.
Revised: Remember that r = τ − t is the remaining sampling time.

Original: For ease of exposition we assume β t = 1 , as shown in Section 4 the term β t corresponds to a time warping of an underlying time scale.
Revised: For ease of exposition we assume β t = 1 , as shown in Section 4 the term β t corresponds to a time-warping.

Original: To start with, the expectation terms in (20) and (21), to which we commonly refer as E [ Y | x, t ] ,  are always convex linear combinations of the samples x ( n ) from D .
Revised: The terms a ( t, τ ) , a (0 , r ) are “integrated scalings”. They are equal to 1 for (11) and the same holds for (12) as r → 0 .

Original: The weights ω ( x, t ) ( n ) , n = 1 , . . . , N , of the linear combination, i.e. the discrete probabilities deﬁning E
Revised: The terms v ( t, τ ) , v (0 , r ) are “integrated variances”. They are equal to r for (11) and the same holds for (12) as r → 0 .

Original: [ Y | x, t ] , are given by the probabilities, under the assumed stochastic process, of reaching each state x ( n ) at terminal time τ from x at time t . The term a ( t, τ ) gives the “integrated scaling” on ∆ t , which goes to 1 as ∆ t → 0 , and is not critically important.
Revised: We commonly refer to E

Original: In the case of (11) it holds that a ( t, τ ) = 1 .
Revised: [ X τ | x, t ] for expectation terms in (20) and (21).

Original: The term v ( t, τ  ) gives the “integrated variance” on ∆ t , which goes to 0 as ∆ t → 0 . In the case of (11), v ( t, τ ) =
Revised: Both drift adjustments (20) and (21) are thus essentially of the form ( E [ X τ | x, t ] − x ) v − 1 r where the term v − 1 r diverges as r → 0 . The expectations E

Original:  τ − t and the same remains true asymptotically for (12) as ∆ t → 0 .
Revised: [ X τ | x, t ] are convex linear combinations of the samples x ( n ) from D .

Original: In summary, both drift adjustments (20) and (21) are essentially of the form ( E [ Y | x, t ] − x  ) v − 1∆ t where the term v − 1∆ t diverges as ∆ t → 0 .
Revised: Explicitly, E [ X τ | x, t ] = (cid:80) Nn =1 ω ( x, t ) ( n ) x ( n ) , where the weights ω ( x, t ) ( n ) are the probabilities, under the distributions Q (time-reversal sampling process (3)) and Π (mixture of diffusions process M from

Original:  By construction, the initial weights entering expectation (20) are all equal to 1 / N when X starts from a ﬁxed value x 0 , and are so on average when X 0 is stochastic.
Revised: Section 3.2), of reaching each state x ( n ) at terminal time τ from x at time t . By construction, the initial weights entering expectation (20) are all equal to 1 / N when X starts from a ﬁxed value x 0 , and are so on average when X 0 is stochastic.

Original: The initial weights entering expectation (21) are on average approximately equal to 1 / N , depending on the quality of the approximation P Z ≈ Q τ .
Revised: The initial weights entering expectation (21) are on average approximately equal to 1 / N , depending on the quality of the approximation P Z ≈ Q τ .

Original: Thus E [ Y | X 0 , 0] is an averaging of many samples x ( n ) .
Revised: Thus, E [ X τ | X 0 , 0] is an averaging of many samples x ( n ) .

Original: As time progresses, changes in X t correspond to changes in E [ Y | X t , t ] through changes in the weights ω ( x, t ) ( n ) .
Revised: As time progresses, changes in X t correspond to changes in E [ X τ | X t , t ] through changes in the weights ω ( x, t ) ( n ) .

Original: Eventually all mass concentrates on a single weight ω ( x, t ) ( ∗ ) corresponding to a  sample x ( ∗ ) .
Revised: Eventually all mass concentrates on a single weight ω ( x, t ) ( ∗ ) corresponding to a dataset sample x ( ∗ ) .

Original: Ultimately, the attractor dynamics implied by ( E [ Y | x, t ] − x ) v − 1∆ t drive X t to x ( ∗ ) .
Revised: Ultimately, the attractor dynamics implied by ( E [ X τ | x, t ] − x ) v − 1 r drive X t to x ( ∗ ) .

Original: We provide an illustration in Figure 1 (top). From here onward D ( CIFAR ) stands for the training portion of the CIFAR10 dataset (images are linearly scaled to [0 , 1] ), and Euler(T) corresponds to the Euler scheme (Kloeden & Platen, 1992) applied with T discretization steps.
Revised: We provide an inspection in Figure 1, where D ( CIFAR ) stands for the training portion of the CIFAR10 dataset, and Euler(T) corresponds to the Euler scheme (Kloeden & Platen, 1992) applied with T discretization steps.

Original: In the VESDE and VPSDE of Song et al.
Revised: In the VESDE and VPSDE of Song et al.

Original: we have G ( x, u ) =
Revised: we have G ( x, r ) =

Original: β u I and reversing (21) gives
Revised: β r I and reversing (21) gives

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original:  We can thus take a trained score model s φ ( x, u ) ≈ ∇ x ln q u ( x ) , plug it in (22), and verity the extend to which E
Revised: where ∇ y ln q r ( y ) is the true score . We can thus take a trained score model s φ ( x, r ) ≈ ∇ x ln q r ( y ) , plug it in (22), and verity the extent to which E

Original: [ Y | x, t ] has been approximated. We provide a ﬁrst investigation in Figure 1 (bottom).
Revised: [ X τ | x, t ] has been approximated, see Figure 1.

Original: [Figure]
Revised: [Figure]

Original: We pause for a moment to review the ﬁndings of Figure 1 (see Appendix D for additional related plots).
Revised: We pause for a moment to review the ﬁndings of Figure 1 (see Appendix D for additional related plots).

Original: First of all, we can classify the dynamics of E [ Y | X t , t ] and of the associated weights in three stages.
Revised: Firstly, we can classify the dynamics of E [ X τ | X t , t ] and of the associated weights in three stages.

Original: In the 1 st stage the weights do not move much.
Revised: In the 1 st stage the weights do not move much.

Original: During the 2 nd stage, roughly t ∈ [0 . 4 , 0 . 6] , the weights’ mass gets distributed over a limited number of samples.
Revised: During the 2 nd stage, roughly t ∈ [0 . 4 , 0 . 6] , the weights’ mass gets distributed over a limited number of samples.

Original: Interestingly, the weights’ dynamics are not monotonic.
Revised: Interestingly, the weights’ dynamics are not monotonic.

Original: As time progresses the weights’ mass shifts to different objects from different classes.
Revised: As time progresses the weights’ mass shifts between different objects from different classes.

Original: From the beginning of the 3 rd stage all mass is allocated to a single weight, the terminal image gets decided well in advance of the terminal time.
Revised: From the beginning of the 3 rd stage all mass gets allocated to a single weight, the terminal image is decided well in advance of the terminal time.

Original: These dynamics are suboptimal.
Revised: These dynamics are suboptimal.

Original: We would like to shorten the 1 st stage but it is associated with large values of β t (i.e. quick time passing) which are required to decouple Q τ from P D .
Revised: We would like to shorten the 1 st stage, but it is associated with large values of β t (i.e. quick time passing) which are required to decouple Q τ from P D .

Original: This is an intrinsic limitation of time-reversal approaches.
Revised: This is an intrinsic limitation of time-reversal approaches.

Original: It is also dubious that (partially) sampling multiple objects over t is beneﬁcial for efﬁcient generative modeling when we make use only of the terminal sample.
Revised: It is also dubious that (partially) sampling multiple objects over t is beneﬁcial for efﬁcient generative modeling when we make use only of the terminal sample.

Original: This issue applies to trained models as well, as the 3 rd row of Figure 1 shows.
Revised: This issue applies to trained models as well, as the 3 rd row of Figure 1 shows.

Original: An interesting open question is how to obtain more suitable dynamics, where perhaps class transitions happen rarely.
Revised: An interesting open question is how to obtain more suitable dynamics, where perhaps class transitions happen rarely.

Original: Secondly, E [ Y | X t , t ] provides a denoised representation of X t across the whole 3 rd stage.
Revised: Secondly, E [ X τ | X t , t ] provides a denoised representation of X t across the whole 3 rd stage.

Original: An alternative to the noise removal step applied to X τ in Song et al.
Revised: An alternative to the noise removal step applied to X τ in Song et al.

Original: (2021) is to consider E [Y | X τ  , τ ] as the terminal sample instead.
Revised: (2021) is to consider E [ X τ | X t , t ] as the sampling process instead.

Original: Thirdly, Figuremakes it clear that lowering the number of discretization steps affects generative sampling in multiple ways.
Revised: Thirdly, Figure 1 makes it clear that lowering the number of discretization steps affects generative sampling in multiple ways.

Original: On the one hand the terminal sample X τ is more noisy.
Revised: On the one hand the terminal sample X τ is more noisy.

Original: This is not very surprising: close to τ the drift adjustment is approximately ( x ( ∗ ) − X t)( τ − t )  − 1 , for a sample x ( ∗ ) , which is the drift of a Brownian bridge.
Revised: This is not very surprising: close to τ the drift adjustment is approximately ( x ( ∗ ) − X t ) v − 1 r , which is the drift of a Brownian bridge.

Original: Bridge sampling is notoriously problematic (Bladt et al., 2016).
Revised: Bridge sampling is notoriously problematic (Bladt et al., 2016).

Original: On the other hand larger discretization errors  signiﬁcantly affect the dynamics of E [ Y | X t , t ] resulting in less coherent samples.
Revised: On the other hand larger discretization errors also signiﬁcantly affect the dynamics of E [ X τ | X t , t ] resulting in less coherent samples.

Original: We remark that none of these insights could have been gained by observing X t alone, i.e. the odd rows of Figure 1 (bottom).
Revised: We remark that none of these insights could have been gained by observing X t alone, i.e. the even cells’ rows of Figure 1.

Original: To conclude, (20) and (21) give an additional meaning to “denoising”.
Revised: To conclude, (20) and (21) give an additional meaning to “denoising”.

Original: Neural network approximators need to map from a noisy input X t to an adjustments toward a smoother superimposition of samples.
Revised: Neural network approximators need to map from a noisy input X t to an adjustment toward a smoother superimposition of samples.

Original: The desire to minimize the discrepancy between the smoothness properties of X t and  of E [ Y | X t , t ] motivates the developments of Section 7.
Revised: The desire to minimize the discrepancy between the smoothness properties of X t and that of E [ X τ | X t , t ] motivates the developments of Section 8.

Original:  T RANSPORTS A PPROXIMATION
Revised:  T RANSPORTS A PPROXIMATION

Original: As in Song et al.
Revised: As in Song et al.

Original: (2021), computing the multiplicative drift adjustment A ( x t , t ) requires O ( N ) operations.
Revised: (2021), computing the multiplicative drift adjustment A ( x t , t ) requires O ( N ) operations.

Original: In this Section we introduce three training objectives for which unbiased and scalable, i.e.
Revised: In this Section we introduce three training objectives for which unbiased and scalable, i.e.

Original: O (1) with respect to N , MC estimators can be immediately derived.
Revised: O (1) with respect to N , MC estimators can be immediately derived.

Original: The ﬁrst training objective applies only to A ( x t , t, x 0 ) .
Revised: The ﬁrst training objective applies only to A ( x t , t, x 0 ) .

Original: It relies on the identity (Appendix A)
Revised: It relies on the identity (Appendix A)

Original: [Equation]
Revised: [Equation]

Original: It is advantageous to consider the right hand side of (23) because from (8) we know that π t | 0 ( x t | x 0 ) has mixture representation.
Revised: It is advantageous to consider the right-hand side of (23) because from (8) we know that π t | 0 ( x t | x 0 ) has mixture representation.

Original: As in Song et al.
Revised: As in Song et al.

Original: (2021), we can rely on Vincent (2011) to obtain a scalable objective to train a neural network approximator s φ ( x t , t )
Revised: (2021), we can rely on Vincent (2011) to obtain a scalable objective to train a neural network approximator s φ ( x t , t )

Original: ≈ ∇ x t ln π t | 0 ( x t | x 0 ) , i.e.
Revised: ≈ ∇ x t ln π t | 0 ( x t | x 0 ) , i.e.

Original: [Equation]
Revised: [Equation]

Original: FD , DBMT
Revised: where J t = E [ (cid:107)∇

Original: [Equation]
Revised: [Equation]

Original: =
Revised: |

Original: [Equation]
Revised: [Equation]

Original: 24) where r t = E [ (cid:107)∇ X t ln p t | 0 ,τ ( X t | x 0 , X τ ) (cid:107) 2 ] − 1 is a regularization term.
Revised: is a regularization term.

Original: The remaining training objectives rely on the identities (20) and (21).
Revised: The remaining training objectives rely on the identities (20) and (21).

Original: The goal is directly approximate the expectations of (20) and (21) which, as in Section 5, we denote with a generic E [ Y | x, t ] .
Revised: The goal is directly approximate the expectations of (20) and (21) which, as in Section 5, we denote with a generic E [ X τ | x, t ] .

Original: That is, we aim to train a neural network approximator s φ ( x, t )
Revised: That is, we aim to train a neural network approximator s φ ( x, t )

Original: ≈ E
Revised: ≈ E

Original: [ Y | x, t ] .
Revised: [ X τ | x, t ] .

Original: As conditional expectations are mean squared error minimizers, suitable objectives for the expectation terms of (20) and (21) are
Revised: As conditional expectations are mean squared error minimizers, suitable objectives for the expectation terms of (20) and (21) are

Original: [Equation]
Revised: [Equation]

Original: CE , DBMT

Original: [Equation]
Revised: [Equation]

Original: In Table 1 we summarize the operations needed to implement the plain MC estimators for the four objectives considered in this work.
Revised: In Table 1 we summarize the operations needed to implement the plain MC estimators for the four objectives considered in this work.

Original: We reference where to ﬁnd the required quantities for SDEs (11) and (12).
Revised: We reference where to ﬁnd the required quantities for SDEs (11) and (12).

Original: The MC estimators for the Fisher divergence losses L FD , ∗ involve multiplications by Γ − 1 (by (15) and (17)).
Revised: The MC estimators for the Fisher divergence losses L FD , ∗ involve multiplications by Γ − 1 (by (15) and (17)).

Original: Moreover, computing the drift adjustment at generation time requires multiplications by Γ .
Revised: Moreover, computing the drift adjustment at generation time requires multiplications by Γ .

Original: In Section 7 we discuss how to manage the computational burden.
Revised: In Section 8 we discuss how to manage the computational burden.

Original: An appealing property of L CE , ∗ is that computing the drift adjustment only requires the application of simple scalar functions (see (20) and (21)), and that their MC estimators only requires sampling operations.
Revised: An appealing property of L CE , ∗ is that computing the drift adjustment only requires the application of simple scalar functions (see (20) and (21)), and that their MC estimators only requires sampling operations.

Original: A further advantage of L CE , ∗ is that no regularization is required.
Revised: A further advantage of L CE , ∗ is that no regularization is required.

Original: In contrast, in the absence of regularization terms, L FD , ∗ are divergent for t
Revised: In contrast, in the absence of regularization terms, L FD , ∗ are divergent for t

Original: ≈ τ due to the term v − 1∆ t discussed in Section 5.
Revised: ≈ τ due to the term v − 1 r (Section 5).

Original: [Equation]
Revised: [Equation]

Original: Evaluation
Revised: Evaluation

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table] 
Revised: [Table] [Equation] [Table] [Equation]

Revised: [Table]

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Original:   N ON -D ENOISING D IFFUSIONS
Revised:  DBMT O VERVIEW AND N UMERICAL E XPERIMENT

Revised: In this section we ﬁnalize the DBMT construction, putting together the results of Sections 3, 4 and 6. The unconstrained SDE follows (11) or (12). It remains to choose the mixing distribution Π 0 ,τ .

Revised: The marginal Π τ needs to match P D , but there is ﬂexibility in the choice of Π 0 | τ .

Revised: Song et al.

Revised: (2021) derived a random ordinary differential equation (RODE) matching the marginal distribution of a generative SDE, leading to faster sampling and to likelihood evaluation.

Revised: RODE-matching requires Π 0 to have density.

Revised: A natural implementation is given by the factorial distribution Π 0 ,τ = P Z ⊗ P D with P Z = N D (0 , Γ) and the unconstrained SDE following (12) with α t = 1 / 2 which preserves

Revised: P Z .

Revised: If instead the DBMT starts from a ﬁxed value x 0 , i.e. Π 0 ,τ = δ x 0 ⊗ P D , we can choose x 0 = 1 / N (cid:80)

Original: {+Nn =1 x ( n ) a (0 , τ ) − 1 to remove the drift adjustment at t = 
+}
Revised: {+Nn =1 x ( n ) a (0 , τ ) − 1 to remove the drift adjustment at t = 
+}

Revised: (see (20)) and reduce the work required to transport x 0 to P D .

Revised: Finally, the use of non-factorial distributions can lead to a more efﬁcient implementation, by linking the initial distribution to P D .

Revised: The training steps for the simplest objective (25) of Section 6 are reported in Algorithm 1.

Revised: Batch size is assumed to be 1 to ease the description.

Revised: It is also assumed that Π 0 ,τ is factorial, otherwise the obvious modiﬁcation applies to line 2 (Algorithm 2 is unaffected) where the endpoints are sampled.

Revised: At line 3 a random central time and the corresponding state are sampled.

Revised: The function optimizationstep implements a step of stochastic gradient descent update based on the loss L .

Revised: The corresponding sampling algorithm is reported in Algorithm 2 where the Euler(T) discretization is assumed in line 6.

Revised: P t | 0 ,τ ( dx t | X 0 , X τ ) , a ( t, τ ) , v ( t, τ ) , β t are deﬁned in Section 4.

Revised: Section 8 shows how to sample efﬁciently from P t | 0 ,τ ( dx t | X 0 , X τ ) and N D (0 , Γ) in computer vision applications.

Revised: Input: P D , P Z , SDE (11) or (12), NN s φ ( x, t )

Revised: Output: trained s φ ( x, t ) 1: repeat 2:

Revised: X τ ∼ P D , X 0 ∼ P Z 3: t ∼ U [0 , τ ) , X t ∼ P t | 0 ,τ ( dx t | X 0 , X τ ) 4:

Revised: L ← (cid:13)(cid:13)

Revised: X τ − s φ ( X t , t ) (cid:13)(cid:13)5: φ ← optimizationstep ( φ, L ) 6: until convergence

Revised: Input: P Z , SDE (11) or (12), trained s φ ( x, t )

Revised: Output: Discretized path X 0: T 1: X 0 ∼ P Z 2: for s = 1 , . .

Revised: , T do 3: t ← ( s − 1) τT , x ← X s −4: u s ← β t (cid:16) 1 a ( t,τ ) s φ ( x, t ) − x (cid:17) a 2 ( t,τ ) v ( t,τ ) 5:

Revised: E s ∼ N D (0 , Γ) 6:

Revised: X s ← x +( f ( x, t )+ u s )

Revised: τT + g ( x, t ) (cid:112)

Revised: τT

Revised: E s 7: end for

Original: {+We consider a toy numerical example with P Z = P D = 1 / 3 ( δ − 2 + δ 
+}
Revised: {+We consider a toy numerical example with P Z = P D = 1 / 3 ( δ − 2 + δ 
+}

Revised: + δ 2 )

Revised: , D = τ = 1 .

Revised: The unconstrained SDE follows the standard Brownian motion.

Revised: We consider two mixing distributions: independent mixing Π ⊥⊥ 0 , 1 where X 0 and X 1 are independent and fully dependent mixing Π =0 , 1 where

Original: {+X 
+}
Revised: {+X 
+}

Revised: = X 1 .

Revised: The results are reported in Figure 2.

Original: In computer vision applications, color images of resolution H × W have S =
Revised: For both couplings the correct terminal distribution P D is recovered, as can be seen by taking the row-wise sum of the transition matrices. The initialterminal distribution of X solving (8), which realizes the DBMT, is different from the corresponding mixing distribution Π 0 , 1 which is realized by the mixture process M . Π ⊥⊥ 0 , 1 results in a transition matrix of equal entries 1 / 9 , Π ⊥⊥ 0 , 1 results in a diagonal transition matrix of equal diagonal entries 1 / 3 .

Original: HW pixels and correspond to D = 3 S .
Revised: [Figure]

Original: Even in the case of datasets featuring images with the very modest resolution of 32 × 32 pixels, such as CIFAR10, the resulting dimensionality is already D = 3072 .
Revised: [Figure] [Figure] [Figure]  N ON -D ENOISING D IFFUSIONS

Original: A direct modeling approach allowing an arbitrary covariance matrix Γ in (11) and (12) would require to store a D × D matrix Γ , and perform its Cholesky (or equivalent) decomposition with cost O ( D 3 ) .
Revised: In computer vision applications, images of resolution H × W corresponds to D = 3 HW . The use of an arbitrary covariance matrix Γ in (11) and (12) requires its Cholesky (or equivalent) decomposition with cost O ( D 3 ) .

Original: As the resolution increases the computational burden gets intractable very quickly.
Revised: As the resolution increases the computational burden gets intractable very quickly.

Original: Indeed, to the best of the authors knowledge, all prior DDPM literature only considers independent transitions, that is Γ
Revised: Indeed, to the best of the authors’ knowledge, all prior DDPM literature only considers independent transitions, that is Γ

Original: = I .
Revised: = I .

Original: We propose to view SDEs (11) and (12) as corresponding to the space discretization on a H × W grid of a spatio-temporal process deﬁned over the spatial domain [0 , 1] 2 .
Revised: We suggest to view SDEs (11) and (12) as corresponding to the space discretization on an H × W grid of a spatio-temporal process deﬁned over the spatial domain [0 , 1] 2 .

Original: This different perspective can be formalized (Prato & Zabczyk, 2014) giving raise to a stochastic partial differential equation (SPDE) featuring derivatives with respect to time and space (despite the misleading nomenclature, both SDEs and SPDEs solutions are actually nowhere differentiable with respect to time, and often with respect to space, and are interpreted as solutions to stochastic integral equations).

Original: Doing this rigorously is not trivial, but for our needs an heuristical application of the principle sufﬁces.

Original: Consider the Euler discretization of (12): X t +∆ t =
Revised: Consider the Euler discretization of (12): X t +∆ t =

Original: X t + α t β t X t ∆ t + √ ∆ t E t , where E t ∼ N D (0 , Γ) . The idea is  adopt a functional perspective: X ( t + ∆ t, s ) = X ( t, s ) + α t β t X ( t, s )
Revised: X t + α t β t X t ∆ t + √ ∆ t E t , where E t ∼ N D (0 , Γ) , the idea is to adopt a functional perspective: X ( t +∆ t, s )

Original: ∆ t + √ ∆ t E ( t, s ) where s ∈ [0 , 1]deﬁnes space coordinates. That is, both X ( t ) and E ( t ) at each time t are random processes over [0 , 1] 2 . We assume the innovations E ( t ) to be distributed as a Gaussian process (GP) for each t . To start with, it is necessary to model the GP.
Revised: =

Original: For simplicity, we assume a factorial distribution over the channels and an isotopic stationary covariance function. We rely on the semi-variogram approach (Cressie, 1993) to compare how different covariance functions ﬁt D ( CIFAR ) . A semivariogram is a measure of dependency across space. In the case of isotropic stationary covariances it simpliﬁes to a scalar function of the Euclidean distance between points: γ ( (cid:107) ∆ s(cid:107) )=
Revised: X ( t, s )+ α t β t X ( t, s )

Original: E [(∆ x s ) 2 ] /with ∆ x s = x s +∆ s − x s . The rate of decrease of γ ( (cid:107) ∆  s(cid:107) ) toward 0 as (cid:107) ∆ s (cid:107) → 0 gives a measure of the strength of the spatial dependency, i.e. the smoothness of the spatial process. Semi-variograms corresponding to different covariance functions are here ﬁtted to their empirical counterparts via a weighted minimum-least-squares procedure (Cressie, 1993). Figure 2 illustrates the exponential and RBF semi-variogram ﬁts for two images of D ( CIFAR ) .
Revised: ∆ t + √ ∆ t E ( t, s ) where s ∈ [0 , 1] 2 deﬁnes space coordinates.

Original: The exponential covariance, which corresponds to rougher paths, provides a much better ﬁt to the shown samples. This results is consistent across D ( CIFAR )  .We remark that Γ = I corresponds to a pure white-noise process with a perfectly ﬂat semi-variogram which would clearly result in a very poor ﬁt to the empirical semi-variograms shown in Figure 2.
Revised: That is, both X ( t ) and E ( t ) at each time t are random processes over [0 , 1] 2 .

Original: Based on these ﬁndings, we model the innovations of each image channel as a GP with exponential covariance function with length-scale θ = 0 . 205 , the median estimated value (Figure 2 (right)). We match the marginal variance to that of D ( CIFAR ) , σ 2 = 0 .p ( )
Revised: We assume the innovations E ( t ) to be a Gaussian process (GP) for each t .

Original: As the GPs E ( t ) are deﬁned on a 2D domain we can leverage on scalable inference techniques fom spatial statistics.
Revised: As the GPs E ( t ) are deﬁned on a 2D domain we can leverage on scalable inference techniques from spatial statistics.

Original: As an example, we consider the circulant embedding method (CEM) (Wood &
Revised: As an example, we consider the circulant embedding method (CEM) (Wood &

Original: Chan, 1994; Dietrich & Newsam, 1997) which exploits a connection with the fast Fourier transform (FFT).
Revised: Chan, 1994; Dietrich & Newsam, 1997) which exploits a connection with the fast Fourier transform (FFT).

Original: See Appendix C for a cursory review of the CEM.
Revised: See Appendix C for a cursory review of the CEM.

Original: Consider a H × W uniform grid S of size S =
Revised: Consider an H × W uniform grid S of size S =

Original: HW discretizing [0 , 1] 2 , i.e. the support of images.
Revised: HW discretizing [0 , 1] 2 , i.e. the support of images.

Original: For a stationary covariance function, such as the exponential covariance, the CEM samples E ( t ) on S with cost O ( S ln( S )) .
Revised: For a stationary covariance function the CEM samples E ( t ) on S with cost O ( D ln( D )) .

Original: This is close to the O ( S ) cost of sampling from a pure white-noise process, and compares very favourably to the O ( S 3 ) cost that a Cholesky decomposition entails.
Revised: This is close to the O ( D ) cost of sampling from a pure white-noise process, and compares very favorably to the O ( D 3 ) cost of a Cholesky decomposition.

Original: One limitation of CEM is that generated samples, while always Gaussian, might not have the correct covariances.
Revised: One limitation of CEM is that generated samples, while always Gaussian, might not have the correct covariances.

Original: Whether this happens, and in that case the quality of the approximation, depends on the covariance function.
Revised: Whether this happens, and in that case the quality of the approximation, depends on the covariance function.

Original: For the estimated exponential covariance function with length-scale θ = 0 .
Revised: In Appendix C we select and ﬁt an isotropic GP to the microscale properties of D ( CIFAR ) .

Original: 205 sampling is exact.
Revised: For this estimated GP sampling is exact.

Original: Figure 3 shows samples from a pure white-noise GP, i.e. Γ =
Revised: Figure 3 shows samples from a pure white-noise GP, i.e. Γ =

Original: I , (1 st row) and from the estimated GP using CEM (2 nd row).
Revised: I , (1 st row) and from the ﬁtted GP using CEM (2 nd row).

Original: As noted in Section 6, sampling is enough to implement the MC estimators for L CE , ∗ , but the MC estimators and drift adjustments for L CE , ∗ involve additional matrix multiplications by Γ and Γ − 1 .
Revised: As noted in Section 6, sampling is enough to implement the MC estimators for L CE , ∗ , but the MC estimators and drift adjustments for L CE , ∗ involve additional matrix multiplications by Γ and Γ − 1 .

Original: The CEM allows to compute these at the same O( S ln( S ))
Revised: The CEM allows to compute these at the same O

Original:  cost if we deﬁne the GP E ( t ) on a 2D torus (Rue & Held, 2005, Chapter 2.1).
Revised: ( D ln( D )) cost if we deﬁne the GP E ( t ) on a 2D torus (Rue & Held, 2005, Chapter 2.1).

Original: This corresponds to introducing dependencies between “opposing” boundaries of [0 , 1] 2 .
Revised: This corresponds to introducing dependencies between “opposing” boundaries of [0 , 1] 2 .

Original: Figure 3 (3 rd row) shows some samples, in the highlighted patch the opposing-boundaries dependency is evident.
Revised: Figure 3 (3 rd row) shows some samples, in the highlighted patch the opposing-boundaries dependency is evident.

Original: Either way, all samples from the 2 nd and 3 rd rows of 
Revised: Either way, all samples from the 2 nd and 3 rd rows of Figure 3 match the smoothness properties of D ( CIFAR ) .

Original: F INAL  C ONSIDERATIONS
Revised:  C ONCLUSIONS

Original: The DBMT construction of Section 3 is general, and applies to essentially arbitrary initial-terminal distributions Π 0 ,τ and SDE coefﬁcients f ( · ) , σ ( · ) .
Revised: The DBMT construction of Section 3 is exact.

Original: The SDE class considered in Section 4 is tractable as it results in linear in the narrow-sense (Kloeden & Platen, 1992) SDE bridges.
Revised: The SDE class of Section 4 is tractable as it results in linear diffusion bridges.

Original: Within this SDE class, the time-space factorization of the diffusion coefﬁcient g ( x, t ) = √ β t Γ 1 / 2 separates modeling concerns. β t corresponds to a non-linear time-wrapping which impacts the transport’s dynamics. An analysis for the DTRT of Song et al. (2021) is carried out in Section 5.
Revised: The time-space factorization of the diffusion coefﬁcient g ( x, t ) = √ β t Γ 1 /separates

Original: As shown in Section 7, Γ can be efﬁciently modeled by ﬁtting the micro-scale properties of P D .The spatial modeling could deﬁnitely be improved by introducing across-channel dependencies.
Revised: modeling concerns: β t corresponds to a time-wrapping, Γ can be efﬁciently modeled by ﬁtting the microscale properties of P D .

Original: The widespread availability of GPU-accelerated FFT implementations stands behind our focus on the CEM.
Revised: Availability of GPU-accelerated FFT implementations motivates our focus on the CEM.

Original: Within this context, we stress the importance of the connection with spatio-temporal modeling.

Original: Alternative scalable approaches abound, from Gaussian Markov Random Fields (Rue & Tjelmeland, 2002; Rue, 2001) to Karhunen–Loève expansions (Betz et al., 2014).
Revised: Alternative scalable approaches abound, from Gaussian Markov Random Fields (Rue & Tjelmeland, 2002; Rue, 2001) to Karhunen–Loève expansions (Betz et al., 2014).

Original: Π τ needs to match P D , but the choice of Π 0 | τ has so far received little attention. An interesting result from Song et al. (2021) is the derivation of a random ordinary differential equation (RODE) matching the marginal distribution of a given SDE. This result requires the SDE to start from an initial distribution with a density.
Revised: It remains to apply the results of this work to perform an empirical benchmarking.

Original: From this perspective, a non degenerate Π 0 is preferable as the application ofthe RODE approach leads to faster sampling and to likelihood evaluation. A natural implementation is then Π 0 = N D (0 , Γ) for (12) with α t = 1 / 2 which preserves Π 0 . We now consider a DBMT starting from a ﬁxed value x 0 , i.e. Π 0 =
Revised: Sectiondevelops three novel training objectives, two of which with desirable properties compared to the objective of Song et al.

Original: δ x 0 . In this case, the goal of minimizing discretization errors suggests to choose the initial value that cancels the drift adjustment at t = 0 . This is achieved by x 0 = 1 / N (cid:80) Nn =1 x ( n ) a (0 , τ ) − 1 , see (20). Finally, a dependent Π 0 | τ can reduce the effort required to transport samples from Π 0 to Π τ .
Revised: (2021), especially for non-factorial transitions.

Original: As an example, we could ﬁrst sample a length-scale from the histogram of Figure 2 (right), and then condition both Π 0 and Π τ to match this length-scale.
Revised: We remark the simplicity of the proposed DBMT approach (Algorithms 1 and 2) compared to alternatives grounded in the Schrödinger bridge problem (De Bortoli et al., 2021;

Original: This paper has been entirely dedicated to the methodological study of generative modeling through diffusion processes. The next logical step is to apply the insights contained in this work to the actual training of neural networks. Three novel training objectives have been proposed in Section 6, two of which offering desireable properties compared to the Fisher divergence objective of Song et al. (2021), especially in the case of non-factorial transitions.
Revised: Wang et al., 2021; Vargas et al., 2021).

Original: We argue that the increased understanding of the target mappings of (20) and (21) can, and should, guide the development of neural network architectures more closely matching the target structure compared to the U-Net default choice.
Revised: The understanding of the target mappings ((20) and (21)) can guide the development of neural networks more closely matching the target structure compared to the U-Net default choice.

Original: https://github.com/?
Revised: https://github.com/?

Original: links to the code accompanying this paper which is made available under the MIT license (link to be shared with a comment directed to the reviewers and area chairs).
Revised: links to the code accompanying this paper which is made available under the MIT license.

Original: A T HEORETICAL F RAMEWORK A. A SSUMPTIONS
Revised: A T HEORETICAL F RAMEWORK A. A SSUMPTIONS

Original: Assumption 1 (SDE solution) .
Revised: Assumption 1 (SDE solution) .

Original: A given D -dimensional SDE ( f, g ) with associated initial distribution
Revised: A given D -dimensional SDE ( f, g ) with associated initial distribution

Original: V 0 and integration interval [0 , τ ] admits an unique strong solution on [0 , τ ] .
Revised: V 0 and integration interval [0 , τ ] admits a unique strong solution on [0 , τ ] .

Original: Assumption 1 can be checked through the application of the standard existence and uniqueness theorems for SDE solutions.
Revised: Assumption 1 can be checked through the application of the standard existence and uniqueness theorems for SDE solutions.

Original: Of particular relevance to our setting is the formulation of Krylov (1995,
Revised: Of particular relevance to our setting is the formulation of Krylov (1995,

Original: Chapter 5, Theorem 1) that limits the monotonicity requirement to, informally speaking, drifts that pull the process toward inﬁnities.
Revised: Chapter 5, Theorem 1) that limits the monotonic requirement to, informally speaking, drifts that pull the process toward inﬁnities.

Original: Assumption 2 (SDE density) .
Revised: Assumption 2 (SDE density) .

Original: A given D -dimensional SDE ( f, g ) with associated initial distribution
Revised: A given D -dimensional SDE ( f, g ) with associated initial distribution

Original: V 0 and integration interval [0 , τ ] admits a marginal / transition density on (0 , τ ) with respect to the
Revised: V 0 and integration interval [0 , τ ] admits a marginal / transition density on (0 , τ ) with respect to the

Original: D -dimensional Lebesgue measure that uniquely satisﬁes the Fokker-Plank / Kolmogorov-Forward partial differential equation (PDE).
Revised: D -dimensional Lebesgue measure that uniquely satisﬁes the Fokker-Plank / Kolmogorov-forward partial differential equation (PDE).

Original: We refer to Särkkä & Solin (2019, Chapter 5) and to Karatzas & Shreve (1996, Chapter 5.7) for connections between SDEs and PDEs.
Revised: We refer to Särkkä & Solin (2019, Chapter 5) and to Karatzas & Shreve (1996, Chapter 5.7) for connections between SDEs and PDEs.

Original: All theoretical results of this work rely on simple algebraic manipulations and re-arrangements of quantities of interest.
Revised: All theoretical results of this work rely on simple algebraic manipulations and re-arrangements of quantities of interest.

Original: The main complication stems from the need to justify differentiation and integration exchanges, i.e. exchange of limits.
Revised: The main complication stems from the need to justify differentiation and integration exchanges, i.e. exchange of limits.

Original: Assumption 3 (exchange of limits) .
Revised: Assumption 3 (exchange of limits) .

Original: We assume that limits exchanges are justiﬁed in the steps marked with ( (cid:63) ) and ( (cid:63)(cid:63) ) .
Revised: We assume that limits exchanges are justiﬁed in the steps marked with ( (cid:63) ) and ( (cid:63)(cid:63) ) .

Original: Similarly, various steps in the derivations involve considering fractional quantities with densities appearing in the denominators.
Revised: Similarly, various steps in the derivations involve considering fractional quantities with densities appearing in the denominators.

Original: Assumption 4 (positivity) .
Revised: Assumption 4 (positivity) .

Original: For a given stochastic process, all ﬁnite-dimensional densities, conditional or not, are strictly positive.
Revised: For a given stochastic process, all ﬁnite-dimensional densities, conditional or not, are strictly positive.

Original: Assumption 4 is easy to verify.
Revised: Assumption 4 is easy to verify.

Original: We resorted to the practical but somewhat unsatisfactory formulation of Assumption 3 because in full generality it is complicated to give easily checkable conditions.
Revised: We resorted to the practical but somewhat unsatisfactory formulation of Assumption 3 because in full generality it is complicated to give easy to check conditions.

Original: We just note that when Π T =
Revised: We just note that when Π T =

Original: P D , the limit exchange marked with ( (cid:63) ) is always justiﬁed.
Revised: P D , the limit exchange marked with ( (cid:63) ) is always justiﬁed.

Original: So are the limits exchanges marked with ( (cid:63)(cid:63) ) when in addition Π 0 puts all the mass to a ﬁxed initial value, or (by direct veriﬁcation) when Π 0 is Gaussian for the SDE class of Section 4.
Revised: So are the limits exchanges marked with ( (cid:63)(cid:63) ) when in addition Π 0 puts all the mass to a ﬁxed initial value, or (by direct veriﬁcation) when Π 0 is Gaussian for the SDE class of Section 4.

Original: Thorough this paper, both in the main text and in the proofs that follow, it is supposed that Assumptions 1, 2 and 4 are satisﬁed by SDEs (1), (6) and (7).
Revised: Thorough this paper, both in the main text and in the proofs that follow, it is supposed that Assumptions 1, 2 and 4 are satisﬁed by SDEs (1), (6) and (7).

Original: This is the case for the SDE class of Section 4, i.e. (11) and (12), for any Π 0 with ﬁnite variance.
Revised: This is the case for the SDE class of Section 4, i.e. (11) and (12), for any Πwith ﬁnite variance.

Original: Remark:
Revised: Remark:

Original: For ease of exposition it is assumed thorough this paper that all diffusions take values in the state space R D .
Revised: For ease of exposition it is assumed thorough this paper that all diffusions take values in the state space R D .

Original: There is no impediment in extending the presented results to the case of diffusions taking values in a subset X ⊂ R D .
Revised: There is no impediment in extending the presented results to the case of diffusions taking values in a subset X ⊂ R D .

Original: The obvious changes to Assumptions 1 to 4 apply, the proofs carry over without substantial modiﬁcations.
Revised: The obvious changes to Assumptions 1 to 4 apply, the proofs carry over without substantial modiﬁcations.

Original: This extension could be of practical interest as images are often represented as ﬂoating point values in [0 , 1] .
Revised: This extension could be of practical interest as images are often represented as ﬂoating point values in [0 , 1] .

Original: A. S TATEMENT AND P ROOF OF  M IXTURE OF D IFFUSIONS T HEOREM
Revised: A. S TATEMENT AND P ROOF OF D IFFUSION M IXTURE R EPRESENTATION T HEOREM

Original: Theorem 2 (Mixture of diffusions) .
Revised: Theorem 2 (Diffusion mixture representation) .

Original: Consider the family of D -dimensional SDEs on t ∈ [0 , τ ] indexed by λ ∈ Λ
Revised: Consider the family of D -dimensional SDEs on t ∈ [0 , τ ] indexed by λ ∈ Λ

Original: [Equation]
Revised: [Equation]

Original: (27) where the initial distributions
Revised: (27) where the initial distributions

Original: V λ 0 and the BMs W λt are all independent.
Revised: V λ 0 and the BMs W λt are all independent.

Original: Let ν λt , t ∈ (0 , τ ) denote the marginal density of X λt .
Revised: Let ν λt , t ∈ (0 , τ ) denote the marginal density of X λt .

Original: For a generic mixing distribution L on Λ , deﬁne the mixture marginal density ν t for t ∈ (0 , τ ) and the mixture initial distribution V 0 by
Revised: For a generic mixing distribution L on Λ , deﬁne the mixture marginal density ν t for t ∈ (0 , τ ) and the mixture initial distribution V 0 by

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Consider the D -dimensional SDE on t ∈ [0 , τ ] deﬁned by
Revised: Consider the D -dimensional SDE on t ∈ [0 , τ ] deﬁned by

Original: [Equation]
Revised: [Equation]

Original: It is assumed that all diffusion processes X λ and the diffusion process X solving (29) satisfy the regularity assumptions Assumptions 1, 2 and 4 and that Assumption 3 holds.
Revised: It is assumed that all diffusion processes X λ and the diffusion process X solving (29) satisfy the regularity assumptions Assumptions 1, 2 and 4 and that Assumption 3 holds.

Original: Then the marginal distribution of the diffusion X is ν t .
Revised: Then the marginal distribution of the diffusion X is ν t .

Original: Proof of Theorem 2.
Revised: Proof of Theorem 2.

Original: We start by establishing that the law of X is indeed given by the solution of (29).
Revised: We start by establishing that the law of X is indeed given by the solution of (29).

Original: In this proof we make use of the following notation: for f scalar-valued ( f ) t = ddt f , for a vector-valued ( a ) x = (cid:80) D i =1 d dx i a , for A matrix-valued ( A ) xx
Revised: In this proof we make use of the following notation: for f scalar-valued ( f ) t = ddt f , for a vector-valued ( a ) x = (cid:80) D i =1 d dx i a , for A matrix-valued ( A ) xx

Original: = (cid:80) D i,j = d 2 dx i dx j A .
Revised: = (cid:80) D i,j = d 2 dx i dx j A .

Original: This notation allows for a compact representation of PDEs reminiscent of the 1-dimensional setting.
Revised: This notation allows for a compact representation of PDEs reminiscent of the 1-dimensional setting.

Original: Then, for 0 < t < τ we have that
Revised: Then, for 0 < t < τ we have that

Original: [Equation]
Revised: [Equation]

Original: ν
Revised: ν

Original: [Equation]
Revised: [Equation]

Original: µ λ ( x, t ) ν λ ( x, t ) ν
Revised: µ λ ( x, t ) ν λ ( x, t ) ν

Original: ( x, t ) ν
Revised: ( x, t ) ν

Original: ( x, t ) (cid:19) x
Revised: ( x, t ) (cid:19) x

Original: + (cid:18) σ λ ( x, t ) ν λ ( x, t ) ν
Revised: + (cid:18) σ λ ( x, t ) ν λ ( x, t ) ν

Original: ( x, t ) ν
Revised: ( x, t ) ν

Original: ( x, t ) (cid:19)
Revised: ( x, t ) (cid:19)

Original: xx
Revised: xx

Original: L ( dλ ) =
Revised: L ( dλ ) =

Original: (cid:18)(cid:90) Λ µ λ ( x, t )
Revised: (cid:18)(cid:90) Λ µ λ ( x, t )

Original: ν λ ( x, t ) ν
Revised: ν λ ( x, t ) ν

Original: ( x, t )
Revised: ( x, t )

Original: L ( dλ ) ν ( x, t ) (cid:19) x
Revised: L ( dλ ) ν ( x, t ) (cid:19) x

Original: + (cid:18)(cid:90) Λ σ λ ( x, t )
Revised: + (cid:18)(cid:90) Λ σ λ ( x, t )

Original: ν λ ( x, t ) ν
Revised: ν λ ( x, t ) ν

Original: ( x, t )
Revised: ( x, t )

Original: L ( dλ ) ν ( x, t ) (cid:19) xx .
Revised: L ( dλ ) ν ( x, t ) (cid:19) xx .

Original: ( (cid:63)(cid:63) )
Revised: ( (cid:63)(cid:63) )

Original: The second line is an exchange of limits, the third line is the application of the Fokker-Plank PDEs for the collection of processes X λ , the fourth line is a rewriting in terms of ν ( y, t ) , the last line is another exchange of limits.
Revised: The second line is an exchange of limits, the third line is the application of the Fokker-Plank PDEs for the collection of processes X λ , the fourth line is a rewriting in terms of ν ( y, t ) , the last line is another exchange of limits.

Original: The result follows by noticing that the last line gives the Fokker-Plank representation of (29).
Revised: The result follows by noticing that the last line gives the Fokker-Plank representation of (29).

Original: A. D RIFT A DJUSTMENTS
Revised: A. D RIFT A DJUSTMENTS

Original: Limitedly to this section, we lighten the notation by removing subscripts from probability measures and densities.
Revised: Limitedly to this section, we lighten the notation by removing subscripts from probability measures and densities.

Original: The missing time points can be inferred without ambiguity from the variables.
Revised: The missing time points can be inferred without ambiguity from the variables.

Original: A.3. D RIFT A DJUSTMENT I DENTITIES FOR C ONSTANT I NITIAL V ALUE
Revised: A.3. D RIFT A DJUSTMENT I DENTITIES FOR C ONSTANT I NITIAL V ALUE

Original: First identity:
Revised: First identity:

Original: [Equation]
Revised: [Equation]

Original: Second identity:
Revised: Second identity:

Original: [Equation]
Revised: [Equation]

Original: A.3. D RIFT A DJUSTMENTS AS E XPECTATIONS
Revised: A.3. D RIFT A DJUSTMENTS AS E XPECTATIONS

Original: To establish (20) notice that from (14) we have
Revised: To establish (20) notice that from (14) we have

Original: [Equation]
Revised: [Equation]

Original: To establish (21) note that from (15) we have
Revised: To establish (21) note that from (15) we have

Original: [Equation]
Revised: [Equation]

Original: B SDE S C LASS F ORMULAS
Revised: B SDE S C LASS F ORMULAS

Original: The transition densities of (9) and (10) are given respectively by
Revised: The transition densities of (9) and (10) are given respectively by

Original: [Equation]
Revised: [Equation]

Original: Here we used the notation f t : τ = 1 τ − t (cid:82)
Revised: Here we used the notation f t : τ = 1 τ − t (cid:82)

Original: τ t f u du , i.e. f t : τ is the average value of a function f u on the interval [ t, τ ] .
Revised: τ t f u du , i.e. f t : τ is the average value of a function f u on the interval [ t, τ ] .

Original: The time-homogenous case of (10), where α t : τ = α , is thus immediately recovered.
Revised: The time-homogenous case of (10), where α t : τ = α , is thus immediately recovered.

Original: The scalar functions a bm ( t, τ ) , a ou ( t, τ ) , v bm ( t, τ ) and v ou ( t, τ ) are given by
Revised: The scalar functions a bm ( t, τ ) , a ou ( t, τ ) , v bm ( t, τ ) and v ou ( t, τ ) are given by

Original: [Equation]
Revised: [Equation]

Original: τ .
Revised: τ .

Original: The scalar functions v br (0 , t, τ ) , a br (0 , t, τ ) and a br (0 , t, τ ) are given by
Revised: The scalar functions v br (0 , t, τ ) , a br (0 , t, τ ) and a br (0 , t, τ ) are given by

Original: [Equation]
Revised: [Equation]

Original: The scalar functions β ve ,u and β vp ,u are given by
Revised: The scalar functions β ve ,r and β vp ,r are given by

Original: [Equation]
Revised: [Equation]

Original: The constants σ min , σ max , ¯ β min , ¯ β max depend in part on the dataset considered, but are consistently chosen to have β ve ,u , β vp ,u small for u ≈ 0 and large for u ≈ τ .
Revised: The constants σ min , σ max , ¯ β min , ¯ β max depend in part on the dataset considered, but are consistently chosen to have β ve ,r , β vp ,r small for r ≈ 0 and large for r ≈ τ .

Original: The approximating distributions P Z in Song et al.
Revised: The approximating distributions P Z in Song et al.

Original: (2021) are P ve Z = N D (0 , Iσ 2max ) for VESDE,
Revised: (2021) are P ve Z = N D (0 , Iσ 2max ) for VESDE,

Original: P vp Z = N D (0 , I ) for VPSDE.
Revised: P vp Z = N D (0 , I ) for VPSDE.

Original: C A DDITIONAL M ATERIAL C. C LOSELY R ELATED W ORK
Revised: C A DDITIONAL M ATERIAL C. C LOSELY R ELATED W ORK

Original: A work closely related to the present paper is that of Wang et al.
Revised: A work closely related to the present paper is that of Wang et al.

Original: (2021) as it similarly avoids the time-reversal construction.
Revised: (2021) as it similarly avoids the time-reversal construction.

Original: Wang et al.
Revised: Wang et al.

Original: (2021) construct a 2-stages diffusion process from a constant initial value x 0 to P D by relying on the theory of Schrödinger bridges.
Revised: (2021) construct a 2-stages diffusion process from a constant initial value x 0 to P D by relying on the theory of Schrödinger bridges.

Original: The most notable differences with respect to the DBMT transport are: (i) the dynamics considered in Wang et al.
Revised: The most notable differences with respect to the DBMT transport are: (i) the dynamics considered in Wang et al.

Original: (2021) are less general, in our notation they correspond to f ( · ) = 0 , g ( · ) = σI for a ﬁxed scalar σ ; (ii) the transport proposed in Wang et al.
Revised: (2021) are less general, in our notation they correspond to f ( · ) = 0 , g ( · ) = σI for a ﬁxed scalar σ ; (ii) the transport proposed in Wang et al.

Original: (2021) necessarily starts from x 0 , the general result of (8) allows for (almost) arbitrary initial distributions and initial-terminal dependencies.
Revised: (2021) necessarily starts from x 0 , the general result of (8) allows for (almost) arbitrary initial distributions and initial-terminal dependencies.

Original: For an initial x 0 ,  and for the more limited dynamics considered in Wang et al.
Revised: For an initial x 0 , i.e. for case of A ( x t , t x 0) in Section 3.2 and for the more limited dynamics considered in Wang et al.

Original: (2021), the achieved transport is the same.
Revised: (2021), the achieved transport is the same.

Original: In this sense the DBMT generalizes the ﬁrst stage diffusion of Wang et al.
Revised: In this sense the DBMT generalizes the ﬁrst stage diffusion of Wang et al.

Original: It is interesting to note that in the case of a constant x 0 the DBMT can also be obtained by an application of Doob h -transforms as we show in the following section.
Revised: It is interesting to note that in the case of a constant x 0 the DBMT can also be obtained by an application of Doob h -transforms as we show in the following section.

Original: [Equation]
Revised: We now review two additional works grounded in the Schrödinger bridge problem: De Bortoli et al. ; Vargas et al. Both works rely on the Iterative Proportional Fitting (IPF) procedure to solve the (dynamic) Schrödinger bridge problem. Both works leverage on time-reversal results to carry out the alternated Schrödinger half-bridge IPF iterations.

Revised: The main difference between the two works is that De Bortoli et al.

Revised: (2021) estimates the optimal SDE drifts via neural network approximations and score-matching, while Vargas et al.

Revised: (2021) relies on Gaussian Processes and maximum likelihood ﬁtting.

Revised: The work of De Bortoli et al.

Revised: (2021) can be seen as an extension of Song et al.

Revised: (2021), and similarly to our work allows the use of shorter time intervals.

Revised: Compared to our proposal, it solves a harder problem but also presents additional difﬁculties.

Revised: Training is more involved as all the neural network approximations, one for each IPF iterate, need to converge.

Revised: Moreover, there is limited guidance on how to optimally choose the number of integration steps over the number of

Revised: IPF iterates.

Original:  C ONNECTION WITH D OOB H - TRANSFORMS
Revised: C. C ONNECTION WITH D OOB H - TRANSFORMS

Original: The previously established identity
Revised: The previously established identity

Original: [Equation]
Revised: [Equation]

Original: shows that the drift adjustment can be equivalently expressed as
Revised: shows that the drift adjustment can be equivalently expressed as

Original: [Equation]
Revised: [Equation]

Original: as x 0 is a constant.
Revised: as x 0 is a constant.

Original: It can be veriﬁed that the h function satisﬁes the required space-time regularity property (Särkkä & Solin, 2019, Eq.
Revised: It can be veriﬁed that the h function satisﬁes the required space-time regularity property (Särkkä & Solin, 2019, Eq.

Original: As such, it is a genuine Doob h -transform.
Revised: As such, it is a genuine Doob h -transform.

Original: That p h [Equation] ( x 
Revised: That p h t (cid:48) | t ( x t (cid:48) | x t ) =

Original: [Equation]
Revised: p t (cid:48) | t ( x t (cid:48) | x t ) h ( x t (cid:48) , t (cid:48) ) /h

Original:  , t ) is the transition density of the DBMT transport from δ x 0 to Π τ follows by direct computation.
Revised: ( x t , t ) is the transition density of the DBMT transport from δ x 0 to Π τ follows by direct computation.

Original: [Figure]
Revised: For simplicity, we assume a factorial distribution over the channels and an isotopic stationary covariance function. We rely on the semivariogram approach (Cressie, 1993) to compare how different covariance functions ﬁt D ( CIFAR ) . A semivariogram is a measure of dependency across space.

Revised: In the case of an isotropic stationary covariance it simpliﬁes to a scalar function of the

Revised: Euclidean distance between points: γ ( (cid:107) ∆ s (cid:107) ) =

Revised: E [(∆ x s ) 2 ] / 2 with ∆ x s = x s +∆ s − x s .

Revised: The rate of decrease of γ ( (cid:107) ∆ s (cid:107) ) toward 0 as (cid:107) ∆ s (cid:107) → 0 gives a measure of the inﬁnitesimal spatial dependency, i.e. the smoothness of the spatial process.

Revised: Semivariograms corresponding to different covariance functions are here ﬁtted to their empirical counterparts via a weighted minimum-leastsquares procedure (Cressie, 1993).

Revised: Figure 4 illustrates the exponential and RBF semivariogram ﬁts for two images of D ( CIFAR ) .

Revised: The exponential covariance, which corresponds to rougher paths, provides a much better ﬁt to the shown samples.

Revised: This result is consistent across D ( CIFAR ) .

Revised: We remark that Γ =

Revised: I corresponds to a pure white-noise process with a perfectly ﬂat semivariogram which would clearly result in a very poor ﬁt to the empirical semivariograms shown in Figure 4.

Revised: Based on these ﬁndings, we model the innovations of each image channel as a GP with exponential covariance function with length-scale θ = 0 .

Revised: 205 , the median estimated value (Figure 4 (right)).

Revised: We match the marginal variance to that of D ( CIFAR ) , σ 2 = 0 .

Original:  We start by providing a cursory explanation leading to efﬁcient sampling in the 1D case, before giving the intuition behind the extension to the 2D case.
Revised: [Figure] [Figure] We start by providing a cursory explanation leading to efﬁcient sampling in the 1D case, before giving the intuition behind the extension to the 2D case.

Original: We refer to Wood & Chan (1994); Dietrich & Newsam (1997) for a complete explanation and to Rue & Held (2005) for the results underlying efﬁcient density (likelihood) computation.
Revised: We refer to Wood & Chan (1994); Dietrich & Newsam (1997) for a complete explanation and to Rue & Held (2005) for the results underlying efﬁcient density (likelihood) computation.

Original: Let [0 , 1] be the spatial domain of interest.
Revised: Let [0 , 1] be the spatial domain of interest.

Original: Let S
Revised: Let S

Original: = { s i } Mi =1 be an uniform grid (regular lattice) discretizing [0 , 1] , where the points s i are assumed to be ordered.
Revised: = { s i } Mi =1 be a uniform grid (regular lattice) discretizing [0 , 1] , where the points s i are assumed to be ordered.

Original: The ﬁrst key observation is that for a stationary covariance function ρ ( · , · ) the covariance matrix C with entries C i,j = ρ ( s j , s j ) is symmetric and Toeplitz, i.e. with constant-diagonals.
Revised: The ﬁrst key observation is that for a stationary covariance function ρ ( · , · ) the covariance matrix C with entries C i,j = ρ ( s j , s j ) is symmetric and Toeplitz, i.e. with constant-diagonals.

Original: See
Revised: See

Original: Figure 4 (leftmost) for an example where M = 9 .
Revised: Figure 5 (leftmost) for an example where M = 9 .

Original: A property of symmetric Toeplitz matrices is that they can always be embedded in larger symmetric circulant matrices.
Revised: A property of symmetric Toeplitz matrices is that they can always be embedded in larger symmetric circulant matrices.

Original: A circulant matrix of size
Revised: A circulant matrix of size

Original: M (cid:48)
Revised: M (cid:48)

Original: × M (cid:48) is deﬁned by the property that all its rows (and columns) are obtained by cycling through the same M (cid:48)
Revised: × M (cid:48) is deﬁned by the property that all its rows (and columns) are obtained by cycling through the same M (cid:48)

Original: -dimensional vector.
Revised: -dimensional vector.

Original: Circulant matrices correspond to covariance matrices of GPs deﬁned on a (here 1D) torus (Rue & Held, 2005, Chapter 2.1).
Revised: Circulant matrices correspond to covariance matrices of GPs deﬁned on a (here 1D) torus (Rue & Held, 2005, Chapter 2.1).

Original: The circulant embedding matrix just introduced corresponds to an artiﬁcial enlargement of the spatial domain [0 , 1] to a larger interval leading to a torus.
Revised: The circulant embedding matrix just introduced corresponds to an artiﬁcial enlargement of the spatial domain [0 , 1] to a larger interval leading to a torus.

Original: See Figure 4 (2 nd from left) for a circulant embedding of C .
Revised: See Figure 5 (2 nd from left) for a circulant embedding of C .

Original: The second key observation is that a circulant matrix is diagonalized by the 1D FFT matrix.
Revised: The second key observation is that a circulant matrix is diagonalized by the 1D FFT matrix.

Original: Having obtained the eigenvalues of C , efﬁcient sampling on the enlarged domain is achieved by the 1D FFT applied to complex standard random numbers multiplied by the (square root of the) eigenvalues.
Revised: Having obtained the eigenvalues of C , efﬁcient sampling on the enlarged domain is achieved by the 1D FFT applied to complex standard random numbers multiplied by the (square root of the) eigenvalues.

Original: The real and imaginary part of the generated samples are independent.
Revised: The real and imaginary part of the generated samples are independent.

Original: The main issue wih the CEM is that the circulant matrix embedding might fail to be positive deﬁnite.
Revised: The main issue with the CEM is that the circulant matrix embedding might fail to be positive deﬁnite.

Original: The issue can be avoided by considering progressively larger embeddings, see the theoretical and empirical ﬁndings of Dietrich & Newsam (1997).
Revised: The issue can be avoided by considering progressively larger embeddings, see the theoretical and empirical ﬁndings of Dietrich & Newsam (1997).

Original: Otherwise a level of approximation can be accepted by modifying the covariance function or by truncating the eigenvalues to be positive.
Revised: Otherwise, a level of approximation can be accepted by modifying the covariance function or by truncating the eigenvalues to be positive.

Original: The development of the 2D CEM follows very similar steps.
Revised: The development of the 2D CEM follows very similar steps.

Original: The domain of interest is now [0 , 1] 2 , the uniform grid is S
Revised: The domain of interest is now [0 , 1] 2 , the uniform grid is S

Original: = { s i,j } Mi,j =1 and the points s i,j are assumed to be lexicographically ordered.
Revised: = { s i,j } Mi,j =1 and the points s i,j are assumed to be lexicographically ordered.

Original: The stationarity of the covariance functions results in a symmetric block-Toeplitz covariance matrix, as shown in Figure 4 (3 rd from left).
Revised: The stationarity of the covariance functions results in a symmetric block-Toeplitz covariance matrix, as shown in Figure 5 (3 rd from left).

Original: Again, symmetric block-Toeplitz matrices can be embedded in symmetric block-circulant matrices, as exempliﬁed by Figure 4 (rightmost, zooming might be required to see the block structure).
Revised: Again, symmetric block-Toeplitz matrices can be embedded in symmetric block-circulant matrices, as exempliﬁed by Figure 5 (rightmost, zooming might be required to see the block structure).

Original: Block-circulant matrices can be shown the be diagonalized by the 2D FFT matrix, and efﬁcient sampling follows from similar steps to the ones seen in the 1D case.
Revised: Block-circulant matrices can be shown to be diagonalized by the 2D FFT matrix, and efﬁcient sampling follows from similar steps to the ones seen in the 1D case.

Original: D A DDITIONAL F IGURES
Revised: D A DDITIONAL F IGURES

Original: [Figure]
Revised: [Figure]