Flow matching achieves almost minimax optimal convergence
Authors: Kenji Fukumizu
Taiji Suzuki
Noboru Isobe
Kazusato Oko
Masanori Koyama
Venue: ICLR 2025 Poster (2025)
Keywords: flow matching
generative model
convergence rate
optimality
Primary area: generative models
TL;DR: We establish that FM can achieve an almost minmax optimal convergence rate in terms of 2-Wasserstein distance.

Abstract: Flow matching (FM) has gained significant attention as a simulation-free generative model. Unlike diffusion models, which are based on stochastic differential equations, FM employs a simpler approach by solving an ordinary differential equation with an initial condition from a normal distribution, thus streamlining the sample generation process. This paper discusses the convergence properties of FM in terms of the $p$-Wasserstein distance, a measure of distributional discrepancy. We establish that FM can achieve an almost minimax optimal convergence rate for $1 \leq p \leq 2$, presenting the first theoretical evidence that FM can reach convergence rates comparable to those of diffusion models. Our analysis extends existing frameworks by examining a broader class of mean and variance functions for the vector fields and identifies specific conditions necessary to attain these optimal rates.

Published as a conference paper at ICLR 2025
FLOW MATCHING ACHIEVES ALMOST MINIMAX OPTI- MAL CONVERGENCE
Kenji Fukumizu
Taiji Suzuki
The Institute of Statistical Mathemmatics/Preeereed Networks
University of Tokyo/RIKEN AIP
Tokyo, Japan
Tokyo, Japan ta j @mi wtw..s......ooo.ee...
fukumi zuê i sm. ac jp
Noboru Isobe
Kazusato Oko
University of Tokyo
University of Tokyo/RIKEN AIP Tokyo, Japan
Tokyo. Japan
nobo04098g .eee999g.co ecc. ucc.u-toky..ac.
cko-kazusatoßg www..eoo...............oo...
Masanori Koyama
Preferred Networks/Universin of Tokyo
Tokyo, Japan
masanori koyama@weblab. u...kkyo...ako....p.p....
ABSTRACT
Flow matching (FM) has gained significant attention simulation-free gener- ative model. Unlike diffusion models, which are based on stochastic differential equations, FM employs simpler approach by solving equation with an initial condition from normal distribution, thus streamlining the sample generation process. This paper discusses the convergence properties of FM for large sample size under the Wasserstein distance. We establish that FM can achieve an almost minimax optimal convergence rate for S S 2, presenting the first theoretical evidence that FM can reach convergence rates comparable those of diffusion models. Our analysis extends existing frameworks by examining broader class of mean and variance functions for the vector fields and identifies specific conditions necessary to attain almost optimal rates.
ordinary differential
INTRODUCTION
Flow matching (FM) (Lipman al. 2023 Albergo and Vanden-Ein Eijnden 2023 Liu et al. 2023b) is recent simulation-free generative model that produces samples of the target distribution by solving an ordinary differential equation (ODE) initialized with source normal distribution. The vector field to define the ODE is trained by neural networks with the teaching data of random conditional vectors. This approach bypasses the computationally intensive Monte Carlo sampling required the diffusion model, which is currently the standard in generative modeling. Various variations have been proposed refine the learning vector fields, such as OT-CFM (Tong al., 2024), rectified flow (Liu et al..
2023b), consistent velocity field (Yang et al. 2024), equivariant flow (Klein et al.] (023)) etc. A series of studies also emerge from the viewpoint interpolating distributions (Albergo et al. 2023c a).
FM has already been applied various domains with promising performance. Among many others. the rectified flow method has been extended to high-resolution text image generation (Esser et al. 2024). and there are also many works the application of FM to molecule generation (Hoogeboom
ct al. 2022 Guan et al. 2023) Bose et al. 2023 Dunn and Koes 2024), text generation (Hu et al.
2024 speech generation (Le et al. 2023). motion synthesis (Hu al.] 2023), etc.
Although the methods have been developed on the solid theoretical basis of the flows and continuity equation, their statistical behaviors remain less understood. Recent works have established the con- vergence of the FM estimator to the true distribution under some distributional metrics (Albergo and Vanden-Fijndee 2023 Benton et al. 2023b). Beyond the convergence, more detailed understandings,

---

Published as a conference paper at ICLR 2025
such convergence rates, still open question. In contrast, diffusion models have gained various theoretical understandings. including the convergence rate in terms the number of steps (Chen et al. 2023: Benton et al. 2023a) and the sample size (Oko et al. 2023 Zhang et al., 2024)) Among others, Oko et al. (2023) has shown that diffusion models achieve the minimax optimal convergence rate for large sample size under the total variation metric and the almost minimax optimal rate under the 1- I-Wasserstein distance, where the max is taken over the true densities of the Besov space. This result theoretically supports the high generation ability of diffusion models.
This paper aims to bridge this gap by demonstrating that FM can achieve an almost minimax optimal convergence rate for a large sample size under the p- Wasserstein distance W, for S p I suggesting that FM has theoretical ability comparable to diffusion models. This problem is significant for comparing the ability FM methods and diffusion models, and revealing the difference between SDE and ODE in the generative models. Drawing on the methodologies of of et al. (2023), our analysis not only extends a broader class of mean and variance parameters of Gaussian smoothing for conditional vector fields, but also specifies the conditions on these parameters under which the almost minimax optimal convergence rate can be achieved.
The contributions of this paper are as follows.
We establish that widely used class of conditional FM methods achieves almost minimax optimal convergence rate under the p- p~Wasserstein distance (1 S p S 2), marking the first theoretical demonstration of such optimal performance of FM.
We provide an analytical derivation of the convergence rate under various settings of the parameters, mean and variance, to make path that connects source and target point.
We reveal that the variance parameter, which specifies the contribution of the source, must be decreased around the target a specific rate attain an almost minimax optimal convergence rate.
2 FLOW MATCHING
Throughout the paper, data are in the d-dimensional space Rd The d-dimennional normal distribution with mean μ and covariance matrix V is denoted by Na(u, V). For a probability Pa with index a, the lowercase Pa denotes its probability density function (p.d.f.).
2.1 REVIEW OF FLOW MATCHING
This subsection provides a general review of FM, following Lipman et al. (2023) and Tong al. (2024). The aim of FM is to generate samples from the true probability Perze- FM methods realize it by a flow (x) (T E [0, 1)" that maps a sample from the standard normal distribution Na(0,, La)
that of Pirue- The flow [P(+] (x) is defined by a solution to the ODE
d
dr [[-] = (2()))
(r E [0,
given by desired vector field mp-- FM generates sample by solving the ODE with initial point z0) from Po No Ia); in other words, the distribution time is the pushforward Pr] = [[]] P(o.- The pushforward P() is expected to approximate Ptrue- In practice, we need to
construct the vector field given training data {æ')n y of size n, which is i..d. samples from Pirue-
The relation between the vector field i/ (x) and the p.d.f. (xx) given by the continuity equation:
07 (x) + div (1))0))))))) = 0.
Typically, a neural network (NN) used construct (z). However, it is not obvious how to prepare the desired B/y (a=) to teach NN. In FM methods, conditional random vectors U/y (2(r))>) given z, which are to be easily prepared, are used to teach NN; a location Tir) is sampled by conditional probability Prr (I[=||>) [2) and the vector |/ [-]]] 2) is assigned at [[r] as teaching data.
Wee use [7] to denote the time [0, 1] in this section and preserve a. for the reverse time indexing, which adopted from Section to align with the notation of diffusion models.
2

---

Published as a conference paper at ICLR 2025
Throughout this paper, the condition is given by (2 F(1) with Z(0] Poj and 2[1] Ptrue- The vector (1)((())])) is made so that it satisfies the conditional continuity equation:
o, Prr P- (2 + div
= O.
(I)
A typical construction of ||r is to use path [|r] (T E [0, 1]) from 2(0 to 2[1) and define the d
conditional vector by its time derivative 2[(]] (=]](=) == dr [//]] (see Sec. 2.2). For deterministic path, (+]]1= 2) is the delta function at a point in the path [[r]-
Note that, given ()) the vector (z] x) is random by the choice of x = (2))) Tn); different vectors may be assigned the same location (x. r). Most importantly, by averaging over 2 ~ Q. where Q is the joint distribution with marginals P(uj and P1I. we can see that the p.d.f. of æ at time
P()((z) = (=))()))))
and the averaged vector field (x) given by
P[]](/=>) := (())(a(2)((2))
(r) (z) ==
(3)
p((x))
satisfy the continuity equation
Ôr (x)) + div
= O.
(4)
This provides the theoretical basis for FM methods; the averaged vector field [1) transports Na(0, La) to Perne- U[r is learned by NN Φ(x, r) with noisy training data [< [ Em- pirically, the conditional [/=] is given by the random sample = = (æ) x") and the location Tir]] ~ (x/2)) (or path) with uniform T. The NN is trained with the mean square error (MSE):
ming E (0)))) r) - 1|r] [+1]))1))]]
(5)
Note that ((x(r): 7) does not depend on z. Since the MSE minimizer is the conditional expectation of the teaching data, the empirical minimizer Ø is estimator of (z)) Using the estimator & and the corresponding flow (-))] given by ODE, we obtain the estimator P11 for Perue by sampling. In practice to reduce the variance of (Z p11) and simplify the ODE solution, the optimal transport for pairing F[o and Ø(1j is applied effectively (Tong et al. 2024 Pooladian et al. (023).
2.2 PATII CONSTRUCTION
This paper focuses on the following class paths to construct the conditional vector field. Let Loo ~ Pioj = Na Ia), and ønj be sample of Pin (or the empirical distribution Pevain = (1/n) En i= Szi in practice). A conditional path is defined by
3tr == | + 1p]
(0 r S 1, > 0,1 0).
(6)
We assume that oe and my monotonic, [r]] - 1, m(+] - 0 as - 0+, and [r) 0, m[r] 17 Let o'r (m/r) resp.) be the time derivative of [r) (m resp.). With sampling Unif(o, 1]. random conditional vector is assigned at (2)+), E Rd x [0, by
U[r] (+|+)) [ [1] == of o1o + m())
(7)
Note that, due to 21j]] (0. Ia), the distribution of [[r) given z[j] equals (=+ (= Na(m of. Ta), and thus we call m(r] and of: the mean and variance parameters, respectively. [r),
Since (6) leads 2ooo (w++) - m //))) the conditional vector 7 is written as
x m4 + m(]]
(+ = o
(8)
(r)
This class covers some popular constructions of conditional vector fields in the literature.
3

---

Published as a conference paper at ICLR 2025
Affine path: one of the most popular constructions is the following,
[(r] == (1 [)) + [1] [[=] (2)) [()) 211] x -(0]-
This corresponds to / 7 and [] 1 - r. In Lipman et al. 2023) æ and [1 are generated
independently, while in Tong al. (2024) they are taken by the optimal transport in minibatch.
This case is covered by our result, which does not depend on the construction of joint distribution.
Diffusion: [iipman et al. (2023) presents the diffusion path, which corresponds the deterministic probability flow (Song et al. (020). The conditional density is given by P(r] Pir) (+|) [1]] = y) =
Na(m= (.) La). The setting [r] 1 mir] and [r) vI- is typically used.
3 CONVERGENCE RATE OF FLOW MATCHING
We assume that the true density P(1] is included in the Besov space B (s> 0, 0.0 ='> co)
the unit cube [-1, 1j while the results be extended any size straightforwardly The parameter s specifies the degree of smoothness and is most relevant in this paper. The definition of the Besoy space is deferred Appendix A We use the r-Wasserstein distance W, to measure the accuracy of the estimator. The distance W, of the probabilities P1 and P2 on R is defined by
(F P2) = (infqer(Ps,PP)) [|| x2|| "Q(Q(z,,,)) x2)) 1/r
where I(P,P3) denotes the joint distribution of (x) x2) with marginals P1 and Pz. It is well known that WP,,,, P2) S W,, (P1, P2) holds for r' 2 2 1.
As discussed in Sec. 331 to obtain an accurate estimator, we need to adopt early stopping of ODE and use P1- -Tyl with small To- Our aim is to derive bound of Wp WP(PP (Pl1-T) Ptrue) for a large sample n + 0o. The informal version of our main result is summarized in the following theorem.
Theorem (Informal). Suppose that the target probability Pij has p.d.f. Pnl in the Besov B 1/ ay (1-1, of smoothness degree s, and that n training data {x) is i.i.d. samples from Pil- Assume that [r] ~ ~1-7)" (T - 1 with k 2 1/2, the conditinal vector field is given by 46) and ( and that time- tine-divided neural networks are used (see Sec. (4.3). Then, under several assumptions, the FM estimator P(I-To) with To Ra with appropriate Ro satisfies, for any >
B[Wa(P1-15)) Pirna)] 0
(n xc))
(10)
where E denotes the expectation over the training data.
It is known that a lower bound of the minimax convergence rate exists for the Wasserstein distance for probability estimation. We use the notation > to mean the lower bound up to constant factor.
Proposition 2 (Niles-Wecd and Berthet 2022 ) Let p', 1. 1,8 0. r 2 1, and d 2 2. Then,
inf p suppe (I-..))))) EWW P)] Z 2
(n 00),
where P runs over all estimators based on n i.i.d. samples from P.
For 1/2, by Theorem and Proposition the upper bound 2+4d is almost the optimal convergence rate up to an arbitrarily small 8 > 0. In addition, this convergence rate coincides with that of the diffusion model given in Oko et al. (023) for W,. The above result indicates that the flow matching is as good as the diffusion model regarding the minimax convergence rate under W,, where the max in minimax means sup the Besov space.
3.1 KERNEL DENSITY ESTIMATION AND EARLY STOPPING OF ODE
In practice, with conditional density m-) = (a
La), la), the parameter dn is often set as small positive value T[I) = Jmin > 0 so that 7 is well defined up to 7 = 1 (e.g. Lipman
ct al. 2023). If am is sampled from P Perain 1 Σn der the obtained distribution equals
(x) = J P[ (20 p Peman (1i1)) Σn
(2m0 Ja7z exp
|| ||ar-ar11
2
4

---

Published as a conference paper at ICLR 2025
which is exactly the kernel density estimator (KDE) with the Gaussian kernel of bandwidth min- If the ODE is solved up to r rigorously, the pushforward realizes this KDE. As is well known (Scott 1992), the convergence rate of this KDE under MSE is O(n ((n=4/((+))) at best by choosing the optimal min depending on n, which is much slower than the optimal rate n -2=/(2x+dd under MSE for the true density in B (I4) (Liu et al. 2023a). Based on this consideration, we discuss the early To with small To > 0 and consider the convergence rate of the estimator Pn- -T]-- Notice that Pn Tal differs from KDE, since it is given by the trained vector field. For diffusion models, OOko et al. (2023) and Zhang et al. (2024) also discuss the estimator obtained by stopping the reverse SDE at To to derive the convergence rate.
stopping of the ODE, where we stop T r
3.2 RELATED WORKS
Among many literatures on the statistical convergence of diffusion models, the most relevant to this
work Oko al. (2023). Although our analysis is based on mko et al. (2023) and derives comparable results, there are significant differences. First, we analyze the more general settings for [r]r] and [-]
in the conditional distribution Ppp((= lv) = Na(m)-)). (- Oko et al. (0023 considers only the
case of ar Vt and m, ~ 1 - f (in reverse time t), which is a typical choice for diffusion models. Consequently, we have shown that for rr ~
(1- with x 1/2, only k 1/2 achieves the almost minimax optimal convergence rate. Second, due to the difference between the ODE and diffusion processes, the proof technique for relating the Wasserstein metric and the L2-risk is very different. Our technique is based on Sieksevv-Gööbng lemma to derive the bound for r-Wasserstein with S S 2, while Oko et al. (2023) obtained the bound only for 1-Wasserstein. Third, this the first theoretical result for FM showing convergence rate that is almost optimal. Although FM has been recently used in many applications with competitive results to diffusion models, thcoretical comparisons in terms of convergence rates have been lacking. The results of this paper show that both FM and DM can attain the same almost minimax optimal convergence rate for generalization error.
recent works convergence. Albergo and Vanden-Elijnenn (2023) and Benton et al. (202bb) relate the Wasserstein distance to the L2-risk of the vector fields and show convergence for a large sample size, but did not derive convergence rate. Jiao et al. (024) discusses convergence rates of FM applied in the latent space of the autoencoder and considers the discretization effect of the numerical ODE solution in their analysis. However, they did not include the degree of
For FM, there are
smoothness in developing the convergence rate. AAlbergo et al. (2023b) present a unifying view of the theory of diffusion models and FM with the upper bounds of discrepancy measures.
THEORETICAL DETAILS
This section rigorously presents the main result with the assumptions and shows the proof outline. In the sequel, we use reverse time index t = 1 - T (7 E [0, 1)); t 0 for Perue and t = for (2 Ia). which align with the notations of the diffusion models. We poly(log n) to indicate the term of Ollog" norder with some natural number r, and Ō mean the order up to poly(log n) factor.
4.1
PROBLEM SETTING AND ASSUMPTIONS
With reverse time t, the definitions
and 3 are modified by replacing [1 - t| with f;
P. PI-t). Po Perue: P1 Na(0, Ia).
The flows s and S. are defined by solving the ODE from i. 1 in the reverse time direction:
d
d _ ( (x),,
di p(())) v. (x)(
(11)
where (x) and (z) are the vector field 3 and its neural estimate, respectively. The distributions at t E [o, 1] are given by
Pt = (1
P (G:
(12)
where (1p:) # and ( denote the pushforward by the respective flows sp and P.
In the remainder of this paper, is an arbitrarily small positive value. As in Oko et al. (2023). introduce N specify the number of basis functions of the B-spline for approximating pr pr(
5

---

Published as a conference paper at ICLR 2025
and (x)). This number N depends on the sample size (N n. is used), balancing the approximation error and complexity of the B-spline and NN. We set the stopping time To = N- Ro as discussed in Sec. (RR, is specified later), and solve the ODE from 1 To. For simplicity, the
(Ro
dimensional cube 1, 1 and the reduced cube [-1 N (1-166), - N --(1-=0) (1-x6)]d are denoted by Id
and I respectively, where k >> 0 is specified below in (A3). We make the following assumptions.
(AI) The target probability Po has support 1d and its p.d.f. po satisfies po E Bi (1 and po E
By.g (1 with 8 > max - 1, 1,
(A2) There exists Co > such that Co S po(x) S Co for all x E 14.
(A3) There is K 2 1/2, bo > 0, R > 0, and by > 0 such that
ot but"
1 my but
for sufficiently small 2 To. Also, there are Do > 0 and Ko such that
DT S of m S Do.
|o + |m/l S No
(Vt [To, 1)).
(A4) If 1/2, there is b, and Di >> 0 such that for any 0 S 7 < Ro
1o A- {(o{)² + (m4)² - Dlog N)
(A5) There a constant Ct > such that
a 8m
(y x dyyl op S Cr for any E [To. 1].
The higher degree of smoothness is assumed around the boundary of 1d in (A1) for a technical reason compensate for the nondifferentiabiiiiy of (o(x) at the boundary by (A2). In (A3), it may be more natural to require of + m = so that signal power can be maintained. However, in this paper, to pursue the flexibility of choosing o; and my. we allow bounded changes of af + m? over (A4) is required to limit the complexity of the neural network model (see Lemma 5). In (A3), is assumed to be not less than 1/2, because for x < 1/22, the integral Jr, (of) with To = N Ra diverges to infinity as N - +oc, which causes the divergence of the complexity bound Lemma 5 Note that the boundary case = 112 is, in fact, popularly used for the diffusion model. In this case, (a¹)² is the order 1/t for + 0 and the integral from To is of the order log N, which still diverges to infinity as oo. As discussed in Section 4.3 we consider this integral only for a short time interval, and we will see that the W2 distance converges to zero as n + co. (A5) is made to bound the Lipschitz factor in Theorem 3 Jnder (A3) (see Lemma 10
4.2 GENERALIZATION BOUND
It is known (Albergo and Vanden- Eijnden 2023 Benton al. 2023b) that, given two vector fields, the We-distance the pushforwards of the same distribution by the corresponding flows admits an upper bound by the L2-risk of the vector fields;
Theorem 3. Let vx(x) and (x) be vector fields such that + (x) is L Li-Lipschitz for each t, and Pt and P. be the pushfowards of distribution Po by the corresponding flows at time t from Then, for any t E [0, 1], we have
Pt) Vi (
1/2
(13)
See Appendix B for the proof. From Theorem we can consider the L2- E[ [ [ v(x, dPP, dæ)ds of the vector field to obtain the bound of the W₂ distance of the distributions. From the fact W, < W,, (1 < r S r'), the same upper bound holds for W, for S < 2.
We first review general method for bounding the generalization. We consider training within the general time interval [Te, Tw] where To < Te T. < 1. For estimator o(x, t) of the true vector field vx(),, we define the loss function TT..T. (x) for 1d by
(T) [ [10(0s,)) t) v=(2x(2))1 ( (x x do,,t,
To (x) = 0
(14)

---

Published as a conference paper at ICLR 2025
where æ is the condition of (x)) Although the definition depends on T₃ and Tu. omit them when there is no confusion. Given the training data h==1 the vector field is trained with the teaching data (x (x') at the location (æ,, (t E [T,,Tu]) which is sampled from Pa (r (a') and the uniform distribution U[TT), Tul). Note that given æ we can generate any number of (æz, t). Thus, the sampling error in (14) is negligible and the training by NN can be regarded minimization of
EE (o=1((((("))
(15)
See Oko al. (2023) Section 4) for the discussion of the effect of sampling.
Let & be the minimizer of (15) among the function class S. The generalization error is then given by
To [: t) (at = IE |/ & ((())o)))) (16) [
Egen : E
Let := {&o & E S} and W[C, |1 (I=), be the covering number of the function class L with the || -1 (1a)-norm Then, a standard argument on the generalization error analysis derives the following upper bound (see (see et al. (Theorem C.4, (203) and also Hayakawa and Suzuki (2020))
Theorem 4. The generalization enror of the minimizer of (13) among & is upper bounded by
Egen 2 inf $E8 /
Tu |1((,, t) - 01( (ax)dd
supses 1. 37 log N V[C, (1,) 32 + 3a. (17)
n
From Theorems 3 and it suffices to consider the approximation error (Ist term) and complexity (2nd term) in (17) for deriving the W3 distributional bound.
4.2.1 COMPLEXITY TERM IN GENERALIZATION BOUND
We first consider the complexity term, where the class S is given by NN. A class of NN M(L, W. B) with height L, width W, sparsity constraint S, and norm constraint B is defined as
M(L, W,L, B) = (WA pL) o...........................oo .........))))))................. y (A x +600) A E R R ((0) E IR W
E + (b(0))) (1)) S S, max; || || || S B), where (x) = A ReLU(z)) + b. As shown in Theorems 7 and later, it suffices to consider the NNs that satisfy
[ l) < D(o''l log + (m4))
for constant D. Also, Lipschitz constant proportinal to 1/t under (A3) and (A5). Reflecting these facts, we define the following NN class for training the vector field:
in Lemma A.2 that - v. is Lipschitz continuous with
H,p = { E M(L W, B) [ ll I D(oo'lv/ogn + |m/l) for Vt E [To. 1], - (0x t) is Lg Le-Lshhciitz for each E (To 1] where L, = ÖL/t), (18)
where and Čz are some positive constants.
The supremum norm and the covering number in Theorem 4 are given in the following lemmas. Lemma 5. Let To < T3 Tw 1. Under Assumption (A4), there is C, > such that Sup (olle>(ra S Colggn))
(19)
where b by (A4) for 1/2, and b for 1/2.
See Appendix C.I for the proof. To obtain this bound, we need to impose the upper bound of Φ as in (88). In practice the vectors in the teaching data satisfy this upper bound, and thus & will naturally satisfy the same bound by the least square error solution. The following bound the covering number for neural networks is given by Suzuki (Lemma 3, 2019). Lemma 6. For the function class H.. the covering number satisfies
log.N Ng,,(L,. (1 < ST. || Bn)
7

---

Published as a conference paper at ICLR 2025
4.2.2 APPROXIMATION ERROR FOR SMALL
Recall that N specifies the number of basis functions of the B-spline for the approximation. We derive upper bounds of the approximation error of the NN model MiL, S, B), where L, W; S, and B are specified in terms of N. We will separate [To. 1] into two intervals, [To. 3T.] and [T., 1], where T. == (--xx=-6)/6 and provide different upper bounds. The reason for this choice of division point T. is sketched as follows and is detailed in C.2 In the approximation of the vector field, the B-spline approximation of densities as in Oko et al. (0023). To show fast convergence rate, the first interval is more subtle because (t(x)) is rougher. In approximating the density on the smoother boundary region, we divide the region into small cubes, each of which uses N bases for B-spline approximation. To make the total number of B-spline bases comparable with N, the width dg of the region should be ag = (1-xxd/// On the other hand, in Theorem 7 we need a concentration of an integral around the boundary region for better approximation by the higher smoothness, and this limits the variance of the Gaussian kernel so that or 18 < 0g. This derives S NN-(x"166)) As result, the division point is small enough as T. := N (xr -6)/d
The approximation bound for t E [To, 3T.] with T. = N (see Appendix C.4 for the proof).
is given in the following Theorem
Theorem 7. Under assumptions (AI)-A5), there is a neural network Φ1 E MML,WW, B) and constant Co. which is independent of such that, for sufficiently large N.
/ [10((2,)) t) (())) I (o, log N + +mm))
(20)
for any E [To, 3T,), where L = O(log" N). | = O(N log" N). S = O(N log N). B = exp(O(log N log log N)). Additionally, we can take Φi to satisfy
[z, 8)ll S
+ |m|]],
where Cg is constant independent tf
4.2.3 APPROXIMATION ERROR FOR LARGE t
We derive a bound of the approximation error on any interval 12....), where 1.2 T, = N This is used to discuss the optimal convergence rate in Section 4.3
Theorem 8. Fix tu E [T.. 1] and take an arbitrary >> O. Under Assumptions (AI)-(A5), there is
neural network Φ2 E MT. W, S, B) and C7 > O, which does not depend on I, such that the bound
/ (1))(z;)) -
S CT((o!)² log N + (m))² J.N-
(21)
holds for all t e [2tz, 1], and the NN model satisfies O(log* N), |o O(N), S O(1, de N6x), and B exp(O(log N log log N). Moreover, 622 can be laken so that
(l) S (o)( log N [m/]]
with constant Ö₂ > 0 independent of
See Appendix C.5 for the proof. The approximation error N is arbitrarily small and is not dominant, while S may be dominant in the complexity term.
4.3 CONVERGENCE RATE UNDER WASSERSTEIN DISTANCE
We can consider generalization bound based on Theorems [To 2T.] and [2T., 1]. However, if we apply Theorem 8 for 2t., with f, T. = -- -8-- the dominant factor of the log covering number in 17 is the sparsity 5 =
and 8 deriving the bounds for
From Theorems 3 and 4. the complexity part gives ((NNn) term in the Wz generalization error. If we plug N n (2s+//d which is optimal for the MSE generalization, we have O(n -x((s++d) for the upper bound of W2 generalization, which is slower than the lower bound in Proposition 2 To
8

---

Published as a conference paper at ICLR 2025
achieve better convergence rate, will make use of the factor vt in front of (13) by dividing the interval [To, 1] into small pieces and using a NN for each small interval, as in Oko et al. (2023) for diffusion models.
Notice that, when time is far from O, the convolution of (r y) with larger o, results in a smoother target vector field (z),, which is easier to approximate with a low-complexity model. On the other hand, when t approaches O, with the fixed number of B-spline bases N. the approximation error bound {( /log N + (m²)² J.N 2-/d - can increase for large of or m4, (e.g. dz ~ t" with K < 1). We therefore need a more complex model (that is, larger N) than the one needed for larger t. Thus, it will be more efficient if the number of B-spline hases N, which controls the approximation error and complexity, is chosen adaptively to the time region t.
Specifically, we make a partition To = to < 4 < <... lK = such that i, = 21 for 1 S j S K K-1 with 2 2 I, and build neural network for each [t]-1,6]] (,,.... K). Note that we train each network for interval [()-1,4] 4g] with n training data (0;) We assume that 4 with T. 1j_ 37. serves the boundary to apply two different error bounds. The total number
of intervals K is Ollog N)- = Ollog n), since 2 To 2 1 with To = NN can be achieved by K 2 Ro log, N. The constant Ro is fixed as Ro 2 min(K.R) so that W2(PT,, is negligible (see
2+1
the proof sketch of Theorem 9 In this setting, we have the following main result.
Theorem 9 (Main result). Assume (AI) and d 2 2. If the above time- time-partition is applied and neural network is trained for each time division, for arbitrarily small 5 > 0 and 1 1 S 2, we have
E [W.,(Pro Paruc))] (.ru.))))r))))
= =00, ++226]11
2542
(n + on))
(22)
Proof Sketch. Let J- = 11-1, (l..... K). We use a smaller neural network model for larger j. Specifically, the number of B-spline bases for Jy is Nj t No for Ja, while Nj N for j S je, where j. is defined as above. Note that N + 0o as N + due to 8 >> 0, and that NJ S 11=5x = N for j 2 j. due to t2 2 N also Figure E.I in Appendix.
which means a lower complexity. See
Next, we consider the bound of Wy-distance based on the partition. For each of j = 1,..,, K, we introduce vector field 5(5) such that it coincides with the target v; fort E [t,, 1] and with the learned G. for E [To, 4g). Let Q(i) be the pushforward from Pi Na(0, [a) to / To by the flow of the vector field e
Then, Q(0) = Pr.. the pushforward by the flow of the target o, from t = 1 to To. and also ((() = Pr. Note also that g(T) and ((1-1) differ only in Jy by v,(x)) - (a()). Therefore,
from Theorem and the Lipschitz assumption on Hn, we have
Wa(Po, Pro) W2(Po, Pro) W2(W-1(
CE Vtg { (C/(do flō((, - v(x, (1) 1/2
s 0n + Ei=1
where 82 dbbn 2. + [1yll/ (o Rān 3 which is derived from Lemma 11 and (A3). We take constant RS (s 1)/ min(k, R) so that 0, is of O(n = and thus 0,, is negligible. It is easy to see that the factor ell' (C/a)du is bounded by constant because of tý 2t,-1 1 by definition. For simplicity, let to : t,,, From Theorems 7 and the generalization bound of Pr, is given by E [aa(Po Pto))
1/2
s On + Ej VTo { Co Srj ((o)) log N + ( 24/d dt + x (poolyy m))
EK vt, C7 /o ((0 log N (m.) ndt
5 d=d O(poly(logn))
1/2
50, CO"" N/ + C"" V (log))
+C" ER { (TTA--//2(pooyy00g)) +,,,"
dn//2 6x/2 O(poly(logm))
vn

---

Published as a conference paper at ICLR 2025
S Pn + t5n a/ ((polyloog))) C" +C" Ejzk,
vt,n s/120+
( O(poly(logn))
1-ds
+ n
a (pog
In the third inequality, we use J/- ((m))))+mm)))))))d)))))))))))))) dt = (l) for K = 1/2, and the fact that it bounded by a constant for 1/2. Since 77 is arbitrarily large and k 2 1/2, neglecting the factors of poly(log the candidates of the dominant terms in the above expression are /!/2 n उनंत ع=ير dôn
in the third term and t. (22(++)) in the last summation. By balancing these two terms, the upper bound can be minimized by setting
4. C.ni"
(23)
for some contact C,, and the dominant term of the upper- bound is given by
(+io)) 1-6/2 Ó
2x42
(24)
This proves the claim by replacing 8/2 with and absorbing the poly (log) factor in 6.
From Proposition and Theorem 9 if 1/2, the FM method achieves an almost optimal rate up to the poly(log n) factor and arbitrary small 8 > O. On the other hand, for λ. > 1/2, the obtained upper bound is not optimal. This suggests that the choice of or ~ /t around + 0+ is theoretically reasonable. This is also a popular choice for the diffusion model.
4.4 DISCUSSION
In the derivation of the almost minimax optimal convergence rate, the use of neural networks for divided time intervals is a limitation of the current theoretical analysis. As discussed before Theorem 9 without this partition, the current analysis gives only Ö(n =>42), which is not optimal for W2. It is obviously an important question of how to avoid such a time division. In Oko et al. (2023), the optimal convergence rate for the diffusion model has been proved without time division for the total variation, which is Ö(n x.ィz), ). The bound is based on Girsanov's theorem, which gives an upper bound of the KL divergence of SDE by the L2 losses of the drift estimation. To the best of our knowledge, no bounds are known for ODE with respect to the KL or total variation for the difference
of vector fields. This an important future direction for understanding the ability of FM theoretically.
CONCLUSION
This paper has rigorously analyzed the convergence rate of flow matching, demonstrating for the first time that FM can achieve the almost minimax optimal convergence rate under the 2- 2-Wasserstein distance. This result positions FM as competitive alternative diffusion models in terms of asymptotic convergence rates, which concurs with empirical results in various applications. Our findings further reveal that the convergence rate is significantly influenced by the variance decay rate in the Gaussian conditional kernel, where d, Vt is shown to yield the optimal rate. Although there are several popular proposals for the mean and variance functions, theoretical justification or comparison has not been explored intensively. The current result on the upper bound (Theorem 9 provides theoretical insight on the influence of the choice of these functions.
Although this study offers substantial theoretical contributions, these insights are still grounded in specific modeling assumptions that limit broader applicability. In addition to the time-partitio discussed in Sec. Sec.[4.4] this paper focuses primarily on assumptions utilizing Gaussian conditional kernels. However, other FM implementations might employ different path constructions, as suggested by recent proposals Kerrigan al.) (003);; Isobe et al. (2024). The theoretical implications of these alternative approaches remain an essential area for future research.
ACKNOWLEDGGENNTS
This work has been supported in part by JST CREST JPMJCR2015 and JSPS Grant-in-Ale for Transformative Research Areas (A) 22H05106.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Michael S Albergo. Nicholas M Boffi, Michael Lindsey, and Eric Vanden-Eijnden. Multimarginal generative modeling with stochastic interpolants. arXiv, October 2023a.
Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden-Fijnden. Stochastic interpolants: A unifying framework for flows and diffusions, 2023b.
08797
Michael Albergo, Mark Goldstein, Nicholas M Boffi, Rajesh Ranganath, and Eric Vanden- Eijnden. Stochastic interpolants with data-dependent couplings. arXiv [cs.LG], October 2023c.
Michael Samuel Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations, 2023.
Joe Benton, Valentin De Bortoli, Arnaud Doucet, and George Deligiannidis. Nearly d-Linear convergence bounds for diffusion models via stochastic localization. arXiv [stat.ML], 2023a.
Joe Benton, George Deligiannidis, and Arnaud Doucet. Error bounds for flow matching methods. arXiv, May 2023b.
Joey Bose, Tara Akhound-Sadegh, Guillaume Huguet, Kilian Fatras, Jarrid Rector-Brooks, Cheng- Hao Liu, Andrei Cristian Nica, Maksym Korablyov, Michael M Bronstein, and Alexander Tong. SE(3)-Stochastic flow matching for protein backbone generation. arXiv,, October 2023.
Hongrui Chen, Holden Lee, and Jianfeng Lu. Improved analysis of score-based generative modeling: User-Friendly bounds under minimal smoothness assumptions. arXiv, June 2023.
Ian Dunn and David Ryan Koes. Mixed continuous and categorical flow matching for 3D de novo molecule generation. arXiv, April 2024.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion En- glish, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for High-Resee Reihh-eesotttion image synthesis. arXiv, March 2024.
Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma. 3D equiv- ariant diffusion for Target-Aware molecule generation and affinity prediction. In The Eleventh International Conference Learning Representations, 2023.
Satoshi Hayakawa and Taiji Suzuki. On the minimax optimality and superiority deep neural network learning sparse parameter spaces. Neural Netw,, 123:343-366 March 2020.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3D. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 8867- 8887. PMLR, November 2022.
Vincent Hu, Di Wu, Yuki Asano, Pascal Mettes, Basura Fernando, Bjöm Ommer, and Cees Snoek. Flow matching for conditional text generation few sampling steps. In Yvette Graham and Matthew Purver, editors, Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 380-392, St. Julian ss Malta, March 2024. Association for Computational Linguistics.
Vincent Tao Hu, Wenzhe Yin, Pingchuan Ma, Yunlu Chen, Basura Fernando, Yuki M Asano, Efstratios Gavves, Pascal Mettes, Bjorn Ommer, and Cees G M Snoek. Motion flow matching for human motion synthesis and editing. arXiv, December 2023.
Noboru Isobe, Masanori Koyama, Kohei Hayashi, and Kenji Fukumizu. Extended flow matching: method of conditional generation with generalized continuity equation. arXiv. February 2024.
Yuling Jiao, Yanming Lai. Yang Wang, and Bokai Yan. Convergence analysis of flow matching in latent space with transformers. J. Mach. Learn. Res., 23:1-65, April 2024.

---

Published as a conference paper at ICLR 2025
Gavin Kerrigan, Giosue Migliorini, and Padhraic Smyth. Functional flow matching. arXiv, 2023.
Leon Klein, Andreas Krämer, and Frank No'e. Equivariant flow matching. Neural Inf Process Syst, 0bs/2306.15030) June 2023.
Matthew Lc, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vi- mal Manohar, Yossi Adi, Jay Mahadeokar, and Wei-Ning Hsu. Voicebox: rext-Guided multilingual universal speech generation at scale. arXiv, November 2023.
Yaron Lipman, Ricky T Q Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling, The Eleventh International Conference on Learning Repre- sentations, 2023.
Linxi Liu, Dangna Li, and Wing Hung Wong. Convergence rates of class of multivariate density estimation methods based on adaptive partitioning. J. Mach. Learn. Res., 24(50):1-64, 2023a.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023b.
Ryumei Nakada and Masaaki Imaizumi. Adaptive approximation and generalization of deep neural network with intrinsic dimensionality. J. Mach. Learn. Res., 21((14))11338 2020.
Jonathan Niles-Weed and Quentin Berthet. Minimax estimation of smooth densities in Wasserstein distance, Ann. Stat,, 50(3):1519- 1540, 2022.
Kazusato Oko, Shunta Akiyama, and Taiji Suzuki. Diffusion models are minimax optimal distribution estimators. volume 202, pages 26517-26582, PMLR, 4 2023. URL httpps:/pprocedddngg
mlr. D....prrss//222/ko22333 html
Philipp Petersen and Felix Voigtlaender. Optimal approximation of piecewise smooth functions using deep ReLU neural networks. Neural Netw., 108:296-3300 330, December 2018
Aram-Alexandre Pooladian, Carles Domingo-Enrich, Ricky TT. Q. Chen, and Brandon Amos. Neural optimal transport with Lagrangian costs. In ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems, 2023. URL https: /tte::://opezzeviee new. http://wwwweeeoooummmma.
mybCFXBBC9
Johannes Schmidt-Hieber. Deep ReLU network approximation of functions on manifold. arXiv, August 2019.
David W Scott. Multivariate density estimation: Theory, practice, and visualization. Wiley Series in Probability and Statistics. John Wiley & Sons, Nashville, TN, August 1992.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. basre generative modeling through stochastic differential equations. In International Conference on Learning Representations, October 2020.
Taiji Suzuki. Adaptivity of deep ReLU network for learning in besov and mixed smooth besov spaces: optimal rate and curse of dimensionality, In International Conference on Learning Representations, 2019.
Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector- Brooks, Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. Transactions on Machine Learning Research, 2024.
Ling Yang, Zixiang Zhang, Zhilong Zhang, Xingchao Liu, Minkai Xu, Wentao Zhang, Chenlin Meng, Stefano Ermon, and Bin Cui. Consistency flow matching: Defining straight flows with velocity consistency. arXiv (cs. CV], July 2024.
Kaihong Zhang, Heqi Yin, Feng Liang, and Jingbo Liu. Minimax optimality of score-based diffusion models: Beyond the density lower bound assumptions. In International Conference on Machine Learning, pages 601 34-60178. PMLR, July 2024.
12

---

Published as a conference paper at ICLR 2025
Appendix
This section summarizes some basic mathematical facts, which can be easily derived and used in the proof our main results, and known facts developed in Oko et al. (023
A BASIC MATHEMATICAL RESULTS
A.1 DEFINITION OF BESOV SPACE
Besov space is an extension of the Sobolev space, allowing for non-integer orders of smoothness, and effective in measuring both the local regularity and the global behavior of functions. It formally defined as follows.
Let Ω be a domain in Fd For a function f E Lg (St) for some p' E (0, oo], the r-th modulus of smoothness of f is defined by
Wr = sup ||A||25e
|
where
(x))()) = E f(x +
if + jh for all j,
otherwise.
For 0 < <// S x0,8 0,r |s| + 1, let the seminorm
11 Pig' be defined by
(10 wr (f,1))
(9' oo), (d'= DO).
Ml
supr t (1,
The norm of the Besov space Bi (22) is defined by
[Ifll p'.u = |1fll + lfll
and
By o -= E ( IIfIl < oo).
The parameter 8 serves as the order of smoothness. If = q' and is an integer, By coincides with the Sobolev space. For details of Besov spaces, Triebel (1992), for example.
A.2 LIPSCHITZ CONDITION
This section shows a lemma that provides a Lipschitz constant of - vy (x) under the assumptions (A3) and (A5). Lemma 10. Let (x( be a vector field defined by (3) and B). i.e.
vz(x)) = / {00 at
myy
+ m/y pe po(y)x))dy
where
(e( y)po (P)
((())=)= J pe(x ÿ) ( Then, under (A3) and (A5), (z(x) is Lipschitz continuous with Lipschitz constant CL/t for any sufficiently small t, where Č2 is independent tft.
(r(z)))
20
(V2Toa))
Proof. By the definition of and the form of a. and m. in (A3), can compute explicitly
@v,(x)
Ia / { (x m.y)
+ R Op.
(pp(yy)))
dy.
dx
t
dx
8. As Da
| = am
8 1 = O, we further obtain
(v)(
K La
mck θ
Ox. / (p:()/
0x
t
The claim is obvious under (A5).
13

---

Published as a conference paper at ICLR 2025
A.3 WASSERSTEIN DISTANCE FOR CONVOLUTION
The following lemma is used in the proof sketch of Theorem 9 where W2(Pr, Pro) is bounded.
Lemma 11. Let P be probability distribution HR with V == |Pl)) < cc and Pm, be given by the density J (v2ne)d exp(
| m
)dP(yy) Then,
20
W2(P, P P) V(1 m) do²
Proof. The proof is elementary, but we include it for completeness. Let Y and Z be independent random variables with probability P and Na(0, Ia), respectively. Let == my oZ, then the distribution of X is Pm.o. Considering coupling (X,Y), Y).
Pm, < E X Y
E Fm 1)Y + 2
(1 m) m) do?
which completes the proof.
A.4 APPROXIMATION OF FUNCTION IN BESOV SPACE
In this subsection, we present several approximation results developed in Suzuki (2019 Oko et al. (203). Although these results are already known, we include them here for ease of relerence.
Let (x) be the function defined by (x) 1 for x E [0, 1] and 0 otherwise. The cardinal B-spline of order E N is defined by
N=( := N N (z)).
which is the convolution (l + 1) times of N. Here, the convolution f is defined by
(f * (x)) / f(x v)g(y)dy.
For a multi-index & E Nd and j E Zá, the tensor product B-spline basis in Rdd of order E defined by
d
M (z) = II NG x, -j).
i=1
The following theorem says that function f in the Besov space is approximated by superposition of M (a:) of the form
[N(Ø) = Σ on. M Ma (zc).
(k.f)
In the sequel, we fix the order E of the B-spline. (a)+ denotes max(0, Theorem 12 (Oko et al.
(2023); Suzuki (019)). Let C O and 0 < p',4'r S Under s > d(1/p' I/r)+ and < s
I+ 1/p'), where E N is the order of the cardinal B-spline bases, for any B of (I-C (I-C, C,C)), there exists IN that satisfies
< minfe
|f fv (. S C (- (1-000))
for N 1 and has the following form:
7 ak M (z)
fx(x)= Σ
Σ
aks ML)ME
(EJ(K)
with
K"
Σ
[(K)+
Σ
n& = =,
k=
k=
where J(k) (-C2 l, - l 1...., C2 1, (1i) J(K). K K' (0(1) log( log(N/C"))u-L K, ny 0((N/C"M2-Mt--K) (k
K + I,..., K" for v= = (s w)/(2w) with w d(l/p' - -1/r)4. Moreover, we can take Ork. 50 that Jakal & (v= (d/p' -0)+
14

---

Published as a conference paper at ICLR 2025
Based on the above theorem, the following result shows the accuracy of approximating the true density Po by the B-spline functions with support restriction.
Theorem 13 (Oko et al. ).023), Under Assumptions (AI)-(A5), there exists fN of the form
3N
[N(M) Σ a ||
S M)MM Re (x)
Σ N=++1
05
||01|00 S N N- 1-8
ME.. (z)), (25)
that satisfies
||Po fNllE2(re) CaN-a/d
(26)
IIo - IxIILP(II))))) S CoN-
(27)
for some Ca 0 and (x) for any with ||> 2 1. Here. -2(k)) l (i)) I 2(())
(i = 1,2....,N m = ..2.....ce d). |ky I K = (0(1) + log N )v + +(d Oog for v
log (2s (2w) with d(1/p- 1/2)+ +- The notations (ki) and (js) the m-th component of the multi-indices k; and ji. respectively. Moreover, we can take las S NNv + d d- (d/p 8)+-
Proof. See Oko et al. 2023 Lemma B.4).
The following result shows the accuracy of approximating the "smoothed" B-spline basis function by a neural network. This essential consider the approximation of p. (a) and v, by a neural network taken for (z).
Theorem 14. Let C 0, k E j E Zd, E Z. with -C2 S js S C2 (i ........ d). and Cp and &4 (x. t) in M(L, W, S, B) with
(-(1-)) For any E (0 < E < 1/2), there exists neural network Φ3 (z. t)
E 1 + log 2 C' k²²), L Olog
| || x O 6 1 + log" C' + k²),
S (log -- log C k*),
B exp(O loglog -1 log C k))
(28)
such that
1 1llvlloo S Co.1l Mf Oss (x, t) -1)) f
my 20
ME, (v) (v2TT,))
dy
Se
(29)
and
m-myy 1 ll S Ct,1] M M1,(() (3))
mll-m
04
(z,
20?
dy
SE
(30)
(V
Tt
hold for all x E [-C, Furthermore, we can choose the networks so that |105 1l and 11 lloc of class O(1).
Proof. See Oko et al. 2023 Lemma B.3)
A.5 APPROXIMATION OF GAUSSIAN INTEGRALS
The following lemma is elementary fact about Gaussian integrals used in the proof of Theorem in Section C.4 We include it here for completeness.
Lemma 15. Let æ E Rd 0 < < 1/2, and E {0, 1]. For any function Fly) supported 14 there is Cn that depends only on such that
Ur (V2TO.)) where
1ammaall 20?
/ An (v2nos))
I1-- wil 20
Fly)dy
F
S E, (31)
A =
y E ld
x y my
SO.
Te avVlo N
m
Proof. See Oko et al. 20023 Lemma F.9).
15

---

Published as a conference paper at ICLR 2025
B PROOF OF THEOREM 3
Although the proof of Theorem is basically the same as the proof of Benton et al. 2023b Theorem 1), slight difference appears since the current bound shows for arbitrary time t. We include the proof here for completeness.
Let v(z t) and ((;;;) be smooth vector fields and sa,t and P.,t be respective flows;
(..1(()) v()).((2))
(s,)(x)) I,
d
di (ac(z) = (x) (2)
Lemma (Alekseev-Gröbner), Under the above notations, for any T
(2o) - (p,,r = L." (VeGer(t)) r (z(P... s) ( s)) (32)
Note that on the right-hand side, the vector fields e and are evaluated at the same point Po.. (x.)
Proof of Theorem Let P = (50,c) Po and P. == (PP,,) Po be pushforwards. By the definition of the 2- 2-Wasserstein metric.
Wa(Pi, Pt) S ( |1 dd) 1/2
(33)
From Lemma T6
(zo)) (x.)l < S. \
(to (zo). =((po, 8) ds.
As general relation of the largest singular value, we have
a
(x)
Then, follows from
V
G.. (x) V (_ V (x)
that the inequality
00
i at
(x)) (mp))
holds by the Li-Lipschitzness of 6(- t). Accordingly, noting that | S (x) = //V the standard ODE argument leads to
V 2 (x)) (=pp. 11, S ell Ludu
and therefore
(x)) (x.((=)))l s S ( v((po,((Zn) s)llds
From Cauchy-Schward inequality,
|| (wo) (2.1(2)))12
I 1'ds (' L. (((po. 8) v(po,, (2x)), ds|
f e S budy (:d0 8) - (x.. ds.
Since the distribution of Po.. (2)) with un ~ Po is given by P,, combining the above bound with 33
completes the proof.
16

---

Published as a conference paper at ICLR 2025
C PROOF OF MAIN THEOREMS
C.I PROOF OF LEMMA
J SUP OF LOSS FUNCTIONS
Proof.
1.m / (1( vx(+مم=)|| Pe (a( dxx
To
(x(x)
Tt
To
/ /1ob ( + 1%
(T) / [1xe((2)))))
2
(at
To
Tr
From the definition of Hn and Assumptions (A3-A4), the first term is bounded by
4C [" (oo])logg + (m!)²) dt s SČ ((log m)6+1 + (logn)")
where Ö is a constant, and b = for K > 1/2 and b by for k = 1/2. From the definition
0y ( [z)) o. Tt m4 + m/.
ot
and the fact ||>|||| s 1, the second term is upper bounded by
4 f'' / (00) - - m2a0l? (m/)ll=ll de(
< /" (d(o))² (m))22 di S AdD(logn))
To
for some constants D > 0. Note that the first inequality is given by pr(a)l Na(ma of
This completes the proof.
C.2 DIVISION POINT
The appropriate division point of the time interval [Tg, 1] arises from the width ag of the smoother region around the boundary of Id which is assumed in (A1). Suppose that the smoothness of Po(x)) is 8, higher than 8, in the region [d\[-1 + an, 1 - ag) dd_ We should make the smoother region as small possible ensure that po is essentially in B
We ag N and consider the partition of V-1 + ap, 1 - ao] by N - (NV - 2) cubes of size ag ~ (see Figure C.I). Suppose that we use N bases (N >> 1) of B-spline for each small cube with arbitrarily small 8' >> 0. The condition 8' > guarantees that the approximation can be arbitrarily accurate in each small cube. Since Po restricted to each cube has smooth degree 3, Theorem tells that it can be approximated by a B-spline function with the accuracy
Gn N /d
The total number of B-spline bases is then
(Ndy (N - 2) ~ Nid-11)+4''
To make the number of bases equal to or less than N, which is the number used for the -r..iooeion of Po. we set 7 (1 5x) /d, i.e., dg = (N11-6x) (we set 8' 8n notational simplicity).
As seen in the proof of Theorem 7 to obtain the desired bound, the deviation should satisfy at S ag to bound the integral around the boundary. When it small that a, t" a; S ag means I S T. = (- 1-6)/d which gives a constant on T., Consequently, we divide the time interval into [To, T.] and [T.; 1], and show the different bounds for the approximation error.
C.3 BASIC BOUNDS (.))) AND (x)
Recall that
P((x)= 1 J1-1111 (2
2ng
Po(y)dy
17

---

Published as a conference paper at ICLR 2025
Figure C.I: Division of the cube into smoother small regions and the general region.
Lemma 17. There exists C₁ = Ci Co) > O such that
(11201100- -m:)
- mx)
OT exp
S pr())) S C1 exp
(34)
20
for any a E I&d and E [To.
Proof. The proof elementary and the same Oko et (2023 Lemma A.2). We omit it. Lemma 18.
(i) Let k N be arbitrary. There is C2
C2(d, Co) such that
10., -2, p((x))
(Vt E [To 1]). C2
of
(ii) There is C3 C3(dd Co) such that
(1101100 (1 me)+ v1) |m/}
|[)(())] s Cs Csfor
for any x E Rd and t [To. 1].
Proof. (i) is standard and the same (Lemma A.3 Oko al. 2023 We omit it.
For (ii), let g(æ; myy, of) = (x:l)) (2mmr) 227z
exp(
kl-m,yl? 2n
}. Note that
f (,(()))) m.y. (o)
ve (α)
(r(x)
The norm of the numerator is upper bounded by
x m.y + m4y g((:; myy of (o(())d))
may
g(a;may, of )oll))d + |m//l /
|yyll((x; m.y, o. oo))ody)
σe
Since Supp(po) = [] by assumption (Al), the second term in the last line is upper bounded by |mi| / g(x:; m.y of (o (x()).
(35)
To bound the first term, we use the restriction of the integral region as Lemma F.9, Oko et al. (202)) Namely, letting == CT exp( approximated as for any j = l,...,d,
[||= m the lower bound of pz(x), the integral is
20
m,y ) g(x; o
I --mmy)
g((;; of S
Rd
Ax
18

---

Published as a conference paper at ICLR 2025
where Ag IT 2i Cae v log(1/E), my + Cai only d, and {0, 1). Then, the first term is upper-hounded by
my [log(//E]] C is a positive constant depending
may oz
Noting that y E A₂ is equivalent to - S CV/o the above quantity further upper-bounded by
/ g(x; o(::: voe s c'((;, C"((di//og(1//)mol + e).
As
Noting that 5 0T exp( [| [| and (x 2 E, we obtain
C² (ooo//log(1/3 (l/g(L)))p((@)/ + E) + (m/lp((x)
||0(())]
(x
o (|| - mi)+ + o% + |mm'l
50"
( -me)+
[| (1 - me)+ v 1) + /mm/
=C""!
of
Ot
This completes the proof.
The following lemma shows upper bound of vz( when æ is in bounded region of oz I/e. and presents bounds of relevant integrals.
Lemma 19. Let e > 0 be sufficiently small.
(i) For any C4 > O. we have
[|,,())]| S (lod)// (m//)
(36)
for any æ with |00 my Coooy/lo((14 and t [To. 1].
(ii) For any Cs O. there is C > 0 such that
J 2mll (ogr/// (og(1/e)
po(x)]lu(<))] s (((( log (s-))++)mmm)) 1ogolog"
d-2
{5-1)} c7
(37)
and
Pe Pc(x)dæl < Ö'og + + (5- c3
(38)
Jall 2m Cea, (oR(1/)) 0 and t [To 1].
hold for any
Proof. (i) is obvious from Lemma 18 since (1101100 - me)+/or S C4vlog(1/s) under the assump- tion of æ.
We show (ii). follows from Lemmas] 17 and 18 that
(||=]| - mma)
((o/)e - m + (m/)2} oā
(,,)) < 20,03 exp
20?
and B, = far = (xi...., x.)) E Rdd |xs]| maxisisd |4yy1 The space Letr = (|=)
divided into d regions UgL_ B, with measure-zero intersections. In B,, the variables T2. Fd satisfy |2,| & m + C.o, (1g((/E), and integral (37) is upper bounded by
201Csd /
e - (1))),22 + (ml)² (oor + mz) dr
CasToR(1/E)
sC' Sanvbattrs
- ((6)))2,041) + 2 Idr.
19

---

Published as a conference paper at ICLR 2025
where we use (ore + m.) (r + 1)²=1 < 22
For E N U [0]. define
4e(z) { p'c - dr.
It is casy to see 1/1 (2) e * and (x)2) x a + (l 1)v'c- by partial integral. Using these formulas, we can see that for 2
vx(2) < Be e-
where Bg is positive constant that depends only l. Thus, we obtain an upper bound
(((o))² log (-) + (m,)² log 3 - o
15
8
which proves
37 The assertion
38 is similar.
C.4
BOUNDS OF THE APPROXIMATION ERROR FOR SMALL t
This subsection shows the proof of Theorem
(I) Restriction of the integral.
We first show that the left-hand side of 20 can be approximated by the integral over the bounded region
D (a E R
||>|| oo I m, + C4o. log.//o N).
To see this, observe that from Lemma 18 (ii,, for x E Dr.N we have
[<=](=)] < 20 ((a))² log + |m
(39)
From the bound of v. (x), we can restrict the neural network ((x, () so that it satisfies the same upper bound. Therefore, (3) is applied to [| also. Combining this fact with Lemma (ii), we have
J | () vx(x)) "x (x))a
< <00 {(,1)² log N + [m]
20
me
dx
DEN (V2TTO)
< 4C3C'' {(o1)² log N |mm }] N /2/2 log
4 N.
If C4 is taken large enough to satisfy C3/2 >
28 d it follows that
f [1o(x, t) ue (2.
$
[0(x,)) t) - (x)) + C' {(o))² log N + [m/ N- -
(40)
D.,x
for some constant C' > 0. Thus, we can consider the first term on the right-hand side.
Let > 0 be arbitrary positive number. The integral over D in (40) be restricted to the
region {a (x())= 2 N 2rt"" }. This can be easily seen by
/ D.N
1pp((x) SN N 2nt"" I1(oz(#)) 6lx, t) 11 (x
5 1C3
/
{(o)) log N + +mm N 2m da
Dr,N
< S4GN 20t0 {(ot) log N + +mm( 2
Caor / log NYd
< n ((o1)² log N + |m| 2] N 2ntw
log N,
20

---

Published as a conference paper at ICLR 2025
where C" depends only on d, Cs. and C4. This bound is of smaller order than the second term the right-hand side of (40). and thus negligible. In summary, we have
f [1bb(, () vr (r
= S (xp)(x)) 2 N 2
[1,((x,1) - N
(( + C' {(o))2 N + +mmm]]]] N -
DEN
(41)
for sufficiently large N.
(II) Decomposition of integral. Here, we give a bound of the integral / [1 (z. (x) (x
over the region DI.Nv n {x Pa(2)) 2 N "="" }. The norm (1 t) (x))l is bounded in detail. First, recall that
Ur (x) I (x) (3))d
Pe( - pr(arl))ooo))d
(42)
Pr (x))
Based on Theorem 13 can find fN in 25 such that
Po fN ( S C Il fallE((1a)))) S CaN-afa
(43)
As an approximate of pz()), define a function fI (x, by
(e = / (V2TO,)
we myyl? 20
fx(y)dy.
(44)
Since we consider the region where Pt (x()) 2 N (2 we further define
fi fi f) fi
In similar manner, the numerator of (42) can be approximated by
ff, fx,,(x)tt.
where TR- functions f2 and fa defined by
f2(x, = / x m:y
1
miwi? fN(
e
2o
(V2TT,))*
fs(2,t) == / 1 (V2TTo))
Jon-muu12
20
fN(
(45)
We have an approximate of (xx) by
fa(z,t) = o'f2(x t) + m t) 1
fz fi
fs N
CsVlog.N
S
<C²]
fi
fi
We want to evaluate
S JO.N 1/px(x)) 2 N 2eeis || 1) U2 (x)
{
1p(( 2 N 204z |1 t) - fa 1) (x)
<
Dr.N
+ #f
1[pr(x)) 2 N س2ح fa( (z
D.NN
TA + TB.
(46)
(III) Bound of TA (neural network approximation of B-spline)
21

---

Published as a conference paper at ICLR 2025
We will approximate Si, S2. and f3 by neural networks. For ht E4 and j 24, let E
(a 1,2,3, 0, 1) denote the functions defined by
EbJ.u
(2)
11ly/lb I Co,1l MM (1)
2ng
dy.
(47)
JRd
(v2To,)
k.j.a =
Jen
{
x mmy
111/llo S C6,,1) M
I- mmmmall
20
dy.
(48)
(v (/TOT)))
and
((3)
| Rd
yl |11//100 S Cb,1] (1) (V2TOr)) )a
1m-maul 201
dy
(49)
R.J.u
where C₁,1 = for u = and -1 written as a linear combination of 1 11llyll I Co,1lA4, with coefficients ag. From Theorem for any E > 0, there are neural networks cis do and Φ7 such that
for u 1. Then, from Theorem 12 fN
is
1/i () az () _ D5 max logle
[ t) Φ6(a, 1) _ N&N max axle,
ff,( (, t)ll - D,N DaNmax ag|=.
Since max, |asl S (v td (dd/--a)+, by taking e sufficently small, for any > we have (a 1) (2, 1)l < D
(xx -0o(1, t)ll < D
(xx -x,(x,t)) - DyN
The operations obtain the approximation of (x)) based on Φs. Φo, and Φ7 are given by the following procedures:
Ci = cl ( (2+tw)/ N
C2 recip((i),
C3 = mult((2, Ø6),
Ç4 = clip((s; -oy
CoVlog log N, Covlog
C. = mult((2, Φ7),
Ç6 = clip((s; -C, Cs).
S7 = mu ((1,
&3 mult((o. m²),
Φg = +
As shown in Section D the neural networks clip, recip, and mult achieve the approximation error N- with arbitrarily large 7), while the complexity of the networks increases only at most polynomials of N for B and ||]]] and at most poly N) factor for L and S. In the upper bound of the generalization error, the network parameters B and ||W|] and the inverse error S N appear only in the log() part to the log covering number, and S and B appear as linear factor. We also need to use the approximations dj and mi of oj and mV, respectively, by neural networks in construction. However, this can be done in similar manner to (Section B1, Oko et al.) 2023) with all network parameters (log" E11 for approximation accuracy E, and thus they have only Opooy N)) contributions. We omit the details in this paper. Consequently, the increase of the neural networks to obtain Φg from Φs. de. and Φ contributes the log covering number only by the poly(log N) factor.
As a result, we obtain
LA OON
(50)
for arbitrary 7 > 0, while the required neural network increases the complexity term only by Oppoly(log N)) factor.
22

---

Published as a conference paper at ICLR 2025
(IV) Bound of Is (B-spline approximation of the true vector field)
We evaluate here
IB = D.N
(pp.(x)
N 22 111
pr
(51)
Let 22(2 1) and h 1) be functions defined by
I my vil
h2(x,,) t) = Jes x m.y
20
Po(y)dy,
at
(TT,))
h3(2r,0 1) = 1 y
1
"V22Tor)a
e
20
poly)dy
(52)
Then,
[Ifa(2),)
1) vx
f2 5
fa
of off2(1, 1) + mifa(a,t) t)
o/ 1) + m/hes(a
=
s CSvlllo N
S Cs
f(
Pr p+(x))
(53)
We evaluate the integral (51) by dividing D..N into the two domains { {ær m, S ||oooo S m. C N).
| |oc S m² and
Due to condition (x)) N-(2e+w)/d, it suffices to take the network (ax, t) so that it satisfies fa t) 2 N2(2++w)/d by clipping the function if necessary. We therefore assume in the sequel that fi t) N w2d holds.
(aV-a) case: ||a:||o S Mi
In this case, Lemma 17 shows that CT S px) S S1 for some C1 > O, which depends only on d
and po. Using this fact and the conditions || S Css Covlog and
f Cs. we have
f2,2 t) + max,((xtt) ,( t) m/ (xx,
(x)
(x))
f2 t)
h2(2, t)
fs(
hz(x,t)
$ $/01
+ |m/il
fi(
pr(x)
fx
Pr(x)
fx
f2(x,t)
f2(
122xx,t)
|o'l
(x)
(x)
(x)
(r(x))
{11 ff((a)
fs(a, t)
fs(x, 1)
h3(2r, t)
+ +mmil
+
f1(
(x(
P((x)
pr(
<C Cs Vlog (e) - fi 6) + |1f2(x t) - h2(
+ (Calp(())) - fixx,)) + [ t) - t=s
$ sCi(ll))) Vlog N + mm - (x()) + f2 t) - h2 t)ll + |m] f) - 1ss t)]1
(54)
We evaluate the integral on fx || 30 m,). From the bound 54 have
TB =
Sm [pp((x)) 2 N 2nd 1 fax t) ve(xx)) (x
{(o')8 log N + (mb)²} /
S c"
1lp((x) - [1 t) (2) (x)
/1lll Sm.
+ (o())
1pp((x) > N 2atu
12 t) 12( 1) (x
+mm))² { Smm
1(p((x) > N 2 f1s t) 12,( t) (x)
(55)
23

---

Published as a conference paper at ICLR 2025
We write J J and 8 for the three integrals that appear in the right-hand side of (55)
We will show only the derivation of an upper bound for J since the other two cases are similar. Recall that by the definition of f2 and hgg
fa( 1) h2( () - f
my
2
(NN(D) - po (o)))d
or (V2To,))
Then, using (x)) & C1, we have
2
1 S
(x) 2 N 2-
m4y
20
(fN(z) po(L))dd
dx
Smllasmme
14
Tt
(v2To,))
1 1lllrlloo
1 1]
may
m
Iu-n/me1 ((re)
(JN(U) - po(())dy dx
Siz
mg JRd
0t
V2TTO
<
2d
C1
/|te|l
20
1lllerloo 51]
my
2
m
d
e
lu-m/me12 2(te/mm)
(JN(y) - -o((u))² dydde
11
Sma
at
C1
/
11 $
11] Te
x myy
2
Immmaal 201
(fN(y) - -o(l))) dydx
= m
|1ll
Smm
(V2mor )d
(56)
where the third line uses Jensen's inequality for || 11. For / E CNN with sufficiently large
N, we can find co such that m, on on the time interval [To.3N 3N We can thus further
obtain for some C² > O
J SC SCSC.
||00 mayl?
20
dx(fN (y) - -o(l)))
1a JR.
of
(V2no,))
dC' J. (IN(U) - dd
dC'I|SN - -poll2-0 S NC
(57)
by the choice of J'N. Similarly, we can prove that (( and ( have the same upper bounds of N - order. This proves that there is CB.1 > O such that
BB CB.1 (o!)) log N + (m))² N 3.
(58)
(VI-b) case: m. < ||0||xx S m. + Coorylog N.
Unlike case (i), we do not have a constant lower bound of (x:) in this region, and thus we resort
to the bound (x) 2 (d that is 1/p,(x) S N(2=+w)/d and (zfi t) S (22a+a)/dd We have
off2(x, t) m/ (xx,
o(h2(x, t) mx)
f1(z)
(x(x)
< tx |1 1) + m(f2(a, (oiha(a, () (x))
+ +[])((a))] f1 t) [f((z) - (xx)
S N120tw//d (lo' (lo'iVToge log N + m mm))) fi (ax, 1)I + (ollllaaax 1) - 12( 1)ll
+ + t) t)(
(59)
where in the last inequality we use Lemma 19 (i).
24

---

1-dx Published as a conference paper at ICLR 2025
Let ANN == fær R my < | I m + Cior Vlog N). By the same argument using Jensen's inequality as the derivation of (56). we can obtain
IB2
[p)((x))
2 N 2 fll t) pe(2))l(p(<<))d
D.N
S C N 21120
((o))2 log N + (ml)²}
/
20
mavl wll (fN(z) - p
De, J1a (v2TTo,)
+ (o'))
SS
m
a - myy
1
m
(fN(y) (pol))) dydd
JAN JTa
SAIN /
1
myyl?
+(m))²
|1y11?
(N((y)
dydæ
(60)
(V2Tor,)
Due to the factor [/(+++=w//d the integrals must have orders smaller than in the case of (66) to derive the desired bound of I B,2. We will make use of Assumption (A1) about the higher-order smoothness around the boundary of Id
Because the three integrals can be bounded in a similar manner, we focus only on the second one,
denoted by k-(2) Since 6,w > 0 can be taken arbitrarily small, we can assume A >> 68 1 + ōx 2w. R
From Lemma 15 with E = N- there is C. O, which is independent of æ, t, and sufficiently
large N, such that
myy
Im-mayl2 20?
(fN(y) Pol
Te
(v2TTO))
m,y
myyl?
(fN(z) -poy)))dd I SN-3, (61)
(v 12TO,)
where Ar is given by
divlog N
As = y E Idd
y
S Co
m
00
m
Note that if x E DN, and y E An. then
-1 Øj < -1 Crorvlog N
Csoevlog! N I y3 2
or
m
m4
for each j = I,...,d. d. Because we assume that t < 3N 3N- and oN ~ bnt" we can assume
that my 2 Du/2 from Assumption (A3). Then, for sufficiently large N, we see that y Id
This can be seen in o, ~ bot" < 6o3 1-dx and thus Coo.vlog N /mm S 2Cahu3" N a For
vDo
E 1 we can use the second bound in 25 : IIfN pollza(1au1)) S N s/d It follows from
(6 that
K
x
myy
1
Im-ma 21
20
(Jv(y) (
Do,
0t
12Tor)a
ln-mayl2
M:y
(JN - po(l))) + NN-N-/ 8/d da 20
JAIN
An
O2
(v (/TOTO dd
myy
lm-mayl 204
dx( (
A
Rd
T2
(v2TT,,)
Since the volume A is upper bounded by D'o.yoo with some constant D' > 0, we have
k!2 < dllfN Poll2-(443)) Nog
< C" (N-28/d + N N- 1/2 N
o N الن log 1/2 N).
(62)
25

---

Published as a conference paper at ICLR 2025
where the last line uses 8 >> 1 (A1). The integrals K and K have upper bound of the same order. As a result, there is CB.2 > 0 which does not depend on n. or / such that
IB CB2 {a/1) log N + (m/)² N
Since have taken that 8 6s di + 2w,, we have
TB2 - CB22 (o1) log N + (m/)² N 2
(63)
(VI-c)
follows from 58 and (63 that there CB > such that
TB S CB {o/)) log N (m,) N
(64)
for sufficiently large N.
(V) Concluding the proof.
Combining 41 (46). (50), and (64) leads to the upper bound of the statement of the theorem. The argument of the network size in part (III) proves the corresponding statement.
C.5 BOUNDS OF THE APPROXIMATION ERROR FOR LARGER !
This subsection gives proof of Theorem g The proof is parallel to that of Theorem in many places, while the smoothness of the target density is more helpful.
(I)' Restriction of the integral. In similar manner part (I) of Section C.4 there is Cs 0, which does not depend on / such that for any neural network Φ(x, t) with ( a t)ll S Ca(lo/IyTog [m/]) } the bound
u, dx) 2 floy! / Vlg N |ml))N" Pe p((z) J
holds for any E [N- 1]. Also, in a similar way, we can restrict the integral to the region {or (x) N ") to negligible difference. Consequently, obtain
(=)))(((zz =2)
[ellssm (x) 2 "l (<)(,()) - 2 + +Ol'o'l + |m]l)
(65)
(II)' Decomposition of integral. We consider B-spline approximation of Pr (x). Unlike Section C.4 we can regard Pt, as the target distribution, which of class C This will cause a tighter bound than in Section C.4 easier analysis. More precisely, it is easy see that Pt. the convolution between Po and the Gaussian distribution Na(m,y, of Ta), can be rewritten as the convolution between pt.
and Na(may, for any t.. that is.
Px(x) Jgd (v
28
Pt. (y)dy,
where
m
m
o7
m
o²
(66)
mi.
m,.
We thus apply similar argument Section C.4
We B-spline approximation of Pt.- For v O, take N such that that the derivatives of Pt. (x:) satisfy
3dq It is easy
ak
C₂ oi.
Per (2:
<
Or, ... -a
26

---

Published as a conference paper at ICLR 2025
for any & I a and (d1...., .x). From Assumption (A3), there are tt E [0, and by > 0 such that
oi 2 bit for any 0 $ tt. If we set c¹ := (tt)² y > O, then o, 2 bit V for any t E [0, 1]. We
can see that
Pt.
E Be (
aa V ct
holds, because for any & S we have
Ok Pr. (2))
Ca A (cl)
S 1 A(c)) (a(((((e)))
Or., drin to an Vet
ar V ct
which implies
Pe. vel
W
(RR))
and
y I1w2 (B4) S Ca (k+) (constant).
Notice that, by a similar argument as in the proof of Lemma (i), there is C3 > 0 such that
Jw 20 Vog
(1lv 1)pr (y S N
(67)
holds. We therefore consider a B-spline approximation on [-Cs v log N, Cs Clylog Njg. Letting
N, = [t_dd No01
be the number of B-spline bases, from Theorem 12 we can find function fN- of the form
fN-(2) at V cf)
1 ||]]||>00> S Cs Covlog N MIM (x)
1=1
with las1 I 1 and Cg > 0 such that llr.. SN- 1x N.C, Aog Mj1 S Collog (Nt) (1_ v ct)
holds.
From No 2 t- dx 26 and a
3dn
the right-hand side is bounded by
25x
C² (oooogN) N)a/ N NaPeN-oaan don/dd < C²N 33//2 -3r/22
for sufficiently large N, which implies
lp.. fx- Poll-CooloNN.Cvleee
S C10.N 3n/2
(68)
for sufficiently large N. Note also that the coefficient a; == (1, v ct) of the basis Md. kej. in fN- bounded by
| S t. v ct <N 1- N oll=án)
In similar manner to part (II) of Section C.4 define fI, f2, fs. and fa using fN- as
(1x, = (1x, N with (xx, = /
20
JN- (y)dy,
(69)
(V2Trō))
f2(x0 : f
x muy
1s-maul2
20
(N-
ōt
(V2TTØ,
fs( = [ (V
1n-maul? 207
fN- (v)dy.
(70)
fa(x, 6.52(1r, 1) m/Ja(aa
S Cs Csvlog N
fa
S
<Cs
fx
f1
fi
We have a similar decomposition of the integral to (46):
f/00 D.N 1(p((x) 2 N "1l@(((z, - 22 (z) (x)
1(p((x) 2 N~"|| fa (x))
DAN
of D.N
N t) ox(=))
=: ĪA + îB
(71)
27

---

Published as a conference paper at ICLR 2025
(III) Bound of A (neural network approximation of B-spline)
Using exactly the same argument as in part (III) of Section C.4 we can show
A = Opolly N)N
for arbitrary v/ 0, and thus it is negligible.
The size of the neural network is given by L = Ollog* N), || O(N), S O(N*) O(t,a ((*), and B = exp(O(log N log log N).
(IV)' Bound of Is (B-spline approximation of the true vector field)
By replacing po with pt.. in 52) define (2 6) and ha(a 1) with my and o. by
[ I -myy
12( () =
Pe. (y)dy
JRd
(V2Tō))
12(2, t) = [ (V2mmo)d
Ilm-m u0l 2ng
Pe. (v)dy.
Then, by a similar argument to 19p. we have
(z)) < ~"C (151) Voll + (m/)) (z(x) f1(x,)) t)
f1 12 t)ll fmllllfs(a 12( E)
for some constant Ö and thus
I Da.x Jec (V2Tōr))d
na 21 (fN-(V) Dr. (x))dy dæ
12
In s 22 {(o1)2 log N + (mb)²)
+ (3))²
{
Jpu ōe
m4y
207
m
(fN-(y) - Pp: (2))) dx
(V2TTr))
De.x
Iane y
le mevll
+ (m,)
JoLN
20
(NN(2)) Pt. (e)())))))) dx
(72)
(V2TTo))
Here, we show a bound of
JB.2 = 1
Jo
x my
Im meul
207
(fx-(y) Pt. (u))dy
dx.
J.,N
JRd
ōt
(V2rrō.)a"
The other two integrals can be bounded similarly.
Let p = VED, > O, where Do is given in Assumption (A3). We derive bound of J in the two cases of separately: (IV-a)" mi, 2 p. and (IV-b)' th S p-
Case (IV-a)': m 2 p.
28

---

Published as a conference paper at ICLR 2025
By rewriting the inner integral on by Gaussian integral, we have
in Iy m/mml
JB2 =
m.y
m
28f
(fN- (3) Pt. (w)(dd) dæ
DU.N
Rd
V2
llu- /mml
x m
m
(fN. (y) - -p:- (3)) dyde 201
2d
DUN
Rd
x m4y
Ne
<
20
(JN-(z) Pt. (y)) dydæ
DAN my JBd
(V2
)
< (2Do) 4/2
x m
1
28
(fN- Pt., (v)) dxdy
Bd Rd
0t
(v2Tō,))
S d/2d
(JN- - (y)
S 4/2d
J-CCvVoorN.oswoor (fN- Pt. (t.(()))) (y)) +
|1v| /IIl2CeViONN Pt. (x)
S p 4/2d { [1.NN - Pc. Pr..1::21 oow.l/ee + -N-31]
SCN 37
where the second line uses Jensen's 's inequality and the last two lines are based on 67 and constant C' does not depend on or N.
688 The
Case (IV-b)' m. S
In this case, we can show oī 2 2Db 1. In fact, from mi S p. we have
o2 = o mł 2 of p² p²oo...
From Assumption (A3), m of 2 Dj Since mi p² mm, by assumption, we have
of 2 DT m 2 DT - p'm
Combining these two inequalities, we obtain
o, z - (m,, + p Dj p² Do = 2Do
where the last equality holds from the definition p == v2D=
We divide the integral of Jn,2 into the regions {y |/ 2 Cavlog N} and {y |ly|l S Csylog N). In the region {y |l 2 Csglog N}. using oā 2 1/(2Do) and fN. (y) = 0, we have a bound
M
m.y
|11 (1llll 205vTng
25?
(SN- (2) - Pt. (y)
ōr
(V2TTō,)
Do
(2Dn)2
||
muyl (pt, ?y
Do (2Dn)²
( (C'log N + 11 pt. dy)
C
< C 3m log N,
where we 67 and the fact Dr, implies
|| < C' log N for some C².
29

---

Published as a conference paper at ICLR 2025
For the other region fy moyl//o s D' Vlolog N for some D' > 0. Application of Cauchy-Schward inequality derives
llylloo
Css /log N), first notice that for a DI,N, we have |
M (1lw) oo So
m Ôt
muyl? 202
(Jx-(y) - Pr. (y))dy
(V2nō,)
Do / log
/
dy
(JN- (y) - Pr. (2))
5Co
(llv) do SCovlog
S D"ogg ) [| I - Pt. [P N², Coyloo
< D N
From the above cases (IV-a)' and (IV-b)', we have for any & [Lx, 1]
JB2 S poly(log N) N
As consequence, there is a constant C" that does not depend on / and m N such that
I S C" ((a)) log N + (mN)))NN"" "ply (log N).
The factor poly(log N) can be erased if we take a larger in the proof.
D APPROXIMATION OF FUNCTIONAL OPERATIONS BY NEURAL NETWORKS
This section reviews the accuracy the approximation and the increase complexity when we approximate functional operations by neural networks. The following results are shown in Oko et al. (2023) Section F) as well and in more original literature Nakada and Imaizumi (2020) Petersen and Voigtlaender (2018) Schmidt-Hieber (2019).
The operations used directly in this article are recip, mult, clip, and aw. The usage in Section (III) is explained examples.
D.1 CLIPPING FUNCTION
First, we consider the realization of the component-wisc clipping function.
Lemma 20. For any a, E IRd with a; < b, (i 1.2,...,ce d), there exists neural net-
W (d, d) ; S 7d, and B work clip(æ; b) E _M[L, ([S,W, with L 2. maxigisa maxf (asl. by} such that
clip(æ;; a, b); min(he max (xx, (vi,0;))
1,2,.... d)
holds. When = Cmin and b, Cman for all i, we also use the notation clip(æ;; Cmin Cmax) == clip(æ;; a, b).
D.2 RECIPROCAL FUNCTION
Second, the reciprocal function + 1/x is approximated by neural networks follows.
Lemma 21. For any 0 < s < L, there is recip(æ') MML, W. B) such that
x -
recip(a') $ +
(73)
z2
holds for any E [1 and x' E R with L = O(log (s (s-))), || O(log((=-1)), S O(log(e-11), and B -
D.3 MULTIPLICATION
Lemma 22. Let 2 2, C2 1,0 < Car < 1. For any E > 0, there exists a neural network
mult( F2, ... s:a) E M[.., W,S, B) with I (d - dlog C)), || 48d,
30

---

Published as a conference paper at ICLR 2025
O(dloge=' + dlog C)), B Cd. such that (i)
mult(z' x'd)
II x S dCdl derrs
Cerrs
holds for all [-C, C7d and x' Rd with ||e x and (iii) mult(z'; s'dd = 0 if at least one of a44 is
2'1100 S cerr, (ii) [mul((()) S Cdd for all R
We note that some of 2t (i + j) can be shared; for II Io, with Oa Zz (i (i= 1i=1,.... I) and Σ a; = d, there exists a neural network satisfying the same bounds as above; the network is denoted by mult(æ;;))
D.4 SWITCHING
Lemma 23. Let t < l2 < < 52, and f(x,t) t) be a scaler-valued function. Assume that
(x, t) - [ax t)l S s on [1] and (g2(x, f(x f)] [L2, 82]. Then, there exist neural networks sW1 (t: (t; t2, $1) and (2, $1) in MM W. S, B) with L = W (1,2,1.1) S = 8,
and max ($1 t2) [ such that
|sw1 (4; 12, 81 (x) sw2(l; s1)pp(2, -(x)
holds for any L [11182]
D.5 CONSTRUCTION OF NETWORK SECTION C.4 (III)
We detail the network size required for the approximation procedure presented in Section C.4 (III).
Clipping to Ç1 From (3) and assumption (A3), Ø can be upper bounded by No for sufficiently large N. Then, from Lemma 20 we
that in the clipping of Φs. the increase in model sizes is constant depending on d except B, which is multiplied by the upper bound N Ko+1
Approximating fi' by C2: Since we assume fi 2 (2 clipping ds from below by
N- N-(2++w)/d does not increase the difference; thus [[1 - fil S D5N-", Lemma 21 substituting x' -1. x. f1, and e = N-x for x > Xo (2s + w)/d with arbitrary Xo O, we have
recip(Si(z;))) Ji (ax 1)
N- + N (ix fa
Since 17 > is arbitrary, by setting 7 and x so that 17 3x, we have
recip((i(æx t)) fi t)
$ (Ds +IN
(74)
This is achieved by a neural network with Ollgg N). O(log* N), || O(log log N) and B = (N)2x).
So mult({2. de): Note that (cll < (2(2+ww//d |1 - Φbl O( and
K fi <
x from (4.. We have also taken 7) so that r 3x. In applying Lemma
22
we can set
C // because |C2| < (22++w)/d Also, cerr = max - 1/fi fil. (
fell
max (O(N x), ON }}} ONN-)NN))))) With d = 2 and E = N N we have
(3 f) (2 Φ6 t) ON Xo + (2+ N N-) = O(N Xo + 222--o0 =O
where we use the fact that x is taken to satisfy x > Xo + (2s + w)/d.
We then obtain
f2 f1
fg S [[ - Φ0 + 2ol2
Ca
fz5z f2 fl
S C|3 - Φb52ll + 1666 - feCel +
xx) + O(N (1 /d + NN NN
= x(N-xo),
(75)
31

---

Published as a conference paper at ICLR 2025
where in the second last line use filz
O(vlooglVV N) and thus | O(vlog N).
similar argument derives
fz
<5
O(N 20).
(76)
For the neural network architecture in this multiplication, L and S are added by the order of O(log + log = O(logN) The width W of constant order and thus negligible. The width W is O(N2(2stw))))
Clipping to obtain Ç4 and Ço: For these clipping procedures, the approximating networks have L, W and S of constant order, while the weight values are of O O(vlog N) and O(1), respectively, and thus they are negligible. The approximation errors are kept N
Multiplication to obtain S7 and Ç3: As in the previous procedures, clipping by (Vylog N) and (1)) does not increase the approximation enror, while the increase in the size of the network is negligible.
In similar manner to Oko al. (Lemma B.1 2023).
approximate of and m² by neural networks so that lo! oj = O(N ") and |m m/| = O(N "). The network sizes are = Ollog N), ||W]| = Olog" N), S = Oloog N). and B = Ollog N). With arguments similar to those of the previous procedures,
show
f2 fi
f2
<7 (o:)
Cs Csvloy log N
-(N
f1
and
La
fz
Ss
|mm|
1
s Cs
X(N
f1
f1
In total, find neural network to approximate (x)x)) with the approximation error of O so that the network has the size of most polynomial orders for B and |W]lso. while (poly(log N)) for S and L. As result, the contributions the log cover number are only Oppoly (log N)).
E IDEA OF TIME DIVISION
The basic idea of the time division used derive the almost optimal minimax convergence rate is depicted in Figure E.1
32

---

Published as a conference paper at ICLR 2025
(a(P..))) v (
@ (00 v,
Division of time interval
Pa(x)
Pr.
Px(x)
(x)
Pr. (x)
Pa. (x)
Px())
number spline bases continues.
Adapted number of B-spline bases for spproximetior.
Lipschitz coefficient
Lipschitz coefficient is constant:
2 chole 20 logtt/t.) (t/t.)
211 sAudu
= const.
can be large around t. f.
Coefficient vf does not contribute for the large interval [].
Coefficient be small around t, which suppresses the complexity term.
Figure E.I: The idea of time division for deriving convergence rate
33


Reviews:
[{"review_id": "w9avvZGAGP", "reviewer": "Reviewer_nHx4", "summary": "The paper provides estimates for the 2-Wasserstein distance for the sample-based distribution obtained in the Flow-Matching framework relative to the exact distribution. These estimates depend on the number of samples used in training, the smoothness of the true distribution as an element of the Besov space, the asymptote of the growth of the conditional map at the initial time instant.\nThe paper considers the early stopping mode of the ODE, when the solution stops at time $T_0<1$, and the estimates of $T_0$ are also given.", "questions": "- How do estimates change if you take a distribution other than the Gaussian distribution as the initial distribution $P_{[0]}$?\n\n- Can the obtained estimates be easily extended to the case of estimation error in the total variation (TV) distance?\n\n- Would your estimates change if you use different heuristics for Flow Matching, such as OT-minibatch?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The paper is the first to present estimates for the Flow Matching framework, which shows that almost optimal minimax converges rates are achieved under several assumptions.\n- The paper is well written", "weaknesses": "This paper contains many points in common with the paper cited therein [1].  In particular, using Besov space for target density, B-splines for its approximation, etc. Many estimates are based on thouse from [1], see, for example, Appendix A.4--A.5 of the presented paper, where the citation on [1] are explicit. In paper [1] diffusion models are considered, but as shown in paper [2], Flow Matching approach includes, under certain conditions, Diffusion models approach. Thus, generalization or obtaining similar results for Flow Matching is rather straightforward. Namely, in essence, the difference is to use Alekseev-Gröbner Theorem (Lemma 16 about error of a perturbed solution of ODE) instead of Girsanov’s Theorem (Proposition D.1 of [1] for error of a perturbed solution of SDE).\nOne of the main differences is the presence in the estimates of the degree of growth of the parameter $\\sigma_t$ at 1, but the authors come to the well-known (empirical) conclusion that the optimal asymptotics is $\\sqrt t$. Does this provide the first theoretical justification for this empirically observed optimal scaling?  How can one intuitively realize that the degree of $\\sigma_t$ growth near the time point $t=1$ is important if the ODE solution is considered on the interval $[0, T_0]$, where $T_0<1$?\n\n\n[1] Kazusato Oko, Shunta Akiyama, and Taiji Suzuki. Diffusion models are minimax optimal distribution estimators. volume 202, pages 26517–26582. PMLR, 4 2023\n\n[2] Aaron Lipman, Ricky T Q Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EyrQ0Jk3qk", "reviewer": "Reviewer_yJWQ", "summary": "The paper applies the same framework as in Oko et al. (a paper on convergence rates of diffusion models as timesteps and/or sample size goes to infinity), to Flow Matching. Due to the application to a different model, some of the proofs are different but the results are of the same strength.", "questions": "1) In Theorem 1, what is a good bound for R_0?\n2) Still in theorem 1, why do you chose $\\sigma_{[\\tau]}$ in that form? In what applications does it appear that way?\n3) Can you test the sharpness of the bounds of theorem 9 for some FM famous use cases? \n\nat line 310 \"in general we generally consider\" may be rephrased", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "It's interesting to know that FM have the same standard guarantees as DMs.", "weaknesses": "Mathematically, the paper does not shine as to novelty, it mostly chains known estimates, and applies them to FM. \nLike in similar papers for other models, some of the setups look like toy models, this may be because the mathematical theory is unavailable in general.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4sToYQwNBC", "reviewer": "Reviewer_xqeo", "summary": "This paper provides near-minimax convergence guarantees for the flow matching (FM) algorithm for $p$-Wasserstein distances. Distinct from diffusion models, FM use ordinary differential equations (ODEs) at inference time instead of stochastic differential equations (SDEs). Their estimator is based on time-partitioned estimators, similar to the analysis of Oko et al (2023). They adopt the estimator for more general parameters (specifically the mean and covariance parameters) to provide an estimator for flow matching.", "questions": "My comments are minor (these are mostly typos I found, but this is far from exhaustive)\n\n1. Line 188 \"for probability *density* estimation\"?\n2. L193 (this might happen in many places): I believe grammatically it makes sense to say \"i.id. sample*s*\" instead of a single sample\n3.  L222: *reverse* not revserve\n4. L254: \"respectivly\" is misspelled\n5. This is a question: is there a clean way to track the dependence on the diameter of the set of the support? The assumptions assume the support of the density is in the unit cube. What is the dependence if the radius was arbitrary? This might fall outside the scope of the paper, but I'm curious if the authors have the answer\n6. L516: \"diffrence\" is misspelled", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "This paper provides (to my understanding) the first estimation rates for flow matching in the context of classical statistical estimation rates. This paper is similar in spirit to the FM paper of Lipman et al (2023), where they use different combinations of mean-variance parameters to define their path. While this work leverages many ideas from Oko et al. (2023), what is especially interesting is this idea that \"optimal parameter choices\" lead to minimax convergence rates, whereas other choices do not enjoy the same statistical rates.", "weaknesses": "The paper could be more clearly written, and the main text is very technical. This paper would benefit substantially from a small figure explaining the construction of the estimator at a high level, and also explaining the reason why the full minimax estimation is not possible. These are overall minor points, but I do believe the paper would benefit greatly from these modifications overall.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "crreJbwQuo", "reviewer": "Reviewer_SoUb", "summary": "This paper proves an almost minimax optimality result for a class of flow models. Previously, Oko had shown that diffusion models are minimax optimal under the 1-Wasserstein distance. This paper builds on Oko to show that a class of FMs with terminal Gaussian distribution and paths of the form x_t = \\sigma_t x_0 + m_t x_1 (which includes diffusion as a special case) are almost minimax optimal, with a parameter kappa determining the non-optimality. They show this under the p-Wasserstein distance for 1 <= p <= 2.", "questions": "L223: Kappa = 1/2 corresponds to diffusion-FM, correct? So the result actually says that that only diffusion-FM is optimal within the family of FMs you consider? If so, L230 'FM is as strong as diffusion models' seems somewhat misleading -- it seems more like diffusion is actually stronger than FM, except when FM is equivalent to diffusion. Please correct me if I'm wrong; otherwise might want to state this differently.\n\nL225: Can you elaborate on the ways in which your proof technique differs significantly from Oko's? (Since Oko's result is a special case of yours for diffusion-FM and 1-Wasserstein.)\n\nL133: Can anything be said in the more general non-Gaussian case of FMs?\n\nL177: Notation not super clear here. What is P_[1] vs p_[1]?\n\nTheorem 1: It seems like you are further restricting sigma to have the form (1-\\tau)^\\kappa?\n\nL222: Typo 'revserse'\n\nL224: Diffusion can be expressed as an ODE so I am not sure what you mean here?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Provides new theoretical results for a family of flow models, showing almost minimax optimality (with 'almost' depending on a specific parameter). The paper is rigorous, clearly written, and clearly places the results in the context of prior work.", "weaknesses": "Please see Questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)