Towards Hierarchical Rectified Flow
Authors: Yichi Zhang
Yici Yan
Alex Schwing
Zhizhen Zhao
Venue: ICLR 2025 Poster (2025)
Keywords: Generative Model
Flow Matching
Rectified Flow
Primary area: generative models
TL;DR: We formulate a hierarchical rectified flow which couples multiple ordinary differential equations (ODEs) and generates data distributions from a simple source distribution.

Abstract: We formulate a hierarchical rectified flow to model data distributions. It hierarchically couples multiple ordinary differential equations (ODEs) and defines a time-differentiable stochastic process that generates a data distribution from a known source distribution. Each ODE resembles the ODE that is solved in a classic rectified flow, but differs in its domain, i.e., location, velocity, acceleration, etc. Unlike the classic rectified flow formulation, which formulates a single ODE in the location domain and only captures the expected velocity field (sufficient to capture a multi-modal data distribution), the hierarchical rectified flow formulation models the multi-modal random velocity field, acceleration field, etc., in their entirety. This more faithful modeling of the random velocity field enables integration paths to intersect when the underlying ODE is solved during data generation. Intersecting paths in turn lead to integration trajectories that are more straight than those obtained in the classic rectified flow formulation, where integration paths cannot intersect. This leads to modeling of data distributions with fewer neural function evaluations. We empirically verify this on synthetic 1D and 2D data as well as MNIST, CIFAR-10, and ImageNet-32 data. Our code is available at: https://riccizz.github.io/HRF/.

Published as a conference paper at ICLR 2025
TOWARDS HIERARCHICAL RECTIFIED FLOW
Yichi Zhang', Yici Yan', Alex Schwing', Zhizhen Zhao'
'University of Illinois Urbana- -hhampahha
ABSTRACT
We formulate a hierarchical rectified flow to model data distributions. It hierarchi- cally couples multiple ordinary differential equations (ODEs) and defines time- differentiable stochastic process that generates data distribution from a known source distribution. Each ODE resembles the ODE that is solved classic recti- fied flow, but differs in its domain, i.e., location, velocity, acceleration, etc. Unlike the classic rectified flow formulation, which formulates a single ODE the loca- tion domain and only captures the expected velocity field (sufficient to capture a multi- modal data distribution), the hierarchical rectified flow formulation models the multi-modi random velocity field, acceleration field, etc., in their entirety. This more faithful modeling of the random velocity field enables integration paths intersect when the underlying ODE is solved during data generation. Inter- secting paths in turn lead to integration trajectories that are more straight than those obtained in the classic rectified flow formulation, where integration paths cannot intersect. This leads to modeling of data distributions with fewer neural function evaluations. We empirically verify this on synthetic ID and 2D data as well as MNIST, CIFAR- 10, and ImageNet-32 data. Our code is available at: bttps://ricciz... githb 1o/HRF/
INTRODUCTION
Diffusion models (Ho et al., 2020: Song et al. 2021a|b and particularly also flow matching (Liu
al. 2023 Lipman et al., 2023 Albergo Vanden-Eijnden, [0233 Albergo et al.] 2023) have
gained significant attention recently. This is partly due to impressive results that have been reported across domains from computer vision (Ho et al.) (020) and medical imaging (Song et al.) 2022) to robotics (Kapelyukh et al. 2023) and computational biology (Guo et al. 2024). Beyond impressive results, flow matching was also reported to faithfully model multimodal data distributions. In ad- dition, sampling is reasonably straightforward: it requires to solve ordinary differential equation (ODE) via forward integration from time zero time one. The source distribution points are sampled from simple and known source distribution, e.g., a standard Gaussian.
set of source distribution points along an estimated velocity field
The velocity field is obtained by matching velocities from constructed "ground-truth" integration path with a parametric deep net using a mean squared error (MSE) objective. See Fig. TTa for the "ground-truth" integration paths of classic rectified flow. Studying the "ground- truth" velocity dis- tribution at a distinct location and time for rectified flow reveals a multimodal distribution. We derive an analytic expression for the multimodal velocity distribution in case of mixture-off Gaussian -f-ausssaaa data distribution in Section 3.1 It is known that the MSE objective used in classic rectified flow does not permit to capture this multimodal distribution. Instead, classic rectified flow leads to a model that aims to capture the mean of the velocity distribution. This is illustrated in Fig. bb).
We do want to emphasize that capturing the mean of the velocity distribution is sufficient for charac- terizing multimodal data distribution (Liu al.] (002).. However, only capturing the mean velocity also leads to unnecessarily curved forward integration paths. This due to the fact that integration paths cannot intersect when using MSE objective, can be observed in Fig. bb).
In this paper, we hence wonder whether is possible to capture the velocity distribution in its en- tirety. This enables integration paths to intersect during data generation, illustrated in Fig. [[c). Intuitively, and as detailed in Section 3.2 we can capture the velocity distribution by formulating a rectified flow objective in the velocity space rather than the location space. Hence, instead of training

---

Published as a conference paper at ICLR 2025
1.0
X
1.0
10-
9.8
0.8
0.8
0.6
0.6
0.6-
0.4
0.4
02
0.2
0.2
8.0
0.0
0.0
(a) Linear Interpolation
(b) Rectified Flow
(c) Ours
Figure/ 1: Particles flow from starting points (grey) to endpoints (blue) as time increases from 0 to
1. Ideally, the trajectories (green) are straight lines connecting two ends as shown in (a). Rectified Flow captures the expected velocity field while our Hierarchical Rectified Flow can model the true velocity field thus generating intersecting and more straight paths.
deep net estimate the velocity for integration in location space, as done in classic rectified flow, we train a deep net that estimates the acceleration for integration in velocity space. Sampling can then be done by forward integrating two hierarchically coupled processes: first, forward integrate in velocity space to obtain sample from the velocity distribution; then use the velocity sample to per- form a step in location space. While this nested integration of two processes seems computationally more demanding at first, it turns out that fewer integration steps are needed, particularly in the latter process. This is due to the fact that the integration path is indeed less curved, as shown in Fig. T().. We also show in Section 3.3 that capturing the velocity distribution in its entirety permits to capture a multimodal data distribution. The data generation process is governed by a random differential equation (RDE) (Han) 2018) with learned random velocity field.
Going forward, instead of using 'just' two hierarchically coupled processes we can extend the for- mulation to an arbitrary depth, which is detailed in Section 3.4 Using depth of one defaults to classic rectified flow (deep net captures the expected velocity field), while depth of two leads to a deep net that captures the acceleration, etc. We refer to this construction of hierarchically coupled processes as a 'hierarchical rectified flow.'
Empirically, we find that the studied hierarchical rectified flow leads to samples that better fit the data distribution. Specifically, we find that this hierarchical rectified flow leads to slightly better results than the vanilla rectified flow.
PRELIMINARIES
Given dataset D = {<11)} consisting of samples 21 Pl. e.g., images, drawn from unknown target data distribution pi. the goal of generative modeling is to learn a model that faithfully captures the unknown target data distribution #1 and permits to sample from the learned distribution.
Since we focus primarily on rectified flow, we provide its formulation the following. At inference time, rectified flow starts from samples 2o ~ Po drawn from a known source distribution Po. e.g., standard Gaussian. The source distribution samples are pushed forward from time t = 0 to target time & 1 via integration along a trajectory specified via learned velocity field v(2,, t). This learned velocity field depends on the current time É and the sample location 2; at time t. Formally, we obtain samples by numerically solving the ordinary differential equation (ODE)
dæp = v(2 dt, with 20 ~ Po. t [0, 1].
(1)
Notably, this sampling procedure is able to capture multimodal dataset distributions, as one expects from a generative model.
To learn the velocity field, at training time, rectified flow constructs random pairs (æo, I1), consisting of source distribution sample 2g ~ Pu and a target distribution sample 21 ~ DD. The latter drawn from given dataset D consisting of samples which are assumed to be drawn from the unknown target distribution Pt. For a uniformly drawn time U[0, 1], the time-dependent location x, is computed from the pair (ro, x1) using linear interpolation of (zn, 1,,, i.e.,
Ft (1 1)35 + tx1, where 20 Po: 31 D.
2

---

Published as a conference paper at ICLR 2025
At this location 2, and time t, the "ground-truth" velocity (g(( t) 0r./0t = 2y is readily available. is then matched during training with a velocity model u t) via a standard &g loss, ie., during training address
inf E -. [ll=] zo - u [()1,))1
where the optimization is over the set of all measurable velocity fields. In practice, the func- tional velocity model u(,,,) f) is often parameterized via a deep net with trainable parameters 8, i.e, vzz,, t) = (22, t), and the infimum resorts to minimization over parameters 6.
Considering the training procedure more carefully, is easy see that different random pairs (5o, 21) can lead to different "ground-truth" velocity directions at the same time / and at the same location 24- The aforementioned & loss hence asks the functional velocity model v(r,, L) to regress to different "ground-truth velocity directions. This leads to averaging, i.e., the optimal functional velocity model v" (x,, t) = (( ()- =x,] [0(Z;, t)].
According Theorem 3.3 by Liu al. (2023), if we v* for the ODE in Eq. ① then the stochastic process associated with Eq. (1) has the same marginal distributions for E [0, as the stochastic process associated with the linear interpolation characterized in Eq. (2).
1]
Nonetheless, to avoid the averaging, in this paper we wonder whether it is possible to capture the multimodal velocity distribution each time t and at each location Zt, and whether there are any potential benefits to doing so.
TOWARDS HIERARCHICAL RECTIFIED FLOW
In the following Section 3.1 we first discuss the multimodality of the velocity distribution and pro- vide case study with Gaussian mixtures. The case study is designed to provide insights regarding the velocity distribution. We then discuss in Section 3.2 simple way capture the multimodal velocity distribution and how to use it to sample from the data distribution. Then, we show in Section 3.3 that the proposed procedure indeed faithfully captures the data marginals. Finally, we discuss in Section 3.4 an extension towards hierarchical rectified flow formulation.
3.1
VELOCITY DISTRIBUTION AND CASE STUDY WITH GAUSSIAN MIXTURES
The linear interpolation in Eq. defines difeeiifferentiable stochastic process with the random velocity field v( = 21 To. where Po and x Pt- Note, the source and target distributions are independent. The following theorem characterizes the distribution of the velocity specific space time location (2),
Theorem The velocity distribution Ti a1(1; (u,)) the space time location (I4,t) induced by the linear interpolation in Eq. (2) is
T (v; xe, 1) = pVIx, (ulox)) po(æ; - tu)p (F. + t)v)
(4)
pr(
for (z(z)) o with 8* denotes convolution)
(Po
for = 0,
Pe(xx))
PT-1)PP Pa
[-r
2z *P (2)
for E (0. 1),
(5)
(1(1))
for = 1.
The distribution Ii (1(1; x4, is undefined px(
The proof of Theorem | deferred to Appendix A Note that since Pr is typically multimodal, the resulting Ti (v; az, t) is also multimodal. At t 0, we have mi (I, p1 + v), which corresponds to the data distribution shifted by -2,- At t = 1, we have (1 t) = Po(x; v), which corresponds to the flipped source distribution shifted by 24.
To illustrate the multimodality of the velocity distribution, we consider a simple 1-dimensional example. The souree distribution is standard Gaussian (zero mean, unit variance). The target distribution a Gaussian mixture. The following corollary provides the "ground-trutt velocity distribution at any location Tt.
3

---

Published as a conference paper at ICLR 2025
144 hos
0.2
pemplio
645
0.2
6:25
0.2
0.2
605
-55 60
65
2.6
25
6.2
-2
6.
-3
600
-2
-1
vobdy
Veldy
(2r, ( (b) (I 0.4) (c) (2,, (0.5, 0.6) (d) (2 = (1.0,
Figure 2: We verify the derived velocity distribution by comparing its probability density function (blue) to the empirical sample histogram (orange) at different times and locations 24.
Corollary Assume Pu =N(x;;,,1) and P1 =E
K WaN (x:; Mko ( then
(1- (Hz xz) to
of
(1( Σ
(6)
km
where o2 (1 161-t) 2)² + oo and wk,t Des " Mer
To empirically check the fit of Corollary | in Fig. 2 we compare the derived velocity distribution with empirical estimates at different locations (ra, l). We observe great fit and very clearly multimodal distributions.
We defer the proof of Corollary [ to Appendix B
very much worthwhile to study these distributions bit more. In particular, we observe that the velocity distribution at time source distribution. This can be seen from Fig. dj.). Further, at time t O, we observe the velocity distribution be identical a shifted data distribution. This can be seen from Fig. 2 (a).
1 collapses
single Gaussian, more specifically shifted
This is valuable to know as it suggests that the velocity distribution is at least as complex as the data distribution. Indeed, 0, the velocity distribution is identical shifted data distribution.
3.2 MODELING THE VELOCITY DISTRIBUTION
The previous section showed that the velocity distributions can be multimodal. Knowing that the optimal velocity model o (z,, t) of classic rectified flow averages "ground-trutte velocities. can't expect classic rectified flow to capture this distribution. We hence wonder: 1) is it possible to capture the multimodal velocity distribution each time and each location 24; are there any benefits to capturing the multimodal velocity distribution as opposed to 'just' capturing its mean as done by classic rectified flow.
Intuitively, an accurate characterization of the velocity distribution might be beneficial because we obtain straighter integration paths, which in may lead to easier integration with fewer ral function evaluations (NFE). In addition, capturing the velocity distribution provides additional modeling flexibility (an additional time axis), which might yield improved results. Notably, mod- cling of the velocity distribution does not lead to modeling of simpler distribution. As mentioned the velocity distribution is identical to shifted data distribution.
Section 3.1 at time
To accurately model the "ground- "grround-truth" velocity distribution, we can use rectified flow for velocities rather than locations, which are used in the classic rectified flow formulation. This is equivalent to learning the acceleration. To see this, first, consider classic rectified flow again: we construct a time- dependent location z; from pairs ( #1 compute the "ground-truth" velocii Ugg (ar, 1) = 0x./0t and train velocity model Up up(2, (2r, to match this "ground-truth velocity tx),
To learn the acceleration, we introduce a source velocity sample to ~ To drawn from a known source velocity distribution Tn- We also construct a target velocity sample (æ,, t) n (v; xa, f) at time f and at location It, which follows the target velocity distribution Tı (w; 22, t) at time and at location øt- Note, the target velocity sample time t and location 2; (1 t)my tr is obtained via Un (z,, t) = 21 Zp, when considering a rectified flow. The samples Un (æ,, t) follow the "ground-truth" velocity distribution mi It, (2,4) 1) time and location Iz-
4

---

Published as a conference paper at ICLR 2025
Algorithm 1: Hierarchical Rectified Flow Training
The source distributions po and To and the dataset 'D while stopping conditions not satisfied do
Sample xo ~ Po, I1 ~ DD and 0o ~ To
//better to sample mini-batch Sample ~ [0, 1] and U10,1] //different t and t for each mini-bateh
sample
Compute loss following Eq.
8
Perform gradient update on
end
Using both the source velocity sample vo and the target velocity sample en (ar, f), and following clas- sie rectified flow, we introduce a new time-axis E [0, 1] and construct er (z(, t) (1 -)uu + TUI(1z at time and at location 24. Using it, we obtain the "ground-
depeedeenddnt velocity
truth" acceleration from the time-dependent velocity er (2r4, 1) via dgt (Tt, L, T) = 00./00 in (xr, uo 31 -0
vig-
Note, for specific (I,, we can get the following ODE induced from the linear interpolation of the target velocity distribution to convert Un ~ To to u1 ~ T (1;
du (1t,
al L, ur T)TT, with ug T0-
Here, a(Ia, f, Mr: = (vj=,,t) () [V the expected acceleration vector field.
Vol V= = u-] Po. [1 Vo-VVlV,- t/,, X; = is
Our approach aims to learn the acceleration vector field though flow matching for all (æt, t), i.e.,
matching the "ground-truth" acceleration by addressing
inf T TE Po D.t-U 1,00UU ro0t [l](21 vo vn) a(xi, f, Wr: ( +)
(8)
In practice, we use a parametric model ag(x,, t, Ur. 7) to match the target "ground-truth" accel- eration by minimizing the objective w.r.t. the trainable parameters 6, Training of the parametric acceleration model is straightforward. It is summarized in Algorithm
It remains to answer how we use the trained acceleration model ao(x,, 1, Mr, r) during sampling. We have the following coupled ODEs induced from the coupled linear interpolations:
du (Ze, t) = a(2,, t, uy r)dT, with uo(2, t) ~ To:
TE [
dzz un (zz, t)dt,
with 2g Po: E
[].
Those coupled ODEs convert E Po to 21 E P1. After training, the ODEs in Eq. (9) are simulated using the vanilla Euler method and ag as detailed in Algorithm - We first draw two random samples: Vo To from the source velocity distribution and zo Po from the source location distribution. We then integrate the velocity forward to time t modeled velocity distribution V (xo, 0). Subsequently, use this sample to perform one integration step on the location. We continue this procedure until we arrive at a sample 2T.
obtain a sample from the
Remark. The generation of data is governed by random differential equation (RDE) with the random velocity field, where Eq. (9) can be viewed as
dæ, = g t, uz(ze, t))dt, where uo(=,, t) No: 2g Po:
(10)
and g is a deterministic function. The randomness comes from the initial conditions for data and velocity. This is different from the sampling process governed by a stochastic differential equation (SDE) used in diffusion models (Song al. (2021b), where the randomness comes from the Wiener process and the initial condition for data.
Note that rather due to two hierarchically coupled linear processes. We hence refer to this construction as a hierarchical rectified flow.
of the term acceleration is not due second-order derivatives of the location, but
remains to show that the obtained samples indeed follow the target data distribution. We will dive into this topic next.
5

---

Published as a conference paper at ICLR 2025
Algorithm 2: Hierarchical Rectified Flow Sampling
Input The source distributions Po and To, the number of E-discretization steps J, the number of 7-discretizatton steps I,, and the trained network parameters 0. Sample 2g ~ Po and up ~ no:
Compute At J-T and AT T.
for j 1 ,,....J J do
for 1, .,... do
Compute = u1-1 + ae (2 4-1, T-1)) AT
end
Compute 2j = žj-1 ur. Al
end
3.3 DISCUSSIONS ON THE GENERATED DATA DISTRIBUTION
We discuss below the property of the hierarchical rectified flow defined in Eq. ( According to rectified flow theory, we generate samples from the velocity distribution using the expected acceleration field. The following theorem states that the generation process defined in Eq. (9) which uses the velocity distribution, leads correct marginals for all times / [0.
Theorem
The time- limedifferentiable stochastic process Z = [7+ 1 1]} generated by Eq. has the same marginal distribution as the time- iimediiffernntibble stochastic process = {X; [0, }]} generated by the linear interpolation in Eq. (2)
(9)
Intuitively, the marginal preserving property is because at each time [0.1]. 1]. we can express 24 as the linear interpolation of an Ty Pu and an P1 according to Eq. ②
We defer the proof of Theorem 2 to Appendix C
key benefit of our approach is that the process Z can piece-wise straight. Starting with samples from Pt. for 1. E [0, 1]. we propagate each sample by v(2,, () where v(=,, () m (w; 21, t). Since v(2,,t t) = Φ1 zo. where tz - - t) = Zt, the straight path following v(2,, t) will lead to sample from the data distribution. In other words, At can be chosen arbitrarily in the interval 10, t). In practice, the learned velocity distribution is not perfect. Therefore, instead of one-step generation from the initial distribution, shown in Section we typically only use 2-5 steps in the numerical integration for data generation. Computationally. straight paths are very attractive as trajectories with nearly straight paths incur small time-discretization error in numerical simulation.
choose to propagate the samples for couple of steps. As
3.4 EXTENDING TOWARDS HIERARCHICAL RECTIFIED FLOW
Consider the training objective for acceleration matching discussed Eq. (8), and further consider the coupled ODE solved when sampling from the constructed process specified in Eq. (9) It is straightforward to extend both to an arbitrary depth. I.e., instead of modeling the velocity distribu- tion by matching accelerations, we can model the acceleration distribution by matching jerk or go even deeper towards snap, crackle, pop, and beyond.
Formally, the training objective of a hierarchical rectified flow of depth D is given by
inf E Pe [1((x) 15x0)) (z,, t]12
(11)
Here, 1p is the D-dimensional all-ones vector and t = [ta) ( (D)] a D-dimensional vector of time variables drawn from a D-dimensional unit cube 00, 1j Moreover, we use the D-dimensional vector of source distribution samples 2g [mm]] ...., (P))T, drawn from a D-dimensional source distribution Po. e.g., D-dimensional standard Gaussian. We further (1)
the D-dimensional location vector 24 [=] ,...., (.) with its d-th entry given as
2(()) (1 - )(()) +(d) d-1 =0)). In addition, we refer to f as the functional field of directions. Note that Eq. (TT) is identical Eq. 3 if D 1 or Eq. if D 2.

---

Published as a conference paper at ICLR 2025
12
1.75
3.52
20 clge Nivodds
2E the RCvE
deg
1.25
0.75
0.80
02
505
00
05
1.5
25
02
-2
0:3
6
60
Whol
Volocyy
waddy
(a) (2t, (-1.0,0.0) (b) (2e, 1) (0.0, 0.4)
(c) (2t, = (0.5, 0.6) (d) (z,,t)= (1.0,
Figure 3: Numerical estimation of ア (Ze, in HRF2 with different number of integration steps. The blue line shows the ground-truth where pu is standard Gaussian and P1 is a mixture of two Gaussians. The 1 Wasserstein distances (WD) for the estimates w.r.t. Ti are shown in the legend.
Before discussing inference we want to highlight the importance of the first term in Eq. 11 Sub- tracting a large number of Gaussians from a data sample 21 leads to a smoothed distribution. This another potential benefit of a hierarchical rectified flow formulation.
Given a trained functional field of directions f we sample from the defined process via numerical simulation according to the following coupled ODEs:
dzf
(D)
(2
(2t
(2:D-1) , = [(2, t)
with (D) ~ Po po) #0
dit
d2(0-1) (2) (1:D-2) (1:D-2)) = zi (D)
[ (z) (1:D-11) (1u.D-1) dill-1),
with 20
(D-1) 1) ~ Po
(D-1)
dzt = 21
(2)
(=(0) (0,,00) (1) di
(1)
with 20 ~ Po
(1) (1)
(12)
Note that Eq.
12) is identical to Eq. | D 1r Eq. 9 D
Again, note that order derivatives of the location, but rather due to hierarchically coupled linear processes.
of the terms acceleration, jerk, etc. is not due to second, third, and higher-
4 EXPERIMENTS
The studied hierarchical rectified flow (HRF) formulation couples multiple ODEs accurately model the multimodal velocity distribution. To assess efficacy of this formulation, we first vali- date the approach in low-dimensional settings, where the analytical form of the velocity distribution is straightforward to compute. This allows to verify that the model can indeed capture the veloe- ity distribution accurately. We then investigate whether fitting the velocity distribution enhances the model's ability to fit the data distribution in generative tasks. We perform experiments
ID data (Section 1. 2D data (Section 4.2). and high-dimensional image data (Section 4.3) with depth two HRF (HRF2) models: the models not only fit the velocity distribution but also enhance the quality of the generative process. We also include results for depth three HRF (HRF3) models on low dimen- sional data to show the potential for exploring deeper hierarchical structures. Importantly, for all experiments we report total number of function evaluations (NFEs), i.c., the product of the number of integration steps at all HRF levels.
4.1 SYNTHETIC ID DATA
For the ID experiments, we first consider a standard Gaussian source distribution and target dis- tribution represented by a mixture of two Gaussians. Using Eq. (6), we can compute the analytical form of the velocity distribution. As shown in Fig. our model captures the analytic velocity distri- bution with high accuracy. As expected, an increasing number of velocity integration steps increases the accuracy of the estimated velocity. The only exception occurs when approaches 1. The model performance deteriorates, and excessive steps accumulate errors, leading to less accurate results.
Next, examine the data generation quality. We use mixture of five Gaussians as illustrated in Fig. aa) and compare HRF to a baseline rectified flow (RF). We use the 1-Wasserstein distance (WD) as metric assess the quality of the generated data. As shown in Fig. 4b), for the same
7

---

Published as a conference paper at ICLR 2025
64-
64-
05
15
(a) Data distribution
(b) Metrics
(c) RF trajectories
(d) HRF2 trajectories
Figure 4: Results on ID example, where po is a standard Gaussian and p1 a mixture of Gaussians. (a) Histograms of generated samples and pt. (b) The I-Wasserstein distance vs. NFE. (c) and (d)
The trajectories of particles flowing from source distribution (grey) to target distribution (blue).
50 39
15 163
2t 200
(a) Metrics
(b) RF trajectories
(c) HRF2 trajectories
(d) HRF3 trajectories
Figure 5: Results on 2D data. Top row: Po is standard Gaussian and P1 is mixture of 6 Gaussians. Bottom row: Po is mixture Gaussians and p1 is represented by the moons data. (a) Sliced 2- 2-Wasserstein distance with respect to NFE. (b) and (c) show the trajectories (green) of sample particles flowing from source distribution (grey) target distribution (blue).
neural function evaluations (NFEs), the HRF models outperform the baseline, producing data distri- butions with lower WD, indicating superior quality. In Fig. 4 legend, the term "v steps" refers to the number of velocity integration steps. In this ID experiment, HRF3 demonstrates better perfor- mance compared HRF2. More ID results are provided in Appendix H.I
Additionally, we observe a fundamental difference the generated trajectories. Since rectified flow estimates only the mean the velocity distribution, tends to move towards the center of the target distributions initially. In contrast, the HRF model determines the next direction at each space-time location based on the current velocity distribution. As shown in Fig. dd), the HRF2 trajectories nearly linear and can intersect, which permits to use fewer data sampling steps during generation.
For the deep net, we use simple embedding layers and linear layers first process the space and time information separately. Afterward, we concatenate these representations. This combined input then passed through a series of fully connected layers, allowing the model to capture complex interactions and extract high-level features essential for accurate velocity prediction. We use the same architecture for the baseline model but increase the dimension of the hidden layers to opti- mize its performance. In contrast, the HRF2 model contains only 74,497 parameters compared to 997,089 parameters for the baseline model. This demonstrates the potential efficiency of HRF in handling nigher-dimensionn data while maintaining more compact architecture. More details of the experiments are provided in Appendix
4.2 SYNTHETIC 2D DATA
For the 2D experiments, we consider two settings: 1) a standard Gaussian source distribution and a target distribution consisting of mixture of six Gaussians; and 2) mixture of eight Gaussians as the source distribution and the moons dataset as the target distribution. We employ the same network architecture used in the 1D experiments. Due to the 2D data, now have 76,674 parameters for
8

---

Published as a conference paper at ICLR 2025
2
5
3
0
5
6
5
5
4
6
3
5
55 8 81
7 3
7
9
7
1
7
8
0
O
8
5
O
|
/
N.0.05M
575
(44822)
1972 (46.214)
15.
12.
50
15
36c
386
28
1.00
(a) MNIST
(b) CIFAR-10
(c) ImageNet-32
Figure 6: Experimental results on (a) MNIST, (b) CIFAR-IO, and (c) ImageNet-32 datasets. Top
row: samples of generated images, bottom row: FID scores with respect to total NFEs.
the HRF2 model and 329,98 986 parameters for the baseline RF model. We measure the quality of data generation using the sliced 2- Wasseerstein distance (SWD). Fig. shows the results. It is evident that on these more complex datasets, the performance gap between an HRF model and the rectified flow baseline is more pronounced. The trajectories demonstrate similar patterns to those observed in the 1D experiments: HRF2 produces significantly straighter paths, while the rectified flow baseline often exhibits large directional changes. Additionally, the HRF models consistently achieve higher quality in data generation compared to the baseline. HRF3 outperforms HRF2 for generating the moon data from mixture of 8 Gaussians. However, HRF2 works better for the simpler mixture of Gaussian target. There is room to improve the training and scheduling of the integration steps among different layers for deeper HRF models.
4.3 IMAGE DATA
In addition to low-dimensional data, we also conduct experiments on nigh-dimensional image datasets including MNIST (LeCun et al. 1998), CIFAR-10 (Krizhevsky) 2009), and ImageNet- 32 (Deng et al. 2009). We employ the Fréchet Inception Distance (HID) as the metric for evaluating image generation quality. For the baseline, (202)) and successfully reproduced the state-of-the-art results across all datasets. The HRF model builds upon this U-Net structure. We follow the parameter settings and training procedures from Tong et al. (2024) and Lipman et al. (2023). Further details on the architecture and training setup are provided Appendix(F]
the same U-Net architecture as Lipman et al.)
As shown in Fig. 66 for the same total NFEs, the HRF2 model demonstrates better performance on MNIST and CIFAR-10, and on-par performance on ImageNet-32 when compared to the baseline.
5 RELATED WORK
GANs (Goodfellow al. 2014 Arjovsky et al., 2017). VAEs Kingma & Welling 2014), and normalizing flows (Tabak & Turner 2013) Rezende & Mohamed 2015 Dinh
Generative Modeling:
201]] Huang al. 2018 Durkan al. 2019 are classic methods for learning deep generative
models. GANs excel in generating high-quality images but face challenges like training instability
and mode collapse due to their min-max update mechanism. VAEs and normalizing flows rely on maximum likelihood estimation (MLE) for training, which necessitates architectural constraints or special approximations to ensure manageable likelihood computations. VAES often employ a con- ditional Gaussian distribution alongside variational approximations, while the discrete normalizing flows utilize specifically designed invertible architectures and require costly Jacobian matrix cal- culations. Extending the discrete normalizing flow to continuous unstructured yet estimatable using trace estimation methods (Hutchinson 1990, Chen et al., 2018 Grathwohl al. 2019). However, using maximum likelihood estimation (MLE) for this mapping
enabled the Jacobian to be

---

Published as a conference paper at ICLR 2025
requires costly backpropagation through numerical integration. Regularizing the path can mini- mize solver calls (Finlay et al., 2020 Onken et al., (021), but doesn't 't resolve the fundamental optimization challenges. Rozen et al. (2021); [Ben-Hamu et al. (2022) considered simulation-fis -ree
training by fitting a velocity field, but still present scalability issues (Rozen et al. 202T) and biased
optimization (Ben-Hamu al. 20022)
Recent research has utilized diffusion processes, particularly the Orstein- (OU) process, to link the target distribution P with a source distribution po- This involves a stochastic differential equation (SDE) that evolves over infinite time, framing generative model learning as fitting the reverse evolution of the SDE from Gaussian noise to P1 (Sohl-Dickstein et al. 2015 Ho et al. 2020 Song et al., (2021b). This method learns the velocity field by estimating the score function Te using the Fischer divergence instead of the maximum likelihood estimation (MLF) objective. Although diffusion models have demonstrated significant potential for modeling high- dimensional distributions (Rombach et al. 2022 Hoogeboom et al. 2022, Saharia et al. 2022). the
requirement for infinite time evolution, heuristic time step parameterization (Xiao et al. 2022). and
the unclear significance of noise and score (Bansal et al., 2024; Lu et al, 2022) pose challengss.
Notably, the score-based diffusion models typically require large number of time steps generate data samples. In addition, calculating the actual likelihoods necessitates using the ODE probability flow linked the SDE (Song et al. 2021b). These highlight the need for further exploration of effective ODF-driven methods for learning the data distribution.
Flow Matching: Concurrently, Liu al. (2023); Lipman al. (023)) : Albergo & Vanden-Eijnden (2023) presented an alternative to score-based diffusion models by learning the ODE velocity through a fime-differentiable stochastic process defined by interpolating between samples from the source and data distributions, i.e., v2 the (Tr, (1), with Zig Po and æι ~ P1, instead of the OU process. This offers greater simplicity and flexibility by enabling precise connections between any two densities over finite time intervals. [Liu et al. (2023) concentrated on linear interpolation with (1 (1 () + f. i.e., straight paths connecting points from the source and the target distributions. Lipman et al.] (202)) introduced the interpolation through the lens of conditional prob- ability paths leading to a Gaussian. Extensions of Lipman et al. (2023) were detailed by Tong et al. 2024), generalizing the method beyond Gaussian source distribution. Albergo & Vanden-Eijnder
2023
Albergo et al. 2023) introduced stochastic interpolants with more general forms.
Straightening Flows: Liu al. (2023) outlined iterative process called ReFlow for coupling the points from the source and target distributions to straighten the transport path and demonstrated that optimal transport map. Other related studies bypass the iterations by modifying how noise and data are sampled during training. For example,PPooldiia |Pooladian et al. (2023); Tong et al. (2024) calculated mini-batch optimal transport couplings between the Gaussian and data distributions to minimize transport costs and gradient variance. Note that these approaches are orthogonal our approach and can be adopted in our formulation (see Appendix H.
repeating this procedure leads
Concurrently, Guo & Schwing (2025) also study a method to model multi-modal velocity vector fields. In this paper, we discuss use of a hierarchy of ordinary differential equations. Differently, Guo & Schwing (2025) study how to use a lower-dimensional latent space to enable modeling of the velocity distribution via a variational approach. The hierarchy of ordinary differential equations permits to more accurately model the velocity distribution while use of the variational approach enables to capture semantics.
Modeling Velocity Distribution:
DISCUSSION & CONCLUSION
We study a hierarchical rectified flow formulation that hierarchically couples linear ODEs, each akin to classic rectified flow formulation. We find this formulation to accurately model multi- modal distributions for velocity, etc., which in turn enables integration paths to intersect during data generation. As consequence, integration paths are less curved leading to compelling results with fewer neural function evaluations. Currently, our sampling process is relatively simple, relying on the Euler method for multiple integrations. We have only performed basic grid search regarding possible integration schedules and we have not explored other solvers. We suspect, better strategies exist and we leave their exploration to future work.
Acknowledgements: Work supported in part by NSF grants 1934757, 2008387, 2045586, 2106825, MRI 1725729, NIFA award 2020-67021-32799 and the Alfred P. Sloan Foundation.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
M. Albergo and E. Vanden-Eijnden. Building normalizing flows with stochastic interpolants. In Proc. ICLR, 2023.
M. Albergo, N. Boffi, and H. Vanden-Eijnden. Stochastic interpolants: unifying framework for flows and diffusions. arXiv preprint UrXiv:2303.08797, 2023.
M. Arjovsky. S. Chintala, and Bottou. Wasserstein generative adversarial networks. In Proc. ICML, 2017.
A. Bansal, E. Borgnia, H.-M. Chu, Li, Kazemi, Huang, M. Goldblum, J. Geiping, and
T. Goldstein. Cold diffusion: Inverting arbitrary image transforms without noise. In Proc. NeurlPS, 2024.
H. Ben-Hamu, S. Cohen, J. Bose, B. Amos, M. Nickel, A. Grover, R. Chen, and Y. Lipman. Match- ing normalizing flows and probability paths on manifolds. In Proc. ICML, 2022.
Bromiley. Products and convolutions of gaussian probability density functions. Tina- Vision Memo, 2003.
R. Chen, Y. Rubanova, Bettencourt, and D. Duvenaud. Neural ordinary differential equations. In Proc. NearlPS, 2018.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-2442250 255. Ieee, 2009.
L. Dinh, J. Sohl-Dickstein, and S. Bengio. Density estimation using real nvp. In Proc. ICLR, 2017.
John R Dormand and Peter J Prince. A family of embedded runge-kutta formulae. Journal of computational and applied mathematics, 6(1):19-26, 1980.
C. Durkan, A. Bekasov, Murray, and G. Papamakarios. Neural spline flows. In Proc. NeurIPS, 2019.
C. Finlay, J.-H. Jacobsen, Nurbekyan, and A. Oberman. How to train your neural ode: the world of jacobian and kinetic regularization. In Proc. ICMI., 2020.
Goodfellow, Pouget-Abadie, M. Mirza, B. Xu, D. Farley S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Proc. NeurIPS, 2014.
W. Grathwohl, R. Chen, J. Bettencourt, Sutskever, and D. Duvenaud. FFJORD: Free-form con- tinuous dynamics for scalable reversible generative models. In Proc. ICLR, 2018.
W. Grathwohl, R. Chen, J. Bettencourt, and D. Duvenaud. Scalable reversible generative models with free-form continuous dynamics. In Proc. ICLR, 2019.
P. Guo and A. G. Schwing. wwtp://www..rgra//220000000./0/. 2025.
Variational Rectified Flow
Matching.
In
Z. Guo, J. Liu, Y. Wang, M. Chen, D. Wang, D. Xu, and J. Cheng. Diffusion models in bioinformat- ics and computational biology. Nature Reviews Bioengineering. 2024.
Xiaoying. Han. Random Ordinary Differential Equations and Their Numerical Solution. Springer, 2018.
J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. In Proc. NeurIPS, 2020.
E. Hoogeboom, V. G. Satorras, C. Vignac, and M. Welling. Equivariant diffusion for molecule generation in 3d. In Proc. ICML, 2022.
C.W.. Huang, D. Krueger, A. Lacoste, and A. Courville. Neural autoregressive flows. In Proc. ICMI, 2018
11

---

Published as a conference paper at ICLR 2025
M. Hutchinson. A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Communications Statistics-Simullttie Simulation and Computation, 1990.
I. Kapelyukh, V. Vosylius, and E. Johns. DALL-E-Bot: Introducing web-scale diffusion models to robotics. IEEE Robotics and Automation Letters, 2023.
D. Kingma and M. Welling. Auto-Encoding Variational Bayes. In Proc. ICLR, 2014.
A. Krizhevsky, Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
Y. Lipman, R. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling. In Proc. ICLR, 2023.
X. Liu, C. Gong, and Q. Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. In Proc. ICLR, 2023.
C. Lu, K. Zheng, F. Bao, J. Chen, C. Li, and J. Zhu. Maximum likelihood training for score-based diffusion ODEs by high order denoising score matching. In Proc. ICMI., 2022.
D. Onken, S. W. Fung, X. Li, and L. Ruthotto. OT-Flow: Fast and accurate continuous normalizing flows via optimal transport. Proc. AAAI, 2021.
A.-A Pooladian, H. Bam-Hamu, C. Domingo-Eninn Ennich, B. Amos, Y. Lipman, and R. Chen. Multisam- ple flow matching: Straightening flows with minibatch couplings. In Proc. ICML, 2023.
D. Rezende and S. Mohamed. Variational inference with normalizing flows. In Proc. ICML, 2015.
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proc. CVPR, 2022.
N. Rozen, A. Grover, M. Nickel, and Y. Lipman. Moser flow: Divergence-based generative modeling on manifolds. In Proc. NeurIPS, 2021.
C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, J. Ho, D. J. Fleet, and M. Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. In Proc. NeurIPS, 2022.
Skilling. The eigenvalues of mega-dimensional matrices. Maximum Entropy and Bayesian Meth- ods, 1989.
Sohl-Dickstein, Weiss, N. Maheswaranathan, and Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proc. ICML, 2015.
J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. In Proc. ICLR, 2021a.
Y. Song, J. Sohl-Dickstein, D. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. In Proc. ICLR, 2021b.
Y. Song, L. Shen, L. Xing, and S. Ermon. Solving inverse problems in medical imaging with score- based generative models. Proc. ICLR, 2022.
E. Tabak and C. Turner. A family of nonparametric density estimation algorithms. Communications on Pure and Applied Mathematics, 2013.
A. Tong, N. Malkin, G. Huguet, Y. Zhang, J. Rector- Rector-Brooks, K. Fatras, G. Wolf, and Y. Bengio. Im- proving and generalizing flow-based generative models with minibatch optimal transport. TMLR, 2024.
Z. Xiao, K. Kreis, and A. Vahdat. Tackling the generative learning trilemma with denoising diffusion GANs. In Proc. ICLR, 2022.
12

---

Published as a conference paper at ICLR 2025
APPENDIX: TOWARDS HIERARCHICAL RECTIFIED FLOW
The appendix is organized as follows. We first provide a proof Theorem T (the velocity distribu- tion given 2²) in Appendix [A] We then provide proof of Corollary [ (velocity distribution for the special case of mixture of Gaussians target distribution) in Appendix B Afterwards we provide the proof of Theorem 2 (correctness of the marginals) Appendix C
Then discuss density es- timation for HRF models in Appendix [ Next we provide more details regarding the hierarchical
rectified flow formulation in Appendix E Subsequently, we discuss experimental and implemen-
tation details in Appendix [F Finally, we provide additional ablation studies in Appendix G and
additional experimental results in Appendix(H
A PROOF OF THEOREM
Proof of Theorem 1 The velocity at location 2, and time t is 21 20
- The last
equality holds because (1 t) fxı It. Recall that for a random variable Y ax + with R and & 0, we have (v Pp 8ー transform of the random variable X1, we get
Since the random variable V is linear
Ti (w; xe, t) Pvix, (ulzz) (1 /)x,, ((1 t)e xg/21)) Therefore, need to evaluate Px, X, Using Bayes' formula,
(13)
Px,xx, (21 (Ta) = Px,|X, (11/ (x1)
(14)
Px, (1z)
assuming that px, (Ir) jf is undefined ix, (IT) # Now it remains to find Px,(X, and have
It tx
Px,|X, (Ze (x1) P(1- (x+)
7 PNo -
(15)
Plugging Eq.
14 and Eq. 155) into Eq. 113 and using 41 + (1 ()), we have
Ti (0) Tz, Pv x. (v( Px. (2+ tx)pxx x, (I'e + - t)u)
px, (xx)
po( tr)p1 (xt + (1 - t)w) Pc(
(16)
Since the random variable X, is linear combination of two independent random variables Xo and X1 as defined in Eq. (2). we have
p((x))= = P(1 1) (x1)
Pix, (xt) = (=)- z)d=
= /
2
Ft 1
1 PXo
t PX:
dz
1- - t
t
Ft
21
t(1 t) Po
* p1
for t E (0, 1).
(17)
1-t)
At f 0. Pt Po since z0- At = 1, Pt Pl. since I. 21. TI (2; t) is undefined if
(r(xr.) = 0. This completes the proof.
B PROOF OF COROLLARY
Bromiley (2003) summarizes few useful properties for the product and convolution of Gaussian distributions. We state the relevant results here for our proof of Corollary
Lemma For the linear transform of a Gaussian random variable, we have
r: u n- b o²
N(ax + by b,, 02) = -N
a
a
13

---

Published as a conference paper at ICLR 2025
Lemma 2 For the convolution of two Gaussian distributions, have N11;111 11, of) * N(x; P2, 02 = -a + p2, of + o2).
Lemma 3 For the product of two Gaussian distributions, we have
1
(M1 -2) 22)2 of + o3
M102 + p
o903
of) ( +
exp
N x;
of + 0
of + 03
The proof' of the Lemmas are detailed by Bromiley (2003)
Proof of Corollary 1 We first compute the density of X, using Theorem with the specific po and P1:
Ft
It t.
(x(xz) 1(1 1) PD
*
1 -
K
1 /V
(i
Tt
0,1) 1) *
Σ
It WiN
(#+
MRx oi:)
(18)
t(1 t)
By applying Lemma 1 and Lemma to Eq. (8) we get
pe(w,) = N (172 1)3)
Σ WaN (are Cups .o%))
K
wi (N (1-:0.(1-tt)) (zeitue, foot) Σ
WaN (Erituee, xk.)
(19)
Using Theorem I and Eq. 19 we have
N tu; 0, 1)
Ex WWN (z, + (1 - t)e; Mek o/
Pvix, (0/x1))
E Wii kritpw,,0 W..
N (u: #1 1112 EK WwN u; mi-m (1-132)
EK= WiiN (p
EK WiN (v: Se 1 )
0; 1-t (1-0)²
t(1 t) Σ w= w
Ek=1 Wi 2 2) ) exp
K
1(1-1)
(x0-112)2 N (1; (1-1)) --1-18+4000
of 06.
o
1(1 - 1) E)
ww/ Resituw,ft .t.)
===1 WaN (Ziitiue, 02.)) N (1: (1-6)(mx (1-
2
02.1
Σk= wwwN 144 1in +2
(1 (Hu xxx) + tof
o
W VV
(20)
o
1=1
The equality holds by applying Lemma [ The equality b is derived by applying Lemma 3 the product of two Gaussian distributions. Simplifying the expressions, we get equality c and the final expression of pv (w(x)). This completes the proof.
14

---

Published as a conference paper at ICLR 2025
C
PROOF OF THEOREM 2
According to Theorem 3.3 of Liu et al. (2023). the ODE in Eq. 7 generates the samples from the ground-truth velocity distributions at space time location (Iz, t). other words, the random variable
V 11
Now we consider the characteristic function of ZI+ Zz V fort [0, and At E [0, (0.1-t], t]], assuming that Z has the same distribution as X,. If the characteristic functions of Zi+ai and X1+ agree, then Z A and Xt+Ar have the same distribution.
To show this, we evaluate the characteristic function of Zita6.
E (hh
/ (((k.Xx+Va
/ eliitttuse Ti (e ()
/ @ichatttt Pola tv) (1a + (1 - t)p) pe pelri)doda
px()))
/ ((Mi(lttt)))) po(xs 1 + (1 tyu)
b
po (2 (di)dr
EEeess (etihiXtaa))
(21)
We use the notation (:-)) to denote the inner product. The equality a is valid due to Theorem I The equality b holds because It tv and 2x - -1- (1-1)v with the linear interpolation. Therefore, we find that ZI+a/ and X follow the same distribution. In addition, since Zo and X₂ follow the same distribution Pu, can conclude that Zi and X, follow the same marginal distribution at t for t e [0,1].
This completes the proof.
D DENSITY ESTIMATION
In the following, we describe two approaches for density estimation. The resulting procedures are summarized in Algorithm and Algorithm 4 To empirically verify the correctness of the density estimation procedures, we train an RF baseline and an HRF2 model using a bimodal Gaussian target distribution and a standard Gaussian source distribution (see Appendix [H.1] for more details). In Fig. 7 we compare 1) the ground truth density, 2) the density estimated for the RF baseline model, and 3) the densities estimated for the HRF2 model with both procedures. We also report bits per dimension (bpd) for experiments on the ID 1W - 2N, 2D 8N + moon, CIFAR-10, 10, and ImageNet- 32 data. The results are shown in Table We observe that HRF2 consistently outperforms the RF baseline.
To estimate the density, according to Eq. in Theorem we have
log p1 (z1) log 1 (1; =t, !) + log (x())) - log pa Iu), with 1
(22)
This implies that for any given è E [0, 1]. we can use Eq. 22 to estimate the density for generated sample 21. We can choose using the linear interpolation in Eq. with 2g Pa-
For = 0, we observe that (1 = M (21 - 2p: 20, 0), where 2g ~ Py- In this case, we can directly evaluate the likelihood of the generated sample via the velocity distribution. We discuss evaluation of the likelihood below. The procedure to compute the density is summarized in Algorithm 3
Fort t 1, the right-hand side of Eq. 22 becomes log P1 (21) because log mi (u; 21, 1) log Po(21
4), which cancels out with the last term in Eq. (22) Hence, t can't be used to estimate the density.
For E (0, 1), we need to evaluate (x)) to estimate the likelihood of 21. Considering a one step linear flow from zo at time 2, at I, we have 2, 20 vt and pe p=(2, (2)) m (0; 20, 0). Using
15

---

Published as a conference paper at ICLR 2025
Algorithm 3: Density Estimation (t 0)
Input: : Generated sample 21 and the source distribution po-
Sample ~ Po
Compute u 21 20
Compute (21) = I (w; =o. 0) according to Eq. 24
(Optional) Compute (21) = N Ei=l #i (u 0 0), with u(() = - =/00 and 20)) ~ po:
Output: (11)
Algorithm 4: Density Estimation 2 (t E (0, 1))
Input : Generated sample 21 and the source distributions Po and To-
Draw random ~ (0, 1);
Sample 2y ~ Pb
Compute 2g = tz1 + (1 2) and 1-1
Evaluate Po 2 fu), (2(2 according to Eq. (3 and क (w; 2,, t) according to Eq. 24
Compute the log likelihood according to Eq.
Output: أi (21)
it, the density at time & can be computed according
Po(ze) / + 24 20 20,0 po po(z)) dzo
( 20
T
(23)
where 266 Po- Algorithm outlines the procedure for the likelihood computation with randomly drawn t E (0, 1). Optionally, we can average across randomly drawn t 1).
To evaluate the (log-)likeihood of velocity u location 2, and time 4, which is needed in both cases (t- = 0 and t E (0, 1)). we follow the approach introduced by vyCh et al. (2018): Song et al. 2021bb and numerically evaluate
log m (i; Ze, 1) = log no (uoi ze: () 1," Vn, ag(221s Ty, dr.
(24)
Here, the random variable u, backward with a fixed u at r = 1. The term Vu, ag a00 u., is computed by using the Skilling- Hutchinson trace estimator Epp))) (V"V a(z,,, Wr, r)c (Skilling 1989 Hutchinson 1990 Grath-
function of can be obtained by solving the ODE in Eq. 7
wohl et al. 2018). The vector-Jacobian product c'V,, v, a(2,, u, T) can be efficiently computed by using reverse mode automatic differentiation, approximately the same cost as evaluating 4,,,,4,7,,7) T).
In experiments, we the RK45 ODE solver (Dormand & Prince 1980) provided by the scipy.integrate solve_ivp package. We use atol = 1e-5 and rtol = 1e-5. When imple- menting Algorithm 4 we use N 1000 evaluate (zx)).
As mentioned above, to empirically verify the correctness of the density estimation procedures, we train RF baseline and HRF2 model using bimodal Gaussian target distribution and a standard Gaussian source distribution. We compare the density estimated for the RF baseline model and the
densities estimated for the HRF2 model with both Algorithm and Algorithm Fig. 7 aa) compares
the results obtained with Algorithm 3 to the RF baseline and the ground truth. Fig. 7 b) compares
the density estimated for different times with Algorithm 4 to the baseline and the ground truth.
Regardless of the choice of algorithm and time, observe that the HRF2 model obtains better estimation of the likelihood. Importantly, both procedures provide a compelling way to estimate densities.
In Table I we report bits per dimension (bpd) for experiments on the ID LN - 2N, 2D 8N -> moon, CIFAR-10, 10, and ImageNet-32 data. For ID data, 2n = 0 suffices for compelling results. For higher dimensional data, we use 20 samples 2n as shown in the optional line 4 of Algorithm 3 compute the bits per dimension. We observe that HRF2 consistently outperforms the RF baseline.
16

---

Published as a conference paper at ICLR 2025
RF
1.2
t-0.2
HRF2 0.4
1.0-
10.
Lisllooo
0.8
0.8
0.6
-1.5
-1.0
-0.5
0.0
0.5
1.5
-15
-1.0
-0.5
0.0
0.5
1.0
1.5
Data
Data
(a) Algorithm
(b) Algorithm
Figure 7: Density estimation results and comparison to ground truth. Irrespective of the choise of algorithm and the choice of time, we observe compelling density estimation results. We also note that the HRF2 model improves upon the RF baseline.
NLL (BPDJ)
1W + 2N
8N - moon
CIFAR-10
ImageNet-32
Baseline (RF)
0.275
2.119
2.980
3.416
Ours (HRF2)
0.261
2.113
2.975
3.397
Table 1: Density estimation on ID 1N + 2N, 2D &N + moon, CIFAR-10, and ImageNet-32 data using bits per dimension (bpd). We observe consistently better density estimation with the HRF2 model.
E HIERARCHICAL RECTIFIED FLOW FORMULATION DETAILS
In this section, we show how Eq. 8 can be derived from Eq. For convenience we re-state Eq.
inf -, [[[(1] 1Bxo) - (zt, (21, f
(25)
For D 2, note that xι 1720 is equivalent 21 Uo =
(1) 20 = Letting To x and
(1)
(2) = we obtain x1 10 = xι 20 Up-
U[0, 1J2, since and are drawn independently from uniform distribution U [0, 1]. Also, Wo [=0 (1) [x [No, vol Pu+ where Zo and to are drawn independently from standard Gaussian source distributions Po and mo because Pn is D-dimensional standard Gaussian.
Further note that we obtain the time variables t = [E()) (2) [t,
Based on general expression (d) = (1- {(d) Jap (1) + - - 2 (1) and previous results, have
(1) {(0)1 (1 t)zo + tz and e, xx zt (1) (1 {(1)=o
(1) = r) + TU1- This identical to the computation of x, and Uy- Combining all of these results while renaming the function from f to a, arrive
(1 jo (2) + (() 0o
inf Ez Exo D.N~U 001
[][( 20 vn) a(xe, L, Ur, T)
(26)
This program is identical to the one stated in Eq.
F EXPERIMENTAL AND IMPLEMENTATION DETAILS
F.1 Low DIMENSIONAL EXPERIMENTS
For the ID and 2D experiments, we use the same neural network. consists two parts. The first part processes the space and time information separately using sinusoidal positional embedding and
17

---

Published as a conference paper at ICLR 2025
linear layers. In the second part, the processed information is concatenated and passed through series of linear layers to produce the final output. Compared to the baseline, our HRF model with depth D takes D times more space and time information input. Therefore, the first part of the network has D times more embedding and linear layers to handle spatial and temporal information from different depths. However, by adjusting the dimensions of the hidden layers, reduced the network size to just one-fourth of the baseline, while achieving superior performance. For each dataset in the low-dimensional experiments, we use 100,000 data points for training and another 100,000 data points for evaluation. For each set of experiments, we train five different models using five random seeds. During the evaluation, we performed total of 125 experiments and averaged the results to ensure the fairness and validity of our findings.
F.2 HIGH DIMENSIONAL EXPERIMENTS
In the nigh-dimensional image experiments, we used the U-Net architecture described by Lipman jt al. (2023) for the baseline model. To handle extra inputs V, and т, we designed new U-Net-based network architectures for MNIST, CIFAR-10, and ImageNet-32 data.
MNIST. For MNIST, we use single U-Net and modify the ResNet block. Similar to the neural network used in our low-dimensional experiments, each ResNet block has two parts. In the first part, we handle two sets of space-time information, i.c., (Ie, t) and (V,, 7), separately with 2 distinct pathways: convolutional layers for spatial data and linear layers for time embeddings. In the second part, all the spatial data and time embeddings are added together and passed through a series of linear layers capture the space-time dependencies. For fair evaluation, adjusted the number of channels such that the model sizes approximately match (ours: 1.07M parameters vs. baseline: 1.08M parameters). We note that the HRF formulation significantly outperforms the baseline. The results were shown in Fig. 6 More results are provided in Appendix H.3
use U-Nets with the same number of layers but different channel sizes. We use a larger U-Net with channel size 128 to process the velocity U, and time T. We use another smaller U-Net with channel size 32 to process the location I2 and time t. We merge the output of each ResNet block of the smaller U-Net with the corresponding ResNet block of the bigger U-Net. The size of this new U-Net structure is 1.25x x larger than the baseline (44. 4IM parameters in our model and 35.75M parameters in the baseline). Our model achieves slightly better generation quality (see Fig. 6 in Section 3 and Table in Appendix H.3)
CIFAR-10. For CIFAR-10,
ImageNet-322 32. For ImageNet-32, we adopt the same architectural setup for CIFAR-10 but modify the attention resolution to "16,8" instead of just "16" better capture the increased multimodality of the ImageNet-32 dataset. Our U-Net model has a parameter size of 46.2 compared to 37 .06M for the baseline. It demonstrates slightly improved generation quality (see Fig. in Section and Table 7 in Appendix H.
For training, we adopt the procedure and parameter settings from Tong et al. (024)) and Lipman et al. (2023). We use the Adam optimizer with 81 0.9, 82 0.999, and-
10 with no weight decay. For MNIST, the U-Net has channel multipliers [1, 2, 2]. while for CIFAR-10 and Image the channel multipliers are [1, 2, 2). The learning rate is set to 1 x 10-1 with batch size 128 for MNIST and CIFAR-IO. For ImageNet-32, we increase the batch size to 512 and adjust the learning rate x 10-4, 4 We train all models on single NVIDIA RTX A6000 GPU. For MNIST, we train both the baseline and our model for 150,000 steps while we use 400,000 steps for CIFAR-10.
G ABLATION STUDIES
G.1 ABLATION STUDY FOR NFE
The sampling process of HRF with depth D involves integrating D ODES using Euler's method. The total number of neural function evaluations (NFE) is defined as NFE = IT where (d)) is the number of integration steps at depth d. Note, for a constant NFE budget, varying the ((d) values can lead to different results. Therefore, we conduct an ablation study to understand suitable choices for (d)
18

---

Published as a conference paper at ICLR 2025
Total NFEs Sampling Steps
N + 2N
N - 5N
2N
2N
N
62(2D)
8N - moon
1-WD
1-WD
1-WD
2-SWD
2-SWD
100
(1,
0.020
0.031
0.045
0.070
0.172
100
(2,50)
0.025
0.019
0.011
0.037
0.107
100
(5,20)
0.022
0.020
0.010
0.045
0.119
100
(10,10)
0.025
0.019
0.017
0.053
0.163
100
(20,5)
0.026
0.017
0.030
0.062
0.201
100
(50,2) 2)
0.047
0.030
0.075
0.081
0.222
100
(100, 1)
0.032
0.030
0.050
0.085
0.177
Table 2: HRF2 performance for low dimensional experiments under the same NFE 100 budget with different choices of sampling steps. Sampling steps (J, L) indicates that use J steps to integrate= x and L steps to integrate v. 1-WD refers to the 1- I-Wasserstein distance and 2-SWD refers the Sliced 2- 2-Wasserstein distance. Bold for the best. Underline for the runner-up.
4.0
RF
3.5
HRF2
HRF3
3.0
HRF4
HRFS
8
0.5
2500
5000
7500 10000
12500 15000 17500 20000 Training Steps
Figure 8: Training losses of HRF with different depths on ID data, a standard Gaussian source distribution a mixture of Gaussians target distribution. We observe training remain stable.
As shown in Fig. increasing the number of integration steps improves the sampling of the velocity distribution. However, beyond a certain threshold, the benefit of additional steps does not justify the increased computational cost. Table further illustrates that, for fixed NFE budget, compelling strategy is allocate a sufficient number steps to accurately sample v for a precise velocity distribution while using fewer steps integrate over
G.2 ABLATION STUDY FOR DEPTH
Our HRF framework can be extended to an arbitrary depth D. Here, we compare the training loss convergence of HRF with depths ranging from to where HRFI corresponds the baseline RF. As illustrated by the training losses shown in Fig. g training stability remains consistent across different depths, with higher-depth HRFs demonstrating comparable stability to lower-depth models.
Importantly, note that Fig. maminnyy serves to compare convergence behavior and not loss magnitudes as those magnitudes reflect different objects, i.e., velocity for depth of 1, acceleration for depth of 2, etc. Moreover, the deep net structure for the functional field of directions f depends the depth, which makes a comparison more challenging. Table 3 and Table indicate that increasing the depth results in manageable model size, training time, and inference time. These trade-offs are justified by the significant performance improvements observed in Fig. and Fig. 5 See Appendix H.I for details regarding the training data.
19

---

Published as a conference paper at ICLR 2025
Training
ID data
2D data
RF (0.30M)
HRF2 (0.07M)
HRF3 (0.67M)
RF (0.33M)
HRF2 (0.08M)
HRF3 (0.71M)
Time (x10-2 s/iter)
1,292
0.736
2.202
1,503
0.737
2.252
Memory (MB)
2011
1763
2417
2091
2605
Puram. Counts
297,089
74,497
673,793
329,986
76,674
711,042
Table 3: Computational requirements for training synthetic datasets. All models in this table trained for 15000 iterations with a batch size of 51200.
Inference Time (8)
1D data
21> data
Total NFEs
(0.30M)
HRF2 (0.07M)
HRF3 (0.67M)
RF (0.33M)
HRF2 (0.08M)
HRF3 (0.71M)
5
0.030 0.014
0.014 0.005
0.037 0.030
0.035 0.017
0.017 0.006
0.041 0.034
10
0.069 + 0.020
0.033 = 0.000
0.128 0 0.001
0.078 0.025
0.039 0.000
0.145 = 0.001
50
0.372 + 0.024
0.164 # 0.000
0.642 # 0.001
0.440 = 0.001
0.193 = 0.000
0.727 = 0.001
100
0.755 0.001
0.327 0.000
1.291
0.001
0.385
1.455 0.003
Table 4: Inference time comparison for synthetic data using a varying NFE budget. For HRF2, we used sampling step combinations: (1,5). ().5), (5, 10), (10, 10). For HRF3, we used sampling step combinations: (1, 5), (1. 2, (1, 10), (2, 5, 10). For all experiments, we set our batch size to 100,000.
H ADDITIONAL EXPERIMENTAL RESULTS
H.1 ADDITIONAL ID RESULTS
The results for experiments used in Fig. T and Fig. Jare shown in Fig.
H.2 HIERARCHICAL RECTIFIED FLOW WITH OTCFM
As mentioned in Section 5 various approaches for straightening the paths in flow matching models exist. These approaches are orthogonal to our work and can be easily incorporated the HRF formulation. To demonstrate this, incorporate the minibatch optimal transport conditional flow matching (OTCFM) (Tong et al.] (024) into the two layered hierarchical rectified flow (HRF2).
In OTCFM, for each batch of data (120 (4) Ji=1, (m] (B } seen during training, we sample pairs of points from the joint distribution Tbath (To, x1) given by the optimal transport plan between the source and target points in the batch. We follow the same procedure to couple noise with the data points and use the batch-wise coupled Zy and I1 to learn the parameters in ag. We refer to this approach HOTCFM2. We test its performance on two synthetic examples: 1) ID example with a standard Gaussian source distribution and a mixture two Gaussians as the target distribution; and 2D example with mixture of eight Gaussians the source distribution and the moons dataset as the target distribution. Fig. 10 and Fig. I show that hierarchical rectified flow improves the performance of OTCFM.
H.3 ADDITIONAL RESULTS ON MNIST, CIFAR-10, AND IMAGENET-32
Here show additional results for experiments with MNIST, 100AR-10, and ImageNet-32 data. From Tables to 7 we can observe the following: For MNIST, our model is comparable in size, comparable training times, and comparable in inference times, while outperforming the baseline. For CIFAR-10 10 and ImageNet-32, our model is 1.25x larger and has a slower inference time. How- ever, as shown Table 7 it still outperforms the baseline. We believe that the modest trade-off in model size and inference time is acceptable given the performance gains.
20

---

Published as a conference paper at ICLR 2025
GouRd Tah
10.
12
04
62.
65.
05
02
20
50 65
Goued Yan
X
X
12-
68-
00
65-
66-
3.c
1872
1872 20 200
22-115-1--15
Sold MES
(a) Data distribution
(b) Metrics
(c) RF trajectories
(d) HRF trajectories
Figure 9: More experiments on ID data: top row shows results for a standard Gaussian source distribution and a mixture of Gaussians target distribution; bottom row shows results for a mixture of 2 Gaussians source distribution and the same mixture of 2 Gaussians target distribution.
Groud Burk HITENG
1.
1.2
112
64
0.2
02
15 10
45 05
Teaal N7
(a) Data distribution
(b) Metrics
(c) OTCFM trajectories (d) HOTCFM2 trajectories
Figure 10: Results for ID data, with Po being a standard Gaussian and P1 being a mixture of Gaussians. (a) Histograms of generated samples and P1. (b) The I-Wasserstein distance vs. total
NFEs. (c,d) The trajectories of particles flowing from source distribution (grey) to target distribution (blue).
-Jess OTEE HOTOM stegs
Siece VAuasrrrIE Disercee
12:4
25
150
125 150 175 200
Tatal MFT
(a) Metrics
(b) OTCFM trajectories
(c) HOTCFM2 trajectories
Figure 11: Results for 2D data, with pa being a mixture of 8 Gaussians and #1 being represented by the moons data. (a) Sliced 2- 2-Wasserstein distance vs. total NFEs. (b) and (c) show the trajectories (green) of sample particles flowing from source distribution (grey) to target distribution (blue).
Training
MNIST
CIFAR-10
ImageNet-32
RF (L.08D)
HREZ (L.07M)
RF (35.75M)
HRF2 (44.81M)
RE (37.06M)
HRF2 (46.21M)
Time (s/iter) 0.1
0.1
0.3
0.7
0.8
Memory (MB)
3935
3931
8743
10639
27234
33838
Param. Counts
1,075,361
1,065,698
35,746,307
44,807,843
37,064,707
46,210,083
Table 5: Computational requirements during training on image datasets.
21

---

Published as a conference paper at ICLR 2025
Inference time (6)
MNIST
CIFAR-10
ImageNet-32
Total NEES
RF (L08M)
HRF2 (L.7M)
RF (35.75M)
HRF2 (44.1M)
RF (37.06M)
HRF2 (46.21M)
0.084 = 0.001
0.085 : 0.001
0.221 +: 0,000
0.295 0.000
0.229 0.000
0.301 0.000
10
0,168 0.000
0.169 1 0.000
0.441 1 0.001
0.589 1 0.001
0.458 1 0.000
0.601 0,000
20
0,336 = 0.000
0.339 : 0.000
0.889 + 0.001
1.176 + 0.001
0.918 0.001
1.207 = 0.001
0.843
0.851
2.229 0.001
2.953 0.004
2.302 0.002
3.029 0.004
100
1,693 0.002
4,471 + 0.004
5.921 0.003
4.618 0.003
6.100 =
500
8.538 0.030
8.598 0.010
22.375 = 0.011
29.701 1 0.011
23.1 4 0.005
30.863 + 0.083
Table 6: Inference time comparison for MNIST, CIFAR-10, and ImageNet-32 datasets using varying total NFEs budget. For HRF2 MNIST used sampling step combinations: (1, (2, 5), (5, 4), (5, 10), (5, 20), (5, 100). For HRF2 on CIFAR-10 and ImageNet-32 we used sampling step combinations: (1, 5), (1, 10), 20), (1,50), (2, 50), (2, 250). All experiments conducted with a batch size of 128.
Performance (FID)
MNIST
CIFAR-10
ImageNet-32
Total NFEs
RF (L.8M)
HRF2 (L#7M)
RF (35.75M)
HRF2 (44.81M)
RF (37.06M)
HRF2
5
19.187 + 0.188
15.7%% : 0.151
36.209 + 0.142
30,884 : 0.104
6 0.166
48.933 + 0.177
7.974 0.119
6.644 0.076
21.744 0.045
20.286 0.022
20
6.151 0.090
3,408 0.076
8.355 0.065
7.129 : 0.027
12,411 0.002
12.492 : 0.100
50
5.605 0.057
2.664 0.058
5.514 0.034
4.847 0.028
8,910 4 0.137
9.024 0.112
100
5.563 0.049
2,588 0.075
4.588 0.013
4.334 : 0.054
7,873 0.110
7.679 0.022
500
5.453 0.047
2.574 0.121
3.887 0.035
3,706 + 0.043
6.962 + 0.087
6.503 0.035
Table Performance comparison for MNIST, CIFAR-10, and ImageNet-32 datasets using varying total NFEs budget. For HRF2 MNIST used sampling step combinations: (5, 1). (10, 1), (5, (), (10, 5), (10,10). 10), (100, 5). For HRF2 on CIFAR-10 and ImageNet-32 we used sampling step combinations: (L,), 10), (1, 20), 50), 50), (2, 250). Bold for lower mean.
22


Reviews:
[{"review_id": "EGjLXGk7bH", "reviewer": "Reviewer_tEsT", "summary": "This paper introduces a novel framework, hierarchical rectified flow, to model data distributions. It addresses the limitations of conventional rectified flow (RF), which only captures mean velocity, by incorporating distributional information in the velocity space. The objective, equivalent to acceleration matching, derives target acceleration from the data and prior velocity distributions. The framework can be extended beyond acceleration, creating a hierarchical generation model. The proposed approach offers benefits such as easier path integration and improved results with fewer NFEs.", "questions": "Please see the Weaknesses section for my concerns and questions.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The strengths of the paper are listed below:\n- Clear motivation with well-written, easy-to-follow presentation.  \n- Experimental results that partially support the theoretical claims.", "weaknesses": "The weaknesses of the paper are listed below:\n- While the paper's motivation is sound, my main concern lies in the practicality and application of the proposed approach. The generation framework demands significantly more model calls compared to the conventional RF framework, specifically NFEs multiplied by the number of discretizations in the velocity space. Although RF only targets mean velocity, it delivers strong empirical results with far greater efficiency than the proposed method. Additionally, RF can be rectified multiple times for better results. I recommend that the authors conduct a comprehensive comparison between the rectified RF and the proposed method. This inefficiency could lead to significantly limited applicability, especially for larger datasets like ImageNet. Could the authors include a comparison of generation performance and training/inference time between the rectified RF and the proposed method on the existing datasets, or ideally on ImageNet?\n\n- The inefficiency becomes even more pronounced with increased depth; for example, a D-depth model requires a substantially larger architecture compared to a one-depth model.  Does the forgetting problem of the optimal trajectory occur with increased depth, necessitating an expanded architecture? Could the authors provide an ablation study on how model performance and computational requirements change as depth increases?\n\n- The paper lacks a comprehensive comparison of efficiency, including training and inference time, against RF and rectified RF. The paper should include details of the neural network architecture to ensure reproducibility in toy settings. The paper should include more details of the neural network architecture and experimental settings to ensure reproducibility in toy scenarios.\n\n- On the theoretical side, does increasing depth make it harder for the model to converge during training? It would be helpful to include empirical evidence of convergence rates for models with different depths or a discussion of any theoretical bounds on convergence that might exist when using larger depths.\n\n- Minor typo on line 156: it should be \\pi_0. Please review and correct all typos.\n\nNote: The review has been edited to make it more actionable for the authors, as suggested by the PCs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hFsM7feDwm", "reviewer": "Reviewer_Cauw", "summary": "The work enriches the concept of rectified flow for generating data from known base distribution to data distribution. Compared to the base approach, hierarchical rectified flow formulation models the multi-modal random velocity field and acceleration field, which leads to generating more interesting trajectories. The authors evaluate their approach on some low-dimensional use cases and small image benchmark data.", "questions": "Please refer to the weakness section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "***Clarity and Accessibility***: The paper is well-written and presented in a logical manner, making the concepts and methodology easy to follow. Complex ideas are explained in a way that is accessible to readers with varying levels of familiarity with flow-based generative models, facilitating broader understanding and engagement with the research.\n\n***Clear and Impactful Motivation***: The authors provide a compelling motivation for developing hierarchical rectified flow models, addressing limitations in existing flow-based generative models. This work has the potential to significantly impact the field by introducing a more flexible and expressive approach, which could inspire further research and applications in generative modeling.\n\n***Solid Theoretical Foundations***: The paper presents strong theoretical foundations that support the proposed hierarchical model. These well-motivated considerations provide a rigorous basis for understanding how the model improves upon previous approaches, making the proposed framework robust and trustworthy.\n\n***Intuitive Experimental Demonstrations***: The inclusion of low-dimensional toy experiments adds an important educational aspect to the study, offering readers an intuitive way to grasp the dynamics of the approach. These experiments clarify how the model handles multi-modal distributions and complex trajectories, offering insights that make the approach more transparent and easier to analyze.", "weaknesses": "***Ambiguity in Extended Hierarchical Approach***: Although the acceleration-based approach and its motivation are clear and well-justified, the transition to the extended hierarchical flow model lacks clarity. Specifically, while the training objective for the acceleration-based approach is defined by Equation (8), the relationship to the hierarchical model’s training objective, outlined in Equation (10), is not thoroughly explained. The conceptual progression and the structural specifics of the higher-dimensional source distribution (D-dimensional) are left somewhat ambiguous. Additional details on how this source distribution is constructed and how it interfaces with the hierarchical model would clarify the extension.\n\n***Scalability and Computational Complexity***: While the hierarchical rectified flow method is innovative, its computational demands are significant, particularly when scaling to high-dimensional data. Even when considering latent models, the approach appears to be resource-intensive and potentially impractical for high-dimensional images due to its complex sampling procedure. This aspect could hinder its use in real-world scenarios where efficient sampling and rapid training times are crucial. The authors should provide a more detailed analysis of the computational overhead, comparing training and sampling times with other flow-based generative models to highlight both the benefits and trade-offs of their approach.\n\n***Limited Scope of Experiments***: The experimental evaluation primarily uses small, low-dimensional image datasets, which limits the generalizability and relevance of the results. Given current advancements in generative modeling, such datasets do not fully showcase the potential of the hierarchical approach. Adding experiments on higher-dimensional or more complex data—perhaps by incorporating latent diffusion architectures—would better demonstrate the method’s scalability and practical value. More comprehensive experimentation would help readers assess how well the model performs in scenarios closer to those encountered in modern applications of generative models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S24VYvDMVS", "reviewer": "Reviewer_4Etx", "summary": "Flows $p_t$ transform one distribution $p_0$ into another $p_1$ via a corresponding vector field $f(x_t,t)$ that satisfies its continuity equation. The flow is typically defined via conditional flows, for example $p_t(x|x_0, x_1)\\sim t(x_1)+(1-t)x_0+\\sigma\\epsilon$, where one can take the limit $\\sigma\\rightarrow 0$. The corresponding flow is simply $p_t(x)=\\int p_t(x|x_0, x_1) \\pi(x_0, x_1)dx_0 dx_1$. The conditional vector fields $v_t(x_t|x_0,x_1)$ that satisfy the continuity equation of such flows are $x_1-x_0$, while the unconditional vector field is $f(x_t,t)=\\int v_t(x_t|x_0,x_1) p_t(x_0, x_1|x_t) dx_0 dx_1=\\int v_t(x_t) \\pi_t(v_t|x_t) dv_t$.\n\nThis paper studies the possibility of using a sample of $\\pi_t(v_t|x_t)$ at each integration step during data generation, instead of using the mean $f(x_t,t)$ which is akin to using stochastic gradient descent instead of gradient descent. . The result is a stochastic sampling process, which by using a derivation of $\\pi_t(v_t|x_t)$ is proven to have marginals that coincide with the flow $p_t$.\n\nThe distribution $\\pi_t(v_t|x_t)$ is itself modeled using the methodology of rectified flow matching, and the methodology is applied to synthetic and high dimensional image data.", "questions": "1. What are the parameter counts for both HRF and the baseline HF? Also what is the training time per iteration?\n\n2. Is the reported $NFE=J\\cdot L$? What is the time required per NFE in both HFR and RF?\n\n3. The paper claims that the resulting paths are straight as the trajectories can intersect. However the trajectories are stochastic, and thus they are likely to increase overall length. I would appreciate a clarification from the authors regarding this matter.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is generally well written, even though a few parts could be improved.\n\nTheorem 1 and 2 are interesting results, and provide insights about the conditional distribution of velocities at each current point $x_t$ and they enable using higher order stochastic sampling.", "weaknesses": "My **main** concerns are the following:\n\n1. The proposed method requires taking the position and the current velocity as input in order to predict the acceleration. The authors mention expanding the Resnet for their framework and increasing the amount of data processed. Can authors provide a detailed table comparing HRF and RF models, including parameter counts, training time per iteration, and memory usage across all experiments?\n\n2. It is not clear if the NFE includes the steps required to sample the velocities. The paper should report both L and J in each experiment. Also the compute time for each step should be reported and compared with RF.\n\n3. The results for CIFAR-10 unfortunately are not encouraging. Considering the results on MNIST and CIFAR-10 it seems that the proposed model does not scale as well. This is why it is important to also test the model on Imagenet.\n\n4. The resulting model is not a diffeomorphism and as such we lose the ability to perform density estimation.\n\nIn my opinion the main contributions of the work are theoretical, and applying the proposed methodology in practice is challenging.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)