Constant Acceleration Flow
Authors: Dogyun Park
Sojin Lee
Sihyeon Kim
Taehoon Lee
Youngjoon Hong
Hyunwoo J. Kim
Venue: NeurIPS 2024 poster (2024)
Keywords: Generative model
Rectified flow
Fast generation
Primary area: diffusion_based_models
TL;DR: We introduce the Constant Acceleration Flow (CAF) to enhance generative models, outperforming previous methods in speed and precision with new strategies.

Abstract: Rectified flow and reflow procedures have significantly advanced fast generation by progressively straightening ordinary differential equation (ODE) flows under the assumption that image and noise pairs, known as coupling, can be approximated by straight trajectories with constant velocity. However, we observe that the constant velocity modeling and reflow procedures have limitations in accurately learning to couple with flow crossing, leading to suboptimal few-step generation. To overcome the limitations, we introduce the Constant Acceleration Flow (CAF), a novel framework based on a simple constant acceleration equation. Additionally, we propose two techniques to improve estimation accuracy: initial velocity conditioning for the acceleration model and a reflow process for the initial velocity. Our comparative studies show that CAF not only outperforms rectified flow with reflow procedures in terms of speed and accuracy but also demonstrates substantial improvements in preserving coupling for fast generation.

Constant Acceleration Flow
Dogyun Park Korea University gg933@korea.ce ac .rr
Sojin Lee Korea University
Sihyeon Kim Korea University sh bs 1511krrea, ac kr
sojin_ Lein_lee@korea ac kr
Tachoon Lee Korea University 98hoonOkorea.ck kc.
Youngjoon Hong" KAIST
Hyunwoo J. Kim
Korea University
mongyj@kaist ac .kr
hyunwoojkiimkkoeea. kr
Abstract
Rectified flow and reflow procedures have significantly advanced fast generation by progressively straightening ordinary differential equation (ODE) flows. They operate under the assumption that image and noise pairs, known couplings, can be approximated by straight trajectories with constant velocity. However, we observe that modeling with constant velocity and using reflow procedures have limitations in accurately learning straight trajectories between pairs, resulting in suboptimal performance in few-step generation. To address these limitations, we introduce Constant Acceleration Flow (CAF), a novel framework based on a simple constant acceleration equation. CAF introduces acceleration an additional learnable variable, allowing for more expressive and accurate estimation of the ODE flow. Moreover, we propose two techniques to further improve estimation accuracy: initial velocity conditioning for the acceleration model and reflow process for the initial velocity. Our comprehensive studies on toy datasets, CIFAR- 10, and ImageNet 64x64 demonstrate that CAF outperforms state state-of-tthe off-hhe-rt baselines for one-step generation. We also show that CAF dramatically improves few-step coupling preservation and inversion over Rectified flow. Code is available at https://ittub..oom///aaaCCCCCC
Introduction
Diffusion models [1, 2] learn the probability flow between target data distribution and a simple Gaussian distribution through an iterative process. Starting from Gaussian noise, they gradually denoise to approximate the target distribution via series of learned local transformations. Due to their superior generative capabilities compared to other models such as GANs and VAEs, diffusion models have become the go-to choice for high-quality image generation. However, their multi-step generation process entails slow generation and imposes significant computational burden. To address this issue, two main approaches have been proposed: distillation models [3, 4, 5, 6, 7, 9] and methods that simplify the flow trajectories [10, 11, 12, 13, 14] to achieve fewer-step generation. An example of the latter is rectified flow [10, 11, 13], which focuses straightening ordinary differential equation (ODE) trajectories. Through repeated applications of the rectification process, called reflow, the trajectories become progressively straighter by addressing the flow crossing problem. Straighter flows reduce discretization errors, enabling fewer steps in the numerical solution and, thus, faster generation.
Rectified flow [10, 13) defines the straight ODE flow over time i with a drift force v, where each sample x, transforms from X æo to X₃ Ra under constant velocity
Xy- It
"Correpooningg authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Sampling trajectory
Sampling trajectory
a01..0
Xt
Pol (.)
X
aQi2. O
x Flow crossing
x
x
Flow crossing
(a) Rectified Flow
(b) Constant Accelcration Flow
Figure 1: Initial Velocity Conditioning (IVC), We illustrate the importance of IVC to address the flow crossing problem, which hinders the learning of straight ODE trajectories during training. In Fig. la, Rectified flow suffers from approximation errors at the overlapping point X2 (where x = =x?). resulting in curved sampling trajectories due flow crossing. Conversely, Fig. 1b demonstrates that CAE, utilizing IVC, successfully estimates ground-truth trajectories by minimizing the ambiguity at X--
approximates the underlying velocity v with a neural network Vg. Then, it iteratively applies the reflow process to avoid flow crossing by rewiring the flow and building deterministic data coupling. However, constant velocity modeling may limit the expressiveness needed for approximating complex couplings between Ty and TT. This results in sampling trajectories that fail to converge optimally to the target distribution. Moreover, the interpolation paths after the reflow may still intersect- a phenomenon known flow crossing-whic leads eurved rectified flows because the model estimates different targets for the same input. As illustrated in Fig. 1a, instead of following the intended path from x x|. sampling trajectory from Rectified flow erroneously diverts towards x due to the flow crossing. Such flow crossing can make the accurate learning of straight ODE trajectories more challenging.
In this paper, we introduce the Constant Acceleration Flow (CAF), a novel ODE framework based on constant acceleration equation, outlined in (4). Our CAF generalizes Rectified flow by introducing acceleration as an additional learnable variable. This constant acceleration modeling offers the ability to control flow characteristics by manipulating the acceleration magnitude and enables a direct closed-form solution of the ODE, supporting precise and efficient sampling in just few steps. Additionally, we propose two strategies to address the flow crossing problem. The first one initial velocity conditioning (IVC) for the acceleration model, and the second one is to employ reflow to enhance the learning of initial velocity. Fig. 1b presents that CAF, with the proposed strategies, can accurately predict the ground-truth path from x] to x even when flow crossing occurs. Through extensive experiments, from toy datasets real-world image generation on CIFAR-10 [15] and ImageNet 64 64, we demonstrate that our CAF exhibits superior performance over Rectified flow and state-of- ofute-of-the-arr thtther-ttttt baselines. Notably, CAF achieves superior Fréchet Inception Distance (FID) scores on CIFAR- 10 and ImageNet 64 64x66 64 in conditional settings, recording FIDs of 1.39 and 1. 1.69, respectively, thereby surpassing recent strong methods. Moreover, we show that CAF provides more accurate flow estimation than Rectified flow by assessing the 'straightness' and 'coupling preservation' of the learned ODE flow. CAF is also capable of few-step inversion, making it effective for real-world applications such as box inpainting.
To summarize, our contributions are as follows:
We propose Constant Acceleration Flow (CAF), a novel ODE framework that integrates acceleration controllable variable, enhancing the precision of ODE flow estimation compared to the constant velocity framework.
We propose two strategies to address the flow crossing problem: initial velocity conditioning for the acceleration model and a reflow procedure to improve initial velocity learning. These strategies ensure more accurate trajectory estimation even in the presence of flow crossings.
Through extensive experiments on synthetic and real datasets, CAF demonstrates remarkable performance, especially achieving the superior FID CIFAR- 10 and ImageNet 64 x 64 over strong baselines. We also demonstrate that CAF learns more accurate flow than Rectified flow by assessing the straightness, coupling preservation, and inversion.

---

0
Generated
2-Rectificd Flow
CAF (Ours) h
CAF (Ours) h
CAF (Ours) = 2
Figure 2: 2D synthetic dataset. We compare results between 2-Rectified flow and our Constant Acceleration Flow (CAF) on 2D synthetic data. To (blue) and Ti (green) are source and target distributions parameterized by Gaussian mixture models. Here, the number of sampling steps is N = 1. While 2-Rectified flow frequently generates samples that deviate from Ti, CAF more accurately estimates the target distribution T1. The generated samples (orange) from CAF form a more similar distribution the target distribution æ1-
2 Related work
Generative models. Learning generative models involves finding a nonlinear transformation be- tween two distributions, typically denoted Ho and Iι, where To simple distribution like Gaussian, and ) is the complex data distribution. Various approaches have been developed to achieve this transformation. For example, variational autoencoders (VAE) [16, 17] optimize the Evidence Lower Bound (ELBO) to learn nonlinear mapping from the latent space distribution to the data distribution πT. Normalizing flows [18, 19, 20] construct a series of invertible and differentiable map- pings to transform into Similarly, GANs [21, 22, 23, 24, 25] carn generator that transforms Ty into J through adversarial process involving discriminator. These models typically perform one-step generation from to In contrast, diffusion models [2, 26, 27, 28, 29, 30] propose learning the probability flow between the two distributions through an iterative process. This iterative process ensures stability and precision, as the model incrementally learns to reverse a diffusion process that adds noise to data. Diffusion models have demonstrated superior performance across various domains, including images [12, 31, 32, 33], 3D [34, 35, 36, 37], and video [38, 39, 40].
Few-step difflusion models Addressing the slow generation speed of diffusion models has become major focus in recent research: Distillation methods [3, 4, 5, 6, 7, 9] seek to optimize the inference steps of prentrdi diffusion models by amortizing the integration of ODE flow. Consistency models [6, 7, 8] train model to map any point on the pre- mre-trained diffusion trajectory back to the data distribution, enabling fast generation. Rectified flow [10, 11, 13] is another direction, which focuses straightening ODE trajectories under a constant velocity field. By straightening the flow and reducing path complexity, it allows for fast generation through efficient and accurate numerical solutions with fewer Fuler steps. Recent methods such as AGM [41] also introduce acceleration modeling based on Stochastic Optimal Control (SOC) theory instead of relying solely on velocity. However, AGM predicts time-varying acceleration, which still requires multiple iterative steps to solve the differential equations. In contrast, our proposed CAF ODE assumes that the acceleration term is constant with respect to time. Therefore, there is need iteratively solve complex time- dependent differential equations. This simplification allows for a direct closed-form solution that supports efficient and accurate sampling in just few steps.
3 Preliminary
Rectified flow [10, 13] is an ordinary differential equation-based framework for learning a mapping between two distributions Ru and Ti- Typically, in image generation, To is simple tractable distribution, e.g. the standard normal distribution, defined in the latent space and Tı is the image distribution. Given empirical observations of xo xo and X1 Ha over time E [0, 1]. a flow is defined as
dx. dt
v(x,,t).
(1)
3

---

Sampling direction
m
h
2
Figure 3: Sampling trajectories of CAF with different h. The sampling trajectories of CAF are displayed for different values of h, which determines the initial velocity and acceleration. To and иi are mixtures of Gaussian distributions. We sample across sampling steps of N show how sampling trajectories change with h.
where = I(xo. x1,6)
1) is sime-differentiable interpolation between Xy and X1, and v : Rd x [0, 1] + Rd a velocity field defined on data-time domain. Rectified flow learns the velocity field with neural network Vo by minimizing the following mean square objective:
min Xxxxx [ vo(x,,
where y represents coupling of (Tp, T.) and p(t) is time distribution defined on [0, 1]. The choice of interpolation I leads to various algorithms, such as Rectified flow [10], ADM [30], EDM [29], and LDM [42]. Specifically, Rectified flow proposes simple linear interpolation between Xy and X1 as x, = (1 1)xo + [x1, which induces the velocity field v in the direction of (x1 Xn), ie., v(x,, f) = X1 Xg. This means the Rectified flow transports Xg to x along straight trajectory with a constant velocity. After training Va. we can generate sample x using off-the-shelf ODE solvers 4 such as the Euler method:
X X, At tt), E At. ...................... (N ...
1) At).
where At and N is the total number of steps. To achieve faster generation with fewer steps without sacrificing accuracy, it is crucial to learn straight ODE flow. Straight ODE flow minimize numerical errors encountered by the ODE solver.
Reflow and flow crossing. The trajectories of interpolants x; may intersect-a phenomenon known as flow crossing-due to stochastic coupling between To and (e.g., random pairing of x and x)). These intersections introduce approximation errors in the neural network, leading curved sampling trajectories [10]. Our toy experiment, illustrated in Fig. la, clearly demonstrates this issue: the simulated sampling trajectories become curved due to flow crossing, rendering one-step simulation inaccurate. To address this problem, Rectified flow [10] introduces a reflow procedure. This procedure iteratively straightens the trajectories by reconstructing more deterministic and direct pairing of Xo and X1 without altering the marginal distributions. Specifically, the reflow procedure involves generating new coupling 7 of (Xo, X₁ & (xx; v²)) using pre-trained Rectified flow model v
where & denotes the iteration of the reflow procedure, and xxxx; v X + fo vt (x,, l)ddi By iteratively refining the coupling and the velocity field, the reflow procedure reduces flow crossing, resulting in straighter trajectories and improved accuracy in fewer steps.

---

4 Method
We aim develop a generative model based on the ODE framework that enables faster generation without compromising quality. To achieve this, we propose a novel approach called Constant Acceleration Flow (CAF). Specifically, CAF formulates an ODE trajectory that transports x; with constant acceleration, offering a more expressive and precise estimation of the ODE flow compared constant velocity models. Additionally, we propose two novel techniques that address the problem of flow crossing: 1) initial velocity conditioning and 2) reflow procedure for learning initial velocity, The overall training pipeline is presented in Alg. 1.
4.1 Constant Acceleration Flow
We propose a novel ODE framework based on the constant acceleration equation, which is driven by the empirical observations X To and X1 πi over time t E [0, 1] as:
dx, v 0)dt + axx,, f)tdt,
where v: R x [0] + Rd the initial velocity field and I [0, 1] - Rd is the acceleration field. We abbreviate time variable / for notation simplicity, i.e., v(xo. 0) = v(xo). a(x,, () = a (xx)). By integrating both sides of (4) with respect and assuming a constant acceleration field, ie..
a(x,,) = ,, vt1, t2 E [0, 1], we derive the following equation:
x = X + v(xi)t it + a(x))t².
(5)
2
Given the initial velocity field v, the acceleration field a can be derived as
a(x,) 2(x1 - xn) - 2v(x)).
(6)
by setting and the constant acceleration assumption. Then, we propose a time-differentiable interpolation I as:
I(xo, X,, v(x)) = (1 (1- + fx + (t-
by substituting (6) to (5). Using this result, we can easily simulate an intermediate sample x on our CAF ODE trajectory.
Learning initial velocity field. Selecting an appropriate initial velocity field is crucial, as different initial velocities lead distinct flow dynamics. Here, we define the initial velocity field as a scaled displacement vector between X1 and Xo:
v(x)) = h(x1 - xo);
(8)
where h E TR is a hyperparameter that adjusts the scale of the initial velocity. This configuration enables straight ODE trajectories between distributions To and æ1, similar those in Rectified flow. However, varying h changes the flow characteristics: 1) h = 1 simulates constant velocity flows, 2) leads model with positive acceleration, and 3) h > 1 results in a negative acceleration, as illustrated in Fig. 3. Empirically, we observe that the negative acceleration model is more effective for image sampling, possibly due its ability finely tune step sizes near data distribution.
The initial velocity field is learned using a neural network Va. which optimized by minimizing the distance metric .(.,,) between the target and estimated velocities
min Ex,, X: try))) ~-I [d(v(xo), v=(x)))]
(9)
where p(t) is a time distribution defined on [0. 1]. Note that our velocity model learns target initial velocity defined at 0. This differs from Rectified flow, which learns target velocity field defined over E [0, 1].
Learning acceleration field. derived from (6) as
Under the assumption of constant acceleration, the acceleration field
a(x,) 2(x1 - xo) 2x0)).
(10)
We learn the acceleration field using a neural network as by minimizing the distance metric d(---) as: min xxxx) (d(a))))) ax())))
(11)
In Sec. C, we theoretically show that CAF ODE preserves the marginal data distribution.
5

---

Algorithm Training process of Constant Acceleration Flow
Require: deterministic coupling 7, initial velocity scale h. Vo, ag.
while not converge do
2: Xo, X1 ~ t ~ Unif( [0, 1])
3:
v(Xo) x( xo)
Target initial velocity b Eq. (7)
4:
X, = -[(x,. (,
5:
fiel d( d(v(x)), vx())
6:
0 xt- 0 - Vvvl
> update 0 using SGD with gradient
7:
end while
8:
while not converge do
9:
Xo, X1 ~ 7, 1 Unif([O, 1]), Va = vo(xo)
10:
a(x:) 2(x1 xn)
25g
Target acceleration b Eq. (7)
11:
X; = I(x,, X1, vo)
12:
Lace d(sgg(((x:) (x:)] as (x vo))
13:
0 の Lace
update using SGD with gradient
14:
end while
15:
return Vg, as
4.2 Addressing flow crossing
Rectified flow addresses the issue of flow crossing by a reflow procedure. However, even after the procedure, trajectories may still intersect each other. Such intersections hinder learning straight ODE trajectories, as demonstrated in Fig. la. Similarly, our acceleration model also encounters the flow crossing problem. This leads to inaccurate estimation, as the model struggles to predict estimation on these intersections correctly. To further address the flow crossing, we propose two techniques.
Initial velocity conditioning (IVC). We propose conditioning the estimated initial velocity v. (xg) the input of the acceleration model, i.e., (x,,,, va). This approach provides the acceleration model with auxiliary information on the flow direction, enhancing its capability to distinguish correct estimations and mitigate ambiguity the intersections trajectories, as illustrated Fig. 1. Our IVC circumvents the non-intersecting condition required in Rectified flow (see Theorem 3.6 in [10], which is a key assumption for achieving a straight coupling y. By reducing the ambiguity arising from intersections, CAF can learn straight trajectories with less constrained couplings, which is quantitatively assessed in Tab. 4.
To incorporate IVC into learning the acceleration model, we reformulate (11) as:
min E (xx.x
[d (sg [ggaaxx))] (x,, vo))].
(12)
where sg[.] indicates goo-gradieett operation. Since our velocity model learns to predict the initial velocity (see (9)), we ensure that the model can handle both forward and reverse CAF ODEs, which start from Xo and x1, respectively. Thus, our acceleration model can generalize across different flow directions, enabling inversion as demonstrated in Sec. B.2.
Reflow for initial velocity. is also important improve the accuracy of the initial velocity model. Following [10], we address the inaccuracy caused by stochastic pairing of X and X by employing pre-trained generative model v. which constructs a more deterministic coupling 7 of xo and x,. We subsequently use this new coupling to train the initial velocity and acceleration models.
4.3 Sampling
After training the initial velocity and acceleration models, we generate samples using the CAF ODE introduced in (4). The discrete sampling process given by:
Xr+ x, At ve(xo) + + At a (X,, t, ve(xo)),
(13)
where N is the total number of steps, At 1. 1 i-At, and t' (2i+1) At where E (O. N-1)
(See Alg. 2). We adopt l' since it empirically improves accuracy, especially in the small N regime. Notably, when N (one-step generation), L' simplifies to leading to the closed-form solution in (5). See Alg. 3 for inversion algorithm.

---

Algorithm 2 Sampling process of Constant Acceleration Flow
Require: velocity model Vg, acceleration model ag. sampling steps N, mp-
I: Xu T0
2: Ve f vo(Xo)
do 3: for i to
4: tt N
5: l't lit1
6: à f as (x vo)
7: x t X₁ + N vo + Kā
1
8: end for
9: return X1
5 Experiment
We evaluate the proposed Constant Acceleration Flow (CAF) across various scenarios, including both synthetic and real-world datasets. In Sec. 5.1, our investigation begins with a simple two- dimensional synthetic dataset, where we compare the performance of Rectified flow and CAF to clearly demonstrate the effectiveness of our model. Next, we extend our experiments to real-world image datasets, specifically CIFAR-IO (32x32) and ImageNet (64x64), in Sec. 5.2. These experiments highlight CAF's ability to generate high-quality images with a single sampling step. Furthermore, we conduct an in-depth analysis of CAF through evaluations of coupling preservation, straightness, inversion tasks, and an ablation study in Sec. 5.3.
5.1 Synthetic experiments
We demonstrate the advantages of the Constant Acceleration Flow (CAF) over the constant velocity flow model, Rectified Flow [10], through synthetic experiments. For the neural networks, we use multilayer perceptrons (MLPs) with five hidden layers and 128 units per layer. Initially, we train I-Rectified flow on 2D synthetic data establish deterministic coupling. We then train both CAF and 2-Rectified flow. For CAF, we incorporate the initial velocity into the acceleration model by concatenating it with the imput, ensuring that the model capacities of both CAF and 2-Rectified flow remain comparable. We set d as le distance. Fig. 2 presents samples generated from CAF in one step and from 2-Rectified flow in two steps. Our CAF more accurately approximates the target distribution Ti than 2-Rectified flow. In particular, CAF with h = 2 (negative acceleration) learns the most accurate distribution. In contrast, 2- 2-Rettifeed flow frequently generates samples that significantly deviate from T. indicating its difficulty in accurately estimating straight ODE trajectories. This experiment shows that reflowing alone may not overcome the flow crossing problem, leading to poor estimations, whereas our proposed acceleration modeling and IVC effectively address this issue. Moreover, Fig. shows sampling trajectories from CAF trained with different hyperparameters It clearly demonstrates that h controls the flow dynamics acceleration, h 1 represents constant velocity, and < 1 corresponds to positive acceleration flows. Additional synthetic examples are provided in Fig. 6.
intended: h > indicates negative
5.2 Real-data experiments
To further validate the effectiveness of our approach, we train CAF on real-world image datasets, specifically CIFAR-10 at 32x32 resolution and ImageNet at 64x64 resolution. To create a determin- istic coupling y. utilize the pre-trained EDM models [29] and adopt the U-Net architecture of ADM [30] for the initial velocity and acceleration models. In the acceleration model, we double the input dimension of first layer to concatenate the initial velocity to the input x, of the accelera- tion model, which marginally increases the total number of parameters. We set h = 1.5 and d as loss [43] for all real-data experiments.
Baselines and evaluation. We evaluate state-of-the-ar the-trr diffusion models 2, 28, 29], GANs [22, 23, 24], and few-step generation approaches [6, 7]. We primarily assess the image generation quality of our method using the Fréchet Inception Distance (FID) [50] and Inception Score (IS) [51]. Additionally, we evaluate diversity using the recall metric following [6, 7, 10].
7

---

Table 1: Performance on CIFAR-10.
Table Performance on ImageNet 64
64.
Model
N
Unconditional
Conditional
Model
FID4
iSt
Reet
FID
FID4
GAN Models
GAN Models
BigGAN [23]
StyleGAN-Ada [23]
8.51
StylaGAN-XL [24]
BigGAN-deop [22]
4.06
82.35
0.48
DiftusionnConsistnney Models
StyloGAN-XL [24]
1.85
Difuusion/ Models
DDIM [2]]
50
13.7
0.56
SDE [1]
18.3
0.49
DDPM [2]
1000
3.17
DDPM [1]
250
11.0
0.58
VDM [27]
1000
IDDM [47)
250
2.92
0.62
LSGM [28]
138
2.10
ADM [30]
250
2.07
0.63
DDIM [26]
10
13.36
EDM [29]
2.44
48.88
0.67
EDM [29]
37:75
2.01
35,54
1.82
DPM-solver [18]
20
55.3
3.42
7.93
CT [6]
8.70
DEIS (49)
20
3.10
Models Distillation
6.65
Dift' Instruct
CT [6]
11.1
13.0
0.56
0.47
DMD [6:]
3.77
DiTlusoonCommisteneyy Mudels Distillation
DENO
TRACT [45]
3.78
DifT- Diff-Instmett [9]
5.57
KD [46]
DMD [44]
2.62
CD [6]
TRACT H51
7:43
3.55
DENO
7.83
CTM [7]
PD [3]
15.39
0.62
1.98
1.73
4.70
Rectifled Flow Models
CD [6]
6.20
40.08
0.57
Rectiffed Flow [10]
11.81
6.88
CTM [7]
1.73
1,92
64.29
70,38
0.57
2-Rectified Flow Distill [10)
4.84
Rectified Models
CAF (Ours)
4.81
2.68
CAF' (Ours)
6.52
37.45
0.62
CAF GAN (Ours)
CAF + GAN (Ours)
1,69
62.03
0.64
Distillation. Distilling a few-step student model from a pre-trained teacher model has recently
become essential for high-quality few-step generation [6, 7, 10, 11]. InstaFlow [11] has observed that learning straighter trajectories and achieving good coupling significantly enhance distillation performance. Moreover, CTM [7] and DMD [44] incorporate an adversarial loss as an auxiliary loss to facilitate the training of the student model. We empirically found that incorporating the adversarial loss alone sufficient to achieve superior performance for one-step sampling without introducing instability. For training details, please refer to Sec. A.
CIFAR-10. We present the experimental results on CIFAR- 10 in Tab. 1. Our base unconditional
CAF model (4.81 FID, N = 1) significantly improves the FID compared to recent seute-of-thh-ase diffusion models (without distillation), including DDIM [26] (13.36 FID,
10), EDM (37.75
FID, 5), and 2-Rectified flow (7.89 FID, N = 2) in few-step generation (e.g., N < 10). We retrained 2-Rectified flow using the official codes of [10], achieving a slightly better performance than the officially reported performance (12.21 FID) for one-step generation [10]. CAF's remarkable 3.08 FID improvement over 2-Rectified flow (N 2) highlights the effectiveness of acceleration modeling in fast generation. Our approach is also effective in class-conditional generation, where the base CAF model (2.68 FID, N 1) shows a significant FID improvement over EDM (35.54 FID, N 5) and 2-Rectified flow (3.74 FID, N 2). Additionally, after adversarial training, CAF achieves a superior FID of 1.48 for unconditional generation and 1.39 for conditional generation with N 1. Lastly, we qualitatively compare the 2-Rectified flow and our CAF in Fig. 4, where CAF generates more vivid samples with intricate details than 2-Rectified flow.
ImageNet. We extend our evaluation to the ImageNet dataset 64x64 resolution demonstrate the scalability and effectiveness of our CAF model on more complex and higher-resolution images. Similar to the results on CIFAR-1O, our base conditional CAF model significantly improves the FID compared to recent saate-of-the-ase art diffusion models (without distillation) in the small N regime (e.g..
N < 10). Specifically, CAF (6.52 FID, N 1) outperforms models such as DPM-solver [48] (7.93 FID, N 10), CT [6] (11.1 FID, N = 2), and EDM [29] (55.3 FID, N = 5). This validates that the superior performance of CAF can be effectively generalized to complex and large-scale datasets. Additionally, after adversarial training, CAF outperforms or is competitive with state-of-the-are distillation baselines one-step generation. Notably, CAF achieves the best FID performance of 1.69, surpassing strong baselines. We also demonstrate one-step qualitative results in Fig. 14.

---

247
CAF CAFibur
CAP (Oung)
RF Detill
Bestled
ART Dittel
CAP Ditiiel
step
stops
step
sleps
(b)
Figure 4: Qualitative results on CIFAR-10. We compare the quality of generated images from 2 Rectified flow and CAF (Ours) with N = 1 and 10. Each image X is generated from the same Xo for both models. CAF generates more vivid images with intricate details than 2-RF for both N.
Table 3: Coupling preservation.
Table 5: Ablation study on CIFAR-10 (N = 1).
Metric 2-Rectified Flow
CAF (ours)
Config
Constant acceleration
v0 condition
Reflow procedure
FID4
LPIPS PSNR
0.092
0.041 33.16
29.79
378
Table 4: Flow straightness comparison.
B
6.88
C
(h=1.55
V
3.82
Dataset
2-Rectified Flow
CAF (ours)
D
(b=1.5)
V
2.68
2D CIFAR-10
0.065 0.043
0.058
E
((==1)
3.02
0.034
F
(b=0.55
2.73
5.3 Analysis
Coupling preservation. We evaluate how accurately CAF and Rectified flow approximate the deterministic coupling obtained from pre-trained models via reflow procedure. To analyze this, we first conduct synthetic experiments where the interpolant paths I are crossed, as illustrated in Fig. 5a. Due to the flow crossing, the sampling trajectory of Rectified flow fails to preserve the ground-truth coupling (interpolation path Z), leading to a curved sampling trajectory. contrast, our CAF learns the straight interpolation paths by incorporating acceleration, demonstrating superior coupling preservation ability.
Moreover, evaluate the coupling preservation ability real data from CIFAR-10. We randomly sample IK training pairs (xo, from the deterministic coupling y and measure the similarity between x and &, where x generated sample from Xo. In other words, between ground truth image and a generated image corresponding to the same noise. If the coupling well-prre preseredd,, the distance should be small. We use PSNR and LPIPS [52] as distance measures. The result in Tab. 3 demonstrates that CAF better preserves coupling. In terms of PSNR, CAF outperforms Rectified flow by 3.37. This is consistent with the qualitative result in Fig. 5b, where X from CAF resembles more to X1 (ground truth) than & from Rectified flow.
measure the distance
Flow straightness. Flow Straightness Score (NFSS). Similar previous works [10, 11], we measure flow straightness S by the L didstance between the normalized displacement vector (Xo xx) and the normalized velocity vector x; as below:
To evaluate the straightness of learned trajectories, we introduce the Normalized
Xi |x|
X0
*;
S = Exo.x
(14)
X Xol12
Here, smaller value of indicates straighter trajectory. We compare S between CAF and Rectified flow using synthetic and real-world datasets, presented in Tab. 4. For Rectified flow, we use &, vx(x,,, while for CAF, we use x, = vo(xo) + (x,)). The results show that CAF outperforms Rectified flow in flow straightness.

---

(0.250)
(0.081)
(0.181)
(9.048)
CAF CAF(Ous)
(0.328)
(0.057)
(9.023)
GT
(0.34)
2-RF
CAF (0.848
GT
2-RF
CAF (8,050)
(a)
(b)
Figure 5: Experiments for coupling preservation. (a) We plot the sampling trajectories during training where their interpolation paths I are crossed. Due to the flow crossing, RF (top) rewires the coupling, whereas CAF (bottom) preserves the coupling of training data. (b) CAF accurately generates target images from the given noise (e.g., a frog from the car noise). LPIPS [52] values are in parentheses.
from the noise), while RF often fails (e.g..
Inversion We further demonstrate CAF's capability in real-world applications by conducting zero- shot tasks such reconstruction and box inpainting using inversion. We provide implemenetation details and algorithms in Sec. B.2. As shown in the Tab. and 7, our method achieves lower reconstruction errors (CAF: 46.68 PSNR RF: 33.34 PSNR) and better zero-shot inpainting capabilities even with fewer steps compared to baselines. These improvements are attributed to CAF's superior coupling preservation capability. Moreover, present qualitative comparisons between CAF and the baselines in Fig. 12 and 13, which further validates the quantitative results.
Ablation study. We conduct ablation study evaluate the effectiveness of components in our framework under the one-step generation setting (N = 1). We examine the improvements achieved by 1) constant acceleration modeling, 2) initial velocity (vo) conditioning, and 3) the reflow procedure for Vo. The configurations and results are outlined in Tab. 5. Specifically, A I-Rectified flow and 2-Rectified flow, respectively. Configurations C to represent our CAF frameworks, with C being our CAF without IVC. By comparing A,B,C, and F, we demonstrate that all three components in our framework substantially improve the performance. In addition, we analyze the final model across various acceleration scales controlled by h. The performance difference between D and is relatively small, indicating that our framework robust to the choice of hyperparameters, Empirically, we observe that configuration F. i.e., CAF (h 1.5) with negative acceleration, achieves best FID 2.68. Notably, our CAF without vo conditioning. still outperforms rectified flow (configuration B) by 3.06 FID. This highlights the critical role of constant acceleration modeling in enhancing the quality of few-step generation. Also, we verify the significance of reflowing by comparing configurations A and B, which achieve 378 FID and 6.88 FID, respectively.
and correspond
Conclusion
In this paper, we have introduced the Constant Acceleration Flow (CAF) framework, which enhances precise ODE trajectory estimation by incorporating a controllable acceleration variable into the ODE framework. To address the flow crossing problem, we proposed two strategies: initial velocity conditioning and a reflow procedure. Our experiments on toy datasets, real-world dataset demonstrate CAF's capabilities and scalability, achieving of-te-of-the-art FID scores. Furthermore, we conducted extensive ablation studies and amalyses-includigg including assessments of flow straightness, coupling preser- vation, and real-world applications- validate and deepen our understanding of the effectiveness of our proposed components in learning accurate ODE trajectories. We believe that CAF offers promising direction for efficient and accurate generative modeling, and we look forward exploring its applications in more diverse settings such as 3D and video.
10

---

Acknowledgement
This work was supported by ICT Creative Consilience Program through the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government 2120--11122--111111,,, 10%), the National Research Foundation of Korea (NRF) A0C2005/222003773 45%). and the Virtual
(MSIT)
grant funded by the Korea government (MSIT) Engineering Platform Project (Grant No. PO022336, 45%), funded by the Ministry of Trade, Industry & Energy (MoTIE, South Korea).
References
[1] Yang Song. Jascha Sohl- Sill-Dickstein Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, ICIR, 2021.
[2] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances Neural Information Processing Systems, NeurIPS, 2020.
[3] Tim Salimans and Jouathan Ho. Progressive distillation for fast sampling diffusion models. In International Conference on Learning Representations, ICLR, 2022.
[4] Chenlin Meng. Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho,, and Tim Salimans. On distillation of guided diffusion models. Conference on Computer Vision and Pattern Recognition, CVPR, 2023.
[5] Hongkai Zheng, Weili Nie, Arash Vahdat, Kamyar Azizzadenesheli, and Anima Anandkumar. Fast sampling of diffusion models via operator learinng. In International Conference on Machine Learning, ICML, 2023.
[6] Yang Song, Prafulla Dhariwal, Mark Chen, and llya Sutskever. Consistency models. In International Conference on Machine Learning. JCML, 2023.
[7] Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He. Yuki Mitsufuji. and Stefano Ermon. Consistency trajectory models: Learning probability flow ode trajectory of diffusion. International Conference Learning Representations, ICLR, 2024.
[8] Simian Luo. Yigin Tan, Longbo Huang. Jian Li, and Hang Zhao. Latent consistency models: Synthesizing tigh-resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023.
[9] Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff-instruct: universal approach for transferring knowledge from pre-trained diffuusin models. Advances Neural Information Processing Systems. NeurIPS, 2024.
[10] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In International Conference on Learning Representations, ICLR, 2023.
[11] Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. Instaflow: One step enough for high-quality diffusion-based text-to-image generation. In International Conference on Learning Reprexentations, ICLR, 2023.
[12] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for hightiosolution image synthesis. arXiv preprint arXiv:2403.033: 03206. 2024.
[13] Qiang Liu. Rectified flow: A marginal preserving approach optimal transport. arXiv preprint 2.Xi::2099.14577, 2022.
[14] Yaron Lipman. Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. International Conference Learning Reprexentations, ICLR, 2022.
[15] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[16] Diederik P Kingma and Max Welling, Auto-encoding variational bayes. In International Conference on Learning Representations, ICLR, 2014.
[17] Aaron Van Den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In Advances in Neural Information Processing Systems, NeurIPS, 2017.

---

[18] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. In International Conference on Learning Representations. ICLR, 2017.
[19] Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1xl convolutions. In Advances in Neural Information Processing Systems, NeurIPS, 2018.
[20] Derek Onken, Samy Wu Fung, Xingjian Li, and Lars Ruthotto. Ot-flow: Fast and accurate continuous normalizing flows via optimal transport. Association for the Advancement of Artificial Intelligence, AAAI, 2021.
[21] Ian Goodfellow, Jean Pouget-Abadics Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, NeurIPS, 2014.
[22] Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. In International Conference Learning Representations, ICLR, 2018
[23] Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data, In Advances in Neural Information Processing Systems, NeurIPS, 2020.
[24] Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets, In SIGGRAPH, 2022.
[25] Yujin Kim, Dogyun Park, Dohee Kim, and Suhyun Kim. Naturalinversion: Data-free image synthesis improving real-world consistency. In Association for the Advancement of Artificial Intelligence, AAAI, 2022.
[26] Jiaming Song, Chentin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, ICLR, 2020.
[27] Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In Advances in Neural Information Processing Systems, NeurIPS, 2021.
[28] Arash Vahdat, Karsten Kreis, and Jan Kautz. Score- generative modeling in latent space, Advances in Neural Information Processing Systems, NeurIPS, 2021.
[29] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of difuusionbbasee generative models. In Advances in Neural Information Processing Syxtems, NeurlPS, 2022.
[30] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In Advances in Neural Information Processing Systems, NeurIPS, 2021.
[31] James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. hmps://odn. openai, com/papers/dall-e- pdf. 2023.
[32] Sojin Lee, Dogyun Park, Inho Kong, and Hyunwoo J Kim. Diffusion prior-based amortized variational inference for noisy inverse problems. In European Conference on Computer Vision, ECCV. 2024.
[33] Juyeon Ko, Inho Kong, Dogyun Park, and Hyunwoo Kim. Stochastic conditional diffusion models for robust semantic image synthesis. In International Conference on Machine Learning. ICML, 2024.
[34] Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey 7akharov, and Carl Vondrick. Zero-1-to-3: Zero-shot one image to 3d object. In International Conference on Computer Vision, ICCV, 2023.
[35] Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. In International Conference on Learning Representations, ICLR, 2024.
[36] Vikram Voleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion. arXiv preprint 2tt::2400312000 2024.
[37] Dogyun Park, Sihyeon Kim, Sojin Lee, and Hyunwoo J Kim. Ddmi: Domain Domain-agnostic latent diffusion models for synthesizing high-qualit, implicit neural representations. In International Conference on Learning Representations, ICLR. 2024.
12

---

[38] RunwayML Team. Runwayml- gen2. 2023.
[39] Pika Art. Pika art home. 2023.
[40] Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng. Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024.
[41] Tianrong Chen, Jiatao Gu, Laurent Dinh, Evangelos A Theodorou, Joshua Susskind, and Shuangfei Zhai. Generative modeling with phase stochastic bridges. In International Conference on Learning Representations, ICLR, 2024.
[42] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Conference on Compuder Vision and Pattern Recognition, CVPR, 2022.
[43] Sangyun Lee, Zinan Lin, and Giulia Fanti. Improving the training of rectified flows. In arXiv preprint m05.22205.20330, 2024.
[44] Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Conference on Computer Vision and Pattern Recognition, CVPR, 2024.
[45] David Berthelot, Arnaud Autef, Jierui Lin, Dian Ang Yap. Shuangfei Zhai, Siyuan Hu, Daniel Zheng, Walter Talbott, Fric Gu. Tract: Denoising diffusion models with transitive closure time-distillation. In arXiv preprint arXiv:2303.0424 042444 2023.
[46] Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. In arXiv preprint wwXiv:2101.025500 2021.
[47] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference Machine Learning, ICML, 2021.
[48] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfci Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. In Advances in Neural Information P Processing Systems, NeurlPS, 2022.
[49] Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. In arXiv preprint arXiv:2204. 13902, 2022.
[50] Martin Heusel. Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to local nash equilibrium. Advances in Neural Information Processing Systems. NeurlPS, 2017.
[51] Tim Salimans, Ian Goodfellow. Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, NewrIPS, 2016.
[52] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang- The unreasonable effectiveness of deep features perceptual metric. In Conference on Computer Vision and Pattern Recognition, CVPR, 2018
[53] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, ICLR, 2019.
[54] Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved vggan. In International Conference on Learning Representations, ICLR, 2022.
[55] Mingxing Tan and Quoc Le. Efficientne:: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, ICML, 2023.
[56] Hugo Touvron, Matthieu Cord, Mathhij Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning, ICML, 2023.
[57] Ron Mokady. Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Conference on Computer Vision and Patiern Recognition. CVPR, 2023.
[58] Inbar Huberman Spiegelglas. Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: Inversion and manipulations. In Conference on Computer Vision and Pattern Recognition, 2024.
13

---

A Implementation details
We utilize the pre-trained EDM model [29] to build the deterministic coupling Y for training our models. To construct deterministic couplings for CIFAR-10 10 and ImageNet, we select N = 18 and N = 40, respectively, using deterministic sampling following the protocol in [29]. For CIFAR-10 10 and ImageNet, we generate IM and 3M pairs, respectively. We use the batch size of 2048 and train for 700K/700K iterations on ImageNet. For CIFAR-IO, we use the batch size of 512 and train for 500K/500K iterations. For all experiments, we use Adam W [53] optimizer with learning rate of 0.0001 and apply an Exponential Moving Average (EMA) with a 0.999 decay rate. For training acceleration model, initialize it with initial velocity model for faster convergence.
For adversarial training, we employ adversarial loss Laan using real data X1,real from [24]:
Cgan. (Ø) Exi [log ( reat)] )] + 1xx [log(1 (x)()))),+
(15)
where d is a discriminator and & - X + us (xn) + Las (Xo, va(xo)). In the end, we use the following combined loss to update the acceleration model:
L(),n) (Ø) Agan 7):
(16)
where Luce corresponds to (12) and À are weight hyperparameters. Following [42, 54]. we employ adaptive weighting as Agan = a Cxr(e))
[V where Φi is the last layer of the acceleration model. Without Laxe. found the training unstable and frequently exhibit mode collapse issue, which is common problem with adversarial training. We follow the training configuration from StyleGAlI AN- XL [24]. We bilinearly upscale the image to 224x2224 resolution and use EfficientNet [55] and DeiT- base [56] for extracting features. During the adversarial training, we only optimize the acceleration model and discriminator with the learning rate of 2e-5 and 1e-3, respectively. We keep the parameters of the initial velocity model fixed for stable training. The total training takes about 21 days with 8 NVIDIA A100 GPUs for ImageNet, and takes 10 days NVIDIA RTX3090 GPUs for CIFAR-10.
B Additional results
B.1
Additional qualitative results
2D toy dataset. In Fig. 6, we provide additional generation results and sampling trajectories on various 2D synthetic datasets with N 1, demonstrating the effectiveness of approach for fast generation. Fig. provides additional examples of coupling preservation on 2-RF and CAF,
Real-world dataset. In Fig. and 9, we show additional generation results from our base CAF model on CIFAR- with N = 10, and 50. In Fig. 10, we compare the generation result between 2-RF and CAF distilled versions. Fig. shows sampling results from our base CAF models with different hyperparameters h. Lastly. Fig. 14 shows the generation results on ImageNet with N = 1.
B.2 Real-world applications
Inversion techniques are essential for real- applications such as image and video editing [57, 58]. However, existing methods typically require 25-100 steps for accurate inversion, which can be computationally intensive. In contrast, our method significantly reduces the inference time by enabling inversion in just few steps (e.g., N < 20). We demonstrate this efficiency in two tasks: reconstruction and box inpainting.
To reconstruct X1. we first invert x to obtain &o. as described in Alg. 3. We then use the generation process (Alg. 2) with &n and same initial velocity vo(x,) used in Alg. to generate x1 For box inpainting, we inject conditional information the non- maskmddeed image region- -into the iterative inversion and generation procedures, as detailed in Alg. As demonstrated in Tab. 6 and 7, our method achieves better reconstruction quality (CAF: 46.68 PSNR vs. RF: 33.34 PSNR) and zero- shot inpainting capability even with fewer steps compared to baseline methods. Qualitative results are presented in Fig. 12 and 13, which further illustrate the effectiveness of our approach. This demonstrate that our method can be efficiently used for real-world applications, offering both speed and accuracy ad advantages over existing techniques.
14

---

Algorithm 3 Inversion process of Constant Acceleration Flow
Require: velocity model Va, acceleration model ag. sampling steps N, T.
I: X1 "1
2: Ve f (X1)
3: for . to do
4:
tt N
5: l'+ 2i-1
6: å f ag (x,, Vo)
7: x t X N
v0
8: end for
9: return X0
Algorithm 4 Box inpainting of Constant Acceleration Flow
Require: velocity model Vg. acceleration model ag. sampling steps N, reference image X binary image mask 52 where indicates the missing pixels.
1: a ~ (0,
2: X X1
Q) +oo
Create image with missing pixels and add noise
3: Va f Vx(x)
4: for i N to do
Inversion steps
5:
+ N. t' + 26-1
2N
6: à c ao (x,: v1)
7:
x1 t Xt
N Vgg -ā
8:
X4- - x
3 Q2) + +1 0 N(0,1)
9: end for
10: VO f- vx(Xo)
11: for 1 to N 1
Generation steps
12:
t 1, t' t 2j+1
2N
13: ås - ag(x, Vo)
14: X4+ t X1 N vo + N a
x t x1 @ (1 (2) + x
end for 17: return inpainted image
B.3 Comparison with previous acceleration modeling literatures
Here, we elaborate on the crucial differences between AGM [41] and CAF. The main distinction is that CAF assumes constant acceleration, whereas AGM predicts time-dependent acceleration. Since the CAF ODE assumes that the acceleration term is constant with time, there is no need to solve time-dependent differential equations iteratively. This allows for closed-form solution that supports efficient and accurate sampling, given that the learned velocity and acceleration models are accurate. Specifically, the solution for CAF ODE is given by:
X1 = No + / v(xo) a(xi) tdt = xo v(xo) + /'' a(x;) - tdf
(17)
1 Xo v + a(xi) / tdt xo v(xu) + a(x;)
(18)
The integral simplifies thanks to the constant acceleration assumption, leading to one-step sampling. In contrast, AGM's acceleration time-varying. meaning that the differential equation cannot be reduced in an analytic form. It requires multiple steps to approximate the true solution accurately. In Tab. 8, we systemically compare AGM with our CAF, where CAF consistently outperforms AGM. Moreover, we conducted additional experiments where AGM was trained with deterministic couplings as in our reflow setting. Incorporating reflow into AGM did improve its performance in the stw-step regime, which further highlights the distinct advantage of CAF over AGM.
15

---

Table 6: Reconstruction error.
Table 7: Box inpainting.
Model
N
PSNR
LPIPS
Model
NFE
FID 4
CM
N/A
N/A
18
13.16
CTM
N/A
N/A
CTM
N/A
EDM
4
13.85
0.447
EDM
N/A
2-RF
2
33,34
0.094
2-RF
12
16.41
2-RF
29.33
0.204
CAF (Ours)
12
10.39
CAF (Ours)
46.68
0.007
CAF (+GAN) (Ours)
40.84
0.028
CAF (+GAN) (Ours)
12
10.91
Table Comparison between AGM and CAF
Model
Acceleration
Closed-form solution
Reflow for velocity
FID on CIFAR-10 +
AGM [41]
Time-varying
No
No
11.88 (N 5)
AGM (enhanced ver.)
Time-varying
No
Yes
15.23 (N 5)
CAF (Ours)
Constant
Yes
Yes
4.81 (N
C Marginal preserving property of Constant Acceleration Flow
We demonstrate that the flow generated by our Constant Acceleration Flow (CAF) ordinary differential equation (ODE) maintains the marginal of the data distribution, as established by the definitions and theorem in [10].
Definition C.1. For a path-wise continuously differentiable process x fxx : t E [0, 1]}, we define its expected velocity vx and acceleration a* as follow:
[xx
dx
(xx,1) =E
x,
=
(=, =
x
Vo supp(x;).
(19)
dt
dp
For supp(xx). the conditional expectation not defined and we set v* and a² arbitrarily, for example vx(x, t) = O and a² (x,,t) 0.
Definition C.2. [10] We denote that x is rectifiable v is loccall bounded and the solution the integral equation of the form
2, - 20 + L' v² (z,,t))dt Vt E0,10,1]]
Zo X0.
(20)
exists and is unique. In this case, z= (z :t t [0, 1]} called the rectified flow induced by x. Theorem 1. [10] Assume x is rectifiable and z is its rectified Then Law(z:) = Law(x)), Vt E [0, 1].
Refer to [10] for the proof of Theorem 1.
We will now show that our CAF ODE satisfies Theorem by proving that our proposed ODE (4)
induces z, which is the rectified flow as defined in Definition C.2. In (4), we define the CAF ODE as
dx,
dx;
dxx,
(21)
di
dt
d12
r=
By taking the conditional expectation on both sides, we obtain
vx (I, (z, a² tt
(22)
from Definition C.I. Then, the solution of the integral equation of CAF ODE identical to solution in Definition C.2 by (22):
2, = 2g +
v* v* 0) a* (Z,,t) tdt
(23)
0
Zo +
(x( t)dt.
(24)
This indicates that z induced by CAF ODE is also rectified flow. Therefore, the CAF ODE satisfies the marginal preserving property, i..., Law(z,) Law(x)), as stated in Theorem 1.
16

---

D Limitation and Broader impacts
D.1 Limitations
One limitation of our model is the increased number of function evaluations (NFE) required for N-step generation. While Rectified flow achieves an NFE of N by only computing the velocity at each step, our method necessitates an additional computation, resulting in a total NFE of N + 1. This because we compute the initial velocity the beginning and the acceleration at each step. Although this extra evaluation slightly increases the computational burden, it is relatively minor in terms of overall performance and still enables efficient few-step generation. Moreover, this additional step can be reduced by jointly predicting velocity and acceleration terms with single model, which we leave for future work. Another limitation is the additional effort required to generate supplementary data. We utilize generated data to create a deterministic coupling of noise and data samples for training CAF. While generating more data enhances our model's performance, it can increase GPU usage, leading to higher carbon emissions.
D.2 Broader Impacts
Recent advancements in generative models hold significant potential for societal benefits across wide array of applications, such image and video generation and editing, medical imaging analysis, molecular design, and audio synthesis. Our CAF framework contributes to enhancing the efficiency and performance of existing diffusion models, offering promising directions for positive impacts across multiple domains. This suggests that in practical applications, users can utilize generative models more rapidly and accurately, enabling broad spectrum of activities. However, it is crucial to acknowledge potential risks that must carefully managed. The increased accessibility of generative models also broadens the potential for misuse. As these technologies become more widespread, the possibility of their exploitation for fraudulent activities, privacy breaches, and criminal behavior increases. It is vital ensure their ethical and responsible use to prevent negative impacts. Establishing regulated ethical standards for developing and deploying generative AI technologies is necessary to prevent such misuse. Additionally, imposing restricted access protocols verification systems to trace and authenticate generated contents will help ensure their responsible use.
17

---

Generated
2-Rectified Flow
CAF (Ours) h
CAF (Ours) h
CAF (Ours) h
(a) Generation results
x-1
Sampling direction
(b) Sampling trajectories with different
Generated
2-Rectified Flow
CAF (Ours) h
CAF (Ours) h
CAF (Ours) h
(c) Generation results
Sampling direction
à 0
h=1
(d) Sampling trajectories with different
Figure 6: Experiments on various 2D synthetic dataset. We compare results between 2-Rectified Flow and our Constant Acceleration Flow (CAF) on 2D synthetic data. my (blue) and मं (green) are source and target distributions parameterized by Gaussian mixture models. The generated samples (orange) from CAF form a more similar distribution as the target distribution nT.
18

---

(Ours)
2-RF
(Ours)
2-RF
CAF
(Our)
2.RF
(Ours)
2-RF
Figure 7: Additional visualizations of coupling preservation CIFAR-10. CAF accurately generates target images (x₃) from the given noise (Xn), while Rectified Flow often fails to preserve coupling of Xg and X1
19

---

10 steps
stops
10 steps
steps
10 stops
stops
10 slups
Figure 8: Qualitative results on unconditional generation (CIFAR-10). We illustrate generating
images with varying sampling steps, demonstrating consistency quality even for one-step generation.
20

---

airplane
deer
dog
frog
horse
ship
step
10 steps
50 steps
stop
10 steps
50 steps
step
steps
50 steps
step
steps
50 steps
Figure 9: Qualitative results conditional generation (CIFAR-10). We illustrate generating
images with varying sampling steps, demonstrating consistency quality even one-step generation.
21

---

1step
2-RF
CAF (Ours)
10 step
2-RF
CAF (Ours)
Figure 10: Comparisons on unconditional generation (CIFAR-10). We compare distilled model
from 2-Rectified Flow (2-RF-Disiil+GAN)) and CAF (CAF+Distill+GAN) with qualitative results.
22

---

step
h 2
steps
h
Figure 11: Unconditional generation for different h on CIFAR-10. We display qualitative results of CAF for different values of h, indicating that our framework is robust to the choice of h.
23

---

(a) Ground Truth
(b) CAF (Ours) (I step, PSNR 46.68, LPIPS 0.007)
(c) (1 step, PSNR-29.333 LPIPS -P2PP--0224)
Figure 12: Reconstruction results using inversion.
(a) Masked Images
(b) CAF (ours) (12 step, FID=10.39)
(c) 2-RF (12 step, FID=16.41)
(d) CM (18 step, FID=13.16)
Figure 13: Zero-shot box inpainting results. We use 16x16 size mask for masked images in (a). For consistency model (d), we followed their official code for inpainting.
24

---

Figure 14: Qualitative results on conditional generation for ImageNet 64x64 (N FI691169).
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We have done thorough experiments to verify our contribution and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of our work in the supplementary material.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
26

---

Justification:
We have provided complete proof in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We have provided implementation details in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
27

---

Answer: [Yes]
Justification:
We will provide an official code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/ mublic/gudedn http://uidc/edddeeeeeeeee Stw.deeeeoooSSSmssiiiiiiceoo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ //nips. wttpccccpppccccceeccecceccccccct /ttpa//ggudieeeeeeeuuuubiiiiiiii /CodeSubmie ssisnnollll onPolicy))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have provided details in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Single run.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
28

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided details in the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https / /ttp///eerpp/.ee/ee//ee cc
Answer: [Yes]
Justification:
We have followed the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed the broader impact in supplementary materials.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We will provide code and checkpoint with the safeguards.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have credited and respected the assets in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
30

---

Answer:
[NA]
Justification:
We have not released new asset.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/codeemoodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: We have not done any crowdsourcing experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: We have not done any crowdsourcing experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "VSVvPaPEMw", "reviewer": "Reviewer_GaYE", "summary": "This paper develops a new flow based generative model whose vector field is constructed similar to rectified flows (straight path between source and target samples), but instead of using a path with a constant speed, uses a path with constant acceleration.  This requires learning a neural network to parametrize the initial velocity field and another neural network to parametrize the acceleration field.  To improve their model's performance, the authors also propose parametrizing the acceleration neural network using the initial velocity and also applying reflows to the initial velocity.  The proposed method helps mitigate the flow crossing problem at a model level, which should in theory avoid excess reflows to learn non-crossed flows.  The experiments section demonstrates the ability of the proposed method to learn high performing generative models on single step generation on the CIFAR-10 dataset.", "questions": "- Is there a reason why algorithm 1, line 6, updates $\\theta$ before computing the loss for $a_\\phi$?", "limitations": "The authors adequately addressed the limitations.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea is novel, solves a relevant problem and fits nicely into the current landscape of the flow based generative modeling research area.  The paper was easy to read and the method itself is simple enough to understand and implement after a single pass over the paper.  The main contribution of the paper, in my opinion, is the conditioning of the acceleration neural network on the initial velocity.  This gives the final trained model a non-markov path measure, which is something that is of current interest to the field (Augmented Bridge Matching, De Bortoli et al., 2023).", "weaknesses": "- The related work should include references to building diffusion models with non-markov path measures (see De Bortoli et al., 2023 and its related work).\n- For completeness, the paper should include proofs to show that the learned distribution is the same as the data distribution.  This follows directly from interpreting Eq. 2 as a conditional flow matching objective (Lipman et al., 2023).\n- In the initial presentation of the vector field $a(x_t,t)$ at the start of section $4.1$, it would make more sense to drop the dependence on $t$ to emphasize that the ground truth acceleration field is constant in time.\n- The scope of the empirical evaluation is limited.  The authors should have evaluated on more datasets than just CIFAR-10.  There are other small scale image datasets, like ImageNet-32, that could have been evaluated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q.1 [Missing discussion with AugBM]**  The related work should include references to building diffusion models with non-markov path measures (see De Bortoli et al., 2023 and its related work).\n\n**Response to Q.1**\n\nWe appreciate the reviewer for pointing out the related works that we have missed. We recognize that AugBM shares a similar motivation with our method in preserving couplings and addressing flow (bridge) crossing by conditioning auxiliary information to the network. \nHowever, while AugBM aims to improve coupling preservation specifically for image-to-image translation tasks, our main contribution lies in developing an ODE framework that enables fast sampling with high accuracy. To achieve this, we proposed a flow with constant acceleration that provides a one-step closed-form solution (Eq. 5 in our paper). This closed-form solution enables few-step (N<3) sampling (Eq.11 in our paper) with high accuracy when learned with deterministic couplings, as demonstrated by our experimental results.\n\nWe will include a detailed discussion of AugBM and related works in the revised paper to provide a more comprehensive comparison.\n\n---\n**Q.2 [Proof of marginal preserving]** \nFor completeness, the paper should include proofs to show that the learned distribution is the same as the data distribution.\n\n**Response to Q.2**\nWe highly appreciate the feedback and acknowledge the importance of demonstrating the marginal preservation property of our approach. To address the reviewer’s concern, we provide a derivation that the flow induced by our Constant Acceleration Flow (CAF) ordinary differential equation (ODE) preserves the marginal of the data distribution, established by the definitions and theorems in [1].\n\n***Definition 1***: *For a path-wise continuously differentiable process $\\mathbf{x} = {\\mathbf{x}_t : t \\in [0,1]}$, we define its expected velocity $v^{\\mathbf{x}}$ and acceleration $a^{\\mathbf{x}}$ as follow:*\n$$\n\\\nv^{\\mathbf{x}}(x,t) = \\mathbb{E}\\left[\\frac{d\\mathbf{x}_t}{dt} \\ \\bigg| \\ \\mathbf{x}_t = x\\right], \\quad a^{\\mathbf{x}}(x,t) = \\mathbb{E}\\left[\\frac{d^2\\mathbf{x}_t}{dt^2} \\ \\bigg| \\ \\mathbf{x}_t = x\\right], \\quad \\forall x \\in \\text{supp}(\\mathbf{x}_t).\n\\\n$$\n*For $x \\notin \\text{supp}(\\mathbf{x}_t)$, the conditional expectation is not defined, and we set $v^{\\mathbf{x}}$ and $a^{\\mathbf{x}}$ arbitrarily, for example, $v^{\\mathbf{x}}(x,t) = 0$ and $a^{\\mathbf{x}}(x,t) = 0$.*\n\n***Definition 2*** [1]: *We denote that $\\mathbf{x}$ is rectifiable if $v^{\\mathbf{x}}$ is locally bounded and the solution of the integral equation of the form*\n$$\n\\\n\\mathbf{z}_t = \\mathbf{z}_0 + \\int_0^t v^{\\mathbf{x}}(\\mathbf{z}_t, t) dt, \\quad \\forall t \\in [0,1], \\quad \\mathbf{z}_0 = \\mathbf{x}_0,\n\\\n$$\n*exists and is unique. In this case, $\\mathbf{z} = {\\mathbf{z}_t : t \\in [0,1]}$ is called the rectified flow induced by $\\mathbf{x}$.*\n\n***Theorem 1*** [1]: *Assume $\\mathbf{x}$ is rectifiable and $\\mathbf{z}$ is its rectified flow. Then, $\\text{Law}(\\mathbf{z}_t) = \\text{Law}(\\mathbf{x}_t)$ for all $t \\in [0,1]$.*\n\n\\\nRefer to [1] for the proof of ***Theorem 1***. We will now show that our CAF ODE satisfies ***Theorem 1*** by proving that our proposed ODE induces $\\mathbf{z}$, which is the rectified flow as defined in ***Definition 2***. In our paper, we defined the CAF ODE as\n$$\n\\frac{d\\mathbf{x}_t}{dt} = \\left. \\frac{d\\mathbf{x}_t}{dt} \\right|{t=0} + \\frac{d^2\\mathbf{x}_t}{dt^2} \\cdot t.\n$$\nBy taking the conditional expectation on both sides, we obtain\n$$\nv^{\\mathbf{x}}(x,t) = v^{\\mathbf{x}}(x,0) + a^{\\mathbf{x}}(x,t) \\cdot t,\n$$\nfrom ***Definition 1***. Then, the solution of the integral equation of CAF ODE is identical to the solution in ***Definition 2***:\n$$\n\\mathbf{z}_t = \\mathbf{z}_0 + \\int_0^t \\left( v^{\\mathbf{x}}(\\mathbf{z}_0, 0) + a^{\\mathbf{x}}(\\mathbf{z}_t, t) \\cdot t \\right) dt \\\n= \\mathbf{z}_0 + \\int_0^t v^{\\mathbf{x}}(\\mathbf{z}_t, t) dt.\n$$\nThis indicates that $\\mathbf{z}$ induced by CAF ODE is also a rectified flow. Therefore, CAF ODE satisfies the marginal preserving property, i.e., $\\text{Law}(\\mathbf{z}_t) = \\text{Law}(\\mathbf{x}_t)$, as stated in ***Theorem 1***.\n\n---\n**Q.3 [Expression of acceleration model]** \nIn the initial presentation of the acceleration field $a(x_t,t)$ at the start of section 4.1, it would make more sense to drop the dependence on $t$ to emphasize that the ground truth acceleration field is constant in time.\n\n**Response to Q.3**\n\nThank you for your suggestion. We agree that emphasizing that the ground truth acceleration field is constant in time can enhance clarity. We will consider revising the notation.\n\n---\n**Q.4 [Limited evaluation]**\nThe scope of the empirical evaluation is limited. The authors should have evaluated on more datasets than just CIFAR-10. There are other small-scale image datasets, like ImageNet-32, that could have been evaluated.\n\n**Response to Q.4**\n\nThank you for your feedback regarding the scope of our empirical evaluation. To address the reviewer’s concern, we have extended our evaluation to include additional generation results on ImageNet 64x64. These results demonstrate the broader applicability and effectiveness of our framework beyond CIFAR-10. For the quantitative results, please refer to **1. [ImageNet 64x64 results]** in the “**Author Rebuttal**” section above.\n\n---\n**Q.5 [Training algorithm]** Is there a reason why algorithm 1, line 6, updates  $\\theta$ before computing the loss for $a_\\phi$?\n\n**Response to Q.5**\n\nThank you for your question. In our experiments, we found that updating the initial velocity model before computing the loss for the acceleration model leads to more stable training dynamics.\n\n---\n[1] Liu, Xingchao, Chengyue Gong, and Qiang Liu, Flow straight and fast: Learning to generate and transfer data with rectified flow, ICLR 2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tqurKSkf9W", "reviewer": "Reviewer_rYLJ", "summary": "This work proposes Constant Acceleration Flow (CAF), which, instead of learning a velocity-based flow model like in Flow Matching, jointly trains an initial velocity model together with a constant time-dependent acceleration model. This framework aims to learn straighter paths and thus enable better results for few-step generation.\n\nAdditionally, they propose to condition the acceleration model on the initial velocity and propose to improve the initial velocity using a reflow procedure. CAF is empirically validated on toy data and CIFAR-10, including ablations on the different elements of the proposed framework.", "questions": "- As mentioned above, how dependent is CAF on the pre-trained model? Does it also work well without any pre-trained model?\n- Does the 2-Rectified Flow comparison for the CIFAR-10 experiments also use generated samples by EDM for its first training round?\n- Can the velocity and acceleration models be trained separately? E.g. first train the initial velocity model and then train the acceleration model?", "limitations": "- Again, how dependent is CAF on the pre-trained model?\n- Training both an acceleration and a velocity prediction model effectively doubles the amount of total parameters. This could be mentioned as another limitation.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The Constant Acceleration Flow is a new formulation for achieving straighter trajectories in flow-based generative models derived based on the assumption of constant acceleration.\n- Good empirical results on CIFAR-10\n- Includes important ablations on the velocity conditioning as well as on the magnitude of the initial velocity", "weaknesses": "\"Reflow\" for initial velocity:\n- The reflow procedure was proposed to straighten the paths of an existing generative model. The authors propose to use a \"reflow\" procedure for training CAF. However, they don't generate the couplings with a model trained with CAF but instead with a pre-trained EDM model, which is very different from how the reflow procedure was proposed. Additionally, it is unclear how dependent CAF is on this procedure and, thus, from the pre-trained generative model. In the CIFAR-10 ablation, the setting with constant acceleration and velocity conditioning but no \"reflow\" procedure (and thus no pre-trained generative model) is crucial to answer the question of whether CAF also works without a pre-trained model. If it does not, then I would argue that the proposed framework is more of a new distillation technique rather than a new generative model. This contextualization should also be more clearly explained in the main text.\n\nMissing discussion and contextualization of related work:\n- Acceleration Generative Modeling has been proposed in [1]. While CAF is based on Flow Matching and a constant velocity is chosen, [1] considers Bridge Matching in a stochastic optimal control framework with a changing velocity induced by the acceleration prediction. However, their trained model is also a parameterized acceleration prediction that takes as input the current data point $x_t$, current time $t$, and the current velocity $v_t$ effectively conditioning on the velocity. This framework is very related to the approach proposed by the authors, and the connection should be discussed in detail in the main paper. Additionally, an experimental comparison illustrating the differences between the two approaches could be beneficial.\n- Coupling preservation through initial velocity conditioning: The authors refer to the reflow approach as existing related work. However, the reflow approach was proposed to straighten paths, not preserve couplings. Moreover, the problem of coupling preservation has been thoroughly analyzed in [2], which proposes to use a source point conditioning, i.e. conditioning on $x_0$. How does Augmented Flow/Bridge Matching (Flow/Bridge Matching with $x_0$ conditioning) compare to CAF? As mentioned, velocity conditioning is also used in [1].\n- A possible way to empirically compare to these competing methods would be to extend the ablation study on CIFAR-10 to include [1] and [2].\n\nMinor Weaknesses:\n- Experiments are only conducted on CIFAR-10. Including at least one more dataset, e.g. 64x64 ImageNet/CelebA/etc, could strengthen the empirical results.\n\n[1] Tianrong Chen and Jiatao Gu and Laurent Dinh and Evangelos A. Theodorou and Joshua Susskind and Shuangfei Zhai. \"Generative Modeling with Phase Stochastic Bridges\". In ICLR 2024.\n\n[2] Valentin De Bortoli and Guan-Horng Liu and Tianrong Chen and Evangelos A. Theodorou and Weilie Nie. \"Augmented Bridge Matching\". In Arxiv 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We truly appreciate the reviewer for the constructive feedback. We would like to clarify the reasoning behind our approach and address the concerns raised.\n\n**Q.1** **[Why use EDM for reflow?]**\n\n**Response to Q.1**\nIn response to the reviewer’s concern regarding the reflow procedure, we would like to emphasize that the reflow procedure can be flexibly applied to any pre-trained diffusion model, which is a notable advantage of reflow, as demonstrated in InstaFlow [1]. Based on these findings and the inherent flexibility of the reflow procedure, we utilized the pre-trained EDM model to ensure a fair comparison against other fast sampling models, such as CM and CTM, which also utilized pre-trained EDM models.\n\n---\n**Q.2 [Contextualization of work]**\nThe proposed framework seems to be more of a new distillation technique rather than a new generative model. This contextualization should also be more clearly explained in the main text.\n\n**Response to Q.2** We acknowledge the importance of contextualizing the method. We will include the below discussions and contextualize our work more clearly in the main text of the final version. To address the reviewer’s concern comprehensively, we have considered two key points:\n\n- **[Fast sampling as our main motivation]**: We would like to emphasize that our primary focus is to propose an ODE framework based on Rectified Flow that enables fast generation with high accuracy (as stated in the abstract, introduction, and preliminary sections), rather than introducing a new generative model family. To achieve this goal, we have proposed three techniques: constant acceleration, initial velocity conditioning (IVC), and reflow. Each of these techniques independently improves the estimation accuracy for single-step generation, as demonstrated in Table 2 of our paper.\n- **[CAF as an independent framework]**: To address the reviewer’s concern regarding the reliance on reflow, we conducted experiments where CAF was trained solely from real data (CIFAR-10) without the deterministic couplings generated from the pre-trained model (denoted as 1-CAF). For the quantitative results, please refer to section **2. [Training without pre-trained model]** in the \"**Author Rebuttal**\". The results show that 1-CAF effectively learns the real data distribution without reflow, even outperforming 1-Rectified Flow (RF). However, similar to 1-RF, 1-CAF loses its accuracy in extremely few-step regimes. We believe that improving generation quality in these regimes without the deterministic couplings is an interesting direction for future research.\n\n---\n**Q.3 [Missing discussion with AGM]**\n\n**Response to Q.3** Thank you for bringing out the related work that we have missed. We found that AGM elegantly formulates a new generative model using the acceleration term based on SOC theory. Since this concurrent work was published a few weeks before our submission, we unfortunately missed this important contribution. We will ensure that discussions of AGM and related works are included in the revised paper.\n\nThe main difference between AGM and CAF is that CAF assumes constant acceleration, whereas AGM predicts time-dependent acceleration. Our constant acceleration framework leads to a simpler sampling procedure (Eq. 11 in our paper) based on an analytically closed-form solution (Eq. 5 in our paper). This closed-form solution enables few-step (N<3) sampling with high accuracy when learned with deterministic couplings. To demonstrate this, we additionally compare the generation results of AGM and CAF on CIFAR-10. For the quantitative results, please refer to section **3. [Comparison with AGM]** in the \"**Author Rebuttal**\".  The result demonstrates that CAF outperforms AGM in terms of FID scores in an extremely few-step regime. Moreover, we provide qualitative comparisons between CAF and AGM in the attached PDF file, where CAF generates images with more high-frequency details than AGM for few-step sampling. These results highlight the distinct advantages of our framework over AGM, particularly in terms of single-step sampling accuracy.\n\n---\n**Q.4 [Missing discussion with AugBM]**\n\n**Response to Q.1**\nThank you for pointing out the related work that we have missed. We recognize that AugBM shares a similar motivation with our method in preserving couplings and addressing flow (bridge) crossing by conditioning auxiliary information to the network. However, while AugBM aims to improve coupling preservation specifically for image-to-image translation tasks, our main contribution lies in developing an ODE framework that enables fast sampling with high accuracy. We will include a detailed discussion of AugBM and related works in the revised paper to provide a more comprehensive comparison.\n\n---\n**Q.4 [Training setting of 2-Rectified Flow]** \n\n**Response to Q.4**\nYes, in our experiments, we used the same dataset generated by the pre-trained EDM for both CAF and RF.\n\n---\n**Q.5 [Separate training]**\n\n**Response to Q.5** \nThank you for the insightful suggestion. We investigated the possibility of training the velocity and acceleration models separately. In our experiments on the CIFAR-10 conditional setting, we found that training each network separately led to more stable training dynamics. This approach resulted in a slight improvement in the FID score, reducing it from 4.18 to 3.32.\n\n---\n**Q.6 [Computational cost]** \n\n**Response to Q.6** We agree with the reviewer that our framework requires additional learnable parameters for both acceleration and velocity prediction models. This can be a limitation, particularly for low-resource systems. One potential mitigation strategy could be training a single model to learn both velocity and acceleration by conditioning the model appropriately. We will include this discussion in the limitations section of the revised paper.\n\n---\n[1] Liu, Xingchao, et al. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation, ICLR 2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OGPRLIjQTn", "reviewer": "Reviewer_oUYA", "summary": "The paper presents an extension to rectified flow by Liu et al., 2023, a method based on optimal transport that can match samples from two distributions. This class of methods may have different uses but is often employed for the efficient generation of new samples and for morphing one sample to another. As such rectified flow and the extension proposed in this paper may be seen as finding efficient trajectories for use during generation by diffusion models. The method solves the transport flow in a simple Euler approximation of the ODE describing the flow. The paper identifies crossing of flow trajectories as a major obstacle in generating efficient paths in rectified flow and proposes constant acceleration flow (CAF). The paper proposes Initial Velocity Conditioning to produce the initialize the acceleration field in solving the ODE. Rectified flow uses constant velocity to solve the ODE using a reflow procedure to optimize the paths in a second step. Similarly, the proposed CAF also uses reflow to improve the initial paths. The paper reports experimental evaluation of the model with synthetic distributions and with CIFAR-10. The experiments demonstrate improved flow between distributions for synthetic data and improved FID score on CIFAR-10. The paper also reports an increase in straightness of trajectories.", "questions": "A discussion of the use of rectified flow on real-world tasks would enhance the justification for the proposed CAF. Can such examples be provided and discussed?\n\nAre there other options to provide the initial velocity field rather than using a pre-trained generative model? \n\nThe paper is generally well written. The following are minor typos:\nl. 20  a huge computation burden -> a large computational burden\nl. 128 thet -> the\nl. 176 Experiment -> Experiments or Experimental Evaluation \nl. 188 128-dimensional dimensions", "limitations": "The limitations section should be moved from the appendix into the main paper as the computational impact of CAF over rectified flow is essential.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The constant acceleration flow is novel to the best of my knowledge. \n\nThe paper describes the Initial Velocity Conditioning and related steps to make the approach work on CiFAR-10. The approach is suitably summarized in pseudo-code in two algorithms. \n\nThe synthetic experiments demonstrate the paper's motivation visually. \n\nThe ODE framework for the transport and the forward Euler solver makes the steps in the procedure easy to follow. The analogy makes it easy to see that constant velocity and constant acceleration lead to straight trajectories is the initial coupling of the samples x_0 and x_1 is correct.\n\nA comparison with many alternative methods is provided in Table 1. \n\nThe proposed CAF visually leads to improved quality and stability on CIFAR-10 over rectified flow.", "weaknesses": "The paper presents arguably a substantial improvement of results obtainable with rectified flow in terms of quality at the price of increased computation. The paper does not attempt any argument why this is significant, i.e., has the improved quality the potential to lead to wider adaptation of rectified flow type methods. The paper has no real-world use case to motivate the work. The claim that experiments with CIFAR-10 at 32x32 pixels is a real-world example seems far-fetched.    \n\nThe initialization of the method and its impact on the result is not well explored. The initial velocity calculation requires sample x_1 drawn from the target distribution. Both, the paper under consideration and rectified flow employ a pre-trained generative model to obtain that sample. The quality of initial samples may impact the results of CAF. The authors should provide evidence to the contrary or provide an experimental evaluation of the influence. \n\nThere is no theoretical justification why constant acceleration is superior to constant velocity flow generation. While the experimental evaluation appears clear, it remains unclear if this behavior is due to the tasks considered or holds more broadly. \n\nThe paper claims reflow as a contribution but it has been proposed by rectified flow. The same is true for the measurement of straightness of flow trajectory with Normalized Flow Straightness Score (NFSS) in Eqn 12. \n\nThe paper does not discuss how CIFAR-10 is used. One can speculate based on earlier work such as GLOW that unconditional generation and conditional generation may refer to use of class conditioning but a concise description of the experimental setup is required. \n\nThe use of a pre-trained generative model should also be taken into account in the required computational effort during training. It may be application-dependent if such a model is available or if it need to be trained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We highly appreciate the reviewer's effort for the detailed feedback and for spotting the typos. We will make corrections in the final version.\n\n**Q.1 [Significance of work & real-world application]**\n\n**Response to Q.1**\nIn response to the concern regarding the significance of our method, we would like to emphasize three key points:\n- **[Fast sampling]:** To address the slow inference of diffusion models, there have been efforts to reduce inference steps at the cost of additional training, such as CM and CTM. Our work shares this motivation, effectively addressing slow inference without compromising quality. We believe our work has substantial potential to make a generative model both fast and accurate, enhancing its applicability.\n- **[Efficient editing]:** Inversion has been an essential technique for real-world applications such as image/video editing [1,2]. However, current methods require 25-100 steps for accurate inversion, whereas our method can significantly reduce the inference time by a few-step inversion. We demonstrate this in two additional tasks: *reconstruction* and *box inpainting*. For the results, please refer to **4. [Inversion & Zero-shot image editing]** in the \"**Author Rebuttal**\" above.\n- **[ImageNet 64x64 results]:** For real-world examples, we provide additional results on ImageNet 64x64. These results demonstrate the broader applicability of our framework. For the quantitative results, please refer to **1. [ImageNet 64x64 results]** in the “**Author Rebuttal**” above.\n\n---\n**Q.2 [Impact of pre-trained model]**\n\n**Response to Q.2**\nAs the reviewer commented, the performance of CAF can be influenced by the quality of the pre-trained model. This phenomenon is common in other distillation methods, where the student's performance is bounded by the teacher. To address this, we have incorporated auxiliary adversarial loss with real data. This auxiliary loss can be considered as the divergence between the real data distribution and the learned distribution, helping the model surpass the performance of the pre-trained model. Our empirical results in the paper also support this, where CAF’s performance with distillation (FID 1.7) surpasses EDM (FID 2.01).\n\n---\n**Q.3 [Why constant acceleration is better than constant velocity?]**\n\n**Response to Q.3**\nCAF employs 2nd-order momentum by incorporating acceleration (2nd derivative of position w.r.t time). In contrast, constant velocity flow relies on 1st-order momentum, representing only the velocity. The inclusion of the 2nd-order term in CAF enables it to account for the time-varying nature of velocity, providing a more expressive approximation of the dynamics. This is analogous to higher-order terms in a Taylor series expansion, leading to lower numerical errors during the sampling phase. The superiority of higher-order schemes in reducing numerical errors is well-documented in numerical analysis literature (see LeVeque, 2002, Finite Volume Methods for Hyperbolic Problems).\n\n---\n**Q.4 [Contribution of reflow and NFSS]**\n\n**Response to Q.4**\nThank you for your feedback. We would like to clarify that we do *not* intend to claim credit for the reflow procedure or the measurement of straightness, both of which were introduced in the rectified flow. In our paper, we have provided appropriate references and discussions of these concepts in the abstract, introduction, and preliminary sections to acknowledge their original sources and their use in our work. Our primary contributions lie in the development of the CAF framework, which leverages these established techniques to achieve and measure improved performance. We will ensure that our references and discussions are appropriately highlighted to avoid any misunderstanding.\n\n---\n**Q.5 [Details of CIFAR10]** \n\n**Response to Q.5**\nWe will include the details in the revised paper. For training, we generated 12 million pairs from a pre-trained EDM model using 10 class labels. We do not use class labels for unconditional training. For class conditional training, the class labels are transformed into one-hot class embeddings and then element-wise added to the time-step embeddings. \n\n---\n**Q.6 [Additional training efforts for the pre-trained model]** \n\n**Response to Q.6**\nWe acknowledge that the use of a pre-trained model should be considered for the required computational effort during training. To clarify this, we have considered the following cases:\n- **[With pre-trained models]:** In scenarios where pre-trained models are readily available, leveraging them can significantly reduce the overall training time and computational resources required.\n- **[Without pre-trained model]:** In scenarios where pre-trained models are not available, our CAF can be trained from scratch to serve as a model to generate deterministic couplings. To show this, we conducted experiments where CAF was trained solely from real data without any pre-trained models. For the results, please refer to section 2. [Training without pre-trained model] in the \"Author Rebuttal\". This approach increases the computational efforts but effectively addresses the scenario where pre-trained models are not available.\n\nWe will include these discussions in the revised version of our paper to clarify the required computational effort during training.\n\n---\n**Q.7 [Real-world applications]**\n\n**Response to Q.7**\nPlease refer to our response to **Q.1 [Significance of work & real-world application].**\n\n---\n**Q.8 [Other options to provide the initial velocity]** \n\n**Response to Q.8**\nThank you for pointing out possible directions for future study. We believe that improving generation quality in few-step regimes without relying on reflow is a promising direction for future research.\n\n---\n[1] Mokady, Ron, et al, Null-text inversion for editing real images using guided diffusion models, CVPR2023 \\\n[2] Huberman-Spiegelglas, Inbar, et al, An edit friendly DDPM noise space: Inversion and manipulations, CVPR2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely appreciate all reviewers' detailed feedback. Here, we present additional quantitative results to address their concerns.

---
**1. [ImageNet 64x64 Results]**

In response to the concerns about the limited evaluation of our method, we provide additional quantitative results (FID, Inception Score, and Recall) on ImageNet 64x64. The table below summarizes the results:
| Model  |  N  |  FID $\downarrow$  |   IS $\uparrow$  | Recall $\uparrow$ |
|----------------------------------------------------|:---:|:-----:|:-----:|:------:|
| _GAN Models_     |     |       |       |        |
| BiGGAN-deep    |  1  |  4.06 |   -   |  0.48  |
| StyleGAN-XL    |  1  |  2.09 | **82.35** |  0.52  |
|       |     |       |       |        |
| _Diffusion Model / Consistency Model_    |     |       |       |        |
| ADM     | 250 |  2.07 |   -   |  **0.63**  |
| EDM   |  79 |  2.44 | 48.88 |  0.67  |
| iCT-deep  |  2  |  2.77 | - |  0.62  |
| iCT-deep  |  1  |  3.25 | - |  **0.63**  |
|                                                    |     |       |       |        |
| _Diffusion Models - Rectified flow_                |     |       |       |        |
| **CAF (Ours)**   |  1  |  9.29 | 42.73 |  0.627 |
|                                                    |     |       |       |        |
| _Diffusion Models - Distillation_                  |     |       |       |        |
| CD  |  1  |  6.2  | 40.08 |  **0.63**  |
| Diff-Instruct  |  1  |  5.77  | - |  -  |
| CTM  |  2  |  1.73 | 64.29 |  0.57  |
| CTM  |  1  |  1.92 | 70.38 |  0.57  |
|
| _Diffusion Models - Rectified Flow + Distillation_ |     |       |       |        |
| **CAF (+distill + GAN) (Ours)**    |  1  |  **1.69** | 62.03 |  0.621 |

These results demonstrate that our method achieves comparable performance compared to recent state-of-the-art models, highlighting the generalization capability of our approach to large-scale datasets. Additionally, we have included the ImageNet generation results from our trained model with a single step in **Figure 4** of the attached PDF file.

---
**2. [Training without pre-trained model]** 

To address the reviewer’s concern regarding the reliance on a pre-trained model, we report additional results where CAF was trained solely from real data (CIFAR-10) without the deterministic couplings generated by the pre-trained model (denoted as 1-CAF).  
| Model | N |  FID $\downarrow$ |
|-------|:---:|:----:|
| 1-RF  |  1  |   335   |
|   **1-CAF**  |  1 |  328 |
| 1-RF |  70  |   4.7   |
|  **1-CAF**  |  70 | **3.53** |

The results show that 1-CAF effectively learns the real data distribution without the deterministic couplings, even outperforming 1-Rectified Flow (RF). However, similar to 1-RF, we lose its accuracy in extremely few-step regimes. We believe that improving quality in these regimes without the deterministic couplings is an interesting direction for future research.

---
**3. [Comparison with AGM]**

As the reviewer rYLJ requested, we additionally compare the generation results of Acceleration Generative Model (AGM) [1] and CAF in the table below.
| Dataset        | Model      |  N |  FID $\downarrow$  |
|----------------|------------|:--:|:-----:|
| CIFAR-10       | AGM-ODE    |  50 | 2.46 |
|    | AGM-ODE |  5 |  11.88 |
|    | **CAF (Ours)** |  1 |  **1.7** |
| ImageNet 64x64 | AGM-ODE    | 30 | 10.07 |
|    | AGM-ODE    | 20 | 10.55 |
|    | **CAF (Ours)** |  1 |  **1.69** |

The results demonstrate that CAF outperforms AGM in terms of FID scores in an extremely few-step regime. Moreover, we provide qualitative comparisons between CAF and AGM in **Figure 3** of the attached PDF file, where CAF generates images with more high-frequency details than AGM for few-step sampling. These results highlight the distinct advantages of CAF over AGM, particularly in terms of few-step sampling accuracy. For a detailed discussion, please refer to our response to the revierwer rYLJ **Q.3 [Missing discussion with AGM]**.

---
**4. [Inversion & Zero-shot image editing]**

As the reviewer oUYA requested, we demonstrate our CAF's capability on real-world applications by conducting zero-shot tasks on CIFAR-10 test datasets (*reconstruction* and *box inpainting*). Since our framework is based on ODEs, we can solve our CAF ODE in reverse to perform inversion like DDIM [2]. For reconstruction, we followed the procedure as DDIM. For box inpainting, we inject conditional information (non-masked image region) into the iterative inversion and reconstruction procedure. As demonstrated in the table below, we achieve better reconstruction quality and zero-shot inpainting capability even in a few steps compared to the baselines. This improvement is due to the superior coupling preservation capability of CAF (in Table 4 of our paper). Moreover, we provide qualitative results in **Figure 1 and 2** of the attached PDF file that align with the quantitative results.

|  Reconstruction | N |  PSNR $\uparrow$ | LPIPS $\downarrow$ |
|-------------|:-:|:-----:|:-----:|
| CM  | - |  N/A  |  N/A  |
| CTM  | - |  N/A  |  N/A  |
| EDM  | 4 | 13.85 | 0.447 |
| 2-RF  | 1 | 29.33 | 0.204 |
| **CAF (Ours)** | 1 | **30.27** | **0.171** |

| Box inpainting | N |  FID $\downarrow$  |
|----------------|:---:|:-----:|
| CM  |  18 | 13.16 |
| 2-RF  |  10  | 16.41 |
| **CAF (Ours)** |  10  |  **9.79** |

These results demonstrate that our method can significantly reduce the inference time required for recent methods that utilize inversion for various real-world applications such as image/video editing.

---
[1] Chen, Tianrong, et al, Generative modeling with phase stochastic bridges, ICLR 2024 \
[2] Song, Jiaming, Chenlin Meng, and Stefano Ermon, Denoising diffusion implicit models, ICLR 2021

Meta-review: This paper received two weak acceptances and one borderline rejection. Reviewer rYLJ generally agreed with the two positive reviews but raised a key concern regarding the concurrent work AGM [1], which might have a similar approach to the acceleration generation modeling framework. In the rebuttal, the authors provided a comparison with AGM, highlighting differences in the acceleration strategy and the closed-form solution. The comparison results also further verified the effectiveness. These points effectively justify the novelty of this work, especially since constant acceleration is a central claim in the title. Consequently, the AC agreed that the work retains its novelty despite the concurrent work on arXiv earlier, which had not been fully published before the NeurIPS submission deadline.

Additionally, the AC considered the generalization of this approach to other datasets, as noted by Reviewer oUYA. The rebuttal included further experiments on ImageNet 64x64, which strengthened the justification and made the work more convincing.

Based on the novelty, solid experiments presented in the paper, and additional evaluations in the rebuttal, the AC concurs with the majority of reviewers to accept the paper. For the camera-ready version, it is suggested to add a subsection discussing AGM and include this work in the related works section. Furthermore, it is encouraged to add generalization results on other datasets in the final version.
Decision: Accept (poster)