Constant Acceleration Flow

Authors: dogyun  park
sojin  lee
sihyeon  kim
taehoon  lee
youngjoon  hong
hyunwoo j kim
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 13:02:19

Abstract: Rectified flow and reflow procedures have significantly advanced fast generation by progressively straightening ordinary differential equation (ODE) flows under the assumption that image and noise pairs, known as coupling, can be approximated by straight trajectories with constant velocity. However, we observe that the constant velocity modeling and reflow procedures have limitations in accurately learning to couple with flow crossing, leading to suboptimal few-step generation. To overcome the limitations, we introduce the Constant Acceleration Flow (CAF), a novel framework based on a simple constant acceleration equation. Additionally, we propose two techniques to improve estimation accuracy: initial velocity conditioning for the acceleration model and a reflow process for the initial velocity. Our comparative studies show that CAF not only outperforms rectified flow with reflow procedures in terms of speed and accuracy but also demonstrates substantial improvements in preserving coupling for fast generation.

Main paper: 
 introduction: 
 Diffusion models [1, 2] learn the probability flow between a target data distribution and a simple Gaussian distribution through an iterative process. Starting from Gaussian noise, they gradually denoise to approximate the target distribution via a series of learned local transformations. Due to their superior generative capabilities compared to other models such as GANs and VAEs, diffusion models have become the go-to choice for high-quality image generation. However, their multi-step generation process entails slow generation and imposes a significant computational burden. To address this issue, two main approaches have been proposed: distillation models [3, 4, 5, 6, 7, 8, 9] and methods that simplify the flow trajectories [10, 11, 12, 13, 14] to achieve fewer-step generation. An example of the latter is rectified flow [10, 11, 13] , which focuses on straightening ordinary differential equation (ODE) trajectories. Through repeated applications of the rectification process, called reflow, the trajectories become progressively straighter by addressing the flow crossing problem. Straighter flows reduce discretization errors, enabling fewer steps in the numerical solution and, thus, faster generation.Rectified flow [10, 13] defines the straight ODE flow over time t with a drift force v, where each sample x t transforms from x 0 ∼ π 0 to x 1 ∼ π 1 under a constant velocity v = x 1 -x 0 . It Figure 1 : Initial Velocity Conditioning (IVC). We illustrate the importance of IVC to address the flow crossing problem, which hinders the learning of straight ODE trajectories during training. In Fig. 1a , Rectified flow suffers from approximation errors at the overlapping point x t (where x 1 t = x 2 t ), resulting in curved sampling trajectories due to flow crossing. Conversely, Fig. 1b demonstrates that CAF, utilizing IVC, successfully estimates ground-truth trajectories by minimizing the ambiguity at x t .approximates the underlying velocity v with a neural network v θ . Then, it iteratively applies the reflow process to avoid flow crossing by rewiring the flow and building deterministic data coupling. However, constant velocity modeling may limit the expressiveness needed for approximating complex couplings between π 0 and π 1 . This results in sampling trajectories that fail to converge optimally to the target distribution. Moreover, the interpolation paths after the reflow may still intersect-a phenomenon known as flow crossing-which leads to curved rectified flows because the model estimates different targets for the same input. As illustrated in Fig. 1a , instead of following the intended path from x 1 0 to x 1 1 , a sampling trajectory from Rectified flow erroneously diverts towards x 2 1 due to the flow crossing. Such flow crossing can make the accurate learning of straight ODE trajectories more challenging.In this paper, we introduce the Constant Acceleration Flow (CAF), a novel ODE framework based on a constant acceleration equation, as outlined in (4) . Our CAF generalizes Rectified flow by introducing acceleration as an additional learnable variable. This constant acceleration modeling offers the ability to control flow characteristics by manipulating the acceleration magnitude and enables a direct closed-form solution of the ODE, supporting precise and efficient sampling in just a few steps. Additionally, we propose two strategies to address the flow crossing problem. The first one is initial velocity conditioning (IVC) for the acceleration model, and the second one is to employ reflow to enhance the learning of initial velocity. Fig. 1b presents that CAF, with the proposed strategies, can accurately predict the ground-truth path from x 1 0 to x 1 1 , even when flow crossing occurs. Through extensive experiments, from toy datasets to real-world image generation on CIFAR-10 [15] and ImageNet 64×64, we demonstrate that our CAF exhibits superior performance over Rectified flow and state-of-the-art baselines. Notably, CAF achieves superior Fréchet Inception Distance (FID) scores on CIFAR-10 and ImageNet 64×64 in conditional settings, recording FIDs of 1.39 and 1.69, respectively, thereby surpassing recent strong methods. Moreover, we show that CAF provides more accurate flow estimation than Rectified flow by assessing the 'straightness' and 'coupling preservation' of the learned ODE flow. CAF is also capable of few-step inversion, making it effective for real-world applications such as box inpainting.To summarize, our contributions are as follows:• We propose Constant Acceleration Flow (CAF), a novel ODE framework that integrates acceleration as a controllable variable, enhancing the precision of ODE flow estimation compared to the constant velocity framework.• We propose two strategies to address the flow crossing problem: initial velocity conditioning for the acceleration model and a reflow procedure to improve initial velocity learning. These strategies ensure a more accurate trajectory estimation even in the presence of flow crossings.• Through extensive experiments on synthetic and real datasets, CAF demonstrates remarkable performance, especially achieving the superior FID on CIFAR-10 and ImageNet 64×64 over strong baselines. We also demonstrate that CAF learns more accurate flow than Rectified flow by assessing the straightness, coupling preservation, and inversion.  
related work: 
 Generative models. Learning generative models involves finding a nonlinear transformation between two distributions, typically denoted as π 0 and π 1 , where π 0 is a simple distribution like a Gaussian, and π 1 is the complex data distribution. Various approaches have been developed to achieve this transformation. For example, variational autoencoders (VAE) [16, 17] optimize the Evidence Lower Bound (ELBO) to learn a nonlinear mapping from the latent space distribution π 0 to the data distribution π 1 . Normalizing flows [18, 19, 20] construct a series of invertible and differentiable mappings to transform π 0 into π 1 . Similarly, GANs [21, 22, 23, 24, 25] earn a generator that transforms π 0 into π 1 through an adversarial process involving a discriminator. These models typically perform a one-step generation from π 0 to π 1 . In contrast, diffusion models [2, 26, 27, 28, 29, 30] propose learning the probability flow between the two distributions through an iterative process. This iterative process ensures stability and precision, as the model incrementally learns to reverse a diffusion process that adds noise to data. Diffusion models have demonstrated superior performance across various domains, including images [12, 31, 32, 33] , 3D [34, 35, 36, 37] , and video [38, 39, 40] .Few-step diffusion models Addressing the slow generation speed of diffusion models has become a major focus in recent research: Distillation methods [3, 4, 5, 6, 7, 8, 9] seek to optimize the inference steps of pre-trained diffusion models by amortizing the integration of ODE flow. Consistency models [6, 7, 8] train a model to map any point on the pre-trained diffusion trajectory back to the data distribution, enabling fast generation. Rectified flow [10, 11, 13] is another direction, which focuses on straightening ODE trajectories under a constant velocity field. By straightening the flow and reducing path complexity, it allows for fast generation through efficient and accurate numerical solutions with fewer Euler steps. Recent methods such as AGM [41] also introduce acceleration modeling based on Stochastic Optimal Control (SOC) theory instead of relying solely on velocity. However, AGM predicts time-varying acceleration, which still requires multiple iterative steps to solve the differential equations. In contrast, our proposed CAF ODE assumes that the acceleration term is constant with respect to time. Therefore, there is no need to iteratively solve complex timedependent differential equations. This simplification allows for a direct closed-form solution that supports efficient and accurate sampling in just a few steps.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: We have discussed the limitations of our work in the supplementary material.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminary: 
 Rectified flow [10, 13] is an ordinary differential equation-based framework for learning a mapping between two distributions π 0 and π 1 . Typically, in image generation, π 0 is a simple tractable distribution, e.g., the standard normal distribution, defined in the latent space and π 1 is the image distribution. Given empirical observations of x 0 ∼ π 0 and x 1 ∼ π 1 over time t ∈ [0, 1], a flow is defined as where x t = I(x 0 , x 1 , t) is a time-differentiable interpolation between x 0 and x 1 , and v : R d × [0, 1] → R d is a velocity field defined on data-time domain. Rectified flow learns the velocity field v with a neural network v θ by minimizing the following mean square objective:EQUATIONEQUATION)where γ represents a coupling of (π 0 , π 1 ) and p(t) is a time distribution defined on [0, 1]. The choice of interpolation I leads to various algorithms, such as Rectified flow [10] , ADM [30] , EDM [29] ,and LDM [42] . Specifically, Rectified flow proposes a simple linear interpolation between x 0 and x 1 as x t = (1 -t)x 0 + tx 1 , which induces the velocity field v in the direction of (x 1 -x 0 ), i.e., v(x t , t) = x 1 -x 0 . This means the Rectified flow transports x 0 to x 1 along a straight trajectory with a constant velocity. After training v θ , we can generate a sample x 1 using off-the-shelf ODE solvers Φ, such as the Euler method:EQUATIONwhere ∆t = 1 N and N is the total number of steps. To achieve faster generation with fewer steps without sacrificing accuracy, it is crucial to learn a straight ODE flow. Straight ODE flow minimize numerical errors encountered by the ODE solver.Reflow and flow crossing. The trajectories of interpolants x t may intersect-a phenomenon known as flow crossing-due to stochastic coupling between π 0 and π 1 (e.g., random pairing of x 0 and x 1 ). These intersections introduce approximation errors in the neural network, leading to curved sampling trajectories [10] . Our toy experiment, illustrated in Fig. 1a , clearly demonstrates this issue: the simulated sampling trajectories become curved due to flow crossing, rendering one-step simulation inaccurate. To address this problem, Rectified flow [10] introduces a reflow procedure. This procedure iteratively straightens the trajectories by reconstructing a more deterministic and direct pairing of x 0 and x 1 without altering the marginal distributions. Specifically, the reflow procedure involves generating a new coupling γ of (x 0 , x 1 = Φ(x 0 ; v k θ )) using a pre-trained Rectified flow model v k θ , where k denotes the iteration of the reflow procedure, and Φ(x 0 ; v k θ ) = x 0 + 1 0 v k θ (x t , t)dt. By iteratively refining the coupling and the velocity field, the reflow procedure reduces flow crossing, resulting in straighter trajectories and improved accuracy in fewer steps.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes]• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
method: 
 We aim to develop a generative model based on the ODE framework that enables faster generation without compromising quality. To achieve this, we propose a novel approach called Constant Acceleration Flow (CAF). Specifically, CAF formulates an ODE trajectory that transports x t with a constant acceleration, offering a more expressive and precise estimation of the ODE flow compared to constant velocity models. Additionally, we propose two novel techniques that address the problem of flow crossing: 1) initial velocity conditioning and 2) reflow procedure for learning initial velocity. The overall training pipeline is presented in Alg. 1.We propose a novel ODE framework based on the constant acceleration equation, which is driven by the empirical observations x 0 ∼ π 0 and x 1 ∼ π 1 over time t ∈ [0, 1] as:EQUATIONwhere v : R d × [0] → R d is the initial velocity field and a : R d × [0, 1] → R d is the acceleration field. We abbreviate time variable t for notation simplicity, i.e., v(x0 , 0) = v(x 0 ), a(x t , t) = a(x t ).By integrating both sides of ( 4) with respect to t and assuming a constant acceleration field, i.e., a(x t1 ) = a(x t2 ), ∀t 1 , t 2 ∈ [0, 1], we derive the following equation:EQUATION)Given the initial velocity field v, the acceleration field a can be derived asEQUATIONby setting t = 1 and the constant acceleration assumption. Then, we propose a time-differentiable interpolation I as:EQUATIONby substituting ( 6) to (5) . Using this result, we can easily simulate an intermediate sample x t on our CAF ODE trajectory.Learning initial velocity field. Selecting an appropriate initial velocity field is crucial, as different initial velocities lead to distinct flow dynamics. Here, we define the initial velocity field as a scaled displacement vector between x 1 and x 0 :EQUATIONwhere h ∈ R is a hyperparameter that adjusts the scale of the initial velocity. This configuration enables straight ODE trajectories between distributions π 0 and π 1 , similar to those in Rectified flow. However, varying h changes the flow characteristics: 1) h = 1 simulates constant velocity flows, 2) h < 1 leads to a model with a positive acceleration, and 3) h > 1 results in a negative acceleration, as illustrated in Fig. 3 . Empirically, we observe that the negative acceleration model is more effective for image sampling, possibly due to its ability to finely tune step sizes near data distribution.The initial velocity field is learned using a neural network v θ , which is optimized by minimizing the distance metric d(•, •) between the target and estimated velocities asEQUATIONwhere p(t) is a time distribution defined on [0, 1]. Note that our velocity model learns target initial velocity defined at t = 0. This differs from Rectified flow, which learns target velocity field defined over t ∈ [0, 1].Learning acceleration field. Under the assumption of constant acceleration, the acceleration field is derived from (6) asEQUATIONWe learn the acceleration field using a neural network a ϕ by minimizing the distance metric d(•, •) as:EQUATIONIn Sec. C, we theoretically show that CAF ODE preserves the marginal data distribution.Require: deterministic coupling γ, initial velocity scale h, v θ , a ϕ . 1: while not converge do 2:x 0 , x 1 ∼ γ, t ∼ Unif([0, 1]) 3: v(x 0 ) = h(x 1 -x 0 ) ▷ Target initial velocity 4:x t = I(x 0 , x 1 , t, v(x 0 )) ▷ Eq. ( 7)5:L vel = d(v(x 0 ), v θ (x t )) 6:θ ← θ -∇L vel ▷ update θ using SGD with gradient 7: end while 8: while not converge do 9:x 0 , x 1 ∼ γ, t ∼ Unif([0, 1]), vθ = v θ (x 0 ) 10: a(x t ) = 2(x 1 -x 0 ) -2v θ ▷ Target acceleration 11:x t = I(x 0 , x 1 , t, vθ ) ▷ Eq. ( 7)12:L acc = d(sg[a(x t )], a ϕ (x t , vθ )) 13:ϕ ← ϕ -∇L acc ▷ update ϕ using SGD with gradient 14: end while 15: return v θ , a ϕRectified flow addresses the issue of flow crossing by a reflow procedure. However, even after the procedure, trajectories may still intersect each other. Such intersections hinder learning straight ODE trajectories, as demonstrated in Fig. 1a . Similarly, our acceleration model also encounters the flow crossing problem. This leads to inaccurate estimation, as the model struggles to predict estimation on these intersections correctly. To further address the flow crossing, we propose two techniques.We propose conditioning the estimated initial velocity vθ = v(x 0 ) as the input of the acceleration model, i.e., a ϕ (x t , vθ ). This approach provides the acceleration model with auxiliary information on the flow direction, enhancing its capability to distinguish correct estimations and mitigate ambiguity at the intersections of trajectories, as illustrated in Fig. 1 . Our IVC circumvents the non-intersecting condition required in Rectified flow (see Theorem 3.6 in [10] ), which is a key assumption for achieving a straight coupling γ. By reducing the ambiguity arising from intersections, CAF can learn straight trajectories with less constrained couplings, which is quantitatively assessed in Tab. 4.To incorporate IVC into learning the acceleration model, we reformulate (11) as:EQUATIONwhere sg[•] indicates stop-gradient operation. Since our velocity model learns to predict the initial velocity (see ( 9)), we ensure that the model can handle both forward and reverse CAF ODEs, which start from x 0 and x 1 , respectively. Thus, our acceleration model can generalize across different flow directions, enabling inversion as demonstrated in Sec. B.2.Reflow for initial velocity. It is also important to improve the accuracy of the initial velocity model. Following [10] , we address the inaccuracy caused by stochastic pairing of x 0 and x 1 by employing a pre-trained generative model ψ, which constructs a more deterministic coupling γ of x 0 and x 1 . We subsequently use this new coupling γ to train the initial velocity and acceleration models.After training the initial velocity and acceleration models, we generate samples using the CAF ODE introduced in (4). The discrete sampling process is given by:EQUATION)where N is the total number of steps, ∆t = 1 N , t = i•∆t, and t ′ = (2i+1) 2•∆t where i ∈ {0, ..., N -1} (See Alg. 2). We adopt t ′ since it empirically improves accuracy, especially in the small N regime. Notably, when N = 1 (one-step generation), t ′ simplifies to 1 2 , leading to the closed-form solution in (5) . See Alg. 3 for inversion algorithm.Require: velocity model v θ , acceleration model a ϕ , sampling steps N , π 0 .1:x 0 ∼ π 0 2: vθ ← v θ (x 0 ) 3: for i = 0 to N -1 do 4: t ← i N 5: t ′ ← 2i+1 2N 6:âϕ ← a ϕ (x t , v θ )7:x t+ 1 N ← x t + 1 N vθ + t ′ N âϕ 8: end for 9: return x 1 
experiment: 
 We evaluate the proposed Constant Acceleration Flow (CAF) across various scenarios, including both synthetic and real-world datasets. In Sec. 5.1, our investigation begins with a simple twodimensional synthetic dataset, where we compare the performance of Rectified flow and CAF to clearly demonstrate the effectiveness of our model. Next, we extend our experiments to real-world image datasets, specifically CIFAR-10 (32×32) and ImageNet (64×64), in Sec. 5.2. These experiments highlight CAF's ability to generate high-quality images with a single sampling step. Furthermore, we conduct an in-depth analysis of CAF through evaluations of coupling preservation, straightness, inversion tasks, and an ablation study in Sec. 5.3.We demonstrate the advantages of the Constant Acceleration Flow (CAF) over the constant velocity flow model, Rectified Flow [10] , through synthetic experiments. For the neural networks, we use multilayer perceptrons (MLPs) with five hidden layers and 128 units per layer. Initially, we train 1-Rectified flow on 2D synthetic data to establish a deterministic coupling. We then train both CAF and 2-Rectified flow. For CAF, we incorporate the initial velocity into the acceleration model by concatenating it with the input, ensuring that the model capacities of both CAF and 2-Rectified flow remain comparable. We set d as l 2 distance. Fig. 2 presents samples generated from CAF in one step and from 2-Rectified flow in two steps. Our CAF more accurately approximates the target distribution π 1 than 2-Rectified flow. In particular, CAF with h = 2 (negative acceleration) learns the most accurate distribution. In contrast, 2-Rectified flow frequently generates samples that significantly deviate from π 1 , indicating its difficulty in accurately estimating straight ODE trajectories. This experiment shows that reflowing alone may not overcome the flow crossing problem, leading to poor estimations, whereas our proposed acceleration modeling and IVC effectively address this issue. Moreover, Fig. 3 shows sampling trajectories from CAF trained with different hyperparameters h. It clearly demonstrates that h controls the flow dynamics as we intended: h > 1 indicates negative acceleration, h = 1 represents constant velocity, and h < 1 corresponds to positive acceleration flows. Additional synthetic examples are provided in Fig. 6 .To further validate the effectiveness of our approach, we train CAF on real-world image datasets, specifically CIFAR-10 at 32×32 resolution and ImageNet at 64×64 resolution. To create a deterministic coupling γ, we utilize the pre-trained EDM models [29] and adopt the U-Net architecture of ADM [30] for the initial velocity and acceleration models. In the acceleration model, we double the input dimension of first layer to concatenate the initial velocity to the input x t of the acceleration model, which marginally increases the total number of parameters. We set h = 1.5 and d as LPIPS-Huber loss [43] for all real-data experiments.Baselines and evaluation. We evaluate state-of-the-art diffusion models [1, 2, 7, 28, 29] , GANs [22, 23, 24] , and few-step generation approaches [6, 7] . We primarily assess the image generation quality of our method using the Fréchet Inception Distance (FID) [50] and Inception Score (IS) [51] . Additionally, we evaluate diversity using the recall metric following [6, 7, 10] . Distillation. Distilling a few-step student model from a pre-trained teacher model has recently become essential for high-quality few-step generation [6, 7, 10, 11] . InstaFlow [11] has observed that learning straighter trajectories and achieving good coupling significantly enhance distillation performance. Moreover, CTM [7] and DMD [44] incorporate an adversarial loss as an auxiliary loss to facilitate the training of the student model. We empirically found that incorporating the adversarial loss alone was sufficient to achieve superior performance for one-step sampling without introducing instability. For training details, please refer to Sec. A. CIFAR-10. We present the experimental results on CIFAR-10 in Tab. 1. Our base unconditional CAF model (4.81 FID, N = 1) significantly improves the FID compared to recent state-of-the-art diffusion models (without distillation), including DDIM [26] (13.36 FID, N = 10), EDM (37.75 FID, N = 5), and 2-Rectified flow (7.89 FID, N = 2) in a few-step generation (e.g., N < 10). We retrained 2-Rectified flow using the official codes of [10] , achieving a slightly better performance than the officially reported performance (12.21 FID) for one-step generation [10] . CAF's remarkable 3.08 FID improvement over 2-Rectified flow (N = 2) highlights the effectiveness of acceleration modeling in a fast generation. Our approach is also effective in class-conditional generation, where the base CAF model (2.68 FID, N = 1) shows a significant FID improvement over EDM (35.54 FID, N = 5) and 2-Rectified flow (3.74 FID, N = 2). Additionally, after adversarial training, CAF achieves a superior FID of 1.48 for unconditional generation and 1.39 for conditional generation with N = 1. Lastly, we qualitatively compare the 2-Rectified flow and our CAF in Fig. 4 , where CAF generates more vivid samples with intricate details than 2-Rectified flow.ImageNet. We extend our evaluation to the ImageNet dataset at 64×64 resolution to demonstrate the scalability and effectiveness of our CAF model on more complex and higher-resolution images. Similar to the results on CIFAR-10, our base conditional CAF model significantly improves the FID compared to recent state-of-the-art diffusion models (without distillation) in the small N regime (e.g., N < 10). Specifically, CAF (6.52 FID, N = 1) outperforms models such as DPM-solver [48] (7.93 FID, N = 10), CT [6] (11.1 FID, N = 2), and EDM [29] (55.3 FID, N = 5). This validates that the superior performance of CAF can be effectively generalized to complex and large-scale datasets. Additionally, after adversarial training, CAF outperforms or is competitive with state-of-the-art distillation baselines in one-step generation. Notably, CAF achieves the best FID performance of 1.69, surpassing strong baselines. We also demonstrate one-step qualitative results in Fig. 14 . Coupling preservation. We evaluate how accurately CAF and Rectified flow approximate the deterministic coupling obtained from pre-trained models via a reflow procedure. To analyze this, we first conduct synthetic experiments where the interpolant paths I are crossed, as illustrated in Fig. 5a . Due to the flow crossing, the sampling trajectory of Rectified flow fails to preserve the ground-truth coupling (interpolation path I), leading to a curved sampling trajectory. In contrast, our CAF learns the straight interpolation paths by incorporating acceleration, demonstrating superior coupling preservation ability.Moreover, we evaluate the coupling preservation ability on real data from CIFAR-10. We randomly sample 1K training pairs (x 0 , x 1 ) from the deterministic coupling γ and measure the similarity between x 1 and x1 , where x1 is a generated sample from x 0 . In other words, we measure the distance between a ground truth image and a generated image corresponding to the same noise. If the coupling is well-preserved, the distance should be small. We use PSNR and LPIPS [52] as distance measures.The result in Tab. 3 demonstrates that CAF better preserves coupling. In terms of PSNR, CAF outperforms Rectified flow by 3.37. This is consistent with the qualitative result in Fig. 5b , where x1 from CAF resembles more to x 1 (ground truth) than x1 from Rectified flow.Flow straightness. To evaluate the straightness of learned trajectories, we introduce the Normalized Flow Straightness Score (NFSS). Similar to previous works [10, 11] , we measure flow straightness S by the L 2 distance between the normalized displacement vector (x 0 -x 1 ) and the normalized velocity vector ẋt as below:EQUATION)Here, a smaller value of S indicates a straighter trajectory. We compare S between CAF and Rectified flow using synthetic and real-world datasets, as presented in Tab. 4. For Rectified flow, we use ẋt = v θ (x t ), while for CAF, we use ẋt = v θ (x 0 ) + a ϕ (x t )t. The results show that CAF outperforms Rectified flow in flow straightness. generates target images from the given noise (e.g., a car from the car noise), while RF often fails (e.g., a frog from the car noise). LPIPS [52] values are in parentheses.Inversion We further demonstrate CAF's capability in real-world applications by conducting zeroshot tasks such as reconstruction and box inpainting using inversion. We provide implemenetation details and algorithms in Sec. B.2. As shown in the Tab. 6 and 7, our method achieves lower reconstruction errors (CAF: 46.68 PSNR vs. RF: 33.34 PSNR) and better zero-shot inpainting capabilities even with fewer steps compared to baselines. These improvements are attributed to CAF's superior coupling preservation capability. Moreover, we present qualitative comparisons between CAF and the baselines in Fig. 12 and 13, which further validates the quantitative results.Ablation study. We conduct an ablation study to evaluate the effectiveness of components in our framework under the one-step generation setting (N = 1). We examine the improvements achieved by 1) constant acceleration modeling, 2) initial velocity (v 0 ) conditioning, and 3) the reflow procedure for v 0 . The configurations and results are outlined in Tab. 5. Specifically, A and B correspond to 1-Rectified flow and 2-Rectified flow, respectively. Configurations C to F represent our CAF frameworks, with C being our CAF without IVC. By comparing A,B,C, and F, we demonstrate that all three components in our framework substantially improve the performance. In addition, we analyze the final model across various acceleration scales controlled by h. The performance difference between D and F is relatively small, indicating that our framework is robust to the choice of hyperparameters. Empirically, we observe that configuration F, i.e., CAF (h = 1.5) with negative acceleration, achieves the best FID of 2.68. Notably, our CAF without v 0 conditioning, still outperforms rectified flow (configuration B) by 3.06 FID. This highlights the critical role of constant acceleration modeling in enhancing the quality of few-step generation. Also, we verify the significance of reflowing by comparing configurations A and B, which achieve 378 FID and 6.88 FID, respectively. 
conclusion: 
 In this paper, we have introduced the Constant Acceleration Flow (CAF) framework, which enhances precise ODE trajectory estimation by incorporating a controllable acceleration variable into the ODE framework. To address the flow crossing problem, we proposed two strategies: initial velocity conditioning and a reflow procedure. Our experiments on toy datasets, real-world dataset demonstrate CAF's capabilities and scalability, achieving state-of-the-art FID scores. Furthermore, we conducted extensive ablation studies and analyses-including assessments of flow straightness, coupling preservation, and real-world applications-to validate and deepen our understanding of the effectiveness of our proposed components in learning accurate ODE trajectories. We believe that CAF offers a promising direction for efficient and accurate generative modeling, and we look forward to exploring its applications in more diverse settings such as 3D and video. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: We have provided details in the supplementary materials. Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: We have followed the code of ethics. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: We have discussed the broader impact in supplementary materials. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to 
 
 Appendix: We utilize the pre-trained EDM model [29] to build the deterministic coupling γ for training our models. To construct deterministic couplings for CIFAR-10 and ImageNet, we select N = 18 and N = 40, respectively, using deterministic sampling following the protocol in [29] . For CIFAR-10 and ImageNet, we generate 1M and 3M pairs, respectively. We use the batch size of 2048 and train for 700K/700K iterations on ImageNet. For CIFAR-10, we use the batch size of 512 and train for 500K/500K iterations. For all experiments, we use AdamW [53] optimizer with a learning rate of 0.0001 and apply an Exponential Moving Average (EMA) with a 0.999 decay rate. For training acceleration model, we initialize it with initial velocity model for faster convergence.For adversarial training, we employ adversarial loss L gan using real data x 1,real from [24] :EQUATIONwhere d η is a discriminator andx1 = x 0 + v θ (x 0 ) + 1 2 a ϕ (x 0 , v θ (x 0 )).In the end, we use the following combined loss to update the acceleration model:EQUATIONwhere L acc corresponds to (12) and λ are weight hyperparameters. Following [42, 54] , we employ adaptive weighting as λ gan = ∥∇ ϕ l Lacc(ϕ)∥ ∥∇ ϕ l Lgan(ϕ,η)∥ , where ϕ l is the last layer of the acceleration model. Without L acc , we found the training unstable and frequently exhibit mode collapse issue, which is a common problem with adversarial training. We follow the training configuration from StyleGAN-XL [24] . We bilinearly upscale the image to 224×224 resolution and use EfficientNet [55] and DeiTbase [56] for extracting features. During the adversarial training, we only optimize the acceleration model and discriminator with the learning rate of 2e-5 and 1e-3, respectively. We keep the parameters of the initial velocity model fixed for stable training. The total training takes about 21 days with 8 NVIDIA A100 GPUs for ImageNet, and takes 10 days 8 NVIDIA RTX3090 GPUs for CIFAR-10.2D toy dataset. In Fig. 6 , we provide additional generation results and sampling trajectories on various 2D synthetic datasets with N = 1, demonstrating the effectiveness of our approach for fast generation. Fig. 7 provides additional examples of coupling preservation on 2-RF and CAF.Real-world dataset. In Fig. 8 and 9 , we show additional generation results from our base CAF model on CIFAR-10 with N = 1, 10, and 50. In Fig. 10 , we compare the generation result between 2-RF and CAF distilled versions. Fig. 11 shows sampling results from our base CAF models with different hyperparameters h. Lastly, Fig. 14 shows the generation results on ImageNet with N = 1.Inversion techniques are essential for real-world applications such as image and video editing [57, 58] . However, existing methods typically require 25-100 steps for accurate inversion, which can be computationally intensive. In contrast, our method significantly reduces the inference time by enabling inversion in just a few steps (e.g., N < 20). We demonstrate this efficiency in two tasks: reconstruction and box inpainting.To reconstruct x 1 , we first invert x 1 to obtain x0 , as described in Alg. 3. We then use the generation process (Alg. 2) with x0 and same initial velocity v θ (x 1 ) used in Alg. 3 to generate x1 . For box inpainting, we inject conditional information-the non-masked image region-into the iterative inversion and generation procedures, as detailed in Alg. 4. As demonstrated in Tab. 6 and 7, our method achieves better reconstruction quality (CAF: 46.68 PSNR vs. RF: 33.34 PSNR) and zeroshot inpainting capability even with fewer steps compared to baseline methods. Qualitative results are presented in Fig. 12 and 13 , which further illustrate the effectiveness of our approach. This demonstrate that our method can be efficiently used for real-world applications, offering both speed and accuracy advantages over existing techniques.Require: velocity model v θ , acceleration model a ϕ , sampling steps N , π 1 .1:x 1 ∼ π 1 2: vθ ← v θ (x 1 ) 3: for i = N to 1 do 4: t ← i N 5: t ′ ← 2i-1 2N 6:âϕ ← a ϕ (x t , vθ )7:x t- σ ∼ N (0, I) 2: x ← x1 ⊙ (1 -Ω) + σ ⊙ Ω▷ Create image with missing pixels and add noise σ3: vθ ← v θ (x) 4: for i = N to 1 do ▷ Inversion steps 5: t ← i N , t ′ ← 2i-1 2N 6:âϕ ← a ϕ (x t , vθ )7:x t-1 N ← x t -1 N vθ -t ′ N âϕ 8: x t-1 N ← x t-1 N ⊙ (1 -Ω) + (1 -t)σ ⊙ Ω, σ ∼ N (0, I) 9: end for 10: vθ ← v θ (x 0 ) 11: for j = 0 to N -1 do ▷ Generation steps 12: t ← j N , t ′ ← 2j+1 2N 13:âϕ ← a ϕ (x t , vθ )14:x t+ 1 N ← x t + 1 N vθ + t ′ N âϕ 15:x t+ 1 N ← x1 ⊙ (1 -Ω) + x t+ 1 N ⊙ Ω 16: end for 17: return inpainted image x 1Here, we elaborate on the crucial differences between AGM [41] and CAF. The main distinction is that CAF assumes constant acceleration, whereas AGM predicts time-dependent acceleration. Since the CAF ODE assumes that the acceleration term is constant with time, there is no need to solve time-dependent differential equations iteratively. This allows for a closed-form solution that supports efficient and accurate sampling, given that the learned velocity and acceleration models are accurate. Specifically, the solution for CAF ODE is given by:EQUATIONThe integral simplifies thanks to the constant acceleration assumption, leading to one-step sampling.In contrast, AGM's acceleration is time-varying, meaning that the differential equation cannot be reduced in an analytic form. It requires multiple steps to approximate the true solution accurately. In Tab. 8, we systemically compare AGM with our CAF, where CAF consistently outperforms AGM. Moreover, we conducted additional experiments where AGM was trained with deterministic couplings as in our reflow setting. Incorporating reflow into AGM did not improve its performance in the few-step regime, which further highlights the distinct advantage of CAF over AGM.One limitation of our model is the increased number of function evaluations (NFE) required for N -step generation. While Rectified flow achieves an NFE of N by only computing the velocity at each step, our method necessitates an additional computation, resulting in a total NFE of N + 1. This is because we compute the initial velocity at the beginning and the acceleration at each step. Although this extra evaluation slightly increases the computational burden, it is relatively minor in terms of overall performance and still enables efficient few-step generation. Moreover, this additional step can be reduced by jointly predicting velocity and acceleration terms with a single model, which we leave for future work. Another limitation is the additional effort required to generate supplementary data. We utilize generated data to create a deterministic coupling of noise and data samples for training CAF. While generating more data enhances our model's performance, it can increase GPU usage, leading to higher carbon emissions.Recent advancements in generative models hold significant potential for societal benefits across a wide array of applications, such as image and video generation and editing, medical imaging analysis, molecular design, and audio synthesis. Our CAF framework contributes to enhancing the efficiency and performance of existing diffusion models, offering promising directions for positive impacts across multiple domains. This suggests that in practical applications, users can utilize generative models more rapidly and accurately, enabling a broad spectrum of activities. However, it is crucial to acknowledge potential risks that must be carefully managed. The increased accessibility of generative models also broadens the potential for misuse. As these technologies become more widespread, the possibility of their exploitation for fraudulent activities, privacy breaches, and criminal behavior increases. It is vital to ensure their ethical and responsible use to prevent negative impacts. Establishing regulated ethical standards for developing and deploying generative AI technologies is necessary to prevent such misuse. Additionally, imposing restricted access protocols or verification systems to trace and authenticate generated contents will help ensure their responsible use. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 


Decision: This paper received two weak acceptances and one borderline rejection. Reviewer rYLJ generally agreed with the two positive reviews but raised a key concern regarding the concurrent work AGM [1], which might have a similar approach to the acceleration generation modeling framework. In the rebuttal, the authors provided a comparison with AGM, highlighting differences in the acceleration strategy and the closed-form solution. The comparison results also further verified the effectiveness. These points effectively justify the novelty of this work, especially since constant acceleration is a central claim in the title. Consequently, the AC agreed that the work retains its novelty despite the concurrent work on arXiv earlier, which had not been fully published before the NeurIPS submission deadline.

Additionally, the AC considered the generalization of this approach to other datasets, as noted by Reviewer oUYA. The rebuttal included further experiments on ImageNet 64x64, which strengthened the justification and made the work more convincing.

Based on the novelty, solid experiments presented in the paper, and additional evaluations in the rebuttal, the AC concurs with the majority of reviewers to accept the paper. For the camera-ready version, it is suggested to add a subsection discussing AGM and include this work in the related works section. Furthermore, it is encouraged to add generalization results on other datasets in the final version.

Reviews: review: paper_summary: paper_summary: This paper develops a new flow based generative model whose vector field is constructed similar to rectified flows (straight path between source and target samples), but instead of using a path with a constant speed, uses a path with constant acceleration.  This requires learning a neural network to parametrize the initial velocity field and another neural network to parametrize the acceleration field.  To improve their model's performance, the authors also propose parametrizing the acceleration neural network using the initial velocity and also applying reflows to the initial velocity.  The proposed method helps mitigate the flow crossing problem at a model level, which should in theory avoid excess reflows to learn non-crossed flows.  The experiments section demonstrates the ability of the proposed method to learn high performing generative models on single step generation on the CIFAR-10 dataset.
strength_weakness: strength_weakness: - The related work should include references to building diffusion models with non-markov path measures (see De Bortoli et al., 2023 and its related work).
- For completeness, the paper should include proofs to show that the learned distribution is the same as the data distribution.  This follows directly from interpreting Eq. 2 as a conditional flow matching objective (Lipman et al., 2023).
- In the initial presentation of the vector field $a(x_t,t)$ at the start of section $4.1$, it would make more sense to drop the dependence on $t$ to emphasize that the ground truth acceleration field is constant in time.
- The scope of the empirical evaluation is limited.  The authors should have evaluated on more datasets than just CIFAR-10.  There are other small scale image datasets, like ImageNet-32, that could have been evaluated.
questions: questions: - Is there a reason why algorithm 1, line 6, updates $\theta$ before computing the loss for $a_\phi$?
limitations: limitations: The authors adequately addressed the limitations.
paper_summary: paper_summary: This work proposes Constant Acceleration Flow (CAF), which, instead of learning a velocity-based flow model like in Flow Matching, jointly trains an initial velocity model together with a constant time-dependent acceleration model. This framework aims to learn straighter paths and thus enable better results for few-step generation.

Additionally, they propose to condition the acceleration model on the initial velocity and propose to improve the initial velocity using a reflow procedure. CAF is empirically validated on toy data and CIFAR-10, including ablations on the different elements of the proposed framework.
strength_weakness: strength_weakness: "Reflow" for initial velocity:
- The reflow procedure was proposed to straighten the paths of an existing generative model. The authors propose to use a "reflow" procedure for training CAF. However, they don't generate the couplings with a model trained with CAF but instead with a pre-trained EDM model, which is very different from how the reflow procedure was proposed. Additionally, it is unclear how dependent CAF is on this procedure and, thus, from the pre-trained generative model. In the CIFAR-10 ablation, the setting with constant acceleration and velocity conditioning but no "reflow" procedure (and thus no pre-trained generative model) is crucial to answer the question of whether CAF also works without a pre-trained model. If it does not, then I would argue that the proposed framework is more of a new distillation technique rather than a new generative model. This contextualization should also be more clearly explained in the main text.

Missing discussion and contextualization of related work:
- Acceleration Generative Modeling has been proposed in [1]. While CAF is based on Flow Matching and a constant velocity is chosen, [1] considers Bridge Matching in a stochastic optimal control framework with a changing velocity induced by the acceleration prediction. However, their trained model is also a parameterized acceleration prediction that takes as input the current data point $x_t$, current time $t$, and the current velocity $v_t$ effectively conditioning on the velocity. This framework is very related to the approach proposed by the authors, and the connection should be discussed in detail in the main paper. Additionally, an experimental comparison illustrating the differences between the two approaches could be beneficial.
- Coupling preservation through initial velocity conditioning: The authors refer to the reflow approach as existing related work. However, the reflow approach was proposed to straighten paths, not preserve couplings. Moreover, the problem of coupling preservation has been thoroughly analyzed in [2], which proposes to use a source point conditioning, i.e. conditioning on $x_0$. How does Augmented Flow/Bridge Matching (Flow/Bridge Matching with $x_0$ conditioning) compare to CAF? As mentioned, velocity conditioning is also used in [1].
- A possible way to empirically compare to these competing methods would be to extend the ablation study on CIFAR-10 to include [1] and [2].

Minor Weaknesses:
- Experiments are only conducted on CIFAR-10. Including at least one more dataset, e.g. 64x64 ImageNet/CelebA/etc, could strengthen the empirical results.

[1] Tianrong Chen and Jiatao Gu and Laurent Dinh and Evangelos A. Theodorou and Joshua Susskind and Shuangfei Zhai. "Generative Modeling with Phase Stochastic Bridges". In ICLR 2024.

[2] Valentin De Bortoli and Guan-Horng Liu and Tianrong Chen and Evangelos A. Theodorou and Weilie Nie. "Augmented Bridge Matching". In Arxiv 2023.
questions: questions: - As mentioned above, how dependent is CAF on the pre-trained model? Does it also work well without any pre-trained model?
- Does the 2-Rectified Flow comparison for the CIFAR-10 experiments also use generated samples by EDM for its first training round?
- Can the velocity and acceleration models be trained separately? E.g. first train the initial velocity model and then train the acceleration model?
limitations: limitations: - Again, how dependent is CAF on the pre-trained model?
- Training both an acceleration and a velocity prediction model effectively doubles the amount of total parameters. This could be mentioned as another limitation.
paper_summary: paper_summary: The paper presents an extension to rectified flow by Liu et al., 2023, a method based on optimal transport that can match samples from two distributions. This class of methods may have different uses but is often employed for the efficient generation of new samples and for morphing one sample to another. As such rectified flow and the extension proposed in this paper may be seen as finding efficient trajectories for use during generation by diffusion models. The method solves the transport flow in a simple Euler approximation of the ODE describing the flow. The paper identifies crossing of flow trajectories as a major obstacle in generating efficient paths in rectified flow and proposes constant acceleration flow (CAF). The paper proposes Initial Velocity Conditioning to produce the initialize the acceleration field in solving the ODE. Rectified flow uses constant velocity to solve the ODE using a reflow procedure to optimize the paths in a second step. Similarly, the proposed CAF also uses reflow to improve the initial paths. The paper reports experimental evaluation of the model with synthetic distributions and with CIFAR-10. The experiments demonstrate improved flow between distributions for synthetic data and improved FID score on CIFAR-10. The paper also reports an increase in straightness of trajectories.
strength_weakness: strength_weakness: The paper presents arguably a substantial improvement of results obtainable with rectified flow in terms of quality at the price of increased computation. The paper does not attempt any argument why this is significant, i.e., has the improved quality the potential to lead to wider adaptation of rectified flow type methods. The paper has no real-world use case to motivate the work. The claim that experiments with CIFAR-10 at 32x32 pixels is a real-world example seems far-fetched.    

The initialization of the method and its impact on the result is not well explored. The initial velocity calculation requires sample x_1 drawn from the target distribution. Both, the paper under consideration and rectified flow employ a pre-trained generative model to obtain that sample. The quality of initial samples may impact the results of CAF. The authors should provide evidence to the contrary or provide an experimental evaluation of the influence. 

There is no theoretical justification why constant acceleration is superior to constant velocity flow generation. While the experimental evaluation appears clear, it remains unclear if this behavior is due to the tasks considered or holds more broadly. 

The paper claims reflow as a contribution but it has been proposed by rectified flow. The same is true for the measurement of straightness of flow trajectory with Normalized Flow Straightness Score (NFSS) in Eqn 12. 

The paper does not discuss how CIFAR-10 is used. One can speculate based on earlier work such as GLOW that unconditional generation and conditional generation may refer to use of class conditioning but a concise description of the experimental setup is required. 

The use of a pre-trained generative model should also be taken into account in the required computational effort during training. It may be application-dependent if such a model is available or if it need to be trained.
questions: questions: A discussion of the use of rectified flow on real-world tasks would enhance the justification for the proposed CAF. Can such examples be provided and discussed?

Are there other options to provide the initial velocity field rather than using a pre-trained generative model? 

The paper is generally well written. The following are minor typos:
l. 20  a huge computation burden -> a large computational burden
l. 128 thet -> the
l. 176 Experiment -> Experiments or Experimental Evaluation 
l. 188 128-dimensional dimensions
limitations: limitations: The limitations section should be moved from the appendix into the main paper as the computational impact of CAF over rectified flow is essential.

Comments: title: Rebuttal well-addressed my concerns
comment: Thanks for the further explanations and results. This further clarified my concerns and I've decided to raise my score 3 -> 4.
Dear Reviewer rYLJ,

We sincerely appreciate your dedication and constructive feedback on our work. As the reviewer-author discussion period ends soon, we would like to kindly ask if you have any remaining concerns or questions.


Best regards, \
Authors
Thank you for the response. We'd like to clarify the additional questions the reviewer raised.

> ***AGM was uploaded to Arxiv October 2023, which does not make it concurrent given the NeurIPS Contemporaneous Work guidelines but previous work.***

First, we would like to clarify and sincerely apologize for any confusion: at the time of our submission, we were *genuinely unaware* of the AGM. As mentioned in our previous response, we fully recognize the AGM's contribution to the field of acceleration modeling and will ensure that discussions of AGM and related works are included in the revised paper.

> ***Is there an intuition of why this closed-form enables few-step with deterministic coupling over AGM with deterministic couplings?***

Since the CAF ODE assumes that the acceleration term is *constant* with respect to time, there is no need to iteratively solve complex time-dependent differential equations. This simplification allows for a direct closed-form solution that supports efficient and accurate sampling in just a few steps, given that the learned velocity and acceleration models are sufficiently accurate.

In contrast, AGM’s acceleration term is *time-varying*. This variability means that the differential equation cannot be simplified in the same way, often necessitating multiple iterative steps to approximate the true solution accurately. The constant acceleration assumption in CAF is similar to the concept of rectified flow (constant velocity), which is known to make solving differential equations more efficient than in diffusion models. 

Below are the details:

In CAF ODE (equation 4 in the paper), the solution for the final sample is given by:

$x_1 =x_0 +\int_0^1v(x_0,0)+a(x_t,t)tdt=x_0+v(x_0,0) +\int_0^1a(x_t,t)tdt$

Thanks to the constant acceleration assumption, the integral simplifies to:

$x_1=x_0+v(x_0,0)+ a(x_0,0)\int_0^1tdt=x_0+v(x_0,0)+ \frac{1}{2}a(x_0,0)$

This result corresponds to the one-step sampling version of equation 11 in our paper. 

> ***For the rebuttal, the authors compare CAF with reflow vs AGM without reflow. I believe the contribution of the [i] constant acceleration flow should be disentangled from the [ii] use of reflow. [ii] Reflow should also significantly improve the few-step results of AGM. To fairly compare the two frameworks, a comparison of just [i] vs AGM is needed or/and a comparison of [i] + [ii] vs AGM + [ii]. In my opinion, the reported results in the author rebuttal do not constitute a fair comparison between AGM and CAF.***

Thank you for your valuable feedback. We understand your concern regarding the need to disentangle the use of the reflow.

First, we would like to clarify the rationale behind our comparisons. In our proposed method, the reflow procedure is an essential component designed to achieve an accurate solution of our CAF ODE. AGM, on the other hand, does not incorporate reflow in its original methodology. In our rebuttal, we have fully utilized the AGM method in its **standard form** as proposed by the authors, using their official code without making any alterations. This approach reflects the typical use cases and capabilities of both methods as they were originally intended to be used.

To address the reviewer's concern, we conducted additional experiments where AGM was trained with deterministic couplings generated from EDM (the same as our reflow setting). We replaced $\epsilon_0$ and $x_1$ in AGM with noise-image pairs from EDM and followed the experimental setup in the official AGM code. The results are summarized in the table below:
|                        | N |   FID $\downarrow$   |
|------------------------|:-:|:-----:|
| AGM-ODE without reflow | 5 | 11.88 |
| AGM-ODE with reflow    | 5 |  15.23 |
| CAF                    | 1 |  5.15 |

In fact, incorporating reflow into AGM did not necessarily improve its performance in the few-step regime. We assume that this is because AGM may require noise-velocity-image triplets. This indicates that effectively integrating the reflow mechanism within the AGM framework is not straightforward with a pre-trained diffusion model, and it requires extra effort to build these triplets that go beyond the scope of our current comparison.

---
We hope our explanations address the reviewer’s concerns regarding AGM.
Thank you for the detailed answers and added experiments.

> ***The result demonstrates that CAF outperforms AGM in terms of FID scores in an extremely few-step regime.***

For the rebuttal, the authors compare CAF with reflow vs AGM without reflow. I believe the contribution of the [i] constant acceleration flow should be disentangled from the [ii] use of reflow. [ii] Reflow should also significantly improve the few-step results of AGM. To fairly compare the two frameworks, a comparison of just [i] vs AGM is needed or/and a comparison of [i] + [ii] vs AGM + [ii]. In my opinion, the reported results in the author rebuttal do not constitute a fair comparison between AGM and CAF.

> ***This closed-form solution enables few-step (N<3) sampling with high accuracy when learned with deterministic couplings.***

Is there an intuition of why this closed-form enable few-step with deterministic coupling over AGM with deterministic couplings? As mentioned above, this should also be confirmed empirically.

> ***Since this concurrent work was published a few weeks before our submission.***

AGM was uploaded to Arxiv October 2023, which does not make it concurrent given the NeurIPS Contemporaneous Work guidelines but previous work.
We greatly appreciate the reviewer’s response and are truly thankful for the constructive feedback that has helped enhance our work. Thank you for dedicating your time and effort to providing us with such valuable insights.
We sincerely appreciate the reviewer’s prompt response and are grateful for the constructive comments that have strengthened our work. Thank you for dedicating your time and effort to providing us with such valuable insights.
Thank you for the detailed rebuttal. 
Based on the explanations and especially the demonstrations of additional applications, I have revised my recommendation to weak accept.
Thank you for the response and for the updated experiments.  My opinion on the paper is the same so I am going to keep the same score.