Implicit Dynamical Flow Fusion (IDFF) for Generative Modeling

## Abstract

Abstract Conditional Flow Matching (CFM) models can generate high-quality samples from a non-informative prior, but they can be slow, often needing hundreds of network evaluations (NFE). To address this, we propose Implicit Dynamical Flow Fusion (IDFF); IDFF learns a new vector field with an additional momentum term that enables taking longer steps during sample generation while maintaining the fidelity of the generated distribution. Consequently, IDFFs reduce the NFEs by a factor of ten (relative to CFMs) without sacrificing sample quality, enabling rapid sampling and efficient handling of image and time-series data generation tasks. We evaluate IDFF on standard benchmarks such as CIFAR-10 and CelebA for image generation, where we achieve likelihood and quality performance comparable to CFMs and diffusion-based models with fewer NFEs. IDFF also shows superior performance on time-series datasets modeling, including molecular simulation and sea surface temperature (SST) datasets, highlighting its versatility and effectiveness across different domains.

## 1 Introduction

Diffusion models have emerged as powerful tools for modeling complex, high-dimensional data by iteratively transforming random noise into meaningful information (Cachay et al., ; Myers et al., ). These models have demonstrated remarkable success in various domains, including high-quality image generation (Song et al., ) and text production (Liu et al., ; Kim et al., ). However, their training process remains computationally intensive, often requiring hundreds of function evaluations (NFEs) to achieve high-quality results (Ho et al., ). While techniques such as DPM-solver (Lu et al., , ) and Denoising Diffusion Implicit Models (DDIMs) (Song et al., ) have made strides in reducing NFEs for diffusion-based generative models, significant computational challenges persist.

In contrast to diffusion models, which parameterize the dynamics of noise-to-data transformation, Conditional Flow Matching (CFMs) directly parameterize vector fields governing this transformation (Liu et al., ; Albergo and Vanden-Eijnden, ; Albergo et al., ). CFMs leverage the change of variable theory in statistics to define conditional probability paths, enabling faster convergence during training compared to diffusion models (Lipman et al., ). However, generating high-quality samples with CFMs still requires hundreds of NFEs, making the process computationally expensive (Dao et al., ) and limiting the extension of CFM models to time-series data modeling, as NFEs scale with sequence length.

To address these challenges, we introduce a novel method to enhance the efficiency of CFMs, allowing for longer sampling steps without compromising the fidelity of the target distribution. Our approach draws inspiration from Hamiltonian Monte Carlo (HMC) algorithms (Neal, ; Neal et al., ), which utilize the conservation properties of Hamiltonian dynamics to couple target and momentum variables.
The key contributions of our work are as follows:

- 1.
We propose Implicit Dynamical Flow Fusion (IDFF), which integrates a momentum term into the vector field of conditional flow models. Similar to how HMC reduces the number of samples required to reach the target distribution (Neal, ; Neal et al., ), IDFF reduces the NFE by a factor of 10 compared to traditional CFMs while maintaining identical marginal distributions. Additionally, we introduce a training objective that directly aligns generated samples with actual data rather than solely focusing on matching conditional flows.
- 2.
We validate IDFF through extensive experiments on image generation using datasets such as CIFAR-10 and CelebA. Additional results on ImageNet-64, CelebA-HQ, LSUN-Church, and LSUN-Bedroom are presented in Figure 1. Our models achieve a superior balance between computational cost and sample quality compared to both traditional diffusion models (e.g., DDPM (Ho et al., ), DDIM (Song et al., )) and current CFMs.
- 3.
We demonstrate that reducing NFEs makes IDFF practical for other generation tasks, such as time-series modeling. We showcase IDFF’s effectiveness in capturing complex dynamical systems, including 3D attractors, molecular dynamics (MD), and sea surface temperatures (SST). Our results indicate that IDFF surpasses other methods in sample quality for MD and SST datasets while maintaining computational efficiency, with NFE $\leq$ 5 for each generated sample.

Figure: Figure 1: Image generation using IDFF (with NFE=10) applied to multiple datasets: CIFAR10, CelebA-64, ImageNet-64, LSUN-Bedroom, LSUN-Church, and CelebA-HQ. The quality of the generated images demonstrates IDFF’s ability to capture and reproduce realistic visuals across varying levels of complexity and resolution. Additional samples and further analysis can be found in Appendix A.5.
Refer to caption: extracted/5899840/figures/Figure_1.png

## 2 Background

##### Score-based generative models.

Let $p_{\text{data}}=\{\mathbf{x}^{1},\mathbf{x}^{2},\dots,\mathbf{x}^{N}\}$, where $\mathbf{x}^{i}\overset{\text{iid}}{\sim}p(\mathbf{x})$ and $\mathbf{x}^{i}\in\mathbb{R}^{d}$ for all $i=1,2,\dots,N$. We aim to build a generative model for this dataset given empirical samples. Moving forward, we suppress the superscript on each $\mathbf{x}^{i}$.

Score-based models (Song and Ermon, ) represent a broad class of diffusion models that describe a continuous-time stochastic process characterized by a stochastic process $\mathbf{x}_{t}$, which is governed by the following Itô stochastic differential equation (SDE):

$$ $d\mathbf{x}_{t}=\mathbf{f}(\mathbf{x}_{t},t)dt+g(t)d\mathbf{w},$ (1) $$

where $t\in[0,1]$, $\mathbf{f}(\cdot,t):\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$
is the drift coefficient, $g(\cdot):\mathbb{R}\rightarrow\mathbb{R}$ is
the diffusion coefficient of $\mathbf{x}_{t}$, and $\mathbf{w}\in\mathbb{R}^{d}$ is a standard Wiener process.

The time-reversed version of this diffusion process, derived from the Fokker–Planck equations (Song and Ermon, ), is also a diffusion process. This reverse-time SDE is defined as:

$$ $d\mathbf{x}_{t}=\left(\mathbf{f}(\mathbf{x}_{t},t)-g(t)^{2}\nabla_{\mathbf{x}_ {t}}\log p_{t}(\mathbf{x}_{t})\right)dt+g(t)d\bar{\mathbf{w}},$ (2) $$

where $\bar{\mathbf{w}}$ is a standard Wiener process. Equation 2 produces the same marginal distributions as the forward diffusion process defined by Equation 1.

Equation 2 must be solved to generate samples from this model. The score-matching approach simplifies this process by redefining $\mathbf{f}(\mathbf{x}_{t},t)=h(t)\mathbf{x}_{t}$, where $h:\mathbb{R}\rightarrow\mathbb{R}$ typically assumes an affine form, such as $\mathbf{f}(\mathbf{x}_{t},t)=\mathbf{x}_{t}$. Moreover, $g(\cdot)$, which governs the noise intensity added to the process, often follows a linear or exponential schedule in time.

This reformulation enables a simplification of solving Equation 2 by focusing solely on learning $\nabla_{\mathbf{x}}\log p_{t}(\mathbf{x}_{t})$ for all $t$. With this approach, and initializing with random noise, e.g., $\mathbf{x}_{0}\sim\mathcal{N}(0,\mathbf{I})$, the process gradually converges to the data distribution $p_{\text{data}}$, reaching it at $t=1$.

##### (Conditional) Flow Matching (CFM).

Flow-matching (FM) transforms a simple prior distribution into a complex target distribution. FM is defined by a time-dependent vector field $\mathbf{v}_{t}$, which governs the dynamics of an ordinary differential equation (ODE):

$$ $\frac{d}{dt}(\mathbf{x}_{t})=\mathbf{v}_{t}(\mathbf{x}_{t})$ (3) $$

The solution to this ODE, represented by $\phi(\mathbf{x}_{t})$, evolves along the vector field $\mathbf{v}_{t}$ from time 0 0 to time $1$, producing $\mathbf{x}_{1}\sim p_{1}$. As samples move along the vector field $\mathbf{v}_{t}$, $p_{t}({\mathbf{x}}_{t})$ changes over time, described by the continuity equation.

$$ $\frac{\partial p_{t}}{\partial t}=-\nabla\cdot(p_{t}\mathbf{v}_{t})$ (4) $$

where $p_{t}(\mathbf{x}_{t})$ is determined by $\mathbf{v}_{t}(\mathbf{x}_{t})$, $\nabla\cdot$ is the divergence operator, with $p_{1}\approx p_{\text{data}}$.

If the probability path $p_{t}({\mathbf{x}}_{t})$ and the vector field ${\mathbf{v}}_{t}({\mathbf{x}}_{t})$ are given, and if $p_{t}({\mathbf{x}}_{t})$ can be efficiently sampled, we can then define a parameterized vector field $\hat{{\mathbf{v}}}_{t}({\mathbf{x}}_{t};\bm{\theta})$ using a neural network with weights $\bm{\theta}$. This parameterized vector field approximates $\mathbf{v}_{t}(\mathbf{x}_{t})$, and our goal is to learn how to generate samples from the ODE defined in equation 3. The parameters of the neural network are trained using the following objective: $\mathcal{L}_{\text{FM}}(\bm{\theta}):=\mathbb{E}_{t\sim\mathcal{U}(0,1),{
\mathbf{x}}_{t}\sim p_{t}({\mathbf{x}}_{t})}\|\hat{\mathbf{v}}_{t}({\mathbf{x}
}_{t};\bm{\theta})-\mathbf{v}_{t}({\mathbf{x}}_{t})\|^{2}$ to solve the ODE defined in equation 3. Here, $\mathbf{v}_{t}({\mathbf{x}}_{t})$ can be defined as $\mathbf{v}_{t}({\mathbf{x}}_{t}):=\mathbb{E}_{q({\mathbf{z}})}\left[\frac{
\mathbf{v}_{t}({\mathbf{x}}_{t}|{\mathbf{z}})p_{t}({\mathbf{x}}_{t}|{\mathbf{z
}})}{p_{t}({\mathbf{z}})}\right]$ with some conditional variable ${\mathbf{z}}$ with distribution $q({\mathbf{z}})$; refer to Theorem 3.1 in Tong et al. () for more details.

However, this learning objective becomes intractable for general source and target distributions. To mitigate this, we can focus on cases where the conditional probability paths $p_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})$ and vector fields ${\mathbf{v}}_{t}({\mathbf{x}}_{t}|{\mathbf{x}}_{1})$ associated with $p_{t}(\mathbf{x}_{t})$ and ${\mathbf{v}}_{t}({\mathbf{x}}_{t})$ are known and have simple forms (Tong et al., ). In such cases, we can recover the vector field ${\mathbf{v}}_{t}({\mathbf{x}}_{t})$ using an unbiased stochastic objective known as the Conditional Flow Matching (CFM) loss, defined as:

$$ $\mathcal{L}_{\text{CFM}}(\bm{\theta}):=\mathbb{E}_{t,\mathbf{x}_{1},\mathbf{x} _{t}}\left\|\hat{\mathbf{v}}_{t}(\mathbf{x}_{t}|\mathbf{x}_{1};\bm{\theta})- \mathbf{v}_{t}(\mathbf{x}_{t})\right\|^{2}$ (5) $$

where $t\sim\mathcal{U}[0,1]$, $\mathbf{x}_{1}\sim p_{\text{data}}$, and $\mathbf{x}_{t}\sim p_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})$. The training objective in Equation 5 ensures that the marginalized vector field $\hat{\mathbf{v}}_{t}(\mathbf{x}_{t}|\mathbf{x}_{1};\bm{\theta})$, denoted as $\hat{\mathbf{v}}_{t}(\mathbf{x}_{t})$, generates $p_{t}(\mathbf{x}_{t})$, similar to FM models.

##### CFMs vs Diffusion Models.

CFMs share similarities with diffusion models; both rely on defining continuous probability paths over time. However, diffusion models use stochastic processes to transform data distributions, typically governed by an SDE. However, CFMs utilize a vector field to directly map an initial prior distribution to the target distribution via a deterministic process (Equation 4). Diffusion models, such as denoising score matching models, rely on stochastic diffusion paths to approximate the data distribution. CFMs bypass this stochastic process by constructing a conditional vector field and sampling from the ODE defined in Equation 3.

##### Optimal Transport CFMs (OT-CFMs).

The CFM objective defined in Equation 5 appears simple. However, it is nearly intractable due to the absence of prior knowledge regarding an appropriate mapping that links between the initial ($p_{0}$) and target ($p_{\text{data}}$) distributions. To address this challenge, optimal transport (OT) (Tong et al., ) is used to couple the initial and target distributions. This approach seeks a mapping from $p_{0}$ to $p_{1}$ that minimizes the displacement cost between these two distributions, resulting in flows that can be integrated accurately. This CFM variant is known as Optimal Transport CFMs (OT-CFMs), and the conditional vector field corresponds to this is $\hat{\mathbf{v}}_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})=\frac{\mathbf{x}_{1}-
\mathbf{x}_{t}}{1-t}$. While OT improves training efficiency in CFMs, traditional sampling methods for CFMs require over a hundred function evaluations (NFE) to produce high-quality samples (Tong et al., ). The lack of flexibility in sampling steps during training further limits their ability to efficiently generate samples, often resulting in a high NFE.

Additionally, the OT-CFM vector field, which can be reformulated as $\hat{\mathbf{v}}_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})=\mathbf{x}_{1}-\mathbf{x}_
{0}$ Pooladian et al. (), focuses on exactly transporting all points ${\mathbf{x}}_{0}$ to ${\mathbf{x}}_{1}$ along straight-line paths. This implies that all points along the path from 0 0 to $t$ means that the transport map shares the same regression weights when training with the $\mathcal{L}_{\text{CFM}}(\bm{\theta})$ objective. However, this uniform weighting across the trajectory may overlook that points closer to $t=1$ need finer attention to detail and, consequently, a more complex flow; this may limit the model’s ability to effectively represent complex structures in the final generated samples. Furthermore, the deterministic sample generation in CFMs, ${\mathbf{x}}_{t}={\mathbf{x}}_{t-\Delta t}+\Delta t\hat{{\mathbf{v}}}_{t}({
\mathbf{x}}_{t};\bm{\theta})$ (see Figure 2.B for visualization), may restrict the flexibility of CFM, as it prevents the introduction of stochasticity that could otherwise enhance the diversity and richness of generated samples. This might manifest in a lack of expressiveness when generating fine-grained or high-dimensional data. These limitations underscore the need for new approaches to sample diversity and NFE of CFMs. Our paper tackles these limitations by proposing an alternative vector field that better balances deterministic transport with elements of stochasticity to improve sample quality.

Figure: Figure 2: A) Comparison of trajectory sampling between IDFF and OT-CFMs: The figure displays 4096 final samples generated by IDFF. As shown, IDFF takes larger steps toward the target distribution, guided by the momentum term. While OT-CFM follows a nearly straight path to reach the target distribution, it requires a higher number of function evaluations (NFEs). B) OT-CFMs sampling process. C) IDFF sampling process. In this process, $\hat{\mathbf{x}}_{1}(.)$ approximates the data sample ${\mathbf{x}}_{1}$, and $\hat{\epsilon}(.)$ approximates the scores associated with $\bm{\xi}_{t}$. For further details, see Algorithm 2. The key difference between IDFF and OT-CFMs is that instead of directly generating the vector field with, IDFF generates $\hat{\mathbf{x}}_{1}(.)$ and $\hat{\epsilon}(.)$, and then reconstructs the vector field, following it. Additionally, IDFF uses a Gaussian distribution to sample ${\mathbf{x}}_{t}$ rather than calculating it deterministically.
Refer to caption: extracted/5899840/figures/fig2d_toy_trj.png

## 3 Implicit Dynamical Flow Fusion Model (IDFF)

IDFF aims to improve sampling efficiency while enhancing sample quality of OT-CFMs through two key innovations: (1) the incorporation of a learnable Hamiltonian momentum term, $\bm{\xi}_{t}$, into the conditional vector field, which enhances sampling efficiency; and (2) the proposal of a new objective function that learns the IDFF’s vector field in input space.

### 3.1 Learning CFMs with Momentum

To improve sampling efficiency while preserving distribution accuracy in CFMs, we introduce a learnable Hamiltonian momentum term, $\bm{\xi}_{t}$, integrated with the original vector field $\mathbf{v}_{t}(\mathbf{x}_{t})$. This integration yields a new, more flexible vector field:

$$ $\tilde{\mathbf{v}}_{t}(\mathbf{x}_{t})=\sqrt{1-\sigma^{2}_{t}}{\mathbf{v}}_{t} (\mathbf{x}_{t})+\sigma^{2}_{t}\bm{\xi}_{t},$ (6) $$

where $\sigma_{t}$ is a time-dependent interpolation factor balancing ${\mathbf{v}}_{t}$ and $\bm{\xi}_{t}$.

*Interpolating the vector field:* We define the interpolation factor $\sigma_{t}$ as $\sigma_{t}=\sigma_{0}\sqrt{t(1-t)}$, where $\sigma_{0}$ is a hyperparameter that controls the overall influence of the momentum term. While the IDFF vector field converges to OT-CFMs in either limit, outside of it, it enables the flow to leverage the momentum variables to guide the flow based on the momentum variables. Our empirical results demonstrate that this approach significantly improves sampling efficiency by substantially reducing the NFEs required. Our approach also overcomes the limitations of uniform weighting across trajectories by prioritizing samples near $t=0,1$, allowing finer attention to complex structures; see Figure 2.A.

*Designing momentum variables:* The term $\bm{\xi}_{t}$ is designed to guide the new vector field toward the correct probability flow for the next time step, ensuring its role in accurately modeling the distribution. There are many potential choices to make here. Still, we take inspiration from Hamiltonian Monte Carlo, which showcases how to leverage momenta to rapidly sample complex multi-modal posterior distributions Neal (). A potential design for $\bm{\xi}_{t}$ is $\bm{\xi}_{t}=\gamma\nabla_{x}\log p_{t}(\mathbf{x}_{t})$, which aligns with the gradient of the log probability of the samples $\mathbf{x}_{t}$. If $p_{t}(\mathbf{x}_{t}\mid\mathbf{x}_{1})$ is a Gaussian with variance $\sigma_{t}^{2}$, $\bm{\xi}_{t}$ can be simplified to $-\frac{\bm{\epsilon_{0}}}{\sigma_{t}}$, where $\bm{\epsilon_{0}}\sim\mathcal{N}(\bm{0},\bm{\text{I}})$. Alternatively, higher-order choices for momentum terms could capture even more complex interactions in the flow. $\bm{\xi}_{t}=\gamma\nabla_{x}\log p_{t}(\mathbf{x}_{t})$ drives the vector field toward regions of high probability, thereby improving the accuracy of the generated samples. $\gamma$ serves as a weighting factor that controls the influence of this term.

To maintain the same probability path as the original CFM, the vector field must satisfy the continuity constraint defined in equation 4. By incorporating $\tilde{\mathbf{v}}_{t}(\mathbf{x}_{t})$ into the continuity equation 4 (detailed in Appendix A.1.1), we derive the IDFF probability flow ODE:

$$ $\mathbf{w}_{t}(\mathbf{x}_{t})=\tilde{\mathbf{v}}_{t}(\mathbf{x}_{t})-\frac{ \sigma_{t}^{2}}{2}\nabla\log p_{t}(\mathbf{x}_{t})=\sqrt{1-\sigma_{t}^{2}}{ \mathbf{v}}_{t}(\mathbf{x}_{t})+\frac{(2\gamma-1)}{2}\sigma_{t}^{2}\nabla\log p _{t}(\mathbf{x}_{t}),$ (7) $$

To ensure that our new marginal vector field $\mathbf{w}_{t}(\mathbf{x}_{t})$ converges to the standard ${\mathbf{v}}_{t}(\mathbf{x}_{t})$ at the start and end points of the process, $\sigma_{t}$ must approach zero as $t\to{0,1}$. This convergence is crucial for maintaining consistent marginal distributions at these points.

Therefore, ${\mathbf{w}}_{t}({\mathbf{x}}_{t})$ now replaces the vector field ${\mathbf{v}}({\mathbf{x}}_{t})$ in the probability flow ODE (equation 3), enabling the construction of a new generative process that follows the same probability path $p_{t}(\mathbf{x}_{t})$. This process starts with $\mathbf{x}_{0}\sim\mathcal{N}(0,\mathbf{I})$ and follows $p_{t}(\mathbf{x}_{t})$ to produce $\mathbf{x}_{1}$.
The resulting model thus *implicitly* models the score function using flow matching which motivates the name *Implicit Diffusion Flow Fusion*.

###### Lemma 1 .

Given that ${\mathbf{v}}_{t}(\mathbf{x}_{t})$ generates $p_{t}(\mathbf{x}_{t})$ and $\sigma_{t}$ approaches 0 for $t\in{0,1}$, the IDFF probability flow ODE (Eq. 7) follows the same marginal distribution as the CFMs with the ODE defined in equation 3.

###### Proof Schema for Lemma 1 .

Our goal is to show that the IDFF probability flow ODE shares the same marginal distribution as CFMs when $\sigma_{t}$ is small for $t\in{0,1}$.

- •
Step 1: Define the vector fields. We introduce $\tilde{\mathbf{v}}_{t}(\mathbf{x}_{t})$ governed by the $\bm{\xi}_{t}$ and ${\mathbf{v}}_{t}(\mathbf{x}_{t})$ from the original CFM.
- •
Step 2: Apply the continuity equation. By substituting $\tilde{\mathbf{v}}_{t}(\mathbf{x}_{t})$ into the continuity equation, we derive the IDFF ODE, ${\mathbf{w}}_{t}(\mathbf{x}_{t})$.
- •
Step 3: Analyze boundary conditions. We show that at $t=0$ and $t=1$, where $\sigma_{t}$ approaches 0, the IDFF ODE converges to the CFM-ODE, ensuring the same marginal distributions.

∎

Appendix A.1.1 provides the complete proof. This lemma and its proof establish the consistency of IDFF concerning existing CFMs. The approach showcases how to design a new flow augmented with momentum variables with an interpolant that guarantees that it matches OT-CFM in the limit. In what follows, we discuss how to parameterize and learn such a flow.

###### Lemma 1 .

###### Proof Schema for Lemma 1 .

### 3.2 Learning Flows in Input Space

One of the challenges in our formalism is learning ${\mathbf{w}}_{t}$.
This challenge arises because ${\mathbf{v}}_{t}$ operates in vector space while $\bm{\xi}_{t}$ remains in the input space. Rather than training one part of the model in vector space and the other in input space, we define a new objective defined entirely within the input space to address this. Learning the vector field in the input space implies optimizing the vector itself, which is the reasoning behind naming the IDFF model an implicit model of dynamical flows.
We begin by defining an OT path $p_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})$, which acts as a probability bridge between $\mathbf{x}_{0}$ and $\mathbf{x}_{1}$. This path is a stochastic interpolator between the source $\mathbf{x}_{1}$ and target $\mathbf{x}_{0}$, as follows:

$$ $p_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})=\mathcal{N}(\mathbf{x}_{t}\mid t\mathbf{x }_{1}+(1-t)\mathbf{x}_{0},\sigma_{t}^{2}\mathbf{I}),\ \ \sigma_{t}=\sigma_{0} \sqrt{t(1-t)}$ (8) $$

The corresponding conditional vector field is given by:

$$ $\hat{\mathbf{v}}_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})=\frac{\mathbf{x}_{1}- \mathbf{x}_{t}}{1-t}$ (9) $$

This definition aligns with the conditional OT paths between $\mathbf{x}_{1}$ and $\mathbf{x}_{0}$. For large datasets, computing this OT map can be challenging. We employ a minibatch approximation of OT to address this, similar to (Fatras et al., ). While this introduces some errors compared to the exact OT solution, it has proven effective in numerous applications such as domain adaptation and generative modeling (Genevay et al., ).

Current CFM objectives typically use a loss function (defined in equation 5) that optimizes the prediction of the vector field. We propose an alternative approach: defining a denoising neural network $\hat{\mathbf{x}}_{1}(\mathbf{x}_{t},t;\theta)$ and reparameterizing $\hat{\mathbf{v}}_{t}(\mathbf{x}_{t}\mid\mathbf{x}_{1};\theta)$ as follows:

$$ $\hat{\mathbf{v}}_{t}(\mathbf{x}_{t}\mid\mathbf{x}_{1};\theta)=\frac{\hat{ \mathbf{x}}_{1}(\mathbf{x}_{t},t;\theta)-\mathbf{x}_{t}}{1-t}$ (10) $$

Since both $\hat{\mathbf{x}}_{1}(\mathbf{x}_{t},t;\theta)$ and $p_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})$ are easy to sample, we use a single neural network with two separate output heads: the first head, $\hat{\mathbf{x}}_{1}(\mathbf{x}_{t},t;\theta)$, to approximate the data samples, and the second, $\hat{\bm{\epsilon}}(\mathbf{x}_{t},t;\theta)$, to approximate the score function. The network generates these outputs simultaneously. The score function $\nabla_{\mathbf{x}_{t}}\log p_{t}(\mathbf{x}_{t}\mid\mathbf{x}_{1})$, using a Gaussian $p_{t}(\mathbf{x}_{t}\mid\mathbf{x}_{1})$, can be simplified to $-\frac{\bm{\epsilon_{0}}}{\sigma_{t}}$, where $\bm{\epsilon_{0}}\sim\mathcal{N}(\bm{0},\mathbf{\text{I}})$. The neural network can be trained using the following loss function:

$$ $\displaystyle\mathcal{L}_{\text{IDFF}}(\theta)=\mathbb{E}_{t,\mathbf{x}_{0}, \mathbf{x}_{1},\mathbf{x}_{t}}\big{[}\beta(t)^{2}|\hat{\mathbf{x}}_{1}(\mathbf {x}_{t},t;\theta)-\mathbf{x}_{1}|^{2}+\lambda(t)^{2}\left|\hat{\bm{\epsilon}}( \mathbf{x}_{t},t;\theta)-\nabla_{x}\log p_{t}(\mathbf{x}_{t}|\mathbf{x}_{1}) \right|^{2}\big{]}$ (11) $$

Where ${t\sim\mathcal{U}(0,1),\mathbf{x}_{0}\sim\mathcal{N}(\bm{0},\mathbf{\text{I}})
,\mathbf{x}_{1}\sim p_{\text{data}},\mathbf{x}_{t}\sim p_{t}(\mathbf{x}_{t}|
\mathbf{x}_{1})}$. Here ${\lambda(\cdot),\beta(\cdot)}$ are two sets of positive weights.

Our proposed loss function comprises two terms:
The first term optimizes the denoising model $\hat{\mathbf{x}}_{1}(\mathbf{x}_{t},t;\theta)$, with $\mathbf{x}_{t}$ as the noisy input.
The second term represents a scoring model that approximates $\nabla_{x}\log p_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})$. The first term of the loss incorporates a $\beta(t)$ weighting schedule, which emphasizes the quality of samples as they approach $t=1$. For more details about the loss function, please refer to Appendix A.1.2. The complete training process is described in Algorithm 1.

##### Sampling

After training $\hat{\mathbf{x}}_{1}(.;\theta)$ and $\bm{\epsilon}(.;\theta)$, we can generate samples from IDFF by solving an SDE starting from the source samples $\mathbf{x}_{0}$. Refer to Figure 2.C for visualization. This procedure is outlined in Algorithm 2. The SDE is defined as $d\mathbf{x}_{t}=\mathbf{w}_{t}dt+\sigma_{t}d\mathbf{w}$, with initial samples $\mathbf{x}_{0}$ and target samples $\mathbf{x}_{1}$ at $t=1$. Therefore, we can draw samples from $p(\mathbf{x}_{0})$, run them through the SDE, and obtain the final samples $\mathbf{x}_{1}$.
First, we draw $\mathbf{x}_{0}\sim\mathcal{N}(0,\mathbf{I})$, then compute $\hat{\mathbf{x}}_{1}(\mathbf{x}_{0},0,k;\theta)$. Next, we calculate $\hat{\mathbf{v}}_{t}$ using equation 10, followed by the computation of $\mathbf{w}_{t}$. After calculating $\mathbf{w}_{t}$ and using the SDE, we compute the mean for the next time step as $\bm{\mu}_{t+\Delta t}=(\mathbf{x}_{t}+\mathbf{w}_{t}\Delta t)$, and finally draw a sample from a small neighborhood of it: $\mathbf{x}_{t+\Delta t}\sim\mathcal{N}(\mathbf{x}_{t}+\mathbf{w}_{t}\Delta t,
\sigma_{t}^{2}\Delta t\mathbf{I})$. The $\Delta t$ is determined by the NFE as $\Delta t=\frac{1}{\text{NFE}}$, where smaller NFE results in finer time discretization and tighter neighborhoods.

##### Likelihood calculation.

To evaluate the likelihood of data under IDFFs, we can leverage the change in the probability density as the sample ${\mathbf{x}}_{t}$ evolves according to the velocity field ${\mathbf{w}}_{t}$ as described by the continuity equation 4, which gives the time evolution of the probability density under the learned flow Lipman et al. ():

$$ $\log p_{1}(\mathbf{x}_{1})=\log p_{0}(\mathbf{x}_{0})-\int_{0}^{1}\nabla\cdot{ \mathbf{w}}_{t}(\mathbf{x}_{t})\,dt$ (12) $$

This integral is numerically approximated, often using Monte Carlo methods.

### 3.3 IDFF time-series adaptation

To use IDFF for time-series applications, we need to modify the training and sampling algorithms for IDFF as shown in Algorithms 4 and 3, respectively. To accomplish that, we define another random variable, $k$, which is a discrete variable and can take values of ${1,...,K}$ with equal probability, where $K$ is the length of a time series. Therefore, $k$ represents the index of the data sample in a time series. We also have $t$ as a continuous time variable interpolated between two subsequent values of $k$ (i.e., $k$ and $k-1$). Therefore we have $\forall k\in{1,...,K}\rightarrow p(t\mid k)\sim\mathcal{U}(0,1)$. This conditioning does not change the original assumption over $t$ since $k$ can take any values up to the time-series length with the same probability. With this in hand, we can directly pass the $k$ as additional inputs to $\hat{\bm{\epsilon}}_{1}(\mathbf{x}_{t},t,k;\theta)$ and $\hat{\mathbf{x}}_{1}(\mathbf{x}_{t},t,k;\theta)$ and suggest Algorithm 1 for training and Algorithm 2 for sampling time-series data with IDFF. Note that for static data, the sequence length ($K$) equals one; otherwise, it equals the time series length.

Figure: Algorithm 1 IDFF training algorithm.

## 4 Related Work

Recent advancements in CFMs include Schrödinger bridge-based methods (Richter et al., ) and conjugate/splitting-based integrators (Pandey et al., ), both aimed at improving sampling efficiency. However, these methods still struggle to significantly reduce the NFEs, which remains a critical factor for computational efficiency.

In diffusion models, denoising diffusion implicit models (DDIMs) (Song et al., ) were introduced to address the issue of large NFEs by offering a more efficient sampling process. DDIMs deterministically generate samples from latent variables, unlike DDPMs, which rely on Langevin dynamics. By employing a variational approach, DDIMs accelerate sampling while preserving output quality. Other approaches to enhance sampling in diffusion-based generative models include distillation frameworks such as Flash (Kohler et al., ), which generate diverse samples in fewer steps by mitigating training-inference discrepancies.

A distinguishing feature of IDFF is its ability to jointly learn an implicit flow from the generated samples alongside the scoring model. This sets it apart from models like DDIM, which focuses solely on learning a scoring function. By leveraging OT to couple samples between $t=0$ and $t=1$, IDFF accelerates convergence during training while maintaining a low NFE and improving sample quality.

Our image generation experiments, presented in Tables 1 and 2, demonstrate IDFF’s superior performance. Compared to DDPM (Ho et al., ), DDIM (Song et al., ), and DPM-Solver (Lu et al., ) (designed to reduce NFEs in diffusion models), IDFF consistently outperforms these models in both efficiency and quality.

**Table 1: Comparison of Likelihood (BPD), FID, and NFE between IDFF and various methods on the CIFAR-10 dataset.**
| Model | NLL$\downarrow$ | FID$\downarrow$ | NFE$\downarrow$ |
| --- | --- | --- | --- |
| DDPM | 3.12 | 7.48 | 274 |
| DDIM | – | 6.84 | 20 |
| DDIM | – | 13.36 | 10 |
| DPM-Solver | – | 6.03 | 12 |
| Score Matching | 3.16 | 19.94 | 242 |
| ScoreFlow | 3.09 | 20.78 | 428 |
| OT-CFM | 2.99 | 6.35 | 142 |
| OT-CFM | – | 11.87 | 10 |
| IDFF (Ours) | 3.09 | 5.87 | 10 |

In summary, IDFF integrates conditional flow and score-based learning strengths, offering a novel and efficient framework for generative modeling that significantly reduces computational costs while producing high-quality samples. The following section provides more details about these comparisons.

## 5 Experiments

We conducted two general classes of experiments to demonstrate the effectiveness of IDFF in generative modeling on static data, such as images and time-series data, including simulated chaotic systems (e.g., attractors), molecular dynamics, and sea surface temperature forecasting. The anonymous code for our implementation is available at [this link](https://anonymous.4open.science/r/IDFF-D2E5/README.md).

### 5.1 Image Generation

IDFFs exhibit superior performance compared to CFMs in image generation. We trained IDFF models on datasets such as CIFAR-10 and CelebA (additional results for ImageNet-64, CelebA-HQ, LSUN Bedrooms, and LSUN Church are available in Appendix A.5). We utilized a CNN-based UNet (Dhariwal and Nichol, ) to simultaneously model both $\hat{{\mathbf{x}}}_{1}(.,t;\theta)$ and $\bm{\epsilon}(.,t;\theta)$ from Equation 11 for all image data except ImageNet-64. For ImageNet-64, we employed a DiT architecture (’DiT-L/4’)(Peebles and Xie, ). To implement this, we doubled the input channels of the networks and fed the augmented input, $({\mathbf{x}}_{t},{\mathbf{x}}_{t})$, into the network. The outputs were then split into $(\hat{{\mathbf{x}}}_{1}(.,t;\theta),\bm{\epsilon}(.,t;\theta))$.

**Table 2: Comparison of FID and NFE metrics between IDFF and various methods on the CelebA ($64\times 64$) dataset.**
| Model | FID$\downarrow$ | NFE$\downarrow$ |
| --- | --- | --- |
| DDPM | 45.20 | 100 |
| DDIM | 13.73 | 20 |
| DDIM | 17.33 | 10 |
| FastDPM | 12.83 | 50 |
| IDFF (Ours) | 11.83 | 10 |

To ensure a fair comparison with other models, we used commonly employed metrics to evaluate CFMs’ performance, including negative log-likelihood (NLL) using equation 12 measured in bits per dimension (BPD) Lipman et al. (), sample quality as measured by the Frechet Inception Distance (FID), and the average NFE required for the reported FID and NLL, averaged over 50k samples, to assess the computational efficiency of our method. We generate 50k images to calculate the FID against the corresponding reference statistics of each dataset. All models use the same architecture, with specific details provided in Appendix A.4.

We also examined the effect of $\sigma_{0}$ on sample quality for both CIFAR-10 and CelebA datasets in Appendix A.2, finding that a value of $\sigma_{0}=0.6$ achieved the best FID for both datasets, as shown in Table 1 and Table 2. IDFF achieves comparable FID scores with ten times fewer NFEs than CFM and DDPM on the CIFAR-10 and CelebA datasets. IDFF’s qualitative results are provided in Figure 1 (additional samples are available in Appendix A.5).

Figure 4 (A) displays the FID score for the initial 50k generated samples plotted against NFEs for both the CIFAR-10 and CelebA datasets. Sample quality improves rapidly when the NFE exceeds 4, highlighting the significantly lower sampling cost of IDFF compared to traditional CFMs and diffusion-based models.

### 5.2 Time Series Generation

To assess the performance of IDFF in modeling time series data, we evaluated IDFF on three distinct tasks: 3D-attractor generation (see Appendix A.6), molecular dynamics simulation, and sea surface temperature forecasting.

**Table 3: MAE, RMSE, and CC between true and predicted trajectories for the MD simulation.**
| Method | MAE$\downarrow$ | RMSE $\downarrow$ | CC (%)$\uparrow$ |
| --- | --- | --- | --- |
| SRNN | 82.6±28 | 91.9±25 | 10.2±0.27 |
| DVAE | 78.1±27 | 88.1±25 | 30.4±0.35 |
| NODE | 25.3±6.3 | 28.8±6.2 | 10.5±0.41 |
| OT-CFM |  |  |  |
| NFE=50 | 13.3±1.1 | 16.3±2.4 | 86.1±0.1 |
| NFE=10 | 15.5±1.3 | 19.2±2.7 | 82.4±0.2 |
| IDFF |  |  |  |
| NFE=5 | 10.3±1.3 | 14.9±2.7 | 93.2±0.05 |
| NFE=12 | 9.5±1.1 | 13.4±2.8 | 95.2±0.1 |

#### 5.2.1 Molecular Dynamics Simulation

IDFFs can predict dynamics for complex combinatorial structures such as molecules. Molecular dynamics simulation is a crucial tool in quantum mechanics for understanding the dynamics of molecular behavior at the atomic level (Cazorla and Boronat, ). Here, we simulated a fully extended polyalanine structure for 400 picoseconds in a vacuum environment at a temperature of $T=300K$.
This molecular structure consists of 253 atoms with 46 dihedral angles. We trained an IDFF model to accurately generate the dihedral angles for the polyalanine structure from scratch. The distributions of the actual and generated dihedral angles for all 46 angles are presented in Figure 3 (A) and (B), respectively. Figure 3 (C) illustrates the dihedral angles for one of the alanine molecules and its associated dihedral angles. A complete trajectory showcasing a pair of actual and generated dihedral angles is depicted in Figure 3 (D).

To assess the performance of IDFF in the MD simulation, we utilized several evaluation metrics, including root mean squared error (RMSE), mean absolute error (MAE), and correlation coefficients (CC) between the generated and actual trajectories of dihedral angles. We benchmarked IDFF against well-known dynamical models for time series, including dynamical VAE models such as SRNNs (Fraccaro et al., ), VRNNs (Chung et al., ), and Neural ODEs (NODEs) (Garnelo et al., ). As shown in Table 3, IDFF outperforms the baselines by a large margin, indicating its potential for simulating molecular dynamics from scratch.

Figure: Figure 3: (A) True and (B) generated dihedral angles. (C) The dihedral angles for an alanine molecule. (D) True and generated dihedral angle trajectories using IDFF.
Refer to caption: extracted/5899840/figures/md_sim.png

#### 5.2.2 Sea Surface Temperature Forecasting

IDFFs can predict spatiotemporal dynamics for sea-surface temperature.
This experiment centers on forecasting sea surface temperature (SST) (Worsfold et al., ). Using the NOAA OISSTv2 dataset, which contains daily high-resolution SST images spanning from 1982 to 2021 (Cachay et al., ), we partitioned the data to predict SST intervals from 1 to 7 days ahead. To achieve this, we strategically subsampled eleven grid tiles of size $60\times 60$ (latitude $\times$ longitude) from the eastern tropical Pacific region (Huang et al., ).

We used a class-conditional UNet for the SST experiment, with the class encoder part encoding the long-range time dependencies (i.e., for encoding the days) to allow continuous forecasting using the UNet. See Appendix A.4 for the experiment’s network structure and hyperparameter settings. We used the training 4 and sampling 3 algorithms suggested for time-series data. We evaluated the performance of IDFF using the best validation continuous ranked probability score (CRPS) (Matheson and Winkler, ) and mean squared error (MSE) on forecasts up to 7 days. CRPS is a proper scoring rule and a popular metric in probabilistic forecasting literature (Gneiting and Katzfuss, ). We calculate the CRPS by generating a 20-member ensemble and the MSE on the ensemble mean predictions in Table 4.

To assess IDFF’s performance, we compared it against various baseline approaches, including DDPM, DDPM with enabled dropout (Dropout) (Gal and Ghahramani, ), DDPM with random perturbation for the initial values (Perturbation) (Pathak et al., ), MCVD (Voleti et al., ), Dyffusion (Cachay et al., ), and OT-CFM (with $NFE=40$) as shown in Table 4. The results for all the baselines are with $NFE=1000$. These baselines provide a robust framework for evaluating the IDFF framework in time-series forecasting. The results show that IDFF (with $NFE=5$) outperforms the baselines in SST forecasting across both metrics, demonstrating great potential for weather forecasting applications. See samples of the forecasting results in Figure 13.

**Table 4: Results for sea surface temperature forecasting of 1 to 7 days ahead, averaged over the evaluation horizon.**
| Method | CRPS$\downarrow$ | MSE$\downarrow$ |
| --- | --- | --- |
| Perturb. | 0.281 $\pm$ 0.004 | 0.180 $\pm$ 0.011 |
| Dropout | 0.267 $\pm$ 0.003 | 0.164 $\pm$ 0.004 |
| DDPM | 0.246 $\pm$ 0.005 | 0.177 $\pm$ 0.005 |
| MCVD | 0.216 | 0.161 |
| Dyffusion | 0.224 $\pm$ 0.001 | 0.173 $\pm$ 0.001 |
| OT-CFM | 0.231 $\pm$ 0.005 | 0.175 $\pm$ 0.006 |
| IDFF (Ours) | 0.180 $\pm$ 0.024 | 0.105 $\pm$ 0.029 |

## 6 Conclusion and Discussion

IDFF models represent a significant advancement in generative modeling, particularly for time series data. By offering superior computational efficiency compared to traditional flow models like CFMs, IDFF achieves a favorable balance between computational cost and sample quality. This makes IDFF a compelling choice for tasks requiring efficient and high-quality sample generation.
There are myriad future possibilities for the application of IDFF in domains such as music generation (Briot et al., ), speech synthesis (Yu and Deng, ), and modeling biological time-series data (Anumanchipalli et al., ; Golshan et al., ; Rezaei et al., , ; Gracco et al., ).

*Limitations and Future Work:*
Computing the OT map for large datasets can be challenging. To address this, we use a minibatch approximation of OT; however, increasing the batch size results in longer training times for larger batches. Second, our experiments primarily utilized a CNN-based UNet structure (Dhariwal and Nichol, ) as the backbone of the models. We anticipate that some of these limitations may be mitigated by utilizing new neural network structures (Peebles and Xie, ).

*Broader Impacts:* The computational efficiency of IDFF models makes it attractive for use in applications where speed and high sample quality are crucial, e.g., in resource-constrained environments. As with any powerful generative technology, the ethical implications must be considered. Safeguards and incentives should be established to prevent misuse of this technology. The synthetic data generated using IDFF must be used following societal norms and ethical standards.

## NeurIPS Paper Checklist

- 1.
Claims
- Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
- Answer: [Yes]
- Justification: The paper provides evidence for all the claims made in the abstract mainly in the experiments section.
- Guidelines:
•
The answer NA means that the abstract and introduction do not include the claims made in the paper.
•
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
•
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
•
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
- 2.
Limitations
- Question: Does the paper discuss the limitations of the work performed by the authors?
- Answer: [Yes]
- Justification: See the conclusion section
- Guidelines:
•
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
•
The authors are encouraged to create a separate "Limitations" section in their paper.
•
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
•
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
•
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
•
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
•
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
•
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
- 3.
Theory Assumptions and Proofs
- Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
- Answer: [Yes]
- Justification: We provided proof for our method.
- Guidelines:
•
The answer NA means that the paper does not include theoretical results.
•
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
•
All assumptions should be clearly stated or referenced in the statement of any theorems.
•
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
•
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
•
Theorems and Lemmas that the proof relies upon should be properly referenced.
- 4.
Experimental Result Reproducibility
- Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
- Answer: [Yes]
- Justification: See the experiment and method sections.
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
•
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
•
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
•
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a)
If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b)
If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c)
If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
- 5.
Open access to data and code
- Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
- Answer: [Yes]
- Justification: Upon acceptance of the paper, we provided the GitHub repository enabling the reproduction of all the results presented in the paper.
- Guidelines:
•
The answer NA means that paper does not include experiments requiring code.
•
Please see the NeurIPS code and data submission guidelines ([https://nips.cc/public/guides/CodeSubmissionPolicy](https://nips.cc/public/guides/CodeSubmissionPolicy)) for more details.
•
While we encourage the release of code and data, we understand that this might not be possible, so “No” is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
•
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ([https://nips.cc/public/guides/CodeSubmissionPolicy](https://nips.cc/public/guides/CodeSubmissionPolicy)) for more details.
•
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
•
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
•
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
•
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
- 6.
Experimental Setting/Details
- Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
- Answer: [Yes]
- Justification: The experiment section and appendices fully support this.
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
•
The full details can be provided either with the code, in appendix, or as supplemental material.
- 7.
Experiment Statistical Significance
- Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
- Answer: [Yes]
- Justification: See the result tables 4, 3
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
•
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
•
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
•
The assumptions made should be given (e.g., Normally distributed errors).
•
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
•
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
•
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
•
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
- 8.
Experiments Compute Resources
- Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
- Answer: [Yes]
- Justification: See Table5
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
•
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
•
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper).
- 9.
Code Of Ethics
- Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics [https://neurips.cc/public/EthicsGuidelines](https://neurips.cc/public/EthicsGuidelines)?
- Answer: [Yes]
- Justification: We reviewed the NeurIPS Code of Ethics.
- Guidelines:
•
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
•
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
•
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).
- 10.
Broader Impacts
- Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
- Answer: [Yes]
- Justification: The papers societal impacts are discussed in the last section of the paper.
- Guidelines:
•
The answer NA means that there is no societal impact of the work performed.
•
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
•
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
•
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
•
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
•
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
- 11.
Safeguards
- Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
- Answer: [N/A]
- Justification: The paper presents a methodological approach to learning diffusion models. It is not instantiated on problems with very large datasets such as ImageNet and ThePile.
- Guidelines:
•
The answer NA means that the paper poses no such risks.
•
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
•
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
•
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
- 12.
Licenses for existing assets
- Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
- Answer: [Yes]
- Justification: All datasets used from other sources are properly referenced in the paper.
- Guidelines:
•
The answer NA means that the paper does not use existing assets.
•
The authors should cite the original paper that produced the code package or dataset.
•
The authors should state which version of the asset is used and, if possible, include a URL.
•
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
•
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
•
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, [paperswithcode.com/datasets](paperswithcode.com/datasets) has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
•
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
•
If this information is not available online, the authors are encouraged to reach out to the asset’s creators.
- 13.
New Assets
- Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
- Answer: [Yes]
- Justification: The experiment section provides details of the datasets and experiments. Additionally, model details are also provided in appendix A.4.
- Guidelines:
•
The answer NA means that the paper does not release new assets.
•
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
•
The paper should discuss whether and how consent was obtained from people whose asset is used.
•
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
- 14.
Crowdsourcing and Research with Human Subjects
- Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
- Answer: [N/A]
- Justification: We don’t have any human subject
- Guidelines:
•
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
•
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
•
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
- 15.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
- Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
- Answer: [N/A]
- Justification: We don’t have any human subject
- Guidelines:
•
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
•
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
•
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
•
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

## Appendix A Appendix

### A.1 Proofs

#### A.1.1 Continuity proof for Lemma 1

We define a vector field as follows:

$$ $\tilde{\mathbf{v}}_{t}(\mathbf{x}_{t})=\sqrt{1-\sigma_{t}^{2}}{\mathbf{v}}_{t} (\mathbf{x}_{t})+\sigma_{t}^{2}\bm{\xi}_{t},\quad\bm{\xi}_{t}=\gamma\nabla\log p _{t}(\mathbf{x}_{t})$ (13) $$

where $\gamma$ is a coefficient controlling the contribution of the momentum term $\bm{\xi}_{t}$.

Consider a Stochastic Differential Equation (SDE) of the standard form:

$$ $d\mathbf{x}_{t}=\tilde{\mathbf{v}}_{t}(\mathbf{x}_{t})\,dt+\sigma_{t}\,d \mathbf{w}$ (14) $$

with time parameter $t$, drift $\tilde{\mathbf{v}}_{t}$, diffusion coefficient $\sigma_{t}$, and $d\mathbf{w}$ representing the Wiener process.

The solution $\mathbf{x}_{t}$ to the SDE is a stochastic process with probability density $p_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})$, characterized by the Fokker-Planck equation:

$$ $\frac{\partial p_{t}(\mathbf{x}_{t})}{\partial t}=-\nabla\cdot(\tilde{\mathbf{ v}}_{t}(\mathbf{x}_{t})p_{t}(\mathbf{x}_{t}))+\frac{\sigma_{t}^{2}}{2}\Delta p _{t}(\mathbf{x}_{t})$ (15) $$

where $\Delta$ represents the Laplace operator in $\mathbf{x}_{t}$.

We can rewrite this equation in the form of the continuity equation:

$$ $\displaystyle\frac{\partial p_{t}(\mathbf{x}_{t})}{\partial t}$ $\displaystyle=-\nabla\cdot\left(\tilde{\mathbf{v}}_{t}(\mathbf{x}_{t})p_{t}( \mathbf{x}_{t})-\frac{\sigma_{t}^{2}}{2}\frac{\nabla p_{t}(\mathbf{x}_{t})}{p_ {t}(\mathbf{x}_{t})}p_{t}(\mathbf{x}_{t})\right)$ (16) $\displaystyle=-\nabla\cdot\left(\left(\tilde{\mathbf{v}}_{t}(\mathbf{x}_{t})- \frac{\sigma_{t}^{2}}{2}\nabla\log p_{t}(\mathbf{x}_{t})\right)p_{t}(\mathbf{x }_{t})\right)$ (17) $\displaystyle=-\nabla\cdot\left(\mathbf{w}_{t}p_{t}(\mathbf{x}_{t})\right)$ (18) $$

where the vector field $\mathbf{w}_{t}$ is defined as:

$$ $\mathbf{w}_{t}(\mathbf{x}_{t})=\tilde{\mathbf{v}}_{t}(\mathbf{x}_{t})-\frac{ \sigma_{t}^{2}}{2}\nabla\log p_{t}(\mathbf{x}_{t})=\sqrt{1-\sigma_{t}^{2}}{ \mathbf{v}}_{t}(\mathbf{x}_{t})+\frac{2\gamma-1}{2}\sigma_{t}^{2}\nabla\log p_ {t}(\mathbf{x}_{t})$ (19) $$

As $t$ approaches 0 or 1, $\sigma_{t}\rightarrow 0$, causing $\tilde{\mathbf{v}}_{t}(\mathbf{x}_{t})$ to converge to $\hat{\mathbf{v}}_{t}(\mathbf{x}_{t})$. This ensures that the continuity equation is satisfied with the probability path $p_{t}(\mathbf{x}_{t})$ for the original equation.

#### A.1.2 Training objective derivations

As described in Theorem 2 of [Lipman et al., ], the FM loss is defined as:

$$ $\mathcal{L}_{\text{FM}}(\theta)=\mathbb{E}_{t,\mathbf{x}_{1},p_{t}(\mathbf{x}_ {t}|\mathbf{x}_{1})}\big{\|}\mathbf{v}_{t}(\mathbf{x}_{t})-\hat{\mathbf{v}}_{t }(\mathbf{x}_{t})\big{\|}^{2}$ (20) $$

The CFM loss, as defined in equation 5, is equivalent to the FM loss up to a constant value, implying that $\mathcal{L}_{\text{FM}}(\theta)=\mathcal{L}_{\text{CFM}}(\theta)$. Based on this equivalence, we can express the training objective for approximating the vector field in equation 7, $\hat{f}(\mathbf{x}_{t},t\mid\mathbf{x}_{1},\mathbf{x}_{0};\theta)$, as:

$$ $\mathcal{L}_{\text{IDFF}}^{1}(\theta)=\mathbb{E}_{t,\mathbf{x}_{1},p_{t}( \mathbf{x}_{t}|\mathbf{x}_{1})}\big{\|}\hat{\mathbf{v}}_{t}(\mathbf{x}_{t}\mid \mathbf{x}_{1};\theta)-\mathbf{v}_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})\big{\|}^{2}$ (21) $$

To recover unbiased samples, we replace $\hat{u}(.)$ with equation 10 and $\hat{\mathbf{v}}_{t}(\mathbf{x}_{t}|\mathbf{x}_{1},\mathbf{x}_{0})$ with equation 9, resulting in:

$$ $\mathcal{L}_{\text{IDFF}}^{1}(\theta)=\mathbb{E}_{t,\mathbf{x}_{1},p_{t}( \mathbf{x}_{t}|\mathbf{x}_{1})}\bigg{[}\bigg{\|}\frac{\hat{\mathbf{x}}_{1}( \mathbf{x}_{t},t;\theta)-\mathbf{x}_{t}}{1-t}-\frac{\mathbf{x}_{1}-\mathbf{x}_ {t}}{1-t}\bigg{\|}^{2}\bigg{]}$ (22) $$

$$ $=\mathbb{E}_{t,\mathbf{x}_{1},p_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})}\big{[} \beta(t)^{2}\big{\|}\hat{\mathbf{x}}_{1}(\mathbf{x}_{t},t;\theta)-\mathbf{x}_{ 1}\big{\|}^{2}\big{]}$ (23) $$

where $\beta(t)=\frac{1}{1-t}$. Therefore, we have:

$$ $\mathcal{L}_{\text{IDFF}}^{1}(\theta)=\mathbb{E}_{t,\mathbf{x}_{1},p_{t}( \mathbf{x}_{t}|\mathbf{x}_{1})}\big{[}\beta(t)^{2}\big{\|}\hat{\mathbf{x}}_{1} (\mathbf{x}_{t},t;\theta)-\mathbf{x}_{1}\big{\|}^{2}\big{]}$ (24) $$

To approximate $\nabla_{\mathbf{x}}\log p_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})$, we can employ a time-dependent score-based model, $\epsilon(\mathbf{x}_{t},t;\theta)$, using a continuous loss function with a weighting schedule $\lambda(t)$. Since $\nabla_{\mathbf{x}}\log p_{t}(\mathbf{x}_{t}|\mathbf{x}_{1})$ approaches infinity as $t$ tends to 0 or 1, it is necessary to standardize the loss to maintain consistency over time. We set $\lambda(t)$ such that the target has zero mean and unit variance, predicting the noise added in sampling $\mathbf{x}_{t}$ before multiplying by $\sigma_{t}=\sigma_{0}\sqrt{t(1-t)}$. This leads to $\lambda(t)=\sigma_{t}=\sigma_{0}\sqrt{t(1-t)}$, ensuring that the regression target for $\hat{\bm{\epsilon}}$ is distributed as $\mathcal{N}(\mathbf{0},\mathbf{I})$. Notably, this loss function is independent of the $\gamma$ value, allowing us to use the same model with different $\gamma$ values during the sampling process without altering the minima. The loss is defined as:

$$ $\mathcal{L}_{\text{IDFF}}^{2}(\theta)=\mathbb{E}_{t,\mathbf{x}_{1},p_{t}( \mathbf{x}_{t}|\mathbf{x}_{1})}\big{[}\lambda(t)^{2}\left\|\hat{\bm{\epsilon}} (\mathbf{x}_{t},t;\theta)-\nabla_{\mathbf{x}}\log p_{t}(\mathbf{x}_{t}|\mathbf {x}_{1})\right\|^{2}\big{]}$ (25) $$

where $\lambda(\cdot)$ is a set of positive weights. With a proper choice of $\lambda(t)$, this loss is equivalent to the original DDPM loss up to a constant value, according to Theorem 1 of [Song et al., ].
Combining the two components, the IDFF loss function is defined as:

$$ $\displaystyle\mathcal{L}_{\text{IDFF}}(\theta)=\mathcal{L}_{\text{IDFF}}^{1}( \theta)+\mathcal{L}_{\text{IDFF}}^{2}(\theta)$ (26) $$

Which leads to the final objective function:

$$ $\displaystyle\mathcal{L}_{\text{IDFF}}(\theta)=\mathbb{E}_{t,\mathbf{x}_{1},p_ {t}(\mathbf{x}_{t}|\mathbf{x}_{1})}\big{[}\beta(t)^{2}\|\hat{\mathbf{x}}_{1}( \mathbf{x}_{t},t;\theta)-\mathbf{x}_{1}\|^{2}+\lambda(t)^{2}\left\|\epsilon( \mathbf{x}_{t},t;\theta)-\nabla_{\mathbf{x}}\log p_{t}(\mathbf{x}_{t}|\mathbf{ x}_{1})\right\|^{2}\big{]}$ (27) $$

### A.2 Balancing the IDFF flow

Figure: Figure 4: IDFF performance against A) the number of function evaluations (NFEs) and B) the hyperparameter $\sigma_{0}$ (the flow and score model mixing term) with fixed NFE=10 for the CIFAR-10 dataset. Similar analysis for CelebA-64 is shown in Figure5
Refer to caption: extracted/5899840/figures/FigureFID_cifar.png

IDFF consists of a scoring model and an implicit flow component coupled with a parameter $\sigma_{0}$ to generate flow $\bm{{\mathbf{w}}}(.)$. We examine the effect of $\sigma_{0}$ on sample quality for the CIFAR-10 and CelebA datasets, with the NFE fixed at 10. As shown in Figure 4 (B), a value of $\sigma_{0}=0.6$ achieves the best balance between the diffusion and flow components regarding sample quality for both datasets. This plot also demonstrates that sample quality is susceptible to $\sigma_{0}$, particularly for diffusion-based models like DDPMs, compared to CFMs, especially at small NFEs, which is already noticed in some research[Jolicoeur-Martineau et al., , Jain et al., ].

Figure: Figure 5: IDFF performance against A) the number of function evaluations (NFEs) and B) the hyperparameter $\sigma_{0}$ (the flow and score model mixing term) with fixed NFE=10, for CelebA-64 dataset.
Refer to caption: extracted/5899840/figures/FigureFID_cifar.png

### A.3 IDFF training and sampling algorithms for time series

Figure: Algorithm 3 IDFF sampling algorithm for time-series data.

Figure: Algorithm 4 IDFF training algorithm for time-series data.

### A.4 Implementation details

Network configuration:
We adopt UNet (ADM) for our image generation and SST forecasting experiments. Table 5 shows detailed configurations of the ADM network on different datasets.

**Table 5: ADM network configuration for different datasets.**
|  | CIFAR-10 | CelebA64 | CelebA 256 | Church & Bed | SST |
| --- | --- | --- | --- | --- | --- |
| # of ResNet blocks | 2 | 2 | 2 | 2 | 2 |
| Base channels | 128 | 128 | 128 | 256 | 128 |
| Channel multiplier | 1,2,2,4 | 1,2,2,4,4 | 1,1,2,2,4,4 | 1,1,2,2,4,4 | 1,2,4 |
| Attention resolutions | 16 | 16 | 16 | 16 | 16 |
| Label dimensions | 1 | 1 | 1 | 1 | 10 |
| Params (M) | 65.6 | 102.14 | 453.45 | 108.41 | 55.39 |

Training hyper-params. In Table 6, we provide training hyperparameters for unconditional image generation on ADM and SST forecasting datasets.

**Table 6: Hyper-parameters of ADM network.**
|  | ImageNet-64 | CIFAR-10 | CelebA64 | CelebA 256 | Church & Bed | SST |
| --- | --- | --- | --- | --- | --- | --- |
| lr | $1\text{e-4}$ | $1\text{e-4}$ | $1\text{e-}5$ | $1\text{e-}5$ | $1\text{e-}5$ | 1e-4 |
| Batch size | 64 | 128 | 128 | 16 | 16 | 8 |
| # of iterations | 3M | 700K | 1.2M | 1.5M | 1.5M | 700K |
| # of GPUs | 4 | 1 | 1 | 1 | 1 | 1 |

### A.5 Additional results for image generation experiment

Figure: Figure 6: Generated samples for CelebA64 ($64\times 64$) dataset with different $\sigma_{0}$s and $NFE=10$.
Refer to caption: extracted/5899840/figures/celeba_samples.png

Figure: Figure 7: Generated samples for CIFAR-10 ($32\times 32$) dataset with different $\sigma_{0}$s and $NFE=10$.
Refer to caption: extracted/5899840/figures/cifar10_samples.png

Figure: Figure 8: Generated samples for CelebA-HQ ($256\times 256$) dataset with $\sigma_{0}=0.2$ and $NFE=10$.
Refer to caption: extracted/5899840/figures/celeb_HQ_samples.png

Figure: Figure 9: Generated samples for ImageNet64 dataset with $\sigma_{0}=0.2$ and $NFE=10$.
Refer to caption: extracted/5899840/figures/Imagenet_samples.png

Figure: Figure 10: Generated samples for LSUN-church ($256\times 256$) dataset with different $\sigma_{0}$s and $NFE=10$.
Refer to caption: extracted/5899840/figures/lsun_church_samples.png

Figure: Figure 11: Generated samples for LSUN-bed ($256\times 256$) dataset with different $\sigma_{0}$s and $NFE=10$.
Refer to caption: extracted/5899840/figures/lsun_bed_samples.png

### A.6 3D-attractors

In this experiment, we assess IDFF’s performance in generating trajectories of chaotic systems from scratch. We generate trajectories with $K=2000$ samples in each trajectory from 3D attractors, specifically the Lorenz and Rössler attractors, which are chaotic systems with nonlinear dynamics.
The parameters for the Lorenz and Rössler models are set to ${\sigma=10,\rho=28,\beta=8/3}$ and ${a=.2,b=.2,c=5.7}$, respectively, to produce complex trajectories in 3D space. We then train the IDFF model based on these trajectories.
To model each attractor, we use an MLP with two hidden layers of 128 dimensions and two separate heads for $\hat{{\mathbf{x}}}_{1}(.,t,k;\theta)$ and $\epsilon({\mathbf{x}}_{t},t,k;\theta)$. Additionally, we incorporate two separate embedding layers for embedding $t$ and $k$, which are directly concatenated with the first hidden layer of the MLP. The optimized IDFF successfully generates samples of these trajectories from scratch. The generated trajectories are shown in Figure 12.
The quality of the results demonstrates that IDFF can successfully simulate the behaviors of highly nonlinear and nonstationary systems such as attractors.

Figure: Figure 12: Time-series simulation. IDFF trajectory generation for the chaotic systems.
Refer to caption: extracted/5899840/figures/ts_sim.png

### A.7 SST forecasting visualization

Figure: Figure 13: SST forecasting result conditioned on day 1st for 9 days with $\sigma_{0}=0.2$ and fixed $NFE=5$. Same results for different $\sigma_{0}$s is shown in Figure14
Refer to caption: extracted/5899840/figures/sst_sigma02.png

Figure: Figure 14: SST forecasting result conditioned on day 1st for 9 days for different values of $\sigma_{0}$ and fixed $NFE=5$.
Refer to caption: extracted/5899840/figures/ssdt_samples.png

### A.8 2D-simulated static data and time-series

Additional results for 2D simulations for both static and time-series generation are shown in Figure15.

Figure: Figure 15: 2D synthetic simulation.
Refer to caption: extracted/5899840/figures/2d_synthetic.png

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: Please refer to weakness section.
soundness: 3
strengths: 1. The authors present a new objective for training and sampling in CFMs that combines some aspects of stochasticity of score-based, making the NFEs lower at inference and improving sample generation quality.
2. The authors have provided thorough proof of the derivations for the formulation.
3. The authors have not only shown that their method performs better in image generation but also that it can be used in tasks of other domains like time series.
summary: 1. The authors address two of the main challenges of prior CFMs:
a. During inference, CFMs require a large number of steps for high-quality/fidelity generation.
b. Due to a lack of stochasticity, CFMs may generate less diverse and detailed samples.
In essence, the authors present a modification to CFMs that combines the objective of CFM and score-based models, which have the benefits of CFMs for faster convergence during training while being able to use fewer NFEs during inference (an attribute of score-based models).
weaknesses: 1. Figure 2 C) is a bit confusing by itself. Only after reading Algorithm 2 was the figure easy to understand. The authors can try to make the figure self-sufficient in understandability.
2. The authors compare image generation quality w.r.t sampling strategies like DDIM and DPM and show that their model achieves the best quality. 3. However, they have not compared against better strategies like EDM. A comparison against EDM would be even more insightful.
How does the method proposed by authors compare to objectives like Rectified Flow for diffusion models, which have shown the ability to converge faster?
4. Are there any limitations to the author's methods compared to traditional CFMs after introducing some stochasticity? Does the training convergence speed become slower compared to traditional CFMs? The authors have not discussed this in the main paper.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: Follow up  weakness 2., what is the main contribution of the paper upon [2]?

[1] Tong, Alexander, et al. "Improving and generalizing flow-based generative models with minibatch optimal transport." arXiv preprint arXiv:2302.00482 (2023).

[2] Tong, Alexander, et al. "Simulation-free schr\" odinger bridges via score and flow matching." arXiv preprint arXiv:2307.03672 (2023).

[3] Lu, Cheng, et al. "Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps." Advances in Neural Information Processing Systems 35 (2022): 5775-5787.

[4] Lu, Cheng, et al. "Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models." arXiv preprint arXiv:2211.01095 (2022).

[5] Zheng, Kaiwen, et al. "Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics." Advances in Neural Information Processing Systems 36 (2023): 55502-55542.

[6] Zhao, Wenliang, et al. "Unipc: A unified predictor-corrector framework for fast sampling of diffusion models." Advances in Neural Information Processing Systems 36 (2024).
soundness: 3
strengths: The paper tackles the important problem of reducing NFE in sampling process of flow/bridge models. The method is compared on number of different domians.
summary: The paper derive framework for simulation free training for Schrödinger bridge. The proposed method trains a combined score matching loss and a denoiser loss, then samples by solving an SDE.
weaknesses: 1. The proposed loss seems similar to the loss proposed in  [2] for simulation free training of Schrödinger bridges up to parametrizing the the velocity field with a denoiser.

2. In the CIFAR10 experiment the author chose to compare to DPM-solver [3] only, while there are already two follow up works DPM++ [4] and DPM-v3-solver [5] which in [5] are reported to perform better than the proposed IDFF method. Additionally, Uni-PC solver [6] is reported to perform better.

3. of less importance but I also note there seems to be some inconsistencies or typos in the Background section.
    1. In equation 5 of conditional flow matching loss it seems as if the network is dependent on the target point $x_1$ and the conditional target velocity is independent of the target point $x_1$.
    2. In the paragraph "Optimal Transport CFMs" the author state that the flow matching loss in equation 5 "is nearly intractable", this state is a bit unclear since flow matching has been used for many large scale application such as image, video, and audio generation. Additionally, it is un clear whether the authors refer to the conditional optimal transport (also know as linear scheduler)  with an independent coupling of $x_0$ and $x_0$ or with the optimal transport between source and target distribution. On the one hand, equation 5 is written for an independent coupling, and on the other hand the authors cite [1] and the OT-CFM path which to my understanding is referred to the optimal transport coupling.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: Please review the weaknesses.
soundness: 3
strengths: 1. The idea presented is novel and effective, as demonstrated by the experiments.
2. The proofs provided in this paper are solid.
3. This paper also discusses the topic of time-series generation.
summary: This paper introduces a new method called IDFF, which facilitates rapid sampling without compromising sample quality and efficiently handles both image and time-series data generation tasks. The experimental results demonstrate the superiority of IDFF.
weaknesses: I believe the main drawback is that I cannot find an explanation for how this momentum term helps reduce the NFE. It appears that the motivation for designing this method is inspired by Hamiltonian Monte Carlo, which, by the way, is not mentioned in the background section. If I have overlooked this part, please let me know. If not, I recommend adding an explanation. If it is well justified, I will reconsider my score.

Here are some specific suggestions.
1. It would be beneficial to include a discussion in Section 3.1 that explicitly explains how the momentum term contributes to reducing the NFE without compromising sample quality, drawing parallels to Hamiltonian Monte Carlo's efficiency improvements. This discussion is crucial, as it constitutes a key claim of the paper.
2. Please include Hamiltonian Monte Carlo in the background section (Section 2) to provide context for readers and establish a clearer connection to the paper's approach.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: 1. Could you provide a clearer explanation of the transition from $\tilde{\mathbf{v}}_t\left(\mathbf{x}_t\right)$ to $\mathbf{w}_t\left(\mathbf{x}_t\right)$ in Section 3.1 of the paper?
soundness: 2
strengths: The paper proposes a novel approach for training Conditional Flow Matching (CFM), which adds an additional correction term to the vector field, allowing for larger step sizes during the sampling process, thereby accelerating sampling. The paper also conducts extensive experiments to validate the effectiveness of the method, including tasks in image generation and time-series datasets modeling.
summary: The paper introduces Implicit Dynamical Flow Fusion (IDFF) for generative modeling. IDFF learns a vector field with an additional momentum term, allowing for the use of larger step sizes during sampling without compromising the quality of the generated data, thereby accelerating the generation process. This method has demonstrated effectiveness across different domains, including image generation and time-series datasets modeling.
weaknesses: 1. Some parts of the paper need improvement in their descriptions. For instance, in Section 3.1, during the transition from $\tilde{\mathbf{v}}_t\left(\mathbf{x}_t\right)$ to $\mathbf{w}_t\left(\mathbf{x}_t\right)$, a stochastic differential equation (SDE) is introduced (in Appendix A.1.1), but this is not explicitly mentioned in the main text, nor is its purpose clearly explained. Perhaps the introduction of the SDE here is intended to eliminate the effect of introducing the momentum term on the probability density path $p_t(\mathbf{x}_t)$?
2. Some experiments in the paper are insufficient. For example, in the image generation task, there is no comparison with some classic Conditional Flow Matching (CFM) methods, such as rectified flow[1].

References:
[1] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003, 2022.

Meta-review: The paper introduces Implicit Dynamical Flow Fusion (IDFF) for generative modeling. IDFF learns a vector field augmented with a momentum term, enabling the use of larger step sizes during sampling without compromising the quality of the generated data. This innovation accelerates the generation process and demonstrates effectiveness across various domains, including image generation and time-series modeling.

Overall, while the proposed method shows promise, the current submission has several limitations:

1.	Similarity with Prior Work: The novelty of the approach is not sufficiently differentiated from existing methods.

2.	Limited Performance Gains for Slightly Larger NFEs: The method shows no significant improvement when the number of function evaluations (NFEs) is moderately large (e.g., 10).

3.	Missing Comparisons with State-of-the-Art (SOTA) Methods: The manuscript lacks a thorough comparison with more recent and competitive SOTA approaches, which weakens the evaluation.

4.	Clarity Issues: Certain sections, such as the discussion on the benefits of momentum for reducing NFEs (e.g., in relation to Hamiltonian Monte Carlo) and Section 3.1, are unclear and require further explanation.

Considering the current acceptance rate and these limitations, the paper is not yet ready for publication. We encourage the authors to address these concerns comprehensively, as outlined in the reviewers' feedback, to enhance the clarity, novelty, and rigor of the work for future submissions.

Decision: Reject