Generation Properties of Stochastic Interpolation under Finite Training Set

## Abstract

Abstract This paper investigates the theoretical behavior of generative models under finite training populations. Within the stochastic interpolation generative framework, we derive closed-form expressions for the optimal velocity field and score function when only a finite number of training samples are available. We demonstrate that, under some regularity conditions, the deterministic generative process exactly recovers the training samples, while the stochastic generative process manifests as training samples with added Gaussian noise. Beyond the idealized setting, we consider model estimation errors and introduce formal definitions of underfitting and overfitting specific to generative models. Our theoretical analysis reveals that, in the presence of estimation errors, the stochastic generation process effectively produces convex combinations of training samples corrupted by a mixture of uniform and Gaussian noise. Experiments on generation tasks and downstream tasks such as classification support our theory.

## 1 Introduction

In recent years, generative models have demonstrated remarkable capabilities across various domains, including image generation (Rombach et al., 2022; Peebles & Xie, 2023), image restoration (Wu et al., 2024b; a), text generation (Nie et al., 2025; Ye et al., 2023; Gong et al., 2024), and macromolecular prediction in biomedicine (Trippe et al., 2022). These models aim to learn the target distribution from observed samples and generate new data accordingly. Despite their impressive performance, several studies (Bhattacharjee et al., 2023; Meehan et al., 2020) have reported instances of data-copying behavior in generative models, raising serious concerns about data security, privacy leakage, and copyright infringement. More recent works (Zhang et al., 2023; Yoon et al., 2023; Kadkhodaie et al., 2023; Gu et al., 2023; Somepalli et al., 2023) have systematically examined the memorization behavior of generative models through extensive empirical experiments. These studies reveal that generative models may replicate samples from the training set, exhibiting strong memorization effects, which is particularly prominent with small-sample training sets.

A series of outstanding papers (Oko et al., 2023; Huang et al., 2023) have investigated the theoretical properties of generative models in the asymptotic regime when the number of training samples tends to infinity. However, practical applications often involve finite-sample settings, where the model can only approximate the empirical distribution defined by the training data rather than the true underlying distribution. This raises fundamental questions about the nature and behavior of generative models under such conditions. Several recent works (Gao & Li, 2024; Yi et al., 2023; Bertrand et al., 2025) have taken preliminary steps toward addressing these questions by studying the optimal learning objectives for diffusion and flow-based models in finite-sample regimes and analyzing their generative behaviors.

In this paper, we investigate the memorization and generalization behaviors of generative models through the stochastic interpolation framework(Albergo et al., 2023; Albergo & Vanden-Eijnden, 2022). This framework supports both deterministic and stochastic generation processes, and it encompasses a wide range of diffusion-based approaches, including score-based models (Song et al., 2020) and flow-based models (Lipman et al., 2022). It has also demonstrated strong empirical performance in image generation tasks (Ma et al., 2024; Yu et al., 2024). As such, the findings of this study offer both broad generalizability and practical relevance.

Our paper analyzes the properties of the stochastic interpolation model under the constraint of limited training samples. We theoretically demonstrate that the model memorizes the training data through its velocity field. Under ideal scenario without estimation error, deterministic generation reproduces the training samples exactly, while stochastic generation produces the training samples with additive Gaussian noise. Furthermore, we investigate the generative behavior under estimation errors and introduce formal definitions of underfitting and overfitting in the context of generative models. These findings lay a theoretical foundation for understanding memorization phenomena in generative modeling.

Contributions of our paper are summarized as follows

- •
We provide a systematic analysis of the optimal estimation for stochastic interpolation models with finite training populations, and we investigate the theoretical properties of both deterministic and stochastic generation.
- •
We analyze the effect of estimation errors on generation result and provide a theoretical understanding of overfitting and underfitting phenomena in generative models.
- •
We reveal that samples generated by stochastic interpolation models can be viewed as training samples perturbed by a combination of uniform and Gaussian noise. We validate our theoretical findings through downstream experiments on datasets such as MNIST, CIFAR-10 and Imagenet.

### 1.1 Notion

In this paper, we adopt the following notions. Let $W_{t}$ be the standard Brownian motion. $I_{d}$ denotes the $d$-dimensional identity matrix. The Minkowski sum of two sets is defined as $A*B=\{a+b\mid a\in A,\,b\in B\}$. The asymptotic notations $f(t)\lesssim g(t)$, $f(t)\gtrsim g(t)$, and $f(t)\asymp g(t)$ indicate that the relationship between $f(t)$ and $g(t)$ is understood up to constant multiplicative factors. The symbol $\delta(x)$ denotes the Dirac delta function. $f^{\prime}(x)$ represents the derivative of the function $f(x)$ and the gradient operator is denoted as $\nabla$. We use $P_{X}(\cdot)$ to denote the probability density of the random variable $X$. Specifically, we denote $\rho_{t}$ as the probability density of the random variable $Z_{t}$ involved in the differential equation.

## 2 Preliminary: Stochastic Interpolation

In this section we introduce the stochastic interpolation model. Given two random variables $Z_{0}$ and $Z_{1}$ with probability distribution $\rho_{0}$ and $\rho_{1}$, a stochastic interpolation is defined as

$$ $Z_{t}=\mathcal{I}(t,Z_{0},Z_{1})+\gamma(t){\eta},\;t\in[0,1],$ (1) $$

where ${\eta}$ is the standard Gaussian random variable independent of $Z_{0}$ and $Z_{1}$, $\gamma$ is a real-valued function defined on $[0,1]$ and $\mathcal{I}$ is the function that $[0,1]\times\mathbb{R}^{d}\times\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$. It has the following conditions.

1. The interpolation function $\mathcal{I}$ satisfies $\mathcal{I}(0,Z_{0},Z_{1})=Z_{0}$ and $\mathcal{I}(1,Z_{0},Z_{1})=Z_{1}$.

2. The function $\zeta$ satisfies one of the following conditions

- •
For $t\in\{0,1\}$, $\gamma(t)\equiv 0$.
- •
For $t\in(0,1)$, $\gamma(t)>0$ and for $\gamma(t)=0$ elsewhere.

A widely used form of stochastic interpolation is $\mathcal{I}(t,Z_{0},Z_{1})=\alpha(t)Z_{0}+\beta(t)Z_{1}$, which satisfies $\alpha(0)=1,\,\alpha(1)=0,\,\beta(0)=0,\,\beta(1)=1$. Therefore, $Z_{t}$ has the following form

$$ $Z_{t}=\alpha(t)Z_{0}+\beta(t)Z_{1}+\gamma(t)\eta.$ $$

Without loss of generality, we assume $\alpha(t)$ and $\beta(t)>0$ when $t\in(0,1)$. Based on the theoretical frameworks provided by the Fokker–Planck equation and the continuity equation, the velocity field $b(\bm{z},t)$ and the score function $s(\bm{z},t)$ are defined as follows

$$ $\displaystyle b(\bm{z},t)$ $\displaystyle=\mathbb{E}\Big[\alpha^{\prime}(t)Z_{0}+\beta^{\prime}(t)Z_{1}+{\gamma}^{\prime}(t){\eta}|Z_{t}=\bm{z}\Big],$ (2) $\displaystyle s(\bm{z},t)$ $\displaystyle=\nabla_{\bm{z}}\log\rho_{t}(\bm{z}).$ (3) $$

Starting from an initial point $Z_{1}\sim\rho_{1}$, the generated result obtained from the following generative models satisfy $Z_{0}\sim\rho_{0}$,

$$ $\displaystyle\text{Deterministic generation: }dZ_{t}$ $\displaystyle={b}(Z_{t},t)dt,$ (4) $\displaystyle\text{Stochastic generation: }dZ_{t}$ $\displaystyle=\Big({b}(Z_{t},t)-\zeta(t){s}(Z_{t},t)\Big)dt+\sqrt{2\zeta(t)}dW_{t}.$ (5) $$

In practical applications, the explicit forms of $b(\bm{z},t)$ and $s(\bm{z},t)$ are typically unknown and should be estimated through the minimization of the following loss function

$$ $\begin{split}&\hat{b}(\bm{z},t)=\arg\min_{b}\mathbb{E}\Big\|b(Z_{t},t)-\Big(\alpha^{\prime}(t)Z_{0}+\beta^{\prime}(t)Z_{1}+{\gamma}^{\prime}(t){\eta}\Big)\Big\|^{2},\\ &\hat{s}(\bm{z},t)=\arg\min_{s}\mathbb{E}\Big\|s(Z_{t},t)-\frac{1}{\gamma(t)}{\eta}\Big\|^{2}.\end{split}$ $$

The stochastic interpolation model constitutes a general class of generative models. When choosing $\alpha(t)=t$, $\beta(t)=1-t$ and $\gamma(t)\equiv 0$, the stochastic interpolation model coincides with the flow mathcing framework. When the initial distribution $\rho_{0}$ is Gaussian distribution and the time-dependent coefficients $\alpha(t)$ and $\beta(t)$ are suitably selected, the stochastic interpolation model degenerates to the score-based generative model. Therefore, the conclusions discussed in this paper under the stochastic interpolation framework can be readily extended to other diffusion generative models.

## 3 Generation Results under Finite Training Sets

### 3.1 Optimal Generation

In this section, we consider the ideal scenario in which the estimators $\hat{b}$ and $\hat{s}$ are assumed to be error-free, i.e., they exactly coincide with the true underlying functions $b$ and $s$. We begin by analyzing the setting where $\rho_{1}$ is a Gaussian distribution and $\rho_{0}$ corresponds to the data distribution, reflecting the classical generative modeling framework. In Section [A.1](https://arxiv.org/html/2509.21925v1#A1.SS1), we will extend our analysis to the more practical case where both $\rho_{0}$ and $\rho_{1}$ are empirical distributions constructed from finite populations. The expression of the velocity field $b$, as given in Eq. [2](https://arxiv.org/html/2509.21925v1#S2.E2), involves expectations with respect to both $\rho_{0}$ and $\rho_{1}$. In many real applications, we are limited to a finite training populations, and therefore $\rho_{0}$ should be characterized via discretized empirical distributions as follows.

$$ $\rho_{0}=\frac{1}{n}\sum_{i=1}^{n}\delta(X_{i}).$ (6) $$

Therefore, taking the expectation with respect to the distribution $\rho_{1}$ reduces to computing the empirical expectation over the training samples. It is worth noting that we are still taking the expectation with respect to the distribution $\rho_{1}$, as the Gaussian distribution permits infinite sampling during training. This allows us to obtain the optimal velocity field under finite samples, denoted as $b^{*}(\bm{z},t)$.

###### Proposition 1

When $\rho_{1}$ is Gaussian distribution and $\rho_{0}$ has the discretized empricial distribution form defined in Eq. [6](https://arxiv.org/html/2509.21925v1#S3.E6), the optimal velocity field has the following expression

$$ $b^{*}(\bm{z},t)=\sum_{i=1}^{n}\frac{1}{C_{3}(t)}\Big[C_{1}(t)\bm{z}-C_{2}(t)X_{i}\Big]\frac{\exp\big(-\frac{\|\bm{z}-\alpha(t)X_{i}\|^{2}}{2C_{3}(t)}\big)}{\sum_{j=1}^{n}\exp\big(-\frac{\|\bm{z}-\alpha(t)X_{j}\|^{2}}{2C_{3}(t)}\big)},$ (7) $$

where

$$ $\begin{split}&C_{1}(t)=\gamma(t)\gamma^{\prime}(t)+\beta^{\prime}(t)\beta(t),\\ &C_{2}(t)=\Big[\gamma(t)\gamma^{\prime}(t)+\beta^{\prime}(t)\beta(t)\Big]\alpha(t)-\Big[\gamma^{2}(t)+\beta^{2}(t)\Big]\alpha^{\prime}(t),\\ &C_{3}(t)=\gamma^{2}(t)+\beta^{2}(t).\end{split}$ $$

$s(\bm{z},t)$ can be expressed by $b(\bm{z},t)$ and $z$, which is also reported in Huang et al. (2023). Thus, we can obtain the optimal score function under finite populations, denoted as $s^{*}(\bm{z},t)$.

###### Proposition 2

When $\rho_{1}$ is Gaussian distribution and $\rho_{0}$ has the empirical distribution form defined in Eq. [6](https://arxiv.org/html/2509.21925v1#S3.E6), the optimal score function has the expression that

$$ $s^{*}(\bm{z},t)=\frac{\alpha(t)}{B(t)}b^{*}(\bm{z},t)-\frac{\alpha^{\prime}(t)}{B(t)}\bm{z},$ (8) $$

where

$$ $B(t)=\beta(t)\Big[\alpha^{\prime}(t)\beta(t)-\alpha(t)\beta^{\prime}(t)\Big]+\gamma(t)\Big[\gamma(t)\alpha^{\prime}(t)-\gamma^{\prime}(t)\alpha(t)\Big].$ $$

It can be observed that the optimal velocity $b^{*}(\bm{z},t)$ has a closded form of a weighted sum of $\{\bm{z}-X_{i}\}_{i=1}^{n}$, which is an expression that can only be derived under finite training sets. Due to the softmax form of the weighting terms, one specific $\bm{z}-X_{i}$ will dominate. Therefore there exists $i\in\{1\cdots n\}$ that the generated results will gradually approach $X_{i}$, which motivates the following theorem about deterministic generation.

###### Theorem 1

There exists $i\in\{1,\dots,n\}$ such that, based on the optimal velocity field $b^{*}(\bm{z},t)$, the deterministic generated result satisfies $Z_{0}=X_{i}$.

Furthermore, we now consider the case of stochastic generation. Since $s^{*}(\bm{z},t)$ is a linear combination of $b^{*}$ and $\bm{z}$, it also gradually becomes a weighted sum of $\{\bm{z}-X_{i}\}_{i=1}^{n}$. This observation motivates the following theorem about stochastic generation.

###### Theorem 2

Based on the optimal velocity field $b^{*}(\bm{z},t)$ and score $s^{*}(\bm{z},t)$, assume that $\zeta(t)\lesssim B(t)$ when $t\rightarrow 0$, the generated result of stochastic generation defined in Eq. [5](https://arxiv.org/html/2509.21925v1#S2.E5) will be the Minikowski sum of the training set and the Gaussian distribution. In particular, there exists $i\in\{1,\dots,n\}$ such that $Z_{0}\sim N(X_{i},\sigma^{2}I_{d})$, where $\sigma^{2}=2\int_{0}^{t}\zeta(t)dt$.

###### Remark 1

The assumption that $\zeta(t)\lesssim B(t)$ is satisfied in some common cases, such as $\gamma(t)=\sqrt{t(1-t)}$.

We use a two-dimensional toy data to help understanding and verifying Theorems [1](https://arxiv.org/html/2509.21925v1#Thmtheorem1) and [2](https://arxiv.org/html/2509.21925v1#Thmtheorem2). Specifically, we randomly select 5 points within the range $[0,1]^{2}$ as the training set and compute the corresponding $b^{*}(\bm{z},t)$ and $s^{*}(\bm{z},t)$ using Eq. [7](https://arxiv.org/html/2509.21925v1#S3.E7) and Eq. [8](https://arxiv.org/html/2509.21925v1#S3.E8). The deterministic and stochastic generation results are shown in Fig. [1](https://arxiv.org/html/2509.21925v1#S3.F1). When using deterministic generation, the model precisely reproduces the training samples, as the red generated points almost coincide with the blue points in Fig. [1(a)](https://arxiv.org/html/2509.21925v1#S3.F1.sf1). In the case of stochastic generation, the generated samples form elliptical Gaussian distributions centered around the training samples. Furthermore, as the noise level increases, the spread of the Gaussian distributions also increases accordingly.

Figure: (a)
Refer to caption: plot/oracle,eta=0.jpg

###### Proposition 1

###### Proposition 2

###### Theorem 1

###### Theorem 2

###### Remark 1

### 3.2 Generation with Estimation Error

In practical training, we often adopt various neural networks architectures to estimate the oracle $b^{*}(\bm{z},t)$. Therefore, we further consider the generation process with estimation error. For theoretical simplicity, we focus only on the deterministic generation scenario, since, as analyzed in Section [3.1](https://arxiv.org/html/2509.21925v1#S3.SS1), stochastic generation can be viewed as deterministic generation with added Gaussian noise.

Let $\hat{b}(\bm{z},t)$ denote the estimated veloity field and $\epsilon(\bm{z},t)$ denote as the estimation error, i.e.,

$$ $\epsilon(\bm{z},t):=\hat{b}(\bm{z},t)-b^{*}(\bm{z},t),$ $$

which characterizes the discrepancy between the estimated $\hat{b}(\bm{z},t)$ and the optimal $b^{*}(\bm{z},t)$. The following theorem reveals the impact of estimation error on the generated results.

###### Theorem 3

When using $\hat{b}(\bm{z},t)$ as the velocity field and performing deterministic generation defined by Eq. [4](https://arxiv.org/html/2509.21925v1#S2.E4), there exists $i\in\{1\dots,n\}$ such that the generated result ${Z}_{0}$ satisfies

$$ ${Z}_{0}=X_{i}-\lim_{t\rightarrow 0}\frac{C_{3}(t)}{C_{1}(t)}\epsilon(\bm{z},t),$ (9) $$

where $C_{1}(t)$ and $C_{3}(t)$ are defined in Proposition [1](https://arxiv.org/html/2509.21925v1#S3.Ex3).

This result indicates that the generated result is a sample from the training set combined with estimation error.
In the following, we state formal assumptions on $\epsilon(\bm{z},t)$ and investigate its impact on the generation result. The first assumption is the uniform boundness of the error term $\epsilon(\bm{z},t)$, which is commonly adopted in convergence analysis (Huang et al., 2023). Under this regularity condition, we establish the following theoretical result.

###### Corollary 1

Assume that there exists a constant $\lambda>0$ such that the approximation error $\epsilon(\bm{z},t)$ satisfies

$$ $\|\epsilon(\bm{z},t)\|^{2}\leq\lambda,\;\forall\bm{z},t\in\mathbb{R^{d}}\times[0,1],$ $$

Then the generation result of deterministic generation defined by Eq. [4](https://arxiv.org/html/2509.21925v1#S2.E4) will lie within the set $\{X_{i}\}_{i=1}^{n}$.

Corollary [1](https://arxiv.org/html/2509.21925v1#Thmcorollary1) establishes that, provided the error is uniformly bounded, the generated samples remain within the training set, irrespective of the magnitude of the bound. This result follows from the fact that, during the deterministic generation process, the error is explicitly corrected at each iteration. This property highlights a key strength of diffusion-based generative frameworks, namely their robustness to bounded perturbations during the generation trajectory. We visually illustrate the conclusion of Corollary [1](https://arxiv.org/html/2509.21925v1#Thmcorollary1) through experiments on two-dimensional data. Setting $\lambda$ to 1, 5, and 25 the generated results with different $\lambda$ are shown in Fig. [2](https://arxiv.org/html/2509.21925v1#S3.F2). Even when $\lambda$ is large, the generated results still coincide with the samples in the training set, which supports the conclusion of the Corollary [1](https://arxiv.org/html/2509.21925v1#Thmcorollary1).

Figure: (a)
Refer to caption: plot/error_type=1,error=1.jpg

The assumption that $\epsilon(\bm{z},t)$ is uniformly bounded constitutes a strong condition under which the aforementioned conclusions hold. However, in practice, $\epsilon(\bm{z},t)$ typically diverges due to the divergence of $b^{*}(\bm{z},t)$ as $t\to 0$ or $t\to 1$. To address this, we consider a more general setting in which the divergence of $\epsilon(\bm{z},t)$ is regulated by a function $\gamma(t)$. Under this formulation, we demonstrate that the quality and validity of the generation process are governed by the relative decay rates of $\beta(t)$ and $\gamma(t)$ as $t\to 0$.

###### Corollary 2

Assume that there exists a constant $\lambda>0$ such that the estimation error $\epsilon(\bm{z},t)$ satisfies

$$ $\|\epsilon(\bm{z},t)\|^{2}\leq\frac{\lambda}{\gamma(t)},\;\forall\bm{z},t\in\mathbb{R^{d}}\times[0,1].$ $$

Under the common choice of $\beta(t)=t$, the behavior of the generated samples depends critically on the relative scaling between $\gamma(t)$ and $\beta(t)$ as $t\rightarrow 0$:

- •
If $\gamma(t)\gtrsim\beta(t)$, the generated samples asymptotically converge to the training set.
- •
If $\gamma(t)\asymp\beta(t)$, the generated samples remain concentrated in the vicinity of the training set.
- •
If $\gamma(t)\lesssim\beta(t)$, the generated samples diverge, resulting in outputs that deviate substantially from the training set and lack semantic consistency.

In a more general scenario, the generation results are related to the relative scaling between $C_{1}(t)\gamma(t)$ and $C_{3}(t)$, as can be seen in Proof. [A.3.7](https://arxiv.org/html/2509.21925v1#A1.SS3.SSS7).

Corollary [2](https://arxiv.org/html/2509.21925v1#Thmcorollary2) offers theoretical guidance for the selection of the functions $\gamma(t)$ and $\beta(t)$ in the deterministic generation process. To develop an intuitive understanding of the implications of Theorem [2](https://arxiv.org/html/2509.21925v1#Thmcorollary2), we consider a fixed choice of $\beta(t)=t$ and evaluate three representative forms of $\gamma(t)$: $t(1-t)$, $\sqrt{t(1-t)}$, and $t(1-t)^{2}$. The corresponding empirical results are presented in Fig. [3](https://arxiv.org/html/2509.21925v1#S3.F3).

Figure: (a)
Refer to caption: plot/error_type=2,gamma=1.jpg

Next, we discuss a more complex but also more realistic error scenario, which is

$$ $\epsilon(\bm{z},t)\propto\frac{1}{\rho_{t}(\bm{z})}\propto{\sqrt{C_{3}(t)}}\left\{\sum_{i=1}^{n}\exp\left(\frac{\|\bm{z}-\alpha(t)X_{i}\|^{2}}{2C_{3}(t)}\right)\right\}^{-1}.$ $$

The intuition behind this scenario is that when the marginal density $\rho_{t}(\bm{z})$ is relatively high, the likelihood of observing the sample $\bm{z}$ during training increases, thereby leading to a smaller estimation error at the corresponding pair $(\bm{z},t)$. Based on this observation, we assume that the estimation error is inversely proportional to the density $\rho_{t}(\bm{z})$. Furthermore, we assume that

$$ $\epsilon(\bm{z},t)=\lambda\left\{\sum_{i=1}^{n}\exp\Big(-\frac{\|\bm{z}-\alpha(t)X_{i}\|^{2}}{2C_{3}(t)}\Big)\right\}^{-1},$ (10) $$

where $\lambda$ is used to control the norm of the estimation error. We neglect the denominator term ${\sqrt{C_{3}(t)\beta(t)}}$ in our analysis for the reason that it is a polynomial function of $t$, which is asymptotically dominated by the exponential term in the full expression’s denominator. To streamline the theoretical discussion, we therefore retain only the dominant exponential factor. Moreover, to better reflect practical implementation settings, we adopt the Euler method for discrete-time generation, i.e.

$$ $Z_{t-1}=Z_{t}-\hat{b}(\bm{z},t)h,$ $$

where $h$ is the step size. Then, we have the following conclusion.

###### Corollary 3

Assume that $\epsilon(\bm{z},t)=\lambda\left\{\sum_{i=1}^{n}\exp\Big(-\frac{\|\bm{z}-\alpha(t)X_{i}\|^{2}}{2C_{3}(t)}\Big)\right\}^{-1},$ the generated result of the deterministic method defined in Eq. [4](https://arxiv.org/html/2509.21925v1#S2.E4) satisfy

- •
Underfitting: There exists $\overline{\lambda}$ such that when $\lambda>\overline{\lambda}$, for all $i\in\{1,\dots,n\}$ and $t$, it holds that
$\|Z_{t-1}-X_{i}\|^{2}>\|Z_{t}-X_{i}\|^{2}.$
- •
Overfitting: Assume the start point in generation is bounded. For any $\tau>0$, there exists $\underline{\lambda}$ such that when $\lambda<\underline{\lambda}$, there exists $i\in\{1,\dots,n\}$ satisfying
$\|Z_{0}-X_{i}\|^{2}<\tau.$

The assumption that the start point is bounded is not particularly restrictive due to the finite numerical precision and constrained sampling ranges in practical implementations. Corollary [3](https://arxiv.org/html/2509.21925v1#Thmcorollary3) offers a comprehensive characterization of generative behavior within the stochastic interpolation framework. Specifically, if the estimation error during training is excessively large, the generated samples tend to diverge and fail to capture meaningful structure—this scenario corresponds to underfitting. On the other hand, if the estimation error is extremely small, the generated samples may concentrate too closely around the training data, effectively memorizing them and failing to produce novel outputs, which is referred to as overfitting.

We provide a visual validation of the conclusions drawn in Corollary [3](https://arxiv.org/html/2509.21925v1#Thmcorollary3), with the corresponding results presented in Fig. [4](https://arxiv.org/html/2509.21925v1#S3.F4). When the error bound $\lambda$ is large, the generated samples exhibit divergent behavior and tend toward infinity, which is the underfitting case. In contrast, when $\lambda$ is sufficiently small, the generated samples remain concentrated around points in the training set, which is the overfitting case. This is consistent with the theoretical predictions.

Figure: (a)
Refer to caption: plot/error_type=3,error1=0.01.jpg

Section [3.1](https://arxiv.org/html/2509.21925v1#S3.SS1) reveals that the stochastic generation process inherently injects Gaussian noise into the final outputs. Excluding the underfitting regime, generation with estimation error yields samples that lie within a ball around the training data, which can be effectively interpreted as the addition of uniform noise. Therefore, in practical scenarios, the essence of the generated samples from the generative model is the training samples perturbed by Gaussian noise and uniform noise, where the intensity of Gaussian noise is related to the stochasticity during generation, and the intensity of uniform noise is related to the fitting error.

The conclusions presented in this work do not challenge the fundamental validity of generative models. When the primary objective is sample generation, and the underlying data space exhibits sufficient continuity, producing samples in the vicinity of the training set can still be considered consistent with the data distribution and may yield high-quality outputs. Furthermore, as the size of the training dataset increases, the generated samples tend to better approximate the true data distribution. Therefore, in classical generative tasks, such as training models on large-scale datasets like ImageNet, it remains feasible to attain both high generation quality and sample diversity.

However, some prior works have utilized samples generated by generative models in downstream tasks, such as training classification models. Our theoretical findings suggest that the use of generated samples in such settings may be effectively equivalent to augmenting the training set with noise-perturbed versions of the original samples. Moreover, when the original training dataset is limited in size, generative models are prone to overfitting, producing samples that closely resemble the training data without introducing substantial novelty. This necessitates caution when applying generative models to downstream tasks.

###### Theorem 3

###### Corollary 1

###### Corollary 2

###### Corollary 3

## 4 Experiments

### 4.1 Performance of Generated Samples in Classification Tasks

In this section, we empirically validate our theoretical findings through experiments on real datasets. Specifically, we assess the effectiveness of generative models in downstream classification tasks using the MNIST and FashionMNIST datasets. To illustrate the sample composition, we consider a total training size of $100$ and the training configurations is defined as follow

- •
Half Samples: A baseline classifier trained on $50$ real samples.
- •
Full Samples: An upper-bound classifier trained on $100$ real samples.
- •
Generated Samples: A classifier trained on $50$ real samples and $50$ generated samples, where the generative model is trained on the $50$ real samples.
- •
Full Generated Samples: A classifier trained on $50$ real samples and $50$ generated samples, where the generative model is trained on the full dataset of $60{,}000$ samples.
- •
Noised Samples: A classifier trained on $50$ real samples and $50$ noisy versions of them.

The classification accuracies under these settings are reported in Tab. [1](https://arxiv.org/html/2509.21925v1#S4.T1) and Tab. [2](https://arxiv.org/html/2509.21925v1#S4.T2), where the values in the table represent the classification accuracy (%) on the test set.

**Table 1: Classification Accuracy on MNIST**
| Sample Size | Half Samples | Noised Samples | Generated Samples | Full Generated Samples | Full Samples |
| --- | --- | --- | --- | --- | --- |
| 100 | 68.84 | 75.3 | 68.61 | 73.74 | 77.57 |
| 200 | 77.57 | 83 | 78.33 | 81.8 | 81.7 |
| 400 | 81.7 | 84.96 | 80.43 | 84.88 | 85.97 |
| 1000 | 87.29 | 88.65 | 86.18 | 88.69 | 89.22 |
| 2000 | 89.22 | 90.44 | 89.78 | 90.48 | 91.3 |
| 4000 | 91.3 | 92.94 | 91.82 | 92.24 | 93.15 |
| 10000 | 93.15 | 94.23 | 93.36 | 93.42 | 94.48 |

**Table 2: Classification Accuracy on FashionMNIST**
| Sample Size | Half Samples | Noised Samples | Generated Samples | Full Generated Samples | Full Samples |
| --- | --- | --- | --- | --- | --- |
| 100 | 62.26 | 68.69 | 65.89 | 60.8 | 68.32 |
| 200 | 68.32 | 73.78 | 70.78 | 67.81 | 71.04 |
| 400 | 71.04 | 76.18 | 73.07 | 72.5 | 75.77 |
| 1000 | 77.64 | 79.6 | 78.55 | 76.9 | 80.14 |
| 2000 | 80.14 | 82.32 | 81.41 | 80 | 81.83 |
| 4000 | 81.83 | 84.21 | 82.65 | 81.5 | 82.64 |
| 10000 | 84.17 | 84.82 | 83.72 | 83.31 | 84.14 |

The experimental results demonstrate that synthetic samples generated by a generative model trained on the full dataset can enhance the performance of downstream classification tasks, thereby validating the data augmentation potential of high-quality generated samples. However, under small-sample conditions, the limited availability of training data leads the generative model to overfit, resulting in generated samples with low diversity that largely replicate the training distribution. This overfitting behavior is directly manifested in the downstream task performance: specifically, when the training data are scarce, the classification accuracy achieved using generation-augmented data is even lower than that obtained using simple noise-perturbation techniques.

### 4.2 Performance of Generated Samples in Contrastive Learning

We further assess the applicability of our theoretical framework in the context of contrastive learning. Prior work Wang et al. (2024) has shown that incorporating synthetic samples generated by a generative model, together with weaker data augmentation strategies, can enhance contrastive learning performance. Our theory offers a plausible explanation for this observation. Specifically, data augmentation in contrastive learning can be interpreted as generating new samples within a local neighborhood of the original data points. In our theory, the samples generated by the generative model can be regarded as the result of adding Gaussian noise within the unit ball neighborhood of the original training samples. Consequently, combining generated samples with weak augmentation yields a comparable perturbation magnitude to that of conventional strong augmentation strategies (e.g., the $\lambda_{k+1}$ in Wang et al. (2024)), while simultaneously introducing greater sample diversity. This dual effect contributes to improved generalization. We validate this claim through comparions between using generated samples and noised samples under different contrastive learning method on the CIFAR-10 dataset, including SwavCaron et al. (2020), SimsiamChen & He (2021), SimclrChen et al. (2020a) and Mocov2Chen et al. (2020b). Experiments results are reported in Tab. [3](https://arxiv.org/html/2509.21925v1#S4.T3), where the values in the table represent the classification accuracy (%) on the test set.

**Table 3: Results of Contrastive Learning on CIFAR-10**
|  | Swav | Simsiam | Simclr | Mocov2 |
| --- | --- | --- | --- | --- |
| Baseline | 88.94 | 88.82 | 90.14 | 91.72 |
| Weak Augmentation + Generated samples | 90.11 | 90.80 | 91.71 | 92.95 |
| Weak Augmentation + Noised samples | 90.09 | 90.68 | 91.41 | 92.84 |

The experimental results reveal that incorporating noisy samples in conjunction with weak data augmentation yields modest performance gains over the baseline, although it remains slightly inferior to directly leveraging samples generated by the generative model. This observation is consistent with expectations: the generative model, having been trained on the full dataset of 60,000 samples, possesses strong generative capacity. Nonetheless, the fact that noise based synthetic samples achieve performance comparable to that of the generative model provides indirect empirical support for our theoretical framework.

## 5 Discussion and Future Work

In this paper, we discuss the properties of generative models under limited sample training set based on the stochastic interpolation framework. We also provide a comprehensive understanding for the underfitting and overfitting phenomena in generative models with theoretical support. This conclusion can also be extended to other algorithms in the diffusion model series. However, due to space limitations, there are still some issues that this article does not address, including (1) whether mainstream generative models such as VAE, GAN, and Autoregressive exhibit similar memorization characteristics; (2) whether conditional generative models follow the same patterns under this theoretical framework. These open questions provide valuable directions for future research.

## References

- Albergo & Vanden-Eijnden (2022)
Michael S Albergo and Eric Vanden-Eijnden.
Building normalizing flows with stochastic interpolants.
*arXiv preprint arXiv:2209.15571*, 2022.
- Albergo et al. (2023)
Michael S Albergo, Nicholas M Boffi, and Eric Vanden-Eijnden.
Stochastic interpolants: A unifying framework for flows and diffusions.
*arXiv preprint arXiv:2303.08797*, 2023.
- Bertrand et al. (2025)
Quentin Bertrand, Anne Gagneux, Mathurin Massias, and Rémi Emonet.
On the closed-form of flow matching: Generalization does not arise from target stochasticity.
*arXiv preprint arXiv:2506.03719*, 2025.
- Bhattacharjee et al. (2023)
Robi Bhattacharjee, Sanjoy Dasgupta, and Kamalika Chaudhuri.
Data-copying in generative models: a formal framework.
In *International Conference on Machine Learning*, pp. 2364–2396. PMLR, 2023.
- Caron et al. (2020)
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin.
Unsupervised learning of visual features by contrasting cluster assignments.
*Advances in neural information processing systems*, 33:9912–9924, 2020.
- Chen et al. (2020a)
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.
A simple framework for contrastive learning of visual representations.
In *International conference on machine learning*, pp. 1597–1607. PmLR, 2020a.
- Chen & He (2021)
Xinlei Chen and Kaiming He.
Exploring simple siamese representation learning.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 15750–15758, 2021.
- Chen et al. (2020b)
Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He.
Improved baselines with momentum contrastive learning.
*arXiv preprint arXiv:2003.04297*, 2020b.
- Gao & Li (2024)
Weiguo Gao and Ming Li.
How do flow matching models memorize and generalize in sample data subspaces?
*arXiv preprint arXiv:2410.23594*, 2024.
- Gong et al. (2024)
Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, et al.
Scaling diffusion language models via adaptation from autoregressive models.
*arXiv preprint arXiv:2410.17891*, 2024.
- Gu et al. (2023)
Xiangming Gu, Chao Du, Tianyu Pang, Chongxuan Li, Min Lin, and Ye Wang.
On memorization in diffusion models.
*arXiv preprint arXiv:2310.02664*, 2023.
- Huang et al. (2023)
Ding Huang, Jian Huang, Ting Li, and Guohao Shen.
Conditional stochastic interpolation for generative learning.
*arXiv preprint arXiv:2312.05579*, 2023.
- Kadkhodaie et al. (2023)
Zahra Kadkhodaie, Florentin Guth, Eero P Simoncelli, and Stéphane Mallat.
Generalization in diffusion models arises from geometry-adaptive harmonic representations.
*arXiv preprint arXiv:2310.02557*, 2023.
- Lipman et al. (2022)
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le.
Flow matching for generative modeling.
*arXiv preprint arXiv:2210.02747*, 2022.
- Ma et al. (2024)
Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie.
Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers.
In *European Conference on Computer Vision*, pp. 23–40. Springer, 2024.
- Meehan et al. (2020)
Casey Meehan, Kamalika Chaudhuri, and Sanjoy Dasgupta.
A non-parametric test to detect data-copying in generative models.
In *International conference on artificial intelligence and statistics*, 2020.
- Nie et al. (2025)
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li.
Large language diffusion models.
*arXiv preprint arXiv:2502.09992*, 2025.
- Oko et al. (2023)
Kazusato Oko, Shunta Akiyama, and Taiji Suzuki.
Diffusion models are minimax optimal distribution estimators.
In *International Conference on Machine Learning*, pp. 26517–26582. PMLR, 2023.
- Peebles & Xie (2023)
William Peebles and Saining Xie.
Scalable diffusion models with transformers.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pp. 4195–4205, 2023.
- Rombach et al. (2022)
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-resolution image synthesis with latent diffusion models.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 10684–10695, 2022.
- Somepalli et al. (2023)
Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein.
Understanding and mitigating copying in diffusion models.
*Advances in Neural Information Processing Systems*, 36:47783–47803, 2023.
- Song et al. (2020)
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.
Score-based generative modeling through stochastic differential equations.
*arXiv preprint arXiv:2011.13456*, 2020.
- Trippe et al. (2022)
Brian L Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi Jaakkola.
Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem.
*arXiv preprint arXiv:2206.04119*, 2022.
- Wang et al. (2024)
Yifei Wang, Jizhe Zhang, and Yisen Wang.
Do generated data always help contrastive learning?
*arXiv preprint arXiv:2403.12448*, 2024.
- Wu et al. (2024a)
Rongyuan Wu, Lingchen Sun, Zhiyuan Ma, and Lei Zhang.
One-step effective diffusion network for real-world image super-resolution.
*Advances in Neural Information Processing Systems*, 37:92529–92553, 2024a.
- Wu et al. (2024b)
Rongyuan Wu, Tao Yang, Lingchen Sun, Zhengqiang Zhang, Shuai Li, and Lei Zhang.
Seesr: Towards semantics-aware real-world image super-resolution.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 25456–25467, 2024b.
- Yang et al. (2022)
Sidi Yang, Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang.
Maniqa: Multi-dimension attention network for no-reference image quality assessment.
In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 1191–1200, 2022.
- Ye et al. (2023)
Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, and Quanquan Gu.
Diffusion language models can perform many tasks with scaling and instruction-finetuning.
*arXiv preprint arXiv:2308.12219*, 2023.
- Yi et al. (2023)
Mingyang Yi, Jiacheng Sun, and Zhenguo Li.
On the generalization of diffusion model.
*arXiv preprint arXiv:2305.14712*, 2023.
- Yoon et al. (2023)
TaeHo Yoon, Joo Young Choi, Sehyun Kwon, and Ernest K Ryu.
Diffusion probabilistic models generalize when they fail to memorize.
In *ICML 2023 workshop on structured probabilistic inference $\{$$\backslash$&$\}$ generative modeling*, 2023.
- Yu et al. (2024)
Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie.
Representation alignment for generation: Training diffusion transformers is easier than you think.
*arXiv preprint arXiv:2410.06940*, 2024.
- Zhang et al. (2023)
Huijie Zhang, Jinfan Zhou, Yifu Lu, Minzhe Guo, Peng Wang, Liyue Shen, and Qing Qu.
The emergence of reproducibility and generalizability in diffusion models.
*arXiv preprint arXiv:2310.05264*, 2023.

## Appendix A Appendix

### A.1 Property of Finite Training Sets on Both End

In this section, we extend the results to the case where both $\rho_{0}$ and $\rho_{1}$ are finite sample distributions, which deduces the following proposition.

###### Proposition 3

When $\rho_{0}=\frac{1}{n}\sum_{i=1}^{n}\delta(X_{i})$ and $\rho_{1}=\frac{1}{m}\sum_{i=1}^{m}\delta(Y_{i})$, the optimal velocity has the following expression

$$ $\begin{split}b^{*}_{2}(\bm{z},t)=\frac{\gamma^{\prime}(t)}{\gamma(t)}\bm{z}+\sum_{i=1}^{n}\sum_{j=1}^{m}&\Big\{\Big[\alpha^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\alpha(t)\Big]X_{i}+\Big[\beta^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\beta(t)\Big]Y_{j}\Big\}\\ &\frac{\exp(-\frac{\|\bm{z}-\alpha(t)X_{i}-\beta(t)Y_{j}\|^{2}}{2\gamma^{2}(t)})}{\sum_{k=1}^{n}\sum_{l=1}^{m}\exp(-\frac{\|\bm{z}-\alpha(t)X_{k}-\beta(t)Y_{l}\|^{2}}{2\gamma^{2}(t)})}.\end{split}$ (11) $$

Similar to Theorem [1](https://arxiv.org/html/2509.21925v1#Thmtheorem1), we have the following corollary about the generation result.

###### Corollary 4

Using $b^{*}_{2}(\bm{z},t)$ as the velocity field and performing determinstic generation as defined by Eq. [4](https://arxiv.org/html/2509.21925v1#S2.E4), starting from a time $t=1$, the generated result will lie in $\{X_{i}\}_{i=1}^{n}$. Similarly, starting from time $t=0$, the generated result will lie in $\{Y_{i}\}_{i=1}^{m}$.

Corollary [4](https://arxiv.org/html/2509.21925v1#Thmcorollary4) provides the result for error-free deterministic generation. Similarly, following the conclusions in Sections [3.1](https://arxiv.org/html/2509.21925v1#S3.SS1) and [3.2](https://arxiv.org/html/2509.21925v1#S3.SS2), we can readily obtain the results for stochastic generation and generation with estimation error.

###### Proposition 3

###### Corollary 4

### A.2 The Relation between Generation Quality and Training Set

We further investigate the phenomena of memorization and generalization in generative model training by conducting experiments on the ImageNet dataset. Specifically, we sample 256, 1024, 4096 images randomly from ImageNet to construct the training sets, and adopt the SiT-B/2 Ma et al. (2024) architecture as the backbone of the generative model. To assess the quality of generated images, we employ the CLIPIQA Yang et al. (2022) metric, which provides a reference-free evaluation of perceptual quality, in contrast to metrics such as SSIM and PSNR that require ground-truth comparisons. For quantifying memorization, we follow the evaluation protocol introduced in Yoon et al. (2023). For a generated sample ${Z}$, find $X_{1}$ and $X_{2}$ in the training set with the smallest and second smallest $L_{2}$ distance. The sample ${Z}$ is marked as a memorized instance if it satisfies $\frac{\|X_{1}-Z\|^{2}}{\|X_{2}-Z\|^{2}}\leq\frac{1}{3}$ following Yoon et al. (2023). The visual results are presented in Fig. [5](https://arxiv.org/html/2509.21925v1#A1.F5).

Figure: (a)
Refer to caption: plot/memory_and_generalization.png

The results in the figure indicate that if the training set has very few samples, such as 256 samples, the model will quickly memorize the samples in the training set, leading to overfitting. As the number of training samples increases, with 1024 samples, although the quality of the training images improves, the memorization rate also rises. When there are 4096 samples, the model no longer memorizes the samples in the training set and achieves better generation metrics. This memorization phenomenon supports the motivation of this paper.

To further examine this behavior, we display generated samples alongside their closest counterparts in the training set, which is shown in Fig. [5(b)](https://arxiv.org/html/2509.21925v1#A1.F5.sf2). The results suggest that, particularly under limited-data settings and prolonged training, the model tends to memorize specific training instances, thereby compromising its generalization capability.

### A.3 Theoretical proofs

In this section, we provide the proofs of the theorems in the main text.

#### A.3.1 Proof of Proposition 1

Proof. According to the definition of $b(\bm{z},t)$, when $\rho_{1}$ is the Gaussian distribution, the optimal velocity field can be derived as

$$ $\begin{split}b^{*}(\bm{z},t)&=\mathbb{E}\Big[\alpha^{\prime}(t)Z_{0}+\beta^{\prime}(t)Z_{1}+\gamma^{\prime}(t)\eta|Z_{t}=\bm{z}\Big]\\ &=\frac{1}{p(Z_{t}=\bm{z})}\int\int\Big[\alpha^{\prime}(t)z_{0}+\beta^{\prime}(t)z_{1}+\gamma^{\prime}(t)\eta\Big]\rho_{0}(z_{0})\rho_{1}(z_{1})P_{\eta}\Big(\frac{\bm{z}-\alpha(t)z_{0}-\beta(t)z_{1}}{\gamma(t)}\Big)dz_{0}dz_{1}\\ &=\frac{1}{p(Z_{t}=\bm{z})}\int\int\Big\{\big[\alpha^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\alpha(t)\big]z_{0}+\big[\beta^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\beta(t)\big]z_{1}+\frac{\gamma^{\prime}(t)}{\gamma(t)}\bm{z}\Big\}\\ &\quad\quad\rho_{0}(z_{0})\rho_{1}(z_{1})P_{\eta}\Big(\frac{\bm{z}-\alpha(t)z_{0}-\beta(t)z_{1}}{\gamma(t)}\Big)dz_{0}dz_{1}\\ &=\left\{\int\int\rho_{0}(z_{0})\rho_{1}(z_{1})P_{\eta}\Big(\frac{\bm{z}-\alpha(t)z_{0}-\beta(t)z_{1})}{\gamma(t)}\Big)dz_{0}dz_{1}\right\}^{-1}\\ &\quad\quad\int\int\Big\{\big[\alpha^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\alpha(t)\big]z_{0}+\big[\beta^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\beta(t)\big]z_{1}+\frac{\gamma^{\prime}(t)}{\gamma(t)}\bm{z}\Big\}\\ &\quad\quad\rho_{0}(z_{0})\rho_{1}(z_{1})P_{\eta}\Big(\frac{\bm{z}-\alpha(t)z_{0}-\beta(t)z_{1}}{\gamma(t)}\Big)dz_{0}dz_{1}\\ &=\left\{\int\int\rho_{0}(z_{0})\exp\Big(-\frac{\|z_{1}-\frac{\beta(t)}{\gamma^{2}(t)+\beta^{2}(t)}(\bm{z}-\alpha(t)z_{0})\|^{2}}{2\frac{\gamma^{2}(t)}{\gamma^{2}(t)+\beta^{2}(t)}}\Big)\exp\Big(-\frac{\|\bm{z}-\alpha(t)z_{0})\|^{2}}{2(\gamma^{2}(t)+\beta^{2}(t))}\Big)dz_{1}dz_{0}\right\}^{-1}\\ &\quad\quad\int\int\Big\{\big[\alpha^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\alpha(t)\big]z_{0}+\big[\beta^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\beta(t)\big]z_{1}+\frac{\gamma^{\prime}(t)}{\gamma(t)}\bm{z}\Big\}\\ &\quad\quad\rho_{0}(z_{0})\exp\Big(-\frac{\|z_{1}-\frac{\beta(t)}{\gamma^{2}(t)+\beta^{2}(t)}(\bm{z}-\alpha(t)z_{0})\|^{2}}{2\frac{\gamma^{2}(t)}{\gamma^{2}(t)+\beta^{2}(t)}}\Big)\exp\Big(-\frac{\|\bm{z}-\alpha(t)z_{0})\|^{2}}{2(\gamma^{2}(t)+\beta^{2}(t))}\Big)dz_{1}dz_{0}\\ \end{split}$ $$

Treating $z_{1}$ as a Gaussian random variable, we obtain

$$ $\begin{split}&\rho_{1}(z_{1})P_{\eta}\Big(\frac{\bm{z}-\alpha(t)z_{0}-\beta(t)z_{1}}{\gamma(t)}\Big)\\ \propto&\exp\Big(-\frac{\|z_{1}\|^{2}}{2}\Big)\exp\Big(-\frac{\|\bm{z}-\alpha(t)z_{0}-\beta(t)z_{1}\|^{2}}{2\gamma^{2}(t)}\Big)\\ \propto&\exp\Big(-\frac{\|z_{1}\|^{2}}{2}\Big)\exp\Big(-\frac{\beta^{2}(t)\|z_{1}\|^{2}-2\beta(t)z_{1}^{T}(\bm{z}-\alpha(t)z_{0})+\|\bm{z}-\alpha(t)z_{0}\|^{2}}{2\gamma^{2}(t)}\Big)\\ \propto&\exp\Big(-\frac{(\gamma^{2}(t)+\beta^{2}(t))\|z_{1}\|^{2}-2\beta(t)z_{1}^{T}(\bm{z}-\alpha(t)z_{0})+\|\bm{z}-\alpha(t)z_{0}\|^{2}}{2\gamma^{2}(t)}\Big)\\ \propto&\exp\Big(-\frac{\|z_{1}-\frac{\beta(t)}{\gamma^{2}(t)+\beta^{2}(t)}(\bm{z}-\alpha(t)z_{0})\|^{2}}{2\frac{\gamma^{2}(t)}{\gamma^{2}(t)+\beta^{2}(t)}}\Big)\exp\Big(-\frac{\|\bm{z}-\alpha(t)z_{0})\|^{2}}{2(\gamma^{2}(t)+\beta^{2}(t))}\Big).\end{split}$ $$

Therefore $b^{*}(\bm{z},t)$ has expression that

$$ $\begin{split}b^{*}(\bm{z},t)&=\left\{\int\rho_{0}(z_{0})\exp\Big(-\frac{\|\bm{z}-\alpha(t)z_{0}\|^{2}}{2(\gamma^{2}(t)+\beta^{2}(t))}\Big)dz_{0}\right\}^{-1}\int\Big\{\big[\alpha^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\alpha(t)\big]z_{0}+\big[\beta^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\beta(t)\big]\\ &\quad\quad\frac{\beta(t)(\bm{z}-{\alpha(t)}z_{0})}{\gamma^{2}(t)+\beta^{2}(t)}+\frac{\gamma^{\prime}(t)}{\gamma(t)}\bm{z}\Big\}\rho_{0}(z_{0})\exp\big(-\frac{\|\bm{z}-\alpha(t)z_{0}\|^{2}}{2(\gamma^{2}(t)+\beta^{2}(t))}\big)dz_{0}\\ &=\int\Big[\alpha^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\alpha(t)-\frac{\beta^{\prime}(t)\gamma(t)-\gamma^{\prime}(t)\beta(t)}{\gamma^{3}(t)+\beta^{2}(t)\gamma(t)}\alpha(t)\beta(t)\Big]z_{0}+\Big[\frac{\gamma^{\prime}(t)}{\gamma(t)}+\frac{\beta^{\prime}(t)\gamma(t)-\gamma^{\prime}(t)\beta(t)}{\gamma^{3}(t)+\beta^{2}(t)\gamma(t)}\beta(t)\Big]\bm{z}\\ &\quad\quad\frac{\exp\big(-\frac{\|\bm{z}-\alpha(t)z_{0}\|^{2}}{2(\gamma^{2}(t)+\beta^{2}(t))}\big)}{\int\exp\big(-\frac{\|\bm{z}-\alpha(t)z_{0}\|^{2}}{2(\gamma^{2}(t)+\beta^{2}(t))}\big)\rho_{0}(z_{0})dz_{0}}\rho_{0}(z_{0})dz_{0}.\\ \end{split}$ $$

Considering that $\rho_{0}=\sum_{i=1}^{n}\delta(X_{i})$, the integral can be written as a summation over the samples $\{X_{i}\}_{i=1}^{n}$, thus $b^{*}(\bm{z},t)$ has the following expression

$$ $\begin{split}b^{*}(\bm{z},t)&=\sum_{i=1}^{n}\Big\{\Big[\frac{\gamma(t)\gamma^{\prime}(t)-\beta^{\prime}(t)\beta(t)}{\gamma^{2}(t)+\beta^{2}(t)}\Big]\bm{z}-\Big[\frac{\gamma(t)\gamma^{\prime}(t)-\beta^{\prime}(t)\beta(t)}{\gamma^{2}(t)+\beta^{2}(t)}\alpha(t)-\alpha^{\prime}(t)\Big]X_{i}\Big\}\frac{\exp\big(-\frac{\|\bm{z}-\alpha(t)X_{i}\|^{2}}{2(\gamma^{2}(t)+\beta^{2}(t))}\big)}{\sum_{j=1}^{n}\exp\big(-\frac{\|\bm{z}-\alpha(t)X_{j}\|^{2}}{2(\gamma^{2}(t)+\beta^{2}(t))}\big)}\\ &:=\sum_{i=1}^{n}\frac{1}{C_{3}(t)}\Big[C_{1}(t)\bm{z}-C_{2}(t)X_{i}\Big]\frac{\exp\big(-\frac{\|\bm{z}-\alpha(t)X_{i}\|^{2}}{2C_{3}(t)}\big)}{\sum_{j=1}^{n}\exp\big(-\frac{\|\bm{z}-\alpha(t)X_{j}\|^{2}}{2C_{3}(t)}\big)},\end{split}$ $$

where

$$ $\begin{split}&C_{1}(t)=\gamma(t)\gamma^{\prime}(t)+\beta^{\prime}(t)\beta(t),\\ &C_{2}(t)=\Big[\gamma(t)\gamma^{\prime}(t)+\beta^{\prime}(t)\beta(t)\Big]\alpha(t)-\Big[\gamma^{2}(t)+\beta^{2}(t)\Big]\alpha^{\prime}(t),\\ &C_{3}(t)=\gamma^{2}(t)+\beta^{2}(t).\end{split}$ $$

$\square$

#### A.3.2 Proof of Proposition 2

We begin by presenting two related lemmas that establish some useful properties of $Z_{t}$ with respect to $Z_{0}$ and $Z_{1}$.

###### Lemma 1 (Tweedie’s Formula)

Let $Y$ be a random variable following an exponential family distribution, such that $Y=\theta+\varepsilon$, where

- •
$\theta$ is an unknown parameter follows a prior distribution,
- •
$\varepsilon$ is independent noise with $\mathbb{E}[\varepsilon]=0$ and $\text{Var }(\varepsilon)=\sigma^{2}$.

Then, the posterior expectation of $\theta$ given $Y=\bm{y}$ is:

$$ $\mathbb{E}\big[\theta\mid Y=\bm{y}\big]=\bm{y}+\sigma^{2}\frac{d}{dy}\log P_{Y}(\bm{y}).$ $$

###### Lemma 2

The score function $s(\bm{z},t)$ can be expressed as

$$ $s(\bm{z},t)=-\frac{1}{\gamma(t)}\mathbb{E}\big[\eta|Z_{t}=\bm{z}\big].$ $$

Proof. Recalling that the definition of $Z_{t}$ is

$$ $Z_{t}=\alpha(t)Z_{0}+\beta(t)Z_{1}+\gamma(t)\eta.$ $$

Taking conditional expectation with respect to $Z_{t}$ on both sides, we obtain

$$ $Z_{t}=\mathbb{E}\big[\alpha(t)Z_{0}+\beta(t)Z_{1}|Z_{t}\big]+\gamma(t)\mathbb{E}\big[\eta|Z_{t}\big].$ $$

Applying Lemma [1](https://arxiv.org/html/2509.21925v1#Thmlemma1), it follows that

$$ $\mathbb{E}\big[\alpha(t)Z_{0}+\beta(t)Z_{1}|Z_{t}=\bm{z}\big]=\bm{z}+\gamma^{2}(t)s(\bm{z},t).$ $$

Combining the above equation, we have

$$ $s(\bm{z},t)=-\frac{1}{\gamma(t)}\mathbb{E}\big[\eta|Z_{t}=\bm{z}\big].$ (12) $$

$\square$

Next we present the proof of Proposition [2](https://arxiv.org/html/2509.21925v1#S3.Ex4).

Proof. The velocity field $b(\bm{z},t)$ can be further expressed as

$$ $\begin{split}b(\bm{z},t)&=\mathbb{E}\big[\alpha^{\prime}(t)Z_{0}+\beta^{\prime}(t)Z_{1}+\gamma^{\prime}(t)\eta|Z_{t}=\bm{z}\big]\\ &=\alpha^{\prime}(t)\mathbb{E}\big[Z_{0}|Z_{t}=\bm{z}\big]+\beta^{\prime}(t)\mathbb{E}\big[Z_{1}|Z_{t}=\bm{z}\big]+\gamma^{\prime}(t)\mathbb{E}\big[\eta|Z_{t}=\bm{z}\big].\end{split}$ (13) $$

Applying Lemma [1](https://arxiv.org/html/2509.21925v1#Thmlemma1), it follows that

$$ $\alpha(t)\mathbb{E}\big[Z_{0}|Z_{t}=\bm{z}\big]=\bm{z}+(\beta^{2}(t)+\gamma^{2}(t))s(\bm{z},t).$ (14) $$

In addition, by Lemma [2](https://arxiv.org/html/2509.21925v1#A1.Ex18), we have

$$ $\begin{split}Z_{t}&=\mathbb{E}\big[\alpha(t)Z_{0}+\beta(t)Z_{1}|Z_{t}\big]+\gamma(t)\mathbb{E}\big[\eta|Z_{t}\big]\\ &=\alpha(t)\mathbb{E}\big[Z_{0}|Z_{t}\big]+\beta(t)\mathbb{E}\big[Z_{1}|Z_{t}\big]+\gamma(t)\mathbb{E}\big[\eta|Z_{t}\big].\end{split}$ (15) $$

Combining Eq. [12](https://arxiv.org/html/2509.21925v1#A1.E12), Eq. [13](https://arxiv.org/html/2509.21925v1#A1.E13), Eq. [14](https://arxiv.org/html/2509.21925v1#A1.E14), and Eq. [15](https://arxiv.org/html/2509.21925v1#A1.E15), we arrive at the closed-form expression for the score function

$$ $s(\bm{z},t)=\frac{\alpha(t)}{B(t)}b(\bm{z},t)-\frac{\alpha^{\prime}(t)}{B(t)}\bm{z},$ $$

where

$$ $B(t)=\beta(t)\big[\alpha^{\prime}(t)\beta(t)-\alpha(t)\beta^{\prime}(t)\big]+\gamma(t)\big[\gamma(t)\alpha^{\prime}(t)-\gamma^{\prime}(t)\alpha(t)\big].$ $$

Thus we have

$$ $s^{*}(\bm{z},t)=\frac{\alpha(t)}{B(t)}b^{*}(\bm{z},t)-\frac{\alpha^{\prime}(t)}{B(t)}z.$ $$

$\square$

###### Lemma 1 (Tweedie’s Formula)

###### Lemma 2

#### A.3.3 Proof of Theorem 1

Proof. Let us define $A(t)=\exp\big(-\int_{t}^{1}\frac{C_{1}(u)}{C_{3}(u)}du\big)$, $X=[X_{1},\dots X_{n}]$ and introduce the weight function

$$ $\omega(\bm{z},t)=\text{softmax}\Big[\frac{\|\bm{z}-\alpha(t)X_{1}\|^{2}}{2(\gamma^{2}(t)+\beta^{2}(t))},\cdots,\frac{\|\bm{z}-\alpha(t)X_{n}\|^{2}}{2(\gamma^{2}(t)+\beta^{2}(t))}\Big]^{T}.$ $$

Let $Z_{t}=A(t)\kappa(t)$, where $\kappa(t)$ is a time-dependent auxiliary function. Then, by differentiating $\kappa(t)$, we have

$$ $\begin{split}\frac{d\kappa(t)}{dt}&=\frac{A(t)\frac{dZ_{t}(x)}{dt}-Z_{t}A^{\prime}(t)}{A^{2}(t)}\\ &=\frac{A(t)\frac{C_{1}(t)}{C_{3}(t)}Z_{t}-A(t)\frac{C_{2}(t)}{C_{3}(t)}\cdot X\cdot\omega\big(Z_{t},t\big)-\frac{C_{1}(t)}{C_{3}(t)}A_{t}Z_{t}}{A^{2}(t)}\\ &=-\frac{C_{2}(t)}{C_{3}(t)A(t)}\cdot X\cdot\omega\big(Z_{t},t\big).\end{split}$ $$

Let $Z_{t}(x)$ denote the random variable at time $t$ of the process initialized at point $x$. Then, $Z_{t}(x)$ can be expressed as follows

$$ $\begin{split}Z_{t}(x)&=A(t)\kappa(t)\\ &=\exp\Big(-\int_{t}^{1}\frac{C_{1}(u)}{C_{3}(u)}du\Big)\Big(x+\int_{t}^{1}\frac{C_{2}(u)}{C_{3}(u)A(u)}\cdot X\cdot\omega\big(Z_{u}(x),u\big)du\Big).\end{split}$ $$

Therefore, at $t=0$, there exists $i\in\{1\cdots n\}$ that the generation result $Z_{0}(x)$ takes the following value

$$ $\begin{split}Z_{0}(x)&=\lim_{t\rightarrow 0}\exp\Big(-\int_{t}^{1}\frac{C_{1}(u)}{C_{3}(u)}du\Big)\Big(x+\int_{t}^{1}\frac{C_{2}(u)}{C_{3}(u)A(u)}\cdot X\cdot\omega\big(Z_{u}(x),u\big)du\Big)\\ &=\lim_{t\rightarrow 0}\frac{\int_{t}^{1}\frac{C_{2}(u)}{C_{3}(u)A(u)}\cdot X\cdot\omega\big(Z_{u}(x),u\big)du}{\frac{1}{A(t)}}\\ &=\lim_{t\rightarrow 0}\frac{\frac{C_{2}(t)}{C_{3}(t)A(t)}\cdot X\cdot\omega\big(Z_{t}(x),t\big)}{\frac{A^{\prime}(t)}{A^{2}(t)}}\\ &=\lim_{t\rightarrow 0}\frac{C_{2}(t)}{C_{1}(t)}\cdot X\cdot\omega\big(Z_{t}(x),t\big)\\ &=X\cdot\lim_{t\rightarrow 0}\omega\big(Z_{t}(x),t\big)\\ &=X_{i}.\end{split}$ (16) $$

The sixth equation holds because as $t$ approaches 0, due to the nature of the softmax function, one element in $\omega$ tends to $1$, while the other elements tend to 0. Therefore, there exists $i\in\{1,\dots,n\}$ such that the final result will be one of the $X_{i}$.

$\square$

#### A.3.4 Proof of Theorem 2

Proof. Recalling that the stochastic generation process has the expression that

$$ $\begin{split}&dZ_{t}=\Big(b^{*}(Z_{t},t)-\zeta(t)s^{*}(Z_{t},t)\Big)dt+\sqrt{2\zeta(t)}dW_{t},\\ &\text{where }s(\bm{z},t)=\frac{\alpha(t)}{B(t)}b(\bm{z},t)-\frac{\alpha^{\prime}(t)}{B(t)}\bm{z}.\end{split}$ $$

Starting from initial point $s$, $Z_{t}(x)$ has the following expression

$$ $Z_{t}(x)=x-\int_{t}^{1}\Big(1-\frac{\zeta(u)\alpha(t)}{B(u)}\Big)b\big(Z_{u}(x),u\big)+\frac{\zeta(u)\alpha^{\prime}(u)}{B(u)}Z_{u}(x)du+\int_{t}^{1}\sqrt{2\zeta(t)}dW_{t}.$ $$

Let $\tilde{Z}_{t}$ denote the result generated by the drift component alone (i.e., without the stochastic noise), initialized from $x$

$$ $\tilde{Z}_{t}(x)=x-\int_{t}^{1}\Big(1-\frac{\zeta(u)\alpha(u)}{B(u)}\Big)b(Z_{u}(x),u)+\frac{\zeta(u)\alpha^{\prime}(u)}{B(u)}\tilde{Z}_{u}(x)du.$ $$

Similar to the proof of Theorem [1](https://arxiv.org/html/2509.21925v1#Thmtheorem1), denoting $A(t)=\exp(-\int_{t}^{1}\frac{C_{1}(u)}{C_{3}(u)}+\frac{\zeta^{\prime}(u)\alpha^{\prime}(u)}{B(u)}du)$, we obtain that

$$ $\begin{split}\tilde{Z}_{0}(x)&=\lim_{t\rightarrow 0}\exp\Big(-\int_{t}^{1}\frac{C_{1}(u)}{C_{3}(u)}+\frac{\zeta^{\prime}(u)\alpha^{\prime}(u)}{B(u)}du\Big)\Big(x-\big(1-\frac{\zeta(t)\alpha(t)}{B(t)}\big)\int_{t}^{1}\frac{-C_{2}(u)}{C_{3}(u)A(u)}\cdot X\cdot\omega\big(\tilde{Z}_{u}(x),u\big)\\ &=\lim_{t\rightarrow 0}\frac{C_{2}(t)}{C_{1}(t)}\big(1-\frac{\alpha^{\prime}(t)\zeta(t)C_{3}(t)}{C_{1}(t)B(t)}\big)^{-1}\big(1-\frac{\zeta(t)\alpha(t)}{B(t)}\big)\cdot X\cdot\omega\big(\tilde{Z}_{t}(x),t\big).\\ \end{split}$ $$

Therefore, when $\zeta(t)\lesssim B(t)$, there exists $i\in\{1,\cdots,n\}$ such that $\tilde{Z}_{0}(x)=X_{i}$. Consequently, the stochastic generation process satisfies

$$ $Z_{0}(x)=X_{i}+\int_{0}^{1}\sqrt{2\zeta(t)}dW_{t},$ $$

where $\int_{0}^{1}\sqrt{2\zeta(t)}\,dW_{t}$ is a Gaussian random variable with mean zero and variance $2\int_{0}^{1}\zeta(t)\,dt$.
$\square$

#### A.3.5 Proof of Theorem 3

Proof. When an error term $\epsilon(\bm{z},t)$ is present, the generation process can be expressed as follows

$$ $\frac{dZ_{t}}{dt}=b^{*}(\bm{z},t)+\epsilon(\bm{z},t).$ $$

Analogous to the proof in Theorem [1](https://arxiv.org/html/2509.21925v1#Thmtheorem1), define, let $A(t)=\exp\left(-\int_{t}^{1}\frac{C_{1}(u)}{C_{3}(u)}du\right)$ and $Z_{t}=A(t)\kappa(t)$, we have

$$ $\begin{split}\frac{d\kappa(t)}{dt}&=\frac{A(t)\frac{dZ_{t}}{dt}-Z_{t}A^{\prime}(t)}{A^{2}(t)}\\ &=\frac{A(t)\frac{C_{1}(t)}{C_{3}(t)}Z_{t}-A(t)\frac{C_{2}(t)}{C_{3}(t)}\cdot X\cdot\omega\big(Z_{t},t\big)+A(t)\epsilon(Z_{t},t)-\frac{C_{1}(t)}{C_{3}(t)}A_{t}Z_{t}}{A^{2}(t)}\\ &=-\frac{C_{2}(t)}{C_{3}(t)A(t)}\cdot X\cdot\omega\big(Z_{t},t\big)+\frac{\epsilon(Z_{t},t)}{A(t)}.\end{split}$ $$

Starting from initial point $x$, $Z_{t}(x)$ has the following expression

$$ $\begin{split}Z_{t}(x)&=A(t)\kappa(t)\\ &=\exp\Big(-\int_{t}^{1}\frac{C_{1}(u)}{C_{3}(u)}du\Big)\Big(x-\int_{t}^{1}-\frac{C_{2}(u)}{C_{3}(u)A(u)}\cdot X\cdot\omega\big(Z_{u}(x),u\big)+\frac{\epsilon(Z_{u},u)}{A(u)}du\Big).\end{split}$ $$

Therefore, at $t=0$, there exists $i\in\{1\cdots n\}$ that the generation result $Z_{0}(x)$ takes the following value

$$ $\begin{split}Z_{0}(x)&=\lim_{t\rightarrow 0}A(t)\Big(x-\int_{t}^{1}-\frac{C_{2}(u)}{C_{3}(u)A(u)}\cdot X\cdot\omega_{u}(Z_{u}(x))+\frac{\epsilon(Z_{u},u)}{A(u)}du\Big)\\ &=\lim_{t\rightarrow 0}\frac{\int_{t}^{1}\frac{C_{2}(u)}{C_{3}(u)A(u)}\cdot X\cdot\omega\big(Z_{u}(x),u\big)du}{\frac{1}{A(t)}}-\lim_{t\rightarrow 0}A(t)\int_{t}^{1}\frac{\epsilon(Z_{u},u)}{A(u)}du\\ &=\lim_{t\rightarrow 0}\frac{\frac{C_{2}(t)}{C_{3}(t)A(t)}\cdot X\cdot\omega\big(Z_{t}(x),t\big)}{\frac{A^{\prime}(t)}{A^{2}(t)}}-\lim_{t\rightarrow 0}\frac{\epsilon(Z_{t},t)/A(t)}{A^{\prime}(t)/A^{2}(t)}\\ &=\lim_{t\rightarrow 0}\frac{C_{2}(t)}{C_{1}(t)}\cdot X\cdot\omega\big(Z_{t}(x),t\big)-\lim_{t\rightarrow 0}\frac{A(t)\epsilon(Z_{t},t)}{A^{\prime}(t)}\\ &=X\cdot\lim_{t\rightarrow 0}\omega\big(Z_{t}(x),t\big)-\lim_{t\rightarrow 0}\frac{C_{3}(t)}{C_{1}(t)}\epsilon(Z_{t},t)\\ &=X_{i}-\lim_{t\rightarrow 0}\frac{C_{3}(t)}{C_{1}(t)}\epsilon(Z_{t},t).\end{split}$ $$

The fifth equality holds because $A(t)=\frac{C_{1}(t)}{C_{3}(t)}A^{\prime}(t)$.
$\square$

#### A.3.6 Proof of Corollary 1

Proof. Recalling that the expression of $C_{1}(t)$ and $C_{3}(t)$ are

$$ $\begin{split}&C_{1}(t)=\gamma(t)\gamma^{\prime}(t)+\beta^{\prime}(t)\beta(t),\\ &C_{3}(t)=\gamma^{2}(t)+\beta^{2}(t).\end{split}$ $$

As $t\rightarrow 0$, we observe that $C_{3}(t)/C_{1}(t)\rightarrow 0$. Therefore, if there exists a constant $\lambda>0$ such that $\|\epsilon(\bm{z},t)\|^{2}\leq\lambda$ for all $t$, then

$$ $\lim_{t\rightarrow 0}\left\|\frac{C_{3}(t)}{C_{1}(t)}\epsilon(\bm{z},t)\right\|^{2}=0.$ $$

This implies that the error term vanishes in the limit as $t\rightarrow 0$, and hence the generated sample at time $t=0$ concentrates around one of the training samples.
Therefore, there exists an $i\in\{1\cdots n\}$ such that $Z_{0}=X_{i}$.
$\square$

#### A.3.7 Proof of Corollary 2

Proof. Since the error term $\epsilon(\bm{z},t)$ is bounded by a function related to $\gamma(t)$, if $C_{3}(t)\lesssim\gamma(t)C_{1}(t)$, then

$$ $\lim_{t\rightarrow 0}\left\|\frac{C_{3}(t)}{C_{1}(t)}\epsilon(\bm{z},t)\right\|^{2}=0.$ $$

If $C_{3}(t)\asymp\gamma(t)C_{1}(t)$, there exists a constant $c>0$ such that

$$ $\lim_{t\rightarrow 0}\left\|\frac{C_{3}(t)}{C_{1}(t)}\epsilon(\bm{z},t)\right\|^{2}=c.$ $$

If $C_{3}(t)\gtrsim\gamma(t)C_{1}(t)$, then

$$ $\lim_{t\rightarrow 0}\left\|\frac{C_{3}(t)}{C_{1}(t)}\epsilon(\bm{z},t)\right\|^{2}=\infty.$ $$

Next, we demonstrate the case when $\beta(t)=t$, where $\beta^{\prime}(t)=1$.
we consider the following equation

$$ $\begin{split}\frac{C_{3}(t)}{C_{1}(t)\gamma(t)}&=\frac{\gamma^{2}(t)}{\gamma^{2}(t)\gamma^{\prime}(t)+\gamma(t)\beta(t)\beta^{\prime}(t)}+\frac{\beta^{2}(t)}{\gamma^{2}(t)\gamma^{\prime}(t)+\gamma(t)\beta(t)\beta^{\prime}(t)}\\ &=\frac{\gamma^{2}(t)}{\gamma^{2}(t)\gamma^{\prime}(t)+\gamma(t)t}+\frac{t^{2}}{\gamma^{2}(t)\gamma^{\prime}(t)+\gamma(t)t}.\end{split}$ $$

Therefore, as $t\rightarrow 0$, if $\gamma(t)\lesssim\beta(t)$, then

$$ $\lim_{t\rightarrow 0}\left\|\frac{C_{3}(t)}{C_{1}(t)}\epsilon(\bm{z},t)\right\|^{2}=0.$ $$

If $\gamma(t)\asymp\beta(t)$, there exists a constant $c>0$ such that

$$ $\lim_{t\rightarrow 0}\left\|\frac{C_{3}(t)}{C_{1}(t)}\epsilon(\bm{z},t)\right\|^{2}=c.$ $$

If $\gamma(t)\gtrsim\beta(t)$, then

$$ $\lim_{t\rightarrow 0}\left\|\frac{C_{3}(t)}{C_{1}(t)}\epsilon(\bm{z},t)\right\|^{2}=\infty.$ $$

$\square$

#### A.3.8 Proof of Corollary 3

Proof. (1) We begin by proving the first case, that is,
there exists a constant $\overline{\lambda}$ such that for any $\lambda>\overline{\lambda}$, and for all $i$ and $t$, $\|Z_{t-1}-X_{i}\|^{2}>\|Z_{t}-X_{i}\|^{2}.$

(i) If for all index $j\in\{1\dots,n\}$,

$$ $\|Z_{t}-\alpha(t)X_{j}\|^{2}>2\max_{l,k}\|X_{l}-X_{k}\|^{2},$ $$

then, for a fixed index $i$, it follows that for all $j\in\{1\dots,n\}$,

$$ $\begin{split}\Big\|Z_{t}-\alpha(t)X_{j}\Big\|^{2}&\leq\Big\|Z_{t}-\alpha(t)X_{i}\Big\|^{2}+\alpha(t)\Big\|X_{i}-X_{j}\Big\|^{2}\\ &\leq\Big\|Z_{t}-\alpha(t)X_{i}\Big\|^{2}+\frac{\alpha(t)}{2}\Big\|Z_{t}-\alpha(t)X_{i}\Big\|^{2}\\ &\leq\frac{2+\alpha(t)}{2}\Big\|Z_{t}-\alpha(t)X_{i}\Big\|^{2}.\end{split}$ $$

Consequently, we have

$$ $\left\{\sum_{j=1}^{n}\exp\big(-\frac{\|Z_{t}-\alpha(t)X_{j}\|^{2}}{2C_{3}(t)}\big)\right\}^{-1}\gtrsim\left\{n\exp\big(-\frac{\|Z_{t}-\alpha(t)X_{i}\|^{2}}{2C_{3}(t)}\big)\right\}^{-1}\gtrsim\exp\big(\|Z_{t}-\alpha(t)X_{i}\|^{2}\big).$ (17) $$

Moreover, there exist universal constants $c_{1},c_{2}>0$ such that

$$ $\begin{split}&\Big\|Z_{t}-\frac{C_{1}(t)}{C_{3}(t)}Z_{t}h+\frac{C_{2}(t)}{C_{3}(t)}\cdot X\cdot\omega(Z_{t},t)h-X_{i}\Big\|^{2}\\ &=\Big\|(1-\frac{C_{1}(t)}{C_{3}(t)}h)Z_{t}-(1-\frac{C_{1}(t)}{C_{3}(t)}h)X_{i}+(1-\frac{C_{1}(t)}{C_{3}(t)}h)X_{i}-X_{i}+\frac{C_{2}(t)}{C_{3}(t)}\cdot X\cdot\omega(Z_{t},t)h\Big\|^{2}\\ &\leq(1-\frac{C_{1}(t)}{C_{3}(t)}h)\Big\|Z_{t}-X_{i}\Big\|^{2}+\Big\|\frac{C_{1}(t)}{C_{3}(t)}hX_{i}\Big\|^{2}+\Big\|\frac{C_{2}(t)}{C_{3}(t)}\cdot X\cdot\omega(Z_{t},t)h\Big\|^{2}\\ &\leq c_{1}\Big\|Z_{t}-X_{i}\Big\|^{2}+c_{2}.\end{split}$ (18) $$

The constants $c_{1}$ and $c_{2}$ are independent of both $t$ and $j$ because the functions $\beta(t)$, $C_{1}(t)$, $C_{3}(t)$ and $\|X\|^{2}$ are all uniformly bounded.

Suppose that the error term $\epsilon(\bm{z},t)$ satisfies the following condition

$$ $\Big\|\epsilon(Z_{t},t)\Big\|^{2}h>\Big\|Z_{t}-\frac{C_{1}(t)}{C_{3}(t)}Z_{t}h+\frac{C_{2}(t)}{C_{3}(t)}\cdot X\cdot\omega(Z_{t},t)h-X_{i}\Big\|^{2}+\Big\|Z_{t}-X_{i}\Big\|^{2},$ (19) $$

then, by virtue of the update equation

$$ $Z_{t-1}-X_{i}=Z_{t}-\frac{C_{1}(t)}{C_{3}(t)}Z_{t}h+\frac{C_{2}(t)}{C_{3}(t)}\cdot X\cdot\omega(Z_{t},t)h+\epsilon(Z_{t},t)h-X_{i},$ (20) $$

and applying the triangle inequality, it follows that

$$ $\begin{split}\Big\|Z_{t-1}-X_{i}\Big\|^{2}&=\Big\|Z_{t}-\frac{C_{1}(t)}{C_{3}(t)}Z_{t}h+\frac{C_{2}(t)}{C_{3}(t)}\cdot X\cdot\omega(Z_{t},t)h+\epsilon(Z_{t},t)h-X_{i}\Big\|^{2}\\ &\geq\Big|\|\epsilon(Z_{t},t)h\|^{2}-\|Z_{t}-\frac{C_{1}(t)}{C_{3}(t)}Z_{t}h+\frac{C_{2}(t)}{C_{3}(t)}\cdot X\cdot\omega(Z_{t},t)h-X_{i}\|^{2}\Big|\\ &\geq\|Z_{t}-X_{i}\|^{2}.\end{split}$ $$

Combining Eq. [18](https://arxiv.org/html/2509.21925v1#A1.E18) and Eq. [19](https://arxiv.org/html/2509.21925v1#A1.E19), the condition in Eq. [19](https://arxiv.org/html/2509.21925v1#A1.E19) can be rewritten as

$$ $\|\epsilon(Z_{t},t)\|^{2}h\geq(1+c_{1})\|Z_{t}-X_{i}\|^{2}+c_{2}.$ (21) $$

According to Eq. [17](https://arxiv.org/html/2509.21925v1#A1.E17), for each discrete time $t_{i}$, there exists a constant $\overline{\lambda}_{t_{i}}$ such that Eq. [21](https://arxiv.org/html/2509.21925v1#A1.E21) holds. We denote $\overline{\lambda}_{1}=\max_{t_{i}}\overline{\lambda}_{t_{i}}$.

(ii) If there exists an index $j$ such that

$$ $\|Z_{t}-\alpha(t)X_{j}\|^{2}\leq 2\max_{l,k}\|X_{l}-X_{k}\|^{2},$ $$

then for any $i\in\{1,\dots,n\}$, it holds that

$$ $\|Z_{t}-\alpha(t)X_{i}\|^{2}\leq\big(2+\alpha(t)\big)\max_{l,k}\|X_{l}-X_{k}\|^{2}.$ $$

In this case, there exist universal constants $c_{3}\geq 0$ such that

$$ $\|Z_{t}-X_{i}\|^{2}\leq\|Z_{t}-\alpha(t)X_{i}\|^{2}+\|\alpha(t)X_{i}-X_{i}\|^{2}\leq c_{3},$ $$

where $c_{3}$ is independent of both $t$ and $j$. Furthermore, we have

$$ $\|\epsilon(z,t)\|^{2}={\lambda}\left\{\sum_{j=1}^{n}\exp\left(-\frac{\|Z_{t}-\alpha(t)X_{j}\|^{2}}{2C_{3}(t)}\right)\right\}^{-1}\geq\frac{\lambda}{n}.$ $$

Therefore, there exists a constant $\overline{\lambda}_{2}$ such that the error term $\epsilon(\bm{z},t)$ satisfies Eq. [21](https://arxiv.org/html/2509.21925v1#A1.E21).
By defining $\overline{\lambda}=\max(\overline{\lambda}_{1},\overline{\lambda}_{2})$, the proof of the first case is complete.

(2) Next, we prove the second case, that is, for any $\tau>0$, there exists $\underline{\lambda}$ such that when $\lambda<\underline{\lambda}$, there exists $i\in\{1,\dots,n\}$ satisfying

$$ $\|Z_{0}-X_{i}\|^{2}<\tau.$ $$

According to Eq. [20](https://arxiv.org/html/2509.21925v1#A1.E20), there exist universal constants $c_{1},c_{2},c_{3}>0$ such that

$$ $\begin{split}\Big\|Z_{t-1}-X_{i}\Big\|^{2}&\leq\big(1-\frac{C_{1}(t)}{C_{3}(t)}h\big)\Big\|Z_{t}-X_{i}\Big\|^{2}+\Big\|\frac{C_{1}(t)}{C_{3}(t)}hX_{i}+\frac{C_{2}(t)}{C_{3}(t)}\cdot X\cdot\omega(Z_{t},t))h\Big\|^{2}\\ &\quad\quad+\Big\|\epsilon(\bm{z},t)\Big\|^{2}h\\ &=c_{1}\Big\|Z_{t}-X_{i}\Big\|^{2}+c_{2}+\Big\|\epsilon(\bm{z},t)\Big\|^{2}h\\ &\leq c_{3}\lambda\exp(\|Z_{t}-X_{i})\|^{2}.\end{split}$ $$

Since the start point $Z_{1}$ is bounded and $\{X_{i}\}_{i=1}^{n}$ is a finite set, let

$$ $M=\max_{i}\|Z_{1}-X_{i}\|^{2}.$ $$

Define $C_{T}:=\exp^{(T)}(0)$ as the T-fold exponential, where $T$ is the number of generation steps. Thus there exists $\underline{\lambda}_{1}$ such that $c_{3}\underline{\lambda}_{1}M<\ln^{(T)}(C_{T})$, which implies $\exp^{(T)}(c_{3}\underline{\lambda}_{1}M)<C_{T}$.
Simultaneously, there exists $\underline{\lambda}_{2}$ satisfying $\underline{\lambda}_{2}<\frac{\tau}{c_{3}C_{T}}.$ By defining $\underline{\lambda}=\min(\underline{\lambda}_{1},\underline{\lambda}_{2}),$ we obtain $c_{3}\underline{\lambda}\exp^{(T)}(c_{3}\underline{\lambda}M)\leq\tau,$ and consequently, $\|Z_{0}-X_{i}\|^{2}\leq\tau$, which completes the proof of the second case.

$\square$

#### A.3.9 Proof of Proposition 11

Proof. Similar to the proof of Proposition [1](https://arxiv.org/html/2509.21925v1#S3.Ex3), when $\rho_{0}=\frac{1}{m}\sum_{i=1}^{m}\delta(X_{i})$ and $\rho_{1}=\frac{1}{n}\sum_{i=1}^{n}\delta(Y_{i})$, the optimal velocity field has the expression that

$$ $\begin{split}b^{*}_{2}(\bm{z},t)&=\mathbb{E}\big[\alpha^{\prime}(t)Z_{0}+\beta^{\prime}(t)Z_{1}+\gamma^{\prime}(t)\eta|Z_{t}=\bm{z}\big]\\ &=\frac{1}{p(Z_{t}=\bm{z})}\int\int\Big[\alpha^{\prime}(t)Z_{0}+\beta^{\prime}(t)Z_{1}+\gamma^{\prime}(t)\eta\Big]\rho_{0}(z_{0})\rho_{1}(z_{1})P_{\eta}\Big(\frac{\bm{z}-\alpha(t)z_{0}-\beta(t)z_{1}}{\gamma(t)}\Big)dz_{0}dz_{1}\\ &=\frac{1}{p(Z_{t}=\bm{z})}\int\int\Big\{\Big[\alpha^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\alpha(t)\Big]z_{0}+\Big[\beta^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\beta(t)\Big]z_{1}+\frac{\gamma^{\prime}(t)}{\gamma(t)}\bm{z}\Big\}\\ &\quad\quad\rho_{0}(z_{0})\rho_{1}(z_{1})P_{\eta}\Big(\frac{\bm{z}-\alpha(t)z_{0}-\beta(t)z_{1}}{\gamma(t)}\Big)dz_{0}dz_{1}\\ &=\frac{\gamma^{\prime}(t)}{\gamma(t)}\bm{z}+\sum_{i=1}^{n}\sum_{j=1}^{m}\Big\{\Big[\alpha^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\alpha(t)\Big]X_{i}+\Big[\beta^{\prime}(t)-\frac{\gamma^{\prime}(t)}{\gamma(t)}\beta(t)\Big]Y_{j}\Big\}\\ &\quad\quad\frac{\exp(-\frac{\|\bm{z}-\alpha(t)X_{i}-\beta(t)Y_{j}\|^{2}}{2\gamma^{2}(t)})}{\sum_{k=1}^{n}\sum_{l=1}^{m}\exp(-\frac{\|\bm{z}-\alpha(t)X_{k}-\beta(t)Y_{l}\|^{2}}{2\gamma^{2}(t)})}.\end{split}$ $$

$\square$

#### A.3.10 Proof of Corollary 4

Proof. According to the definition of $b_{2}^{*}(\bm{z},t)$ and the proof of Theorem [1](https://arxiv.org/html/2509.21925v1#Thmtheorem1), we denote $A(t)=\exp\Big(-\int_{t}^{1}\frac{\gamma^{\prime}(u)}{\gamma(u)}du\Big)$. Starting from an initial point $x$ at time $t=1$, $Z_{t}(x)$ has expression that

$$ $\begin{split}Z_{t}(x)&=\exp\Big(-\int_{t}^{1}\frac{\gamma^{\prime}(u)}{\gamma(u)}du\Big)\Big(x-\int_{t}^{1}\sum_{i=1}^{n}\sum_{j=1}^{m}\Big\{\Big[\alpha^{\prime}(u)-\frac{\gamma^{\prime}(u)}{\gamma(u)}\alpha(u)\Big]X_{i}+\Big[\beta^{\prime}(u)-\frac{\gamma^{\prime}(u)}{\gamma(u)}\beta(u)\Big]Y_{j}\Big\}\\ &\quad\quad\frac{\exp(-\frac{\|Z_{u}(x)-\alpha(u)X_{i}-\beta(u)Y_{j}\|^{2}}{2\gamma^{2}(u)})}{\sum_{k=1}^{n}\sum_{l=1}^{m}\exp(-\frac{\|Z_{u}(x)-\alpha(u)X_{k}-\beta(u)Y_{l}\|^{2}}{2\gamma^{2}(u)})}\frac{1}{A(u)}du\Big).\end{split}$ $$

Therefore, there exists an $i\in\{1,\dots,n\}$ such that

$$ $\begin{split}Z_{0}(x)&=\lim_{t\rightarrow 0}\exp\Big(-\int_{t}^{1}\frac{\gamma^{\prime}(u)}{\gamma(u)}du\Big)\Big(\int_{t}^{1}\sum_{i=1}^{n}\sum_{j=1}^{m}\Big\{\Big[\alpha^{\prime}(u)-\frac{\gamma^{\prime}(u)}{\gamma(u)}\alpha(u)\Big]X_{i}+\Big[\beta^{\prime}(u)-\frac{\gamma^{\prime}(u)}{\gamma(u)}\beta(u)\Big]Y_{j}\Big\}\\ &\quad\quad\quad\quad\quad\frac{\exp(-\frac{\|Z_{u}-\alpha(u)X_{i}-\beta(u)Y_{j}\|^{2}}{2\gamma^{2}(u)})}{\sum_{k=1}^{n}\sum_{l=1}^{m}\exp(-\frac{\|Z_{u}-\alpha(u)X_{k}-\beta(u)Y_{l}\|^{2}}{2\gamma^{2}(u)})}\frac{1}{A(u)}du\Big)\\ &=\lim_{t\rightarrow 0}-\frac{\gamma(t)}{\gamma^{\prime}(t)}\sum_{i=1}^{n}\sum_{j=1}^{m}\Big\{\Big[\alpha^{\prime}(u)-\frac{\gamma^{\prime}(u)}{\gamma(u)}\alpha(u)\Big]X_{i}+\Big[\beta^{\prime}(u)-\frac{\gamma^{\prime}(u)}{\gamma(u)}\beta(u)\Big]Y_{j}\Big\}\\ &\quad\quad\quad\quad\quad\quad\quad\frac{\exp(-\frac{\|z_{u}-\alpha(u)X_{i}-\beta(u)YY_{j}\|^{2}}{2\gamma^{2}(u)})}{\sum_{k=1}^{n}\sum_{l=1}^{m}\exp(-\frac{\|Z_{u}-\alpha(u)X_{k}-\beta(u)Y_{l}\|^{2}}{2\gamma^{2}(u)})}\\ &=\lim_{t\rightarrow 0}\frac{-1}{\gamma^{\prime}(t)}\Big[\sum_{i=1}^{n}\sum_{j=1}^{m}\Big(\gamma(t)\alpha^{\prime}(t)-\gamma^{\prime}(t)\alpha(t)\Big)X_{i}+\Big(\gamma(t)\beta^{\prime}(t)-\gamma^{\prime}(t)\beta(t)\Big)Y_{j}\Big]\\ &\quad\quad\quad\quad\quad\quad\quad\frac{\exp(-\frac{\|Z_{u}-\alpha(u)X_{i}-\beta(u)Y_{j}\|^{2}}{2\gamma^{2}(u)})}{\sum_{k=1}^{n}\sum_{l=1}^{m}\exp(-\frac{\|Z_{u}-\alpha(u)X_{k}-\beta(u)Y_{l}\|^{2}}{2\gamma^{2}(u)})}\\ &=X_{i}.\end{split}$ $$

According to the symmetry of $\{X_{i}\}_{i=1}^{n}$ and $\{Y_{j}\}_{j=1}^{m}$, starting from time $t=0$, the generated samples belong to $\{Y_{j}\}_{j=1}^{m}$.

$\square$

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 4
presentation: 2
questions: It appears that this submission may not fully comply with the official ICLR 2026 LaTeX style and template requirements. In particular, the section headings, theorem statements, and corollaries are not displayed in boldface as specified in the ICLR formatting guidelines. This results in weak visual separation between structural elements, thereby reducing readability and making it difficult to distinguish different logical components of the paper. Could the authors confirm whether the manuscript was prepared using the correct ICLR 2026 LaTeX style files and templates?
soundness: 4
strengths: - The core strength of this work lies in its rigorous theoretical contributions. It successfully derives closed-form expressions for the optimal velocity field and score function within the SI framework under the practical and important finite-sample assumption. The resulting softmax structure of the weights provides a clear and intuitive understanding of the "attractor effect" pulling the generation trajectory towards the training data points. This is followed by clean, limiting conclusions for deterministic (exact sample recovery) and stochastic (sample + Gaussian noise) generation in the ideal case.
- The paper does not stop at the ideal error-free setting. It systematically characterizes the impact of estimation errors on the generation process. This analysis provides actionable theoretical guidance for practitioners, linking generation outcomes to the properties of the error and the relative decay rates of the interpolation schedules (e.g., $\gamma(t)$ and $\beta(t)$), which is valuable for practical understanding and parameter tuning.
summary: This paper provides a theoretical investigation into the behavior of generative models under a finite training set, focusing specifically on the stochastic interpolation framework. The authors derive closed-form expressions for the optimal velocity field and score function given a finite number of training samples. The analysis reveals that in an ideal (error-free) setting, the deterministic generation process exactly recovers the training samples, while the stochastic generation process yields training samples corrupted by additive Gaussian noise. Furthermore, the paper extends this analysis to the practical setting involving model estimation errors, introducing formal definitions for "overfitting" and "underfitting" in this context. The theoretical results suggest that generated samples in this scenario can be viewed as a mixture of training samples, uniform noise (from estimation error), and Gaussian noise (from stochasticity). The paper's theoretical claims are supported empirically by 2D data visualizations and downstream tasks (classification, contrastive learning) on datasets like MNIST, CIFAR-10, and ImageNet, providing a theoretical foundation for understanding memorization and generalization in diffusion and flow-based models under limited data.
weaknesses: - The analysis is confined to the stochastic interpolation framework (and the diffusion models it encompasses). It remains an open question whether these conclusions generalize to other prominent generative model families, such as VAEs, GANs, and Autoregressive models. Furthermore, several core conclusions rely on idealized assumptions:
(1) The standard setup assumes $\rho_1$ is a Gaussian distribution.
(2) Specific technical conditions (e.g., $\zeta(t) \lesssim B(t)$) are required for some results.
(3) The analysis in Sec 3.2 depends on strong, formal assumptions about the error term $\epsilon(z,t)$. For instance, Corollary 1 assumes the error is "uniformly bounded" , while Corollary 3 assumes it is "inversely proportional to the density." These assumptions, while facilitating theoretical derivation, may not hold in practice, and their robustness for industrial-scale models or multi-modal generation is yet to be verified.
- The empirical support for the core theoretical claims (especially regarding the impact of different error bounds) relies heavily on 2D "toy" data visualizations. While the downstream tasks on small- to medium-scale datasets (MNIST, CIFAR-10) are illustrative, it remains unclear whether the insights derived from these low-dimensional error structures generalize to the complex, high-dimensional data manifolds encountered in state-of-the-art image generation. The paper would be strengthened by validation on larger-scale or conditional generation settings.

---

confidence: 5
contribution: 1
initial_rating: 2
presentation: 1
questions: 1. **Regarding Theorem 1 and Equations (4) and (5):**
   If my understanding is correct, you take for granted that Equations (4) and (5) have solutions and that their marginals are given by $\rho_0$ and $\rho_1$. If this is the case, could you clarify what is novel in **Theorem 1**?
   Given that we already know the marginal of the solution corresponds to $\rho_0$, and assuming $\rho_0$ is defined as in line 145 (i.e., concentrated on a finite set of points $\{X_i\}$), it logically follows that $Z_0$ must coincide with one of the $X_i$.
   What, then, does **Theorem 1** contribute beyond this observation?

2. **Regarding the claim on lines 175–176:**
   Is my understanding correct that the claim on lines 175–176 implies the formula for $b^\star$ can *only* be derived when $\rho_0$ is concentrated on a finite set?
   If so, could you explain why this restriction is necessary?
   To my knowledge, at least in the case where $\gamma = 0$, similar formulas for $b^*$ are available for a much broader class of distributions, not limited to discrete ones.
soundness: 1
strengths: Given the current prominence of generative modeling, any effort to deepen our theoretical understanding of existing methods or to design new, theoretically grounded approaches is highly valuable.
summary: This paper addresses the problem of generating samples from a target distribution using the stochastic interpolation method. Unlike previous work, the target distribution here is assumed to be discrete, with its support concentrated on a finite set of points.
The main contributions are theoretical: we demonstrate that both the deterministic and stochastic versions of the generator produce synthetic samples that are either elements of the support (in the deterministic case) or samples formed by adding a Gaussian vector to an element of the support. The case of an estimated score is also examined.
weaknesses: ### General Assessment
The paper suffers from significant issues in both language and mathematical rigor, making it difficult to follow. Below is a detailed list of the problems identified:

---

### **1. Language and Mathematical Issues**

- **Lines 76, 78:** The term "Notion" should be replaced with "Notation."
- **Line 83:** The authors state that $P_X$ denotes the probability density function (PDF) of a random variable $X$.
  - First, there is no evidence in the text where this notation is actually used.
  - Second, the reference measure for the density should be explicitly clarified.
  - Instead, the notation $p$ appears on line 681 without any prior definition.
- **Line 97:** The notation $\zeta$ is introduced without explanation. It remains unclear whether $\zeta$ is equivalent to $\gamma$ or refers to a different quantity.
- **Line 107:** The phrase *"Without loss of generality"* lacks justification—it is unclear why the condition does not restrict the generality of the setting.
- **Line 113:** A reference is needed to support the claim that Equations (4) and (5) have solutions and that these solutions possess the desired marginals.
- **Lines 120–123:** While it is acknowledged that $b$ and $s$ are unknown in practice, the text should specify what *is* known to properly define the estimation problem.
  - The formulas in these lines include $Z_t$ on the right-hand side, but it is unclear how $Z_t$ can be accessed when $b$ and $s$ are unknown.
  - The left-hand side depends on $\boldsymbol{z}$, yet $\boldsymbol{z}$ does not appear on the right-hand side.
- **Line 142:** The statement *"In many real applications ..."* is either incorrect or requires reformulation.
- **Line 147:** It is unclear why expectations with respect to $\rho_1$ are treated as empirical means. While this may hold for $\rho_0$ (as defined on line 145), it does not necessarily apply to $\rho_1$.
- **Formatting:** Boldface font should be used consistently for **Theorems**, **Propositions**, etc.
- **Proposition 1:** The meaning of *"optimal"* is ambiguous, rendering the claim unclear.
- **Theorem 3:** In Equation (9), $z$ should be replaced with $Z_0$. Additionally, the existence of the limit should be explicitly stated and proven.

---

### **2. Problem Formulation**
The paper fails to clearly specify the target distribution for sampling. In generative modeling, the typical setup involves:
- An unknown target distribution $P_0$.
- An empirical distribution $\hat{P}_n$ derived from a sample of size $n$ drawn independently from $P_0$.

In this paper, there is confusion between $P_0$ and $\hat{P}_n$. As far as I understand, the authors assume $P_0$ is a uniform distribution over a finite set, with elements denoted by $X_i$. However, the fact that backward ODEs and SDEs lead to elements from the support of $P_0$ was already established in prior work.

---

### **3. Limited Novelty and Impact**
The paper's contributions are modest and of limited practical interest:
- The discretization of deterministic and stochastic differential equations is not addressed, despite its critical impact on the generative model's error.
- The selection of functions $\alpha$ and $\beta$ is not discussed, even though these choices may significantly affect performance.


---

### **4. Experimental Setting**
The experimental section lacks clarity:
- The choice of parameters ($\alpha$, $\beta$, $\gamma$), the discretization scheme, and other experimental details should be explicitly described.
- The theoretical insights that the experiments aim to validate should be clearly stated.

---

### **5. Prior work**
The paper claims to investigate memorization and generalization behavior of genertaive models (line 49). But unfortunatly, it does not contain any discussion of what has been done in the prior work on these aspects. Relevant references include
-  Vaishnavh Nagarajan, Colin Raffel, and Ian J Goodfellow. Theoretical Insights into Memorization in GANs. Neural
Information Processing Systems (NeurIPS) 2017 - Integration of Deep Learning Theories Workshop, 2018.
- Ishaan Gulrajani, Colin Raffel, and Luke Metz. Towards GAN Benchmarks Which require Generalization. In ICLR 2019
- Matthew Jagielski et al.  Measuring Forgetting of Memorized Training Examples. In The Eleventh International Conference on Learning Representations, 2023
- Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models. In Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, pages 6048-6058, 2023a.
- Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Understanding and Mitigating Copying in Diffusion Models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023b.
- Elen Vardanyan, Sona Hunanyan, Tigran Galstyan, Arshak Minasyan, Arnak S. Dalalyan:
Statistically Optimal Generative Modeling with Maximum Deviation from the Empirical Distribution. ICML 2024

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 2
questions: Could the authors clarify what the first column ("Sample Size") in Tables 1 and 2 represents? The paper does not define this term, and it seems inconsistent with the statement in the main text that the training sample size is fixed at 100. If "Sample Size" does not denote the actual number of training examples used, please specify precisely what it measures and how it relates to the experimental setup.
soundness: 2
strengths: Overall the paper is clearly written with simple and intuitive main theorems. Thm. 1/2 formalize the ``exact copy vs. noisy copy" intuition, and the visualization in Fig. 1 matches the theory. It is interesting that the author takes into account the estimation error and connect it to the under/overfitting in SI with finite‑data training. The toy studies in Fig. 3–4 qualitatively support the predictions.
summary: The paper analyzes memorization/generalization of the Stochastic Interpolation. With one end being empirical measure and the other end being Gaussian, the authors derive closed‑form expressions for the optimal velocity being a softmax‑weighted combination of the training samples, and show the optimal score is an affine transform of 
the optimal velocity. Using these forms, they prove that deterministic generation recovers exact training samples, while stochastic generation yields training samples plus Gaussian noise; they then study how estimation error affects outcomes, and present toy examples and tasks (MNIST/FashionMNIST classification; CIFAR‑10 contrastive learning) to support the theory.
weaknesses: 1. The memorization effect and the closed‑form optimal scores in finite‑sample settings are well established in many score‑based diffusion literature. The SI perspective substantially generalize the setting, but the incremental conceptual gain over prior SGM memorization results is limited without new theoretical consequences or scaled experiments on modern, larger models. The paper would benefit from a clearer comparisons of what SI enables beyond standard SGM results.
2. While Section 4 provides two intuitive experiments, the evidence is indirect. Since the paper derives closed‑form optimal estimators for the velocity and score, it would be informative to include generation results obtained with the closed‑form optimal scores as a comparison to the trained model. Additionally, reporting the discrepancy between the learned score function and the theoretical optimal score function would more concretely validate the proposed theory and help isolate the effect of training errors.
3. There are several formatting problems in the manuscript. For example, theorem statements appear inline with the main text without typographic emphasis. Please consider using standard theorem environments. Also there are problem with the citation format in Section 4.2.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: See weaknesses
soundness: 1
strengths: 1. *Link between drift, score, and noise:* The paper explicitly relates the forms of the velocity (drift) and score functions to the type and magnitude of stochastic noise, synthesizing how Gaussian or uniform perturbations emerge in generation.
2. *Characterization of memorization:* It offers a formal description of how deterministic generation reproduces training data and how stochasticity leads to noisy versions, providing a claimed theoretical explanation of memorization phenomena in generative models.
3. *Introduction of overfitting/underfitting notions:* It discusses these regimes specifically for generative models, linking them to the size of estimation errors and to the behavior of the velocity field.
4. *Data augmentation perspective:* The paper includes technical examples and experiments showing how generated samples can act as noise-perturbed or augmented data, connecting generation quality to data availability.
5. *Conceptual contribution:* Overall, it attempts to systematize how finite-sample size, stochasticity, and model estimation errors jointly shape the generative process.
summary: This paper aims at analyzing the behavior of a class of generative models trained on limited data through the stochastic interpolation framework. It asserts that, under ideal conditions, deterministic generation exactly reproduces the training samples, while stochastic generation corresponds to the same samples perturbed by Gaussian noise. When estimation errors are introduced, the generated samples become convex combinations of training data corrupted by mixed uniform and Gaussian noise. It also proposes definitions of overfitting and underfitting specific to generative models. The theoretical discussion is supported by experiments on MNIST, CIFAR-10, and ImageNet, which are said to confirm that small training sets lead to memorization, while larger datasets yield more diverse generations. Overall, the paper positions itself as providing a theoretical explanation for memorization and generalization phenomena in generative models trained on finite datasets.
weaknesses: **Major Scientific Weaknesses**
1. **Poor referencing and limited contextualization:** The paper insufficiently cites the rapidly growing literature on the theoretical convergence and generalization of diffusion-based models. Several key contributions ([1]–[4]) addressing convergence under manifold hypotheses, log-concavity relaxations, and Wasserstein or KL guarantees are omitted. As a result, the theoretical positioning of this work within the broader landscape remains unclear.
2. **Redundant theoretical content:**
The connection between the drift (velocity field) and score function has already been well established in prior studies ([5]–[7], even though not cited), and some of the presented results appear to directly reproduce existing ones. In particular, Proposition 2 is an almost verbatim restatement of Corollary 2.1 in [8], without distinction regarding novelty.
3. **Questionable correctness of results:** Several claimed results seem mathematically imprecise. In particular, the reasoning in Theorem 2 is problematic: it separates the deterministic and stochastic components of an SDE despite the drift being nonlinear, which invalidates the argument under standard stochastic analysis. The claimed derivation of the Gaussian perturbation outcome therefore lacks rigor.
4. **Unrealistic assumptions:** The assumptions on boundedness and smoothness (e.g., uniform or infinite upper bounds on certain norms and distances) are overly restrictive and not justified under realistic neural network approximations. These conditions make the practical relevance of the theoretical claims doubtful.
5. **Vague definition of overfitting and underfitting:** The proposed definitions are not linked to the learning or optimization process—they are defined in terms of generative outcomes rather than training dynamics. Moreover, the chosen error structure lacks justification: it is unclear why such a specific functional form of the estimation error is meaningful or observable in practice.

**Presentation and Structural Problems**
1. **Formatting and compliance:** The manuscript seems not to follow the official ICLR template (e.g., presentation of theorems and propositions), producing an irregular and somewhat confusing layout.
2. **Frequent typographical errors:** Numerous typos and grammatical issues occur throughout (e.g., lines 78, 126, 152, 175, 190, 223, 455, 456–457). The paper clearly requires thorough proofreading and language polishing.
3. **Misleading or unclear figures:**
The toy illustrations do not convincingly relate to actual generative model behavior. It is unclear whether they stem from trained models or analytical constructions. Without methodological details, these visuals risk being pedantic and misleading rather than informative.

While the paper raises a relevant question about the behavior of generative models under finite training data, it lacks precise referencing, relies on overly strong assumptions, and reproduces known results without sufficient methodological rigor or clear novelty. Significant theoretical and presentation revisions are required before it can be considered a reliable contribution.