Entropic Time Schedulers for Generative Diffusion Models
Authors: Dejan Stancevic
Florian Handke
Luca Ambrogioni
Venue: NeurIPS 2025 poster ()
Keywords: Generative Diffusion Models
Information Theory
Conditional Entropy
Noise Scheduling
Stochastic Differential Equations (SDEs)
Primary area: deep_learning
TL;DR: We present a time scheduler that selects sampling points based on entropy rather than uniform time spacing, ensuring each point contributes an equal amount of information to the final generation.

Abstract: The practical performance of generative diffusion models depends on the appropriate choice of the noise scheduling function, which can also be equivalently expressed as a time reparameterization. In this paper, we present a time scheduler that selects sampling points based on entropy rather than uniform time spacing, ensuring that each point contributes an equal amount of information to the final generation. We prove that this time reparameterization does not depend on the initial choice of time. Furthermore, we provide a tractable exact formula to estimate this \emph{entropic time} for a trained model using the training loss without substantial overhead. Alongside the entropic time, inspired by the optimality results, we introduce a rescaled entropic time. In our experiments with mixtures of Gaussian distributions and ImageNet, we show that using the (rescaled) entropic times greatly improves the inference performance of trained models. In particular, we found that the image quality in pretrained EDM2 models, as evaluated by FID and FD-DINO scores, can be substantially increased by the rescaled entropic time reparameterization without increasing the number of function evaluations, with greater improvements in the few NFEs regime. Code is available at
\url{https://github.com/DejanStancevic/Entropic-Time-Schedulers-for-Generative-Diffusion-Models}.

Entropic Time Schedulers for Generative Diffusion Models
Dejan Stančevié Radboud University
Florian Handke Ghent University
Luca Ambrogioni Radboud University
Abstract
The practical performance of generative diffusion models depends on the appro- priate choice of the noise scheduling function. which can also be equivalently expressed as time reparameterization In this paper, we present time scheduler that selects sampling points based entropy rather than uniform time spacing, ensuring that each point contributes an equal amount of information to the final gen- eration. We prove that this time reparameterization does not depend on the initial choice time. Furthermore, we provide tractable exact formula to estimate this entropic time for trained model using the training loss without substantial over- head. Alongside the entropic time, inspired by the optimality results, we introduce a rescaled entropic time. In our experiments with mixtures of Gaussian distributions and ImageNet, we show that using the (rescaled) entropic times greatly improves the inference performance of trained models. In particular, quality in pretrained EDM2 models, as evaluated by FID and FD-DINO scores, can be substantially increased by the rescaled entropic time reparameterization without increasing the number of function evaluations, with greater improvements in the few NFEs regime. Code is available https: com Dej jnStancevicc Models
found that the image
Entropic- T Schedul for Seanerative-Diffe Sr
1 Introduction
Generative diffusion models (Sohl-Dickstein et al., 2015). and especially score-based diffusion models, have achieved state-of-the-art performance in image (Dhariwal & Nichol 202T Rombach
et al., 2022 Song et al. 202T) and video generation (Ho et al. diffusion models are obtained by reverting a forward diffusion process, which injects noise into the distribution of the data until all information has been lost. In practice, the performance of these models is highly dependent on the choice of noise scheduling function that regulates the rate of noise-injection (Song et al. 2022). In most commonly used models, change of noise scheduling mathematically equivalent to a change of time parameterization. From a theoretical perspective, equivalently of noise scheduling, is not constrained by theory
[022]SSigen et al. 2022). Generative
the choice of time parametrization, since any change of time in the forward process automatically corrected in reverse dynamics (Song et al. (021). However, explained above, the choice of time is very important practically since it affects both the temporal weighting during training and the discretization scheme during inference. Consequently, 'incorrect' choice of time variable can lead to severe inefficiencies due to the under- sampling of some temporal windows and the redundant over-sampling of others. This is particularly problematic since recent theoretical and experimental work suggested that 'generative decisions' tend to be clustered in critical time windows (Li & Chen 2024), which have been connected to
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

symmetry- breaking phase transitions in physics (Raya & Ambrogioni 2023 Ambrogioni 2025
Biroli et al. 2024 Sclocchi al., 2025). The "triviality" of the first phase of diffusion prior to the initial phase transitions has led to the idea that this early phase can be skipped in one 'jump' using pre-trained initialization Lyu et al. (2022). These late initialization schemes can be seen as a special case of time re-scheduling that compresses the high-noise part of the original schedule.
The idea of changing the diffusion time data-dependent way, also known as time- same-warping, was first introduced in Dieleman et al. 20222 in the context of a class of diffusion models for sequences of discrete tokens. However, their implementation required the used of special architectures trained with cross-entropy loss instead of the standard denoising score matching. In this paper, we show that natural data-dependent time parametrization can tractably obtained for any continuous generative diffusion model as the (rescaled) conditional entropy of Xg given Xç- This choice of time leads to constant entropy rate, meaning that each time point contributes the final generation in an equal amount. Furthermore, we show that this entropic time is invariant, meaning that it does not depend on the original choice time parameterization. Examples of the same SDE in the entropic time and standard time are given in figure [ a rescaled entropic time. We provide an exact tractable formula that relates these quantities to the empirical EDM (Karras al. 2022) and DDPM (Song et al. 2022) loss, which can be used to easily define the entropic time for any given trained network.
Furthermore, inspired by the optimality results, we introduce
Space
Liiropp
Standard Time
Entropic Time
Figure 1: An example of the same SDE and its conditional entropy in the standard and entropic time.
2 Related work
Accelerated Sampling Procedures One the most significant challenges in current diffusson models the slow generative process. Since the introduction of the connection between the diffusion models and SDEs (Song et al. 2021]. a wide array of research has aimed to address this issue by designing better numerical integrators. Some of the research in that direction includes the works of Liu et al. (2022) and Lu et al. (2022). An alternative line research focuses on optimizing the sampling time schedule itself. Sabour et al. (2024) presents principled approach to optimizing sampling schedules in diffusion models by aligning them with stochastic solvers, enabling higher efficiency, Wang al. 2024) splits the generation process into three categories (acceleration, deceleration, and convergence steps), identifies imbalances in time step allocation, and introduces methods to address them, leading to faster training and sampling. Lee et al. (2024) uses spectral analysis of images to design a sampling strategy that prioritizes critical time steps, improving quality while reducing the number of steps. Li al. (2023) explores joint optimization of time steps and architectures for more efficient generation without additional training. While much of this research focuses on learning or empirically determining optimal sampling schedules, our work provides more theoretical perspective based on ideas from information theory. The closest work to ours is Dieleman et al. (2022), which they use a cross-entropy loss to deduce a time-warping function for diftusion language models. However, our work differs since we analyze the standard diffusion models, where cross-entropy not available. Furthermore, their expression for the entropy is not exact, as implies an assumption of conditional independence of the tokens given the noisy state. On the other hand,

---

here we provide exact formulas that can be applied to any generative diffusion model trained with denoising score matching, both in the continuous and in the discrete regime.
Connection Between Entropy, Information Theory, and Diffusion Models Diffusion models are inherently tied to concepts from information theory, particularly in the context of denoising Gaussian noise, which is fundamental operation in information- thhooretthhhhcccccceo frameworks. This connection has inspired growing body of work exploring the interplay between diffusion models and information theory. Premkumar (2024) investigates entropy-based objectives for learning more robust generative models. Kong et al. (2023a), Kong al. (2023b), and Franzese al. (2025) aim to provide clearer understanding of diffusion models through an information -ooommtiionthhoottiic lens. Although these works explore the connection between information theory and diffusion models, and employ similar equations to ours, our focus diverges slightly. We use information theory as a guide to design better sampling algorithms. Work exploring similar direction to ours is et al. (2025). However, they explore the conditional entropy between two consecutive time steps given a fixed discretization grid, while we look at the conditional entropy between the current time step and time zero in a way that invariant under the change of time and discretization.
3 Background on score-matching generative diffusion
The mathematics of generative diffusion models can be elegantly formalized in term of stochastic differential equations (SDE). Consider target distribution p(xo) defined by a data source such distribution of, for example, natural images, sound waves, or linguistic strings. We interpret this data source as the initial distribution of a diffusion process governed by the SDE:
dX: f(Xx, t)dt + g())dW
(I)
where dW is a standard Wiener process, ((ee,) () is a vector-valued drift function, and g() is scalar volatility function, which regulates the standard deviation of the input noise. The marginal densities of the process can be obtained from the Fokker- Planck equation:
d @xpc(xt)= Edu, Ous (-1x(x0n)) + g(()) 0m px(x)),
where O; the partial derivative with the respect to time and 02₁, the partial derivative with respect to the j-th component of x,. We denote the forward "solution kernel" of the diffusion process as p(x, y). which the solution of the Fokker- Plank equation for po(y) = 5(y xo). The core idea of generative difffusion is to sample from x. by initializing asymptotic noise state xT (where T is large enough for the SDE to reach its stationary distribution) and by "inverting" the temporal dynamic. This can be done using the reverse SDE:
dX, (f - g(t) logp(()) dt g(t)dW,
which can be proven to give the same marginal densities of eq. when initialized with the appropriate stationary distribution, which is usually Gaussian white noise. We denote the reverse solution kernel of the reverse dynamics q(xo x;), which can be interpreted as the optimal denoising distribution. The data-dependent key component of the reverse dynamics is the so-called score score function, which can be written expectation over the optimal denoising distribution:
V log pr (x.) Egtxa| xx) V log (x xx)]
In most practical forms of generative diffusion, the score function is approximated using deep network sø (x,, t), where the parameters 0 are optimized by minimizing an upper bound on the quadratic score-mater sotrematchhig loss:
LsM (0) = (pp(==)).~A(8)
(xs, t) (ogg((x:)])
(5)
E (pp
[|5o t) V lo xo) xx)1l m0]
LUSM
where X(1) is density defined on the time axis. Note that LsM(O) and LDSM (0) differ only by constant and therefore have the same gradients and optima. However, L(O) is substantially more tractable as it does not require samples from the unknown optimal denoiser g(xo xt).
3

---

4 Optimal sampling schedule as a change of time
In this section, we revisit a result from Sabour et al. 2024 and notice some interesting features. Inspired by it, we formalize what we mean by the change of time.
Obtaining the analytical expression for the optimal sampling schedule difficult and, in most practical cases, impossible. However, (Sabour et al. 2024 shows that for the EDM noise schedule (Karras) al., 2022), the optimal sampling schedule for the ODE flow when data comes from normal distribution with variance e² has an analytical expression. More precisely, the sampling schedule, [Iminn /1, mm.s,, that minimize the KI. divergence is given by
t = ammnn + N (Amaa ammnn))
arctan
(6)
where Ommnn/ /maz arctan
(see theorem 3.1 in Sabour al. 2024). turns out that this
schedule is also optimal for the deterministic DDIM (Song et al. 2002), which we show in Appendix
C Moreover, since the DDIM solver is invariant under time change (Lu et al.] [2022), its optimal
schedule remains invariant under time change, making it particularly suitable for comparing different time parameterizations.
This implies that even in the simple case, the optimal schedule depends on the data distribution. In addition, this result frames the optimization of sampling schedule as a problem of time change. Rather than selecting timesteps differently for different numbers of sampling steps (e.g. EDM scheduler), theorem 3.1 shows that one should think of the sampling schedule as transformation of time such that the sampling schedule becomes linear in the new time. Furthermore, in section 5.3 we will connect equation with the conditional entropy production.
4.1 Change of time
The change of time in SDEs is powerful technique used simplify their analysis and solutions. By altering the time variable, the dynamics of the SDE can be transformed into more manageable form. More information can be found in section 8.3 in Lawler (2010).
Definition 4.1. We say a function @ is proper time change if continuous and strictly increasing.
It can be shown that given a proper time change, J. and a random process, X., that solves the SDE dX,, fe(X,, t)dt + (r),), then Y, = Xo(t)) solves dY, e(t) ((t))
V (())). Guided by the theory of time change, we define an equivalence between SDEs.
Definition 4.2. Given two SDEs dX, f(x,, + g(t)dW, and dX,, = f(x,, s)ds + g(s)dW we say that they are equivalent up to a time change if there exists a proper time change, Φ : + such that
1. ((t)f(xx((t)) = f(x,t)
2. (o(1))((())) =t())
Furthermore, we can require /(0) = 0 without affecting anything (since it equivalent to subtracting constant from the original function). By requiring that, we get that a time change between two SDEs is unique it exists. Under a time change, the forward kernels stay the same, in the sense that p=(z(0o) late) (z(20) holds (this follows from Y; = X Essentially, time change squeezes and stretches the time axis but does not fundamentally change the diffusion process. Algorithm | shows how to implement sampling using time change.
Given this notion of equivalence, a natural question arises: there a preferred or canonical time parameterization? We argue that a conditional entropy, H[zoo[z]], and quantities derived from it are good candidates. However, for H H[zo|[zs]] to make sense, we assume that we are given an initial distribution, (=())) (that is, a data set). Therefore, besides an SDE, we require dataset for the entropic time. the further text, we will always assume that the dataset given and is the same for different time parameterizations of SDEs.

---

Algorithm Sampling using time change
1: procedure TTMECHANGESAMPL.ERR (4;) 1M ( (6(44) (l), Do(a,o);
solver(I, Da, ceur Dext, Scur. Snext)
2:
J f -((o) + x=((((t)) o(4))) ) for j = N -
> Uniform spacing in new time
3:
ij f interp(Ty; {o(t))) M {t+); M) for j = 0, .... ......
> Corresponding old time
4:
Jj,8j + oll)), s({) for j O,... 1
5:
0,8 + {0, (0,0),11 {1,
6:
sample I (0,0
7:
forje {N,... 1 do
8:
Curr next + 0j.
9: Scur: Snexi f $-.
+- solver( Do: oeur Tnext, Sour: 8next)
D e.g. Heun, DDIM, etc.
11: end for
output
13:
end procedure
Entropic time schedules
In this section, we introduce the concepts of entropic time and rescaled entropic time. First, provide some reasons for using the conditional entropy as new time parameterization. Then, we show how to obtain the conditional entropy in practice and show its connection with commonly used quantities in diffusion literature. Furthermore, we demonstrate that the entropic time parameterizations are well-defined and invariant under the initial time parameterization of the SDE. There are several possible choices for the entropy function, which highlight different aspects of information transfer. The most straightforward choice is the information transfer T,. Consider initial source Xg Po transmitted through a noisy channel px,, Xn). which is determined by the solution of the SDE given in eq. 1 The noise-corrupted signal is received and decoded using (x0 x,). The amount of information transferred at time can be quantified as the difference between the prior and posterior entropy:
T, H H[xo] H Xo) x.] = 1[xo X
where H[xo] E (xo) [log p(x)] is the entropy of the source, H[x xx] = Bp(ao.as) [log (xx))) xx)] is the conditional entropy under the optimal denoising distribution, and I ['x;XX, is a mutual infor- mation. Therefore, it is natural to interpret this quantity as the amount of information available at time concerning the identity of the source data. Up to constant shift, this is equivalent to using the time variable (it) H Xy xt] in the forward process. This time axis is defined by having constant conditional entropy rate between the final generated image and the noisy state at time t.
5.1 Characterizing the conditional entropy
Having established that conditional entropy makes sense as new time parameterization, question arises: How do we calculate it in practice? In general, conditional entropy can be written as
H[x, H H[xo] 1[x;: x,] H H(x, [x] H[x.].
In practice, H[x; xt] is easy to get once the forward kernel is known, but difficult to obtain numerical value of H[x,]. However, by looking at time derivative of the conditional entropy, we get a method for obtaining a numerical value. The time derivative is given by
H x.] H[x, H[x.].
(8)
Hence, know the time derivative, need calculate the time derivative of H(x.). In case when an SDE is given by 4.1 the entropy production is given by
o2 E Ep Vll[/Vloop()xx]
[x] = (p.. (V.))
2
The equation is well-known expression in nonequilibrium thermodynamics for entropy production (Premkumar, 2024). The derivation of the expression can be found in the appendix B Similarly, we can obtain the similar expression for H(x,|xa) Combining these two expressions, we obtain
z
H Xx x,]
(Ep( log log xx) (p,(,,) |I/V log
(10)
5

---

Note that this expression depends on the data distribution only through the Euclidean norm of the score function, which is approximated by a neural network in diffusion models.
5.2 Estimating the entropy rate from the training loss
In this section, we present a connection between the conditional entropy rate and training loss. For more details on the derivation of these results, see the Appendix [F In practice, most diffusion models can be written using the framework introduced in Karras et al. (2022) In this framework, the SDE is written as d.X,
A()) X,dt + s(t) (2 (f)dW, with p( mo) (2z,; (t)) as
x(e)
forward kernel. This leads to the following conditional entropy production
Dt (t)
H[xo|xx
=(t))
log
(11)
where D a dimension of the space (e.g. for the MNIST dataset, it would be 28²). In the rest of this paper, we will be using this framework. The squared error, ef encapsulates our uncertainty at time t about the final sample Z0 and is given by
= (x.) (m(ool mn)l| Epprolnn mr) (y 1o]11211 = (o. (tr
(12)
Using the fact that we can write 2 x get
as (1) (1) pe (x+))) (see Appendix G) we
d(t)
[|xooxx]=
(1133 ct.
(13)
Recognizing that SÑ - and H[x xx] -[x xy equation 13 precisely the well-known
I-MMSE identity from information theory (Guo et al. (2055.
Furthermore, integrating over time yields an expression for the conditional entropy:
X x1] = -- S. SNN(()) et dt.
(14)
This expression coincides with the continuous- time (infinite-stepy limit of the variational lower bound derived by Kingma et al. (2021), revealing direct informition-theorrtiic characterization as already hinted by Kong et al. (2023a).
The previous results provide simple way of estimating the conditional entropy rate from the standard loss function of trained diffusion model due to a close connection between the squared error and the loss. This provides a tractable way to estimate the conditional entropy from the training error. Note that, using the error of the model entails an approximation since the entropy is defined with respect to the true score function and, therefore, does not take into account the discrepancy between the learned and true score.
To analyze this deviation, we start from a striking result: the conditional entropy production is, up to multiplicative factor, the gap between the explicit and denoising score matching loss 55 In fact, following the steps from Vincent (201T) and keeping track of the terms that are constant in 0, we have
LSM( (
LDSM (0) (2 A(t))) [ log p(x (x0) l/V
(15)
Using expression 10 we can rewrite the above equality
2 H(x0
CDsM(() (!) + Ex(e)
(16)
oo
This relation can also be expressed at single time point as
H X xx]
+
g2_
(0)
gi
Ex [|so(xe,)) V log (x, x=))
(17)
2
where (0) = E (x, so (x,, t) V log log (ogg)(xx))1)) [2] denotes the mean squared error between the true score and its neural approximation. The right-hand side corresponds to our estimate of the conditional entropy production. It follows that the estimated entropy production always upper-bounds the true value, with the gap determined by the disagreement between the learned and true scores, (0) In this sense, X x.] can be interpreted as the irreducible contribution to the loss, reflecting the intrinsic uncertainty of the optimal denoising process.

---

5.3 The entropic and rescaled entropic times
Here, we introduce a rescaled entropy and show that both rescaled entropy and conditional entropy proper changes of time and are invariant under different time parameterizations of SDE. Proofs can be found in the Appendix [ First, we notice that in the case of continuous data, the conditional entropy goes negative infinity at time equal zero. In practice, this is not observed since diffusson models always start from non-zero initial time. However, it adds arbitrariness to the overall curve of the conditional entropy. To combat this problem, guided by the observation that the change of time for the optimal sampling schedule for normally distributed data, eq. 6 is equal to the rescaled entropy (see Appendix Eb, introduce rescaled entropy fo o(T H [xx) x, |dT. Algorithm shows how to estimate rescaled entropy in practice (and how it was estimated in this work).
Theorem 5.1. Given an SDE and initial data distribution (x) o(t) H(ro[æ and <(t) = Jo ( (T) [xo[x+]dr are proper time changes.
We call these time parameterizations entropic time and rescaled entropic time, respectively. Naturally, an important question emerges: How does the time parameterization of an initial SDE influence its reparameterized form? We show that an SDE written in entropic time is unique and does not rely on its initial parameterization. More precisely, given two SDEs equivalent up to a time change, the SDEs expressed in their respective entropic times are equivalent up time change, with the time change being the identity function (i.e. drift and noise terms of SDEs in entropic times are related by conditions 1. and 2. from definition 4.2 and are the same since the time derivative of the time change is one).
Theorem 5.2. Given two SDEs given in definition 4.2 and following time changes
1. b0:===/7
2. 2, : [x
3. 4. 8 H2(xo/x,)
it follows that
F F: o H,, + H. [x[xo[xs]]
is proper time change implementing the equivalence and is equal the identity map. F == id.
A similar result holds for the rescaled entropic time well. Therefore, reparameterized in entropic time (or rescaled entropic time), no matter the starting SDE time parameterization, drift and noise are always the same.
Algorithm Estimation of rescaled o()
procedure BTT.AABRRRBSLLLDEERROOOOOOOOODOD s(t), (t)) M)
2: sample 2/ ~ Po for j = 1,
3: Xg
4: RANN < 0
5: for i [0, .o,...., N 1} do
6:
sample vj ~ N(0, I) for j = 1,....A M
7:
+ [P,..., vAr]
8:
X1, - x(t;)x0 + (t::)o(i:v
9:
No Da(x,, (1:))
10:
2 1 M
M -==
{ x
11:
R + R, + d²(())
a(t.)
(4:41 - 22)
D Riemann sum
12:
end for output R&N 14: end procedure

---

5.4 Spectral rescaled entropic time
based our definition rescaled entropic time on optimality results for isotropic Gaussian distributions. However, these results do not account for how different directions in anisotropic Gaussian influence the optimal schedule. From equation TO we observe that the total entropy production can be interpreted can be any orthonormal set (since only norms of scores affect the production). This interpretation corresponds to one specific way of weighting different directions. For image data and diffusion in pixel space, we explore an alternative in this paper: setting the rescaled entropy in each Fourier basis direction to be equal to 1 the final time (i.e., giving equal importance for each frequency), and then weighting them by their respective amplitudes. Theorems from the previous section still hold for the spectral rescaled entropy since they hold for each frequency (basis). An example of the resulting rescaled entropy across different frequencies is shown in figure
sum of contributions from all basis directions, where the basis
0.8
Radial Prequency Magniude
Normulized Time Change
33
0.4
0.2
Resealed Entropy
Spectral Rescaled Eatropy
EDM with 128 steps
10
10
10"
10
Time
Figure 2: Normalized rescaled entropy function of radial frequency for the red channel in ImageNet-64, together with normalized rescaled entropy, spectral rescaled entropy, and HDM with 128 steps.
Experiments
We compare the performance of few-step generation in the standard, entropic, and rescaled entropic times for several low- www-dimessionl examples where an analytic expression for score is easy to calculate. Next, we compare the performance of trained EDM and EDM2 models (Karras et al. 2022
2024 on CIFAR10 (Krizhevsky et al.] 2009), FFHQ et al. (019)), and ImageNet (Russakovsky
al.
2015) using the FID (Heusel et al. 2017) and FD-DINOV2 (Oquab et al. 2023 Stein et al.
2023
scores. More details about the setup can be found in the appendix
6.1
One-dimensional experiments
We used an analytic expression of a score function to compare the performance of a few-step generation process different time parameterizations in one dimension. We used equidistant steps in the standard, entropic, and rescaled entropic times. We used the stochastic DDIM solver (Song al. 2022). We compared those schedules for discrete data and a mixture Gaussians, We used the Kullback- Leibler divergence to compare results for different schedules. An example of KL divergence behavior against the number generative steps given in figure 3 In general, we can see that the discrete case, the entropic time outperforms other schedules by large margin, while the standard schedule gives the worst results. Furthermore, we noticed that when variances of Gaussians are much smaller than the distance between them (i.e. there is no significant overlap between Gaussians), the entropic schedule gives better results. However, when the variances are not negligible the mixture of Gaussians case, see that the rescaled entropic schedule gives the best results, while the

---

entropic schedule underperforms. This suggests that the entropic time might significantly improve certain discrete diffusion models.
10P
Det
1071
Eroppic Time
Entropic Time
10-1
Entropi:
10"7
Rescaled Entmpic
Standard Time
Stunkedd Time:
10'
10P
10
Nomber of geacrative
Nomber generative sleps
(a) Discrete
(b) Continuous
Figure 3: Kullback- divergence against the number of generative steps for different time parameterizations for mixture 15 data points (discrete) and Gaus- sians (continuous).
6.2 CIFARIO, FFHQ, and ImageNet
We compared the performance for different numbers of generative steps using standard, entropic, rescaled entropic, and spectral rescaled entropic (for diffusion pixel space) times. To sample, used the deterministic and stochastic DDIM solver. For CIFAR-10 and FFHQ, the EDM unconditional VP models were used (Karras al. 2022). ) For ImageNet-66 64, the EDM2-S and EDM2-L models were used, while for ImageNet-512, the EDM2-XS XS and EDM2- XXL models were used. For ImageNet-512. we used both models: one optimized for FID and the other for FD-DINOv2.
Figure an example of the effect of different schedules generated images. We observed that the rescaled entropic schedule produces images with lower brightness. The results for FFHQ are pre- sented in table while the FID scores for FID-optimized networks and the DI- NOv2 scores for DINOv2-optimizer net-
Table 1: FID scores for different sampling schedules FFHQ 64 x 64
Solver
Schedule
FID
NFE-16
NFE-32
NFF-64
Stochastic DDIM
Rescaled Entropy
30.81
14.89
7.60
Spectral Resealed Entropy
30.61
14.60
7.33
works on ImageNet-512 are reported in table 2 We note that the using the rescaled entropy schedule, model EDM- XS beats the FD-DINOv2 result pro-
EDM
11.13
5:41
Deterministic DDIM
Rescaled Entropy
8.10
4,28
3.16
Spectral Rescaled Entropy
8.10
4.28
3.14
vided in Karras et al. (2024), 103.39, obtained using Heun solver. We observed that the entropic time produced unrecognizable images (see Appendix (H), therefore, have not included it in the results. The difference between spectral rescaled entropy and rescaled entropy is small but noticeable for stochastic DDIM, with spectral rescaled entropy performing better. In contrast, for deterministic DDIM, the difference is negligible. Results on CIFARIO and FFHQ, together with more examples of generated images, given in appendix H
6.3 Limitations
While our results show a clear benefit of using the entropic schedules across wide range of datasets and fast-sampling methods, we note that these benefits are observed specifically for the first-order solvers. We tried second-order solvers as introduced in Karras et al. (2022): Lu et al.) (2022) but noticed worse results (compared to the EDM schedule). We believe this due the use of an inappropriate information transfer function, eq. 7 Specifically, the definition we use considers only current time point, whereas second-order solvers also take into account the future time point when predicting the updated state, thereby altering the entropy rate. As result, the mismatch in temporal perspective may lead to suboptimal performance for higher-order methods as their entropy curves probably need to be readjusted based the features of the solver.

---

Table 2: FID and FD-DINOv2 scores for different sampling schedules for ImageNet-512
HID
HD-DINOO2 4
Solver
Network
Schedule
NFE=16
NFE=32
NFE=64
NFE=16
NFE=32
NFE=64
EDM
32.31
10.01
4.98
294.25
149.91
107.00
EDM2-XS
Stochastic DDIM
Rescaled Entropy
13.64
4.98
3.80
182.11
109.68
97.10
EDM
30.39
8.80
3.81
218.10
95.21
60.79
EDM2-XXL
Rescaled Entropy
13.38
3.83
2.60
108.16
57.05
46.75
EDM
10.42
4.81
3.83
156.46
115.94
107.05
HDM2-X8
Deterministcc DDIM
Rescaled Entropy
7.57
4.44
3.75
157.32
116.52
106.84
EDM
9.68
3.47
2.41
79.56
52.60
46,27
EDM2-XXL
Rescaled Entropy
5.91
2.78
2.14
68.36
48.26
43.83
(a) EDM
(h) Rescaled Entropy
Figure 4: Comparison of generated images using EDM and rescaled entropic schedules with the same random seed. Images were generated using deterministic DDIM with NFE 8, 16, 32, and 64.
Conclusions and future work
Several avenues for future work remain open. We conjecture that using conditional entropy provides an optimal schedule for discrete generative tasks, although we currently lack a theoretical proof; nonetheless, our toy examples showed great promise. Empirically aligning entropic time with discrete diffusion models, potentially in the spirit of time warping techniques such as Dieleman et al. (2022), an exciting direction. Beyond this, entropic time may also offer principled framework for training and model compression: in distillation, entropic time could identify the most informative stages for supervision and reduce redundancy in transferring knowledge from teacher to student models (similarly could be done for consistency models). More broadly, we propose entropic time candidate training schedule, enabling learning that directly aligned with information flow. We encouraged by the unexpected connection between variational objective Kingma et al. (202I). We envision that this perspective could eventually replace the heavy dataset-specific optimization required in approaches such as EDM (Karras et al. 2022), leading to more efficient and adaptive training across diverse modalities, including medical imaging, audio, and text. Lastly, we note that second-order solvers, which incorporate lookahead steps, may require fundamentally different definition of information transfer. Developing entropic analogues tailored to such solvers is another important direction for extending this framework.
formulation and the continuous-time
10

---

References
Luca Ambrogioni. The statistical thermodynamies of generative diffusion models: Phase transitions, symmetry breaking, and critical instability. Entropy, 27(3):291, 2025.
Giulio Biroli, Tony Bonnaire, Valentin de Bortoli, and Mare Mézard. Dynamical regimes of diffusion models. Nature Communications, 15(1), November 2024. doi: 467-8/44447--024442225
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in Neural Information Processing Systems, 2021.
Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, et al. Continuous diffusion for categorical data. arXiv preprint arXiv:2211 1,5009, 2022.
Alex Dytso, H Vincent Poor, and Shlomo Shamai Shitz. Conditional mean estimation in gaussian noise: A meta derivative identity with applications. IEEE Transactions Information Theory, 69 (8):1883-1898, 2022.
Giulio Franzese, Mattia Martini, Giulio Corallo, Paolo Papotti, and Pietro Michiardi. Latent abstrac- tions in generative diffusion models. Entropy, 27(4))371, 2025.
Dongming Guo, Shlomo Shamai, and Sergio Verdú. Mutual information and minimum -quare егтоr in gaussian channels. IEEE transactions on information theory, 5144::2211-22) 2005.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 2022.
Tero Karras, Samuli Laine, and Timo Aila. style-based generator architecture for generative adversarial networks. Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, 2019.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 35:26565-2663 26577, 2022.
Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24174-24184, 2024.
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models, Advances in neural information processing systems, 34:21696-21707,, 2021.
Xianghao Kong, Rob Brekelmans, and Greg Ver Steeg. Information-theored diffusion. arXiv preprint 0tt.:2302.037722 2023a.
Xianghao Kong, Ollie Liu, Han Li, Dani Yogatama, and Greg Ver Steeg. Interpretable diffusion via information decomposition. arXiv preprint 0rXiv:2310.07972. 2023b.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Gregory F Lawler. Stochastic calculus: An introduction with applications. American Mathematical Society, 2010.
Haeil Lec, Hansang Lee, Seoycon Gye, and Junmo Kim. Beta sampling is all you need: Efficient image generation strategy for diffusion models using stepwise spectral analysis. arXiv preprint arXiv:2407.12173. 2024.
Lijiang Li, Huixia Li, Xiawu Zheng, Jie Wu, Xuefeng Xiao, Rui Wang, Min Zheng, Xin Pan, Fei Chao, and Rongrong Ji. Autodiffusion: Training-free optimization of time steps and architectures for automated diffusion model acceleration. Proceedings of the IEEF/CVF International Conference Computer Vision, 2023.
11

---

Marvin Li and Sitan Chen. Critical windows: non-asymptotic theory for feature emergence in diffusion models. arXiv preprint arXiv:2403.01633, 2024.
Shigui Li, Wei Chen, and Delu Zeng. Improving denoising diffusion with efficient conditional entropy reduction. 2025.
Luping Liu, Yi Ren. Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. arXiv preprint arXiv:2202.0977 2022.
C Lu, Y Zhou, F Bao, Chen, and C Li. A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Proc. Adv. Neural Inf. Process. Syst., New Orleans, United States, pp. 1-31, 2022.
Zhaoyang I Lyu, Xudong Xu, Ceyuan Yang, Dahua Lin, and Bo Dai. Accelerating diffusion models via early stop of the diffusion process. arXiv preprint arXiv:2205. /2524, 2022.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprini wrwiv:2304.0011150 2023.
Akhil Premkumar. Neural entropy. arXiv preprint arXiv:2409.038) 2024.
Gabriel Raya and Luca Ambrogioni. Spontaneous symmetry breaking in generative diffusion models. Advances in Neural Information Processing Systems, 36:66377-66389, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffuusin models. Proceedings of the IEEE/CVF conference on computer vision and Pattern Recognition, 2022.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-252, 2015.
Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis, Align your steps: Optimizing sampling schedules in diffusion models. arXiv preprint arXiv:2404. 2024.
Antonio Sclocchi, Alessandro Favero, and Matthieu Wyart. A phase transition in diffusion models reveals the hicrarchical nature of data. Proceedings of the National Academy of Sciences, 122(1): c24087991211 2025.
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint MeXX::2209.1492 2022.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International Conference on Machine Learning. 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. International COnference Learning Representations, 2022.
Yang Song. Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-basee based generative modeling through stochastic differential equations. International Conference on Leurning Representations, 2021.
George Stein, Jesse Cresswell, Rasa Hosseinzadeh,, Yi Sui, Brendan Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L. Caterini, Eric Taylor, and Gabriel Loaiza-Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. Advances in Neural Information Processing Systems, 36:3732-3784, 2023.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural compula- tion, 23(7):1661-1674 2011.
Kai Wang, Mingjia Shi, Yukun Zhou, Zekai Li, Zhihang Yuan, Yuzhang Shang, Xiaojiang Peng, Hanwang Zhang, and Yang You. A closer look at time steps is worthy of triple speed-up for diffusion model training, aiviv preprint arXiv:2405. 17403, 2024.
12

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See sections 6.3 and
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
13

---

Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We believe that the information provided the paper should suffice for reproducing the main results in the paper. Furthermore, our code is made available.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer: [Yes]
Justification:
Link to our GitHub repo is provided in the abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/t.pss.ccppbllc/guuii /
wwt.Subbmiiiioooiiioooooooio for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: For the main results of the paper, we believe that all the relevant information can be found in the paper. See Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: All the relevant metrics for images were calculated only once and that number is reported.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
15

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [No]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http://aaaaaa...cpppaaaa//aaaaaa
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See appendix
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
16

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
17

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
18

---

Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
19

---

A Broader impact
This work proposes entropic and rescaled entropic time schedules for generative diffusion models, improving performance, especially in low NFE regimes, without additional training overhead. Our approach may benefit applications relying fast generation, such medical imaging and digital con- tent creation. Furthermore, our work contributes to deeper understanding of the relationship between information theory and generative modeling, which may inspire further theoretical advancements.
As with all generative models, there potential misuse in generating synthetic media that could be used for disinformation impersonation. Improvements in efficiency may increase such risks by lowering the barrier to large-scale generation.
B Entropy Production
Here, we show
oi R
H x (x.) [V())
(18)
By looking inside the integral of H[x,]. we get
H[x,] = /
+ p(ax) p(az) p(xz) d p(ze)doe
dr
/ p(a,) log
(19)
p(x;) log (ogp(z))dr,
Assuming our dynamic is determined by the SDE 4.1 simplify the derivative as follows
the Fokker-Planck equation
H[,, = /
g 2
v Vlogp(zz) p(zz)) togp(x))dr
ft
/
of
Vlogp(a:)
v Vlogp(z,) p(z,)dz,
2
o v
V lllV
(ooop(<))) log
2
(20)
/
(folVp(z)))de: + /
gt
v Vlogp(z:)] p(x;))
V(i))((s)dda + L E Ealtaa)ll log [
2
= (x.) [V. + o
2 Ep, log p(x))
which is exactly what we wanted show. We used integration by parts in going from the first line to the second and from the fourth to the fifth.
C Optimal schedule for deterministic DDIM
Here, we show that the optimal schedule for the deterministic DDIM (Song al. 2022 for the EDM SDE is the same as the one given in Sabour et al. (2024).
Theorem C.1. Let Pdata (x) NO, T) and the diffusion process is given by the EDM SDE. Sample Ximax ~ p(x, buuxx) and use the deterministic DDIM using n. steps along the schedule
tmax
tr > tn-1 >... t1 to tmin
obtain Xtmin The optimal schedule t" minimizing the KL KL-divergence between p(x, tmin) and the distribution of Xtmi is given by
t ctan ((1-2) amin + -maax
20

---

where
Omin arctan((miu
O'max arctan(fmax/
Proof. The deterministic DDIM update is given as
Xz, = (x((x,))) + lic-1
(xt, - =x(().))
4
where eo (xx,) + 1 V log p(xx)). By using an analytical expression for the score, we get simplified expression for the update
e + 4-144 2 +
This turns be exactly the same update as ODE Euler method (equation 18 Sabour et al. (2024). Therefore, our claim follows from the proof of theorem 3.1. in Sabour et al. (2024).
D Proofs from section 5.3
Theorem D.I. Given an SDE and initial data distribution po()), s(1) [x and Φ(()
Jo dsa, H H(x Xx] are proper time changes.
Proof. As already mentioned, proper time change must be strictly increasing, continuous function. Since H [nn (ra] has a derivative (see section 5.1), we need to show that it is positive. However, our
claim follows from equation 13 (the squared error is equal to zero only when an initial distribution consists of one data point).
Theorem D.2. Given two SDEs as given in definition 4.2 and following time changes
1. 6 + + == f(t
2. 4, [x(]
3. 4, 88 [x)
it follows that
F 0 H + H2 [x]
is proper time change implementing the equivalence and is equal to the identity map, F = id.
Proof. Immediately, changes. Similarly, using a chain rule, is observed that implements the equivalence. Furthermore,
that g is proper time change since it is composed of other time
F(H, (o. ( (H, (xo (xo,))) 4 (o(()) H. H,(xo|xx(0)
(21)
However, since pr(x) = 2o(0) (z) and p( = (=)) it follows
[x|x/|x // p p( 2o) (p(2g (x)=d)=)))drddrr
(22)
// (( mo) In (g(20) (( )drddrr = H,
where v, = Zo(t) (i.e. are the same spatial point) and time subscripts represent at which point in time the probability distribution is evaluated. This proves that F id.
Similarly, we can prove the same claim for the rescaled entropic time since o(1) (d(()) for any proper change of time ø.
21

---

E Rescaled entropy for Gaussian data
Here, we show that, in the case of the EDM noise schedule, the rescaled entropic time is the optimal sampling schedule for the ODE flow when data comes from a normal distribution with variance c? (equation 6
Recall the expression for the rescaled entropy. So (T) (//o///]d...0 From equation
we have
L ( (x dr S( (Dā(r) - (T))) (oogg//)) dr.
(23)
Using the facts that (T) T, s(T) = and Vlogg- "(1))
we get
I d H) ('' D D- 72 +
D
dr
(24)
S 72 +2
De²
(
22
dr Dearctan
Therefore, linear sampling schedule, [tmins t1, mntmm in the rescaled entropic time is given by
4
i
Dearctan
Dc amin +
(amaz Amnin))
(25)
N
where O'min/ arctan (min//ma
Exactly the same as equation
F Connection with squared error and loss
In this Appendix, we show connections between conditional entropy production and some commonly used expressions in the diffusion literature. Firstly, we show how the conditional entropy production is related to the squared error at time t, ez.
// ||:0- io fo(x:))l /aao (xo
et =
Ro {x(x))111]= (5o+ + (())0(())7((ll ||
(26)
2o
s(t)
The squared error encapsulates uncertainty at time about the final sample vo- The following simplification of the above equation gives a more precise meaning.
c² = Ep (2x) [=p( E Ex(volz,) [yo]]]
(27)
(x.( (t
From Appendix G
know
V Var (arp(oole)(xx)) +0)===000==(())()))))))0+0000000
(28)
Hence,
2 = Bp.(o.))
=O(1) - =0 log
a() (Da(1)
o(t) ((()) - (Do() (
(29)
a1)3
o(t) H[x]
22

---

Following notation from Karras et al. 2022) for the loss at time t, we have
(() = ((() Cout (1) Fa s(1)x0 + ( + +())0(())))
(30)
The formula for prediction (21))) is given by
Cour (t) + (akip(t)az
(0)))
(31)
s(1)
We express the loss at time t using the squared error
c(1) x())E|l==((xx= (x)))((x))) =
(32)
Furthermore, using connection between a squared error and a conditional entropy production, we
get
o(t) H[xo|x|]= (()30)9((0)321)) C(I.
(33)
G Tweedie's second-order formula
Assume are given a distribution p(y) that is obtained by adding a Gaussian noise distribution q(r), i.e. (z/2z) Ny: sz, sz,s 2²0))
Now given some y p(y). if are interested in which ((z) generated it, the best can do is guess ß(y) = E [z)). Tweedic's ' formula gives us
V log)(y)
E [x(
(34)
Now, we might ask how sure we are of our guess, To answer that question, we need to look at the variance, Var (Wrg(z/)) [x] (g(ely) [x²]] Bycalas [x]] In this section, derive the following result
V Vang(ria)) [x] so V v En [x] = o² + 2²0 [log p(y))).
(35)
However, a more general result regarding the cumulants of (<<ll)) holds (Dytso et al. 2022). That is, all the cumulants can be calculated using the score function and its derivatives.
Since we already have Eg(arl [z)). we need to find expression for Eg(+1n)
f dx g(vl=)((x)
x2 = / do g(x)x (y)aaa ply)
E Egiz/y) [x-] =
p(y)
/ du ry((x) va(u)a) + 820"N (VlVv)((y)z)
(36)
p(v)
(
8002
V, dzq(vl=))(xxz
sp(y)
Where in going from the first line to the second, we used Vyq(s/=) 25 (()))))))) However, we seem have encountered problem with the second term in our expression. However, by using q(x,)) v) {(vl=){(z) = q(x v))(), for the second term we get
V, / drg(vlz)g(z))= = v, daq(sll))(u))
= V, ((w)) /
drq(=lu)x = Vp(p(y)
(37)
-En(eto) (zr (r)V,,(()) + p(v) V EeEmmn
23

---

Hence,
s²a²
Roo(elo) [x-]
(R. R vP()) p(y) V (
sp(y)
yl vi
0²0² E (2)))) a v log + y R (x))
(38)
V + s 0²0² log
8²0²
Bogall) [x]]
[2]
s
5²0²
Now, we get an elegant expression for the variance
Van (x [z] sa 2 o s2' log
(39)
So far, have been dealing with mne-dimensional random variables, but it is easy to generalize all the steps to arbitrary dimensions, which gives us the general formula
V Yarg(zly) [x] so² vyEgeeln) = Q2 5²0 (logp(y))
(40)
H Experimental details and Additional results
H.1 Algorithms
Here, we present algorithm 3 which estimates the spectral decomposition of the squared error from the data. Having numerical values of the squared error and its spectral decomposition can be used to compute other entropic quantities of interest. A similar algorithm can be used to obtain entropy and squared error for different orthonormal basis.
Algorithm 3 Estimation of spectral decomposition of (t)
1: Input: Dø(a, .). Encoder(x)), (I), s((), 4e.oo...,NY, M
2: for i (O....... N do
3: Sample xd po for j I,...,M
4: 2o = Encoder(Xo)
5: Sample V, (0, I) for j 1....,M M
6: Z, s(t,) zo + s(t,) (t,) V
7:
20 = Do (Z+ (1.))
8:
for
M do
9:
A z/ =
A FFT2D (1))
11:
Modulus squared for each matrix entry
12:
end for
13:
(t.) M Σn 10
Matrix of squared error for different frequencies
14:
end for
15: output (t)
H.2 One-dimensional experiments
We used an analytic expression of a score function to compare the performance of few-step generation process in different time parameterizations one dimension. We used equidistant steps in the standard time, entropic time, and rescaled entropic time. All entropic quantities were obtained from the squared error using equation 13 The squared error was estimated at 10a equidistant timesteps with 10 samples at each timestep. We used mixture of data points (discrete case) and

---

Mixture of
Mixture of 15
10P
107
Discrete
10'
Eitrappcc Time
tintrppic Time
Resualad Eatropic Time
Resualed Entropic Time
Stardand Time
Stardand Time
Number of generative steps
Number of generative
10
10/
10/
Continuous
Dat.
Dat.
10
Entropi: Time
Entropic Time
10
Rescaled Entropic Time
10
Rescaled Eatropic Time
Standard time
Standard time
10P
10
10P
10'
Number generative steps
Number generative steps
Figure 5: Kullback-Leibler divergence against the number of generative steps for different time parameterizations for mixture of data points (discrete) and Gaussians (continuous).
mixture of Gaussians (continuous case). In both cases, the data had a mean of zero and a standard deviation of one. For sample generation, used the stochastic DDIM (Song et al. 2022). Results are given in figure 5
For the discrete case, the performance was measured by creating nonoverlapping bins around data points, [a; -, a; te), and calculating the Kullback-Leillrr divergence between the initial distribution and the binned distribution (pbin (a;) probability of a generated sample ending up in the i-th bin). seriance-presevvngg SDE and EDM SDE were used for our experiments. Datapoints were randomly initialized and mullback-Leibbee divergence estimated 10 times using 10 different paths, the mean and variance of the KL estimate could be obtained.
For the continuous case, the performance was measured by estimating the SDE-generated distribution using Gaussian kernel density estimation (with a standard deviation of 10-2) and then evaluating the KI. divergence using Monte-Card methods with 10 samples. Similarly to the discrete case, the KI. divergence was estimated 102 times using 1 different paths to estimate the SDE-generated distribution.
H.3 CIFARIO, FFHQ, ImageNet
For ImageNet-64, we used EDM2-S and EDM2-L models and for ImageNet-512, we used EDM2-XS and EDM2-XXL models provided by Karras et al.] (2024).
For CIFARIO and FFHQ, we used For generating samples, we used
unconditional VP models provided by Karras et al. (2022).
the stochastic and deterministic DDIM (Song et al." 2022). To compare performance between different runs, we used the FID (Heusel et al. 2017) 7) and, for ImageNet, FD-DINOv2 (Oquab) al., (2023] Stein al., 2023) scores provided by the Karras et al. (2024) implementation. We used FD-DINOv2 as it correlates better with human preferences (Stein et al.) (023). We used implemantations provided by //ttps://githhi com Hom/NVlabs/eedd for CIFAR 10 and FFHQ, and https://g/ thub. WWVabb//ddma for ImageNet. We generated 50000 images and compared them against pre-computed reference statisties. All reported results are from a single (first) run. Class labels for ImageNet were drawn from uniform distribution.
25

---

For all data sets, entropy and rescaled entropy were calculated using an estimation of squared error using equation [3] The squared error was estimated 128 time points according the EDM schedule (p 7, min = 0.002, mazz 80) using the Monte-Carlo method with 1024 samples at each timestep. In order to obtain (rescaled) entropy, any numerical integration technique should work. We decided on the simplest one, taking the difference in time steps, multiplying it by the derivative, and cumulatively summing it up to a time point t.
We decided on 128 time points by comparing it to 512 time points for CIFARIO 10 and FFHQ, and noticing no perceivable difference in the final entropy curves. Also, rescaled entropy was calculated for ImageNet- 64 with both network sizes, S and L, and there was no significant difference between them, as expected (since depends only on forward process and the initial data distribution). Therefore, we used the smallest models to estimate entropic quantities. Regarding a spectral rescaled entropy, 10000 images were used to estimate amplitudes of different frequencies.
As already stated in the main text, the entropic time generated blurry images and was not used for further comparison. An example of images generated with the deterministic DDIM solver using the entropic schedule over 64 steps, with the EDM2-L model, is given in figure 6 Results obtained for CIFARIO and FFHQ are given in tables and T respectively. Results for ImageNet-64 are given in table + Examples of generated images for ImageNet-64 using the EDM and rescaled entropy schedules are given in figures 8 and 9 For the sake of completeness, we include FID and FD FD-DINOv2 scores for models optimized for FID scores and models optimized for DINO scores in tables 5 and o respectively. We observe some interesting behavior of these results, such as DINO-optimized models giving better FID than FIDD-optimized ones for NFF=16. Also, we can see that the FII score can go up while the DINO score steadily decreases for DINO-optimized models. This shows that those two metrics asses and value vastly different properties of generated images. In addition, we notice that our DINO results are comparable to the results provided in Karras et al. (2024) obtained using Heun second-order solver. Figure 7 show how the number of function evaluations affect the generated images when using EDM and rescaled entropic schedules. Examples of generated images for ImageNet-512 using the EDM and rescaled entropy schedules with stochastic DDIM are given in
figures 10
and while Examples of generated with deterministic DDIM are given in figures 13
14 and
15
Table FID scores for different sampling schedules on CIFARIO 32 32
Solver
Schedule
FID..
NFE-16
NPE-32
NFE-64
13.76
6,36
Stochastic DDIM
Resealed Entropy
20.07
8:41
1.65
Spectral Resenled Entropy
19.77
8.28
4.47
FDM
9.06
4.18
2.77
Deterministic DDIM
Rescaled Entropy
6.07
3.30
2.52
Spectral Rescaled Entropy
3.24
2.51
Table 4: FID and FD-DINOV2 scores for different sampling schedules for ImageNet-64
FID 4
FD-DINOv2
Solver
Network
Schedule
NFE=16
NFE=32
NFE=64
NFE=16
NFE=32
NFE=64
FDM
20.03
8.18
3.81
263.60
135.67
86,32
EDM2-S
Rescaled Entropy
11.69
4.95
2.75
194.02
109.55
81.25
Stochastic DDIM
Spectral Rescaled Entropy
11.46
4.76
2.70
193.81
109.16
79.75
EDM
22.60
9.46
4.44
284.74
141.70
79.86
EDM2-L
Rescaled Eatropy
13.56
5.59
3.06
208.27
108.31
72.06
Spectral Rescaled Entropy
13.46
5,51
2.99
207.76
106.42
70.37
EDM
5.00
2.49
1.90
128.25
99.64
92.88
EDM2-S
Rescaled Entropy
3.46
2.15
1.77
117.26
98.28
93.34
Deterministic DDIM
Spectral Rescaled Entropy
3.54
2.12
1.80
118.50
99.01
95.14
EDM
5.49
2.55
1.82
120.35
84,57
74.87
EDM2-L
Rescaled Entropy
3.63
2.09
1.65
104.88
81.98
75.87
Spectral Rescaled Entropy
3.75
2.09
1.61
106.52
82.64
75.16
26

---

Table FID and FD-DINOv2 scores for different sampling schedules for ImageNet-512 for models optimized for FID scores
Solver
Network
Schedule
FID
FD-DINOv2 1
NFE=16
NFE=32
NFE=64
NFE=16
NFE=32
NFE=64
EDM
32.31
10.01
4.98
419.27
199.69
131.94
EDM2-XS
Stochastic DDIM
Rescaled Entropy
13.64
4,98
3,80
280.82
154,24
124.07
EDM
30.39
8.80
3.81
337.23
127.69
68.97
EDM2-XXL
Rescaled Entropy
13.38
3.83
2.60
186.68
82.47
57.41
EDM
10.42
4.81
3.83
212.78
154,41
137.68
HDM2-X8
Deterministic DDIM
Rescaled Entropy
7.57
4.44
3.75
222.15
161.79
142.22
3.47
137.35
81.26
65.75
EDM2-XXL
Rescaled Entropy
2.78
125.35
79.80
Table 6: FID and FD-DINOv2 scores for different sampling schedules for ImageNet-512 for models optimized for DINO scores
FID L.
FD-DINOv2 1
Solver
Network
Schedule
NFE=16
NFE-32
NFE=64
NFE=16
NFE-32
NFE-64
EDM2-XS
EDM
17.14
8.76
7.34
294.25
149.91
107.00
Stochastic DDIM
Rescaled Entropy
7.76
6.39
6,43
182.11
109.68
97.10
EDM
7.36
218.10
95.21
60.79
HDM2-XXI,
Rescaled Entropy
5.02
108.16
57.05
46.75
156.46
EDM2-XS
Deterministic DDIM
Rescaled Entropy
5.31
5.40
157.32
116.52
106.84
EDM
5.48
4.10
4.03
79.56
52.60
46.27
EDM2-XXL XXL
Rescaled Entropy
3.81
3.54
3.70
68.36
48.26
43.83
Figure 6: Images generated with the deterministic DDIM solver using the non-rescaled entropic schedule over 64 steps, with the EDM2-L model. It is clear from these images that rescaling is crucial in the continuous regime, probably due to the divergence of the differential entropy at t - 0.
27

---

(a) EDM
(b) Rescaled Entropy
Figure 7: Comparison of generated images using EDM and rescaled entropic schedules with the same random seed. Images were generated using deterministic DDIM with NFE = 8, 16, 32, and 64.
(a) EDM
(b) Rescaled Entropy
Figure 8: Images generated with the stochastic DDIM solver using the EDM schedule (left) and rescaled entropic schedule (right) over 64 steps, with the EDM2-S model.
(a) EDM
(b) Rescaled Entropy
Figure 9: Images generated with the stochastic DDIM solver using the EDM schedule (left) and rescaled entropic schedule (right) over 64 steps, with the EDM2-L model.
28

---

AIR
(a) EDM
(b) Rescaled Entropy
Figure 10: Images generated with the stochastic DDIM solver using the EDM schedule (left) and rescaled entropic schedule (right) over 64 steps, with the EDM2-XC XS DINO-optimizz model.
(a) EDM
(b) Rescaled Entropy
Figure 11: Images generated with the stochastic DDIM solver using the EDM schedule (left) and rescaled entropic schedule (right) over 64 steps, with the EDM2-XXL DINO-optimized model.
29

---

EDM
(b) Rescaled Entropy
Figure 12: Images generated with the stochastic DDIM solver using the EDM schedule (left) and
rescaled entropic schedule (right) over 64 steps, with the EDM2 XXL FID-optimized model.
T SOMAA!
BOSDIGA!
(a) EDM
(b) Rescaled Entropy
Figure 13: Images generated with the deterministic DDIM solver using the EDM schedule (left) and rescaled entropic schedule (right) over 64 steps, with the EDM2-XS DINO-optimized model.
30

---

EDM
(b) Rescaled Entropy
Figure 14: Images generated with the deterministic DDIM solver using the EDM schedule (left) and
rescaled entropic schedule (right) over 64 steps, with the EDM2-XXL DINO optimmzzdd model.
(a) EDM
(b) Rescaled Entropy
Figure 15: Images generated with the deterministic DDIM solver using the EDM schedule (left) and rescaled entropic schedule (right) over 64 steps, with the EDM2-XXL FID-optimized model.
31


Reviews:
[{"review_id": "HWhHQJFVWV", "reviewer": "Reviewer_hpGc", "summary": "This paper proposes Entropic Time Schedulers, a diffusion sampling scheduler that uses $H[x_0|x_t]$ as the time parametrization. The authors started from the motivating observation from Sabour et al. 2024 that even a simple data distribution such as $\\mathcal{N}(0, c^2I)$, the optimal  sampling schedule has a parameter $c$, which depends on the data distribution. More importantly, the authors draw inspiration from Sabour et al. 2024 to think of the time scheduler as a transformation of time. Then the authors move on to propose the time parametrization of $H[x_0|x_t]$, which is the conditional entropy of the optimal denoising distribution $p(x_0|x_t)$ from Sabour et al. 2024. They formally derive the time derivative of $H[x_0|x_t]$, i.e. the entropy rate, and connect it with the training loss. They then introduce the concept of \"entropic time\" and \"scaled entropic time\", the latter of which is a stretched version of the former when integrating out the time for the entropy rate. To account for the unrealistic isotropic assumption in Sabour et al.'s optimal denoising distribution, the authors further propose spectral rescaled entropic time. In their experiments, the proposed entropic time schedulers show better performance than baselines in FFHQ and ImageNet.", "questions": "Practically, how is the scheduler constructed? How is the integral of the entropy rate calculated? What is the computational complexity? \n\nThe authors analyze the gap between Score Matching and Denoising Score Matching and the influence on the entropy rate, but how would this affect the sampling results?", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "\\+ I like this paper a lot. It is very well written and very easy to follow. \n\n\\+ As I summarized above, the thought process is very clean and principled. \n\n\\+ All derivations are formal, though I didn't check the details of all the derivations and proofs. \n\n\\+ The authors also discussed the gap between ground-truth entropy rate and the one calculated from Denoising Score Matching. \n\n\\- This is probably a limitation that the authors didn't explicitly state: constructing the proposed scheduler seems to need to traverse the whole dataset for multiple times.", "was_revised": true, "final_justification": "This paper propose a diffusion sampling scheduler with a clean and principled derivation, providing new insights in this critical module of diffusion models. I recommend accepting this paper.", "rebuttal": "Thank you for the detailed review and thoughtful feedback. Below we address specific questions and the pointed out weaknesses.\n\n### Q1: Practically, how is the scheduler constructed? How is the integral of the entropy rate calculated? What is the computational complexity?\n\nWe provide the relevant details in Appendix H.2, and we will improve visibility by including a pseudocode in the main paper. Specifically, entropy and rescaled entropy are estimated using the squared error in Equation 13, evaluated at 128 time points according to the EDM schedule ($\\rho = 7$, $\\sigma_{\\min} = 0.002$, $\\sigma_{\\max} = 80$) via Monte Carlo with 1024 samples per step. The entropy integral is computed using a simple numerical integration scheme: we multiply the estimated entropy rate by the time step difference and apply cumulative summation. This method is computationally efficient and easily implementable.\n\n### Q2: The authors analyze the gap between Score Matching and Denoising Score Matching and the influence on the entropy rate, but how would this affect the sampling results?\n\nThis analysis is intended to clarify the theoretical relationship between entropy and different loss functions. Although not directly used in our sampling procedure, it reveals that the estimated conditional entropy using denoising score matching is always greater than or equal to the true conditional entropy. We included this primarily for theoretical insight.\n\n### W1: Constructing the proposed scheduler seems to require traversing the whole dataset multiple times - is this a limitation?\n\nThis is an insightful concern. However, we empirically observe that it is not necessary to traverse the entire dataset to compute the (rescaled) entropy. In practice, using a subset of the data suffices to obtain accurate estimates. A possible explanation is that the score function already captures global information about the dataset, allowing us to sample $x_0, x_t \\sim p(x_0, x_t)$ from a subset without significant loss of accuracy. This observation is consistent with the estimation method described in Appendix H."}, {"review_id": "QdHGjZ9DdP", "reviewer": "Reviewer_3sWm", "summary": "While the paper presents an interesting perspective by reparameterizing diffusion time based on conditional entropy, the proposed method lacks sufficient methodological depth. The core idea revolves around a theoretical reinterpretation of time scheduling rather than the development of a novel algorithmic mechanism. No new training framework, architectural design, or solver adaptation is introduced to complement the entropic schedule. Moreover, the scheduling function is computed post hoc from the training loss and is not jointly optimized with the model, limiting its potential. The method is also tightly coupled to first-order solvers and fails to generalize to higher-order methods, which further restricts its applicability. Finally, the spectral rescaled entropy schedule is introduced heuristically without theoretical grounding or ablation analysis, reducing the overall rigor and innovation of the approach.", "questions": "How do you justify the use of conditional entropy over mutual information as the time-rescaling metric, especially considering both are involved in Eq. (7)?", "limitations": "The paper does not propose any new architectural modification or algorithmic mechanism beyond reparameterizing the time axis based on estimated conditional entropy.", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 2, "significance": 2, "originality": 1, "strengths_and_weaknesses": "The paper does not propose any new architectural modification or algorithmic mechanism beyond reparameterizing the time axis based on estimated conditional entropy. The practical implementation essentially relies on post hoc schedule computation without integration into the model training pipeline.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback. However, we respectfully believe that several core contributions of the paper have been overlooked. Below, we address the major concerns.\n### On Novelty and Theoretical Contribution\nOur work introduces a novel and theoretically grounded framework for rescheduling diffusion time using conditional entropy. This includes:\n\n- The formulation of entropic time and rescaled entropic time as new families of time reparameterizations, grounded in information-theoretic and stochastic process theory.\n- Theoretical results establishing that these schedules are invariant under time reparameterization of the forward process (Theorems 5.1 and 5.2).\n- A practical, tractable estimator of entropic time (Equation 13).\n\n### On Practical Utility and Empirical Performance\nOur method is not only theoretically motivated, it is also practically useful. In all experiments, the proposed rescaled entropic time schedule significantly outperforms the standard EDM schedule. We observe improvements in FID and consistency across a range of datasets and solvers (Tables 1–4).\n\n### On Post Hoc Schedule Optimization\nWe believe that the simplicity of our approach, which does not require retraining and is agnostic to the architecture of the model, is a key strength of the method. While we agree that integrating our schedule into the training pipeline is a valuable direction for future work, our current focus is on improving sampling efficiency at inference time, which remains a critical challenge in modern generative modeling. For instance, the EDM schedule is highly optimized for inference performance. Identifying better sampling schedules can significantly enhance the practical utility of diffusion models.\n\n### On Solver Generality\nWhile our method performs well with first-order solvers, we observed weaker results with second-order methods. As we explain, this is likely due to a mismatch in temporal structure: the current entropy-based schedule only considers uncertainty at a single time point, whereas second-order solvers depend on both current and future states. We believe this limitation can be addressed, and that extending the entropic time framework to account for this “look-ahead” behavior could enable similar benefits for higher-order solvers as well. \n\n\n### On the Spectral Rescaled Entropy Schedule\nThe spectral rescaled entropy schedule, introduced in Section 5.4, is meant to illustrate that our framework accommodates different orthonormal bases beyond the standard pixel-space representation. We chose the spectral basis as it is a natural example that aligns with practitioners' intuitions about frequency content and information distribution. This variant is motivated by the goal of equalizing the contribution of conditional entropy across frequency bands, thereby distributing denoising effort more uniformly.\n\nWe agree that this component could benefit from additional clarification, and we are happy to expand on its motivation and include a more detailed explanation in the final version.\n\n### On Conditional Entropy vs. Mutual Information\nUsing mutual information would lead to the wrong monotonicity, as it strictly decreases in time, therefore violating what constitutes a valid change of time. One could propose to instead use $-I[\\mathbf{x}_0;\\mathbf{x}_t]$. However, this is entirely equivalent to $H[\\mathbf{x}_0|\\mathbf{x}_t]$ as a time change function since it only shifts $\\phi(t)$ by an irrelevant constant factor (remark after definition 4.2.).\n\n### Summary\nWe believe the paper offers a novel and theoretically meaningful contribution to the study of diffusion models. It introduces:\n\n\n- A principled framework for designing entropy-based schedules.\n- Theoretical results demonstrating invariance and structure.\n- Empirical evidence showing improved sampling quality across datasets.\n\nWe hope that this clarification will lead to a reconsideration of the originality and significance of our work."}, {"review_id": "nQfKAOs3XK", "reviewer": "Reviewer_HXTo", "summary": "The paper tackles the task of finding better timestep schedules used during sampling of diffusion models. This is done by introducing *entropic time* which is a dataset-dependent time reparametrization of the diffusion process that ensures a constant rate of increase in the entropy of the denoising distribution $p(x_0 | x_t)$. The authors analyze this concept theoretically, demonstrating its independence of the initial SDE parameters, as well as its direct connection to the optimal schedule in the simple isotropic Gaussian setting. Experiments on both toy data as well as several standard image generation benchmarks demonstrates a clear benefit of using this parametrization over the commonly used schedules in practice.", "questions": "* Given the fact that this entropic time is only dataset-dependent, have the authors also explored the effects of using this parametrization as the noise schedule during the training of the diffusion model? I understand this requires a pretrained score model to compute accurately, but am curious whether a noisy estimation of it mid-training would suffice.\n* How would one scale this idea to large scale diffusion models such as text-to-image and text-to-video models? Would it make sense to learn a class-conditioned (or text-conditioned) time parametrization?", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n* The paper is extremely well written and easy to follow. \n* The core idea is simple, elegant, and well motivated. The paper also does a superb job explaining the connection between theoretical concepts and how to use them in practice.\n* The proposed approach is training-free, and can easily be applied to any pretrained diffusion model. \n* The experimental results are promising, demonstrating a significant boost in performance almost across the board. \n* The limitation section is quite thorough, which is very appreciated.  \n\n**Weaknesses:**\n* As discussed by the authors, the proposed entropic time parametrization seems to fail when used with higher-order solvers.  \n* As the case with all works that rely on optimizing the timestep schedule, this approach requires quite a few number of function evaluations to achieve best results and is not applicable to 1- or 2-step sampling scenarios. \n\n**Minor nitpicks:**\n* Figure 2 is very effective in showcasing the different schedules based on the radial frequency. I suggest also including a version of the same figure where the x-axis (time) is in logarithm scale to be able to better see the curves at the lower noise scales. \n* Figures 7-11 are all using the stochastic DDIM sampler which causes different starting seeds to result in completely different images during sampling due to the different schedules. Including a few more figures that use the deterministic DDIM (similar to Figure 4) would be good to see.", "was_revised": true, "final_justification": "The paper presents a theoretically motivated approach to optimize sampling schedules in order to speed up diffusion inference. The proposed approach is simple, elegant, and efficient to compute and the papers results are promising. As such, I recommend accepting this paper.", "rebuttal": "We appreciate your thorough review and constructive feedback. Please find below our responses addressing your specific questions and the weaknesses you highlighted.\n\n### Q1: Given the fact that this entropic time is only dataset-dependent, have the authors also explored the effects of using this parametrization as the noise schedule during the training of the diffusion model? I understand this requires a pretrained score model to compute accurately, but am curious whether a noisy estimation of it mid-training would suffice.\n\nWe agree that using (rescaled) entropic time as a training-time noise schedule is a promising direction. While our current project focused on exploring the connection between conditional entropy and diffusion models in the context of sampling efficiency, we plan to investigate the training-time use of entropic time in future work. As Equation 11 shows, entropy depends only on the squared norm of the score, not its direction. This suggests that even a semi-trained model might provide a sufficiently accurate estimate of the (rescaled) entropy.\n\n### Q2: How would one scale this idea to large scale diffusion models such as text-to-image and text-to-video models? Would it make sense to learn a class-conditioned (or text-conditioned) time parametrization?\n\nWe believe our approach is readily scalable to large models such as text-to-image or text-to-video diffusion models, especially since the entropic schedule is computed only once and reused at inference time. That said, using entropic time for training might have a larger effect and requires further investigation. We also find the idea of class-conditioned entropy compelling. While estimating class-conditioned entropy across broad categories may be feasible, conditioning on individual prompts likely would not be practical due to their rarity and the need for on-the-fly estimation (in the case of prompts). We plan to explore these directions in future work.\n\n\n### W2: As the case with all works that rely on optimizing the timestep schedule, this approach requires quite a few number of function evaluations to achieve best results and is not applicable to 1- or 2-step sampling scenarios.\n\nThank you for raising this point. Our goal was not to target consistency models or distilled models with extremely few sampling steps. It remains an open question whether an entropic time parametrization could benefit such approaches. We find this idea interesting and might include a discussion of it in the revised version.\n\n### W3: Figure 2 is very effective in showcasing the different schedules based on the radial frequency. I suggest also including a version of the same figure where the x-axis (time) is in logarithm scale to be able to better see the curves at the lower noise scales.\n\nThank you for the suggestion. We agree that applying a logarithmic scaling to the x-axis could offer better insight into the behavior of the schedules at lower noise levels. We will include this version of the figure in the revised paper.\n\n### W4: Figures 7–11 are all using the stochastic DDIM sampler which causes different starting seeds to result in completely different images during sampling due to the different schedules. Including a few more figures that use the deterministic DDIM (similar to Figure 4) would be good to see.\n\nWe agree that including more deterministic DDIM samples would help better isolate the effect of different schedules. We will include such figures in the updated version of the paper."}, {"review_id": "oADJvt89wE", "reviewer": "Reviewer_8dRB", "summary": "This paper proposes an effective noise scheduling method at inference time (generation time) for the continuous-time denoising diffusion generative model family, aiming to achieve high sample quality, even with a low number of function evaluations (NFEs), for both discrete and continuous random variables.\n\nInspired by Sabour et al. (2024), the authors aim to find a new time parameterization under which linear-in-time discretization yields the best sample quality. To achieve this, the goal is then to design a change of time function $\\phi(t)$, which must be invertible — hence, continuous and monotonically increasing.\n\nThe core idea is to use information transfer — or, more precisely, the mutual information between clean data $X_0$ and noisy data $X_t$ — to guide the time change. The authors propose a schedule where the rate of information transfer (i.e., its time derivative) increases linearly. Intuitively, this means that the time scheduling should be designed such that the information gain at each discretization point is maximized equally, leading to better sample quality.\n\nSpecifically, instead of directly using mutual information, the authors define the time change function as $\\phi(t) = H[X_0|X_t]$, and refer to this as the entropic time schedule (Theorem 5.1). Note that the mutual information satisfies $I(X_0; X_t) = H[X_0] - H[X_0|X_t]$ decreases monotonically as $H[X_0]$ is fixed while the forward process is a linear diffusion. As a result, $H[X_0|X_t]$ increases monotonically over time. This ensures that $\\phi$ is invertible and suitable for change-of-variable techniques.\n\nAlthough computing $H[X_0|X_t]$ exactly is technically challenging, the authors show that only the derivative of $\\phi$ is required for sampling (Definitions 4.1 and 4.2), and this derivative can be approximated. In particular, they express the rate of the entropic time schedule in terms of the denoising loss with respect to the true posterior (Equation 12) and propose a method to approximate it by plugging the learned score function into Equation 12 (Section 5.2).\n\nThe paper also points out that in the case of continuous random variables, the entropic time schedule may become unstable as $t \\rightarrow 0$, since $H[X_0|X_t] \\rightarrow -\\infty$. To address this, the authors propose a rescaled time schedule (Theorem 5.1).\n\nAdditionally, they demonstrate that the proposed change-of-time function is invariant to the distribution of $X_0$ when the forward diffusion is fixed and can be applied across different bases.\n\nFinally, they empirically validate their method on both discrete and continuous random variables, showing that it is effective in improving sample quality.", "questions": "While I understand the goal of finding a new time parameterization under which linear-in-time discretization yields the best sample quality, I am not fully convinced why linear time, in particular, is necessary. My question stems from a lack of familiarity with Sabour et al. (2024), which the paper builds upon. Suppose the key idea is to manage the rate of information gain. Would it not be more effective to maximize the information gain rate at each discretization step while minimizing it in regions that are not discretized? I’m not sure if I missed it, but is there any rationale — theoretical or empirical — for favoring a constant information gain rate over a more adaptive one?\n\n\nIn addition, the paper acknowledges that the plug-in approximation of Equation 12 introduces error. However, in lines 195–211, the analysis focuses only on how the ground-truth conditional entropy $H[X_0 | X_t]$ relates to the approximation error. Would it not be more appropriate to analyze the estimation bias introduced by using the learned score function in Equation 12, especially since this directly affects the accuracy of the entropic time schedule?\n\n\nMoreover, do you believe that the proposed principle could also benefit training, not just sampling? It would be interesting to see whether adjusting time sampling during training (based on the current estimate) could further enhance model performance.\n\n\nLastly, I noticed that Kingma et al. (2021) also provide relevant discussion regarding time reparameterization, particularly in relation to Definitions 4.1 and 4.2. While perhaps not a core baseline, this work seems worth citing as a related reference.\n\n\nKingma et al. \"Variational diffusion models.\" NeurIPS 2021", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**  \n\nI consider that the contribution of the paper is clear and important for several reasons. \n\nThe authors provide a compelling motivation for both the problem and the proposed solution. I believe the acceleration of the sample generation with diffusion-based generative models is still an important problem, and the paper proposes a solution based on information information-theoretic perspective. In particular, the authors articulate the core ideas in a way that is intuitive and accessible to readers, whether they are new to the topic or already familiar with the underlying concepts. \n\nThe theoretical derivations are concise and straightforward, and the overall presentation is clean and well-structured, which enhances readability and clarity. \n\n**Weaknesses**\n\nWhile the paper is technically sound and well-executed, it would be interesting to include further discussion on **why** **specifically linear-in-time discretization** under the proposed time parameterization leads to improved sample quality. This may (or may not) be relevant to the question stated below. This is not essential, but such insights could enrich the reader’s understanding.", "was_revised": true, "final_justification": "The revisions have further strengthened an already high-quality paper. While I believe a score of 5 is the most appropriate, I would have chosen 5.5 if such an option were available.", "rebuttal": "We value the detailed review and thoughtful feedback you provided. Our responses to your specific questions and the identified weaknesses are outlined in the following sections.\n\n\n### Q1: While I understand the goal of finding a new time parameterization under which linear-in-time discretization yields the best sample quality, I am not fully convinced why linear time, in particular, is necessary. My question stems from a lack of familiarity with Sabour et al. (2024), which the paper builds upon. Suppose the key idea is to manage the rate of information gain. Would it not be more effective to maximize the information gain rate at each discretization step while minimizing it in regions that are not discretized? I’m not sure if I missed it, but is there any rationale — theoretical or empirical — for favoring a constant information gain rate over a more adaptive one?\n\nOur goal was to design a time parameterization in which each sampling step corresponds to a similar level of denoising difficulty. Since diffusion models progressively reduce uncertainty, we reasoned that a meaningful measure of step difficulty is the amount of remaining uncertainty about the data at a given point. This led us to use conditional entropy as a proxy: by constructing a time axis where each interval between solver steps spans the same amount of conditional entropy reduction, we aim to distribute the solver’s effort uniformly over the denoising process. This is the core motivation behind entropic time, where uniform discretization naturally aligns with consistent denoising effort per step.\n\nRegarding the linearity: in simple Gaussian toy models, the optimal sampling schedule for various noise schedules turns out to be linear in a reparameterized time variable, such as rescaled or entropic time. More generally, for any set of sampling time points, there always exists a reparameterization in which those points are equally spaced. The deeper question is whether a single function should generate all sampling schedules across different number of steps (e.g., entropic time vs. EDM time). While we do not know the answer, it aligns well with the core idea behind entropic time (constructing a time change under which entropy is linear) and is supported by analytical results in simple toy examples.\n\n### Q2: The paper acknowledges that the plug-in approximation of Equation 12 introduces error. However, in lines 195–211, the analysis focuses only on how the ground-truth conditional entropy relates to the approximation error. Would it not be more appropriate to analyze the estimation bias introduced by using the learned score function in Equation 12, especially since this directly affects the accuracy of the entropic time schedule?\n\n\nWe agree that analyzing the bias introduced by using the learned score function in Equation 12 is an important direction. Equation 16 partially addresses this by implying that the estimate using a learned score is always greater than or equal to the true conditional entropy production (there is a typo in the paper: the sentence should state that the right-hand side is our estimate of the conditional entropy production, not the left-hand side). This inequality provides a useful lower bound, but does not fully characterize the bias. While a deeper analysis of the estimation bias would be valuable, it is highly non-trivial in realistic settings where neither the exact score nor a reliable score-free entropy estimator is available. We consider this a promising topic for future theoretical investigation. If you believe this point deserves greater emphasis or clarification in the paper, we would be happy to revise the corresponding section accordingly.\n\n### Q3: Do you believe that the proposed principle could also benefit training, not just sampling? It would be interesting to see whether adjusting time sampling during training (based on the current estimate) could further enhance model performance.\n\nWe agree that using (rescaled) entropic time during training is a promising direction for future work. While this project focused on understanding the relationship between conditional entropy and diffusion models in the context of sampling efficiency, extending the principle to influence training dynamics is a natural next step. We plan to explore this idea in future work and will add a discussion of possible approaches in the discussion of the updated manuscript.\n\n### Q4: Lastly, I noticed that Kingma et al. (2021) also provide relevant discussion regarding time reparameterization, particularly in relation to Definitions 4.1 and 4.2. While perhaps not a core baseline, this work seems worth citing as a related reference. (Kingma et al. \"Variational diffusion models.\" NeurIPS 2021).\n\nThank you for pointing out this relevant connection. Kingma et al. (2021) share conceptual similarities with our approach in exploring time reparameterization, but the goals and methods differ.\n\nKingma et al. show that their continuous variational lower bound is invariant under time reparameterization, and they propose a learnable sampling schedule designed to minimize the variance of the training loss. In contrast, our focus is primarily on the sampling phase and on demonstrating the usefulness of entropy and related quantities for understanding and improving diffusion processes. Rather than learning a schedule during training, we construct the entropic time axis directly from the properties of the learned model. Moreover, our method only requires access to a trained score model, while theirs depends on a specific training loss.\n\nInterestingly, their continuous variational bound also connects to our framework: taking its expectation with respect to $x_0 \\sim p_0(\\cdot)$ yields the conditional entropy. This further supports the motivation for considering entropic time in the training phase of diffusion models and also shows the ubiquity of entropy and information-theoretic concepts in diffusion models. We will be sure to cite and discuss this connection more explicitly in the revised version."}]

Author rebuttal:

Meta-review: Score-based generative models sample random variables approximately distributed as a target distribution by discretizing a backward SDE associated with a  forward diffusion process used to noise data points. It is known both in practice and theory that the performance of these procedures depend on the choice of the noise schedule used in the forward noising process. 

The authors propose to introduce a new time parameterization of the SDE to improve sampling quality and motivate in particular the use of the conditional entropy under the optimal denoising distribution as a good time parameterization. They propose (Theorem 5) two time parameterizations referred to as entropic time and rescaled entropic time parameterizations.   Although  the conditional entropy can be challenging to estimate, the authors propose a way of estimating the conditional entropy rate from the loss function which allows a practical use of the time parameterizations.

Reviewers found the paper well written and the proposed approach well motivated, in particular the practical consequences of the  theoretical framework convinced most reviewers along with the experiments which show promising performance on classical datasets.

The authors answer to most concerns during the rebuttal and this contribution is of great practical interest for a large ML audience in a very active area of research. The authors are encouraged to include the comments provided during the discussion to improve the paper (including in the main paper a pseudocode and the practical implementation of the scheduler, discussing time reparameterization at training time, discussing the bias introduced by using the learned score function).
Decision: Accept (poster)