Spectral Analysis of Diffusion Models with Application to Schedule Design
Authors: Roi Benita
Michael Elad
Joseph Keshet
Venue: NeurIPS 2025 poster ()
Keywords: Spectral analysis
Designing scheduling
Diffusion models
Primary area: general_machine_learning

Abstract: Diffusion models (DMs) have emerged as powerful  tools for modeling complex data distributions and generating realistic new samples. Over the years, advanced architectures and sampling methods have been developed to make these models practically usable. However, certain synthesis process decisions still rely on heuristics without a solid theoretical foundation. 

In our work, we offer a novel analysis of the DM's inference process, introducing a comprehensive frequency response perspective. Specifically, by relying on Gaussianity assumption, we present the inference process as a closed-form spectral transfer function, capturing how the generated signal evolves in response to the initial noise. We demonstrate how the proposed analysis can be leveraged to design a noise schedule that aligns effectively with the characteristics of the data. The spectral perspective also provides insights into the underlying dynamics and sheds light on the relationship between spectral properties and noise schedule structure. Our results lead to scheduling curves that are dependent on the spectral content of the data, offering a theoretical justification for some of the heuristics taken by practitioners.

Spectral Analysis of Diffusion Models with Application to Schedule Design
Roi Benita
Michael Elad
Department of Electrical and Computer Engineering Technion, Haifa, Israel
Department of Computer Science Technion, Haifa, Israel elad ceadssssechhion.ac.l
roibenita wwwpaaaaaaaas.eniin.cc....
Joseph Keshet
Department of Electrical and Computer Engineering Technion, Haifa, Israel
Abstract
Diffusion models (DMs) have emerged powerful tools for modeling complex data distributions and generating realistic new samples. Over the years, advanced architectures and sampling methods have been developed to make these models practically usable. However, certain synthesis process decisions still rely on heuristics without solid theoretical foundation.
In our work, we offer a novel analysis of the DM's inference process, introducing comprehensive frequency response perspective. Specifically, by relying on Gaussianity assumption, we present the inference process as closed-form spectral transfer function, capturing how the generated signal evolves in response the initial noise. We demonstrate how the proposed analysis can be leveraged to design noise schedule that aligns effectively with the characteristies of the data. The spectral perspective also provides insights into the underlying dynamics and sheds light the relationship between spectral properties and noise schedule structure. Our results lead to scheduling curves that are dependent on the spectral content of the data, offering theoretical justification for some of the heuristics taken by practitioners.
Introduction
Diffusion Models (DMs) have become powerful tools for generating high-quality, diverse signals, with applications in image, audio, and video synthesis. Alongside their practical success and the ability handle complex distributions, some aspects the diffusion processes still rely on heuristies rooted in empirical experimentation. A key example is choosing appropriate noise schedule for inference phase. Developing theoretical foundations for these heuristics may provide valuable insights into the diffusion process itself, and enable greater adaptation to different setups. Our work aims to provide such a theoretical backbone from spectral perspective, as outlined below.
Our starting point lies in a well-established observation; while the continuous description of DMs via SDE or the associated flow ODE (28 are mathematically well-founded. their practice necessarily deviates from these theoretical foundations, introducing various errors 4 24). A major source of error discretization, which replaces the analytical formulations by their discrete-time counterparts. Another source of error is the approximation error, originating from the gap between the ideal denoiser and its neural network realization.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Significant efforts have been made to minimize these associated errors, with advanced numerical offering various methods for better treating the discretization of the differential equations. A fundamental aspect is the decision on the time point discretization, which directly affects the synthesis quality. We refer to these anchor points as the noise schedule, highlighting their role in setting the noise variance introduced at each diffusion step. Realizing their importance,
solvers [28
20 21
researchers have recently shifted their focus from custom-tailored heuristics [T0] 23 13 5 to the
development of optimized noise schedules [26] 29 34 32 37
B 38 (6). More on these methods and
their relation to this paper's 's contributions detailed in Section 5
In this work, we analyze the reverse diffusion process in the spectral domain, presenting the generated output signal as the outcome of a linear transfer function operating on the i.i.d. Gaussian input noise. This analysis is enabled by assuming that the destination distribution to sample from is Gaussian, providing an explicit spectral expression of the generated signal and insight on the evolving dynamics throughout the process. Our analysis, applied to the discretized inference procedure, encompasses both DDPM [T] and DDIM [27] frameworks, thereby revealing the effects of deviating from the continuous formulation. In addition, we extend our study for both variance preserving (VP) and variance exploding (VE) numerical schemes (28).
Posing the derived explicit expressions of the transfer systems as functions of the noise scheduling parameters, we can optimize a noise schedule tailored to given dataset, its resolution, and the speci- fied number of sampling steps required. We demonstrate how effectively solving these optimization problems numerically yields a noise schedule that accounts for these data characteristies, discuss the relation between the noise schedule structure and spectral properties, and validate our approach with existing works. Finally, we apply our method to publicly available datasets, including CIFAR-10 [8) AFHQv2 6 MUSIC [22) and SCO9 (33, examining the relation to heuristic choices in past work.
In summary, our contributions are the following: (i) Assuming the target signal is drawn from Gaussian distribution, we present novel spectral analysis on the discrete diffusion reverse process and derive a closed-form expression for its spectral transfer function. (ii) We formulate an optimization problem to find an optimal noise schedule that aligns with the data characteristics. Our approach provides an effective solution without relying on bounds or constraints on the number of diffusion steps. (iii) We analyze the evolving spectral properties of the signal throughout the diffusion process and at its output, demonstrating how handcrafted noise scheduling decisions and related diffusion phenomena are often well- peee-cedd by our approach. (iv) Our spectral analysis examines various setups, including DDIM and DDPM procedures, VP and VE formulations, the selection of loss functions, and additional features such as expectancy drift.
2 Backkround
We introduce the notations and the framework of diffusion probabilistic models, which are designed to generate samples Xo E IR from underlying, unknown probability distribution p(xo). While the sampling procedure can be described as a Stochastic Differential Equation (SDE) or by its associated flow, an Ordinary Differential Equation (ODE), these formulations lack general analytical solutions and are instead discretized and solved using numerical methods (28). Accordingly, we turn to describe the discrete formulations DDPM [10] and DDIM 27 which stand as the basis for our work.
The diffusion process is generative procedure constructed from two stochastic paths: a forward and a reverse trajectories in which data flows [10]. Each process is defined as a fixed Markovian chain composed of latent variables. During the forward process, a signal instance is gradually contaminated with white additive Gaussian noise as follows [
X₁ Vaxi + v1 O1,E
(1)
where ar for é E [1, T) referred to as the incremental noise schedule and E, ~ NO, I). Under the assumption that or is close to zero, we get that the final latent variable becomes 2T ~ (O. direct consequence of the above equation is an alternative relation of the form:
X = / Vo + vT oLE E~N(0,11)
(2)
'Here adopt the variance- preeevviir (VP) setting: see Appendix(Floo the variance-exploding (VE) form.

---

for ār IT Based on the above relationships, the reverse process aims to reconstruct X, from the noise Xy by progressively denoising it. This can be written as
X VI -1-aa eo 1)) + 0,24;
1 1 at
(xxx
(3)
where Eo is the estimator of for given x2 time t with neural network parameterized by 6, Tt - (1 an) and (O. Alongside this stochastic formulation, The authors of 27 introduce DDIM, deterministic framework for the inference process, which can be utilized to enable faster sampling. While DDIM relies non-Markee Markovian forward process, preserves the marginal distribution as in (2). As a result, the sampling trajectory can be expressed by
Cr (x))) ) + V1-1a-a- ug-1 (x,:
X d
(4)
a
where {0, I....,
, {} is the time index in the DDIM formulation. Throughout the rest of the paper. we denote by ā the set of noise schedule parameters, {ār)z=o for DDPM and {ās]==o for DDIM.
3 Analysis of Diffusion processes
We consider the reverse process system that takes as input noisy signal XT and outputs In this section, we develop the transfer function, which characterizes the relationship between these inputs and outputs in the spectral domain. To do we assume that the output signals are vectors drawn from a Gaussian distribution,
Xg N N( Eo)
(5)
where M4 Rd and Eg Rddd A similar assumption used in previous work 24 26 31]
While this model simplifies the signal distribution, we will demonstrate that it facilitates spectral analysis throughout the diffusion process and allows design noise scheduling mechanism for different objectives.
3.1
The optimal denoiser for Gaussian input
The following theorem states a well-known fact: under the above Gaussianity assumption, the Minimum rean-Squared Error (MMSE) denoiser operating on x, recover Xy is linear. It is the Wiener Filter 335 and can be expressed in closed form.
Theorem 3.1. Let Xg ~ N(Ho En) and let x4 be defined by Then, the denoised signal obtained from the MMSE (and the MAP) denoiser is given by:
x Co + - -.))) (Vo,Eoxi + + ā.) m4)
A detailed proof is given in Appendix A
Here, we outline the main steps under the assumption of Gaussian distributions, where applying the Maximum A Posteriori (MAP) estimator is equivalent to minimizing the MSE. The MAP estimator seeks to maximize the posterior probability:
max log p(xo xx) min log pxxxp(xxxx) log p(xo).
(7)
By substituting the explicit density functions p(xo) and (xx:/xo) according to 2 into the above, and differentiating with respect to Xo we obtain the desired result.
3.2 The reverse process in the time domain
now turn to analyze the discrete sampling procedures, as introduced in 10 and described
in Section 2 We begin by focusing on the DDIM formulation presented in [27] as it highlights the fundamental principles more clearly and facilitates the analysis of faster sampling techniques. The lemma below describes the relationship between two adjacent time steps during the inference process.
3

---

Lemma 3.2. Assume ~ (H Eo) and let à be the noise schedule parameters, we have
X5-1 (0g =b, 20 x. + (1 ūs) ã5)
(8)
where Co,s = 0,D0 + (1 -ā,]L. and the coefficients are
02 = V1-a.-1 b, = =Vo---1-
va,y ag-1
vI 02
O25
The above is obtained by plugging the optimal denoiser into the DDIM reverse process in The derivation is given in Appendix [B] This lemma establishes an explicit connection between adjacent diffusion steps, incorporating the characteristics of the destination signal density function as expressed by Mo and Σn. along with the chosen noise schedule parameters Y.
3.3 Migrating to the spectral domain
Analyzing the diffusion process in the time domain can be mathematically and computationally chal- lenging, particularly in high-dimensional spaces (d >> 1). To address this, we leverage the spectral decomposition of the covariance matrix, simplifying the analysis and enabling the examination of the signal behavior along its eigen- directiness.nns.
Consider destination signal Xy drawn from a multivariate Gaussian distribution with a given mean Mg and a covariance matrix Eg, and let U E Ra be a unitary matrix whose columns are the eigenvectors of Σg. The projection of xo onto this eigenbasis, denoted by vo = U Xo, also follows Gaussian distribution. Specifically, Vo ~ N(pg Ao), where 1 U Mo RR is the transformed mean vector, and Ao E Rdd is positive semi-definite diagonal matrix, containing the eigenvalues of En. denoted {A; )-1 Based on Lemma 3.2. we derive the following property:
Lemma 3.3. Assume Xg NN
The covariance matrices associated with all intermediate steps } in the diffusion process are jointly diagonalizable using the eigenbasis U of Ev.
This lemma holds since all the diffusion steps in ( involve only Eg. its inverse, or the addition of the identity matrix, leading to covariance matrices that share the same eigenbasis. We now turn to describe the diffusion reverse process in the spectral domain. By projecting both sides X onto the eigenbasis U, we obtain the following result.
Lemma 3.4. Assume Xg ~ N(Po Eu) and let ā be the noise schedule parameters. The subsequent step in the reverse process can be expressed in the spectral domain via
V G(s) + M(s)
where V, denotes the projection the signal x, onto the eigenbasis U,
(s))
[ by Ctg (a
- Ao] and M(s) (1-ā,) (ā,Ao (1 ā.
The lemma is proven in Appendix C Equation describes the relationship between two consecutive steps in the reverse process. Note that both matrices, G(s) and M(s), diagonal, and thus, the reverse process in the spectral domain turns into a system of d independent scalar equations. Based on the above relationship, denoted as v.. The complete derivation of the following result can be found in Appendix as well.
may derive expression for the generated signal in the spectral domain,
Theorem 3.5. Assume Xa N( Eu) and let a be the noise schedule parameters. The generated signal in the frequency domain vo can be described as function of Vs via
vo Divs + +D2p0
(10)
where Vs ~ ~N D, =I= G(k) and D2 D2= (II G()) M(i).
Moreover, vo follows Gaussian distribution:
v. ND Dř),
v. E Rd.
(11)
Equation T0 provides novel view of the generated signal in the spectral domain. Specifically, we transfer function which models the relationship between the input signal v. and
view L0

---

the output Vo. Furthermore, since the matrices D1 and D2 are diagonal, the expression simplifies to a of scalar transfer functions, with the only parameters being the noise schedule, a.
So far, we examined DDIM. Similarly, closed-form expression for the stochastic DDPM method 10 is presented in the following Theorem and is proven in Appendix [
Theorem 3.6. Assume Xo (H Σo) and let be the noise schedule parameters. The signal vo generated by DDPM can be expressed as a function of VT via
T
1-1
vo = DIVV
+
Σ
(6) qz," + D220 j=1
II
(12)
1=1
where VT NO,,1), q
ش; (1-a) a.)
The terms D1, D2 and the matrices G(j) are
T-ax
defined in Appendix Moreover, vo follows Gaussian distribution:
T
i-1
No ~ ~ -N((Dau DJ
Σ (
(13)
i=1
1=1
Optimal spectral schedules
With the closed-form expressions in 10 and 12
explore different aspects of the diffusion process and examine how subtle changes in its design affect the output distribution. More specifically, key aspect in this design is the choice of the noise schedule. In the discussion that follows we demonstrate how the proposed scheme enables optimal scheduler design.
We start by focusing the direct dependence between the generated distribution and the noise schedule coefficients, a. We define the spectral domain probability density function of the diffusion process output p(vo; @). Our objective is to bring this distribution to become close possible to the original distribution, p(vo). Specifically, given a dataset with a covariance matrix, Eg, defined by the eigenvalues { and S diffusion steps, our goal is identify the coefficients a that minimize some distance 'D between these two distributions. This results in the following optimization problem with set of specified constraints:
á arg min D (p(vo: Vo: Q).((v)))
(14)
subject
ão 80, ag = 851
ā.-1 2 ā. for ,,..., S.
The equality constraints ensure compatibility between the training and the synthesis processes. This involves ending the diffusion process with white Gaussian noise and starting it with very low noise to capture fine details the objective distribution accurately [19]. The inequality constraints align with the core principles of diffusion models and their gradual denoising process
[10]. The distance 'D between the probabilities can be chosen depending on the specific characteristies of the task. this work, we consider the Wasserstein-2 and Kullback-Leible divergence, but other distances can also be used. The theorems presented below are detailed in Appendix [
The Wasserstein-2 distance (or Earth Mover's Distance) measures the minimal cost of transporting mass to transform one probability distribution into another. the case of measuring a distance between two Gaussians, this has closed-form expression.
Theorem 4.1. The Wasserstein-2 distance Dw, between p(vo: ã) and p(vu) is given by:
Dir, ().
Σ (VT -VAs-Diii) +
1 (D2), 1)2
(15)
1=1
1=1
where ( denote the d eigenvalues of Eo-
The KI. divergence DKL. (PIIQ) assesses how much a model probability distribution Q differs from a reference probability distribution P. Note that this divergence not symmetric. with the Wasserstein-2 case, here as well we obtain a closed-fome expression for the two Gaussians considered.
5

---

Theorem 4.2. The divergence between the generated distribution p(vo; a) and the true distribution p(Vo) given by
1 2
d X xx ())()) [
DkL(o(volll((ooca)) log Dil.
Σ log A - -d+
(16)
For solving the resulting optimization problems, we have employed the Sequential Least SQuares Programming (SLSQP) method [17], a well-suited method for minimization problems with boundary conditions, and equality and inequality constraints.
Before turnnng into the empirical evaluation of the proposed optimization, and the implications the obtained noise scheduling on the various schemes, we pause to describe related work in this field.
5 Related work
Recent work has acknowledged the importance of the noise scheduling in diffusion models, and the need to shift the focus from custom-tailore heuristics [10 optimized alternatives. For instance, the authors of [26] introduced the KL-divergence Upper Bound (KLUB), which minimizes the mismatch between the continuous reversc-time SDE and its linearized approximation. However, this approach, along with others 34 32 37 29), aim to minimize the estimation error well, hence requires retraining denoiser entailing substantial computation time and resources when solving the optimization problem.
23
13
5 to the development of
While pursuing the same goal of optimizing the noise schedule, the work reported in 3 notable strides in simplifying the induced optimization problem. Such efficiency enabled in 38 by introducing upper bound on the truncation error and assuming the data-dependent component to be negligible during optimization. Yet, despite this improved scalability, a direct relationship between data characteristics and the derived noise schedule still remains vague.
38
36 made
potential approach bridge this gap spectral analysis, a fundamental tool signal processing. which frequently serves to associate design choices and properties of diffusion models with data characteristics 25 2 39 Z7. The application of frequency representation is diverse, ranging from introducing the coarse-to to-fie behavior in the reverse process [25] to uncover unique architectural fingerprints and identify biases in modeling different frequency ranges
39] However, to the best of our knowledge, no existing approach has connected spectral analysis with noise scheduling design.
From complementary perspective, several works have explored diffusion models by assuming a Gaussian target distribution [24] 31]. In (24), the authors leveraged a centered distribution and commutativity properties derive an exact solution for both the reverse SDE and ODE. Similarly, closed-fore solution to the probability flow ODE provided in 31 which also demonstrated the relation between learned neural scores and their single Gaussian score approximation. Although these analytical solutions provide valuable insights into diffusion dynamics, they are derived in the continuous domain via integration, potentially neglecting discretization effects and missing an explicit connection to the full traversal of the noise schedule.
6 Experiments
We turn to empirically validate the schedules obtained by solving the optimization problem, referred to as the spectral schedule or spectral recommendation. In addition, examine spectral phenomena arising from the diffusion process and their relation to the schedule structure.
6.1 Synthetic Gaussian distribution
In the first set of experiments, we assume Gaussian data distribution, Xu N( Eu), where Xo E Rdd and Eg is a circulant covariance matrix. Although circularity is not essential, it facilitates a direct examination of the frequency components, as Eg diagonalized by the DFT, with eigenvalues {a corresponding to its DFT coefficients 9 Specifically, the covariance is chosen to satisfy Eg= = T where a eirculant matrix whose first row a [-1, 1 1-1/(d-1), The mean vector, Mo. is chosen to be constant- value vector, following the stationarity assumption.

---

(a)
(b)
(c)
Figure 1: Figure 1a presents the optimized spectral schedules for Σg A A with d - 0.1,
and مr0 0.05 Ia, obtained by minimizing the Wasserstein-2 distance for various numbers of diffusion steps S E [10, 28, 38, 60, 90, 112, 250, 334]. Figure Tb compares the spectral schedule (dotted red) for S = 112 with various heuristic alternatives, including linear, EDM (p = 7) Cosine- based schedules such as Cosine (s 0,e 1,T 1) as in [23 5 and Sigmoid-based schedules like Sigmoid (s = -3,6 = 3,7 = 1) from MT Parametric estimations for the Cosine and Sigmoid schedules appear in brown and cyan, respectively. Figure Tearre//ccoomppeeee the Wasserstein-2 distance of the spectral recommendation with that of the baseline schedules across different step counts.
Finding the optimal noise-schedule scheme ā" depends on the target signal characteristics {Aa ]= the resolution and the number diffusion steps applied Figure Ia shows the resulting noise schedules for d = 50, I = 0.1 and μr 0.05-1d, obtained by minimizing the Wasserstein-2 distance for different diffusion steps Further examples involving different forms of Σg and Pl, as well as the use of the KL divergence, are provided in Appendix
At first glance, the optimization-basse solution produces noise schedule that aligns with the principles of diffusion models. Specifically, it exhibits monotonically decreasing behaviour, with linear drop in ā" in the middle of the process and minimal variation near the extremes. Although each schedule was independently optimized for specified number of diffusion steps, it can be observed that the overall structure remains the same.
Interestingly, solving the optimization problem in 14 while altering the initial conditions or removing the inequality constraints yields the same optimal solution. This suggests that these constraints passive, and that known characteristics of noise schedules, such as monotonicity, naturally arise from the problem formulation, as demonstrated in Appendix G.3
key aspect how the spectral schedule aligns with the existing heuristics. Figure Tb provides
comparison with the Cosine [23] the Sigmoid [[]], the linear [10] and the EDM (p 7) [B
schedules, along with a parametric approximation of the spectral recommendation. To achieve this, Cosine and Sigmoid functions were fitted to the optimal solution by minimizing the l2 loss, identifying the closest match. An interesting outcome from Figure iblis that the spectral recommendation provides partial retrospective justification for existing noise schedule heuristics, as the parametric estimation resembles Cosine and Sigmoid functions, when their parameters are properly tuned.
To validate the optimization procedure, Figure Ic compares the Wasserstein-2 distance of various noise schedules with that of the spectral recommendation across different diffusion steps. While the spectral schedule consistently achieves the lowest Wasserstein-2 distance, the optimization is most effective with fewer diffusion steps, where discretization errors are higher. As the number of steps increases, the gap between the different noise schedules narrows.
We also compare our derived schedules with those proposed in prior work. Specifically, the authors in (6 present closed-form solution for the optimal noise schedule in a simplified setring, assuming an initial isotropic Gaussian distribution with standard deviation C, i. Xy ~ (0, C²)). To enable fair comparison, we frame our optimization problem using the DKL. loss (4.2) and adopt the
Variance- Expinnci--xloiiigggggg (VE) framework, following the setup in (26). Appendix G.2 demonstrates the equivalence between the solutions and highlight their relation to commonly used heuristics.
Frequencies components and schedule structure: We turn to explore the relationship between the cigenvalue magnitudes and the structure of the derived schedule. To do so, we solved the optimization problem for each eigenvalue individually, with the contributions from the other eigenvalues set to
This follows the principles outlined in 19 ensuring fair comparison with other noise schedules.
7

---

zero. It can be observed from Figure that the solution becomes more concave as the magnitude of the eigenvalue decreases (yellow curve) and more convex as the magnitude increases (blue curve).
Notably, under the shift-invariance conditions, the eigenvalues directly correspond to the system's 's frequency components. In scenarios where this components follow monotonically decreasing distribution (e.g., the 1/f trend in speech [30]), the first cigenvalues correspond to the low frequencies, having larger amplitudes, while the last correspond high frequencies and smaller amplitudes. This pattern, along with the previous observation, aligns with the well-known coarse-to- fo-fine signal construction behavior of diffusion models. farther details are provided in Appendix K.1
While the optimization problem primarily focuses on the output signal Vo- each diffusion step V, can be also expressed
Relative error evolution:
function of the initial noise as described in 29 in
Appendix C This enables the analysis of spectral properties across all diffusion steps, rather than only at the output signal. Figure 2h presents the relative error of the 10 largest eigenvalues (sorted, largest on the right) over the final 20 steps of a 60-step diffusion process using the Cosine (0,1,1)
[23 5) schedule. While the error consistently decreases for high-amplitude eigenvalues, it increases near the final diffusion steps for those with smaller magnitudes. Additionally, the final relative error (yellow) tends to be smaller for large eigenvalues (i.e., low frequencies). This behavior aligns with the observed bias in mid-to-high frequencies reported in prior works (39] 77]. Further discussion, including an illustration of the eigenvalue and Wasserstein-2 error dynamics, appears in App. K.2
Mean drift: The explicit expres- sions in (T)) and (12) offer fur- ther insight into the diffusion pro- cess. A notable consideration is
whether this process introduces bias, i.e. drifting the mean com- ponent during synthesis. To study this, analyze the mean bias ex- pression for DDIM, (D2 D) derived from the difference be- tween E [vo and E [Vo]. In Ap- pendix L. we further explore the relationship between the target signal characteristics {A, ) the noise schedule a, and the expres- sion [D2 I|. It appears that dif- ferent choices of the noise sched- ule influence the bias value, with some choices effectively mitigat- ing it. Additionally, as the depth of the diffusion process increases, the bias value tends grow, regardless of the selected noise schedule.
CM
(a)
(b)
Figure 2: Figure 2a shows the spectral schedules obtained by solving the optimization problem individually for each eigen- value, with other contributions set to zero (note that the reverse process proceeds from right to left). Figure (bb illustrates the relative error of the 10 largest eigenvalues over the final 20 steps of 60-step diffusion process using the Cosine (0, 1, 1) schedule.
The explicit formulations of DDPM and DDIM in 10 and 12 enable compari- son of their losses across varying diffusion depths and noise schedules. In Appendix M we present such evaluation using the Wasserstein-2 distance. The results clearly show that DDIM sampling is
DDPM vs DDIM:
faster and yields lower loss values, aligning with the empirical observations in 27
6.2 Empirical Gaussian distribution
We now shift towards a more practical scenario in which we refer to real data, while still maintaining the Gaussian- ity assumption. To assess the generality of our approach, we consider different signal types. Specifically, we use CIFAR-10, and the MUSIC dataset [22), which comprises recordings of various musical instruments down-sample to 16kHz. We fit Gaussian distribution for each dataset by estimating their mean and covariance matrix, while for the MUSIC dataset, we extracted piano-only recordings. These datasets are referred to hereafter as the Gaussian MUSIC piano dataset and the Gaussian CIFAR- 10 dataset
66-
Smees 140
Figure 3: Comparison of the spectral schedules for CIFAR-10 and MUSIC Datasets with various heuristic noise schedules, using 112 diffusion steps.

---

(b)
(C)
Figure 4: Figures 4b and 4c show the Wasserstein-2 distance of the proposed spectral noise schedule (in red) compared to existing heuristics, evaluated across various diffusion step sizes, for CIFAR-10 and MUSIC. Figure 4a presents the corresponding FID scores for CIFAR-10 10. Across all comparisons, the spectral schedule generally outperforms the heuristies, with wider gap at lower step counts. For CIFAR- 10, the approximation error is less pronounced, with results showing greater stability.
24 The estimation procedure for temporal signals, including the role of the window size (d) and silence threshold (th), is detailed in Appendix I.4
Figure 3 presents the spectral recommendations obtained by minimizing the Wasserstein-2 distance for each estimated covariance matrix, alongside several heuristic noise schedules. While both spectral schedules retain
resemblance to the hand-crafted approaches, the one derived for the MUSIC dataset, introduces a somewhat different design of slower decay, adapting to the unique data properties. Consequently, adopting a spectral analysis perspective enables the design of noise schedules tailored to specific needs and data characteristics. The spectral recommendations for the SC09 333 dataset, with the Wasserstein-2 evaluations, provided in Appendix [
6.3 Empirical distribution
remains effective when We now aim to evaluate whether the optimized noise schedule from Sec. the Gaussianity assumption is removed and trained neural denoiser is employed within the diffusion process. Notably, this setting, approximation error arises due to suboptimality
6.2
the denoiser [24).
Our approach towards performance assessment of different noise schedules is the following: We run the diffusion process using a trained denoiser, and the DDIM inference procedure 4 to generate large corpus of samples. We then compare the statistics of the synthesized signals with those of real data by measuring the distance between their second moments, applying the Wasserstein-2 distance to the empirical covariance matrices. For CIFAR-10, we also use the standard FID score.
Figure performs such a comparison with various heuristic noise schedules. For each schedule and number of diffusion steps tested, 50,000 signals are synthesized using trained model. For CIFAR- 10, we used a pretrained denoiser from [3 which based on a continuous noise schedule. For the MUSIC dataset, we trained a model based on the architecture presented in [6 D. which employs linear noise schedule with T 1000 diffusion steps during training.
demonstrates that the spectral schedules consistently outperform most heuristic alternatives across both metrics, with the performance gap narrowing as the number of diffusion steps increases and discretization error diminishes. Notably, the Cosine (0,0.5,1) and EDM heuristics, whose structures closely resemble the spectral schedules derived for the CIFAR-10 and MUSIC datasets in Figure 3 also perform competitively (Figures 4a 4c These suggests that the proposed spectral recommendation effectively preserves the core data properties and may support the broader design of scheduling strategies. A more detailed comparison, with results for SCO9, appears in Appendix[]
7 Discussion
By recognizing how the eigenvalues shape the noise schedule, we can refine the loss term to better align with our needs. In Appendix K. we introduce weighted-11 loss, which promotes low frequencies while sacrificing high ones, ultimately yielding solution consistent with the well-known Cosine (0.1,1) heuristic. This relationship can be further explored and may help in mitigating

---

phenomena such as bias in certain frequency ranges 39 Another potential research direction is leveraging the evolution of the stepwise distribution for enabling accelerated sampling (29)
We analyze the optimization times of our method, For small step counts, where discretization error is more pronounced, the solutions are typically obtained within few seconds. However, for higher step counts or high-resolution signals, the computation time may vary. By prioritizing the most significant eigenvalue components during schedule design or initializing the optimization with solutions from smaller step counts, the runtime can be reduced to a few tens of seconds, as detailed in Appendix
Limitations: While our approach demonstrates the benefits of principled scheduler design and outperforms heuristic methods in FID across multiple datasets, natural next step to extend the framework settings closer to real data distributions, such Gaussian mixture models.
This paper presents spectral perspective on the inference process in diffusion models. Under the assumption of Gaussianity, we establish a direct link between the input white noise and the output signal. Our approach enables noise schedule design based on data characteristies, diffusion steps count, and sampling methods. Effective across synthetic and realistic settings, the optimized schedules resemble existing heuristics, offering insights on handcrafted design choices. By leveraging our method to explore various aspects of the reverse process, we hope this work encourages further exploration of diffusion models and their dynamics from a spectral perspective.
References
[1] Roi Benita, Michael Elad, and Joseph Keshet. DiffAR: Denoising diffusion autoregressive model for raw speech waveform generation. In The 12th International Conference on Learning Representations (ICIRR), 2024.
[2] Giulio Biroli, Tony Bonnaire, Valentin De Bortoli, and Mare Mézard. Dynamical regimes of diffusion models. Nature Communications, 15(1):9957, 2024.
[3] Defang Chen, Zhenyu Zhou, Can Wang, Chunhua Shen, and Siwei Lyu. On the trajectory regu- larity of ODE-based diffusion sampling. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024.
[4] Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In The 11th International Conference on Learning Representations (ICLR), 2023.
[5] Ting Chen. On the importance of noise scheduling for diffusion models. arXiv preprint vrXiv:2301.10977 2023.
[6] Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung- Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[7] Riccardo Corvi, Davide Cozzolino, Giovanni Poggi, Koki Nagano, and Luisa Verdoliva. Intrigu- ing properties of synthetic images: from generative adversarial networks to diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 973-982 2023.
[8] Jonathan Crabbé, Nicolas Huynh, Jan Pawel Stanczuk, and Mihaela van der Schaar. Time series diffusion in the frequency domain. In Proceeding of the 41st International Conference on Machine Learning (ICML), 2024.
[9] Philip J. Davis. Circulant Matrices. Wiley, New York, 1970.
[10] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[11] Allan Jabri, David J. Fleet, and Ting Chen. Scalable adaptive computation for iterative gen- eration. In Proceedings of the 40th International Conference on Machine Learning (ICMI., 2023.
10

---

[12] Alexia
Ke Li, Rémi Teiéleaii Tal Kachman, and Ioannis Mitliagkas. Gotta go fast when generating data with score-based models. arXiv preprint arXiv:2105. 140.00 2021.
[13] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 01226665-265777 2022.
[14] Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 35:23593-2368 2022.
[15] Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi. Fry 'recht audio dis- tance: A metric for evaluating music enhancement algorithms. arXiv preprint arXiv: 882.08466, 2018.
[16] Zhifeng Kong, Wei Ping. Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. DiffWave: versatile diffusion model for audio synthesis. In International Conference on Learning Representations (ICLR), 2021.
[17] Dieter Kraft. software package for sequential quadratic programming. Forschungshericht- Deutsche Forschungs- und Versuchsanstalt fur Luft- und Raumfahrt, 1988.
[18] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[19] Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common diffusion noise schedules and sample steps are flawed. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 5404-5411, 2024.
[20] Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. International Conference on Learning Representations (ICLR), 2022.
[21] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
[22] Luan Moura, Emanuel Fontelles, Vinicius Sampaio, and Mardónio França. Music dataset:
Lyrics and metadata from 1950 to 2019, 2020.
[23] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference machine learning (ICML), pages 8162-8171, 2021.
[24] Emile Pierret and Bruno Galerne. Diffusion models for gaussian distributions: Exact solutions and wasserstein errors. arXiv preprint arXiv:2405.142555 2024.
[25] Severi Rissanen, Markus Heinonen, and Arno Solin. Generative modelling with inverse heat dissipation. The 1ith International Conference on Learning Representations (ICLR), 2023.
[26] Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your steps: optimizing sampling schedules in diffusion models. In I Proceedings of the 41st International Conference on Machine Learning, 2024.
[27] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations (ICLR), 2021.
[28] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[29] Vinh Tong, Trung- Trung-Dung Hoang, Anji Liu, Guy Van den Broeck, and Mathias Niepert. Learning discretize denoising diffiusion odes. arXiv preprint 240::2405.1500 2024.
[30] Richard P Voss and John Clarke. "I/F Noise" in Music and Speech. Lawrence Berkeley National Laboratory, 1975.

---

[31] Binxu Wang and John J Vastola. The unreasonable effectiveness of gaussian score approximation for diffusion models and its applications. arXiv preprint arXiv:2412.09926. 2024.
[32] Yunke Wang, Xiyu Wang, Anh-Dung Dinh, Bo Du, and Charles Xu. Learning to schedule in diffusion probabilistic models. In Proceedings of the 29th ACM Conference on Knowledge Discovery and Data Mining (SIGKDD), pages 2478-2483 2488, 2023.
[33] Pete Warden. Speech commands: A dataset for vioibla-vocabulary speech recognition. arXiv preprint arXiv:1804.00 03209, 2018.
[34] Daniel Watson, William Chan, Jonathan Ho. and Mohammad Norouzi. Learning fast samplers for diffusion models by differentiating through sample quality. In International Conference on Learning Representations (ICLR), 2022.
[35] Norbert Wiener. Extrapolation, Interpolation, and Smoothing of Stationary Time Series. MIT Press, Cambridge, MA, 1949.
[36] Christopher Williams, Andrew Campbell, Amaud Doucet, and Saifuddin Syed. Score-optimal diffusion schedules. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[37] Mengfei Xia, Yujun Shen, Changsong Lei, Yu Zhou, Ran Yi, Deli Zhao, Wenping Wang, and Yong-jin Liu. Towards more accurate diffusion model acceleration with a timestep aligner. In Proceedings of the IEEE/CVF Conference Computer Vision and Paltern Recognition (CVPR), 2024.
[38] Shuchen Xue, Zhaoqiang Liu, Fei Chen, Shifeng Zhang, Tianyang Hu, Enze Xie, and Zhenguo Li. Accelerating diffusion sampling with optimized time steps. Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition, pages 8292-8301, 2024.
[39] Xingyi Yang, Daquan Zhou, Jiashi Feng. and Xinchao Wang. Diffusion probabilistic model made slim. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 12552-22566 2023.
[40] Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. In The Eleventh International Conference on Learning Representations, 2023.
[41] Wenliang Zhao, Lujia Bai, Yongming Rao. Jie Zhou, and Jiwen Lu. Unipe: A unified predictor- corrector framework for fast sampling of diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
[42] Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics. Advances Neural Information Processing Systems, 36:55502-5555 55542, 2023.
12

---

A The Optimal Denoiser for a Gaussian Input
This appendix provides the derivation and explanation of Thhoorem 3.1
Let Xy (w:: Σg) represent the distribution of the original dataset, where xo E Rd The probability density function fxx)) can be written as:
1 2 (80- 1o)" 20"" (xo Roo
f(xo) V (2T) Σnl
exp
Through the diffusion process, the signal undergoes noise contamination, leading to the following marginal expression for x,:
X Vaxxo v1 dgE (0,I)
(17)
For the Maximum A Posteriori (MAP) estimation, we seek to maximize the posterior distribution: max log pxxu) x,)
Using Bayes' rule, this can be written as:
min log (xx p(x+)
min ar
log p(, log p(xx)
(18)
JED
The conditional log-likelihoon log p(x, Xx) is given by:
1
exp {- 1 (x, vārxo) ((1 ((1-a))I)- (x, vāixo)
p(x((xx))
v (2x) E
1
1
log p(x) log (2nr)* EE
2(1 - 2(1) (x, vārxo) (xx vouxo)
The log-likelihood l(g)(xx) is given by:
p(xo) = 1
1 (xn "0)
exp
Ev (xo mo)
(2x) Evl
1
log log (2x)* EEp
(xo 10)) Σの¹ (xo 14)
2
We will differentiate the given expression in (18 with respect to zo and equate it zero:
dlogp(x;(xo)
2 (x, / (0xo))
dxo
2(1 - Ō..) -ūx))
dlogp(Xo)
2Σ, (xo M4)
dxo
2
2 2 (xi vaxXu)
2Σ
(Xn Mo) 0
2(1 a.,)
2
This simplifies to:
Vor (x+ - vārXo)
-.0 (Xo - 14) =0
(1)
Resulting in:
01 Enx; + Eo + (1 - ax) (1 -..44
Thus:
(a. + (1 a.)1 X Va,Eox + (1 (1-6))
6z)
Finally:
Xi E0 + (1 +1--a)1)) (Vā,Eoxi (1 1--ag -Gx)
(19)
13

---

B
The Reverse Process in the Time Domain
Here, we present the reverse process in the time domain for the DDIM 27 as outlined in Lemma
3.2
Let X follow the distribution:
X ~ (Ho, En), Xo E RRd
Using the procedure outline in 127]. the diffusion process begins with xs ~ (O,, 1), where X E Rd and progresses through iterative denoising process described follows
vI āg (x), + V a,-1 (n) Eg (x, (x., 50x2 (20)
x,-1(n) vāg-1
vd.
where
1 01
0, (n) mV
(21)
1 ag
Substituting the marginal property from 2
c0(x,, 8) = x, v0,
X, v1 a, Ep (X..8)
80
(22)
VI
vā,
x1-1(1)) 8 v1 ū,-1 (1)
X, v 0₂50
+ (n)
(23)
VI ag
For the deterministic scenario, choose 0 in and obtain (1
0) Therefore:
Xx v /,80
X (11 = 0) Vāg- 180 + a,-l
v1 ā
/1 a=
v va,v / d3-1
X5
vā
X0
(24)
VI 0s
V1 025
We denote the following:
Vo, V Og
0
b, = Va,-1
a
0
Therefore we get the following equation:
x=-1= + b,x
Using the result from the MAP estimator:
x = (á + (1 - - (va,Eoxt + +(1 -1-00 (1))))4))
we get:
X,-1 = 0,8, + h (a, Eo + a,,) (Vo + (1 - -ox)po)
X,-1 (a. + ba Eo +1
vá.En) (1-aa)))
x, + (b (a.Eo (1 -a-a))1)- (1 as)
(25)
Introduce the notation:
Σo,- = E. + -1- (1-
We can rewrite the equation as:
a + by 0. (Eo.s) -1 En) x, + by (1 - as) (Eo.,) 140
X,
(26)
We follow here the DDIM notations that replaces i with where the steps [l...., S] form subsequence of [L, ........ 2] and 5 = T'.
14

---

C Migrating to the Spectral Domain
Here, demonstrate the application of projecting both sides of Eq. (8) onto the eigenbasis of Eo as outlined in Lemma 3.4] Since Eo is symmetric, its eigenvalue decomposition can written as Eo U where the columns of U represent the eigenvectors, and Ao is the diagonal matrix containing the corresponding eigenvalues f
x U [adI+ b, a 2.
+ b,(1 a. az)
(27)
U X a, a x, + U Va 0.5
UU UU EQU - U (0l1-a,)pp 20) _U
po
By introducing the notations v U x and μi = UT Hop we obtain:
V=-1= - a.V. + b, 0. Uv, + bb a.) a.)
By applying eigenvalue decomposition, the symmetric matrix Eo can be diagonalized using its eigenbasis U, where U U Specifically:
Eo = UA Ao = U
a < + bI uT + -UIUT U + U U
2 UAJ'UT, =
Therefore, we obtain:
Σ = a.E0 a1- U + (1-a,))nm T -1
Σ A (1-ā,))
a. a U
Incorporating these elements into the main equation, we obtain:
V3-1 0.V. bova. + (1- Aov. + +b(11 a.) (ā, + +(1 (1-a,)1 m
Va-1 a.II+bbvar + Ao] v. + bb) + -1-60 (1-)))1]- 14
We will denote the following:
G(s) ag + bava, (a (1 a.)II Au
M(s) = b. a
a.) aā,A0 + (1- 11ā,,)) ā.)
and get:
v.- = V-)v= + M(s)
(28)
We then recursively obtain v, for general
V
II G(S') 8'=44
Vs
Σ i=++1
II f=1+1
GD)
M(i)
(29)
'In case the covariance matrix En is circulant, it can be diagonalized using the Discrete Fourier Transform (DFT) denoted by F(.). In this case, the projection exposes the frequency components, enabling their explicit analysis.
15

---

specifically for 1 = 0:
S
-
No
II
G(s')
vs + +
Σ
II
G()
M(ū)
(30)
===1
==1
j=1
We will denote the following:
II
G(+)
II
as + bs v as (āsAo + I(1 a,))=² Auj
(31)
D2
II G(j)
M(i)
(32)
i=1
j=1
S
by 0z A0 + +I1 az)]= - 20]
aj
bi1 a.) (a,Ao + +I(1 -a=)
Substitute D₁ and D2 into the last equation, we get:
vo Divs + D2já
(33)
The resulting vector from Equation 33 is linear combination of Gaussian signals, therefore it
also follows Gaussian distribution. We now aim determine the mean and covariance of that distribution.
Vu ~ NEE Eg.).
vo E Rd
Mean:
E [ = E [Divs + D2 D D, + E [D>uö] Daj Vs ~ 10, E [ol = D240
Covariance:
Σv, = (Divs + D2j -E + D2 (Divs + D22p [DVs + D2
E (Divs + D2HD" D2 (Divs + +224 D2H4)7
(Divs) (Divs)* = D₁E vs (vs)) D,"
Ex, = D,D, D
vo ND Di),
vo E Rd.
(34)
The same procedure used to derive the distribution of vo in (34) from (30) can similarly be applied to any intermediate step vi in 22), enabling exploration of the spectral dynamics throughout the inference process.
For the data distribution xn NN En), where Xo Rd its first and second moments in the frequency domain are given by:
E Vol = U [xo] = D
Σvo = Ao
Vo ~ Nu Ao)
(35)
16

---

D Evaluating loss functions expressions
Here, we present selected loss functions based on the derivations provided in Section
D.1 Wasserstein-2 Distance:
The Wasserstein-22 2 distance between two Gaussian distributions with means μl and fl,, and covariance matrices Σ1 and Σg, and the corresponding eigenvalues {A pd ) and {12 (d)
Ji=1 is given by:
W2( / (1)) ( (M) 122 + Σ
(36)
Since
Vo ~ ~N(22 Dj), ,)),
vo E Rd and Vo NHHH Ao), E Rd
we obtain:
(u) -22)² (M1 (22 (D2Hi) =D)" (Dem² vら) ((H)) - (Hb))) (D2 =i)
(M 3 - (( ( + (. D2140
Σ
(n 2 2 >
(p)):
? D2 + Σ (K)) (D2 ()
= Σ
D2 (1) -1)² 2 (1/))
1=1
D2 (0) -1) 2
(11-1²) (11 (12)) Σ
i=1
Σ (V110 v \1
(1) Σ
v (160)
| (D, (D,
W2(vo, vo)
Σ (ss (0)0) -1) 2
(un))²
Σ
(VAS - (D,,
(37)
D.2 Kullback-Leibler divergence:
The Kullback-Leibler (KI.) divergence between two Gaussian distributions with means μn and #2, and covariance matrices Σ₁ and Σg, and the corresponding eigenvalues (2 limi and (1) lim1 is given by:
1
Σg Σ
DRL (W(,, E1) [ Ex))
log
d + tr (Σ,'E) + (Mz "n)=En ' 11))
2
Given:
No ~ N( D2),
vo e Rd
and vo ~ Ap), vo E Rdd
The KL divergence is given by:
17

---

DKL (vo || vo) DeL (N (V Evol N (E [vol: Ev,))
DEL (W (Hb. Ao) , (
By decomposing the KL divergence elements, obtain the following terms:
D (
ΣS [ D
Di
Σ1 I X
tr (Eg'En) = Нラ
D.002
(12 11) Er (142 14)) (D2jg) 18) (D,3)~((2211) 14))
(D2
D2 (1) -
(1g)²² (D 1) (Di*) ( (D2 1)
(1001)
D (i)
i=1
Applying the substitution, the term results in:
DEL (N (Ho: Ao), N (D2μ Dj D₂
D2 (8)
1
Σ
log D (
log 11
d
2
(1)
1=1
-1) 2
+
D2 (6)
DKL. (Vo || vo)
1
Σ
2 log D (3)
log A
(i)
d d+
(38)
2
D (1)
1=1
i=1
E DDPM Formulation:
Here, apply equivalent procedure to the DDPM scenario, did for the DDIM, outlined in Theorem 3.6
E.1
The Reverse Process in the Time Domain
Using the procedure outline in [00, the diffusion process begins with Xy ~ (O0, where xy e Rd. and progresses through iterative denoising process described follows:
X
Xi
ai ai
(x(xx,t)
+ oiZ; 2, N(0,1)
(39)
Var
VI
Where o,
1- (1 a.).
Given the marginal property from
X, - Va
co
vI ai
18

---

we can incorporate into
39
1
04
X₁ - v |x200
X
X1
+ +
V
04
vI
a₁
1 axe
1
1 ai
(1 -1a))) vāu xx0
X
1
+ 0.2t
Vat
1 at
āt
1
at
at)
Xt-1
Xx +
Xo + 7z
(40)
Vat
1
0z
O2
We denote the following, where the final term in each equation is represented by â₂ and d=-1;
Var
1
V
Va
at
v
āt ar)
b
Vāc-l
0t 1 - at
1 Ctt
- 0/-- (1 ar)
ah-1 1
1 āt
G 0z
āt
C't
Últ-1
Therefore we get the following equation:
X;-1 axx; + b: + ct
Using the result for the MAP estimator from (
x = Eo + - (1)) (Vo,Eox, + =11 -1- -x))))))
-
we get:
Xt-1 = =0x; b (ā,Eo + a.) (
(Va,Loxt + (1 - -x)) + c222
X (at + by Co (1 a.) VarLo) x by (a, + (1 -,)1)= (1 -a)mm)) ta,z
Using the notation from Appendix B
Eo.- = Eo + (1 =;)]
Thus, we can rewrite the equation as:
x; (at biva, (Eo.)) -1 Eo x; + be(1 ae) (Eo.)) -1 po c.2
(41)
E.2 Migrating the Spectral Domain
Next, we project both sides of the Eq. 44 onto the spectral domain. As previously described in Appendix this step relies on the eigenvalue decomposition of the covariance matrix En UA U where the columns of U represent the eigenvectors, and Ao is the diagonal matrix containing the corresponding eigenvalues.
U UT ((a++ bevor (Eo.)) Σn) x, + be(1 - (i)) (Eo.)) -1
c.2i
(42)
U x - -.0 (Eo.)) UUT UU x ST+UT, (Ubb(11ā))) (Eo..) UUT HotaU
By introducing the notations v U x and μull
UT Moo obtain:
19

---

=-1 = + b, U (Eo.) UU EvU, + bi(1 - at) ā.) (Eo.) Upu + czz
By applying eigenvalue decomposition, the symmetric matrix Eg can be diagonalized using its eigenbasis U, where U UT Therefore, we obtain:
[ú, 20 + (1 0.)1
ā. UAOU + (1 a.)
+ (1-
0,,
E A + (1- ā. a U
Incorporating these elements into the main equation, we obtain:
Vz-1 = V, - V Ao + [1-āa)]] Aov, + b1 a.) lā,Ao + (1 (1-111111) -1)) 14 + azz
Vz-1 = axI + b. lār + (1-a,)1] Aoj vz + (1 (ārAo +11-))))) -a) pb + c.
Including those elements in the main equation:
Vt-1 V. b Ao + (1 ==a) Aov, bi(1 āt) (arAo + -1-111))1) (1-a,)1]- 140 a.z
Vi-1 | b, ha + +11 (1-111111)) -,), Ao] v.
+ (1 a,)II=' pD c,z,
We will denote the following:
G(t)
a: + (1- Au]
M(t) b. āz) a., (1 (1- -1))
and get:
Vt-1 G()V +M00, + taze"
We can then recursively obtain vi for a general l:
T
i-1
[-]
V1
II
G()
v
+
Σ
I
GÚ)
M(i)
ub +
Σ
II GO)
c.z
111+1
i=++1
j=l+1
i=/+1
j=1+1
where the process iterates over all the steps:
[I....,
T
T
i-1
v.
c
ve+
Σ
II
G()
+
Σ
II G(j)
M())
A=
==1
We will denote the following:
8
D
Il
G(s)
$ D2
Σ
[[G M(1)
Substitute D and D2 into the last equation, we get:
20

---

Vo- Dive +
Σ
n
G()
czz
D2f
(43)
The resulting vector from Equation 43 is linear combination of Gaussian signals, therefore it also follows Gaussian distribution. We now aim determine the mean vector and the covariance matrix of that distribution.
Vo ~ (V Eva). Vo E R XT ~ (O,1)) and 2, (0,,1) Vi
Mean:
T
E [vo] E
D +
Σ
II G() GI
D2fm
D2f
[o] D2pi
Covariance:
T
T
1-1
11
Evo
= +D2 +
+ D2
Σ
G()
cz," -E E
Div. + Domi +
Σ
G()
cz.t
Dive D2pu +
Σ
II
G
G2,
-E
E
Div. D2p
Σ
II
GV)
az )
)7
T
i-1
T
Evg = E
Dive +
Σ
GIG
Σ
II
GO
Lemma E.1. Let X2, Nn be independent Gaussian random vectors with mean E[x,] and covariance matrices Cov(x,) E, for 1,2,..., n. Let the linear combination defined as:
X2,........
O:X
1=1
where d1; O2,...,,, On constants. The covariance matrix of denoted Cov(y), is given by:
Cov(y)
Σ
Applying the result of Lemma E. to the expression in Equation 44 where XT ~N and N( I) are independent Gaussian noises for all i, we have:
[[vv.'" = U
X = U xT
U=
z
2
uf
=I
Thus, the covariance is given by:
T
H
Σ. = =E [VVVo
= =D,,
+
|| G(j)
21

---

T
i-1
Evo = -Di
Σ
G()
el
1=1
T +
No ~ N Dzuö
Dj
Σ II ==1
(44)
As discussed in Appendix C for data distribution X second moments in the frequency domain are given by:
MMoo En), where xo E R the first and
E Vol = FIE [xo] = 140
Σxg Ao
Vo ~ ~(( Au)
(45)
F Variance preserving and Variance exploding theoretical analysis
The paper 28 distinguishes between two sampling methods: Variance Preserving (VP) and Variance Exploding (VE). The primary difference lies in how variance evolves during the process. while VP maintains a fixed variance, VE results in an exploding variance as f - T. Here, we focus on comparing these approaches within the context of our spectral noise schedule derivation for the DDIM procedure [10]. Throughout this paper, we described our methods based on the Variance Preserving (VP) formulation, given by:
px, xo) =N~ (Varxo, V1 a
(46)
where the only hyperparameters are the noise schedule parameters: {ā,) where ā, E (0, 1].
In contrast, under the Variance Exploding (VE) method, the hyperparameters are given by o where ae [0, oc), and the marginal distribution takes the form:
[xxx N ~ (Ru, II
(47)
We used the notation X, to distinguish from x,, except in the special case where Xo = No. Applying the reparameterization trick, we obtain:
VP: X = VarXo + VI - are c0,,
(48)
VE: = + 0,E (~,
(49)
A key relationship between the VP and VE formulations, as derived in 14 is given by:
X
X
ā (1+
(50)
VI+0 0
F.1 Determining the Optimal Denoiser:
Following the derivation in A we obtained the expression for the optimal denoiser in the Gaussian case under the Variance Preserving (VP) formulation:
&
Wi
(o + [(1 ((1-at)) (Vo,Eox; + .z.) (4)
Leveraging similar approach, we derive the corresponding expression for the Variance Exploding (VE) scenario:
22

---

V a (: o(t) Wiener,VE (Eo Tož) (Eox; + ozMo).
(51)
F.2 Evaluating the Inference Process in the time Domain:
This part can be performed using two equivalent methods: Method 1:
The ODE for the VE scenario in DDIM, as outlined in 27 is given by:
dă 1 5o(() (o)dt g(t) - (t
(52)
di
Additionally, the score expression and the marginal equation are also derived
27 as follows:
(53)
F2 To + oce where c~ (0,,1
(54)
Substituting Equation 53 into Equation 52
1 (1) (t)
6 dt
dx 2 dt (t))
dr
do (t) (n) dt dt c
(55)
Substituting Equation 54 into Equation
55 we obtain:
50
σ. Ft-1 =0,, (0,-0a (0--0(-1)
0t.
Ot-1 -1
+t+ Ft-1
Ft 80
0:
0t-1 It
(1 01-1
F-
+
50
(56)
Method 2:
given the inference process in the VP formulation
27
a' X,- (1) = 0) V āg
O1a /
ag-1
X,
Vo+-1-
20
vI
ā2
By leveraging the comnections in Equation 50 we can derive the following relationship between the two successive steps, X,-1 and xs, the inference process:
2
R- =
Fx +
F6
(57)
o²
o?
The Resulted expressions in (57 and 56 are identical.
By defining the following terms:
0x a
We follow here the DDIM notations that replaces i with where the steps [1,... S] form subsequence of (L....,3 2] and S = T'.
23

---

by 8-11 1 0s
0,
we can express the relationship between x,-1 and X as:
X 0g + 6g80
(58)
F.3 Evaluating the Inference Process in the Spectral Domain
Since similar expression to Equation 58 has already been discussed in Appendix C describe the inference process in the spectral domain follows:
we can now
v G())v, + M())
(59)
where:
G(s) 0. + b, Ao lo
1o,1- Ao]
M(s) b [A Io d3
Following this and alignment with the same methodology described in Appendix
we obtain:
vo = Div. + +D2ui
(60)
v ~ N(E (vo). Dío),
vo E Rd
E Vol
D2H0,
Ev. D,D, T= = =D,
s
Dj = II
(s)
II
a, + b v ā. aa + I(1 - a.)l Au
(61)
1=1
D2
II G(j) M(i) j=1
(62)
aj by 0. 1o,A0 + +I(1 - a,)l Ao]
di (ā, I(1 -a))
(63)
24

---

G Clarifications and Validations:
G.1 Method Evaluations: Temporal and Spectral Domains
We evaluated the compatibility between the diffusion process in the time domain, using the DDIM method [TO), and its counterpart derived from Equation in the spectral domain. Using an artificial covariance matrix, Eg. with parameters / = 0.1 and d 50 from 6.1 we estimated the covariance of 50, 000 signals that were denoised according Equation 6 and computed their eigenvalues, denoted as {X.*he
Equation using the optimal denoiser from
ld 1im1- In the spectral domain, we
applied the formulation from Equation 10 for deriving D1 and extracted (xppectra) Fi=1 from its diagonal elements. The results are illustrated in Figure 5
Spectral damam
Orggin ral Catteet
Spectra donaii indencce
0.6
3.5
0.5
Egernats index-1
Ogenwabu 1150-1
(a) A comparison of the cigenvalues obtained from each method
(b) The absolute relative error between the esti- mated eigenvalues and the data eigenvalues.
Figure 5: Figure compares the eigenvalues derived from the spectral and time domain formulations of the DDIM method [10]. The dataset, described in 6. with 7 0.1 and d = 50, is used for both approaches, involving 112 diffusion steps and following the linear noise schedule proposed in [27].
Furthermore, FFigure 5h Shilstusrsss the absolute error between the estimated and original eigenvalues for both methods.
Figure 5a shows that the derived eigenvalues from both procedures align with each other, thus verifying the transition from the time to spectral domain. However, they are not necessarily identical to the properties of the original dataset. Notably, the number of steps increases, both processes converge toward the original dataset values. Figure 55 allows for an examination of the absolute error in each process relative to the characteristics of the original dataset. It is evident that while the spectral equation exhibits stable behavior, the time-domain equation displays fluctuations that depend on the number of sampled signals. As the number of samples increases, these fluctuations diminish.
25

---

Figure 6: Comparison between the closed-form solution from AYS [26] and our numerical solution for the simplified case where Xp (O, C2T) with C [0.1, 0.5, 1].
G.2 Method Evaluation: Comparison with Prior Works
We also compare our optimal solution with those from previous works. Specifically, The authors in (26) derive closed-form expression for the optimal noise schedule under a simplified case, where the initial distribution an isotropic Gaussian with a standard deviation of C, i.e Xy ~ (0, C frame our optimization problem using the Kullback-Leibler divergence DKL loss (42) as done in [26].
To enable proper comparison,
Figure 6 compares our optimal solution, obtained by numerically solving Equation (1). with the closed-form solution from [26] can be observed that both methods align for arbitrary values of C. Notably, for C = 1, both noise schedules converge exactly to the Cosine (0, 1, 1) noise schedule, which was originally chosen heuristically 23
G.3 Constraint Omission and Different Types of Initializations
We explore the influence of the initializations and the inequality constraints in the optimization problem. Figures 7 and illustrate the evolution of the noise schedule parameter, {āz [==)==0- during the optimization process for two different initializations: uniformly random and linearly decreasing schedules, respectively. In both cases, the diffusion process consists of 28 steps, and the Wasserstein-2 distance used. Each scenario was conducted twice: with the inequality constraints from 4 and once without. The results are plotted at 15 evenly spaced intervals throughout the process to avoid presenting each individual optimization step.
1..
66
0.2
02
stop
(a) Optimization with incquality constraints.
(b) Optimization without incquality constraints.
Figure 7: A Comparison of the noise schedule parameters, { during the optimization process. The optimization conducted over 28 diffusion steps, with uniformly random initialization. Figure 7a shows the results with inequality constraints, and Figure 7 presents those without.
'Since the variance-exploding (VE) formulation of the diffusion process employed in 26 used the corresponding relationship aa 1+,,z transition the resulting noise schedule to the variance- (VP) formulation, derived in Appendix
26

---

02
02
Otaiinattg stop
(a) Optimization with inequality constraints.
(b) Optimization without inequality constraints.
Figure 8: A Comparison of the noise schedule parameters, {ā, [x= during the optimization process. The optimization conducted over 28 diffusion steps, with a Linearly decreasing initialization. Figure Sa shows the results with inequality constraints, and Figure 8b presents those without.
The results reveal that the optimized schedule is consistent across both initializations and independent of the inequality constraints. This suggests that known characteristics of noise schedules, such as monotonicity, naturally emerge from the problem's formulation itself, even without an explicit demand for inequality constraints. Similar consistency observed for other initializations, including linear and cosine schedules, demonstrating the stability of the optimization procedure.
27

---

H Supplementary Experiments for Synthetic Gaussian Distribution:
In the following sections, we demonstrate the received spectral recommendations for various al- ternative selections, applied to the matrix Σn and the vector Po- differing from those presented in Section 6.1 Additionally, we present the solutions obtained for defining the Wasserstein-2 2 distance and the KL-divergence. Through this, we aim to provide a broader perspective on the behavior and applicability of the proposed approach.
Figure visualizes the covariance matrices En and Ag as discussed in Sec. 6.1
200
32
Co
(b) (Ao))
Figure 9: Visualization of Eo and the trace of Ao for d = 50 and / 0.1. The covariance matrix Eo circulant Pah, while Ag is diagonal with symmetric diagonal elements (9)).
H.1
W distance
Figure 10
presents the resulting noise schedule based on the Wasserstein-2 distance.
30
dains
ant Maae
02-
02
a
(b)
Figure 10: Figures 10a and 10b compare the optimized noise schedules from Sec. 6.1 using d = 50,
1= 0.1, and #4 = 0.05, with heuristic schedules 112 diffusion steps, where the optimization
based on the Wasserstein-2 distance. Figure 10a examines the spectral schedule alongside the Linear
[0], EDM [3] and Cosine-basee schedules, including Cosine (s = 1, = 1) from (3] 5)
Likewise, Figure 10b compares it to Sigmoid-basee based schedules [11] The parametric estimations for the Cosine and Sigmoid functions are highlighted in red.
28

---

H.2 KL-Divergence
Figure presents the resulting noise schedule based on the KL-Divergence.
and.ed Cranto and
Ceccral Someralee
24
24
(a)
(b)
Figure 11: Figures 1aa and 11b compare the optimized noise schedules from Sec. 6.1 using d 50,
I = 0.1, and M4 = 0.05, with heuristic schedules for 112 diffusion steps, where the optimization
is based the KL divergence. |Figure Ila examines the spectral schedule alongside the Linear
10.. EDM [3] and Cosine-based schedules, including Cosine (8 = 0, = 1, T = 1) from [23 5
Likewise, Figure 10b) compares it Sigmoid-based schedules 11 The parametric estimations for
the Cosine and Sigmoid functions are highlighted in red.
Notably, under the same conditions, the KL divergence results in a more concave spectral recommen- dation compared to the Wasserstein-2 distance.
29

---

H.3 Variations in Covariance Matrices and Mean Configurations
In we designed specific covariance matrix Eg and mean vector Ho with the intention of resembling characteristics observed in real signals, such as a centered signal with Mo z 0. However, the optimization process not restricted to these particular choices and can be generalized to accommodate various alternative decisions. Figure [2 displays different covariance matrices along with their corresponding do vectors, followed by the resulting spectral schedules computed using the Wasserstein-2 distance for 60 diffusion steps.
32
39
38
(b)
(c)
04
23
(g)
(h)
ii)
(i)
(k)
(I
Figure 12: Various covariance matrices (first column) along with their eigenvalues (second column) and the corresponding optimized noise schedules (third column). The first row presents the matrix from 6. with / 0.05, b4 = 0.01 1g, The second row shows the same matrix scaled by a factor of 20 while keeping the same pl-- The third row displays covariance matrix incorporating Cosine function in the first row a of the circulant covariance, with 14, = 0.3 1a. The fourth row features a circulant matrix derived from sinusoidal signal with frequency of 1000 in the first row of the circulant covariance matrix, scaled by 0.01, and with oo = 0.1 1
30

---

While we cannot cover all possible choices for the covariance matrix Σn and the vector Mo. we aim to provide broader perspective on the KL-divergence loss. Figure illustrates a circulant covariance matrix whose first row is derived from sinusoidal signal with a frequency of 1000 Hz, along with the corresponding spectral recommendation based on Kl--divegeence, where Po 0.1 1
(a) Eg
(b) tr(oo)
(c)
(d)
Figure 13: Figure 13a shows the circulant covariance matrix, Eg. whose first row is derived from sinusoidal signal with frequency of 1000 Hz. Figure disjissysssss the trace of the corresponding Ao matrix. Figures [3e]and] [3d compare the spectral recommendations for d = 50, 112 diffusion steps, using the KL-divergence, with various noise schedule heuristics including Cosine and Sigmoid, respectively. The parametric estimations for the Cosine and Sigmoid are highlighted in red.
The results above show that modifying the dataset properties, such the covariance matrix and expectation, along with altering the loss function, leads to noise schedules with similar overall structure but varying details. In Appendix[K] we explore the connection between the dataset properties, the loss function, and the resulting noise schedules.
31

---

I Supplementary Experiments for Empirical Gaussian Distribution
We present additional details the CIFAR-IO, Gaussian MUSIC piano and SC09 datasets, along with the spectral noise schedules derived from them 22 33
L.1 CIFAR-10 Dataset
The estimated covariance matrix for CIFAR-10 has dimension of d = 3072, corresponding to the original image resolution of 32 x 32 with three color channels. Figure 14a presents the spectral schedule derived for this covariance matrix using 112-step diffusion process. Notably, its structure resembles that of the EDM schedule. Figure 14b illustrates the Wasserstein-2 distance evaluated on the Gaussian distribution associated with CIFAR-IA 10, demonstrating the optimality of the spectral schedule across all diffusion steps. This effect is especially pronounced at lower step counts, where discretization error is more significant. Interestingly, heuristic schedules whose structures closely align with the spectral schedule, such as EDM, also achieve comparable performance, further supporting the connection between schedule structure and diffusion quality.
1001 pi
met
un
D Pution Sep
Tral Defucio deps
(a)
(b)
Figure 14: Figure 14a compares the spectral schedule with several heuristic noise schedules for 112-step diffusion process. The best-fitting parametric approximations for the Cosine and Sigmoid schedules are shown in red and brown, respectively. Figure 14b [4b] demonstrates the optimality of the spectral recommendation in the Gaussian case, revealing higher effectiveness in diffusion processes with fewer steps, where discretization error is more pronounced.
I.2 MUSIC Dataset
To estimate the covariance matrix for the MUSIC dataset, apply sliding window of length = 400 (0.025 seconds) to the original audio, excluding segments with L1 energy below a threshold of th 0.05 to reduce the influence of silent regions. The resulting covariance matrix is symmetric and exhibits an approximately Toeplitz structure.
Figure 15a presents the spectral schedule derived from the estimated covariance matrix using 112-step diffusion process, while Figure I shows the Wasserstein-22 distance comparison for MUSIC Piano Gaussian dataset. Similar conclusions regarding optimality and the alignment with heuristic schedules hold here as well, with the Cosine (0, 0.5, 1) schedule being the closest both structure and performance to the spectral recommendation.
32

---

FOR
Conaen wid ait
Wiuuemtte 1
Queet Colle
Setoed CRLTEET
unc mnc
Figure 15: Figure 15a compares the spectral schedule with several heuristic noise schedules for 1 112-step diffusion process. The best-fitting parametric approximations for the Cosine and Sigmoid schedules are shown in red and brown, respectively. Figure [5b demonstrates the optimality of the spectral recommendation in the Gaussian case, revealing higher effectiveness in diffusion processes with fewer steps, where discretization error is more pronounced.
33

---

1.3 SC09 Dataset
In this section, we apply our method to a different dataset, SC09, SC09 is a subset of the Speech Commands Dataser (33 and consists of spoken digits (0-9). Each audio sample has duration of one second and is recorded at a sampling rate of 16 kHz.
Differing from Sec. 6.2 here we use segments of length d = 16000 samples (one second) and set th = 0.05 in one setting and 0.1 in another. Figure [6 presents the spectral recommendations for th 0.05 and th = 0.1 in the left and right columns, respectively.
(b)
and
DI Seg
(d)
e-1d2 and 1432
Lall
554
(c)
(f)
Figure 16: Spectral noise schedules for the SC09 dataset with d = 16, 000 and thresholds th 0.05 (left column) and Uh = 0.1 (right column). The first row shows spectral recommendations for various diffusion steps, while the second and third rows compare the spectral recommendation for 112 steps with heuristic noise schedules, including the Cosine and Sigmoid schedules. The parametric estimations for the Cosine and Sigmoid functions are shown in red, respectively.
Figures and 16b demonstrate that the spectral recommendation for th 0.05 exhibits a more concave behavior compared to th = 0.1. A more detailed discussion on the influence of the th and d parameters on the covariance matrix and the resulting noise schedule is provided in Appendices I.4
K
We also evaluate the Wasserstein-22 -2 distance for the spectral recommendation on the SC09 Gaussian dataset, using silence threshold of th = 0.1. As shown in Figure 17 similar conclusions hold: the spectral schedule remains optimal, with Cosine (0, 1, 1) and EDM heuristics most closely matching its structure and performance.
34

---

FOMEE
Sigmaid pnd und laumd)
400
stops
Figure 17: Wasserstein-2 distance comparison across different numbers of diffusion steps and various heuristic noise schedules for the SC09 Gaussian dataset. The results highlight the optimality of the spectral recommendation, showing higher effectiveness in diffusion processes with fewer steps. where discretization error is more pronounced.
35

---

1.4 Analysis of Different Aspects
The estimation of the covariance matrix, which is essential for finding the spectral recommendation for real dataset, relies on the choice two key parameters: th and d.
The parameter represents the dimension of the signals and controls the frequency resolution, which affects the eigenvalues. A smaller d may result in more generalized eigenvalue spectrum, reducing accuracy by averaging energy across neighboring eigenvalues. In contrast, a larger d improves the precision in capturing frequency details but increases computational time for both estimation and optimization.
Figure 18 shows that dl increases, the eigenvalue structure becomes more precise, with the maximum eigenvalues magnitude growing larger. Conversely, as d decreases, the eigenvalue structure becomes more generalized, exhibiting monotonic decrease, discussed in Appendix K.
412
1/
(a) d 400
1.2
1.00
(b) 4000
20:00
10:.0
1900
19000
(c) 16000
Figure 18: Eigenvalues of the MUSIC dataset using d [400, 4000, 16000]
An additional consideration is the choice of the threshold value th. This threshold helps prevent the covariance matrix estimation from being overly influenced by silent regions in the signal, which are characterized by low L1 energy. Adjusting th affects both the covariance matrix values and the eigenvalues, i.e. C Au C Xu, thereby influencing the resulting noise schedule, as shown in Appendix K.1
Supplementary Experiments for Empirical Distribution
In the following section, present more comprehensive analysis of the results for the general empirical distribution, where the Gaussianity assumption no longer holds and a neural denoiser is required.
begin by providing technical details of the denoiser architectures used for each dataset. For CIFAR-10, 10, we employed pretrained denoiser from [3]. trained with continuous noise schedule. This approach facilitates a better comparison across noise schedules by reducing approximation error and ensuring alignment between training and testing conditions.
36

---

For the MUSIC dataset, we trained a model based on the architecture proposed in [16] 11 using linear noise schedule with T" 1000 diffusion steps. Training was performed in an unconditional setting raw waveforms with batch size of 64. For the SC09 dataset, we adopted the architecture from GitHub) and similarly trained it with linear schedule of 7 1000 steps.
In the both cases, during synthesis, we mapped the tested noise schedule to the closest matching step in the linear schedule enable use of the trained denoiser. While this approach is suboptimal compared to training with a continuous noise schedule, introducing additional approximation error, it reflects a practical scenario where specific denoiser is given, and therefore worthwhile examine.
We assess the performance of the spectral recommendation on the MUSIC and SC09 datasets using second-orde -odder metric. In particular, we compute the Wasserstein-22 distance between the covariance matrices estimated from 20, 000 generated samples at each diffusion step and those estimated from evaluate how well each noise schedule preserves the core properties
the real data. This allows of the distribution during synthesis (Figure 20)
Additionally, for CIFAR-10, 10, are preserved with our recommendation (see Figures 19). The FID score calculated following the method outlined in [13), using 50, 000 samples for each computation, with seeds ranging from 0 to 49, 999. As for FAD [5] calculation, it is less reliable for the MUSIC dataset due to its short frame lengths, and it provides limited insight for SCO9, given its nature speech dataset.
the standard FID metric to assess whether perceptual properties
350
(a)
Figure 19: Wasserstein-2 distance (Figure 19ah and FID (Figure 19b results on the CIFAR- dataset.
GEEP CONTRE and
BHATHAA Special ccledderr
Sencier
Samar
(a)
(b)
Figure 20: Wasserstein-2 distance the MUSIC-piano dataset (Figure 20ap and SC09 dataset (Figure 20bb. evaluated over diffusion steps: {10, 20, 28, 38, 60, 90, 112, 250}.
The results support the relevance of the spectral recommendation for a broader range of distributions. The improved stability observed on CIFAR-10 10 may be attributed to factors such as the model architecture, the training procedure, differences between discrete and continuous noise schedules. Interestingly, heuristics that achieve similar performance, such EDM for FID on the CIFAR- 10 dataset and cosine (0, 0.5, 1) schedules for Wasserstein distance on the MUSIC dataset, tend to share structural similarities with the spectral recommendation. This suggests a connection between the structure of the noise schedule and overall performance, and points to potential of the proposed method to inform the design of effective noise schedules.
37

---

CLUBM SHIES
Wasserstein-2 distance (Figure 21a and FID results (Figure 21b the AFHQv2 dataset, evaluated over diffusion steps: {10, 20, 30, 40, 50, 70, 90, 112].
Figure 21:
We conducted an additional experiment on the higher-resolution AFHQv2 dataset [6], with covari- ance matrix of size 288 12, 288 (corresponding 64 64 x 3 images). Figure 2 presents Wasserstein distance and FID scores for the spectral recommendation and heuristic baselines, demon- strating the superior performance of the spectral schedule. In this case, EDM with p 5 most closely approximates the spectral schedule in both structure and FID, reflecting dataset-specific adaptation.
Experiment statistical significance
We compute the FID and Wasserstein-2 distance for CIFAR- 10 using three independent runs, each consisting of 50,000 samples generated with different random seeds ([0-4999), [5000-9999], and [10000-1499M) 19999], to capture random variability. The results in Tables TallesTannee [[ report the Wasserstein-2 distance and FID (respectively), along with 95% confidence interval, calculated using the standard error and the t-distribution.
Table 1: Wasserstein scores (mean + 95% CT) across selected diffusion steps for different noise schedules.
10
20
28
38
50
90
Lincar EDM
3.28 0.28
1.61 0.01
1.07
0.90
0.02
0.73
0.02
0.54 0.04
3.77 = 0.02
1.99 = 0.01
1.43
0.03
1.07
0.01
0.83 = 0.02
0.53 0.04
1,
Cosine (0.2, 2)
3.52 - 0.01
1.8
1.35
1,06 0.01
0.88 = 0.02
0.67 + 0.03
Cosine (0. 1)
6.44 0.01
4.97 0.02
4.31 0.02
3.76 0.01
3.29
0.01
2.46 0.01
Sigmoid (0, 0.7)
4.60 1: 0.01
2.75 0.01
2.10
1.63 1 0.01
1.29 1 0.01
0.81 0.02
Sigmaid (-3, I)
5.67 0.01
3.71
2.94
2.36 0.01
1.00
1.15 0.00
Sigmoid (-3,3,
5.00
0.001
2.13
1,60
0.66 0.02
Spectral
3.21 + 0.24
1.59
1.00
0.81 - 0.02
0.63 = 0.02
0.48 0.04
Table 2: FID scores (mean +: 95% CI) across selected diffusion steps for different noise schedules.
20
28
38
50
Lincar
17.59 + 0.26
9.00 + 0.17
7.19 + 0.25
6.04 + 0.26
5.22 = 0.25
4.00 +:0:00 0.26
Cosine (0. 1,
24.31 + 0.14
8.63
0.16
6.00 0.16
4.65
+ 0.18
3.86 - 0.20
2.94
Cosine (0.2, 1.2)
21.92 0.29
1.210:0.00
9.85 0.25
8.90 0.25
0.29
Cosine 0.5,
20.49 0.27
9.60 0.21
6.85 0.07
5.23 0.12
4.22 = 0.14
3.05
1.166
Sigmnid (0, 0.7)
23.66 0.117
14.10
11.90 0.23
10.48 1: 0.26
9.44 14440.25 0.27
7.78
0.29
Sigmoid (-3,
16.28
8.16
5.66 0.21
4.12
Sigmoid 3, 0.7))
13.67 - 0.06
6.60 0.19
5.31 0.20
4.55 1.55::0221
1.05 = 0.25
3.36 0.22
Spectral
0.20
+ 0.24
0.23
2.60
The results presented Tables T and align with the experimental observations Figure 19 supporting the spectral recommendation and its relation alternative noise schedules. The optimality with respect to the Wasserstein loss remains robust, while for the FID metric, as the number of diffusion steps increases and discretization error becomes less significant, schedules such as EDM achieve comparable performance.
38

---

Experimental result reproducibility
To facilitate easier reproduction of our results, provide examples of the derived spectral schedules obtained for the CIFAR-10 dataset by minimizing the Wasserstein-2 loss for [10, 20, 28) diffusion steps. The values rounded to four decimal places.
10 Diffussin Steps: [0.9999, 0.9899, 0.9177, 0.6796, 0.3398, 0.1271, 0.0429, 0.0135, 0.0030, 0.00004]
20 Diffussion Steps: [0.9999, 0.9981, 0.9914, 0.9735, 0.9340, 0.8591, 0.7389, 0.5814, 0.4162, 0.2751,
0. 0.1724, 0.1048, 0.0627, 0.0371, 0.0216, 0.0121, 0.0063, 0.0028, 0.0008, 0.00004]
28 Diffusion Steps: [0.9999, 0.9989, 0.9961, 0.9899, 0.9778, 0.9562, 0.9204, 0.8656, 0.7882, 0.6889,
0.5747, 0.4572, 0.3487, 0.2570, 0.1849, 0.1309, 0.0917, 0.0638, 0.0442, 0.0304,
0.0207, 0.0138, 0.0089, 0.0055, 0.0031, 0.0015, 0.0005, 0.00004]
39

---

K
Further Discussion
K.1
Relationship Between Noise Schedules and Eigenvalues
To explore the relationship between the optimal spectral noise schedule and the data characteristics. we solved the optimization problem for each eigenvalue individually, with the contributions from the other eigenvalues set to zero. Using the eigenvalues of the covariance matrix from 6.1 Figure 22a shows these eigenvalues, while 22b presents the optimal solutions for 50 diffusion steps, computed using the Wasserstein-2 distance in the optimization problem. Each solution corresponds to single eigenvalue (considering only positions to 10 for clarity]
14
18
(a)
10
Dlticiion
(b)
Figure 22: Figure 22a shows the eigenvalues for the covariance matrix from Section 6.1 Figure 22b presents the results solving the optimization problem for each eigenvalue individually ly, with the contributions from the other eigenvalues set zero.
It can be observed from Figure 22b that the solution becomes concave the magnitude of the eigenvalue decreases (yellow) and more convex as the magnitude increases (blue). Notably, this behavior is determined by the magnitude of the cigenvalues (())))) rather than their indices (i), as the objective functions are independent of the index itself.
When the cigenvalues, or equivalently the DFT coefficients, decrease monotonically. a direct rela- tionship emerges between the eigenvalue magnitude and its corresponding frequency (for example, the 1/ʃ behavior observed in speech (30). In such cases, the first eigenvalues correspond to the low frequencies, having larger amplitudes, while the last correspond to high frequencies and smaller ampli- tudes. This pattern, along, with the previous obseryations aligns with the well-known coarse-to-fine signal construction behavior of diffusion models
Interestingly, by examining the spectral recommendation from Figure la it closely resembles the solutions obtained by emphasizing the highest eigenvalue (2b). This suggests that using the Wasserstein-2 loss tends to favor larger magnitude eigenvalues. This behavior is also reflected in the relative error, (I), Aestl)/ )estl)/(A; shown Figure 23 where larger eigenvalues exhibit smaller relative errors.
'The first eigenvalue is excluded as it disrupts the monotonicity. 'Low magnitude eigenvalues relate with concave schedule and high magnitude eigenvalues correspond convex schedule.
"Higher frequency components are emphasized by allocating more steps toward the end of the diffusion process, while lower- freeueryy components emphasized earlier.
40

---

(a)
(b)
(c)
(d)
Figure 23: Figures 23a and present
present the eigenvalues of matrices and 12g respectively. Figures
233 and 22d illustrate illlluuttt the relative eπror of the spectral recommendation, using 60 diffusion steps and the Wasserstein-2 distance. Notably, larger eigenvalues exhibit smaller relative errors. In Figures 23b and 23c the first elements were manually chosen, as they originally had extreme values (1000 and I0 respectively), and we aimed keep the figure within reasonable scale.
The influence of the eigenvalue magnitude, especially the dominant ones, on the resulting schedule further illustrated through additional examples. Figure 12d displays the covariance matrix from Figure 12a scaled by factor of = 20, which amplifies the dominant eigenvalues, as shown by the relation C Au C Av. Consequently, the spectral recommendation in Figure 12 appears more convex than in Figure 12c A similar trend is observed in Figure T6 where the spectral recommendation for t/ 0.05 exhibits
concave shape compared to fh. 0.1. This relationship open up possibility of designing loss functions which focus on specific frequency ranges interest. Further discussion is provided in K.3
Note: We used the Wasserstein-2 loss. However, alternative measures, such as KL divergence, could
also yield similar results.
41

---

K.2 Temporal Dynamics of the Diffusion Process
Our analysis has focused far the final distribution resulting from the diffusion process, as described by (TO). However, the analytical formulation also enables the examination of phenomena occurring throughout the diffusion process. In particular, Equation (29) describes each intermediate state V, as a function of the preceding noise schedule components and the initial noise Vo. This enables the study of spectral properties and dynamics across all diffusion steps, rather than being limited to the final output.
Figures 24a and 24b show the relative error |X, Aesl/A for the 10 largest eigenvalues (sorted, largest on the right) over the final 20 steps of 60-step diffusion process, using the Cosine schedule 1) and the spectral schedule respectively. In both figures, eigenvalues sorted in ascending order, allowing a comparison between high-frequency components (lower-magnitude eigenvalues) and low-frequency components -iggerr-agnitude eigenvalues).
=,
Tech ger
(a)
(b)
Figure 24: The relative errors of the the Cosine 0, e 1, 1) (24a) and the spectral (2bb) schedules for the 10 largest eigenvalues (sorted, largest on the right) over the final 20 steps of 60-step diffusion process.
In both figures, We observe that the error decreases consistently for large eigenvalues, but increases in the final steps for smaller ones. In addition, the final relative error (yellow curve) is notably lower for high-magnitude eigenvalues (i.e., low frequencies). This behavior aligns with the observed bias toward mid-to-high frequencies reported in prior works (39]. Notably, this effect is under the Cosine schedule; while both schedules yield similar errors for large eigenvalues, the Cosine schedule exhibits much higher errors for small eigenvalues (i.e., high-frequency components). This suggests the effectiveness of the proposed optimization procedure in reducing such errors.
pronounced
An additional aspect is the evolution of eigenvalues throughout the diffusson process. Figures 25a and 25b illustrate this for the Cosine (s = 0, e = 1, r = 1) and spectral schedules for a 60-step difiusion process. An interesting observation is that maggitude cigenvalues tend to converge to their final values faster than high- magnitudg ones. Notably, the convergence rate does not necessarily reflect the accuracy of the final values, discussed in Figure 24 Comparing convergence rates across different noise schedules or leveraging the intermediate expressions to design improved schedules are additional aspects that can be explored; however, we leave them for future work.
(a)
(b)
Figure 25: The dynamics of eigenvalues through a 60-step diffusion process for the Cosine (s
= 1, T= 1) (25a) and the spectral (ceduuess
42

---

Figure 26 illustrates the evolution of the Wasserstein-2 error over the final 20 steps of a 60-step diffusion process. It can be observed that the Wasserstein-2 error consistently decreases for both the cosine 10,1,1) (Figure 26a and the spectral noise schedules (Figure 26b ), with the spectral schedule converging lower final value
411
542
DITua
Difuakan
(a)
(b)
Figure 26: The dynamics of the Wasserstein-2 distance over the final 20 steps of a 60-step diffusion process for the Cosine (s 0, 1, T (26a) and the spectral (2bb) schedules.
43

---

K.3
Relationship Between Noise Schedules and the loss functions
Building on the spectral monotonic behavior, the loss function can be adjusted to weight different spectral regions in various ways, shaping the noise schedule based on specific objectives.
We propose weighted II loss for the first and the second moments of two Gaussian distributions P(Vo; @x) and P(vo).
Σ X Σ, [Di
Dr, (P(Vo: P(vo))
(D2). 1)²
(64)
The first term applies a weighted 1/ loss to the eigenvalues, while the second term computes weighted 12 norm of the mean vectors This design ensures that eigenvalues with larger magnitudes and mean components with higher values have greater influence on the overall loss.
Figure 27 illustrates the spectral recommendation obtained by solving the optimization problem in 14 using the Weighted II loss. The results are based on the Gaussian MUSIC-Pi dataset described in 6.2 where d 16, 000 and th 0.05.
Figure 27: Comparison of the spectral schedule with various heuristic noise schedules for S 112 diffusion steps. The figure presents the linear, EDM (p 7). Cosine-based schedules, including Cosine (s = 0, 1, T = 1) from 23 5 The parametric estimation of the cosine function is shown red.
Interestingly, using the weighted I/ loss results in spectral recommendation that aligns with established heuristic methods. Specifically, it corresponds to the manually designed cosine (0, 10.11) schedule proposed in (23). This observation could indicate potential link between the design of widely used noise schedule heuristics and a bias against high-frequency regions, which has been observed in previous research 339
Note: The relationship between the magnitude of the eigenvalues and their corresponding frequencies holds tight only when monotonic behavior is present. In real-world scenarios, as shown in Figure 18c the eigenvalues' magnitudes generally decrease, but the function is not strictly monotonic. In such cases, alternative approach is required, one that either analyzes broader spectral regions considering both the values and indices of the eigenvalues.
We aim maintain the relationship between both components similar the Wasserstein-2 distance.
44

---

L Analysis of Mean Bias
In the following section, analyze the mean bias expression. To maintain consisteney and preserve the connection to the frequency domain, we conduct the analysis on the circulant matrices used in Section 6.1 which can be diagonalized using the discrete Fourier transform (DFT), denoted by F(.). Naturally, the same approach can be extended to any covariance matrix, Eo. using its eigenbasis U.
The mean bias expression (D2 Du arises from the difference between EE xxo and IE [&]]] In particular, We focus the absolute magnitude of the expression [D2 II - The term D2, as defined in (10). depends on Ag and on a, the chosen noise schedule. Notably, for stationary signals, μo deterministic, resulting in the vector on where all entries are zero except for the first element. specifically, for mean-centered signal where M the DDIM process remains unbiased, regardless of D2 expression. In other cases, for given f the primary source of bias originates from the main diagonal of [D2 - 1/.
Figure 28 analyzes the mean bias for two choices of Ao with d = 50. It compares the values of [D2 across several cosine noise schedule heuristics and illustrates how behavior depends on the number of diffusion steps.
35-
32-
(a)
(d)
23012
3,00
(h)
Figure 28: Figures and 28b display the eigenvalues for two choices of Ao with d = 50. Figures 28c and 28d compare the mean bias values across different parametrization of the Cosine noise schedule using 112 diffusion steps. Figures 28c and 281 show the bias for varying numbers of diffusion steps with the Cosine (0,1,1) noise schedule, while Figures 28g and 28h illustrate the same for the Cosine (0,0.5,1) schedule.
Figures 28c and 28d reveal that for certain heuristics, such as Cosine (0,1,1), the bias is negligible, while for others, like Cosine (0,0.5,1), the bias increases. Additionally, the magnitude of the eigenvalues {>; >= plays a significant role in determining the bias; as the eigenvalues grow larger, the bias also tends to increase.
Figures 28e and 28f illustrate the bias across various numbers of diffusion steps {10, 50, 100, 200, 500, 750, 1000} for the Cosine (0.1,1) noise schedule. Similarly, Figures 28g
45

---

and 28h show the bias for the Cosine (0,0.5,1). In both cases, increasing the number of diffusion steps leads gradual rise in the mean bias [D2 -/.
M DDPM vs DDIM
The explicit formulations DDPM and DDIM in 12) and (T0) also enable their comparison in terms of loss across varying diffusion depths and noise schedules. Figure presents such comparison using the Wasserstein-2 distance on a logarithmic scale. The results show that DDIM sampling is faster and yields lower loss values than DDPM for each noise schedule, aligning with the empirical observations in [27]. addition, it can be seen that the spectral schedule with DDIM consistently maintains optimality with respect all other experiments.
LEWE (EEIA
SIEE EHE BROM!
STOPH
Wasembin Daurrc Daurre
Tel Diftusiin
Figure 29: Comparison the Wasserstein-2 distance between DDIM and DDPM for different noise schedules, including the spectral recommendationn across various diffusion steps.
Figure 30: Comparison of the spectral noise schedules in DDPM and DDIM, derived from the covariance matrix outlined in Sec. 6.. with parameters d=50, 1=0.1, 1, and total diffusion steps set to {10, 28, 38, 60, 90, 112, 250, 334, 500, 750, 10003.
N Optimization Time Analysis
The computational cost of solving the optimization problem for time-dependent signals depends on several factors. As discussed in Appendix [.4] increasing the resolution of temporal signals improves both accuracy and adaptation the data characteristics, but also increases computational cost (larger covariance matrices allow for higher frequency accuracy). Additionally, factors like the sampling rate and stationary regions influence the results.
We report the computation times for the covariance matrices of different datasets, with dimensions 400 x 400 for MUSIC, 3072 x 3072 for CIFAR10, and 12288 x 12288 for AFHQv2. For the MUSIC dataset, we applied stopping criteria of 1500 optimization steps or function tolerance of 10-6 For CIFAR10 0 and AFHQv2, we used 2000 steps the same tolerance to ensure convergence for large number of diffusion steps. All computations were performed on a standard CPU. Notably, using slightly more relaxed stopping criteria could achieve the same optimal schedule in less time.
46

---

Table Comparison of the optimization time (in seconds) for 400 400 covariance matrix (e.g..
MUSIC dataset), 3072 x 3072 covariance matrix (e.g.. CIFARIO dataset) and 12288 x 12288 covariance matrix (e.g., AFHQv2 dataset)
Diffusion steps
400 x 400
3072 x 3072
12288 x 12288
10
0.09
0.36
3.88
50
2.94
8.19
93.09
18.31
25.4
292.48
130
55.21
321.76
1555.87
170
69.5
558.76
2316.73
210
172.02
594.51
3884.07
250
261.69
949.00
7417.46
As shown in the Table for low number of steps, where discretization error is significant and the spectral scheduler greatly improves performance, the computation time ranges from less than a second to few tens of seconds. Over approximately 130 diffusion steps, the computation time exceeds one minute for the CIFARIO; however, in this regime, discretization errors become negligible, the overall structure of the scheduler remains unchanged, and its density increases with minimal change.
To accelerate the optimization process, we leverage this structural similarity by adopting a gradual optimization strategy: begin with a small number of diffusion steps (e.g., 10) which requires relatively low computational effort and use the resulting solution (interpolated to the appropriate length) to initialize optimization for a larger number of steps. This approach significantly reduces computation time by reusing information across step counts.
Table optimization times on the AFHQv2 dataset using random initialization (left column) versus iterative inttialization. For the iterative approach, the third column reports the time for each individual run, while the forth shows the total cumulative time to reach the solution for given diffusion step count. As observed in Table iterative initialization consistently reduces optimization time while achieving noise schedules and loss values comparable to random initialization. Further improvements may be possible by refining the strategy for transitioning between diffusion step counts during optimization.
An additional observation from Table Dis is the significant convergence time required for large covari- ance matrices, especially when a high number of diffusion steps is used. While covariance matrices audio signals are typically considered in the stationary domain with moderate dimensions, image data often involve higher resolutions, resulting in larger matrices and increased computational cost. A possible approach reduce computational only the d most significant directions and ignoring near-zero eigenvalues.
to assume a low-rank covariance matrix, selecting
Specifically, apply PCA the original dataset and conduct optimization within reduced- dimensional subspace. For the AFHQv2 dataset, the dimensionality is reduced from 64 x 64 x (12, 288) to 32 32 (3, 072) using PCA, with results shown in the rightmost column of Table As expected, optimization in this reduced subspace is faster than in the original space (64 x 64 3, left column). The subspace dimensionality acts hyperparameter, here, we d 3072, achieving significant reduction while preserving the dominant spectral components.
47

---

Table 4: Comparison of optimization times (seconds) across proposed acceleration methods on
AFHQv2 dataset
Diffusion steps
Random init. (12288 x 12288)
Iterative init. (12288 12288)
Iterative init. (summed)
PCA (3072 3072)
(12288 12288)
10
3.88
3.92
3.92
0.49
50
93.09
39.60
43.52
12.71
90
292.48
21.61
65.13
32.80
130
1555.87
46.05
111.18
350.27
170
2316.73
20.59
131.78
548.57
210
3884.07
32.00
163.78
605.66
250
74117.47
68.23
232.02
919.99
48

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer:[Yes]
Justification:
The main claims and contributions are clearly stated in the Abstract and rein- forced in the Introduction. These further supported by detailed mathematical derivations in Section B
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Throughout the paper, we present the underlying assumptions of approach. In the Discussion section [7] we address the limitations of our work and suggest potential directions for future research. Additionally, we analyze the optimization time of our method in the appendix N
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
49

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: The Gaussianity assumption and all theorems are clearly stated, with accom- panying sketches and intuition provided in Section Complete proofs are included the supplementary material.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Throughout the paper, we explicitly describe the optimization problem, including the loss function, constraints, and our solution method (SLSQP via SciPy). To support reproducibil- ity, we provide links to the open-source models used in Section 6.3 Additionally, Appendix includes example spectral noise schedules for small number of steps on the CIFAR-10 10 dataset.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
50

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: The code for solving the optimization problem is not yet publicly available; However, we provide a detailed description of the optimization procedure in Sections 4 and 6 which can be reproduced using standard tools such the scipy package. Additionally, we include the resulting spectral schedule used for CIFAR-10 Appendix [J] and we refer to the relevant (eevvanttopensouure open-source model for reproducibilityy The corresponding code will be released shortly. All the data used in the experiments is open source and can be accessed as detailed in 6
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips.co wttpc//pppppggucciceee guw /owiideeeoooobimisscceee wwtpbbbisssiooooooooiooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Throughout the paper, we provide complete details the optimization method 4 and the datasets used 6
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
51

---

Answer: [Yes]
Justification: In section we present a comprehensive comparison of the FID and Wasser- stein results, including a confidence interval. A detailed explanation of the calculation process for these results is also provided.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
In section N we compare execution times for the optimization problem across different covariance matrix sizes and discuss the results, along with specifying the compute resources used.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
ines?
Answer: [Yes]
Justification: guidelines.
We have reviewed the NeurlPS Code of Ethics and fully comply with its
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
52

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer:[NA)
Justification: We believe this work does not have direct societal impact, either positive negative, as it focuses on analysis of diffusion models. In the initial sections and theoretical analysis, we assume Gaussianity, and later apply the optimized schedules more general distributions. Although we utilize pretrained models, we do not produce content that could lead potential malicious or unintended uses.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deploymennt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
53

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We acknowledge and properly credit all original sources of the assets used in our work. Specifically, we used the following datasets: CIFAR-10 (custom license), MUSIC (22) (CC BY 4.0), and SC09 (33 (CC BY 4.0); and the following pretrained models: WVlab-EDNM [T3 (CC BY-SA 4.0) and DiffWave [T6] (Apache License 2.0). All licenses and terms of use have been clearly respected and properly cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cww/coo/coaooooo datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's ss creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
54

---

According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.MM policy httt:p/s/aaarrsssscc///ffeeeeees for what should or should not be described.
55


Reviews:
[{"review_id": "u65EwTpx98", "reviewer": "Reviewer_YDdm", "summary": "The paper presents a novel spectral analysis of the discrete diffusion reverse process. This leads to a closed-form expression for the spectral transfer function, which characterizes how the final generated signal results from a linear transformation of the initial Gaussian noise (Theorems 3.5 and 3.6).\n\nBuilding on this analysis, the paper introduces an optimization problem to compute a data-dependent noise schedule that aligns with the spectral properties of the target data. This approach allows for the derivation of an optimal noise schedule without being constrained by heuristic bounds or fixed step counts.\n\nThe experimental results—spanning synthetic setups and real-world data under Gaussian assumptions—demonstrate that the proposed spectral schedule outperforms standard heuristics (e.g., linear or cosine). Specifically, results on CIFAR-10, MUSIC, and SC09 datasets show that the optimal schedules adapt to the unique spectral characteristics of each dataset, highlighting the limitations of one-size-fits-all heuristics.", "questions": "-\tIn Section 6.3 (lines 338–340), the authors use a pretrained denoiser trained with a continuous noise schedule for the CIFAR-10 dataset. Why not use a pretrained model with a standard discrete schedule, such as this one: https://huggingface.co/google/ddpm-cifar10-32 \n\n-\tThe current approach involves an iterative optimization process that scales with the number of diffusion steps. Is it possible to approximate or bypass this optimization to accelerate computation?\n\n-\tHow would the results differ if the denoiser were trained from scratch using the optimized spectral schedule, rather than applying it only at inference time?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Note: I have not verified the theoretical proofs in full detail, and my comments are based on my best understanding. I am open to adjusting my evaluation in light of other reviews.\n\nStrengths\n\n•\tThe idea of a data-dependent, adaptive noise schedule is intuitive and well motivated.\n\n•\tThe method is inference-time only—it does not require retraining the denoiser.\n\n•\tThe method is designed for discrete processes (DDIM/DDPM) rather than continuous-time formulations (SDE/ODE), which aligns better with practical implementations.\n\nWeaknesses\n\n•\tAs acknowledged in the paper’s limitations section, computing the optimal spectral schedule can be slow—scaling with the number of diffusion steps and dataset size—taking several minutes for large setups.\n\n•\tThe assumption that the target data is drawn from a Gaussian distribution may be too strong for real-world applications. While the experiments include pretrained diffusion models, CIFAR-10 is a relatively small and low-resolution dataset and may not strongly violate this assumption.", "was_revised": true, "final_justification": "I thank the authors for further clarifications on the two remaining concerns. I satisfy with the rebuttal and still lean on accepting this paper. \n\nThanks", "rebuttal": "Thank you for your detailed and constructive feedback. You provided insightful comments and we would like to respond to them. \n\n**Answer to weaknesses:**\n1. As the efficiency of the optimization procedure was also noted by Reviewer M8z5, please refer to our response under Weakness 2, where we outline several alternative approaches for solving the optimization problem. These methods scale well with both the number of diffusion steps and the dataset size, while achieving an identical noise schedule and hence performance, in significantly less time (on the order of tens of seconds).\n\n2. In this work, we focus our theoretical analysis on Gaussian signals, which allows us to derive clear and explicit formulations that facilitate the study of both the final output and the intermediate stages of the diffusion process. Additionally, these formulations shed light on various phenomena and their relationship to spectral properties (see Section 6.1 and Appendix K).  We acknowledge that real-world datasets may not follow a Gaussian distribution. However, in Section 6.3, we demonstrate how the recommendations derived from our analysis can remain relevant in practical settings, with solutions achieving optimal results in terms of Wasserstein distance and FID (quality). In addition, Possible extensions of the analytical solution are discussed in our response to Reviewer M8z5 under Question 1.\\\nIn response to the comment regarding the relatively low resolution of CIFAR-10, we conducted an additional experiment on the higher-resolution AFHQv2 dataset [1], which has a covariance matrix of size 12,288 × 12,288 (corresponding to 64×64×3 images). Applying our method, we derived the optimal noise schedule and evaluated its performance on this real-world dataset using the pretrained denoiser from [2]. Tables 1 and 2 show a comparison of Wasserstein distance and FID scores between our optimal noise schedule and various heuristic baselines on AFHQv2.\n\n**Table 1 – Wasserstein-2 distance comparison on the AFHQV2 dataset between the spectral noise schedule and other heuristics**\n\n\n\\\\[\n\\\\begin{array}{lcccccccc}\n & 10 & 20 & 30 & 40 & 50 & 70 & 90 & 112 \\\\\\\\\n\\\\hline\n\\\\text{Edm }(\\\\rho=5) & 8.08 & 4.91 & 3.76 & 3.18 & 2.83 & 2.43 & 2.22 & 2.08 \\\\\\\\\n\\\\text{Edm }(\\\\rho=7) & 8.21 & 5.01 & 3.19 & 3.34 & 2.93 & 2.50 & 2.32 & 2.18 \\\\\\\\\n\\\\text{Linear} & 7.82 & 4.55 & 3.40 & 2.89 & 2.63 & 2.33 & 2.18 & 2.07 \\\\\\\\\n\\\\text{Cosine }(0,1,1) & 15.16 & 9.66 & 6.81 & 5.31 & 4.33 & 3.23 & 2.67 & 2.32 \\\\\\\\\n\\\\text{Cosine }(0,0.5,1) & 13.47 & 11.25 & 10.02 & 9.28 & 8.64 & 7.78 & 7.22 & 6.78 \\\\\\\\\n\\\\text{Sigmoid }(0,3,0.7) & 10.37 & 7.01 & 5.45 & 4.62 & 4.00 & 3.30 & 2.90 & 2.64 \\\\\\\\\n\\\\text{Sigmoid }(-3,3,0.7) & 11.19 & 7.31 & 5.54 & 4.60 & 3.91 & 3.12 & 2.69 & 2.39 \\\\\\\\\n\\\\text{Spectral} & 7.37 & 4.26 & 3.24 & 2.76 & 2.48 & 2.20 & 2.06 & 1.97 \\\\\\\\\n\\\\end{array}\n\\\\]\n\n**Table 2 – FID score comparison for the AFHQV2 dataset between the spectral noise schedule and other heuristics**\n\n\\\\[\n\\\\begin{array}{lcccccccc}\n & 10 & 20 & 30 & 40 & 50 & 70 & 90 & 112 \\\\\\\\\n\\\\hline\n\\\\text{Edm }(\\\\rho=5) & 10.07 & 4.72 & 3.58 & 2.97 & 2.67 & 2.48 & 2.32 & 2.24 \\\\\\\\\n\\\\text{Edm }(\\\\rho=7) & 13.37 & 5.11 & 4.56 & 3.37 & 2.89 & 2.57 & 2.38 & 2.30 \\\\\\\\\n\\\\text{Linear} & 11.42 & 6.73 & 5.22 & 4.49 & 4.00 & 3.47 & 3.18 & 2.98 \\\\\\\\\n\\\\text{Cosine }(0,1,1) & 26.11 & 10.01 & 5.79 & 4.27 & 3.53 & 2.87 & 2.60 & 2.46 \\\\\\\\\n\\\\text{Cosine }(0,0.5,1) & 26.54 & 16.31 & 12.50 & 10.63 & 9.13 & 7.37 & 6.38 & 5.72 \\\\\\\\\n\\\\text{Sigmoid }(0,3,0.7) & 18.22 & 11.42 & 9.08 & 8.02 & 7.22 & 6.31 & 5.78 & 5.39 \\\\\\\\\n\\\\text{Sigmoid }(-3,3,0.7) & 15.09 & 7.08 & 4.87 & 4.01 & 3.48 & 2.99 & 2.76 & 2.62 \\\\\\\\\n\\\\text{Spectral} & 8.14 & 4.13 & 3.06 & 2.72 & 2.56 & 2.41 & 2.30 & 2.23 \\\\\\\\\n\\\\end{array}\n\\\\]\n\n\n\nTables 1 and 2 demonstrate that the spectral recommendation outperforms all other heuristics across both metrics. As noted in the paper, the performance gap is especially pronounced at low number of diffusion steps, where discretization error tends to be higher. Nevertheless, the spectral schedule remains optimal across all number of steps, making it well-suited for real datasets. A heuristic that is closest to the spectral schedule in both structure and FID performance is EDM with ρ = 5 (a figure illustrating this will be included in the paper, as it cannot be shown here). This differs from the results observed on CIFAR-10, MUSIC and SC09, highlighting the unique spectral characteristics of each dataset. \\\nIn addition to our experiment on AFHQv2, a higher-resolution dataset, our response to Reviewer M8z5 shows that our method can effectively leverage spectral properties in high-resolution settings. Moreover, we see extending the theoretical framework to more general distributions, such as GMMs, as a intiruiting direction for future research.\n\n**Answer to questions:**\n1. Section 6.3 evaluates the spectral recommendation against various existing noise schedules by generating thousands of samples for each schedule across different numbers of diffusion steps. We employ a pretrained denoiser trained on a continuous noise schedule for two main reasons. First, it ensures a consistent and fair evaluation by reducing variability that may result from training different denoisers, thereby allowing us to focus directly on assessing the quality of the noise schedules themselves. Second, it allows sampling from multiple noise schedules without the need for retraining, thereby improving efficiency. While relying on a single denoiser trained with a continuous schedule, rather than multiple denoisers trained on specific discrete schedules, may introduce some effects (see our response to Question 3), these are expected to be minor and consistent across all schedules, particularly compared to the variability introduced by training separate models.\n2. In our response to Reviewer M8z5 (under Weakness 2), we discuss the method’s efficiency and present two approaches to enhance the efficiency of the optimization process: one targeting scenarios with a large number of diffusion steps, and another focusing on high-resolution datasets. Additionally, we would like to note the optimization is performed once per dataset and specified number of diffusion steps, following a straightforward procedure.\n3. We anticipate that training the denoiser specifically for the spectral recommendation would improve the final results, as the denoiser would learn to map noisy inputs to clean data based on the precise noise characteristics of that schedule. This tailored training enhances accuracy and stability, allowing the model to fully leverage its capacity for the given scheduler. As noted in our response to Question 1, we expect these improvements to be consistent across different noise schedules, thereby preserving their relative performance. Due to time and resource constraints during the response period, we were unable to complete training the denoiser on CIFAR-10 or AFHQv2 using the spectral schedule. However, we recommend training the denoiser accordingly once the spectral schedule is established to derive better results. We will address this point in the paper.\n\n[1] Yunjey Choi and Youngjung Uh and Jaejun Yoo and Jung-Woo Ha. StarGAN v2: Diverse Image Synthesis for Multiple Domains\n\n[2] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space\nof diffusion-based generative models. Advances in neural information processing systems,\n35:26565–26577, 2022"}, {"review_id": "FOW7a8N8nZ", "reviewer": "Reviewer_rtuD", "summary": "An important aspect of diffusion model sampling is the choice of time/noise discretization, or \"noise schedule\". A bad noise schedule can yield poor sample quality, or at least make sampling require more steps for good sample quality than with a better schedule. Given that noise schedules tend to be chosen on an empirical basis, the authors of this work propose to use theoretical insight to design better noise schedules.\n\nThe key idea is that Gaussian/linear model sampling dynamics is solvable, even when time/noise is discretized. The authors use this fact to express the final sample in terms of the noise schedule and initial noise seed, and then to compute final sample quality (in terms of the Wasserstein-2 distance to the 'true' Gaussian distribution) in terms of the noise schedule. They can then compare their \"spectral recommendation\" to other schedules on real and toy data sets.", "questions": "1. Is it possible to define a measure of sample quality for the audio data set? It seems important to have a sense for how good the schedule is doing at generating high quality samples, not just at reproducing the data set's Gaussian approximation.\n2. Have the authors identified a case where their recommendation is *particularly* effective? In the case of CIFAR-10, it seems like the EDM schedule does about as well, and does not require a Gaussian fit to use. \n3. How feasible / computationally costly is it to do a Gaussian fit on various high-dimensional data sets? Can the authors discuss this point explicitly?", "limitations": "I think the extent to which their proposal yields meaningfully higher sample quality and/or efficiency should be discussed in more detail, and ideally the authors identify one or more cases where their approach yields a large improvement as opposed to a small one. The authors should also discuss the point that their method requires a Gaussian fit, which is difficult or impossible for large conditional diffusion models.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths.** I'm sympathetic to the idea of using theoretical insight to inform the design of things like noise schedules. The idea to exactly solve the (discrete-time) sampling dynamics associated with the Gaussian model, and use this result to determine the theoretical 'goodness' of noise schedules by computing Wasserstein-2 distances (and KLDs), is interesting. I have done similar math before, and do not doubt that the authors' math is correct.\n\n\n\n**Weaknesses.** My main concern is that neither the theoretical analysis nor empirical analysis appear to be major contributions. The central theoretical result of the paper is the solution of the Gaussian model's sampling dynamics for an arbitrary $\\alpha_t$, but this has been done many times now, as this paper explicitly mentions (see, e.g., line 92). Given this result, it's easy to compute other downstream things like Wasserstein-2 distances and KLDs. The main difference, as far as I can tell, is that this paper's result is in *discrete* rather than continuous time. Although this paper appears to view this as a major technical distinction between its calculations and prior work (see lines 215-216), I kind of disagree. Linear recurrence relations are essentially as easy to solve as linear ODEs. The calculation may be valuable, but it's extremely closely related to work that's been done before.\n\nOn the empirical side, the authors use a few data sets to illustrate that their method gives reasonable results. But the evaluation criteria are a little weird, since the authors are mainly concerned with the extent to which their method reproduces the *Gaussian approximation* of a given data set, rather than returns high quality samples. The most interesting plot is Fig. 4a, since it depicts FID scores (a measure of sample quality) for CIFAR-10. According to the plot, the EDM-recommended noise schedule does as good as their method. Since this is the only sample-quality-related plot I can see in the main text, I'm not convinced that the authors' proposed method is particularly good.\n\nA subtle point which the authors don't mention is that even coming up with an 'optimal' noise schedule requires fitting a Gaussian to a data set, which is not always possible for extremely high-dimensional data sets, and is (at least to my knowledge) essentially impossible for text-to-image conditional models. If the EDM schedule performs about as well, but doesn't require this step, that's worth discussing explicitly. \n\nI find all of the figures kind of hard to parse, since the text and labels are very small. It would really help the paper if the sizes of axis labels, numbers, and so on were greatly increased.\n\nA few minor nitpicks. The authors mention the usefulness of a \"spectral\" perspective on diffusion models, which has parallels to signal processing. But I find this link really misleading. In signal processing settings, spectral analysis involves transforming the time domain to a frequency domain; here, since the data (e.g., an image) may not naturally have a time component, the eigendirections of the covariance matrix do not necessarily have any kind of frequency-like interpretation. Also, the aforementioned papers studying the Gaussian model also do this diagonalization trick, so the spectral idea is far from new. \n\nThe authors should probably cite some foundational papers in their first paragraph, where they write \"Diffusion Models (DMs) have become ...\" There are a number of typos, like misspelled words and missing periods (see, e.g., line 270). These should be fixed.", "was_revised": true, "final_justification": "I think the idea for a 'principled' framework for noise schedule design is important and interesting, and that although some of the Gaussian model math has been done before, the paper still offers a worthy contribution. The authors have done additional experiments to bolster the case that their method works well. While it's true that their method sometimes does about as well as the off-the-shelf EDM method, this isn't always true, and they rightfully point out that it's helpful regardless to have some way to predict that one noise schedule or another will be good. They've done a good amount of work to address various reviewers' concerns. I think the paper should be accepted.", "rebuttal": "Thank you for the detailed comments. After carefully reviewing the text, we would like to address the points you raised comprehensively.\n\n**Answer to weaknesses:**\n- Theoretical Observations: In lines 92 and 215–216, we refer to three works that rely on the assumption of a Gaussian distribution, each differing from our approach in a distinct manner. Work [3] proposes a method for optimizing noise schedules, presenting an analytical solution under more restrictive assumptions, specifically zero-mean Gaussians with diagonal covariance matrices (see lines 254–259). Furthermore, they introduce a discretization of the continuous SDE and formulate an optimization problem using linearization and KLUB loss. However, this approach is computationally demanding and limited to small number of diffusion steps. Moreover, the authors do not provide an explicit expression for sampling dynamics. Work [4] evaluates several sampling schemes by analyzing various sources of error. To derive continuous-time sampling expressions, the authors assume a centered data distribution and commutativity of the covariance matrices. Although related to our work, this study does not focus on optimizing the noise schedule or analyzing of the diffusion process. Work [5] explores the relationship between the learned score and the true data score. It derives a continuous-time sampling expression by analyzing components that are either orthogonal or aligned with the basis vectors. Notably, this work does not address the problem of optimizing the noise schedule. In both [4] and [5], the resulting time-domain expressions are continuous, disregarding discretization effects and missing an explicit connection to the full traversal of the noise schedule.\nIn our work, we derive an explicit and compact expression in the spectral domain based on the discrete formulation. This allows for a simple optimization procedure to compute the noise schedule and also provides insights into the diffusion dynamics. In addition, We believe this perspective could serve as a useful basis for further exploration.\n- Empirical observations: Sections 6.1 and 6.2 validate the theory under the Gaussian assumption, where the experiment fully aligns with the theoretical framework; hence, the Wasserstein distance suffices for evaluation. The results demonstrate that our method outperforms the existing heuristics (see Appendix I). Section 6.3 examines the spectral recommendation on real data, where the Gaussian assumption does not hold. We evaluate performance using both the Wasserstein distanc which aligned with the optimization objective, and FID as a quality measure. Figures 4a and 4b show that for CIFAR-10, the spectral schedule  consistently outperforms heuristic baselines across all number of diffusion steps, highlighting the practical relevance of our theoretical insights.\\\n*Interpretation of Results and Proximity to EDM:*\\\nFigure 4a shows that for a small number of diffusion steps (e.g., up to 40), the FID gap between the optimal and heuristic schedules is significant. As the number of steps increases, the sampling intervals become smaller and performance differences gradually diminish. As discussed in lines 344–346, this behavior is expected: discretization error is more pronounced when fewer steps are used, making the optimization problem more significant in these regimes. Nonetheless, our solution remains optimal and effective across all number of diffsuion steps.\\\nWe believe the reviewer’s comment: “According to the plot, the EDM-recommended noise schedule does as good as their method... I’m not convinced that the authors’ proposed method is particularly good”, overlooks a central point. The observed similarity between the optimal solution and a specific heuristic does not diminish the value of our approach. Rather, it shows that certain heuristics may, for unique data characteristics, approximate the optimal schedule.  importantly, our goal is to avoid repeated, ad hoc comparisons across noise schedules. Hence, as aforesaid, our method is not heuristic-based but formulates a principled optimization problem whose solution may, in hindsight, resemble existing heuristics.\\\nFigure 3 shows that for CIFAR-10, the optimal schedule shares structural similarity with EDM using $\\rho = 7$, making EDM one of the closer-performing heuristics. However a notable performance gap remains, especially at low step counts, where scheduling is crucial. For instance, [2] recommends using 18 diffusion steps for CIFAR-10 and the EDM schedule as a practical trade-off between speed and quality; in this regime, our method consistently outperforms all baselines, including EDM. A similar trend observed in the MUSIC dataset with cosine schedules $(0, 0.5, 1)$. While the optimal schedule may resemble an existing heuristic structurally or in performance, this does not reduce the significance of our approach, as heuristics are plausible candidate solutions within the broader optimization framework.\\\nAdditionally, In response to your comment on the experimental scope, we have included additional results for the AFHQV2 dataset at 64×64×3 resolution (see our response to reviewer YDdm, Tables 1 and 2). These results demonstrate that the spectral recommendation remains optimal across all tested heuristics for varying step counts on higher-resolution real data. Furthermore, the optimal noise schedule in this case resembles the EDM schedule with $\\rho = 5$.\n\n- We acknowledge the reviewer’s concern regarding the use of the term spectral. While “spectral analysis” is often associated with the frequency domain, it can broadly refer to the study of general eigenstructures (e.g., spectral clustering, PCA). In Section 6.1, our analysis focuses on circulant covariance matrices for one-dimensional signals, where the spectral perspective corresponds directly to the Fourier domain and frequency components. In Sections 6.2 and 6.3, the term is used more generally to denote the eigendirections of the covariance matrix. As noted in the related work section, other studies have also employed spectral analysis; however, their objectives and focus differ fundamentally from ours.\n- We appreciate your comment regarding the figures, missing citations, and typos, and we will revise the paper accordingly.\n\n**Answer to Questions:**\n1. A commonly used metric for evaluating audio quality is the Fréchet Audio Distance (FAD) [6], which leverages features extracted from VGGISH [7]. As discussed in Appendix J, FAD is less reliable for the MUSIC dataset due to its short frame lengths, and less informative for SC09 given its speech-specific nature. A promising alternative would be to replace VGGISH with a model trained specifically for short audio or speech. \n2. We would like to address this question in light of our response to Weakness 2. Figures 4a and 4b show that the spectral schedule remains optimal across all step counts. For a small number of diffusion steps, the optimal schedule yields a significant improvement over heuristic baselines, as discretization error is more pronounced in this regime. This advantage can help mitigate the tradeoff between sampling speed and synthesis quality.\\\nFor a larger number of diffusion steps, the gap between the optimal solution and similarly structured heuristics becomes smaller. However, as noted, this does not detract from the value of our approach, which remains effective in this regime. A key strength of our method is its ability to analytically determine an optimal schedule without relying on manual tuning. \\\nRegarding the reviewer’s comment that “in the case of CIFAR-10, it seems like the EDM schedule does about as well, and does not require a Gaussian fit to use”, we emphasize that such a conclusion is retrospective and data-dependent. For example, on MUSIC and SC09, cosine schedules with tuned parameters better approximate the optimal solution, while for AFHQV2, a different $\\rho$ value for EDM yields improved alignment. Without an analytical framework, identifying the best-performing schedule is speculative and requires a trial-and-error process.\n3. Please refer to our response to Reviewer M8z5 (Weakness 2), where we provide a detailed comparison of different data resolutions, diffusion step counts, and optimization runtimes. We also propose two approaches for reducing optimization time and demonstrate their effectiveness on the AFHQV2 dataset.\n\n**Limitations:**\n\nAs noted, our work assumes a Gaussian-distributed dataset, which may not fully hold in settings such as large conditional diffusion models. In such cases, extending the analytical framework to more expressive models, such as GMMs or probabilistic models conditioned on degraded signals, could be a promising direction for future research. Furthermore, adapting our method to the latent space of latent diffusion models or to the intrinsic manifold structure of datasets may provide additional avenues for exploration (see discussion by Reviewer M8z5).\n\n[3] Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your steps: optimizing sampling schedules in diffusion models\\\n[4] Emile Pierret and Bruno Galerne. Diffusion models for gaussian distributions: Exact solutions and wasserstein errors.\\\n[5] Binxu Wang and John J Vastola. The unreasonable effectiveness of gaussian score approximation for diffusion models and its applications\\\n[6] Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi. Fr’echet audio distance: A metric for evaluating music enhancement algorithms. \\\n[7] Hershey, Shawn and Chaudhuri. CNN architectures for large-scale audio classification"}, {"review_id": "TqbMWThLPt", "reviewer": "Reviewer_M2PR", "summary": "Diffusion model has a strong ability to model complex data distributions and generate realistic new samples. However, its generation process depends on a heuristically chosen noise schedule without a solid theoretical basis. In this paper, assuming the target distribution is Gaussian, the authors demonstrate the inference process as a closed-form spectral transfer function, showing how the generated signal develops in response to the initial noise. They then use this analysis to determine an optimal noise schedule that effectively matches the data's characteristics.", "questions": "1. The Gaussianity assumption about the target distribution is somewhat unrealistic because it is unlikely to occur in real datasets. This reduces the effectiveness of the proposed method when applied to actual data, as shown in Section 6.3. Figure 4 does not demonstrate the significant performance difference of the learned noise schedule compared to heuristic baselines.\n\n2. Even in the synthetic case where the assumption holds, Figure 1.3 indicates that the performance gap (Wasserstein distance) of the learned noise schedule relative to others is minimal. This raises the question of whether a complex \"trained\" noise scheduler is truly necessary. Is the cost of tuning hyperparameters for heuristic noise schedules (e.g., cosine, sigmoid) higher than the cost of discovering the \"trained\" noise schedule? If not, what is the main reason for developing this \"trained\" noise schedule?\n\n3. The statement in Lines 300-301 about the mean drift, “Additionally, as the depth of the diffusion process increases, the bias value tends to grow, regardless of the selected noise schedule.”, diminishes the motivation behind this paper.\n\n4. Other minor points:\n- In Eq 14, what are $\\epsilon_0$, $\\epsilon_S$? Additional explanations are needed.\n- Visualizations of some figures (e.g., Figures 1b,c, and 4a, b, c) could be improved by combining the legends horizontally and adding color to make certain lines more distinguishable.", "limitations": "Please see the above questions", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Based on the Gaussianity assumption, a novel theoretical spectral analysis of the discrete diffusion reverse process is presented, along with a closed-form expression for its spectral transfer function.\n- Formulates an optimization problem to find an optimal noise schedule that matches data characteristics, regardless of the number of diffusion steps.\n- A well-structured paper with high-quality writing.\n\nWeaknesses:\n- The Gaussianity assumption is too strong and unrealistic.\n- The performance of the identified noise schedule does not appear to outperform other schedules.", "was_revised": true, "final_justification": "Before the rebuttal, my primary concerns were the unrealistic Gaussianity assumption and the minimal improvement achieved, potentially at a higher cost than other methods. After the rebuttal, the authors’ clarifications sufficiently addressed these issues. Therefore, I have decided to maintain my score.", "rebuttal": "Thank you for your constructive feedback. We appreciate your comments and address them below.\n\n**Answer to weaknesses:**\n1. We recognize that real-world datasets may not adhere to a Gaussian distribution. As this concern was also raised by Reviewer YDdm under Weakness 2, please refer to our detailed response in that section.\n2. For a discussion of the optimal noise schedule’s performance, please see our responses to Questions 1 and 2.\n\n**Answer to questions:**\n1. We address Questions 1 and 2 jointly, as they relate to closely connected aspects of the analysis. A key source of error in diffusion models is discretization error, which stems from the discrepancy between continuous sampling equations and the discrete sampling process implemented in practice. A key consideration for any given dataset is the choice of discretization strategy, particularly the selection of an appropriate noise schedule. As illustrated in Figure 3, the optimal noise schedule can vary significantly between datasets. To avoid the need for extensive exploration of numerous candidate schedules (which would require retraining the denoiser and synthesizing many samples for evaluation), we propose a method that efficiently recommends an optimal noise schedule. This recommendation is computed once per dataset and can be obtained in a reasonable amount of time. (see our response to Reviewer M8z5).\\\nIn Section 6.1 and Figure 1.3, we present an empirical evaluation, comparing the spectral schedule to existing heuristic schedules evaluated using Gaussian distributed data. Therefore, this experiment aligns completely with the theoretical framework. As shown in Figure 1.3, the spectral schedule is optimal for any given number of diffusion steps. Importantly, when fewer diffusion steps are used and discretization error becomes more significant, the advantage of this optimized schedule is even more evident. This leads to a clear improvement in balancing synthesis time with sample quality.\\\nDetermining the optimal noise schedule without our proposed method requires extensive experimentation and comparative analysis. Furthermore, even if the optimal schedule for a given dataset closely resembles an existing heuristic in terms of structure or performance, this does not reduce the significance of our approach since each heuristic can be regarded as a plausible candidate solution within the broader optimization framework.\\\nIn Section 6.3, we derive an optimal noise schedule under the Gaussian assumption and evaluate its performance on real-world datasets using FID and Wasserstein metrics. As shown in Figures 4a and 4b, the spectral noise schedule consistently remains optimal across both metrics and for any selected number of diffusion steps. Similar results were observed in our experiments with the AFHQv2 dataset (refer to our response to Reviewer YDdm under Weakness 2).\\\nFigure 4 demonstrates that the spectral recommendation provides a clear advantage at low number of diffusion steps, where discretization error is significant. As the number of steps increases, performance differences across schedules diminish, yet the spectral schedule remains consistently optimal. Notably, for CIFAR-10, the structure of the optimal schedule resembles the EDM heuristic with ρ = 7. Consequently, the performance gap is more pronounced at low number of diffusion steps, and as the schedules become denser, the gap in FID results narrows, though a measurable difference from other baselines persists. As discussed earlier, this behavior does not detract from our method’s validity, since each scheduler can represent a plausible solution, and similarity in structure at high step counts naturally can leads to reduced differences. This is especially clear in the context where the authors of [2] suggest using 18 diffusion steps for EDM as an effective balance between sampling efficiency and synthesis quality. At this particular settings, our method significantly outperforms all other approaches, including EDM.\\\nWe note that the proposed approach is not a “complex trained noise scheduler” but rather the solution to a straightforward optimization problem performed once for a given dataset (see our responses to Reviewer YDdm under Weakness 2). In contrast, the traditional manual heuristic search for an optimal noise schedule is a time-consuming process that requires extensive tuning across numerous candidate schedules. For example, obtaining statistically robust FID measurements demands generating at least 50,000 samples, repeated three times, for each number of diffusion steps and each heuristic noise schedule under consideration. Even with such extensive experimentation, without analytical guidance, it is challenging to confirm whether the chosen schedule is truly optimal. Nonetheless, we are aware that the Gaussian assumption may not fully capture the characteristics of some real-world datasets. Naturally, extending the analytical framework to more general distributions (e.g. GMMs), would broaden the applicability of the recommendation. We view this as a valuable direction for future research.\n\n3. In our work, we formulate the task of finding an optimal noise schedule as an optimization problem based on the Wasserstein loss, which decomposes into two components: the difference in mean and the difference in covariance characteristics. We observe a mean drift that tends to increase as the inference process progresses (lines 300–301). However, as noted in lines 296–300, “different choices of the noise schedule influence the bias value, with some choices effectively mitigating it.” This behavior is further illustrated in Appendix L, Figure 27, which shows how different noise schedules affect the magnitude of the mean drift to varying degrees. While a deeper diffusion process generally tends to increase drift, selecting an appropriate noise schedule can minimize this effect. Moreover, since the mean term explicitly appears in the loss function (${([D_{2}]_i-1)}^2$ in Equation (15)), the noise schedule optimization inherently accounts for this drift, thereby strengthening the motivation for our approach. \n\n4. a) The parameters $\\epsilon_0$ and $\\epsilon_s$ are selected to start the synthesis process as close as possible to white noise and to end it near the clean data distribution, thereby enabling the capture of fine details. This choice also ensures compatibility between the training and synthesis phases when a denoiser is involved [8]. Specifically, we use $\\epsilon_s = 0.04035 \\times 10^{-3}$ and $\\epsilon_0 = 10^{-4}$.  Furthermore, together with the applied inequality constraints, this setting was originally intended to guarantee that during optimization, the $\\bar{\\alpha}$ values remain within the [0, 1] range.\\\nb) We appreciate the comments regarding the figures in the paper and will revise them accordingly.\\\n\\\n[8] Lin, Shanchuan and Liu, Bingchen and Li, Jiashi and Yang, Xiao, Common diffusion noise schedules and sample steps are flawed"}, {"review_id": "KyLdvK2uWY", "reviewer": "Reviewer_M8z5", "summary": "This work analyzes how to schedule the noise based on the data features from a spectral perspective. To achieve this goal, this work first presents a spectral analysis on the reverse process based on the Gaussian target distribution. Then, based on the spectral analysis, they propose an optimization problem to obtain the noise schedule specific to the given data feature. The simulation and real-world experiments (including VP and VE-based models) also support their theoretical results.", "questions": "Please see the Weakness part.\n1.\tIt would be better to discuss the technique challenge when considering GMM target distribution (it has a closed-form score function, but does not have a closed-form denoiser).", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n1.\tThe spectral analysis for the noise schedule design is novel and interesting.\n\n2.\tThe simulation and real-world experiments also support their theoretical results.\n\nWeakness:\n\n1.\tThe analysis of this work is limited to the Gaussian distribution, which does not match the real-world data distribution. The high-dimensional Gaussian assumption will also lead to a time-consuming optimization process for the given optimization problem. \n\nFor the above problems, some works [1][2] assume the target data distribution admits a low-dimensional linear subspace and the latent distribution is Gaussian. [2] further prove that under the linear subspace assumption, the optimization process of diffusion models is equivalent to PCA.\n\nAs mentioned in the limitation part of this work, PCA is a promising method, and it will be interesting to consider this modeling in future work. \n\n[1] Chen, Minshuo, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. \"Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data.\" In International Conference on Machine Learning, pp. 4672-4712. PMLR, 2023.\n\n[2] Yang, Ruofeng, Bo Jiang, Cheng Chen, Baoxiang Wang, and Shuai Li. \"Few-shot diffusion models escape the curse of dimensionality.\" Advances in Neural Information Processing Systems 37 (2024): 68528-68558.", "was_revised": true, "final_justification": "As the authors addressed my concerns, I have raised my score to 5 and vote for accepting this paper.", "rebuttal": "Thank you for your constructive feedback. We appreciate your insightful comments as well as the valuable references you provided. We address your points below.\n\n**Answer to weaknesses:**\n1. We acknowledge that the Gaussian assumption does not fully capture the complexity of real-world data distributions. However, our aim was to present a broader spectral perspective on diffusion models and to develop a method that could potentially generalize to more realistic settings in future work (see answer to question 2). In addition, we include further results on the AFHQV2 dataset (64*64×3), which remain consistent with the theoretical conclusions and further support them (please refer to Weakness 2 in Reviewer YDdm).\n2. As discussed in the limitations section, the computational time for solving the optimization problem scales with the number of diffusion steps and the data resolution. Importantly, this optimization is performed only once per dataset and target step count to determine the noise schedule. Nonetheless, we address these concerns explicitly here. Table 3 reports optimization times in seconds, including results from the paper and additional experiments on the AFHQv2 dataset at 64×64 resolution (see Reviewer YDdm).\n\n**Table 3 – Optimization times (in seconds) for different numbers of diffusion steps and dataset resolutions**\n\\\\[\n\\\\begin{array}{cccc}\n\\\\text{Diffusion steps} & \\\\text{MUSIC } & \\\\text{CIFAR10 }& \\\\text{AFHQv2 } \\\\\\\\\n & (400\\\\!\\\\times\\\\!400) &(3072\\\\!\\\\times\\\\!3072) & (12288\\\\!\\\\times\\\\!12288) \\\\\\\\\n\\\\hline\n10 & 0.09 & 0.36 & 3.88 \\\\\\\\\n50 & 2.94 & 8.19 & 93.09 \\\\\\\\\n90 & 18.31 & 25.4 & 292.48 \\\\\\\\\n130 & 55.21 & 321.76 & 1555.87 \\\\\\\\\n170 & 69.5 & 558.76 & 2316.73 \\\\\\\\\n210 & 172.02 & 594.76 & 3884.07 \\\\\\\\\n250 & 261.69 & 949.0 & 7417.46 \\\\\\\\\n\\\\end{array}\n\\\\]\n\n- Table 3 shows that optimization times increase with both resolution (e.g., covariance dimension) and the number of diffusion steps. To address this, we propose two methods to effectively accelerate the process.\n\n**Approach 1 – Iterative Initialization**\n\nThe timings reported in Table 3 correspond to runs where the optimization was independently initialized from random points for each number of diffusion steps, which can result in high computational cost. However, as noted in Section 6.1, optimal solutions for different step counts tend to exhibit similar structures for a given dataset, even though they are not identical, since the noise schedule induces a different density for each step count (see Figure 1a).\\\nTo leverage this structural similarity, we adopt a gradual optimization strategy: we begin with a small number of diffusion steps (e.g., 10) which requires relatively low computational effort and use the resulting solution (interpolated to the appropriate dimension) to initialize optimization for a larger number of steps. This approach significantly reduces computation time by reusing information across step counts.\\\nTable 4 compares optimization times on the AFHQv2 dataset using random initialization (left column) versus iterative initialization. For the iterative approach, one column reports the time for each individual run, while the other shows the total cumulative time to reach the solution for a given diffusion step count.\n\n*Key observations:*\n- Optimization times with iterative initialization are consistently faster than with random initialization; for example, on AFHQv2 with 250 diffusion steps, it is 31 times faster then using the random initialization.\n- The resulting noise schedules and corresponding loss values match those obtained with random initialization, regardless of the number of diffusion steps. (A figure illustrating this will be added in the revised paper.)\n- Alternative strategies for transitioning between diffusion step counts during optimization may offer additional speedups. Due to space constraints, we defer a detailed discussion of these configurations to the final version of the paper.\n\n**Approach 2 - Optimization over a Principal Subspace**\n\nWe now turn to scenarios involving high-resolution datasets and thank the reviewer for highlighting references [9] and [10]. Paper [9] investigates the relationship between the intrinsic data distribution and the learned score function, along with examining its impact on sample complexity under a low-dimensional linear Gaussian subspace assumption. Similarly, Paper [10] connects few-shot diffusion models to a PCA-like objective. Building on these insights and as noted in our limitations, we propose to evaluate our optimization method on a PCA-derived subspace to explore potential computational acceleration.\\\nLeveraging the fact that the optimization problem is formulated in the spectral domain, and assuming the dataset can be approximated as low-rank with many eigenvalues close to zero (see Figure 18), we propose to concentrate the optimization on the most significant components. Specifically, we apply PCA to the original dataset and perform the optimization in a reduced-dimensional subspace.  We demonstrate our method on the AFHQv2 dataset by reducing the original dimensionality from 64 × 64 × 3 (12288) to 32 × 32 × 3 (3072) using PCA. The results are shown in the rightmost column of Table 4.\n\n*Key observations:*\n- As expected, solving the optimization in the reduced dimension leads to shorter runtimes compared to operating in the original space (64 × 64 ×3 – see left column).\n- The resulting noise schedules and loss values closely match those obtained from the full dimensional baseline. (Following the derivation of the optimal schedule in the subspace, we assess its loss within the original full dimensional space.)\n- The subspace dimensionality serves as a hyperparameter. We chose $d = 3072$, which offers a substantial reduction while preserving the dominant spectral components. A more principled selection of $d$ is possible, which we leave to be addressed in the full paper.\n\n**Table 4 - Comparison of Optimization Time (is seconds) Across Proposed Methods**\n\n\\\\[\n\\\\begin{array}{ccccc}\n\\\\text{Diffusion steps} \n& \\\\text{Random Init. }\n& \\\\text{Iterative init. }\n& \\\\text{Iterative init. (summed) }\n& \\\\text{PCA } \\\\\\\\\n& (12288\\\\!\\\\times\\\\!12288) \n& (12288\\\\!\\\\times\\\\!12288) \n& (12288\\\\!\\\\times\\\\!12288) \n&(3072\\\\!\\\\times\\\\!3072) \\\\\\\\\n\\\\hline\n10  & 3.88     & 3.92   & 3.92    & 0.49   \\\\\\\\\n50  & 93.09    & 39.60  & 43.52   & 12.71  \\\\\\\\\n90  & 292.48   & 21.61  & 65.13   & 32.80  \\\\\\\\\n130 & 1555.87  & 46.05  & 111.18  & 350.27 \\\\\\\\\n170 & 2316.73  & 20.59  & 131.78  & 548.57 \\\\\\\\\n210 & 3884.07  & 32.00  & 163.78  & 605.66 \\\\\\\\\n250 & 7417.47  & 68.23  & 232.02  & 919.99 \\\\\\\\\n\\\\end{array}\n\\\\]\n\n\n*General observations:*\n- As shown in Table 4, the iterative initialization method substantially accelerates the optimization process without necessitating the data dimensionality reduction required by the second approach, making it well-suited for high-resolution datasets and large numbers of diffusion steps. Combining both approaches is a promising option.\n- Another interesting direction comes from the common use of latent diffusion models for high-dimensional data, where the diffusion process is carried out in a lower-dimensional latent space. It would be interesting to explore how our method generalizes in such settings.\n\n**Answer to questions:**\n1. Incorporating Gaussian Mixture Models (GMMs) into the analytical framework is an intriguing possibility, as explicit expressions for the score function are available in this setting. We anticipate that deriving optimal denoiser expressions for GMMs is feasible and leave this to future work. Notably, formulating an optimization problem similar to ours requires the sampling expressions to be jointly diagonalizable. Additional possible research directions include deriving analytical solutions for synthesis conditioned on degraded signals and designing loss functions which emphasize desired solution characteristics (See Appendix k3).\n\n[9] Chen, Minshuo, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. \"Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data.\" In International Conference on Machine Learning, pp. 4672-4712. PMLR, 2023.\n\n[10] Yang, Ruofeng, Bo Jiang, Cheng Chen, Baoxiang Wang, and Shuai Li. \"Few-shot diffusion models escape the curse of dimensionality.\" Advances in Neural Information Processing Systems 37 (2024): 68528-68558."}]

Author rebuttal:

Meta-review: Although stemming from insights based on the Gaussianity assumption in the theoretical analysis, the derived spectral schedules provide a more principled scheduler design through solving an optimization problem, compared to heuristic designs in the literature, and outperform baselines in FID across various datasets. These insights may be useful for broader diffusion model scheduler design. Including additional experiments on larger-scale datasets in a revision would further demonstrate the effectiveness of the proposed method, which I highly recommend. Overall, I recommend **acceptance**.
Decision: Accept (poster)