Constant Rate Scheduling: A General Framework for Optimizing Diffusion Noise Schedule via Distributional Change
Authors: Shuntaro Okada
Kenji Doi
Ryota Yoshihashi
Hirokatsu Kataoka
Tomohiro Tanaka
Venue: Accepted by TMLR ()

Abstract: We propose a general framework for optimizing noise schedules in diffusion models, applicable to both training and sampling.
Our method enforces a constant rate of change in the probability distribution of diffused data throughout the diffusion process,
where the rate of change is quantified using a user-defined discrepancy measure.
We introduce three such measures, which can be flexibly selected or combined depending on the domain and model architecture.
While our framework is inspired by theoretical insights, we do not aim to provide a complete theoretical justification of how distributional change affects sample quality.
Instead, we focus on establishing a general-purpose scheduling framework and validating its empirical effectiveness.
Through extensive experiments, we demonstrate that our approach consistently improves the performance of both pixel-space and latent-space diffusion models,  
across various datasets, samplers, and a wide range of number of function evaluations from 5 to 250.
In particular, when applied to both training and sampling schedules, our method achieves a state-of-the-art FID score of 2.03 on LSUN Horse 256$\times$256, without compromising mode coverage.

Published in Transactions on Machine Learning Research (01/2026)
Constant Rate Scheduling: A General Framework for Opti- mizing Diffusion Noise Schedule via Distributional Change
Shuntaro Okada; Kenji Doi, Ryota Yoshihashi, Hirokatsu Kataoka, Tomohiro Tanaka LY Corporation, Japan
Reviewed on OpenReview: https: // opw.reveeeer......... net/ fttt/form? id Pjg6kdwWB
Abstract
We propose a general framework for optimizing noise schedules in diffusion models, appli- cable to both training and sampling. Our method enforces a constant rate of change the probability distribution of diffused data throughout the diffusion process, where the rate of change is quantified using user-defined discrepancy measure. We introduce three such measures, which can be flexibly selected combined depending the domain and model architccture. While our framework is inspired by theoretical insights, we do not aim to pro- vide a complete theoretical justification of how distributional change alfects sample quality. Instead, we focus on establishing a ceneral-purpose scheduling framework and validating its empirical effectiveness. Through extensive experiments, we demonstrate that our approach consistently improves the performance of both pixel-space and latent-space diffusion models, across various datasets, samplers, and wide range of number of function evaluations from 5 250. In particular, when applied to both training and sampling schedules, our method achieves stato-of-the- FID score of 2.03 on LSUN Horse x56x256, without compromising mode coverage.
1 Introduction
Diffusion models are a class of probabilistic generative models that have attracted significant attention due to their ability to generate high-quality samples across a variety of domains, including image generation (Saharia et al. 2022b; Nichol al. 2022b Podell et al. 2024a), image super-resolution (Saharia el al.] 2023
Doi et al. 2024), video synthesis (Ho et al. 2022b a), and audio generation (Kong et al. 2021 Popov et al. 2021
Dilfusion models were originally rntrede in Sohl-Dickstein et al. 2015 and denoising diffusion proba- bilistic models (DDPMs) (Ho et al.) 2020) demonstrated that diffusion models can achieve state-of-the-art performance in image generation. typical diffusson model consists of two Markovian processes: a forward process and a reverse process. The forward process gradually adds Gaussian noise to the input data, trans- forming its distribution toward Gaussian. The reverse process aims to reconstruct data from Gaussian noise by tracing the forward process in the reverse direction.
The noise levels applied at each step of these processes treated hyperparameters and are collectively referred to as the noise schedule. The noise schedule has critical impact on both the quality and efficiency of training and sampling (Chen 2023). Importantly, the schedules used during training and sampling do not have to be identical, and recent studies have shown that using different schedules for training and sampling
can improve performance (Karras et al. 2022 Kingma Gao 2023 Hang et al. 2025). However, designing effective noise schedules remains a non-trivial and largely empirical process.
In this work, propose general framework for optimizing both training and sampling schedules by enforcing constant rate of change in the probability distribution of diffused data throughout the diffusion
"Correspondence to: shunkadd@@lyoorp.oo..p

---

Published in Transactions on Machine Learning Research (01/2026)
Reverse process
Forward process
2 aza + v a²e E~ 10
370
Noise level should be slowly Noise level should be slowly
Noise level can be rapidly changed
changed for mode coverage
changed for sample fidelity
0.6
0.8
0.97
0.99
stal
Diffused data distribution
Gaussian noise
A same Gaussian noise
Two modes start to emerge
Three modes start to emerge
Distinct modes corresponding to three data
Figure 1: Toy example of diffused data distributions with three data points in one-dimensional data space, Probability distributions barely change when S 0.6, and we rapidly change noise level. Three modes corresponding to data points emerge when 0.6 S a S 0.97, and noise level should be changed slowly for mode coverage. Three modes become distinct when a Z 0.97, requiring careful control of noise level for sample fidelity.
process. We refer to this framework as constant rate scheduling (CRS). Since the reverse process is expected to accurately trace the forward process, the key motivation is to determine noise schedules that can enhance the traceability of the forward process.
Figure I prresens a toy example of diffussd data distributions with three data points in one-dimensional data space (sce Appendix A for details). Here, a represents the decay rate of data at each step of the diffusion process. In this example, the diffusion process can be divided into three regions: 1) where distributions barely change and noise can be decreased rapidly (a S 0.6), 2) where modes corresponding to each data point emerge and require fine control of noise to avoid mode dropping (0.6 2 a & 0.97), and 3) where mode peaks become sharp and noise must be reduced gradually for better fidelity (a 2 0.97). This example illustrates that the distributional dynamics of the diffusion process are highly non-uniform. We hypothesize that the probability
change represents the traceability of the diffusion process, and controlling the rate of distributional change can improve the quality and stability of the generative process.
Recent advances in generative modeling have also emphasized the importance of well-behaved distributional evolution. For example, flow matching (Lipman et al. 2023) and rectified flow transformers (Esser et al. 2024) formulate goneration as a deterministic flow between data and noise distributions, highlighting that stable distributional trajectories can lead to improved sample quality. Although these approaches differ from diffusion models in their formulation, they share the high-level motivation of designing generative processes with controlled and stable distributional dynamics.
CRS offers a unified and flexible framework for designing noise schedules, with the following key properties: 1) it supports both training and sampling schedules, 2) it allows the use of arbitrary discrepancy measures to quantify distributional change, enabling domain-specifie tailoring and independent schedules for training and sampling, and 3) many existing schedules can be viewed as special cases of CRS under different discrepancy measures.
Our contributions are summarized as follows:
We propose a general framework for optimizing noise schedules in diffusion models by enforcing a constant rate of distributional change throughout the diffusion process. Our framework supports both training and sampling schedules and allows the of any user-specified discrepancy measure.
2

---

Published in Transactions on Machine Learning Research (01/2026)
2. We introduce three practical discrepancy measures for quantifying distributional change, each cap- turing different aspects of the diffusion process. These measures can be flexibly selected or combined, depending on the domain or model type (e.g., pixel-space or latent-space diffusion models).
3. While theoretically motivated, this work does not aim to provide a full formal analysis of the link between distributional change and sample quality, Instead, we focus empirically validating the effectiveness of CRS across diverse experimental settings. Through extensive experiments, we demon- strate that CRS consistently improves the performance of both pixel-space and latent-space diffusion models across various datasets, samplers, and numbers of function evaluations (NFEs), ranging from 5 to 250.
The remainder of this paper is organized as follows: Sec. DDPMs. Sec. T presents our proposed framework. Sec. 5 provides the experimental results. Sec. G discusses limitations, and Sec. i concludes the paper.
2 reeicwws relatod work. Soc. briefly introduces
2
Related work
The performance of generative models is typically evaluated along three axes: mode coverage. Diffusion models are known to excel in terms of fidclity and mode coverage, but suffer from slow sampling speed due to the iterative nature of the reverse process (Xiao et al. 2022). To address this limitation, a wide range of acceleration strategies have been proposed. These approaches can be broadly categorized into five groups:
sampling speed, fidelity, and
Conditional generation:
This strategy improves sampling speed by reducing sample diversity through
2021 Ho & Salimans) 2021 Rombach et al. 2022 Saharia et al.) 2022a]: Nichol et al. 2022a Podell et al. 2024b), which guides the model to generate specific outputs and avoid unnocossary exploration.
conditioning (e.g., class labels text prompts) (Dhariwal & Nichol
2022 employ pre-trained au- toencoders to compress data representations, allowing the diffusion process to operate in a lower-dimensional latent space. This improves both training and sampling efficiency.
Dimensionality reduction: Latent diffusion models (LDMss) Rombach et al.
Samplers: These methods improve the update rule of diffused data for the reverse process. Denoising diffu- sion implicit models (DDIMs) (Song et al. 2021a introduce a non-Markovian process to generalize DSPMs and propose deterministic sampler. In continuous-line diffusion models, the forward and reverse processes are formulated as stochastic differential equations (SDEs) and ordinary differential equations (ODEs) (Song
ict al.
2021b). Numerous studics have developed efficient solvers tailored to these formulations JJoicocur-
Martineau et al.
2021 Liu el. al. 2022: Lu el al. 2022a Zhang & Chen 2023
Zhao el al.) 2023 Zheng
et al.
2023b Zhou al.) 2024).
Noisc schedules:
The choice of noise schodule significantly affects training and sampling efficiency. Early schedules, such as linear (Ho et al. 2020 and cosine (Nichol & Dhariwal 2021), were chosen heuristically,
More recently, principled approaches for optimizing sampling schedules have been proposed (Chen et al. 2024 Sabour et al.) 2024 Williams et al.) 2024 Park et al. 2025 Tong et al. 2025). Among them, Align Your Steps (AYS) (Sabour et al. 2024) determines a noise schedule to minimize the discretization errors measured using Kulbbeck-Leibler divergence. One of the discrepancy measures in our approach can be interpreted as simplified variant of Kullback-Leibler divergence upper bound (KLUB) introducod in AYS, and thus CRS generalizes AYS and related methods as special cases.
In contrast, elucidating diffusion models (EDMs) KKarras et al. 2022 and variational diffusion models- (VDM++) (Kingma & Gao. 2023) have proposed optimization of the training schedule. These works also highlight that using different schedules for training and sampling can improve performance. However, most prior work has focused predominantly on sampling schedules, and optimization of training schedules remains relatively underexplored. CRS is designed optimize both schodules within unified framework, supporting the use of arbitrary and even composite discrepancy measures. This flexibility allows practitioners adapt scheduling strategies based empirical behavior and task-specific requirements.
3

---

Published in Transactions on Machine Learning Research (01/2026)
Distillation:
Distillation-basel approaches another major direction for fast sampling (Luhman & Luh-
man 2021 Salimans & Ho 2022 Song et. al. 2023 Yin et al. 2024b a Frankel et al. 2025). These met
compress multi-step diffusion into fewer steps. CRS is orthogonal to this line of work and can provide better teacher models for distillation, as it can improve training schedules.
3
Background
We briefly introduce DSPMs Ho et al. 2020
In the forward process, Gaussian noise gradually added to the data through a sequence of timesteps, transforming the data distribution into an isotropic Gaussian. This process typically modeled a Markov chain that starts from a data sample 2o q(xo):
((p:)) II ( 10)1)) g(2r0); where ((<1(<=-1) = - a1,
(1)
Here, denotes the number of timesteps, 8 is a docay factor controlling the signal strength, & represents the added noise level, and g(vr)) is the data distribution.
It is straightforward show that (()) takes the following form:
g(,, an) ( 11
where the coefficients satisfy the recursive relations: = 3,a=-1 and of = + This formulation allows sampling of Is from To in a single step. In this work, we paramcterize the forward process using (on,, instead of (Be, 6t), and adopt a variance-preserving formulation which both of + o = and B7 o = hold. The sequence {at)== defines the noise schedule, which controls how rapidly the signal component of the data decays during diffusion. Our goal is to optimize this noise schedule to enable more effective and efficient training and sampling in diffusion models.
In the reverse process, data samples are generated from Gaussian noise by tracing the forward process in the reverse direction. This process also modeled as a Markov chain and is defined as:
(o [I po a (100--111))) p(arr) where p(æT) N(rr: I).
Here, 0 denotes the learnable parameters of the diffusion model. In the limit of T - oo, the reverse transitions can be well approximated by the Gaussian distributions [Feller 1949h:
po( (1 (0, (),
(4)
DDPMs employ noise prediction model c0( t), which predicts the noise added at timestep The predicted mean pa( t) is given by:
H 1 (ax t)
1
82 ae
co(
(5)
BB
The noise prediction model trained by maximizing simplified version of the variational lower bound, which yields practical and stable learning objective.
4 General Approach for Optimizing Noise Schedule
We explain our motivation and present CRS.
4

---

Published in Transactions on Machine Learning Research (01/2026)
4.1 Motivation
As stated in Sec. when T is sulficiently large, it is good approxination to assume that the reverse transitions pe - follow a Gaussian distribution, as expressed in Eq. ( This implies that the distri- butional change between adjacent timesteps must be small enough to be well approximated by a Gaussian distribution. In diffusion models, sample quality depends on how faithfully the reverse process can trace the forward process. Large distributional changes can lead to poor approximation, resulting in artifacts or mode dropping. This observation motivates us minimize the maximum distributional change during the diffusion process, in order to maintain the validity of Eq. [4 throughout the trajectory.
The importance of controlling distributional change can also be understood from the perspective of noise conditional score networks (NCSNs) (Song & Ermon 2019 2020 Song et al. 2021b), which are the conceptual predecessors of diffusion models. According to the manifold hypothesis, natural data typically lie on a low-dimensional manifold embedded in high-dimensional space. This leads to two main challenges when gencrating samples using only the score function Va log q(x:) from a randomly initialized point 2nd (Song & Ermon 2019): first, Brnd is likely to fall in regions where ((Zrna) < 1, making difficult estimate accurate gradients; second, even if the score is correctly estimated, it tends to be small in such regions, leading to slow mixing.
To address these challenges, NCSNs interpolate wetween the data distribution and a Gaussian distribution, whose support spans the entire space. the distributional change between adjacent steps kept small during this interpolation, each reverse step is likely to remain within regions of high probability density. This improves the reliability of local updates and enhances overall sample quality.
These theoretical motivations form the basis for our proposed framework: to design noise schedules that ensure constant rate of change in the distribution of the diffused data across timesteps.
4.2 Constant Rate Scheduling
As described above, goal is to optimize noise schedules by minimizing the maximum distributional change between adjacent timesteps in the diffusion process. Let D(t, At) denote a discrepancy measure between the distributions of diffused data q(xe) and ((*++A)). Our criterion can be formulated as:
argmin
(max D + D()) a(t)
(6)
The optimal schedule att) must satisfy the following condition:
+ = D(t, At) D(t,t)
At aD(t.t)
At const.,
(7)
At
Ot' 1'=
where we used D(I, 0.
To explicitly reflect the dependence on the noise schedule, we rewrite Dit,t') as D(a(t),a(t))) = D(a,a)) where att) is the noise schedule to be optimized. This leads to:
D(t, + At) ~ (
da(()) At = const.,
At v(a) e()) 1==
(8)
Oe
dt
where v(a:) represents the rate of change in the probability distribution of the diffused data.
Therefore, the noise schedule must satisfy da(t) v(a)-1. The proportionality constant is computed from the boundary conditions a0))= Omax and a<() Omins yielding:
da(() = -,, where C= c v(a) do.
C'iaa
(9)
dt
Crann
Here, we introduce a hyperparameter ş > 0 to control the dependence of a+t) on v(a). Larger values of g allocate more timesteps regions where v(a) is large, i.c., where the distribution changes rapidly.
The noise schedule optimization using CRS consists of the following steps:
5

---

Published in Transactions on Machine Learning Research (01/2026)
1. Choose discrepancy measure to quantify distributional change.
Compute v(a:) using the selected discrepancy mcasure.
3. Solve Eq.
9
obtain a(t).
4. Apply the resulting schedule as follows:
For training, use a(I) directly.
For sampling, discretize the schedule uniformly in time: {a(l/T)(u= = 1,, where T is the number of sampling steps.
We propose three discrepancy measures for computing v(a), which are detailed in Section 4.3
The derivation of Eqs. 7 and 8 relies on finst-order Taylor expansion with respect to At, which is accurate when the NFE is sufficiently large. However, when optimizing the sampling schedule at small NFEs, At becomes large and the approximation error may longer be negligible. Empirically, we observe that CRS with & = 1 remains effective for moderate NFEs, while the importance of tuning increases the NFE decreases (see Table 27 in Appendix F.2). We attribute this increased sensitivity to the growing discretization error of the first-order expansion.
For small NFEs, the optimal value of & tends to be greater than 1, resulting in more timesteps being allocated to regions where v(a) large. Since discretization error is expected to be more pronounced regions with larger v(a), it intuitively reasonable that values of & become optimal when the NFE is small.
Although introducing & partially compensates for the discretization error, it does not fully resolve the mis- match between the continuous-time formulation and the discrete-time implementation. A fully principled solution would require directly optimizing the sampling schedule in the discrete-time domain, without relying on the continuous-time assumption underlying Eq. We consider this an important direction for future work.
Similar to NCSNs inspired by simulated annealing (Kirkpatrick et al. 1983), our formulation draws additional motivation from the adiabatie theorem (Morita & Nishimori] 2008) in quantum annealing (Kadowaki & Nishimori 1998). Quantum annealing solves combinatorial optimization problems by interpolating between target cost function and an initial one that has a trivial optimum. The optimization starts from the known optimum of the initial function and aims to trace the instanlaneous optimums along the interpolation path. It crucial to accurately trace the optimum of the instantaneous cost function, and the adiabatic theorem provides the traceability of the optimum at each step. Let att) be the weight of the target cost function and 1 1- a(t) be the weight of the initial cost function. In this setting, the inverse traccability expressod in the form of da(t) v(a), which takes the same form as Eq. (8). Prior work has shown that minimizing dt the maximum value of this inverse traccability yields the optimal att) (Roland & Cerf 2002). that ensures consistent traceability throughout the interpolation. In CRS, we regard the rate bilit- iibbuttonal change as the inverse traceability and minimize its maximum value to derive an optimal noise schedule.
4.3 Discrepancy Measures
Any discrepancy measure is applicable to CRS. We introduce three discrepancy measures.
4.3.1
Discrepancy measure based on FID
FID is widely used metric for evaluating sample fidelity in the field of image generation. It computes the Fréchet distance (Heusel et al. 2017) between the feature distributions of real and generated images under Gaussian assumption, where the features are extracted using the Inception- V3 network (Szegedy et al. 2016 trained on ImageNet classification.
When used as discrepancy mcasure our framework, v(a) tends to take large values in the region where sample fidelity significantly degrades (i.e., a ~ 1). As result, CRS allocates more timesteps to this region, thereby improving sample fidelity.
6

---

Published in Transactions on Machine Learning Research (01/2026)
However, limitation of the FID-based measure is that it fails to accurately capture distributional changes in high-noise regions (ax 0), since the feature extractor is not trained on heavily diffuusd images and may not provide meaningful representations in that regime. Therefore, while the FID-based measure is effective for capturing fidelity-related changes in low-noisc regions, may benefit from being complemented by other discrepancy measures that can better caplure high-noise dynamics.
Once the forward process is defined, v(a) can be computed by simulating the forward diffusion using the training dataset (see Algorithm in Appendix C.I Since this procedure is independent of model parameters, it only needs to be executed once prior to training the diffusion model.
The computation time of v(a) is proportional to the number of training images, which may be prohibitively expensive for large-scale training datasets. To address the scalability issue for large-scale datasels, we introduce a practical alternative in Sec. 5.7
We refer to v(a) computed using FID VFID(a).
4.3.2 Discrepancy measure based on data prediction
We introduce discrepancy measure that leverages a trained diffusion model as follows:
aD o!) Oa'
(a(aa)
where Di Eæ, .e rqlas and) (1x a) - xol a [1 (10)
Here, ( Fe)) is the joint distribution of diffused data different noise levels, and 2 20 ar) denotes the data prediction, which is related to the noise prediction e a) via (x, a) + (
This measure is designed to allocate more computational resources to regions where the data prediction changes rapidly with respect to a. As shown in Appendix B Uz (a) can be interpreted as a simplified variant of KLUB, introduced in AYS. Importantly, vz(a) can be used not only to determine the sampling schedule after training, but also to adaptively optimize the training schedule during learning.
We further interpret U2 (a) as measuring the probabilitydddisribbttoaaa change of the diffused data. According to prior work (Ambrogioni 2024), the data prediction 20(x,, a) can be viewed as weighted average over training data points Zo, where the weighting depends on the noise level When are approximately uniform across the dataset. As a + 1, only few data points contribute significantly, indicating sharper and more specific representations. The change in these implicit weights reflects how sample trajectories diverge as noise decreases, and thus (a)a) provides a proxy for the rate of distributional change.
O, the weights
The pseudocode for optimizing training and sampling schedules is provided in Appendix C.2 For sampling schedule optimization, the computation time of Ux (a) is negligible compared to the overall training time. In contrast, training schedule optimization requires updating D a') during training as the model parameters evolve, which increases the total training time by approximately 20-30%. However, our experiments show that the schedule typically stabilizes early in training. Therefore, fixing the schedule after a few epochs or reducing the update frequency of D2 a') may mitigate the computational cost without compromising performance. Developing a more efficient implementation remains an important direction for future work.
4.3.3 Discrepancy measure based on noise prediction
In the experiments presented in Sec. 5
we trained the noise prediction model. In this setting, it is also effective to use the noise predictions themselves to quantify distributional change, leading to the following definition: (a)
aD (ax a²) Oo
where
De a') Eg, R. aele 20.)) [IE0((a,)) - co al [0) (11)
a'=a
We refer to CRS using (F) (a), and (a) as CRS-UFID. CRS- and CRS-r,, respectively, depending the choice of discrepancy measure used to compute v(a)..
7

---

Published in Transactions on Machine Learning Research (01/2026)
4.4 Combining Multiple Discrepancy Measures
Each discrepancy measure has its own strengths and weaknesses. For example, CRS-VFID effectively captures distributional changes relevant to sample fidelity, but fails to represent changes accurately in high-noise regions (a = 0), as the feature extractor is not trained on diffused images. On the other hand, CRS-U, captures distributional change even in high-noise regions, but it is not necessarily optimal for improving sample fidelity. Therefore, it is considered effective to complementarily combining multiple discrepancy measures for optimizing noise schedules.
We introduce the following strategy for combining multiple discrepancy measures:
M Σ
1 Cm Um (a)fm and Cm
f Mame
v(a)=
Wm (a), where (a)
Um (m
(12)
Omiu
where M the number of discrepancy measures, Um is the rale of change computed using the m-th measure, Wm is its corresponding weight (satisfying Em=1 Wm = 1), and Em controls the emphasis on um (a). Since the value ranges of different discrepancy measures can vary significantly, we normalize each Vmm Vm (a) as in Eq. [12p before combining them.
This formulation also enables conventional noise schedules be interpreted and incorporated within the CRS framework. By converting given noise schedule al)) into its corresponding form of v(a;), can treat it implicit discrepancy measure. For example, if (t) cos (2)) then:
v(a)= - (((aaa)
(da(t)
n
sin -1
(13)
VI-a²
This observation implies that any noise schedule can be viewed the optimization result of CRS with an associated (possibly implicit) discrepancy measure, thus providing unified perspective the design of noise schedules.
While combining multiple discrepancy measures increases the flexibility of CRS, it also introduces additional hyperparameters. In our experiments, we found that tuning these hyperparameters is required only when optimizing sampling schedules in pixel-space diffusion models. In all other cases training schedules for both pixel-space and latent-space models, and sampling schodules for latent-space models CRS-u, with & e= consistently performed well and can serve as default setting.
In the next section, we show that CRS-e, + UFID, which combines (a) and (F), is effective for op- timizing the sampling schedule of pixcl-space diffusion models. CRS-E, + VFDD has four hyperparameters: Wa, WFID, &x, and {Fm- The procedure for tuning these hyperparameters is described in Appendix D and the hyperparameter settings used in all experiments are summarized in Table [2] For configurations with NFE S 50, Table 12 shows that W, = WFID 0.5 is optimal in most cases. These obscrvations suggest that, in practical use, generally sufficient tune only & and FFID.
5 Experiments
We experimentally demonstrate that CRS broadly improves the performance of diffusion models.
5.1
Experimental Settings
Below, we provide brief overview of the components and settings used in our experiments.
Dataset: We use six image datascts: LSUN (Church, Bedroom, Horse) Yu et al. 2016 ImageNet (Deng et al. 2009), FFHQ (Karras et al. 2019), and CIFARIO (Krizhevsky 2009). LSUN, FFHQ, and CIFARIO are used for unconditional image generation, while ImageNet is used for class-conditional image generation.
Noise Prediction Model: We adopt the U-Net architecture introduced in the ablated diffusion model (ADM) (Dhariwal & Nichol 2021) as the noise prodiction model. Hyperparameter settings are listed in
Table 13 Appendix
F
8

---

Published in Transactions on Machine Learning Research (01/2026)
Training Schedule: We compare four training noise schedules: linear Ho et al.) 2020 shifted cosine
(Hoogeboom et al. 2023). the adaptive schedule proposed in VDM++ (Kingma & Gao 2023 and our proposed CRS. The CRS hyperparameter & is fixed to 1 in all experiments.
Loss Function: Except for the adaptive schedule in VDM++, we use the simplified loss function from DDPMs:
L Ex, D 1.1. a())) - =1i
(14)
where x a(t)ao + V1 (t) D denotes the training dataset, and (t)) is the training schedule. When using the adaptive schedule from VDM++, the following loss function is used:
(()) p(A) | /eimmiiiii
2
L E Bao P.o
(15)
where x =1=1og log ( is the log signal-to-noise ratio (log-SNR), (A) is the weighting function, and p())
de
denotes the noise schedule. We adopt the EDM monotonic function as w(), and update p(A) at each iteration to approximately satisfy:
P(A) x B. ( Eeo (z, /iigmoid()) -
-
(16)
Sampling Schedule: We evaluate four sampling schedules: linear, shifted cosine, EDM (Karras et al, 2022), and CRS. For pixel-space diffusion models, we additionally evaluate CRS-e, + UFID, which combines (a) and (a) to leverage their complementary strengths, as discussed in Sec. 5.5 Unless otherwise stated, the hyperparameter & for CRS-w2, CRS-r,, and CRS-FFID is fixed to g 1. For the hyperparameter tuning procedure of CRS U, + UFID and the exact values used in our experiments, please refer to Appendix D Following prior work (Karras al. 2022 Kingma & Gao 2023), we allow training and sampling schedules to differ. We thus evaluate all combinations of training and sampling schedules. Details of the conventional schedules are summarized in Table 16
(Appendix E
Sampling Methods: To assess the generality across samplers, we employ two stochastic samplers and three deterministic samplers. The stochastic samplers include Stochastic DDIM (r)=1) (Song et al.) 2021a) and IDF-DDM-Solva+++MM (Lu et al. 2022b). The deterministic samplers include DDIM (Song et al.
2021a), +2M- (Lu et al.] 2002bb), and pseudo numerical methods (PNDM) (Liu al. 2022 At low NFF regimes, also evaluate improved PNDM (iPNDM) (Zhang & Chen 2023 and the unified predictor-corrector framework (UniPC) (Zhao et al.
2023).
Evaluation Metrics: We use four evaluation metrics: FID HHuueel et al. 2017). spatial FID (sFID) Nash
et al. 2021). and improved precision and recall (Sajjadi et al. 2018 Kyn känniiimi et al. 2019 FID, sFID, and precision measure sample fidelity, while recall quantifies mode coverage.
5.2 Results on Pixel-Space Diffusion Models
We trained continuous-time diffusion models on LSUN Horse 256x256 and LSUN Bedroom 256x256 for unconditional image generation. Detailed experimental results are provided in Appendix F.I
LSUN Horse 256x256: The evaluation results are summarized in Table I Rows shaded in gray represent results obtained under different training schedules. Although sample fidelity and mode coverage are generally considered to be in trade-off relationship, CRS-s, achieved the highest scores across all evaluation metrics. This indicates that CRS-U, does not merely balance fidelity and diversity but leads to an overall improvement in sample quality.
The unshaded rows represent the dependence on sampling schedules. CRS-U, + UFID outperformed all basclines across all metries, indicating a fundamental improvement in generation quality. Notably, our model achieved an FID score of 2.03, surpassing the previous state-of-the-a score of 2.11 reported in Kumari al. 2022
9

---

Published in Transactions on Machine Learning Research (01/2026)
Table 1: Performance of pixel-space diffusion models on LSUN Horse x56x256 with NFE 250, using SDE-DPM-Solve++
the sampler. Gray-shaded rows indicate variations in the training schedule, while unshaded rows correspond to different sampling schedules. Bold values highlight the best performance for each metric under varying training or sampling schedules.
Training
Sampling
Metrics
schedule
schedule
FID +
sFID +
Precision t
Recall t
Linear
Linear
2.90
6.82
0.66
0.56
Shifted cosine
2.95
6.43
0.67
0.56
VDM++
3.82
7.40
0.66
0.54
CRS-v,
2.91
6.56
0.66
0.56
CRS-
2.73
6.40
0.67
0.56
CRS-
Linear
2.73
6.40
0.67
0.56
Shifted cosine
3.13
7.09
0.67
0.54
EDM
2.09
6.16
0.69
0.56
CRS-U:
2.87
6.71
0.66
0.56
CRS-
5.46
8.34
0.62
0.52
CRS- + UFID
2.03
6.06
0.69
0.56
Table 2:
FID scores for LSUN Bedroom 256x256 in pixel-space diffusion models under low NFE conditions.
Our results are compared with prior work from Chen et al. 2024). Bold entrics indicate the best FID achieved at each NFE setting.
Model Sampler
Sampling schedule
NFE = 5 NFE = 10
EDM
UniPC (Zhao et al. 2023)
EDM
23.34
5.75
AMED-SoPt Solver [Zhou et al
2024
5.38
iPNDM Zhang & Chen 2023
GITS Chen et al. 2024)
15.85
5.28
Ours
DPM-Solver++(2MN)
EDM
23.72
4.73
CRS-Ux UFID
14.02
4.88
UniPC
EDM
82.52
4.94
CRS- UFID
21.42
3.30
LSUN Bedroom 256x256: We further validated the effectiveness of CRS low-NFE regimes by com- paring it with strong baselines reported in GITS (Chen al. 2024). We trained continuous-ting diffusion models with CRS-u, used for optimizing the training schedule. Table 2 lists the results. At NFF 5 and NFE = 10, the best FID scores were achieved by using CRS-v, and CRS-U, UFID for optimizing training and sampling schedules, respectively.
As shown in Table 20 in Appendix F.] we also tuned the parameter in the EDM sampling schedule. Even with the optimal setting, CRS- + UPID consistently outperformed EDM.
5.3 Results on Latent-Space SSpace Diffusion Models
We trained latent-space diffusion models using the pre-trained autoencoder from LDMs (Rombach et al, 2022). In our experiments, we adopted the VQ-regularized variant for all datasets, which is the configura- tion primarily used in the publicly available pretrained models on GitHub (https://gi thub. com com/CompVis/ difent-ifffsion) Although downsampling factor of = 8 is also commonly used, we set f (VQ-f4), it is expected to yield better FID scores according Table 10 in the LDMs paper. LSUN Church and LSUN Bedroom were used for unconditional image generation, and ImageNet was used for class-conditional image generation. All images were resized to 256x256 and encoded into 64x64 latent representations. Here, we present the evaluation results on LSUN Church 256x256. Detailed results are provided in Appendix
LSUN Church 256 256: Table summarizes the results. Rows shaded in gray show the effect of different training schedules, where CRS-u, achieved the best performance on all metrics except sFID. Unshaded rows show the dependence on the sampling schedule, and again, CRS-u, achieved the best performance on all metrics except. precision.
10

---

Published in Transactions on Machine Learning Research (01/2026)
Table 3: Performance of latent-space diffusion models on LSUN Church 256x256 with NFE 30, using DPM-Solver+ the sampler. Gray-shaded rows indicate variations in the training schedule, while unshaded rows correspond to different sampling schedules. Bold values highlight the best performance for each metric under varying training or sampling schedules.
Training Sampling
Metrics
schedule schedule
FID
sFID +
Precision f
Recall
Linear
Linear
3.82
10.28
0.60
0.58
VDM++
4.00
10.07
0.58
0.59
CRS-
3.78
10.51
0.60
0.58
CRS-U2
3.87
10.30
0.60
0.59
CRS-V,
3.69
10.23
0.60
0.59
CRS-U,
Lincar
3.69
10.23
0.60
0.59
EDM
4.31
11.31
0.60
0.56
CRS-UFID
3.98
9.61
0.58
0.60
CRS-U,
3.63
9.91
0.59
0.60
CRS-
3.59
9.51
0.59
0.60
Table 4: FID scores on CIFAR10 32x32 and FFHQ 64x64 evaluated using the pretrained model provided EDM (Karras al. 2022). Bold numbers indicate the best sampling schedule for each combination of dataset, sampler, and NFE. For reference, the FID scores of AYS (Sabour al. 2024 and LD3 (Tong al.
2025) are taken directly from their original papers.
Sampling
CIFAR10 32x32
FFHQ 64x64
Sampler
schedule
NFE = 10
NFE = 20
NFE = 10
NFE 20
(2M)
AYS
2.98
2.10
5.43
3.29
GITS
4.03
2.32
5.51
3.81
LD3
3.38
2.36
3.98
2.89
CRS-u, + UFID
3.92
2.27
5.35
3.10
UniPC(3M)
GITS
3.50
1.99
4.52
2.71
LD3
2.84
3.27
CRS-U, + UPID
2.52
1.99
5.23
2.64
iPNDM(3M)
GITS
2.73
2.08
3.96
3.09
LD3
2.38
3.25
CRS-U2 + UFID
2.87
2.11
4.07
2.75
iPNDM(4M)
GITS
2.49
2.02
3.62
3.00
CRS-U, + UPID
2.46
2.02
3.45
2.53
These results indicate that CRS is also effective in latent-space diffusion models. Although CRS-U, performs best in most settings, observe that depending the sampler and NFEs, CRS-t, can be preferable for sampling.
5.4 Comparison with Recent Sampling Schedules
Using the pretrained models released with EDM Karras et al.] 2022 we compare the performance of CRS sampling schedule against AYS (Sabour et al. 2024), GITS CChn et al. 2024),, and learning to discretize denoising diffusion ODEs (LD3) (Tong et al. 2025). All evaluations in this subsection are conducted using the official repository provided in the GITS paper, ensuring full consistency with their experimental protocol.
It is important to note that EDM adopts a variance-exploding formulation, whereas all our previous experiments have been conducted under the varianco- proserving setting. However, CRS can still be applied variance-exploding processes by converting between a and the EDM noise level σDDM via the log- SNR, shown below:
a² I-a²
log SNR log
log
(17)
I o EDM
11

---

Published in Transactions on Machine Learning Research (01/2026)
We also note that, whereas all earlier experiments in this paper used the noise-prediction parameterization, EDM adopts the data-prediction parameterization with preconditioning.
The evaluation results are presented in Table ! The optimal sampling schedule varies across datasets, sam plers, and NFE configurations. On average, LD3 achieves the highest performance, while CRS consistently outperforms GITS and can therefore be regarded as the second-best sampling schedule overall. Although CRS underperforms LD3 as a sampling schedule, a key advantage of CRS that it also provides a uni- fied framework for optimizing the training schedule, which LD3 does not address. Further improving the performance of CRS will likely require identifying more effective discrepancy measures, and exploring such alternatives is an important direction for future work.
For completeness and fairness, we note that CRS-t, + OPiD requires both the computation of (Fi(() and hyperparameter tuning, and the time needed to optimize the sampling schedule is substantially larger than that of GITS or LD3. However, sampling-schodull optimization is performed only once after training the diffusion model. Therefore, as long as the tuning cost. remains small relative to the total training time, we believe that performance under the best hyperparameter configuration is the most meaningful criterion for comparison.
5.5 Comparison of Noise Schedules
Figure 2 shows examples of CRS-based noise schedules in pixel-space diffusion models, computed using different discrepancy measures. For CRS-E, + UPID, the hyperparameters were set to wr = WFID = 0.5 and &. = EFID = 1.0.
Since (a) tends to be large regions where sample fidelity degrades (i.e., near a = 1), CRS-VFID allocates more timesteps to these regions. In contrast, CRS-v, allocates more timesteps to regions with smaller a. This is due to the formulation of the data prediction 28(2;,) = ==0=ş(0.0) where a appears in the denominator, making the data prediction more sensitive to small changes in a when a small. Thus, CRS-UFID and CRS-U, exhibit complementary behaviors, and combining both yields an effective and balanced sampling schedule.
Furthermore, we observed that the noise schedule derived from CRS-t, can be approximately reproduced by appropriately tuning the hyperparameters of CRS-v, + UFID- Therefore, in practice vz(a) and (Fi((a) serve as the two principal components for constructing robust sampling schedules.
Figure 3 illustrates CRS-basod noise schodules for latent-space diffusson models. The optimized schodules differ notably between pixel-space and latent-space models. In latent-space diffusion, fewer timesteps are allocated to the region where 1. This difference can be altributed to the nature of latent-space diffusion models: perceptual expression is largely reconstructed by the autoencoder, and the autoencoder is robust to minor deviations in the generated latent variables. Consequently, precise control near a = 1 is less critical. In contrast, pixel-space models directly generate the final image, so the schedule must allocate sufficient timesteps near a ~ to preserve high-fidelity details.
Bocause all discrepancy measures result in similar noise schodules for latent-space models, we did not evaluate combinations of multiple discrepancy measures.
5.6 Computational Overhead of CRS
CRS requires computing v(a). Below, we report the computational overhead in scenarios where the cost is expected to be most significant.
Cost of computing (r) Although (ri needs to be computed only once before training the diffusion model, its computation requires processing the entire training set to compute feature statistios, and therefore the cost scales linearly with the number of images.
Table 5 reports both the cost relative to a single training epoch and the cost relative to the total training time. Because the computation of (a) and the duration of one training epoch both scale proportionally with dataset size, the cost relative to training epoch remains nearly constant across datasets for both
12

---

Published in Transactions on Machine Learning Research (01/2026)
1.0
CRS Vno
1.0
CRS-V,
0.8
CRS-V,
0.8
CRS-V,
0.6
0.6
ol3
Linear
Shifted cosine
Unner
0.4
0.4
0.2
0.2
CRS-V
CRS.v,
CRS.v.
0.0
0.0
0.2
0.8
0.0
0.2
0.4
0.6
0.8
1.0
Figure 2: Noise schedules computed using dif- ferent discrepancy measures on LSUN Horse x56x256 256 the pixel-space diffusion model.
Figure 3: Noise schedules computed using dif- ferent discrepancy measures on LSUN Church x56x256 in the latent-space diffusion model.
Table 5:
Wall-clock computational cost of computing (a). We report both the cost relative to one training epoch and the cost relative to the total training time for each diffusion setting.
Computational cost
Diffusion type
Dataset
Relative to one training epoch
Relative total training time
Pixel space
LSUN Horse
5 epochs
100
LSUN Bedroom
~ epochs
100
Latent space
LSUN Church
14 epochs
11.4%
LSUN Bedroom
14 epochs
~ 14%
ImageNet
~4 epochs
2.8%
Table 6:
Comparison of wall-clock training time when using linear and CRS-v, the training schedule.
CRS-U, requires evaluating D D2(a, a') during training, resulting in a 20 30% increase in overall training time.
Diffusion type
Dataset
Linear
CRS-
Increase
Pixel space
LSUN Horse
170 hours
215 hours
+26.5%
Latent space
LSUN Church
85 hours
105 hours
+23.5%
LSUN Bedroom
190 hours
245 hours
+28.9%
ImageNet
430 hours
525 hours
+22.1%
pixel-space and latent-space diffusion models. The relative cost appears larger for latent-space diffusion models simply because one latent-space epoch is much faster than one pixel-space epoch. When compared to the total training time, the overhead of computing (a) remains below 15% even in the worst case.
Overhead of computing (a) during training: Table 0 compares the training time when using the linear versus CRS-U, as the training schedule. Because CRS-U; requires evaluating D2 a') during training, the overall training time increases by approximately 20-30 30%. Nevertheless, CRS-v, typically accelerates the
convergence of FID scores during training (see Figs. 12 and
13 Appendix F for details), which can largely
compensate for the per-epoch overhead.
5.7 A Practical Alternative FID-Based Scheduling
In pixel-space diffusion models, CRS-u, + uFid consistently achieves the best performance sampling schedule. However, for large-scale training datasets, computing (F) becomes prohibitively expensive.
As shown in the subsection 5.3 CRS-VFID allocates a large number of timesteps to regions near a ~ 1, resulting in schodule shape similar to that of the cosine schodule. Motivated by this obsorvation, we evaluate a lightweight alternative: CRS- + Beos, which combines CRS-v, with the cosine schedule (see Eq. 13 for v(a:) of the cosine schedule).
13

---

Published in Transactions on Machine Learning Research (01/2026)
Table 7:
Evaluation of 22 Voos as a practical alternative to CRS-E, + UFID for sampling in pixel-
space diffusion models on LSUN Bedroom 256x256. Despite not requiring the computation of (FI((a); CRS-U, Vlos achieves similar FID scores.
Sampler
Sampling Schedule
NFE =
NFE = 10
+
CRS-Us + UPID
14.02
4.88
CRS-ca + Ulass
14.10
4.46
UniPC
CRS-r, + UFID
21.42
3.30
CRS-U, + Veas
20.77
3.30
Table 7 compares the FID scores of CRS- + VFID and CRS-u, + Ceos on LSUN Bedroom 256x256. In all scttings, CRS-v, + Uco achieves FID scores comparable to CRS- + VFID demonstrating that CRS-V, + Ueas is a practical and effective alternative when (F) is costly to compute.
6 Limitations
This paper focuses on developing CRS and empirically validating its effectiveness under fundamental ex- perimental settings. While the results are promising, there are several directions for future work to further enhance the generality, applicability, and performance of CRS. Below, summarize six key limitations and avenues for future work.
(1) Thcorctical analysis. The goal of this work is to provide thcoretically motivatod and empirically effective framework capable of incorporating wide range of discrepancy measures. However, we do not provide theoretical guarantee that minimizing the maximum discrepancy necessarily leads to improved sample quality. Establishing formal connection between distributional change and sample quality remains an important direction for future rescarch.
(2) Evaluation on more complex tasks. Our experiments focus on fundamental settings unconditional and conditoondiional image generation. We did not evaluate CRS in more complex practical scenarios such as text-condittonal generation or classifier-free guidance (CFG). CFG is known to improve FID at the cost of reduced recall, and an adaptive noise schedule may help mitigate this trade-off. Evaluating CRS together with CFG or other conditioning mechanisms could further reveal the benefits of schedule optimization.
(3) Applicability beyond image generation. Although experiments are limited to image genera- tion, CRS is derived from general diffusion-model principles and is therefore expected to generalize to other domains. In this work, (F) proved effective as a sampling discrepancy measure for pixel-space diffusion models; however, it specific to image generation. Applying CRS to non-image modalities will require identifying discrepancy measures that reflect the characteristics of each target domain.
(4) Improving discrepancy measures. As sampling schedule optimizer, CRS currently performs slightly worse than LD3, suggesting that further improvements may be achieved by designing better dis- crepancy measures. A promising direction is to build on advances in schedulee-ptiimiztioon theory such as AYS. As shown in Appendix [ CRS-v, can be interpreted as employing a simplified variant of the KLUB introduced AYS. Thus, futuure theoretical progress in schedule optimization may naturally lead to new discrepancy measures compatible with CRS.
With respect to image-specific metrics, although our experiments primarily relied on FID, many feature- based perceptual metrics have been proposed for image generation. For example, LPIPS (Zhang et al. 2018) is a strong candidate for capturing visually meaningful differences. Since FID and LPIPS depend on feature extractors, a key challenge is designing robust feature representations for noised data, which are essential for accurately assessing distributional change. Self-supervised learning on noised samples a promising direction, as it requires no labels and can be made noise-level aware (e.g., by predicting conditioning on a). Such learned embeddings may lead to more expressive and domain-adaptive discrepancy measures.
14

---

Published in Transactions on Machine Learning Research (01/2026)
(5) Generality across prediction parameterizations and architectures. Our experiments primar- fly evaluate CRS under the noise-prediction parameterization. However, the data- and velocity-prediction formulations are also widely used in recent diffusion models, and the generality of CRS under these param- eterizations has not been thoroughly assessed.
Furthermore, modern diffusion architectures such as Diffusion Transformers (Peebles Xie 2023 were not evaluated. Because CRS is grounded in the dynamics of distributional evolution rather than in model architecture, we expect architectural dependence to be limited, but empirical validation remains important direction for future work.
(6) Scalability to dataset size and image resolution. Although CRS-t, + UFID is effective for opti- mizing sampling schedules in pixcl-space diffusion models, it requires both the computation of (Fi and hyperparameter luning. In particular, the computation of UFID (a) scales linearly with the number of Training images, making this variant impractical for large-scale datasets. Although Section 5.7 demonstrates that CRS- Voos
practical alternative, fundamentally improving scalability will require developing new discrepancy measures that are both compulationally elficient and robust. to noise-level varialion.
Furthermore, all of our experiments were conducted at resolution of 256x256. Validating whether CRS re- mains effective for higher-resolution image generation, and analyzing how the optimal noise schedule depends on image size, are compelling directions for future research.
7 Conclusion
We proposed general framework called CRS for optimizing noise schedules in diffusson models. CRS enforces a constant rate of distributional change throughout the diffusion process and provides a unified perspective for designing both training and sampling schodules. Our framework supports arbitrary user- specified discrepancy measures, and we introduced three practical options that capture different aspects of the distributional dynamics.
Through extensive experiments, we demonstrated that CRS consistently improves the performance of both pixel-space and latent-space diffusion models across various datasets, samplers, and NFEs ranging from 5 to 250. In particular, for sampling in pixel-space diffusion models, CRS-u, + eFiD with tuned hyperparameters achieved the best results. In all other settings, CRS-U, with ç = performed consistently well and is recommended as a default configuration due to its simplicity and robustness.
Although this study has limitations, as discussed in Sec. -includng the lack of a complete theoretical analysis and the need for further validation in other domains we believe that CRS provides a solid and extensible foundation for future research schedule optimization in diffusion models. We hope that this work encourages broader exploration of discrepancy-bae based scheduling, and that future research will further generalize CRS and extend its applicability beyond image generation.
Acknowledgments
We thank the anonymous reviewers for their constructivv and insightful comments, which helped improve the quality of this paper. We are also grateful the team responsible for operating and maintaining the AI Cloud Platform, which provides the internal GPU infrastructure used in this work. Without their support. this research would not have been possible.
References
Luca Ambrogioni. The statistical thermodynamics generative diffusion models: Phase transitions, sym- metry breaking and critical instability. arXiv preprint drXiv:2310.17667, 2024.
Defang Chen, Zhenyu Zhou, Can Wang, Chunhua Shen, and Siwei Lyu. On the trajectory regularity of ODF-based diffusion sampling. Tn Forty-forst International Conference Machine Learning, 2024.
15

---

Published in Transactions on Machine Learning Research (01/2026)
Ting Chen. On the importance of noise scheduling for diffusion models. arXiv preprint arX1v:2301.10972 2023.
Jía Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248 255, 2009. doi: (11099/VVR.2000000
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Newral Information Processing Syslems, volume 34, pp. 8780 8794. Curran Associales, Inc., 2021.
Kenji Doi, Shuntaro Okada, Ryota Yoshihashi, and Hirokatsu Kataoka. Real-srgd: Enhancing real-world
image super-resolution with classifier-fres guided diffusion. In Proceedings of the Asian Conference Computer Vision (ACCV), pp. 3739-3755, 3755, December 2024.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marck, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. Xiv preprinl Xrxiv:2403. 00:00000 2024.
William Feller. On the theory of stochastic processes, with particular reference to applications. In Proceedings of the First Berkeley Symposium Malhemalical Stalistics and Probabilily, pp. 403 4332, Berkeley, CA, 1949. The Regents of the University of California.
Eric Frankel, Sitan Chen, Jerry Li, Pang Wei Koh, Lillian J. Ratliff, and Sewoong Oh. S4s Solving for diffusion model solver. arXiv preprint Xrx::2502. 1 17423, 2025.
Tiankai Hang, Shuyang Gu, Jianmin Bao, Fangyun Wei, Dong Chen, Xin Geng, and Baining Guo. Tm- proved noise schedule for diffusion training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4796-4806, October 2025.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two lime-scale update rule converge to local nash equilibrium. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NewrIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (ods.), Advances in Neural Information Processing Systems, volume 33, pp. 6840 --511. Curran Associates, Inc., 2020.
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.022. 00.0200,,,3, 2022a.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.). Advances in Neural Information Processing Systems, volune 35, pp. 8633 8646. Curran Associates, Inc., 2022b.
Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. Simple diffusion: End-to-end diffusion for high resolution images. arXiv preprint arXiv:2301.11093 2023.
Alexia Jolicoeur-Martinenn Ke Li, Rémi Piché Taillefer Tal Kachman, and Ioannis Mitliagkas. Gotta go fast. when generating data with score-based models. arXiv preprint arXiv:2055 14080, 2021.
16

---

Published in Transactions on Machine Learning Research (01/2026)
Tadashi Kadowaki and Hidetoshi Nishimori. Quantum annealing in the transverse ising model. Phys. Rev. E, 58:5355-5330 Nov 1998. doi: 10.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4396-4405, 4405, 2019.
doi:
Tero Karras, Mika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In Alice H. Oh, Alckh Agarwal, Daniolle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Diederik P Kingma and Ruiqi Gao. Understanding diffusion objectives as the ELBO with simple data augmentation. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated anncaling. Science, 220(4598): 671 680, 1983. doi: 2..122//ciienee2225 22..ce..........7...
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2021.
Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009.
Nupur Kumari, Richard Zhang, Eli Shechtman, and Jun- Yan Zhu. Ensembling off-the-shelf models for gan training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognation (CVPR), pp. 10651 10662, June 2022.
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. In H. Wallach, H. Larochelle, A. Beygelzimer, d' d'Alché-But E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Yaron Lipman, Ricky T. Q. Chen, Heli Ben- Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023. URL https://openrrviee http:////peeeeeeeeeeeeeee/ooooop pet//forummid-PqvWob...r
Luping Liu, Yi Ren, Zhijic Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. In International Conference on Learning Representations, 2022.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022a.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxnan Li, and Jun Zhu. DPM-Solver+ Fast solver for guided sampling of diffusion probabilistic models. arXie preprint orXiv:2211.0100 0100099 2022b.
Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiw:2101.022: 02388, 2021.
Satoshi Morita and Hidetoshi Nishimori. Mathematical foundation of quantum annealing. Journal of Math- ematical Physics, 19512), 12 2008. ISSN 0022-2488. doi: 10.1063/1.299588
Charlie Nash, Jacob Menick, Sander Dieleman, and Peter W. Baltaglia. Generating images with sparse representations. arXi preprint arXim:210 11ii:203.000 111.103.3300 03.03841, 2021.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 8162 8171. PMLR, 18-24 Jul 2021.
17

---

Published in Transactions on Machine Learning Research (01/2026)
Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Nlya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image generation and editing with texi- guided diffusion models. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Reseavch, pp. 168044 PMLR, 17 17-23 Jul 2022a.
Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh; Pranav Shyam, Pamela Mishkin, Bob Megrew, llya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image generation and editing with text- guided diffusion models. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 10784-168808 PMLR, 17 23 Jul 2022b.
Yong- Myng- Park, Chieh-Hssin Lai, Satoshi Hayakawa, Yuhta Takida, and Yuki Mitsufuji. Jump your steps: Optimizing sampling schedule of discrete diffusion models. In The Thirteenth International Conference on Learning Representations, 2025.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4195-4205, October 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024a.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024b. URL httts::/openreview..nn
forum?id=d152228xxt
Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadckova, and Mikhail Kudinov. Grad-tts: A diffusion probabilistic model for text-lo-speech. arXiv preprint arxi 06337, 2021.
Jérémie Roland and Nicolas J. Cerf. Quantum search by local adiabatic evolution. Phys. Rev. A, 65:042308, Mar 2002. doi: 10100//hyyRee...oo A.e.RevvA....... 66..044308. URL htttps://iink: aps http:///iiinaapp.or///////////o/ orgg/bb//10.1103//pggaeee
65. 042308
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp- 10684-10695, June 2022.
Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your steps: Optimizing sampling schedules in diffusion models. In Forty-first International Conference on Machine Learning, 2024.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour Raphael Gontijo-Topes, Bureu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusson models with deep language understand- ing. In Alice H. Oh, Alckh Agarwal, Danielle Belgrave, and Kyunghyun Cho (ods.), Advances in Neural Informalion Processing Systems, 2022a.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emiiy T. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understand- ing. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 36479 36494. Curran Associates, Inc., 2022b.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Trensactions on Pattern Analysis and Machine Intelligence, 45(4):4713-4722 2023. doi: 00.109/TPAMMT22222222044.
18

---

Published in Transactions on Machine Learning Research (01/2026)
Mehdi S. M. Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly, Assessing generative models via precision and recall. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianche and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Represenlations, 2022.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamies. In Francis Bach and David Blei (eds.), Proceedingg of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 225622250000 Lille, Franco, 07-09 Jul 2015. PMLR.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Newral Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Newral Informa- tion Processing Systems, volume 33, pp. 12438 12448. Curran Associates, Inc., 2020.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021b.
Yang Song, Prafulla Dhariwal, Mark Chen, and 1lya Sutskever. Consistency models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of PMLR, 23-29 Jul 2023.
Machine Learning Research, pp.
Christian Szegedy, Vincent. Vanhoucke, Sergey Toffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818-2828 2016. doi: 10.1109/CVPR.2011 201...0..
Vinh Tong, Dung Trung Hoang, Anji Liu, Guy Van den Broeck, and Mathias Niepert. Learning to discretize denoising diffusion ODEs. In The Thirleenth International Conference on Learning Represenlations, 2025.
Christopher Williams, Andrew Campbell, Arnaud Doucet, and Saifuddin Syed. Score-optimal diffusion schedules. In The Thirty-eighth. Annual Conference on Neural Information Processing Systems, 2024.
Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tackling the generative learning trilemma with denoising diffusion GANs. In International Conference on Learning Representations, 2022.
Tianwei Yin, Michaöl Gharbi, Tacsung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T. Freeman. Improved distribution matching distillation for fast image synthesis. In The Thirby-eighth Annual Conference Neural Information Processing Systems, 2024a.
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T. Freeman, and Tacsung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEFE/CVP Conference Computer Vision and Pattern Recognition (CVPR), pp. 6613 6623, June 2024b.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Con- struction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.0 N.Xx:1550600365, 2016.
Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. In The Fleventh International Conference Learning Representations, 2023.
19

---

Published in Transactions on Machine Learning Research (01/2026)
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effective- ness of deep features perceptual metric. In Proceedings of the IEEF Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. UniPC: unified predictor-correctn framework for fast sampling of dilfusion models. In Thirly-seventh Conference on Newral Information Processing Systems, 2023.
Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. DPM-solver-v3: Improved difffusion ODE solver with empirical model statistics. In Thirty-seventh Conference on Neural Information Processing Systems, 2023a.
Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. DPM-solver-v3: Improved diffusion ODE solver with empirical model statistics. In Thirly-seventh Conference on Neural Information Processing Systems, 2023b.
Zhenyu Zhou, Defang Chen, Can Wang, and Chun Chen. Fast ode-based sampling for diffusion models in around 5 steps. In Proceedings of the IEEE/CVE Conference on Compuler Vision and Pollern Recognation (CVPR), pp. 7777-7788 June 2024.
A
Probability Distribution of Diffused Data
We derive the probability distribution of the diffused data used in the toy example (Fig. ! The distribution of the diffused data is given by:
((xa) = / g( (x)) (20 g (x (mp) = (N or00, o² I),
(18)
(19)
where g(zo) is the probability distribution of the target dataset.
We approximate g(xo) by the empirical distribution:
1 N Σ 5(z0 x;)),
((x0))
(20)
where N is the number of data samples, xū is the n-th sample, and ó denotes the Dirac delta function.
Substituting Eq. (20) into Eq. (18 yields:
1 N Σ N(xa azg r2
((Fa)
(21)
This derivation uses the following formula Dirac's delta function:
f(=))((- yddx f(y).
(22)
The resulting probability distribution is Gaussian mixture distribution. We used Eq. example.
21 in the toy
B Derivation of Discrepancy Measure Based on Data Prediction from KLUB
We show that Ug (a) be derived simplified variant of the KLUB introduced in AYS.
20

---

Published in Transactions on Machine Learning Research (01/2026)
B.1 KLUB in Variance- Pressrrcnn Process
The forward SDE in contimuous-time diffusion models (Song et al., 2021b) is defined as:
dæ ft)) g(t)dw,
(23)
where f(t)æ is the drift coefficient, g(t) is the diffusion coefficient, and is the standard Wiener process. The corresponding reverse SDE is given by:
dæ = [f(t)z v (x))]] dt + g(t)dw.
(24)
To approximate the score V lo)) we train a noise prediction model (2. a):
ca(a, ()) = vv logp(()))
(25)
The data prediction æo(x, a) is related to the noise prodiction via:
(x) o a).
(26)
By substituting Eq. 25 into Eq. 24 and using Eq.
26 the reverse SDE can be rewritten
dæ f(x, l)dt + g())dw
(27)
f(x, f) f(t) o(()2((t))
1
a(t) (
(28)
Stochastic DDIM numerically solves this reverse SDE. It exactly integrates the linear term and approximates
the remaining term, introducing discrotizaaion error (Lu et al. 2022a bb. According (Saouu et al. (2024).
in the interval [4),,4], Stochastic DDIM solves the discretized SDE:
dx f' t)dt + g(t)dw,
(29)
fx(x,)) [(1) (()as(()))
a(t) 000 a(t)).
a(t)
(30)
Furthermore, the KLUB between probability distributions of at,,, obtained by solving the true reverse SDE and the descretized one is introduced as
[f - f' 011
(LUB((;-155t)) =
2 f1-1
mooo)( (-gl mo)
o(t)²
dt.
(31)
wa wag (a2))
Ta(1)
aooug( memo(mm mo) oe (20 (20s)
((t)=((t) ( a(t)) xo ((t:)) dt.
(32)
2 1e-1
In the variance-preserving setting, the diffusion coefficient satisfies:
/ 50(()²
á
(33)
Substituting Eq.
33 into Eq. 32 gives:
KLUB KLUB(t(-1,tt) = 1 dta(()
a(t)
J
E
mor=g(()))
[| a(t)) xo(x,, att)))
(34)
(t)
ae (a (w)) æi, (30 (oo;)
21

---

Published in Transactions on Machine Learning Research (01/2026)
B.2 KLUB Discrepancy Measure of CRS
To apply KLUB within the CRS framework, reinterpret it as discrepancy measure over adjacent steps in the diffusion process. Because the KLUB is introduced to evaluate the discretization error when solving the reverse SDE, is reasonable to regard the KLUB as the traccability measure of the diffusion process.
We approximate the integral the KLUB using Simpson's 's rule as follows:
KLUB(L, At) a(t)a(t) (1) DE At)at,
(35)
(3(,,t'
(ira) an me (at (mno) ( (ats)
[1 - (av,,
(36)
Then, to treat (t, t') as a function of a rather than t, we introduce ((f((t),((tt)) = tt,;
(3(a, Bm m m (a
[100 a) x0(2 111
(37)
Here, g( Iar) is given by
( Fad) = / (( (2ngg
(38)
g( xD) = (Nxxi aaoo o² I):
(39)
g(xar (xa) = (Nx ax 8² I),
(40)
where 3 = n'
a and 8 = /1 ,3, Using DE a'),
rewrite KLUB(I, + At) as
âa
KLUB(L, At)
D2 (a, a + Ad)
(41)
2n¹
àa D2 a + Aa) ( a)
Aa
(A1)
(42)
2
Aa
Al
do: Va
a Oa'
At
(43)
dl
1202 2
We use (o, ax) 0 in the second equality. Finally, by adopting (L.UB(L, 1") as the discrepancy measure D(t, E') in CRS, we obtain the following equation:
da
Va
DD2
D At) = /KLUB(t, At) = dt
At.
(44)
2202
da'
a'==
#KLCB(a)
which corresponds to the general form of CRS in Eq. We interpret Us (a) simplified variant of UKLUB(()) in which the multiplicative weighting term omitted.
Although it is possible to optimize the noise schedule using (KLUR(a) directly, this can lead to undesirable behavior, as the weighting term V2nz grows very large when a ~ (i.e., a ~ 0), which can overwhelm the influence of (2 a'). Since DZ a') can interpreted as capturing the distributional change between diffused data distributions (as discussed Section 4.3), the simplified variant U2 (a) is more aligned with the objective of CRS.
Va
C Implementation of CRS
We describe how to compute v(a) using each discrepancy measure.
22

---

Published in Transactions on Machine Learning Research (01/2026)
Table 8: Hyperparameters for computing (ri((a).
Parameter Description
Used Value
Number timesteps
103
for simulating the forward process
04 = 1 (7)
Grid of a-values for evaluating v(a)
2, pixel space
where p
1, latent space
Feature extractor for computing FID
Pixel space: Inception- V3 model used in the EDM repository for FID evaluation Latent space: RosNet--0 trained by us on ImageNet classification in the latent space
C.1 Discrepancy Measure Based on FID
Algorithm | presents the pseudocode for computing (F), and hyperparameters summarized in Tabel Once the forward process is defined, v(a) can be computed by simulating the forward diffusion using the training dataset. Since this procedure is independent of model parameters, it only needs to be executed once prior to training the diffusion model.
It is important to note that, in latent-space diffusion models, the probabilityddistiibtttnaal change must be measured in the latent space embedded by the autoencoder, the Gaussian approximation in Eq. 4 applies to the diffusion process defined in the latent space. Therefore, for latent-space models, trained ResNet-50 (He et al., 2016) classifier on ImageNet in the autoencoder's latent space and use it the feature extractor Φ
To compute v(a), we discretize the range of a using a one-dimensional grid {os)t=o,.... and approximate the continuous function via linear interpolation. In our experiments, we define the discretized grid of a values as:
P
a =
(45)
where 1000. For space-space diffusion models, where v(a) exhibits sharp variation near a 1, we set p = 2 to allocate more evaluation points in that region. For latent-space models, where v(a:) is uniformly distributed, we use p= 1.
The computational cost of vaa) is proportional to the number of training images. For the pixel-space diffusion process on LSUN Horse 256 x 256 with 2M images, the computation takes approximately 35 hours using eight A100 GPUs. For the latent-space process on ImageNet 256x28 with 1.28M images, it takes about 12 hours using eight A100 GPUs. To address the scalability issue for large-scale datasets, we introduce a practical alternative in Sec.
C.2 Discrepancy Measure Based on Data Prediction
Algorithm 2 presents the pseudocode for computing v=(a) using trained diffusion model. As in the case of (a), this involves simulating the forward process on the training data and measuring changes in the predicted data across adjacent timesteps. Hyperparameters are summarized in Table 9 We use the same configuration for all our experiments, regardless of the diffusion type or datasct.
The computation time for obtaining (a) negligible compared with the training time of the diffusion model. It takes approximately 5.5 hours using eight H100 GPUs in pixel-space diffusion models. In latent- space diffusion models, it takes approximately 1.5 hours using eight A100 GPUs.
Algorithm 3 shows how (a) can be used to adaptively optimize the training schedule. Since the data prediction expressed as 20(2, a) x= (a) can become numerically unstable + 0. To prevent this, we restrict evaluation to the range Oth S Omax and fix (z(a) (a(n) for a < Oth-
23

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm v(a)) basis of FID
Require: Training dataset (ar(") J==O....N N-1, timesteps T, noise levels O...,, T, feature model Ensure: ddnearlnterpld((a))eettt )o..... (=...)....
1: # Compute feature vectors of diffused data
2: for n ===00 0 to N do
3: (n) < (x)
4: is latent sapcet_sppcceaaaeeeee then
5:
20 + axt
(n)
6: end if
7:
# Simulate the forward process
for = to T do
10:
Bi + 0:11
11:
8, t 11-1 33
(m) + 8, + S,N
12:
13: end for
14: end for
15:
16: # Compute statistics of feature vectors
(z( (2. --
17: Pr + mean
14 E [O....,T]
{((
(0b (r))() )
18: E. cov
1I E (o,...,T)
19:
20: # Compute v(a) based on FID
21: for t= 0 to T do
FID 14: E, Mc+1, E(+1)
22: +
a1 04+1
23: end for
24: VT DT-1
Table 9: Hyperparameters used to compute 0z (a) for optimizing sampling schodule.
Parameter
Description
Used Value
T
Number of timesteps for simulating the forward process
10a
Number of samples used to estimate (a,a,a')
10a
as
Maximum a used evaluate v(a)
1.0
Oe
Minimum used to evaluate v(a)
0.0
Similar to VDM++ (Knggma & Gao 2023). we divide the range of a into uniform bins and estimate D for each bin using an exponential moving average (EMA).
We the same configuration for all our experiments, regardless of the diffusion type dataset. Although not explicitly included in the pseudocode, the update of DJ starts after 1000 iterations, and the noise schodule a(t) is updated every 100 iterations.
Hyperparameters are summarized in Table IO
While this adaptive scheme slightly increases training time (by 20 30% compared to fixed schedules; see Table 15 in Appendix B we find that the schedule stabilizes early: for example, on LSUN Horse x56, U (a) changes very little after epochs. Thus, fixing the schedule after few epochs or roducing the frequency of D2 updates may mitigate the cost without degrading performance. Developing more efficient implementation remains important direction for future work.
24

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm 2 (a) for optimizing sampling schedule.
Require: training datasel (n In= trained diffusion model c0( oi), number of timesteps T, num- ber of samples for mean S, range of (as,ae)
Ensure: (av
{ (=..........)
# Evaluate E,,,,,' [1 a) zo (D a' )112]
2: DJ < 0, 11 E (0, .)
Aa T (a, ae)
4: for n. in randperm (N) S] do
5: To + ==("")
6: if model operates in latent space then
7: Io - autoencoder(zs)
end if
Yo Z0p
10:
11: # Simulate the forward process
12:
for t= 1 to T do
13:
0t +- as l
ot +- /1 oz
14:
01 at
òe t- vI 11-33 87
0-1
15:
2t
B,ri-1 N,,(0,1)
16:
Yt a
(xt o, Eœ( at))
17:
D3 + Di + 5
11y y yr-
18: end for
19: end for
20:
21: Compute v(a) using D
22: for (= 1=1 1 to T do
1
23: Gr -- V Aa
D2
24: end for
25: Up < U1
Table 10: Hyperparameters used for optimizing the training schedule with CRS-Up.
Parameter
Description
Used Value
Omin
Minimum value of a for the training schodule Maximum value of a for the training schedule Small step used for approximating the derivative Threshold of a for clipping v(a)
0.0
Omax
1.0
Aa
10-3
Othh
0.01
A parameter controlling the dependence of a(t) on v(a)
1
B
Number of bins used to divide the range of a:
100
EMA decay rate for approximately evaluating (a,
0.995
a())
Initial noise schedule
1-t
D Hyperparameter Tuning for Combining Multiple Discrepancy Measures
CRS-u, + UFID achieves the best performance for sampling in pixel-space diffusion models. CRS-v, + UFID involves four hyperparameters: way WFID, Ex. and ÉFID- The effects of these hyperparameters on the resulting noise schedules are illustrated in Figs. 1 5 and 6 Increasing either WFID or ÉFID amplifies the influence
of (a)( resulting in more timesteps being allocated to the region where 1 (see Figs.
and 5
25

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm 3 Adaptive optimization of training schedule using (2)a).
Require: training dataset (œ(()) )a= initial noise schedule a(L); number of bins B, decay rate of EMA range of a (Amaxx Omin), derivative step Aa, minimum value for evaluating v(a) ath, hyperparameter of CRS &
Ensure: noise prediction model Es(2,
ag + ath + (Amax - ath) B'
vb E : B]
D < 10-6: vb (O...., -}
3: for in randperm(N) do
4: # Compute loss for training diffusion models
5: zo + -x(m), t~ 1), e~ ((0,
a f a(l); o o a2
Ea + ( + 0e, 0))
8:
loss - 2 Ea - =l12
10:
# Evaluate DJ using EMA (without gradient)
11:
a 2 ath then
a' Aa, o' t V1-a2 + a' + V1- B
12:
13:
To. - F'v. 10,
14:
Ea' + Eg(Ia', a')
1 o' d'En')
15:
Yo (Da oEa);
Ya' a'
(Din'
Othh B B
16:
f- min
amax
Oth
17:
D
D3 + e) 1a
Ya'
1/
18:
D
Aa
19:
end if
20:
21:
# Update training schedule a())
22:
X {amin: ag. ......., QB
23:
Y < foo, to. v1. UB
24:
o(a)) +- Linearlnterpld(2 )/
25:
Update noise schedule att) by solving Eq. 99
26:
27:
# Update model parameters
28:
loss. backward()
29:
optimizer.step()
30:
end for
Conversely, increasing either wy or &u strengthens the contribution of Ug (a), leading to more timesteps being assigned to the region where a ~ (see Figs. and G
We tuned these hyperparameterss in two steps. First, we fixed &.- {FID and searched for appropriate values of Wa and WFID under the constraint w, + WFID = 1. Then, fixed the weights and adjusted & and FPID- Although further iterations between tuning weights and exponents are possible, we found that just one iteration per step was typically sufficient to outperform conventional noise schedules.
The detailed tuning results on LSUN Horse 256x256 are presented in Table I1 In the first step, adjusting we and WFID led to substantial changes in FID both NFE 250 and NFE 50. In the second step, tuning & and EFID had little impact at NFE 250, but resulted significant FID improvement at NFE 50. This suggests that hyperparameter sensitivity increases the NFEs decrease.
26

---

Published in Transactions on Machine Learning Research (01/2026)
Table 11:
Detailed results of hyperparameter tuning for CRS-U, + FFDD on LSUN Horse 256x256 in pixel-
space diffusion model. + was used as sampler. reduce computational cost, FID scores
were calculated using 10K samples.
NFE
Tuning step
WFID
EFID
&
FID (10K)
250
1st
0.9
0.1
1.0
1.0
3.78
0.7
0.3
3.70
0.5
0.5
3.70
0.3
0.7
3.73
0.1
0.9
3.76
2nd
0.3
0.5
0.5
3.77
1.0
4.30
1.2
3.75
1.4
3.94
1.0
0.5
3.76
1.0
3.70
1.2
3.69
1.4
3.78
1.2
0.5
3.82
1.0
3.69
1.2
3.78
1.4
3.77
1.4
0.5
3.83
1.0
3.78
1.2
3.77
1.4
3.77
50
1st
0.9
0.1
1.0
1.0
6.13
0.7
0.3
4.42
0.5
0.5
4.38
0.3
0.7
4.78
0.1
0.9
7.03
2nd
0.5
0.5
0.5
0.5
7.42
1.0
7.44
1.2
7.48
1.4
7.79
1.0
0.5
4.85
1.0
4.38
1.2
4.32
1.4
4.27
1.2
0.5
4.57
1.0
4.09
1.2
3.96
1.4
3.93
1.4
0.5
4.51
1.0
3.95
1.2
3.85
1.4
3.82
27

---

Published in Transactions on Machine Learning Research (01/2026)
1.0
Who = 0.9,W,
6u - 005,0,5,
Wip 0.7, W, 0.3
=00110.,
0.8
0.5, 0.5
0.8
f =1.
W,
& 6u=14,46 =1
0.6
Wysp 0.1, My
0.6
olt
0.2
0.2
0.0
0.2
0.8
0.2
Effect of the weights wy and WFID on the noise schedule generated by CRS-V, + UFID, with fixed exponents &x EFID = 1.
Figure 4:
Effect of the exponent ÉFID on the noise schedule gencrated by CRS-U, OFID, with fixed weights Wz = WFID 0.5.
Figure 5:
So - 1.0,
11
0.6
fm=110f,=12
4o -
olt
0.4
0.0
0.0
0.2
0.4
0.3
100
Effect of the exponent &a on the noise schedule generated by CRS-u2 + UFID with fixed weights uz WFID 0.5.
Figure 6:
The final hyperparameter settings used in our experiments are summarized in Table 12 schedules a(l) for CRS-u, + UFID are visualized in Figs.
The resulting noise
10 and
11
E
Implementation Details
E.1 U-Net Architecture
The typerparameters of the U-Net model are listed in Table 13 For latent-space diffusion models, we use the same settings as those for ImageNet 64x64 in ADM (Dhariwal Nichol 2021). For pixel-space diffusion models, we adopt the same settings as those for LSUN 256x256 ADM.
E.2 Training Hyperparameters
The training hyperparameters are summarized in Table 14 We save ten checkpoints for each model. FID scores computed using 10K samples for all checkpoints, and report the results from the checkpoint with the best FID.
The training times for each model configuration are provided in Table 15 When using CRS to optimize the training schedule, training time increases by 20-30% compared to conventional noise schedules, due to the overhead of computing v (a) during training. In practice, however, we observe that the noise schedule stabilizes early; for example, on LSUN Horse 256x256, us (a) shows minimal change beyond 10 epochs. This suggests that freezing the schedule after small number of epochs reducing the frequency of D7 updates
28

---

Published in Transactions on Machine Learning Research (01/2026)
Table 12: Hyperparameter settings used for CRS-Ua UFID in our experiments on LSUN Horse 256x LSUN Bedroom 256x256 FFHQ 64x64, and CIEAR10 32x32.
Dataset
Sampler HDE.DPM-SOlx+++
NFE
WFID
EFID
LSUN Horse 256x256
250
0.9
0.1
1.4
1.0
50
0.7
0.3
1.4
1.2
PNDM
250
0.9
0.1
1.0
0.5
50
0.5
0.5
1.2
0.5
+2M
250
0.7
0.3
1.0
1.2
50
0.5
0.5
1.4
1.4
LSUN Bedroom 256x256
+
10
0.5
0.5
1.0
1.0
5
0.5
0.5
1.0
1.2
UniPC
10
0.5
0.5
1.0
1.5
5
0.3
0.7
1.0
1.0
FFHQ 64x64
DPM-Solver++
20
0.5
0.5
1.7
1.4
10
0.5
0.5
1.6
1.4
UniPC(3M)
20
0.5
0.5
1.7
1.4
10
0.5
0.5
1.6
1.6
1PNDM(3M)
20
0.5
0.5
1.6
1.2
10
0.5
0.5
1.6
1.2
iPNDM(4M)
20
0.5
0.5
1.4
1.4
10
0.5
0.5
1.6
1.4
CIFARIO 32x32
+2M-
20
0.5
0.5
1.8
1.2
10
0.5
0.5
1.7
1.2
UniPC(3M)
20
0.5
0.5
2.3
1.0
10
0.5
0.5
1.9
1.5
(PNDM(3M)
20
0.5
0.5
1.7
1.4
10
0.5
0.5
1.7
1.4
1PNDM(4M)
20
0.5
0.5
1.6
1.6
10
0.5
0.5
1.6
1.6
Table 13: Hyperparameters of UNet model.
Latent-space diffusion models
Pixel-space diffusion models
Resolution
64 x 64
256 x 256
Number of parameters
296M
552M
Channels
192
256
Depth
2
Channels multiple
1,2,3,4
1,1,2,2,4,4
Hcads channels
64
64
Attention resolution
32,16,8
32, 16,8
BigGAN up/downsample
True
True
Dropout
0.1
0.1
Table 14: Hyperparameters for training diffusion models. Same configuration was used for training latent- space and pixcl-space diffusion models.
Item
Value
EMA decay
0.9999
Optimizer
Adam
81
0.9
B2
0.999
Learning rate
1e-4
Batch size
32 x 8
29

---

Published in Transactions on Machine Learning Research (01/2026)
1.0
1.0
NFE 250
NFE 50
0.8
0.6
0.6
ol3
Linear
Linger
Shifted cosine
Shitted cosine
0.4
0.2
0.2
0.0
0.2
0.8
0.2
0.4
0.6
0.8
Figure 7:
Noise schedules generated by CRS- Ur + UPID for LSUN Horse 256x256, used with SDE-DPM-Solve++#(M0 during sampling.
Figure 8: Noise schedules generated by CRS-
+ UFDD for LSUN Horse 256x256, used with PNDM during sampling.
NFE 50
NFE
0.6
EDM
0.8
EDM
Linear
Linea
Shifted cosine
Shifted cosine
0.4
0.4
0.0
0.0
0.2
0.4
0.9
10
0.0
0.2
0.4
0.6
0.8
10
Figure Noise schedules generated by CRS- Ur + UFID for LSUN Horse 256x 256, used with + (2M) during sampling.
Noise schedules generated by CRS- Vy + UFID for LSUN Bedroom 256x used with + (2M) during sampling.
Figure 10:
0.6
EDM
Limrgl
Shitted cosine
0.4
0.0
0.4
0.3
Noise schedules generated by CRS- Ug UFID for LSUN Bedroom 256 2566, used with UniPC during sampling.
Figure 11:
30

---

Published in Transactions on Machine Learning Research (01/2026)
Table 15:
Training time of diffusion models.
Diffusion type
Dataset
GPUs
Epochs
Training schedule Training time
Linear
Latent space
LSUN Church
A100 8
1000
VDM++
85 hours
CRS-VFID
CRS-U2
105 hours
CRS-ux
LSUN Bedroom
A100
100
CRS-UFID
190 hours
CRS-U,
245 hours
CRS-
Linear
ImageNet
A100 x
500
VDM++
430 hours
CRS-OFID
CRS-U:
525 hours
CRS-U2
Linear
Pixel space
LSUN Horse
H100
50
Shifted cosine
170 hours
VDM+
CRS-u-
215 hours
CRS-
LSUN Bedroom
H100 x 8
50
CRS-Ux
325 hours
can mitigate training overhead without degrading performance. Developing more efficient implementation remains an important direction for future work.
E.3 Noise Schedules for Training and Sampling
The conventional noise schedules used in this study are summarized in Table 16
F
Detailed Experimental Results
We provide detailed experimental results confirm the effectiveness of CRS for optimizing both training and sampling schedules.
F.1 Results on Pixel-Space Diffusion Models
LSUN Horse 256 256: The results showing the dependence training and sampling schedules at NFE 250 are presented in Table 17 To assess the generality across samplers, evaluated three different ones: 22) PNDM, and DPM-Solver+-22M As shown in the gray-shaded rows, using CRS- Ux for training consistently improved multiple evaluation metrics, including FID, regardless of the sampler used.
Figure 12 shows the evolution of the FID score with 10k samples during training, evaluated using DPM- Solver (2M) with NFE = 250. In this experiment, we compare CRS-u, with shifted cosine, which was the strongest-performing baseline training schedule. The results demonstrate that using CRS-u, as the training schedule not only improves the final FID score but also leads to faster convergence throughout training.
Table 18 presents the effect of different sampling schedules at NFE = 250 and NFE = 50. When using (Me-DDMM-oovvr++++2M) CRS-U, + UFID improved all four metrics at NFE 250, achieving new state- of-tho-art FID score. With PNDM and (2M) differences among sampling schedules were minor NFE 250; however, NFE 50, performance became more sensitive the sampling schedule. In this low-NFE regime, CRS-u, UFID achieved the best FID score.
31

---

Published in Transactions on Machine Learning Research (01/2026)
Table 16: Conventional noise schedules evaluated in this study. T2 denotes number of timesteps for sampling.
Name
Phase
Noise schedule
Olinear (t) Lincarlnterpid (f T
fail)=.......
0g
Linear
Training
0iSiST= = V1 - 3:0;-1,
3, Bmin + Bmay - Bmin (i 1).
T
1000, 3mnn 10 Bmaxx 0.02.
Sampling
O'ineea
+.
.... Ts
O'shifted (t) = (sggmoid()(t)),
Shifted cosine
Training
(()) = -2 log lan (x) + log )
where d resolution in diffusion process.
@sshifeed
/ Os..,TT..
Sampling
åshifted (t) amnn (Amax amin (ssiteeatiteee(t).)
Ouiin
0.01, Qmx 1.0.
P(A) E BOODD.NNN0... ((\) (Eo(,,,)) ā(A)) -e11]]
max (()) argmax ((A)
VDM++
Training
((A) ((A),
2 argmax (A),
(()) N(As (e +c)),
μ* 2.4, 2.4, e 0.5.
Sampling
Only for training. Not evaluated in this study.
EDM
Training
an = 1,121 = OEDM T.
@EDM(() / (igmmid(A(U))
Sampling
A(t) -2 -2oog ott),
p
(t) mmax +
0 - Fmax
(1 (1-t) t)
mmax 80, Fmin
0.002, p =
32

---

Published in Transactions on Machine Learning Research (01/2026)
40
Shifted cosine
Linea
score with 10K samples
35
CRS-V,
FID score with 10K 10K samples
12-
CRS-V,
30
10-
25-
10 15
20
Training 9 epochs
25
30
25
40
50
400 600
Training epechs
2000
Figure 12: Evolution of the FID score with 10k samples during training on LSUN Horse 256 256 in the pixel-space diffusion model. Using CRS-v, as the training schedule not only improves the fi- nal FID score but also leads to faster convergence compared to shifted cosinc. Evaluation is per- formed using D2M--olver+++2MM) with NFE 250.
Evolution of the FID score with 10k samples during training on LSUN Church 256x256 in the latent-space diffusion model. CRS-v, achieves slightly better best FID score and consistently lower FID scores before reaching the best value compared to the linear schedule. Evaluation is performed using DPM- + with NFE 30.
Figure 13:
LSUN Bedroom 256x256; Since only FID scores are reported in the main text (Table 2 we include sFID, precision, and recall in Table 19 for reference.
Table 20 compares the FID scores of CRS-uz + UFiD and EDM sampling under various values of the hyper- parameter p. While the default setting for EDM uses م 7, tuning P further improve its performance. However, even with the optimal p, EDM sampling did not outperform CRS-u, + UFID in terms of FID.
F.2 Results on Latent- Latent-Space Diffusion Models
LSUN Church 256x256: The results showing the dependence on training and sampling schedules at NFE 30, using stochastic and deterministic samplers, are summarized in Tables 21 and 22 respec- tively. To evaluate gencrality across samplers, tested five different ones: Stochastic DDIM, SDE-DPM- +2Mv DDIM, PNDM, and DPM-Solver++ As shown in the gray-shaded rows, CRS-t, con- sistently improved FID scores across all samplers except for SDE- MMo-DD-MSoolve++(2221
Figure 13 shows the evolution of the FID score with 10k samples during training, evaluated using DPM- Solver (2M) with NFE 30. In this experiment, we compare CRS-v, with linear, which was the strongest-performing baseline training schedule. Both training schedules reach their best FID scores around 400 epochs, with CRS-U, achieving slightly better best FID score. Furthermore, CRS-u, also showed consistently lower FID scores before reaching the best value, indicating its effectiveness in terms of convergence speed as well.
The effects of different sampling schedules are shown separately for stochastic and deterministic samplers in Tables 23 and 24 respectively. CRS-U, or CRS-v, achieved the best FID scores all cases except when using MMll-DDM-Solve++++MM at NFE = 20. In most cases, multiple evaluation metrics-including sFID, precision, and recall were simultaneously improved.
LSUN Bedroom 256x256: The results showing the dependence sampling schedules using stochastic and deterministic samplers are presented in Tables 25 and 26 respectively. The training schedule was fixed to CRS-v,- CRS achieved the best FID scores in all cases, except when using SDE- M20 at NFE = 30 and NFE = 20. In most cases, multiple evaluation metrics were simulttnnously improved.
FID scores evaluated small NFE scttings are listed in Table 27 Compared to the results reported DP (Zheng et al. (023a)) better FID scores are consistently achieved by using CRS optimize both training and sampling schedules across all NFE values. As with other evaluations in this paper, we
33

---

Published in Transactions on Machine Learning Research (01/2026)
set = by default for CRS-U, and CRS-e,, except at NFE = 5, where was manually tuned to further improve performance. In this experiment, we observed that the importance of tuning & increases as the NFE decreases. We attribute this behavior to the discretization error introduced by the contimuus--tmme approximation used in Eq. 7 comprehensive set of evaluation metrics, including FID, sFID, precision, and recall, is reported Table 28 CRS-based noise schedules consistently outperform the linear schedule across all metrics, except for recall at NFE
ImageNet 256x 256: We demonstrated the effectiveness of CRS in class-conditional image generation ImageNet. It is important to note that experiments do not use classifier-free guidance (CFG), and therefore the FID scores are naturally higher than those obtained under CFG-enabled settings. However, as shown in Table 29 our model achieves better FID score than the LDM-4 results reported in the original LDM paper (Rombach et al. 2022). These observations indicate that our ImageNet results represent a reasonable level of performance for class-conditional generation without CFG, and provide a meaningful basis for evaluating the impact of CRS in this setting.
The results showing the dependence on training and sampling schedules at NFE- = 30, using both stochastic and deterministic samplers, are summarized in Tables 30 and 31 respectively. As shown in the gray-shaded rows, CRS-FFID achieved the best performance for training schedule optimization, while CRS-2,, which consistently performed well on hthe datasets, was the second-best. This result may be attributed to the fact that the feature extractor used in FID computation pre-trained on ImageNet, which may advantage CRS-UFID in this specific setting.
The impact of different sampling schedules using stochastic and deterministie samplers is shown in Tables 32 and 33 respectively. Here, the training schedule was fixed to CRS-uz, which outperformed conventional noise schedules by siguificant margin on ImageNet. and delivered strong results across wide range of datasets. CRS achieved the best FID scores in all cases, except when using PNDM at NFE = 20 and D2M-Solve++++MM at NFE = 30. In most settings, CRS-based schedules also led to improvements in other evaluation metrics, including sFID, precision, and recall.
G Generated Samples
Generated samples are shown in the following figures:
LSUN Horse 256x256 in the pixel-space diffusion model: Figs. 14 and 15
LSUN Bedroom 256 256 the pixel-space diffusion model: Figs. 16 and
LSUN Church x56x256 in the latent-space diffusion model: Fig.
18
LSUN Bedroom 256x256 the latent-space diffusion model: Fig.
ImageNet 256x256 in the latent-space diffusson model: Fig.
20
All models are trained using CRS-t, for the training schedule optimization.
34

---

Published in Transactions on Machine Learning Research (01/2026)
Table 17: Performance of pixel-space diffusion models on LSUN Horse 256x256 at NFE 250, evaluated under various training and sampling schedules. To assess generality across samplers, we used three different samplers: SDE-DDMM DDF--DMM-Soler+++MMMM
PNDM, and DPM-Solver+++(MM) Gray-shaded rows indicate results for different training schedules (with fixed sampling), while unshaded rows show results for different sampling schedules (with fixed training). Bold values indicate the best performance for each metric. Note that 'cosine" refers to the shifted cosine schedule.
Training
Sampling
Metrics
Sampler SDE-DPM-
schedule
schedule
FID. $
sFID 1
Precision f
Recall f
Linear
Linear
2.90
6.82
0.66
0.56
Solver+ (2MM
Cosine
2.95
6.43
0.67
0.56
VDM++
3.82
7.40
0.66
0.54
CRS-U,
2.91
6.56
0.66
0.56
CRS-uz
2.73
6.40
0.67
0.56
CRS-
Linear
2.73
6.40
0.67
0.56
Cosine
3.13
7.09
0.67
0.54
EDM
2.09
6.16
0.69
0.56
CRS-t:
2.87
6.71
0.66
0.56
CRS-v,
5.46
8.34
0.62
0.52
CRS-u, + UFID
2.03
6.06
0.69
0.56
PNDM
Linear
Linear
4.06
6.23
0.57
0.61
Cosine
2.36
5.59
0.63
0.60
+DM++
70.18
36.78
0.20
0.53
CRS-VFID
8.20
7.92
0.51
0.60
CRS-2,
2.09
5.43
0.65
0.58
CRS-uz
Linear
2.09
5.43
0.65
0.58
Cosine
2.19
5.59
0.67
0.57
EDM
2.35
5.85
0.67
0.56
CRS-v:
2.11
5.58
0.66
0.58
CRS-u,
2.50
5.94
0.64
0.58
CRS-U, VFID
2.12
5.50
0.67
0.57
DPM-
Linear
Linear
3.17
5.98
0.60
0.61
Solver (2M)
Cosine
2.31
5.63
0.63
0.60
+
47.63
29.77
0.26
0.56
CRS-U,
7.12
7.45
0.54
0.60
CRS-v,
2.08
5.47
0.65
0.59
CRS-U,
Linear
2.08
5.47
0.65
0.59
Cosine
2.18
5.72
0.66
0.58
EDM
2.35
5.85
0.67
0.56
CRS-r,
2.12
5.65
0.66
0.58
CRS-
2.87
6.15
0.63
0.58
CRS-c, + UFID
2.19
5.56
0.67
0.57
35

---

Published in Transactions on Machine Learning Research (01/2026)
Table 18: Performance pixel-space diffusion models on LSUN Horse 256x 256 under different sampling
schedules. Training schedule was fixed to CRS-V2. Bold values indicate the best sampling schedule for each evaluation metric.
Sampling
Metrics
Sampler
NFE
schedule
FID
sFID 4
Precision
Recall 1
SDE-DPM-
250
Linear
2.73
6.40
0.67
0.56
Solver+ +(2M)
Shifted cosine
3.13
7.09
0.67
0.54
EDM
2.09
6.16
0.69
0.56
CRS-c,
2.87
6.71
0.66
0.56
CRS-c
5.46
8.34
0.62
0.52
CRS-U, + UFID
2.03
6.06
0.69
0.56
50
Linear
7.54
9.73
0.59
0.48
Shifted cosine
11.37
12.46
0.53
0.43
EDM
2.75
6.91
0.68
0.55
CRS-u:
6.08
8.75
0.61
0.50
CRS-E&
12.99
12.59
0.50
0.41
CRS-c, + UFID
3.84
8.16
0.67
0.52
PNDM
250
Linear
2.09
5.43
0.65
0.58
Shifted cosine
2.19
5.59
0.67
0.57
EDM
2.35
5.85
0.67
0.56
CRS-c:
2.11
5.58
0.66
0.58
CRS-E,
2.50
5.94
0.64
0.58
CRS-E, + UFID
2.12
5.50
0.67
0.57
50
Lincar
2.50
5.76
0.62
0.59
Shifted cosine
3.92
6.77
0.60
0.57
EDM
2.36
5.87
0.67
0.56
CRS-s
2.46
5.97
0.64
0.58
CRS-c;
6.03
7.32
0.57
0.56
CRS-e, + UFID
2.04
5.60
0.66
0.58
DPM-
250
Linear
2.08
5.47
0.65
0.59
+
Shifted Cosine
2.18
5.72
0.66
0.58
EDM
2.35
5.85
0.67
0.56
CRS-c:
2.12
5.65
0.66
0.58
CRS-C,
2.87
6.15
0.63
0.58
CRS- + UFID
2.19
5.56
0.67
0.57
50
Linear
3.17
5.98
0.60
0.59
Shifted cosine
5.27
7.12
0.57
0.56
EDM
2.43
5.87
0.67
0.56
CRS-ee
3.10
6.18
0.62
0.58
CRS-ca
7.44
7.83
0.55
0.55
CRS-c, + WEID
2.26
5.90
0.66
0.57
36

---

Published in Transactions on Machine Learning Research (01/2026)
Table 19: Evaluation results on LSUN Bedroom 256x256 in sixcl-space diffusion model. CRS-u, was used for optimizing training schedule.
Sampling
Metrics
Sampler
NFE
schedule
FID 4
sFID
Precision
Recall f
+2M
10
EDM
4.73
7.75
0.53
0.51
CRS-U, + UFID
4.88
8.67
0.49
0.52
EDM
23.72
19.24
0.20
0.28
CRS- + UPID
14.02
17.49
0.28
0.41
UniPC
EDM
4.94
8.20
0.53
0.40
CRS- UFID
3.30
6.60
0.55
0.53
EDM
82.52
33.35
0.06
0.08
CRS- + UFID
21.42
11.26
0.21
0.44
Table 20: Comparison of FID scores for CRS-u, + UFID and EDM sampling on LSUN Bedroom 256x256
the sixce-space diffusion model. Performance of EDM is reported under various values of the hyperparameter
p. Even with the optimal p, EDM does not outperform CRS-u, UPIn-
Sampler
Sampling schedule
NFE 5
NFE = 10
+
EDM:
190.32
102.10
EDM: 3
30.24
13.12
EDM: p=
22.97
7.92
EDM: p p=7
23.72
4.73
CRS V. UFID
14.02
4.88
UniPC
EDM: p= 1
178.24
90.30
EDM: p 3
23.45
4.43
EDM: P 5
55.06
3.96
EDM: P 7
82.52
4.94
CRS- + UFID
21.42
3.30
37

---

Published in Transactions on Machine Learning Research (01/2026)
Performance latent-space diffusion models on LSUN Church x56x256 at NFE 30, evaluated under various training and sampling schedules using stochastic samplers. Two samplers were used: Stochastic DDIM and MIL-DDPM-Solver++(2M))) Gray-shaded rows indicate variations in training schedules (with fixed sampling), while unshaded rows show variations in sampling schedules (with fixed training). Bold values indicate the best result for each evaluation metric.
Table 21:
Training Sampling
Metries
Sampler
schedule schedule
FID
sFID 1
Precision
Recall
Stochastic DDIM
Linear
Linear
10.73
18.31
0.57
0.33
VDM++
11.05
17.33
0.56
0.35
CRS-EFID
10.85
18.77
0.57
0.34
CRS-2,
10.43
17.81
0.57
0.34
CRS-s,
9.82
16.39
0.58
0.34
CRS-
Linear
9.82
16.39
0.58
0.34
EDM
21.73
29.33
0.43
0.22
CRS-UFID
7.36
13.04
0.60
0.39
CRS-ve
7.53
13.73
0.61
0.38
CRS-C,
6.77
12.57
0.63
0.40
SDE-DPM- +
Linear VDM++
Linear
3.49
10.19
0.63
0.57
3.33
9.79
0.63
0.57
CRS-UFID
3.50
10.69
0.63
0.57
CRS-2,
3.57
10.26
0.64
0.56
CRS-v,
3.38
10.09
0.64
0.55
CRS-ez
Linear
3.38
10.09
0.64
0.55
EDM
4.20
11.69
0.63
0.54
CRS-FFID
3.61
9.51
0.61
0.57
CRS-2
3.19
9.56
0.62
0.59
CRS- Up
3.20
9.14
0.62
0.58
38

---

Published in Transactions on Machine Learning Research (01/2026)
Table 22: Performance of latent-space diffusion models LSUN Church 256x256 at NFF 30, evaluated under various training and sampling schodules using deterministic samplers. Three samplers were used: DDIM, PNDM, and DPM-Solver++(2MM) Gray-shaded rows indicate variations in training schedules (with fixed sampling), while unshaded rows show variations in sampling schedules (with fixed training). Bold
values indicate the best result. for each evaluation metric.
Training Sampling
Metrics
Sampler
schedule schedule
FID +
sFID +
Precision
Recall t
DDIM
Linear
Linear
6.18
13.16
0.59
0.51
VDM++
6.50
12.67
0.56
0.54
CRS-UFID
6.06
13.69
0.59
0.51
CRS-u;
6.05
12.49
0.58
0.50
CRS-U,
5.67
12.33
0.59
0.51
CRS-V,
Linear
5.67
12.33
0.59
0.51
EDM
9.15
16.86
0.56
0.44
CRS-UFID
5.35
10.70
0.57
0.53
CRS-V:
4.95
11.01
0.59
0.53
CRS-
4.81
10.79
0.59
0.55
PNDM
Linear
Linear
3.74
10.48
0.60
0.58
VDM++
3.88
10.20
0.59
0.59
CRS-UFID
3.59
10.61
0.61
0.59
CRS-v:
3.71
10.36
0.60
0.59
CRS-u,
3.56
10.37
0.61
0.58
CRS-
Linear
3.56
10.37
0.61
0.58
EDM
3.88
10.76
0.60
0.57
CRS-FFID
3.59
9.69
0.59
0.61
CRS- U2
3.48
9.79
0.59
0.59
CRS-
3.62
10.15
0.59
0.61
DPM-
Lincar
Lincar
3.82
10.28
0.60
0.58
Solver (2M)
VDM++
4.00
10.07
0.58
0.59
CRS-FFID
3.78
10.51
0.60
0.58
CRS-U:
3.87
10.30
0.60
0.59
CRS-U,
3.69
10.23
0.60
0.59
CRS-v,
Linear
3.69
10.23
0.60
0.59
EDM
4.31
11.31
0.60
0.56
CRS-UFID
3.98
9.61
0.58
0.60
CRS-
3.63
9.91
0.59
0.60
CRS-
3.59
9.51
0.59
0.60
39

---

Published in Transactions on Machine Learning Research (01/2026)
Table 23: Performance of latent-space diffusion models on LSUN Church x using stochastic samplers, evaluated under different sampling schedules. The training schedule was fixed to CRS-ca- Bold values indicate the best sampling schedule for each evaluation metric.
Sampling
Metries
Sampler
NFE
schedule
FID +
sFID
Precision t
Recall f
Stochastic DDIM
50
Linear
6.23
11.94
0.65
0.43
EDM
10.05
17.14
0.60
0.34
CRS-VFID
5.17
10.49
0.65
0.44
CRS-v,
5.26
10.94
0.65
0.44
CRS-v,
4.82
11.91
0.62
0.51
30
Linear
9.82
16.39
0.58
0.34
EDM
21.73
29.33
0.43
0.22
CRS-UFID
7.36
13.04
0.60
0.39
CRS-U,
7.53
13.73
0.61
0.38
CRS-v,
6.77
12.57
0.63
0.40
20
Linear
18.24
25.36
0.45
0.24
EDM
48.46
52.53
0.25
0.11
CRS-VFID
12.42
18.38
0.51
0.29
CRS-v,
12.53
19.27
0.52
0.29
CRS-V
10.66
16.54
0.55
0.32
SDE-DPM-
50
Linear
3.42
10.58
0.63
0.57
Solver
EDM
3.57
10.93
0.64
0.57
CRS-UFID
3.32
9.19
0.62
0.58
CRS-V,
3.14
9.65
0.63
0.58
CRS-u,
3.09
9.34
0.63
0.59
30
Linear
3.38
10.09
0.64
0.55
EDM
4.20
11.69
0.63
0.54
CRS-UFID
3.61
9.51
0.61
0.57
CRS-v,
3.19
9.56
0.62
0.59
CRS-
3.20
9.14
0.62
0.58
20
Linear
3.55
10.37
0.63
0.56
FDM
5.92
13.79
0.62
0.52
CRS-UFID
4.39
9.33
0.63
0.48
CRS-v
3.58
9.93
0.61
0.57
CRS-V,
3.57
9.55
0.62
0.56
40

---

Published in Transactions on Machine Learning Research (01/2026)
Table 24:
Performance of latent-space diffusion models on LSUN Church 256x256 using deterministic samplers, evaluated under different sampling schedules. The training schedule was fixed to CRS-Uz- Bold values indicate the best sampling schedule for each evaluation metric.
Sampling
Metrics
Sampler DDIM
NFE
schedule
FID 4
sFID
Precision f
Recall
50
Lincar
4.52
11.03
0.60
0.54
EDM
5.95
13.37
0.59
0.50
CRS-FFID
4.28
10.07
0.59
0.56
CRS-us
4.17
10.66
0.60
0.56
CRS-
4.01
10.10
0.60
0.57
Linear
5.67
12.33
0.59
0.51
EDM
9.15
16.86
0.56
0.44
CRS-FFID
5.35
10.70
0.57
0.53
CRS-ua
4.95
11.01
0.59
0.53
CRS-
4.81
10.79
0.59
0.55
20
Linear
7.85
14.66
0.56
0.47
EDM
15.74
22.86
0.48
0.34
CRS-UFID
7.47
12.74
0.54
0.48
CRS-v:
6.49
12.55
0.56
0.49
CRS-s,
6.09
11.72
0.57
0.50
PNDM
50
Linear
3.52
10.42
0.61
0.58
EDM
3.71
10.59
0.60
0.57
CRS-FFID
3.49
9.95
0.60
0.61
CRS-
3.48
10.18
0.60
0.59
CRS-vz
3.48
9.92
0.60
0.60
30
Linear
3.56
10.37
0.61
0.58
EDM
3.88
10.76
0.60
0.57
CRS-UFID
3.59
9.69
0.59
0.61
CRS-v,
3.48
9.79
0.59
0.59
CRS-
3.62
10.15
0.59
0.61
20
Linear
3.74
10.24
0.60
0.59
EDM
4.57
11.49
0.59
0.56
CRS-UFID
4.06
9.54
0.57
0.61
CRS-s:
3.64
9.98
0.59
0.61
CRS-v,
3.86
9.54
0.62
0.52
DPM-
50
Linear
3.62
10.44
0.61
0.58
+
EDM
3.90
10.85
0.60
0.56
CRS-UFID
3.59
9.68
0.59
0.60
CRS-s,
3.46
9.82
0.60
0.59
CRS-E
3.43
9.45
0.60
0.59
30
Lincar
3.69
10.23
0.60
0.59
EDM
4.31
11.31
0.60
0.56
CRS-FFID
3.98
9.61
0.58
0.60
CRS-U
3.63
9.91
0.59
0.60
CRS-
3.59
9.51
0.59
0.60
20
Linear
3.92
10.03
0.59
0.59
EDM
5.25
12.19
0.59
0.55
CRS-FFID
4.95
9.92
0.56
0.58
CRS-v:
3.89
9.73
0.58
0.60
CRS-s,
4.00
9.44
0.58
0.59
41

---

Published in Transactions on Machine Learning Research (01/2026)
Performance of latent-space diffusion models on LSUN Bedroom 256x256 using stochastic samplers, evaluated under different sampling schedules. The training schedule was fixed to CRS-Uz, Bold values indicate the best sampling schedule for each evaluation metric.
Table 25:
Sampling
Metries
Sampler
NFE
schedule
FID +
sFID
Precision f Recall t
Stochastie DDIM
50
Linear
3.95
8.98
0.56
0.48
EDM
7.43
16.05
0.54
0.38
CRS-VFID
3.09
7.17
0.57
0.50
CRS-v,
3.25
7.81
0.57
0.49
CRS-U,
3.08
7.20
0.58
0.50
30
Linear
7.28
15.70
0.53
0.38
EDM
17.03
33.60
0.40
0.26
CRS-UFID
5.17
10.62
0.57
0.43
CRS-U,
5.58
12.33
0.56
0.42
CRS-v,
4.78
9.85
0.55
0.45
20
Linear
14.53
29.25
0.42
0.28
EDM
38.01
68.62
0.21
0.13
CRS-VFID
9.18
16.67
0.50
0.35
CRS-v,
9.70
19.40
0.49
0.34
CRS-U
8.11
15.96
0.51
0.36
SDE-DPM-
50
Linear
2.47
6.44
0.57
0.54
Solver
EDM
2.48
6.32
0.57
0.54
CRS-UFID
2.21
5.87
0.57
0.54
CRS-V,
2.22
5.76
0.57
0.55
CRS-u,
2.23
5.67
0.57
0.54
30
Linear
2.45
6.70
0.56
0.55
EDM
2.90
7.41
0.57
0.52
CRS-UFID
2.46
6.19
0.57
0.54
CRS-v,
2.50
5.93
0.57
0.53
CRS-
2.48
5.73
0.56
0.54
20
Linear
2.78
6.79
0.56
0.53
FDM
4.00
9.57
0.55
0.50
CRS-UFID
3.19
6.70
0.55
0.53
CRS-v
2.87
6.22
0.57
0.53
CRS-Vp
3.07
6.34
0.56
0.52
42

---

Published in Transactions on Machine Learning Research (01/2026)
Table 26:
Performance of latent-space diffusion models on LSUN Bedroom 256x256 using deterministic samplers, evaluated under different sampling schedules. The training schedule was fixed to CRS-Uz- Bold values indicate the best sampling schedule for each evaluation metric.
Sampling
Metrics
Sampler DDIM
NFE
schedule
FTD 4
sFID 4
Precision f
Recall
50
Lincar
3.02
6.59
0.53
0.54
EDM
4.17
7.64
0.52
0.51
CRS-FFID
2.66
6.03
0.54
0.55
CRS-us
2.71
6.03
0.54
0.55
CRS-
2.62
5.86
0.54
0.55
Linear
3.76
7.50
0.53
0.51
EDM
6.81
9.98
0.47
0.46
CRS-FFID
3.23
6.61
0.53
0.53
CRS-ua
3.22
6.65
0.53
0.53
CRS-
3.10
6.43
0.54
0.53
20
Linear
5.38
9.27
0.49
0.48
EDM
12.57
14.55
0.38
0.40
CRS-UFID
4.39
7.86
0.51
0.49
CRS-v:
4.28
7.97
0.51
0.49
CRS-u,
4.10
7.54
0.51
0.51
PNDM
50
Linear
2.51
6.02
0.54
0.56
EDM
2.64
6.16
0.53
0.57
CRS-FFiD
2.50
5.66
0.54
0.57
CRS-
2.44
5.73
0.54
0.57
CRS-v,
2.45
5.61
0.54
0.56
30
Linear
2.57
6.03
0.54
0.56
EDM
2.93
6.45
0.52
0.57
CRS-UFID
2.64
5.59
0.53
0.57
CRS-v,
2.48
5.58
0.54
0.56
CRS-
2.65
5.53
0.54
0.57
20
Linear
2.77
6.13
0.53
0.57
EDM
3.76
7.25
0.52
0.54
CRS-UFID
3.02
5.66
0.52
0.57
CRS-v:
2.71
5.53
0.54
0.56
CRS-v,
3.63
6.23
0.61
0.57
DPM-
50
Linear
2.59
6.15
0.54
0.56
Solver (2M)
EDM
2.86
6.44
0.52
0.57
CRS-FFID
2.46
5.63
0.54
0.56
CRS-s,
2.46
5.63
0.54
0.56
CRS-E,
2.37
5.42
0.54
0.56
30
Lincar
2.68
6.18
0.54
0.56
EDM
3.40
7.09
0.51
0.56
CRS-FFID
2.60
5.63
0.54
0.56
CRS-U,
2.60
5.53
0.54
0.56
CRS-
2.63
5.45
0.54
0.55
20
Linear
2.89
6.24
0.53
0.56
EDM
4.14
8.07
0.52
0.53
CRS-FFID
3.03
5.81
0.52
0.56
CRS-v:
2.88
5.59
0.53
0.56
CRS-uz
3.00
5.63
0.54
0.55
43

---

Published in Transactions on Machine Learning Research (01/2026)
Table 27: FID scores on LSUN Bedroom 256x256 in latent-space diffusion models small NFE settings. Bold values indicate the best FID score each NFF. Results for other than "Ours' are taken from Zheng et al. (2023a). As with other evaluations in this paper, we set { = 1 by default for CRS-e, and CRS-u,, except at NFE- 5, where was manually tuned to further improve performance, In this experiment, we observed that the importance of tuning & increases as the NFE decreases. We attribute this behavior to the discrctization error introduced by the contimoous-time approximation used in Eq.
Sampling
FID &
Model
Sampler
schedule
NFE 5
NFE
NFF
LDM
DPM-Solver++
Lincar
18.59
3.63
3.16
UniPC
12.24
3.56
3.07
DPM-Solver-v3 Zheng et al. 2023a
7.54
3.16
3.05
Ours
iPNDM
Lincar
10.81
3.71
2.64
CRS-u;
7.88
3.12
2.48
CRS-
8.07
3.45
2.50
CRS-vs: 8 = 1.4
7.64
CRS-V;: & = 2.2
6.96
Table 28: Detailed evaluation results LSUN Bedroom 256x256 in latent-space diffusion models at small NFE settings. Bold values indicate the best sampling schedule for each metric.
Sampling
Metrics
Sampler NFE
schedule
FID 4
sFID 4
Precision t
Recall f
iPNDM
Linear
2.64
6.15
0.53
0.57
CRS-v:
2.48
5.48
0.54
0.56
CRS-ux
2.50
5.41
0.55
0.56
10
Linear
3.71
6.88
0.52
0.54
CRS-2,
3.12
5.69
0.52
0.56
CRS-u,
3.45
6.15
0.54
0.53
Linear
10.81
13.82
0.42
0.49
CRS-u:
7.88
9.84
0.45
0.50
CRS-U,
8.07
10.43
0.45
0.48
CRS- { 1.4
7.64
9.23
0.45
0.52
CRS-V2: 2.2
6.96
8.65
0.45
0.49
Table 29: Comparison between our trained model and LDM-4 (Rombach et al. 2022 on ImageNet under
class-conditional generation without CFG. Nparams denotes the number of parameters in the U-Net model. Our model, trained with CRS-u, as the training schedule, achieves a better FID score than L.D-4 despite having fewer U-Net parameters. Model Training schedule
Sampling schedule
Sampler Stochastic DDIM
NFE
FID(J)
Nparams
LDM-4 Linear
Linear
250
10.56
400M
Ours
CRS-sz
9.67
296M
44

---

Published in Transactions on Machine Learning Research (01/2026)
Table 30: Performance latent-space diffusion models on ImageNet 256x256 at NEE 30, evaluated under various training and sampling schedules using stochastic samplers. Two samplers were used: Stochastic DDIM and -o-ee- Gray-shaded rows indicate variations in training schedules (with fixed sampling), while unshaded rows show variations in sampling schedules (with fixed training). Bold values indicate the best result for each evaluation metric.
Training Sampling
Metries
Sampler
schedule schedule
FID
sFID 1
Precision
Recall
Stochastic DDIM
Linear
Linear
27.31
21.39
0.46
0.55
VDM++
29.00
17.67
0.46
0.57
CRS-EFID
25.50
21.41
0.48
0.56
CRS-2,
26.97
20.71
0.47
0.56
CRS-s,
25.83
19.92
0.47
0.57
CRS-
Linear
25.83
19.92
0.47
0.57
EDM
49.72
50.08
0.31
0.51
CRS-UFID
20.85
13.01
0.52
0.56
CRS-ve
20.43
13.14
0.53
0.58
CRS-C,
17.88
10.14
0.55
0.59
SDE-DPM- +
Linear VDM+
Linear
11.12
5.68
0.62
0.62
29.00
17.67
0.46
0.57
CRS-UFID
10.22
5.81
0.62
0.64
CRS-2,
11.18
5.84
0.62
0.63
CRS-v,
10.25
5.64
0.62
0.64
CRS-ez
Linear
10.25
5.64
0.62
0.64
EDM
11.52
6.13
0.60
0.64
CRS-UFID
10.49
5.39
0.63
0.63
CRS-2
10.24
5.22
0.63
0.63
CRS- Up
10.56
5.32
0.63
0.63
45

---

Published in Transactions on Machine Learning Research (01/2026)
Table 31: Performance of latent-space diffusion models ImageNet 256 at NFE 30, evaluated under various training and sampling schodules using deterministic samplers. Three samplers were used: DDIM, PNDM, and DPM-Solver++ Gray-shaded rows indicate variations in training schedules (with fixed sampling), while unshaded rows show variations in sampling schedules (with fixed training). Bold values indicate the best result for each evaluation metric.
Training Sampling
Metrics
Sampler
schedule schedule
FID +
sFID +
Precision
Recall t
DDIM
Linear
Linear
17.33
6.79
0.55
0.62
VDM++
19.53
5.97
0.54
0.64
CRS-UFID
15.52
6.95
0.56
0.63
CRS-u;
17.02
6.94
0.55
0.63
CRS-U,
16.28
6.27
0.56
0.64
CRS-V,
Linear
16.28
6.27
0.56
0.64
EDM
22.17
9.22
0.49
0.62
CRS-UFID
15.51
6.72
0.56
0.63
CRS-V:
15.26
5.79
0.57
0.64
CRS-
15.28
5.67
0.57
0.64
PNDM
Linear
Linear
13.21
5.35
0.59
0.65
VDM++
16.01
6.31
0.57
0.66
CRS-UFID
11.42
5.31
0.59
0.66
CRS-v:
12.69
5.45
0.59
0.65
CRS-u,
12.29
5.38
0.59
0.66
CRS-
Linear
12.29
5.38
0.59
0.66
EDM
12.86
5.62
0.58
0.66
CRS-FFID
12.18
5.21
0.59
0.65
CRS-
12.57
5.24
0.59
0.66
CRS-uz
12.65
5.38
0.59
0.66
DPM-
Lincar
Lincar
13.87
5.41
0.59
0.64
Solver (2M)
VDM++
16.97
6.45
0.56
0.66
CRS-FFID
12.09
5.40
0.59
0.65
CRS-U:
13.41
5.54
0.58
0.65
CRS-U,
12.92
5.46
0.59
0.66
CRS-vu
Linear
12.92
5.46
0.59
0.66
EDM
13.67
5.79
0.58
0.66
CRS-VFID
13.02
5.19
0.59
0.65
CRS-
13.02
5.17
0.59
0.65
CRS-
13.33
5.13
0.59
0.65
46

---

Published in Transactions on Machine Learning Research (01/2026)
Table 32: Performance of latent-space diffusion models on ImageNet 256x using stochastic samplers,
evaluated under different sampling schedules. The training schedule was fixed to CRS-ca- Bold values indicate the best sampling schedule for each evaluation metric.
Sampling
Metrics
Sampler Stochastic DDIM
NFE
schedule
FID
sFID 4
Precision
Recall
50
Linear
15.61
8.97
0.57
0.60
EDM
25.66
19.17
0.46
0.57
CRS-UFID
13.67
6.86
0.60
0.60
CRS-U; 13.44
7.56
0.59
0.61
CRS-uz 13.05
6.45
0.60
0.61
30
Linear
25.83
19.92
0.47
0.57
EDM
49.72
50.08
0.31
0.51
CRS-UFID
20.85
13.01
0.52
0.56
CRS-u:
20.43
13.14
0.53
0.58
CRS-v,
17.88
10.14
0.55
0.59
20
Linear
44.15
42.96
0.34
0.53
EDM
78.05
99.25
0.20
0.44
CRS-VFID
30.28
24.26
0.44
0.54
CRS-u,
31.86
26.34
0.43
0.55
CRS-U,
27.20
20.09
0.47
0.56
SDE-DPM-
50
Linear
9.96
5.62
0.63
0.64
D +(2M)
EDM
9.72
5.35
0.63
0.64
CRS-UFID
10.26
5.43
0.63
0.63
CRS-u;
9.68
5.33
0.63
0.64
CRS-u,
10.11
5.32
0.63
0.64
30
Linear
10.25
5.64
0.62
0.64
EDM
11.52
6.13
0.60
0.64
CRS-UFID
10.49
5.39
0.63
0.63
CRS-u,
10.24
5.22
0.63
0.63
CRS-V,
10.56
5.32
0.63
0.63
20
Linear
11.47
5.81
0.61
0.63
EDM
16.26
9.38
0.55
0.63
CRS-UFID
11.10
5.50
0.63
0.62
CRS-u,
10.83
5.36
0.62
0.63
CRS-U,
11.64
5.26
0.63
0.62
47

---

Published in Transactions on Machine Learning Research (01/2026)
Table 33: Performance of latent-space diffusion models on ImageNet x56x256 using deterministic samplers,
evaluated under different sampling schedules. The training schedule was fixed to CRS-U,- Bold values indicate the best sampling schedule for each evaluation metric.
Sampling
Metrics
Sampler DDIM
NFE
schedule
FID 4
sFID 4
Precision f Recall
50
Lincar
13.91
5.37
0.58
0.65
EDM
16.25
6.24
0.55
0.65
CRS-FFID
13.45
5.36
0.58
0.64
CRS-us
13.64
5.20
0.58
0.65
CRS-
13.75
5.12
0.59
0.65
Linear
16.28
6.27
0.56
0.64
EDM
22.17
9.22
0.49
0.62
CRS-FFID
15.51
6.72
0.56
0.63
CRS-ua
15.26
5.79
0.57
0.64
CRS-
15.28
5.67
0.57
0.64
20
Linear
20.49
8.82
0.51
0.62
EDM
33.08
15.78
0.40
0.60
CRS-UFID
18.43
7.52
0.54
0.63
CRS-v,
18.30
7.57
0.54
0.63
CRS-s,
18.16
7.05
0.55
0.62
PNDM
50
Linear
12.10
5.32
0.59
0.66
EDM
12.53
5.49
0.59
0.66
CRS-FFID
11.86
5.20
0.59
0.65
CRS-u,
12.35
5.28
0.59
0.66
CRS-vz
12.44
5.28
0.59
0.66
30
Linear
12.29
5.38
0.59
0.66
EDM
12.86
5.62
0.58
0.66
CRS-UFID
12.18
5.21
0.59
0.65
CRS-v,
12.57
5.24
0.59
0.66
CRS-
12.65
5.38
0.59
0.66
20
Linear
12.83
5.52
0.59
0.66
EDM
13.99
5.99
0.58
0.66
CRS-UFID
12.97
5.38
0.58
0.66
CRS-s:
12.92
5.30
0.59
0.65
CRS-s,
13.63
6.10
0.57
0.66
DPM-
50
Linear
12.42
5.41
0.59
0.66
+
EDM
12.90
5.57
0.59
0.66
CRS-EFID
12.27
5.15
0.59
0.65
CRS-s,
12.64
5.23
0.59
0.66
CRS-E
12.75
5.21
0.59
0.65
30
Lincar
12.92
5.46
0.59
0.66
EDM
13.67
5.79
0.58
0.66
CRS-FFID
13.02
5.19
0.59
0.65
CRS-U
13.02
5.17
0.59
0.65
CRS-
13.33
5.13
0.59
0.65
20
Linear
13.71
5.45
0.58
0.65
EDM
15.31
6.23
0.57
0.66
CRS-FFID
14.10
5.25
0.59
0.65
CRS-
13.70
5.13
0.59
0.65
CRS-uz
14.16
5.09
0.59
0.64
48

---

Published in Transactions on Machine Learning Research (01/2026)
shutterstack
JERE DO
Figure 14: Generated samples of LSUN Horse 256x256 using a pixcl-space diffusion model at NFE = 250 (FID 2.03). The training and sampling schedules were optimized using CRS- and CRS-s, UPID respectively. SDE-DPM--SOve++++22M was used as the sampler.
49

---

Published in Transactions on Machine Learning Research (01/2026)
Figure 15: Generated samples of LSUN Horse 256x256 using a pixel-space diffusion model NFE 50 (FID 2.04). The training and sampling schedules were optimized using CRS- and CRS-s, triD. respectively. PNDM was used the sampler.
50

---

Published in Transactions on Machine Learning Research (01/2026)
Figure 16: Generated samples of LSUN Bedroom 256x256 using a pixel-space diffusion model at NFE 10 (FID 3.30). The training and sampling schedules were optimized using CRS-U, and CRS-s, triD. respectively. UniPC was used as the sampler.
51

---

Published in Transactions on Machine Learning Research (01/2026)
Figure 17: Generated samples of LSUN Bedroom 256 using a pixcl-space diffusion model at NFE = 5 (FID 14.02). The training and sampling schedules were optimized using CRS-E, and CRS-U, + UFID respectively. (2M)))) was used as the sampler.
52

---

Published in Transactions on Machine Learning Research (01/2026)
erutree fbuuuss
CORDAOEO
SNATTOT NWTO www.
ck
Figure 18: Generated samples of LSUN Church 256 using a latent-space diffusion model at NFE 20 (FID 3.89). The training and sampling schedules were optimized using CRS- and CRS- respectively. (2M) was used as the sampler.
53

---

Published in Transactions on Machine Learning Research (01/2026)
Figure 19: Generated samples of LSUN Bedroom 256x256 using a latent-space diffusion model NFE 20
(FID= = 2.71). The training and sampling schedules were optimized using CRS-u, and CRS-U, respectively. PNDM was used as the sampler.
54

---

Published in Transactions on Machine Learning Research (01/2026)
0125
Figure 20: Generated samples of ImageNet 256 256 using a latent-space diffusion model at NFE 20 with class condition (FID = 10.83). Six classes were randomly selected, and cach row corresponds to one class. The training and sampling schedules were optimized using CRS-u, and CRS-e,, respectively. SDE-DPM- (2M)) was used the sampler.
55


Reviews:
[{"review_id": "wLtDVjo4F0", "reviewer": "Reviewer_CJ5W", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the sampling efficiency of diffusion models without sacrificing image quality or mode coverage. The authors propose Constant Rate Scheduling (CRS), a method to optimise the noise schedule (the variance added at each diffusion step) so that the probability distribution of the data undergoing diffusion changes at a constant rate throughout the process. The paper presents theoretical motivations for the proposed method. Through extensive experiments on multiple datasets (LSUN, ImageNet, FFHQ) and both pixel-space and latent-space diffusion models, the paper finds that CRS consistently improves generation performance (lower FID, higher precision/recall), demonstrating the approach’s effectiveness.", "claims_and_evidence_yn": "Yes", "claims_explanation": "1. The paper presents strong evidence on performance improvement. This is supported by extensive results on a variety of datasets and settings. We can see consistency across multiple samplers (Stochastic DDIM, SDE-DPM-Solver++(2M), DDIM, PNDM, DPM-Solver++(2M), UniPC), and the paper reports precision/recall alongside FID rather than cherry-picking a single metric.\n\n2. The paper claims that CRS helps both training and sampling, which is well supported in both latent models (LSUN Church, Bedroom, ImageNet), and pixel models (LSUN Horse, Bedroom).\n\n3. The paper claims about generality and arbitrary distance metrics, which is a bit complecated. No non-image domains were tested, and only three metrics were tried. So the “any distance metric” / “expected to work beyond images” parts are unvalidated; however, the authors relegate this to future work in the Limitations section.\n\n4. The code is not submitted / released. I encourage the author to expand on this in their response.", "audience_yn": "Yes", "audience_explanation": "1. This paper studies diffusion models which are central in machine learning.\n\n2. It addresses a real, widely felt pain point in diffusion models.\n\n3. The paper presents conceptual/theoretical interest, not just experiments. Also, the experimental results are impressive.", "requested_changes": "1. Reproducibility and hyperparameter specification for CRS. CRS introduces several hyperparameters, some of which are scattered in the appendix and/or briefly mentioned but not fully specified per experiment. Please include:\n\n- How v(α) was computed (T, number of samples, feature network used).\n- Exact ξ, w_x, w_FID, any clipping/thresholds, EMA parameters.\n- Dataset-specific tuning rules (e.g., different R or ξ for ImageNet).\n\n2. Computational overhead. The paper mentions that computing v_FID and v_x adds offline and/or training-time cost, but the discussion is fairly high-level. So, please add a short subsection (or expand the current discussion) that reports approximate compute overhead for:\n- v_FID estimation (per dataset, relative to one training epoch).\n- v_x estimation during training (e.g., % increase in training time).\n\n3. Discuss choice of distance metric. CRS heavily relies on the metric it uses. It would be great to discuss how to find or learn better metrics/embeddings that can capture distribution changes of diffused data. The authors suggest using self-supervised feature learning on noised data as one idea; please expand on this.\n\n4. CRS does not guarantee an optimal final model in theory – it’s a heuristic guided by distribution change, not a direct optimisation of the model’s loss or sample quality metric. While it correlates well with better performance (as seen empirically), there’s no formal proof.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "eGWsxrEaEx", "reviewer": "Reviewer_T49E", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a general framework called Constant Rate Scheduling (CRS) for optimizing the noise schedules in diffusion models, applicable to both training and sampling. The core idea is to design a schedule that enforces a constant rate of change in the probability distribution of the diffused data throughout the diffusion process. This rate of change is quantified using a flexible, user-defined discrepancy measure.\n\nThe authors introduce three practical discrepancy measures: one based on FID ($v_{FID}$), one on data prediction ($v_x$), and one on noise prediction ($v_\\epsilon$). These measures can also be combined to leverage their complementary strengths.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The primary claim—that optimizing noise schedules via the CRS framework leads to improved performance (e.g., FID, sFID, Precision/Recall) compared to conventional schedules—is convincingly supported by evidence. The authors provide extensive results across pixel-space and latent-space models, multiple datasets, and a wide range of samplers and NFE settings (from 5 to 250). The ablations on different discrepancy measures (Tables 1, 3, etc.) and their combinations (e.g., $CRS-v_{x}+v_{FID}$) effectively demonstrate the framework's utility.", "audience_yn": "Yes", "audience_explanation": "The paper addresses the optimization of noise schedules for diffusion models , which is an active, fundamental, and important area of research within the machine learning community. The TMLR audience is centrally concerned with such foundational improvements to state-of-the-art generative models.\n\nThe proposed Constant Rate Scheduling (CRS) framework is a novel contribution that provides a general and unified method for optimizing both training and sampling schedules. This unification is a conceptual step beyond prior work, which often focused on optimizing only one of the two.\n\nThe paper provides extensive empirical evidence that this framework is not just a theoretical curiosity but delivers practical, consistent performance gains across a wide variety of settings", "requested_changes": "1. Address the Discrepancy in the $\\Delta t$ Assumption: The derivation of the CRS condition in Eq. (7-8) is based on a first-order approximation that assumes $\\Delta t$ is very small. However, the method shows strong performance in low-NFE regimes (e.g., NFE=5 or 10), where $\\Delta t$ is large, and the approximation error should be significant. This is a key theoretical gap. The authors must add a discussion addressing this point. Why does the framework, derived from a continuous-time perspective, still hold in the highly discrete, few-step sampling setting?\n\n2. Expand Experimental Scope to Justify \"Generality\": The paper claims to be a \"general framework\", but its generality is not sufficiently proven. The validation is limited to $\\epsilon$-prediction models with U-Net backbones. To better support this central claim, we request the authors.  \n  a. Investigate Classifier-Free Guidance (CFG): The class-conditional experiments (e.g., on ImageNet ) presumably use CFG, but this is not mentioned. The interaction between noise schedule and guidance strength is a critical factor in practice. The authors should at least conduct a study showing how the optimal schedule and its performance gains are affected by varying the CFG scale   \n  b. Test on Other Parameterizations/Architectures: To truly demonstrate generality, the framework should be validated on other modern setups, such as $v$-prediction and Diffusion Transformer (DiT) architectures. \n\n3. Analyze Training Convergence Speed: The motivation suggests that by making the distribution change constant, CRS provides more reliable local updates, which implies an \"easier\" learning problem. A natural hypothesis from this is that models trained with CRS (e.g., $CRS-v_x$) might converge faster than those trained with standard schedules (like linear or VDM++). It would be a strong addition to the paper to include a plot of FID or loss curves over training steps/epochs to confirm if this is the case.\n\n4. Clarify CIFAR10 Results: The FID of 1.89 reported for CIFAR10 in Table 29  (NFE=50, DPM-Solver++) seems to underperform the original EDM paper, which reported an FID of 1.79\n\n5. Mathematical Phrasing (Eq. 7): Please refine the language around Eq. (7). As written, it implies Eq. (7) is the solution to Eq. (6). More accurately, Eq. (7) (i.e., $D(t, t+\\Delta t) = const.$) is the condition that the optimal schedule must satisfy to solve the min-max problem in Eq. (6)\n\n6. Autoencoder Model (Sec 5.3): The paper states a \"pre-trained $VQ-4$ autoencoder from LDMs\" was used. This is incorrect; the LDM paper used a KL-regularized autoencoder (not VQ). Please correct this to KL-f4 (or the appropriate term). As a follow-up, please add a brief justification for using an f=4 autoencoder ($64 \\times 64$ latent) for $256 \\times 256$ images, as f=8 ($32 \\times 32$ latent) is also a very common choice.\n\n7. ImageNet Performance: The FID scores on ImageNet seem relatively high for a 500-epoch training run with this model size. This does not invalidate the relative improvement shown by CRS, but it would be good to briefly acknowledge if the baseline models are not fully converged and how this might affect the conclusions.\n\nMissing References\n- Scaling Rectified Flow Transformers for High-Resolution Image Synthesis\n- On the Importance of Noise Scheduling for Diffusion Models\n- Flow Matching for Generative Modeling\n- Improved Noise Schedule for Diffusion Training", "broader_impact_concerns": "None. This paper presents a foundational algorithmic improvement for generative models. This work does not introduce any new, specific ethical risks", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fC5Ewobzqu", "reviewer": "Reviewer_DmXo", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes Constant Rate Scheduling (CRS), a framework for designing both training and sampling noise schedules so that the rate of distributional change between adjacent timesteps is (approximately) constant. The authors minimize the discrepancy $D(t, t + \\Delta t)$ between diffused data distributions such that the final schedule $ -d\\alpha(t)/dt = C v(\\alpha)^{-\\xi}$  with $\\xi>0$. The authors demonstrated several ways to instantiate $v(\\alpha)$ (using vFID, vx, and v$\\epsilon$). The authors applied the framework to LSUN horses/church, and they claim fairly consistent gains. \n\nStrengths:\n1. The work's general direction is highly relevant to diffusion models and optimizing trajectories.\n2. The proposed recipe is easy to follow and is unified in training and in inference. The authors adapted multiple complementary discrepancy measures which can be used in conjunction.\n3. The empirical ablation studies are generally extensive (with a few catches) and show general improvements over baselines.\n\nWeakness:\n1. I think the method may not be broadly applicable (vFID is very expensive, and it appears the method is highly sensitive to hyperparameters)\n2. The paper does not justify (from a theoretical perspective) why such distributional changes relate to sample quality. \n3. Some of the baseline comparisons do not strictly match the previously published results.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The core claim is that $ -d\\alpha(t)/dt = C v(\\alpha)^{-\\xi}$ improves training/sampling by allocating steps where distributional change is the fastest. This is reasonably convincing from both logical and empirical perspectives. Nevertheless, I would expect more rigorous justifications (e.g. one could argue that different regions have different importance, where high noise is for structure formation and low noise is for finer details, which would warrant very different rates). Additionally, I think there can be more/better apple-to-apple comparisons with AYS/GITS/LDS as the existing results appear heavily hand-tuned.", "audience_yn": "Yes", "audience_explanation": "There is a significant number of TMLR readers who work on diffusion models, schedulers, and efficient sampling/training, and I expect such an audience to have an interest in this paper. Generally, this paper is well-positioned within the community's ongoing effort to optimize denoising trajectories, with interesting empirical results and some potential for other practitioners to try it out.", "requested_changes": "Critical changes:\n1. The paper requires more comparisons with schedule-optimization baselines under similar training settings and across more tasks. At least one more baseline and one more task should be shown (e.g. AYS, GITS, and potentially LD3 (ICLR'25) with either CIFAR10 or FFHQ). I would also be interested in seeing more empirical examples compared against baselines.\n2. The readers will require a better explanation on how to best choose the hyperparameters. Generally, I believe the tuning budgets should be standardized across baselines.\n3. There are a series of minor mistakes, e.g. incorrect citations to Feller 2015 (retracted chapter?), eq. 7 has a small approximation error in the first-order Taylor expansion, especially at low NFEs.\n\nImportant but not critical:\n1. Theoretical analysis beyond simple intuition will greatly strengthen the paper. I would be interested in seeing a formal connection between distributional change and sample quality. \n2. I would appreciate additional discussion on scalability or broader applicability of the method, especially given so many knobs to tune.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)