Improved Sampling Algorithms for Lévy-Itô Diffusion Models
Authors: Vadim Popov
Assel Yermekova
Tasnima Sadekova
Artem Khrapov
Mikhail Sergeevich Kudinov
Venue: ICLR 2025 Poster (2025)
Keywords: generative modeling
diffusion models
Lévy-Itô models
α-stable Lévy processes
stochastic differential equations
Primary area: generative models

Abstract: Lévy-Itô denoising diffusion models relying on isotropic α-stable noise instead of Gaussian distribution have recently been shown to improve performance of conventional diffusion models in image generation on imbalanced datasets while performing comparably in the standard settings. However, the stochastic algorithm of sampling from such models consists in solving the stochastic differential equation describing only an approximate inverse of the process of adding α-stable noise to data which may lead to suboptimal performance. In this paper, we derive a parametric family of stochastic differential equations whose solutions have the same marginal densities as those of the forward diffusion and show that the appropriate choice of the parameter values can improve quality of the generated images when the number of reverse diffusion steps is small. Also, we demonstrate that Lévy-Itô diffusion models are applicable to diverse domains and show that a well-trained text-to-speech Lévy-Itô model may have advantages over standard diffusion models on highly imbalanced datasets.

Published as a conference paper at ICLR 2025
IMPROVED SAMPLING ALGORITHMS FOR LÉVY-ITÒ DIFFUSION MODELS
Vadim Popoy, Assel Yermekova, Tasnima Sadekova,
Huawei Noah's Ark Lab {popov. vadiml, yermekova assell, sadekova. hataaaaaaaaaaoootttoooaaaaaaaaaa
Artem Khrapov & Mikhail Kudinov
Huawei Noah's Ark Lab khrapov. art w www.aaawwwwaattteesse..ee com, kucinov.mikhhalleuarrees
ABSTRACT
Lévy-Itô denoising diffusion models relying on isotropic a-stable noise instead Gaussian distribution have recently been shown to improve performance conventional diffusion models in image generation on imbalanced datasets while performing comparably in the standard settings. However, the stochastic algo- rithm of sampling from such models consists in solving the stochastic differential equation describing only an approximate inverse of the process of adding a-stable noise to data which may lead to suboptimal performance. In this paper, de- rive parametric family of stochastic differential equations whose solutions have the same marginal densities as those of the forward diffusion and show that the appropriate choice of the parameter values can improve quality of the generated images when the number of reverse diffusion steps small. Also, we demonstrate that Lévy-Itó diffusion models are applicable to diverse domains and show that a well-trained text-to-speech Lévy-Itô model may have advantages over standard diffusion models on highly imbalanced datasets.
INTRODUCTION
Denoising diffusion probabilistic models (Ho al. 2020) are a powerful class of generative models capable of solving tasks related to various continuous domains such as natural images Dhhrriwal & Nichol 202T: Gao et al. (2023), video (Luo al., (023), speech (Popov et al. 2021 Chen al.
2021) and music (Hawthorne et al.) 2022) to name few. Since they were first introduced, there have been numerous attempts to overcome some of their drawbacks such as difficulty of applying them to discrete domains (Lou et al.) 2024) and their inefficiency coming from iterative sampling algorithm. The latter problem has drawn much attention researchers which has led to many successful solutions like the frameworks of flow matching (Lipman et al. 2023), consistency models (Song et al. 2023) Song & Dhariwal (2024) and distribution matching distillation (Yin et al. (024b]aa a) At the same time, other weaknesses diffusion models have received relatively little attention. For example, conventional diffusion models having Wiener process as their driving one show substantial quality degradation imbalanced datasets (Qin et al. 2023), and there has been recent attempt to address this problem by replacing Wiener process with a-stable Lévy process (Yoon et al. 2023). Such models called Lévy-Itô denoising diffusion models (LIMs) were shown to better train on imbalanced datasets achieving, in particular, better performance LIMs allowing to demonstrate the superior behaviour in this scenario consists in relying on Lévy processes with discontinuous paths and employing isotropic a-stable noise with heavier tails than those Gaussian distribution both at training and inference.
rare classes. The advantage of
Yoon al. (2023) have devised necessary techniques allowing to treat Lévy-Itô models through continuous time formalism based on stochastic calculus similar to Song et al. (2021c) who were the first to make it for common diffusion models. Particularly, the authors introduced the notion of fractional score function and developed fractional denoising score matching technique to estimate fractional score function with neural network. Also, they designed forward process bearing

---

Published as a conference paper at ICLR 2025
resemblance to Variance Preserving diffusion (Song al. c021c) consisting in adding isotropic a-stable noise to data until turns to pure noise from the prior which, in this case, is a standard symmetric a-stable distribution. This process equation (SDE (Oksendal 1992)) and has a reverse-time model described by the reverse SDE written down in terms of the fractional score function containing information about data distribution and some intractable data-dependent term skipped when solving the reverse SDE. Thus, the SDE used inference does not lead exact solution (i.e. marginal probabilities of the underlying process at every diffusion time step & are different from those of the forward diffusion). Two sampling algorithms were suggested by] Yoon al. )023): the stochastic one by solving the mentioned SDE and the deterministic one by solving (2021c). contrast with the former differential equation, the latter one relies only on the fractional score function and does not contain intractable terms we have to omit at inference so it provides exact solution. At the same time, both stochastic and deterministic sampling methods of standard diffusion models are exact (both corresponding ODE and SDE rely only on the score function) and, moreover, there exists a parametric family of sampling algorithms including the deterministic one as particular case (Song al.] 2021a). In this paper, bridge the gap between conventional diffusion models and LIMs and derive similar family of SDEs leading to exact sampling and providing an option choose amount of noise at each reverse diffusion step. This possible to improve performance of Lévy-Itô Itô models when the number of function evaluations (NFE) limited by small number, and, as we show through empirical studies on image generation task, this improvement does not come at the cost of samples diversity, thus keeping the main advantage of LIMs intact. Figure Tillini Iillustrates a forward Lévy diffusion and reverse denoising Lévy diffusions for various reverse SDES we propose.
be described by the forward stochastic differential
analogue of what is called probability flow ODE in Song et al.
reverse dynamics makes
As we mentioned, the literature on conventional diffusion models offers a wide range possible applications that so far remain almost unexplored for LIMs. In this work, we also investigate applicability Lévy-Itō diffusion models to speech domain. To demonstrate possible advantages of LIMs over common diffusion models, train Lévy-Itô-be basebased text-to-speech models imbalanced dataset with the amount data representing different speakers varying significantly and study capability of different models produce speech of "rare" and "frequent" speakers.
Our main contributions are threefold:
We derive a parametric family reverse SDEs relying only on the fractional score function whose solutions have the same marginal densities as those of the forward SDE unlike the reverse SDE proposed the original paper.
We demonstrate the benefits of using these SDEs at inference terms of generated samples quality on image generation task and verify that samples diversity does not suffer if we generate data with the proposed SDEs.
We train Lévy-Itô text-to-speech model on a highly imbalanced dataset and evaluate its performance for speakers with different amount of training data.
100
030 325
650
075
140
675
650
045
Figure 1: 1-dimensional Lévy processes with 1.5. Horizontal axis stands for diffusion time t, vertical axis for values of process. Left part of each plot (time t increasing from 0 to 1) is the forward diffusion. Right part (time decreasing from 0) is reverse diffusion with different amount of noise: (left) the same in the forward diffusion (7/ 1.0); (middle) less than in the forward diffusion (7)- 0.2); (right) no noise (N) = 0.0) meaning that trajectories are continuous.
2

---

Published as a conference paper at ICLR 2025
2
RELATED WORK
Denoising diffusion probabilistic models belong to class of energy-based generative models (Song &
Kingma 2021). Diffusion models were first introduced as discrete-time models (Ho et al. 20200. but
more flexible continuous-time approach was proposed later (Song et al. 2021c Song & Ermon 2020
allowing to develop theoretical basis under diffusion training procedure via denoising score matching
(Song et al. [2021b; Hyvärinen 2005) thanks to the fact that the forward diffusion admits reverse- time model [Anderson] 1982). Diffusion models and closely related flow matching models (Lipman et al. 2023) are very popular due their remarkable performance in common generative tasks (Dhariwal Nichol [021 Gao et al., 2023) well as rich capabilities of controllable generation, e.g. editing an image based on various conditions (Meng et al. 2022), generating speech with the desired pitch (Sadekova et al. 2024), or putting more emphasis on input condition via classifier-free guidance (Rombach et al. 2022).
2023 are perhaps the most successful attempt to replace Gaussian noise in energy-based models with alternative distributions, c.g. Gamma distribution in common diffusion models (Nachmani et al. 202T), heavy-tailed distributions in annealed Langevin dynamics Deasy et al.) 2022). Despite Lévy processes (Applebaum) (2004) have lots of applications, e.g. in finance (Eberlein (200T) where they help to better capture dynamics of asset prices due to flexibility in choosing jump distribution (Geman 2002). their application in score-based generative modeling is still very limited. Apart from the original paper on these models, one can mention Paquet et al. (2024) applying LIMs to protein generation, Shariatian et al. (2024) developing a discrete-time version of LIMs, and Hu et al. (2024) employing Physics-Informe Informed Neural Networks to solve fractional partial differential equations in more general setting than the one studied by Yoon
Lévy-Itô diffusion models (Yoon et al.
al. 2023).
Increasing efficiency of diffusion models based on Gaussian noise is a well-studied topic. Like the algorithm we propose in this paper for LIMs, some these methods do not involve model fine-tuning and utilize various numerical methods of solving SDEs (Kloeden & Platen 1992). For example, Lu et al. ) and Zhang & Chen (2023) make use of specific structure of reverse SDEs typically appearing diffusion modeling derive solvers with smaller numerical errors. Some researchers take into account both reverse and forward diffusions and design their solvers accordingly (Bao et al. 2022 Poppv et al., 2022). There solvers tailored specifically for inference with classifier guidance (Wizadwongsa & Suwajanakorn 2023). Although methods of such kind lead to significant quality improvement for small NFE, even better efficiency can be achieved with methods requiring fine-tuning. One of the first attempts was progressive distillation (Salimans & Ho 2022) followed by consistency modeling (Song al., 2023) Song & Dhariwal 2024) whose main idea is that a diffusion model performing well with a few steps should give consistent predictions of clean data on the same ODE trajectories used at inference. Distribution distillation (Yin et al. 2024bb a) involving Generative Adversarial Networks (Goodfellow et al. 2014) in model fine-tuning is recent alternative to consistency models.
LÉVY-ITÔ MODELS
We begin this section with a quick recap of isotropic a-stable random variables and Lévy processes as well as some basic notions from the fractional calculus. Then, general framework of Lévy-Itô diffusion models is described.
3.1 ISOTROPIC a-STABLE DISTRIBUTION
vellev d-dimensional random variable & comes from the distribution Sa.S ), i.c. isotropic a-
stable distribution with scale parameter 7 and zero mean, if its characteristic function is Ee'sug>
e-"" Parameter must belong to (0. 2] and in general case distribution Sa. (7) does not have explicit form for its probability density function. A notable exception a = 2 in which case we have Gaussian distribution (0, / I) where is d-dimensional identity matrix. Isotropic a-stable distribution is heavy-tailed one except for the case 2, and, moreover, has infinite variance for a < 2 (Paulson et al. 1975). An important property of SaS (7) is that is infinitely divisible which makes it proper candidate to build a Lévy process upon as we discuss next.
3

---

Published as a conference paper at ICLR 2025
Figure 2: 1-dimensional Brownian motion (left) and Lévy processes I, with 1.5 (middle) and 1.2 (right). Horizontal axis stands for time t, vertical axis - for values of process. Brownian motion trajectories are almost surely continuous while trajectories a-stable Lévy processes are discontinuous with large jumps more probable for smaller values of a.
3.2 LÉVY PROCESSES
real-valued d-dimensional stochastic process L, defined for i 2 0 starting almost surely (Lo = 0 a.s.) is said to be a Lévy process (Applebaum) 2004) if
L has stationary increments, i.c. Law(L Li) Law(Lc) for all h >> 0;
Le has independent increments, i.e. L, Lg and L, L, are independent for all
0 I <IS
. L, has càdlàg (i.e. right continuous with left limits) paths.
This definition implies that the increments of Lévy processes must be infinitely divisible. so we can choose isotropic o-stable distribution defined above as the one Lévy process increments belong to. In what follows we consider a-stable Lévy processes LE such that Law(L;) = (t for (1, It can be shown that, in contrast standard Brownian motion, o-stable Lévy processes I. have discontinuous paths, and as a decreases, large jumps become more probable. These properties illustrated in Figure 2
Probability density functions of diffusion processes based on Li satisfy Fokker-Planck equations. but unlike standard diffusions based Wiener process they are not usual partial differential equations (PDEs) as Laplace operator is replaced with the fractional Laplacian (Lischke et al. (0200) when 2. To be more precise, diffusion X; satisfies the following SDE:
dX, t)dt + o,dL?
(1)
where X,- lim X then under certain assumptions on drift and diffusion coefficients (xx t) and o, (Schertzer al. (200I) probability density function p(z, t) of the process X, satisfies the fractional Fokker-Plande PDE
i ar
p(z, t) = - (u(x, ()(xx b)) of --
(2)
with the proper initial conditions Yoon et al. 2023). As one can see by comparing this fractional PDE with the common Bocker-Plande PDE, the fractional Laplacian of order a/2 denoted by (-A)a/ plays the role of (negative) Laplacian when we change diffusion driving process from standard Wiener process to a-stable Lévy process with a < 2.
The fractional Laplacian a pseudo-differential operator defined through generalization of Fourier multiplier property of the negative Laplace operator:
(-A)a/2
F (I
where F and FF1 denote Fourier and inverse Fourier transforms correspondingly. For the cases < a and < a < 2 there are alternative expressions without Fourier transforms through just singular integral over Rdd (Lischke et al. 2020).
3.3 LÉVY-ITŌ DIFFUSION MODELS
Consider adding a-stable noise to data Xa according to the forward SDE T on the time interval [0, T] for any reasonable drift and diffusion coefficients such that for the final time T have
4

---

Published as a conference paper at ICLR 2025
Law(Xy Sa.5 Le. the prior is standard a-stable distribution with the unit scale parameter. Yoon et al. (2023) provided a reverse-timn model of a process satisfying the forward SDE (T). It is given by the SDE solved backwards in time from t = T to f 0 starting from Law Law(Xr):
dXx, = (m( (u(Xx+,() o
0 (X++)) (X++) )dd + o,dLi + d2.,
where X,- limp- X(th, Lę is a-stable Lévy process in reverse time (meaning that its in- crements L - Lg are independent of L for < t) and Z, is a reverse-time model of some data-dependent process Ze with zero mean and finite variation. The process Z₃ is intractable since it can be defined only through its characteristic exponent (see Theorem B.1 in Yoon et al. (2023). formula (39)) depending on data density ratios. The reverse SDE is expressed in terms of the fractional score function s(() (I) defined as
Vp(z,t) (Dl) p(r,t)
(x)
(5)
where p(r, i) is the probability density function of the forward diffusion ① at time
During training the goal is to approximate this fractional score function by a neural network. Yoon al. (2023) proposed train score-matching neural network se (2x,) t) with parameters by optimizing the fractional denoising score matching objective:
C(O, t) x. ||s@ (X.,
(X. (x
(6)
where conditional fractional score function is calculated using density p(r oo. t) of conditional distribution Law(X:|Xo)):
(x( = (-A) Vp(x/xo,t)
(())
If drift coefficient fa 1) is linear in x, then it can be shown that for some functions ar and Y
conditional distribution X, X is the same as ar Xo + Sa.5(()) which allows compute training target explicitly:
T 0,20
(8)
ar
There are two ways of sampling from the trained Lévy-Itô diffusion model: either by solving the SDE
dx. = (u((e+,)) - s c(0)) (X++) )dtt) + o,
(9)
or by solving the probability ODE
dX; o5 (Xt))dt
(10)
with the fractional score function replaced with its neural network approximation. Both differential
equations should be solved backwards in time starting from the prior Sa). As for the SDE 9 is an approximate version of the exact reverse SDE (4). Yoon et al. (0023) justified omitting d2, term by the fact that 7, is random process with zero mean and finite variation while T. has infinite
variation. As far as the ODE (IO) is concerned, it leads to exact sampling procedure since marginal
probabilities of the solutions 10) and (I) are the same as proven in Yoon al. (2023)
PARAMETRIC REVERSE-TIME SDE
Conventional diffusion models allow for parametric family of sampling algorithms (Song et al. 2021a Kong & Ping 2021). The parameter in these algorithms stands for amount of noise added at each inference step, and both the probability flow ODE and the reverse-time SDE are particular cases obtained for specific parameter values. The following theorem provides similar result for Lévy-Itô Itô diffusion models:
Theorem 1. Consider stochastic process X, with marginal probability densities p(æ,t) t) driven by s-staele Lévy process Lę and described by the forward SDE ( Under certain regularity assumptions drift and diffusion coefficients p(x. 1) and a non-negative dimeddepnnent function
5

---

Published as a conference paper at ICLR 2025
te and the fractional score function S (x) given by the formula 5 the following reverse SDE driven by reverse-fime ox-stable Lévy process L4
S (X++))dt + a. 1/o a//aii has the solution with the same marginal probability densities p(z, 8) as the forward process X, given that its starting point XT has the same distribution as XT.
dx, = (u(X 1)
(11)
The proof of this theorem essentially relies inspecting fractional Fokker-Plance PDEs for the SDEs (1) and (11) and can be found in Appendix A as well as a brief discussion on the necessary assumptions on (xx, t). or, T) and ST (z). Note that these assumptions are basically the necessary conditions for the Fokker- Planer-----ck----- equations associated with the SDEs (I) and (TI) to have unique solutions (Schertzer et al. 2001 Albeverio et al., 2010 Zhang et al. 2020), and it is quite common to make such kind of assumptions not only when studying diffusion generative models (Song et al., 2021b general Anderson
Yoon al. 2023). but also when studying reverse-ting timee models of stochastic processes in 1982.
Once we have trained the fractional score-matching network so (r. t), we can generate samples by solving the SDE (T1) backwards in time starting from the prior. The parameter ?) controls amount of a-stable noise added at each reverse diffusion time step t and 17, 1/a can be thought of the ratio between the amounts of noise in the forward and reverse diffusions given by (1) and (T]: when " = 1, then we add exactly the same amount of noise as in the forward diffusion, for f) < we add less noise which in some cases may be useful, and when 1/2 = O we do not add any noise, in which case the SDE (T) coincides with the ODE (TO) and its trajectories become continuous (since all jumps come solely from a-stable noise). Figure illustrates this argument.
is worth noting that, contrast to the SDE (4) the reverse SDEs I do not provide a reverse-time model of the forward process X, because distributions of the trajectories of (I) and (TT) are different in general as seen from Figure [ The main advantage of these SDEs is that they do not rely on intractable terms and can be used readily at inference providing exact marginal densities for each time given that the fractional score- mstre-mat network is trained till optimality. As for the SDE cannot use it directly and have to use the approximate SDE instead.
The approximation (9) of the SDE 4 is supposed to work well since it differs by term dZ. where Z, is the inverse of the process Ze with zero mean and finite variation which negligible since a-stable Lévy process has infinite vari- ation. However, this argument may not work well when we use small NFE to sample from Lévy-Itô model.
Finite or infinite variation is a property of the paths of a random process Y, defined by whether Yr, over all partitions II of the inter- val [0, T (i.e. 0 = lo < L1 <... < 1n-1 < In = T) is finite or infinite almost surely. In practice, when we solve SDEs on a partition with relatively large step sizes, the variation of trajectory may be far less than its supremum which may possibly be reached on partitions with smaller step sizes. In this case, the process with finite variation may have comparable contribution to the overall variation of paths as the one with infinite variation. It can be illustrated by a simple example. Let consider two processes: a process of infinite variation L and compensated Gamma process 17 = 17 7 with shape parameter y known to have finite variation. A linear term subtracted from standard Gamma process 1? because we are interested in processes with zero mean. For each process Y, we can simulate its variation Σt |Yr+ Yr, on the interval [0, 1] divided by n + points 44 = k/n into n equal parts and average this value over number of Monte-Carlo simulations. The results of this experiment are demonstrated in Figure 3 As n increases, variation of LE rapidly grows larger since it is the process of infinite variation while variation of 177 tends to some constant. But for small values of the number of solver steps n, variation of these processes are of the same order, and variation of I' can be even bigger. Therefore, if want simulate sum of a process with finite variation and the one with infinite variation on some interval with a small number of steps, we cannot neglect increments of the process of finite variation.
supm Σx [Y,,
157
1.0
Figure 3: Variation of LG for a = 1.5 and 17 for y 1,5 and 20 on [0. 1] depending on number of solver steps n.

---

Published as a conference paper at ICLR 2025
Table 1: Evaluation of models trained CIFAR10 with 1.8, 1.5 and 1.2 in terms of FID.
Euler-Mane Mallr-Maruyama and Exponential Integrator solvers with N steps are used.
Euler-Maruyama
Exponential Integrator
N=20
N=50
N=20
N=50
N=500
SDE-A (LIM with a 1.8)
144.7
61.57
10.42
6.58
2.64
SDE-E (LIM with 1.8)
8.79
4.14
6.86
4.87
3.36
ODE (LIM with & = 1.8)
11.68
5.23
10.31
4.88
3.38
SDE-A (LIM with
1.5)
109.5
22.68
9.05
4.25
2.72
SDE-E (LIM with @ = 1.5)
7.86
4.37
6.27
4.09
3.26
ODE (LIM with a 1.5)
9.95
5.94
10.50
5.14
3.35
SDE-A (LIM with a 1.2)
49.87
4.54
7.70
4.49
3. 3.28
SDE-E (LIM with a 1.2)
7.08
4.22
7.08
4.26
3.74
ODE (LIM with 1.2)
8.43
5.25
9.14
5.12
3.74
Table 2: Evaluation of models trained on CIFARIO with 1.8, 1.5 and 1.2 in terms of coverage. Suler-Maruyama and Exponential Integrator solvers with N steps are used.
Euler-Maruyama
Exponential Integrator
N=20
N=50
N=20
N=50
N=500
SDE-A (LIM with = 1.8)
2.26%
25.08%
82.56%
89.55%
93.27%
SDE-E (LIM with a 1.8)
84.73%
90.85%
86. 36.26%
91. 91.26%
91.46%
ODE (LIM with 1.8)
82.36%
89.29%
80.35%
89.16%
91.05%
SDE-A (LIM with a = L.5)
6.53%
68.67%
85.11%
90.82%
92, 92.63%
SDE-E (LIM with 1.5)
85.39%
90.00%
87.88%
90.24%
91.50%
ODE (LIM with a 1.5)
82.72%
88.68%
80.93%
88.32%
90.73%
SDE-A (LIM with I.2)
40.99%
89.33%
85.92%
90.13%
90.74%
SDE-E (LIM with 1.2)
86.01%
89.77%
86, 36.63%
89.78%
89.92%
ODE (LIM with a = 1.2)
84.07%
88.71%
81.26%
87.72%
89.95%
To sum up. the exact reverse dynamics 11 we propose can outperform the stochastic sampling algorithm (9) when employing solvers with small NFE because in this case the error coming from dropping dz+ term may start to dominate. As for advantages over the deterministic sampling algorithm propose allows to control amount of noise inference thus potentially leading to more diverse samples for appropriate choices of T)- We will experimentally demonstrate these advantages in the next section.
(TO), the dynamics
5 EXPERIMENTS
In this section we present the results of our experiments on image and speech modalities.
5.1 IMAGE GENERATION
We experiment with unconditional image generation precisely following the setting in Yoon et al. (2023). We train Lévy-Itó models with a = 1.8, 1.5 and 1.2 on CIFARTÓ with the same architecture as in the mentioned paper and evaluate different sampling schemes in terms of Fréchet Inception Distance (FID) and diversity as measured by the metric called "coverage" (Nacem et al., (020). Loosely speaking, this metric is probability that among k nearest neighbours (as measured by the same features used to calculate FID) real image there exists at least one generated image. We generate 50k images with each model and compare them with 50k CIFAR10 images from the training set. We use k = 5 to calculate coverage. The results are shown in Tables and Training details can be found in Appendix B
7

---

Published as a conference paper at ICLR 2025
508- 1707 055 53 MC
205 54 MFC
00/17
Figure 4: FID on CIFAR10 test set as function of r for LIMs with a = 1.8 (left), 1.5 (middle) and a 1.2 (right).
Two solvers with different number of steps are used to generate samples: standard Euler- Maller-aaruyama (Kloeden & Platen, 1992) (it becomes a fixed-step Euler method when we test deterministic sam- pling (T0)) and Exponential Integrator (Zhang & Chen 2023) (reducing to DDIM scheme (Song) et al. 202 la) in a deterministic case). The latter one leads to smaller numerical errors by employing the fact that diffusions are always designed such that the term p(x, t) in corresponding differential equations is linear in z. We test the deterministic mode according to the ODE (TO) which we refer to just as ODE, the approximate stochastic dynamics following the SDE ( denoted by SDE-A, and the exact SDE we propose (11) referred to as SDE-E with the parameters chosen as showing the best performance in terms of FID on CIFARIO 10 test set containing 10k images. Table demonstrates that our SDE-E is the best in terms of FID for small number N of solver steps, and for configurations with rather large numerical errors (i.e. Euler- Materrmm with N = 20 and 50 and Exponential Integrator with N 20) performance improvement is quite significant (up to 3.5 FID) with the results being consistent for all values of a. At the same time, the improvement becomes negligible disappears for configurations with relatively small numerical errors. As far generated images diversity is concerned, we observe from Table that the scheme we propose is always better than the deterministic sampling (except for the case 1.2 and Exponential Integrator with N 500 where the difference is less than 0.05%) and also better than sampling from the approximate SDE for schemes with large numerical errors. These observations are consistent with the argument in Section [
As for tuning hyperparameters Te. we use the following heuristics: general, we should add very little no noise at later stages of inference which is aligned with common practice for conventional diffusion models (Karras et al. 2022) while at earlier stages amount of noise should be considerable enough to allow for samples diversity. Therefore, we test functions 1/2 that most of the time (except for some neighbourhood of 0 and 7') equal some constant 7). decrease as t + and increase as t + T. Explicit expression for functions 1} used for every model can be found in Appendix B Figure 4 shows performance of different models and different solvers depending on 17. One can note that typical optimal values of r smaller for small values of a, i.e. the optimal overall amount of o-stable noise added at inference should be less for noise with heavier tails.
We also conducted experiment of training LIM with a = 1.8 on imbalanced CIFAR10 to verify that the sampling method we propose is applicable in this scenario as well. We generated images with 20 steps of Exponential Integrator in 5 different experiment runs, and the average results are presented in Table 3 Based on these
Table 3: Performance on imbalanced CIFARIO.
Metric
SDE-A
SDE-E
ODE
FID
18.44
18.10
23.68
Coverage
80.33%
81.83%
71.73%
results we conclude that the sampling algorithm we propose still outperforms baseline methods both in terms of FID and diversity when training dataset extremely imbalanced (the largest class contains 5000 images while the smallest - only 50; refer to Appendix for more details), although performance gain is less than for models trained on common CIFARIO dataset. It is also worth mentioning that all of the considered sampling methods corresponding to Lévy-Itô diffusion models achieve significantly better results than common diffusion models with only 60 FID in the described imbalanced setting reported in Yoon al. (2023).
In Table we additionally report coverage for each particular class. Since Table repoots percentage of all real images from the training set having generated ones among their nearest neighbours, its
8

---

Published as a conference paper at ICLR 2025
Table 4: Evaluation of the model trained on imbalanced CIFAR10 in terms of coverage measured for each class, The model generated samples with Exponential Integrator with 20 steps. Classes with larger id have less training data.
Class id
2
3
6
7
SDE-A 82% 84%
88%
77%
82%
89%
68%
81%
65%
80%
74%
SDE-E
87%
82%
81%
92%
65%
84%
69%
91%
70%
ODE 73%
83%
63%
73%
80%
58%
87%
55%
86%
71%
Table 5: Evaluation of text-o--ppeech models trained with Gaussian noise and a-stable noise for
1.8 and 1.5 in terms of speaker similarity. Fuler method of solving ODE with N steps is used.
Female speaker (1000 min.)
N 30
N 50
N 100
Gaussian noise
0.815 + 0.011
0.834
0.010
0.850 + 0.010
a-stable noise (a 1.8)
0.822 + 0.010
0.843
0.009
0.853 1 0.010
a-Stable noise 1.5)
0.841 + 0.009
0.859 0.009
0.867 + 0.009
Male speaker (10 min.)
N 30
N 50
= 100
Gaussian noise
0.738 + 0.009
0.762
0.010
0.773 + 0.010
0-Stable noise (a 1.8)
0.761 + 0.009
0.780
+ 0.010
0.794 + 0.010
(--taable noise (a 1.5)
0.782 + 0.009
0.792
0.008
0.800 + 0.009
results are strongly biased towards more represented classes, and, in principle, it could have happened so that the coverage gain of 1.5% achieved by our method SDE-E holds mostly for such classes while its coverage for less represented classes is poor. Table 4 shows that this is not the case: one cannot observe dramatic quality drop terms of coverage SDE-E compared to SDE-A for classes containing small number of images.
5.2 SPEECH SYNTHESSIS
In experiments with LIMS on text-to-speech task we closely follow Popov et al. (2021) and train models with the text encoder, duration predictor and diffusion- based decoder for three kinds of
driving noise: Gaussian as in the original paper and a-stable with a = 1.8 and 1.5. To demonstrate
benefits of utilizing heavy-tailed noise distribution, imbalanced dataset consisting of 16.6 hours (1000 minutes) of an English female speaker (Io 2017)
train text-to-speech models extremely
and 10 minutes of an English male speaker with id 9017 from Bakhturina al. (2021) Appendix C contains training details.
We synthesized 100 sentences for each speaker using 30, 50 and 100 ODE solver steps. CAM++ speaker verification model (Wang et al. 2023) was chosen to evaluate speaker similarity to ground- truth recordings. [a the results of this evaluation. is clear that the models with a-stable noise consistently outperform the one based on Gaussian noise for both speakers in terms of speaker similarity, and the difference is more distinct when small number steps is used for generation. 1.8. However,
Furthermore, the model with 1.5 shows better results than the one with for all models average speaker similarity values are lower for the less represented speaker. We also have to note that training such imbalanced dataset was challenging task for all models under comparison: we observed that speech generated with male speaker's voice had quite a lot of inaccuracies in pronunciation irrespective of the model.
LIMITATION
One of the main limitations of the reverse dynamics derive in the paper is that have to tune parameters 7), corresponding to amount of noise added at inference which may be quite time- consuming. There are papers estimating optimal reverse variance in common diffusion models (Bao et al. (202)), but case of a-stable Lévy processes we have infinite variance, therefore we cannot apply such methods directly. So, finding optimal amount of a-stable noise inference is

---

Published as a conference paper at ICLR 2025
an interesting open question. Estimating increments d2c of process of finite variation skipped in the original reverse dynamics may also be of interest and lead to alternative sampling algorithms. As far as text-to-speech experiments concerned, it could be useful to focus more realistic scenarios when we have several "rare" speakers rather than only one.
7 CONCLUSION
In this paper we propose a parametric family of algorithms of sampling from Lévy-Itô diffusion models relying on the reverse-time SDE, This SDE is derived to have solutions with the same marginal probabilities those of the forward diffusion which makes the proposed sampling algorithms valid from theoretical point view. Our approach is tested on image generation task where it is shown to significantly improve the results of unconditional image generation when using small number of generation steps without sacrificing samples diversity. Besides, we study applicability of Lévy-Itō diffusion models to speech domain and demonstrate the potential of text-to-speech Lévy-Itô models in the scenario when multi-speaker training data is highly imbalanced.
REFERENCES
Sergio Albeverio, Zdzislaw Brzežniak, and Jiang-Lun Wu. Existence of global solutions and invariant measures for stochastic differential equations driven by Poisson type noise with non-Lipschitz coefficients. Jourmal Mathematical Analysis and Applications, 371(():309-33)) 2010.
Brian D.O. Anderson. Reverse-timg Diffusion Equation Models. Stochastic Processes and their Applications, 12(3):313 326, 1982. ISSN 0304-4149.
David Applebaum. Lévy Processes and Stochastic Calculus. Cambridge Studies Advanced Mathematics. Cambridge University Press, 2004. ISBN 9780521832632.
Evelina Bakhturina, Vitaly Lavrukhin, Boris Ginsburg, and Yang Zhang. Hi-fi multi-speaker english tts dataset. arXiv preprint arXiv:2104.014900 2021.
Fan Bao, Chongxuan Li, Jun Zhu, and Bo Zhang. Analytic-DPM: An Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models. In International Conference Learning Representations, 2022.
Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. WaveG- rad: Estimating Gradients for Waveform Generation. In International Conference on Learning Representations, 2021.
Jacob Deasy, Nikola Simidjievski, and Pietro Liò. Heavy-tailed denoising score matching. ArXiv, abs/21 12.09788 2022. URL
09788
Prafulla Dhariwal and Alexander Nichol. Diffusion Models Beat GANs on Image Synthesis. In Advances in Neural Information Processing Systems, volume 34, pp. 3780-8794, Curran Associates, Inc., 2021.
Emst Eberlein. Application Generalized Hyperbolic Lévy Motions Finance, pp. 339-336. Birkhäuser Boston, Boston, MA, 2001.
Sicheng Gao, Xuhui Liu, Bohan Zeng, Sheng Xu, Yanjing Li, Xiaoyan Luo, Jianzhuang Liu, Xiantong Zhen, and Baochang Zhang. Implicit Diffusion Models for Continuous Super- Resolution. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 10021-10030, June 2023.
Hélyette Geman. Pure jump Lévy processes for asset price modelling. Journal of Banking & Finance, 26(7):1297-1316 2002.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, et al. Generative Adversarial Nets. Advances in Neural Information Processing Systems 27, pp. 2672-2680. Curran Associates, Inc., 2014.
10

---

Published as a conference paper at ICLR 2025
Curtis Hawthorne, Ian Simon, Adam Roberts, Neil Zeghidour, Josh Gardner, Ethan Manilow, and Jesse H. Engel. Multi-instrument Music Synthesis with Spectrogram Diffusion. In Proceedings of the 23rd International Society for Music Information Retrieval Conference, ISMIR 2022, Bengaluru. India, December 4-8, 2022, pp. 598-607, 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusson Probabilistic Models. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, volume 33. Curran Associates, Inc.. 2020.
Zheyuan Hu, Zhongqiang Zhang, George Em Karniadakis, and Kenji Kawaguchi. Score-Based Neural Networks for High-Dimensional Bokker-Planck Equations. ArXiv,
abs/2402.07465, 2024. BKLLtttps///axxvv.ooo wtt.:/orrgaab////aaaaaa 07444.22..74...
Aapo Hyvärinen. Estimation of Non-Normalized Statistical Models by Score Matching. Journal of Machine Learning Research, 6(24))695-709, 2005.
Keith Ito. The LJ Speech Dataset, 2017.
URL https://keithtto.comm
LJ- S--peece--aatasee////
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the Design Space of Diffusion- Based Generative Models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho. and A. Oh (eds.), Advances Neural Information Processing Systems, volume 35, pp. 26565-26577. Curran Associates, Inc., 2022.
Peter E. Kloeden and Eekhard Platen. Numerical Solution of Stochastic Differential Equations, volume 23 of Stochastic Modelling and Applied Probability. Springer-Verlag Berlin Heidelberg. 1992.
Jungil Kong, Jachyeon Kim, and Jackyoung Bae. HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, virtual, 2020.
Zhifeng Kong and Wei Ping. On Fast Sampling of Diffusion Probabilistic Models. ICML Workshop Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2021.
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow Matching for Generative Modeling. In The Eleventh International Conference on Learning Representations, 2023.
Anna Lischke, Guofei Pang, Mamikon Gulian, Fangying Song, Christian Glusa, Xiaoning Zheng, Zhiping Mao, Wei Cai, Mark M. Meerschaert, Mark Ainsworth, and George Em Karniadakis. What is the fractional Laplacian? A comparative review with new results. Journal of Computational Physics, 404: 109009, 2020. ISSN 00011-9991.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 328 19-32848, PMLR, 21-27 Jul 2024.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver: Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 5775-5787. Curran Associates, Inc., 2022.
Zhengxiong Luo, Dayou Chen, Yingya Zhang, Yan Huang, Liang Wang, Yujun Shen, Deli Zhao, Jingren Zhou, and Tieniu Tan. VideoFusion: Decomposed Diffusion Models for hugh-Quality Video Generation. In 2023 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10209-10218. IEEE Computer Society, jun 2023.
11

---

Published as a conference paper at ICLR 2025
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Yun-Yan Zhu, and Stefano Er- mon. SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations. In International Conference Learning Representations, 2022.
Eliya Nachmani, Robin San Roman, and Lior Wolf. Denoising Diffusion Gamma Models. ArXiv, abs/2110.05945 2021. URL htttt: org/ aaaaaass/11110 05948
Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models. In Proceedings of the 37th International Conference on Machine Learning, ICML'20 20. IMLR.org. 2020.
Bernt Oksendal. Stochastic Differential Equations: an introduction with applications. Springer- Verlag, Berlin, Heidelberg, 1992. ISBN 3387533354.
Eric Paquet, Farzan Soleymani, Herna Lydia Viktor, and Wojtek Michalowski. Annealed fractional
Lévy-Itō diffusion models for protein generation. Computational and Structural Biotechnology Journal, 23:1641: 10:1641-1653 2024. ISSN 2001-0370.
A.S. Paulson, E.W. W. Holcomb, and R.A. Leitch. The estimation of the parameters of the stable laws. Biometrika, 6261)1163-, 04 1975.
Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova, and Mikhail Kudinov. Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 8599-8608. PMLR, 2021.
Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova, Mikhail Kudinov, and Jiansheng Wei. Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme. In International Conference on Learning Representations, 2022.
Yiming Qin, Huangjie Zheng, Jiangchao Yao, Mingyuan Zhou, and Ya Zhang.
Diffusion Models. In 2023 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 18434-18443, Los Alamitos, CA, USA, jun 2023. IEEE Computer Society.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- Resolution Image Synthesis With Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695, June 2022.
Tasnima Sadekova, Vladimir Gogoryan, Ivan Vovk, Vadim Popov, Mikhail Kudinov, and Jiansheng Wei. A Unified System for Voice Cloning and Voice Conversion through Diffusion Probabilistic Modeling. In Interspeech 2022, pp. 3003-3007, 2022. doi: 1114///ntersppecch.0000 2022- 10879.
Tasnima Sadekova. Mikhail Kudinov, Vadim Popov, Assel Yermekova, and Artem Khrapov. Pitchflow: adding pitch control flow-matching based tts model. In Interspeech 2024, pp. 4418 141-22 2024.
Tim Salimans and Jonathan Ho. Progressive Distillation for Fast Sampling of Diffuusion Models. In International Conference Learning Representations, 2022.
D. Schertzer, M. Larchevêque, J. Duan, V. V. Yanovsky, and S. Lovejoy. Fractional Fokker-Planck equation for nonlinear stochastic differential equations driven by non-Gaussian Lévy stable noises. Journal of Mathematical Physics, 42(1):200-212 01 2001.
Dario Shariatian, Umut Simsekli, and Alain Durmus. Denoising Lévy Probabilistic Models. ArXiv, abs/2407.1 1809, 2024.
100024177799999 18609
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models. In International Conference on Learning Representations, 2021a.
Yang Song and Prafulla Dhariwal. Improved Techniques for Training Consistency Models. In The Twelfth International Conference Learning Representations, 2024.
Yang Song and Stefano Ermon. Improved Techniques for Training Score-Based Generative Models. In Advances Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
12

---

Published as a conference paper at ICLR 2025
Yang Song and Diederik P. Kingma. How to Train Your Energy- Based Models. ArXiv, abs/2101.03288, 2021. URL /tctpp:://xxii.ooo//aaa
03288
Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum Likelihood Training of Score-Based Diffusion Models. In Advances in Neural Information Processing Systems, 2021b.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations. In International Conference on Learning Representations, 2021c.
Yang Song, Prafulla Dhariwal, Mark Chen, and llya Sutskever. Consistency Models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 32211-32252. PMLR, 23-29 Jul 2023.
Hui Wang, Siqi Zheng, Yafeng Chen, Luyao Cheng, and Qian Chen. Cam++: fast and efficient network for speaker verification using context-aware masking, arXiv preprint arXiv:2303.008: 2023.
Suttisak Wizadwongsa and Supasorn Suwajanakorn. Accelerating Guided Diffusion Sampling with Splitting Numerical Methods. In The Eleventh International Conference Learning Representa- tions, 2023.
Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T. Freeman. Improved Distribution Matching Distillation for Fast Image Synthesis. ArXiv, 0bs/2400.11467, 2024a. htlth:tp::/aaae
org/ wtw.a/////.5.. 11005...4 14867
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T. Freeman, and Taesung Park. One-Step Diffusion with Distribution Matching Distillation. 2024 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6613 Los Alamitos, CA, USA, jun 2024b. IEEE Computer Society.
Eunbi Yoon, Keehun Park, Sungwoong Kim, and Sungbin Lim. Score- Generative Models with Lévy Processes. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 40694-40707. Curran Associates, Inc., 2023.
Qinsheng Zhang and Yongxin Chen. Fast Sampling of Diffusion Models with Exponential Integrator. In The Eleventh International Conference on Learning Representations, 2023.
Yanjie Zhang, Xiao Wang, Qiao Huang, Jinqiao Duan, and Tingting Li. Numerical analysis and applications of Pokker-Planck equations for stochastic dynamical systems with multiplicative a-stable noises. Applied Mathematical Modelling, 87:711-730 2020. ISSN 0307-904X.
13

---

Published as a conference paper at ICLR 2025
A PROOF OF THEOREM
We consider the following forward SDE
dX, = au(Xe,ttde + o.dLi
(12)
and reverse SDE
dX, (u(Xe+. (1 mo s(()) (Xx+)) orn 1/a aLd on the time interval [0, [). LE and LE are forward and reverse-time a-stable Lévy processes correspondingly, and the fractional score function s/a) (x) is defined as
(13)
(x) (-A) Vp(a,1)
(14)
p(a,t)
where p(x, t) is the marginal density of the forward diffusion X,, We assume that both 12 and 13
have solutions and p(z, t) is the unique solution of the fractional Fokker-Planne equation associated with the forward diffusion (12)
① ot plz = (p(x,t)p(a of( p(x,t)
(15)
with the proper initial conditions. We also assume that the fractional Fokker-Plant equation associated with the reverse diffusion (1) has unique solution. The necessary conditions include
measurability of μu-, t) and s(()() (.) and certain Lipschitz-type constraints on drift and diffusion coefficients of the SDEs (12) and (13). More details be found e.g. in Albeeveri et al. (2010);
Schertzer et al. 2001 Zhang et al. (2020).
Proof. Definition of the fractional Laplacian and Fourier multiplier property of negative Laplace operator A imply that
(-A(()) = = (-)) (1
(16)
[ (-)))))))
which makes definition of the fractional Laplacian consistent with the standard Laplace operator.
Now we want to make a reverse-time process X, forward-time one and write down the fractional Fokker-Planck equation associated with it. Consider change of time variable T t. Put
Y- = X, and rewrite (1 using di = =dd:
dY, -Mn(,,((
1/a dL (1 ST + -T-
sda)
(17)
The SDE 117 is now forward one (time flows forward from 0 T), and its fractional Fokker- Planck equation is
-1
(18)
where g(x,7) r) is the probability density function of the process Yr. Let us also rewrite the equation (15 in terms of using
0 Dr
0 or plz, p(e, T r) V (u(c,T-7)p(z, T)) of +f (-A))///p(( 7) (19)
We aim to prove that Y, and X, have the same marginal densities, i.c. that q(z, 7) = p(x,t) = p(a, T). By our assumption equation (18 has unique solution, so it sufficient to check that p(x, T - r) satisfies this equation.
The fractional Laplacian property (16) implies that
T - ( (-1) a-2 Vplx,T -
v
= . p(x: (p(xTT
p(r, T -T)
(20)
= --V ((-A) T
-- =
14

---

Published as a conference paper at ICLR 2025
Substituting g(2, r) with p 7) in 18 and employing
20 we have
①
V. - (Sf_.())(...1
θ7 plz, T - -)== V. -)) -
7)) (1
-0 ( p(x; -- V (n(x, T - -),
+ (1 -/ -1)--)00- (-) -) 11 (- p -T)
V (n(x, T T)p(2, T of p -7)
(21)
which is satisfied by p(r, 1) due to
19
B ADDITIONAL DETAILS OF EXPERIMENTS IN IMAGE GENERATION
The model use for CIFARIO experiments is NCSN++ YYoon et al. 2023 Song al. c021c with 8 residual blocks. We train 3 models for a = 1.8, 1.5 and 1.2 with batch size 128 and learning rate 0.0001 for 250k iterations. Diffusion models tend overfit on CIFARIO so we choose the best checkpoint in terms of FID on the test set (100k, 150k and 180k iterations for a = 1.8, 1.5 and 1.2 respectively).
As for tuning hyperparameters T/o we choose the following expression
to
T) = n (6-n)
+ 2n(a((c) - 1),
(22)
T to
where [x]+ = ma and ó 2 1z. Diffusion models we train are designed to have T = 1. The function (22) decays fast towards zero as + 0 because the last sigmoid term and linearly increases from time step t to to d at time step f T. We choose
20 in all experiments to ensure fast decay in the neighhourhood of t = 0. We take to = 0.7, 0.85 and 0.95 for a = 1.8, 1.5 and 1.2 (meaning increase in the neighbourhood of
correspondingly. 5 is always chosen to be either t t= T). Optimal values (1), 8) we choose to calculate FID on CIFAR10 train set are given in Table 6
Table 6: Hyperparameters for Euler- Mallarmm (EM) and Exponential Integrator (EI) with different number of steps (20, 50 or 500).
(n. 8)
EM-20
EM-50
El-20
EI-50
EI-500
a 1.8
(0.10, 00100
(0.25, 0.25)
(0.60, 1.00)
(0.70, 1.00)
(0.10, 0.100
a 1.5
0.100
(0.15, 1.00)
(0.10, 1.00)
(0.05, 1.00)
(0.10, 0.100
a 1.2
(0.15, 0.15)
(0.10, 1.00)
(0.01, 1.00)
(0.10, 1.00)
(0.10, 0.10)
Typical shapes of the function 7/1 as well images generated by the LIM with a = 1.8 and Exponential Integrator solver with 20 steps given in Figure
Imbalanced CIFAR10 contained 5000, 2997, 1796, 1077, 645, 387, 232, 139, 83 and 50 images belonging classes "airplanc", "automobile". "bird", "cat". "deer". "dog". "frog" "horse", "ship" and "truck" correspondingly. It is the same setting as that used in Yoon et al. (203) For the model trained on this dataset we chose 17: = t1/5_
C ADDITIONAL DETAILS OF EXPER IMENTS IN SPEECH SYNTHESIS
In speech synthesis experiments we worked with the model closely following Popov et al. 202 except for several modifications. First, the forward diffusion transforms data distribution into N (0. I) instead of A (p, 1) in the bascline model and into (1.5(() in case of Lévy-Itô model. Second, early experiments revealed the fact that training on imbalanced dataset introduces bias in the text encoder
15

---

Published as a conference paper at ICLR 2025
Figure 5: Typical shapes of nu (left) and images generated with the approximate SDE and the exact one (II) we propose (right) with 20 iterations of Exponential Integrator.
(middle)
towards the voice with the highest ratio. To avoid this issue replaced this block with the separately trained text encoder which predicts "average voice" mel-spectrograms as in Sadekova et al. (2022). These features encoded by the diffusion decoder only.
irecker-independent and preserve linguistic content, speaker information
The encoder consists of stack of three convolutional layers with kernel size 5 and 512 channels followed by a bidirectional LSTM with 256 units and the duration predictor with an upsampling module. Duration predictor is a two-layer bidirectional LSTM with 256 units. The whole block is pre-trained on LibriTTS dataset, fixed and works in the teacher-forcing mode during the decoder training which took 2800 epochs. Ground truth phoneme durations obtained with Montreal Forced Aligner. The pre-tre trained universal HiFi-GAN Kong et al. 2020) is used as vocoder.
16


Reviews:
[{"review_id": "pDcUex3D11", "reviewer": "Reviewer_7oeL", "summary": "This paper introduces a paremetric family of SDEs allowing different sampling scheme from a single pretrained Levy Ito diffusion model. It brings the toolbox for Levy Ito diffusion models (LIM) closer to that of standard \"Gaussian\" diffusion models (DM). Empirically, the authors find that the new paremetric family of SDEs has benefit in term of generated samples quality and diversity at low number of functional evaluations, which lowers the computational cost of sampling from Levy Ito diffusion models. Finally, they train a text-to-speech diffusion model on an imbalanced dataset and evalutate the benefits of LIMs compared to DM.", "questions": "* Please explain weakness 2.\n* For the text to speech model, even on the main mode (female speakers), LIMs outerperform Gaussian DMs. Why is that? Wouldn't we expect Gaussians DM to perform better on the main mode of the distribution? Does fixing the \"bias\" in the text encoder (line 809 to 825) unfairly disadvantage Gaussians DM even on the main mode? What would the results have been with a standard encoder (or alternatively do you have other baselines for text to speech imbalanced modeling that would shed light on the benefits of LIM on both the main mode and the tail mode)? You are also using an ODE sampler, which, in your image generation experiment, favors the less frequent class. \n* It seems that the common accepted reason for which LIMs are better for imbalanced datasets is that Lévy Ito processes, with their heavy tails and jump possibilities, better cover less probable isolated modes of the data distribution. In your opinion, why does this phenomenon subsist when using ODE sampling? In your imbalanced CIFAR experiment, classes 8 and 9 seem favored by the ODE.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "* The parametric family of SDEs derived by the author is new and offer more flexibility for LIM sampling.\n* The authors give a rigorous derivation of the proposed SDEs\n* The paper clearly tries to show how their theoretical results compare with existing litterature on LIMs. It explains clearly what is the limitation of Yoon et al (approximation of the reverse process).\n* Promising results for text to speech models\n* The paper is well written overall", "weaknesses": "* The experiments about text to speech models are interesting and promising but they are not tied to the main results of the paper (theoretical and empirical contributions to LIM sampling). The baseline DM against which authors compare is also new and has the modifications outlined in lines 809 to 825 in the Appendix.\n* The simple toy example explains well why processes with zero mean and finite variations cannot be omitted completely in low NFE schemes. However, in my opinion, the paper does not explain clearly what steps/changes in the parametric family of SDEs prevents any such (intractable) process to come into play. I understand from line 286 that the reverse time SDE is no longer a model of the forward time SDE and that trajectories are different. Is that what allows you to have \"better\" SDEs?\n* (Not a real weakness per se) The new paremetric family of SDEs does not improve the capabilities of LIM in term of imbalance \"correction\". As examplified by table 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1STIXWjc5O", "reviewer": "Reviewer_EF2f", "summary": "Levy diffusion models perform better (especially on rare classes) when it’s trained on imbalanced datasets. However, the previous reverse Levy process doesn’t have same marginal probabilities at each noise level due to the omitted intractable term, resulting in non-exact sampling. \nThe authors propose a novel parametric family of SDE whose solutions have the same marginal densities as the forward levy diffusion process, leading to exact solutions. \nEmpirical experiments demonstrate that the proposed reverse process has superior sample quality with small number of sampling steps, and also performs good in terms of sample diversity.", "questions": "- In table 3, I wonder how many times the authors run the experiments? It seems that for both FID and coverage metrics, the proposed SDE has very close performance to the baseline SDE. \n- In the speech synthesis experiment, could you provide a comparison to the baseline Lévy-Itô diffusion model?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-organized, and the technique derivations are solid. Also most of the arguments are well supported by experiments. \n- Instead of developing a reverse-time process of the forward Levy-Ito SDE, the authors propose parametric reverse-time SDEs that have the same marginal probability densities as the forward process, which gives exact sampling. \n- It seems like compare to the baseline SDE, the proposed one is good with multiple numerical solvers. For example, in Table 1 and 2, SDE(11) performs good with both solvers, while baseline SDE approximate(9) has a different performance with different solvers.", "weaknesses": "- Though the improvement of FID scores given small number of sampling steps (e.g. N=20) is huge, the improvement on imbalanced CIFAR10 is marginal. Also, it could be nice to provide some FID scores of samples from Gaussian diffusion models, which shows a clear improvement in imbalanced dataset. \n- In the speech synthesis experiment, the authors compare the proposed model only with the Gaussian-based diffusion model. However, it remains unclear whether this model is still SOTA when compared to the baseline Lévy-Itô diffusion model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tTJcM2VROx", "reviewer": "Reviewer_9Gsh", "summary": "The paper offers a novel method for sampling using a Lévy-Itô diffusion model based on a new formulation of its corresponding SDE. The authors identify an issue in existing methods for the reverse SDE caused by neglecting one of the terms, and offer a solution. The paper includes experiments which compare the generation results of the proposed solutions to alternatives, as well as a demonstration of its use in addressing skewed training datasets in the text-to-speech field.", "questions": "- I believe it would be easier to follow the comparisons by using a consistent naming scheme, instead of the current SDE(<number>)\n- How do extremely small NFEs (5-15) effect the sampling quality of the proposed method?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper analyzes Lévy processed and their use for diffusion models, a field which is under-explored.\n- The proposition in the paper is straightforward and mathematically justified.\n- The experiments in the paper provide evidence of the benefits of the new method.", "weaknesses": "- The presentation of the paper could be improved, both at the sentence and at the paragraph level.\n- The figures and naming scheme makes it hard to follow the result and illustrations in the paper. (specifically in Fig. (4), Fig (1), Tab. (5))\n- The takeaway from the experiment shown in Tab. (4) is unclear. While Tab. (3) shows a small advantage for the proposed method, the results in Tab. (4) do not reflect that.\n- The evidence for an advantage in underrepresented data could be made stronger (using more examples, specifically in the image domain).\n\nIf some of my weaknesses are properly addressed I am inclined to raise my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oTqpq5npKS", "reviewer": "Reviewer_fdtf", "summary": "A family of inverse dynamics is derived for diffusion models that use isotropic (\\alpha)-stable noise instead of Gaussian noise. Such models have been proposed recently with a deterministic inverse dynamics and a stochastic one. The deterministic dynamics is guaranteed to retrieve the exact marginal distributions. Unfortunately, the stochastic dynamics yields only approximate solutions.  This paper addresses this limitation. The algorithm is expected to be effective for data generation on imbalanced datasets. Its practical usefulness is experimentally examined in applications to image generation and text-to-speech tasks.", "questions": "The relation between the algorithm by Yoon et al (2023) and the current one is unclear. Please explain more details on how equations (4) and (11) are related.", "limitations": "", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "A parametric family of inverse dynamics is derived for diffusion models that use isotropic (\\alpha)-stable noise instead of Gaussian noise. Unlike the previously proposed one, this family is guaranteed to exactly retrieve the target marginal distributions for any parameter setting in ideal cases.", "weaknesses": "Writing in section 3.3 is sloppy. Since \"some intractable data-dependent process Z_t\" is not explained sufficiently, it is difficult to grasp the relation between the previously proposed algorithm and the current one.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)