Original: Wavelet Score-Based Generative Modeling
Revised: Wavelet Score-Based Generative Modeling

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Score-based generative models (SGMs) synthesize new data samples from Gaussian white noise by running a time-reversed Stochastic Differential Equation (SDE)whose drift coefficient depends on some probabilistic score.
Revised: Score-based generative models (SGMs) synthesize new data samples from Gaussian white noise by running a time-reversed Stochastic Differential Equation (SDE)whose drift coefficient depends on some probabilistic score.

Original: The discretizationof such SDEs typically requires a large number of time steps and hence a highcomputational cost.
Revised: The discretizationof such SDEs typically requires a large number of time steps and hence a highcomputational cost.

Original: This is because of ill-conditioning properties of the score thatwe analyze mathematically.
Revised: This is because of ill-conditioning properties of the score thatwe analyze mathematically.

Original: We show that SGMs can be considerably accelerated,by factorizing the data distribution into a product of conditional probabilities ofwavelet coefficients across scales.
Revised: We show that SGMs can be considerably accelerated,by factorizing the data distribution into a product of conditional probabilities ofwavelet coefficients across scales.

Original: The resulting Wavelet Score-based Generative
Revised: The resulting Wavelet Score-based Generative

Original: Model (WSGM) synthesizes wavelet coefficients with the same number of timesteps at all scales, and its time complexity therefore grows linearly with the image size.
Revised: Model (WSGM) synthesizes wavelet coefficients with the same number of timesteps at all scales, and its time complexity therefore grows linearly with the image size.

Original: This is proved mathematically over Gaussian distributions and shownnumerically over physical processes at phase transition and natural image datasets.
Revised: This is proved mathematically over Gaussian distributions and shownnumerically over physical processes at phase transition and natural image datasets.

Original:  Introduction
Revised:  Introduction

Original: Score-based Generative Models (SGMs) have obtained remarkable results to learn and sampleprobability distributions of image and audio signals [43, 4, 24, 36, 37, 6].
Revised: Score-based Generative Models (SGMs) have obtained remarkable results to learn and sampleprobability distributions of image and audio signals [38, 4, 21, 32, 33, 6].

Original: They proceed as follows: thedata distribution is mapped to a Gaussian white distribution by evolving along a Stochastic Differential
Revised: They proceed as follows: thedata distribution is mapped to a Gaussian white distribution by evolving along a Stochastic Differential

Original: Equation (SDE), which progressively adds noise to the data.
Revised: Equation (SDE), which progressively adds noise to the data.

Original: The generation is implemented using thetime-reversed SDE, which transforms a Gaussian white noise into a data sample.
Revised: The generation is implemented using thetime-reversed SDE, which transforms a Gaussian white noise into a data sample.

Original: At each time step,it pushes samples along the gradient of the log probability, also called score function .
Revised: At each time step,it pushes samples along the gradient of the log probability, also called score function .

Original: This score isestimated by leveraging tools from score-matching and deep neural networks [14, 46].
Revised: This score isestimated by leveraging tools from score-matching and deep neural networks [12, 41].

Original: At samplingtime, the computational complexity is therefore proportional to the number of time steps, i.e., thenumber of forward network evaluations.
Revised: At samplingtime, the computational complexity is therefore proportional to the number of time steps, i.e., thenumber of forward network evaluations.

Original: Early SGMs in [43, 45, 11] used thousands of time steps,and hence had a limited applicability.
Revised: Early SGMs in [38, 40, 10] used thousands of time steps,and hence had a limited applicability.

Original: Diffusion models map a Gaussian white distribution into a highly complex data distribution.
Revised: Diffusion models map a Gaussian white distribution into a highly complex data distribution.

Original: Wethus expect that this process will require a large number of time steps.
Revised: Wethus expect that this process will require a large number of time steps.

Original: It then comes as a surprisethat recent approaches have drastically reduced the the number of time steps.
Revised: It then comes as a surprisethat recent approaches have drastically reduced the the number of time steps.

Original: This is achieved byoptimizing the discretization schedule or by modifying the original SGM formulation [18, 17, 26, 52,41, 35, 42, 23, 11, 28, 40, 50].
Revised: This is achieved byoptimizing the discretization schedule or by modifying the original SGM formulation [16, 15, 23, 46,36, 31, 37, 20, 10, 25, 35, 45].

Original: High-quality score-based generative models have also been improvedby cascading multiscale image generations [39, 12, 6] or with subspace decompositions [16].
Revised: High-quality score-based generative models have also been improvedby cascading multiscale image generations [34, 11, 6] or with subspace decompositions [14].

Original: Weshow that with an important modification, these approaches provably improve the performance of
Revised: Weshow that with an important modification, these approaches provably improve the performance of

Original: SGMs.
Revised: SGMs.

Original: A key idea is that typical high-dimensional probability distributions coming from physics or naturalimages have complex multiscale properties.
Revised: A key idea is that typical high-dimensional probability distributions coming from physics or naturalimages have complex multiscale properties.

Original: They can be simplified by factorizing them as a productof conditional probabilities of normalized wavelet coefficients across scales.
Revised: They can be simplified by factorizing them as a productof conditional probabilities of normalized wavelet coefficients across scales.

Original: These conditionalprobabilities are more similar to Gaussian white noise than the original image distribution, and canthus be sampled more efficiently.
Revised: These conditionalprobabilities are more similar to Gaussian white noise than the original image distribution, and canthus be sampled more efficiently.

Original: On the physics side, this observation is rooted in the renormalizationgroup decomposition in statistical physics [48], and has been used to estimate physical energies fromdata [1].
Revised: On the physics side, this observation is rooted in the renormalizationgroup decomposition in statistical physics [43], and has been used to estimate physical energies fromdata [1].

Original: In image processing, it relies on statistical observations of wavelet coefficient properties
Revised: In image processing, it relies on statistical observations of wavelet coefficient properties

Original: A Wavelet Score-based Generative Model (WSGM) generates normalized wavelet coefficientsfrom coarse to fine scales, as illustrated in Figure 1.
Revised: A Wavelet Score-based Generative Model (WSGM) generates normalized wavelet coefficientsfrom coarse to fine scales, as illustrated in Figure 1.

Original: The conditional probability of each set of waveletcoefficients, given coarse scale coefficients, is sampled with its own (conditional) SGM.
Revised: The conditional probability of each set of waveletcoefficients, given coarse scale coefficients, is sampled with its own (conditional) SGM.

Original: The mainresult is that a normalization of wavelet coefficients allows fixing the same discretization scheduleat all scales.
Revised: The mainresult is that a normalization of wavelet coefficients allows fixing the same discretization scheduleat all scales.

Original: Remarkably, and as opposed to existing algorithms, it implies that the total number ofsampling iterations per image pixel does not depend on the image size.
Revised: Remarkably, and as opposed to existing algorithms, it implies that the total number ofsampling iterations per image pixel does not depend on the image size.

Original: After reviewing score-based generation models, Section 2 studies the mathematical properties of itstime discretization, with a focus on Gaussian models and multiscale processes.
Revised: After reviewing score-based generation models, Section 2 studies the mathematical properties of itstime discretization, with a focus on Gaussian models and multiscale processes.

Original: Images and manyphysical processes are typically non-Gaussian, but do have a singular covariance with long- andshort-range correlations.
Revised: Images and manyphysical processes are typically non-Gaussian, but do have a singular covariance with long- andshort-range correlations.

Original: In Section 3, we explain how to factorize these processes into probabilitydistributions which capture interactions across scales by introducing orthogonal wavelet transforms.
Revised: In Section 3, we explain how to factorize these processes into probabilitydistributions which capture interactions across scales by introducing orthogonal wavelet transforms.

Original: We shall prove that it allows considering SGMs with the same time schedule at all scales, independently of the image size.
Revised: We shall prove that it allows considering SGMs with the same time schedule at all scales, independently of the image size.

Original: In Section 4, we present numerical results on Gaussian distributions, the φ 4physical model at phase transition, and the CelebA-HQ image dataset [19].
Revised: In Section 4, we present numerical results on Gaussian distributions, the φ 4physical model at phase transition, and the CelebA-HQ image dataset [17].

Original: The main contributionsof the paper are as follows:
Revised: The main contributionsof the paper are as follows:

Original: • A Wavelet Score-based Generative Model (WSGM) which generates conditional probabilities of normalized wavelet coefficients, with the same discretization schedule at all scales.
Revised: • A Wavelet Score-based Generative Model (WSGM) which generates conditional probabilities of normalized wavelet coefficients, with the same discretization schedule at all scales.

Original: The number of time steps per image pixel does not need to depend upon the image size toreach a fixed error level.
Revised: The number of time steps per image pixel does not need to depend upon the image size toreach a fixed error level.

Original: • Theorems controlling errors of time discretizations of SGMs, proving accelerations obtainedby scale separation with wavelets.
Revised: • Theorems controlling errors of time discretizations of SGMs, proving accelerations obtainedby scale separation with wavelets.

Original: These results are empirically verified by showing that
Revised: These results are empirically verified by showing that

Original: WSGM provides an acceleration for the synthesis of physical processes at phase transition
Revised: WSGM provides an acceleration for the synthesis of physical processes at phase transition

Original: and natural image datasets.
Revised: and natural image datasets.

Original: diffusion white Gaussian noise  a natural image
Revised: diffusion white Gaussian noise into a natural image

Original: WSGM generates increasingly higherresolution images by discretizing reverse diffusions on wavelet coefficients at
Revised: WSGM generates increasingly higherresolution images by discretizing reverse diffusions on wavelet coefficients at

Original: It begins by generating a first low-resolution image.
Revised: It begins by generating a first low-resolution image.

Original: Renormalized wavelet coefficients are then generated conditionally to this low-resolution image.
Revised: Renormalized wavelet coefficients are then generated conditionally to this low-resolution image.

Original: A fast inverse wavelet transform reconstructs a higherresolution image from these wavelet coefficients.
Revised: A fast inverse wavelet transform reconstructs a higherresolution image from these wavelet coefficients.

Original: This process is repeated at each scale.
Revised: This process is repeated at each scale.

Original: The number of steps is the same at each scale, and can be orders of magnitude smaller than for SGM.
Revised: The number of steps is the same at each scale, and can be orders of magnitude smaller than for SGM.

Original:  Sampling and Discretization of Score-Based Generative Models2.1 Score-Based Generative Models
Revised:  Sampling and Discretization of Score-Based Generative Models2.1 Score-Based Generative Models

Original: Diffusions and time reversal A Score-based Generative Model (SGM) [43, 45, 11] progressivelymaps the distribution of data x into the normal distribution, with a forward SDE which iterativelyadds Gaussian white noise.
Revised: Diffusions and time reversal A Score-based Generative Model (SGM) [38, 40, 10] progressivelymaps the distribution of data x into the normal distribution, with a forward SDE which iterativelyadds Gaussian white noise.

Original: It is associated with a noising process ( x t ) t , with x 0 distributed accordingto the data distribution p , and satisfying:
Revised: It is associated with a noising process ( x t ) t , with x 0 distributed accordingto the data distribution p , and satisfying:

Original: [Equation]
Revised: [Equation]

Original: where ( w t ) t is a Brownian motion.
Revised: where ( w t ) t is a Brownian motion.

Original: The solution is an Ornstein-Uhlenbeck process which admits thefollowing representation for any t ≥ 0 :
Revised: The solution is an Ornstein-Uhlenbeck process which admits thefollowing representation for any t ≥ 0 :

Original: [Equation]
Revised: [Equation]

Original: The process ( x t ) t is therefore an interpolation between a data sample x 0 and Gaussian white noise.
Revised: The process ( x t ) t is therefore an interpolation between a data sample x 0 and Gaussian white noise.

Original: The generative process inverts (1).
Revised: The generative process inverts (1).

Original: Under mild assumptions on p [3, 9], for any T ≥ 0 , the reversetime process x T − t satisfies:
Revised: Under mild assumptions on p [3, 8], for any T ≥ 0 , the reversetime process x T − t satisfies:

Original: [Equation]
Revised: [Equation]

Original: where p t is the probability density of x t , and ∇ log p t is called the Stein score .
Revised: where p t is the probability density of x t , and ∇ log p t is called the Stein score .

Original: Since x T is closeto a white Gaussian variable, one can approximately sample from x T by sampling from the normaldistribution.
Revised: Since x T is closeto a white Gaussian variable, one can approximately sample from x T by sampling from the normaldistribution.

Original: We can generate x 0 from x T by solving this time-reversed SDE, if we can estimate anaccurate approximation of the score ∇ log p t at each time t , and if we can discretize the SDE withoutintroducing large errors.
Revised: We can generate x 0 from x T by solving this time-reversed SDE, if we can estimate anaccurate approximation of the score ∇ log p t at each time t , and if we can discretize the SDE withoutintroducing large errors.

Original: Efficient approximations of the Stein scores are the workhorse of SGM.
Revised: Efficient approximations of the Stein scores are the workhorse of SGM.

Original: [14] shows that the score∇ log p t can be approximated with parametric functions s θ which minimize the following scorematching loss:
Revised: [12] shows that the score∇ log p t can be approximated with parametric functions s θ which minimize the following scorematching loss:

Original: [Equation]
Revised: [Equation]

Original: For image generation, s θ is calculated by a neural network parameterized by θ .
Revised: For image generation, s θ is calculated by a neural network parametrized by θ .

Original: In statistical physicsproblems where the energy can be linearly expanded with coupling parameters, we obtain linearmodels s θ ( x ) =
Revised: In statistical physicsproblems where the energy can be linearly expanded with coupling parameters, we obtain linearmodels s θ ( x ) =

Original: θ ⊤ ∇ U ( x ) .
Revised: θ ⊤ ∇ U ( x ) .

Original: This is the case for Gaussian processes where U ( x ) =
Revised: This is the case for Gaussian processes where U ( x ) =

Original: xx ⊤ ; it alsoapplies to non-Gaussian processes, using non-quadratic terms in U ( x ) .
Revised: xx ⊤ ; it alsoapplies to non-Gaussian processes, using non-quadratic terms in U ( x ) .

Original: Time discretization of generation An approximation of the generative process (3) is computed byapproximating ∇ log p t by s t and discretizing time.
Revised: Time discretization of generation An approximation of the generative process (3) is computed byapproximating ∇ log p t by s t and discretizing time.

Original: It amounts to approximating the time-reversed
Revised: It amounts to approximating the time-reversed

Original: SDE by a Markov chain which is initialised by ˜ x T ∼ N (0 , Id) , and computed over times t k whichdecrease from t N = T to t 0 = 0 , at intervals δ k
Revised: SDE by a Markov chain which is initialised by ˜ x T ∼ N (0 , Id) , and computed over times t k whichdecrease from t N = T to t 0 = 0 , at intervals δ k

Original: = t k − t k [Equation]
Revised: = t k − t k − 1 :

Original: [Equation]
Revised: [Equation]

Original: Ignoring the error due to the score model, the minimum number of time steps is limited by the
Revised: Ignoring the error due to the score model, the minimum number of time steps is limited by the

Original: Lipschitz regularity of the score ∇ log p t , see [5, Theorem 1].
Revised: Lipschitz regularity of the score ∇ log p t , see [5, Theorem 1].

Original: The overall complexity of thisgeneration is O ( N ) evaluations of the score s t ( x ) .
Revised: The overall complexity of thisgeneration is O ( N ) evaluations of the score s t ( x ) .

Original: 2.2 Discretization of SGM and Score Regularity
Revised: 2.2 Discretization of SGM and Score Regularity

Original: We study how the regularity of the score ∇ log p affects the discretization of (5).
Revised: We study how the regularity of the score ∇ log p affects the discretization of (5).

Original: Assuming thatthe score is known, i.e., that s t = ∇ log p t , we prove that for Gaussian processes, the number oftime steps to reach a fixed error ε depends on the condition number of its covariance.
Revised: Assuming thatthe score is known, i.e., that s t = ∇ log p t , we prove that for Gaussian processes, the number oftime steps to reach a fixed error ε depends on the condition number of its covariance.

Original: This result isgeneralized to non-Gaussian processes by relating this error to by the regularity of ∇ log p t .
Revised: This result isgeneralized to non-Gaussian processes by relating this error to by the regularity of ∇ log p t .

Original: Gaussian distributions Suppose that the data distribution is a Gaussian p = N (0 , Σ) with covariance matrix Σ , in dimension d .
Revised: Gaussian distributions Suppose that the data distribution is a Gaussian p = N (0 , Σ) with covariance matrix Σ , in dimension d .

Original: Let p t be the distribution of x t .
Revised: Let p t be the distribution of x t .

Original: Using (2), we have:
Revised: Using (2), we have:

Original: [Equation]
Revised: [Equation]

Original: Let ˜ p t be the distribution of ˜ x t obtained by the time discretization (5).
Revised: Let ˜ p t be the distribution of ˜ x t obtained by the time discretization (5).

Original: The approximation errorbetween the distribution ˜ p 0 obtained with the time-reversed SDE and the data distribution p stems from (i) the mismatch between the distributions of x T and ˜ x T , and (ii) the time discretization.
Revised: The approximation errorbetween the distribution ˜ p 0 obtained with the time-reversed SDE and the data distribution p stems from (i) the mismatch between the distributions of x T and ˜ x T , and (ii) the time discretization.

Original: Thefollowing theorem relates these two errors to the covariance Σ of x in the particular case of a uniformtime sampling at intervals δ k
Revised: Thefollowing theorem relates these two errors to the covariance Σ of x in the particular case of a uniformtime sampling at intervals δ k

Original: = δ .
Revised: = δ .

Original: We normalize the signal energy by imposing that Tr(Σ)
Revised: We normalize the signal energy by imposing that Tr(Σ)

Original: =
Revised: =

Original: d , andwe write κ the condition number of Σ , which is the ratio between its largest and smallest eigenvalues.
Revised: d , andwe write κ the condition number of Σ , which is the ratio between its largest and smallest eigenvalues.

Original: Theorem 1.
Revised: Theorem 1.

Original: We have KL( p ∥ ˜ p 0 )
Revised: We have KL( p ∥ ˜ p 0 )

Original: ≤ E T + E δ + E T,δ , with :
Revised: ≤ E T + E δ + E T,δ , with :

Original: [Equation]
Revised: [Equation]

Original: where f ( t ) =
Revised: where f ( t ) =

Original: t − log(1 + t ) and E T,δ is a higher-order term with E T,δ
Revised: t − log(1 + t ) and E T,δ is a higher-order term with E T,δ

Original: = o ( δ + e − 4 T ) when δ → 0and T →
Revised: = o ( δ + e − 4 T ) when δ → 0and T →

Original: + ∞ .
Revised: + ∞ .

Original: Furthermore, for any ε > 0 , there exists T, δ ≥ 0 such that:
Revised: Furthermore, for any ε > 0 , there exists T, δ ≥ 0 such that:

Original: [Equation]
Revised: [Equation]

Original: [-with C ≥ 
-]
Revised: [-with C ≥ 
-]

Original: a universal constant.

Original: This theorem specifies the dependence of the Kullback-Leibler error on the covariance matrix.
Revised: This theorem specifies the dependence of the Kullback-Leibler error on the covariance matrix.

Original: Itcomputes an upper bound on the number of time steps N = T/δ as a function of the condition numberκ of Σ .
Revised: Itcomputes an upper bound on the number of time steps N = T/δ as a function of the condition numberκ of Σ .

Original: As expected, it indicates that the number of time steps increases with the condition numberof the covariance.
Revised: As expected, it indicates that the number of time steps increases with the condition numberof the covariance.

Original: This theorem is proved in a more general case in Appendix S5, which includes thecase where p has a non-zero mean.
Revised: This theorem is proved in a more general case in Appendix S4, which includes thecase where p has a non-zero mean.

Original: An exact expansion of the Kullback-Leibler divergence is alsogiven.
Revised: An exact expansion of the Kullback-Leibler divergence is alsogiven.

Original: For stationary processes of images, the covariance eigenvalues are given by the power spectrum,which typically decays like | ω | − 1 at a frequency ω .
Revised: For stationary processes of images, the covariance eigenvalues are given by the power spectrum,which typically decays like | ω | − 1 at a frequency ω .

Original: It results that κ is proportional to a power ofthe image size.
Revised: It results that κ is proportional to a power ofthe image size.

Original: Many physical phenomena produce such stationary images with a power spectrumhaving a power law decay.
Revised: Many physical phenomena produce such stationary images with a power spectrumhaving a power law decay.

Original: In these typical cases, the number of time steps must increase with theimage size.
Revised: In these typical cases, the number of time steps must increase with theimage size.

Original: This is indeed what is observed in numerical SGM experiments, as seen in Section 3.
Revised: This is indeed what is observed in numerical SGM experiments, as seen in Section 3.

Original: General processes Theorem 1 can be extended to non-Gaussian processes.
Revised: General processes Theorem 1 can be extended to non-Gaussian processes.

Original: The number of timesteps then depends on the regularity of the score ∇ log p t .
Revised: The number of timesteps then depends on the regularity of the score ∇ log p t .

Original: Theorem 2.
Revised: Theorem 2.

Original: Assume that ∇ log p t ( x ) is C 2 in both t and x , and that:
Revised: Assume that ∇ log p t ( x ) is C 2 in both t and x , and that:

Original: [Equation]
Revised: [Equation]

Original: for some K, M, α > 0 .
Revised: for some K, M, α > 0 .

Original: Then ∥ p − ˜ p 

Revised: Then ∥ p − ˜ p

Original: ∥ TV [Equation] δ
Revised: ∥ TV ≤ E T

Original: [Equation]
Revised: [Equation]

Original: E  T,δ , where:
Revised: E δ + E T,δ , where:

Original: [Equation]
Revised: [Equation]

Original: and E δ,T is a higher order term with E T,δ =
Revised: and E δ,T is a higher order term with E T,δ =

Original: o ( √ δ + e − T ) when δ → 0 and T → + ∞ .
Revised: o ( √ δ + e − T ) when δ → 0 and T → + ∞ .

Original: The proof of Theorem 2 is postponed to Appendix S5 and we show that the result can be strengthenedby providing a quantitative upper bound on ∥ p − ˜ p 

Revised: The proof of Theorem 2 is postponed to Appendix S4 and we show that the result can be strengthenedby providing a quantitative upper bound on ∥ p − ˜ p

Original: ∥ TV .
Revised: ∥ TV .

Original: Theorem 2 improves on [5, Theorem 1] 126 by proving explicit bounds exhibiting the dependencies on the regularity constants K and M of thescore and by eliminating an exponential growth term in T in the upper bound.
Revised: Theorem 2 improves on [5, Theorem 1] 125 by proving explicit bounds exhibiting the dependencies on the regularity constants K and M of thescore and by eliminating an exponential growth term in T in the upper bound.

Original: Theorem 2 is muchmore general but not as tight as Theorem 1.
Revised: Theorem 2 is muchmore general but not as tight as Theorem 1.

Original: The first error term (10) is due to the fact that T is chosen to be finite.
Revised: The first error term (9) is due to the fact that T is chosen to be finite.

Original: The second error term(11) controls the error depending upon the discretization time step δ .
Revised: The second error term(10) controls the error depending upon the discretization time step δ .

Original: Since p t is obtained from pthrough a high-dimensional convolution with a Gaussian convolution of variance proportional to t ,the regularity of ∇ log p t ( x ) typically increases with t so ∥∇ 2 log p t ( x ) ∥ and
Revised: Since p t is obtained from pthrough a high-dimensional convolution with a Gaussian convolution of variance proportional to t ,the regularity of ∇ log p t ( x ) typically increases with t so ∥∇ 2 log p t ( x ) ∥ and

Original: ∥ ∂ t ∇ log p t ( x ) ∥ rather 133 decrease when t increase.
Revised: ∥ ∂ t ∇ log p t ( x ) ∥ rather 132 decrease when t increase.

Original: This qualitatively explains why a quadratic discretization schedule withnon-uniform time steps δ k ∝ k are usually chosen in numerical implementations of SGMs [36, 44].
Revised: This qualitatively explains why a quadratic discretization schedule withnon-uniform time steps δ k ∝ k are usually chosen in numerical implementations of SGMs [32, 39].

Original: For simplicity, we focus on the uniform discretization schedule, but our result could be adapted tonon-uniform time steps with no major difficulties.
Revised: For simplicity, we focus on the uniform discretization schedule, but our result could be adapted tonon-uniform time steps with no major difficulties.

Original: If p is Gaussian, then the Hessian ∇ 2 log p is the negative inverse of the covariance matrix.
Revised: If p is Gaussian, then the Hessian ∇ 2 log p is the negative inverse of the covariance matrix.

Original: Weverify in Appendix S5 that in this case, the assumptions of Theorem 2 are satisfied.
Revised: Weverify in Appendix S4 that in this case, the assumptions of Theorem 2 are satisfied.

Original: Furthermore, theconstants K and M , and hence the number of discretization steps, are controlled using the conditionnumber of Σ .
Revised: Furthermore, theconstants K and M , and hence the number of discretization steps, are controlled using the conditionnumber of Σ .

Original: We thus conjecture that non-Gaussian processes with an ill-conditioned covariancematrix will require many discretization steps to have a small error.
Revised: We thus conjecture that non-Gaussian processes with an ill-conditioned covariancematrix will require many discretization steps to have a small error.

Original: This will be verified numerically.
Revised: This will be verified numerically.

Original: As we now explain, such processes are ubiquitous in physics and natural image datasets.
Revised: As we now explain, such processes are ubiquitous in physics and natural image datasets.

Original: Multiscale processes Most images have variations on a wide range of scales.
Revised: Multiscale processes Most images have variations on a wide range of scales.

Original: They require to usemany time steps to sample using an SGM, because their score is not well-conditioned.
Revised: They require to usemany time steps to sample using an SGM, because their score is not well-conditioned.

Original: This is alsotrue for a wide range of phenomena encountered in physics, biology, or economics [22, 31].
Revised: This is alsotrue for a wide range of phenomena encountered in physics, biology, or economics [19, 27].

Original: Wedefine a multiscale process as a stationary process whose power spectrum has a power law decay.
Revised: Wedefine a multiscale process as a stationary process whose power spectrum has a power law decay.

Original: The stationarity implies that its covariance is diagonalized in a Fourier basis.
Revised: The stationarity implies that its covariance is diagonalized in a Fourier basis.

Original: Its eigenvalues, alsocalled power spectrum, have a power law decay defined by:
Revised: Its eigenvalues, alsocalled power spectrum, have a power law decay defined by:

Original: [Equation]
Revised: [Equation]

Original: where η > 0 and 2 π/ξ is the maximum correlation length.
Revised: where η > 0 and 2 π/ξ is the maximum correlation length.

Original: Physical processes near phase transitionshave such a power-law decay, but it is also the case of many disordered systems such as fluid and gasturbulence.
Revised: Physical processes near phase transitionshave such a power-law decay, but it is also the case of many disordered systems such as fluid and gasturbulence.

Original: Natural images also typically define stationary processes.
Revised: Natural images also typically define stationary processes.

Original: Their power spectrum satisfythis property with η = 1 and 2 π/ξ ≈ L which is the image width.
Revised: Their power spectrum satisfythis property with η = 1 and 2 π/ξ ≈ L which is the image width.

Original: To efficiently synthesize imagesand more general multiscale signals, we must eliminate the ill-conditioning properties of the score.
Revised: To efficiently synthesize imagesand more general multiscale signals, we must eliminate the ill-conditioning properties of the score.

Original: This is done by applying a wavelet transform.
Revised: This is done by applying a wavelet transform.

Original:  Wavelet Score-Based Generative Model
Revised:  Wavelet Score-Based Generative Model

Original: The numerical complexity of the SGM algorithm depends onthe number of time steps, which itselfdepends upon the regularity of the score.
Revised: The numerical complexity of the SGM algorithm depends on number of time steps, which itselfdepends upon the regularity of the score.

Original: We show that an important acceleration is obtained byfactorizing the data distribution into normalized wavelet conditional probability distributions, whichare closer to normal Gaussian distributions, and so whose score is better-conditioned.
Revised: We show that an important acceleration is obtained byfactorizing the data distribution into normalized wavelet conditional probability distributions, whichare closer to normal Gaussian distributions, and so whose score is better-conditioned.

Original: 3.1 Wavelet Whitening and Cascaded SGMs
Revised: 3.1 Wavelet Whitening and Cascaded SGMs

Original: Normalized orthogonal wavelet coefficients Let x be the input signal of width L and dimensiond =
Revised: Normalized orthogonal wavelet coefficients Let x be the input signal of width L and dimensiond =

Original: L n , with n = 2 for images.
Revised: L n , with n = 2 for images.

Original: We write x j its low-frequency approximation subsampled atintervals 2 j , of size (2 − j L ) n , with x 0 = x .
Revised: We write x j its low-frequency approximation subsampled atintervals 2 j , of size (2 − j L ) n , with x 0 = x .

Original: At each scale 2 j − 1 ≥ 1 , a fast wavelet orthogonaltransform decomposes x j − 1 into (¯ x j , x j ) where ¯ x j are the wavelet coefficient which carries thehigher frequency information over 2 n − 1 signals of size (2 − j L ) n [30].
Revised: At each scale 2 j − 1 ≥ 1 , a fast wavelet orthogonaltransform decomposes x j − 1 into (¯ x j , x j ) where ¯ x j are the wavelet coefficient which carries thehigher frequency information over 2 n − 1 signals of size (2 − j L ) n [26].

Original: They are calculated withconvolutional and subsampling operators G and ¯ G specified in Appendix S4:
Revised: They are calculated withconvolutional and subsampling operators G and ¯ G specified in Appendix S3:

Original: [Equation]
Revised: [Equation]

Original: The normalization factor γ j guarantees that E [ ∥ ¯ x j ∥ 2 ] =
Revised: The normalization factor γ j guarantees that E [ ∥ ¯ x j ∥ 2 ] =

Original: (2 n − 1)(2 − j L ) n .
Revised: (2 n − 1)(2 − j L ) n .

Original: We consider waveletorthonormal filters where ( G, ¯ G ) is a unitary operator, i.e.:
Revised: We consider waveletorthonormal filters where ( G, ¯ G ) is a unitary operator, i.e.:

Original: [Equation]
Revised: [Equation]

Original: andG
Revised: and

Original: [Equation]
Revised: [Equation]

Original: It results that x j − 1 is recovered from (¯ x j , x j ) with:
Revised: It results that x j − 1 is recovered from (¯ x j , x j ) with:

Original: [Equation]
Revised: [Equation]

Original: The wavelet transform is computed over J ≈ log 2 L scales by iterating J times on (13).
Revised: The wavelet transform is computed over J ≈ log 2 L scales by iterating J times on (12).

Original: The last x Jhas a size (2 − J L ) n ∼ 1 .
Revised: The last x Jhas a size (2 − J L ) n ∼ 1 .

Original: The choice of wavelet filters G and ¯ G specifies the properties of the wavelettransform and the number of vanishing moments of the wavelet, as explained in Appendix S4.
Revised: The choice of wavelet filters G and ¯ G specifies the properties of the wavelettransform and the number of vanishing moments of the wavelet, as explained in Appendix S3.

Original: Renormalized probability distribution A conditional wavelet renormalization factorizes thedistribution p ( x ) of signals x into conditional probabilities over wavelet coefficients:
Revised: Renormalized probability distribution A conditional wavelet renormalization factorizes thedistribution p ( x ) of signals x into conditional probabilities over wavelet coefficients:

Original: [Equation]
Revised: [Equation]

Original: (14) where α (the Jacobian) depends upon all γ j .
Revised: (13) where α (the Jacobian) depends upon all γ j .

Original: Although p ( x ) is typically highly non-Gaussian, the factorization (14) involves distributions thatare closer to Gaussians.
Revised: Although p ( x ) is typically highly non-Gaussian, the factorization (13) involves distributions thatare closer to Gaussians.

Original: The largest scale distribution p J is usually close to a Gaussian when theimage has independent structures, because x J is an averaging of x over large domains of size 2 J .
Revised: The largest scale distribution p J is usually close to a Gaussian when theimage has independent structures, because x J is an averaging of x over large domains of size 2 J .

Original: Remarkably, for large classes of signals and images, each conditional probabilities ¯ p j (¯ x j | x j ) alsoexhibit Gaussian properties.
Revised: Remarkably, for large classes of signals and images, each conditional probabilities ¯ p j (¯ x j | x j ) alsoexhibit Gaussian properties.

Original: In images, the wavelet coefficients ¯ x j are usually sparse and thushave highly non-Gaussian distribution; however, it has been observed [47] that their conditionaldistributions ¯ p j (¯ x j | x j ) become much more Gaussian, due to dependencies of wavelet coefficientsacross scales.
Revised: In images, the wavelet coefficients ¯ x j are usually sparse and thushave highly non-Gaussian distribution; however, it has been observed [42] that their conditionaldistributions ¯ p j (¯ x j | x j ) become much more Gaussian, due to dependencies of wavelet coefficientsacross scales.

Original: Similarly, in statistical physics, factorization of probabilities of high frequenciesconditioned by lower frequencies have been introduced in [49].
Revised: Similarly, in statistical physics, factorization of probabilities of high frequenciesconditioned by lower frequencies have been introduced in [44].

Original: More recently, normalized waveletfactorizations (14) have been introduced in physics to implement renormalization group calculations,and model probability distributions with maximum likelihood estimators near phase transitions [1].
Revised: More recently, normalized waveletfactorizations (13) have been introduced in physics to implement renormalization group calculations,and model probability distributions with maximum likelihood estimators near phase transitions [1].

Original: Wavelet Score-based Generative Model Instead of computing a Score-based Generative Model(SGM) of the distribution p ( x ) , a Wavelet Score-based Generative Model (WSGM) applies an SGMat the coarsest scale p J ( x J ) and then on each conditional distribution ¯ p j (¯ x j | x j ) for j ≤ J .
Revised: Wavelet Score-based Generative Model Instead of computing a Score-based Generative Model(SGM) of the distribution p ( x ) , a Wavelet Score-based Generative Model (WSGM) applies an SGMat the coarsest scale p J ( x J ) and then on each conditional distribution ¯ p j (¯ x j | x j ) for j ≤ J .

Original: It isthus a cascaded SGM, similarly to [12, 39], but calculated on ¯ p j (¯ x j | x j ) instead of p j ( x j − 1 | x j ) .
Revised: It isthus a cascaded SGM, similarly to [11, 34], but calculated on ¯ p j (¯ x j | x j ) instead of p j ( x j − 1 | x j ) .

Original: Thenormalization of wavelet coefficients ¯ x j effectively produces a whitening which can considerablyaccelerate the algorithm by reducing the number of time steps.
Revised: Thenormalization of wavelet coefficients ¯ x j effectively produces a whitening which can considerablyaccelerate the algorithm by reducing the number of time steps.

Original: This is not possible on x j − 1 becauseits covariance is ill-conditioned.
Revised: This is not possible on x j − 1 becauseits covariance is ill-conditioned.

Original: It will be proved for Gaussian processes.
Revised: It will be proved for Gaussian processes.

Original: A forward noising process is computed on each ¯ x j for j ≤ J and x J :
Revised: A forward noising process is computed on each ¯ x j for j ≤ J and x J :

Original: [Equation]
Revised: [Equation]

Original: where the ¯ w j,t , w J,t are Brownian motions.
Revised: where the ¯ w j,t , w J,t are Brownian motions.

Original: Since ¯ x j is nearly white and has Gaussian properties,this diffusion converges much more quickly than if applied directly on x .
Revised: Since ¯ x j is nearly white and has Gaussian properties,this diffusion converges much more quickly than if applied directly on x .

Original: Using (4), we computea score function s J,t ( x J,t ) which approximates the score ∇ log p J,t ( x J,t ) .
Revised: Using (4), we computea score function s J,t ( x J,t ) which approximates the score ∇ log p J,t ( x J,t ) .

Original: For each j ≤ J we also 198 compute the conditional score ¯ s j,t (¯ x j,t | x j ) which approximates ∇ log ¯ p j,t (¯ x j,t | x j ) .
Revised: For each j ≤ J we also 197 compute the conditional score ¯ s j,t (¯ x j,t | x j ) which approximates ∇ log ¯ p j,t (¯ x j,t | x j ) .

Original: The inverse generative process is computed from coarse to fine scales as follows.
Revised: The inverse generative process is computed from coarse to fine scales as follows.

Original: At the largest scale2 J , we sample the low-dimensional x J by time discretizing the inverse SDE.
Revised: At the largest scale2 J , we sample the low-dimensional x J by time discretizing the inverse SDE.

Original: Similarly to (5), thegenerative process is given by:
Revised: Similarly to (5), thegenerative process is given by:

Original: [Equation]
Revised: [Equation]

Original: For j going from J to 1 , we then generate the wavelet coefficients ¯ x j conditionally to the previouslycalculated x j , by keeping the same time discretization schedule at all scales:
Revised: For j going from J to 1 , we then generate the wavelet coefficients ¯ x j conditionally to the previouslycalculated x j , by keeping the same time discretization schedule at all scales:

Original: [Equation]
Revised: [Equation]

Original: The inverse wavelet transform then approximately computes a sample of x j − 1 from (¯ x j, 0 , x j ) :
Revised: The inverse wavelet transform then approximately computes a sample of x j − 1 from (¯ x j, 0 , x j ) :

Original: [Equation]
Revised: [Equation]

Original: The generative process is illustrated in Figure 1 and its pseudocode is given in Appendix S2, see
Revised: The generative process is illustrated in Figure 1 and its pseudocode is given in Appendix S2, see

Original: Algorithm 1.
Revised: Algorithm 1.

Original: The appendix also verifies that if x is of size d then its numerical complexity is O ( Nd )where N is the number of time steps, which is the same at each scale.
Revised: The appendix also verifies that if x is of size d then its numerical complexity is O ( Nd )where N is the number of time steps, which is the same at each scale.

Original: For multiscale processes, weshall see that the number of time steps N does not depend upon d to reach a fixed error measuredwith a KL divergence.
Revised: For multiscale processes, weshall see that the number of time steps N does not depend upon d to reach a fixed error measuredwith a KL divergence.

Original: Related work The use of wavelets in multiscale architectures is often motivated by the spectral biasphenomenon [38]: [13] uses a wavelet CNN for super-resolution, [7] incorporates spectral informationby training GANs in the wavelet domain.

Original: Closer in spirit to our work, [51] introduces Wavelet Flow,a normalizing flow with a cascade of layers generating wavelet coefficients conditionally on lowerscales, then aggregating them with an inverse wavelet transform.

Original: This method yields training timeacceleration and high-resolution (1024x1024) generation.

Original:  WSGM is closely related to other cascadingdiffusion algorithms, such as the ones introduced in[12, 39, 6].
Revised: Related work WSGM is closely related to other cascading algorithms, such as the ones introducedin [11, 34, 6].

Original: The main difference lies in that earlier works on cascaded SGMs do not model the waveletcoefficients { ¯ x j } Jj =

Revised: The main difference lies in that earlier works on cascaded SGMs do not model thewavelet coefficients { ¯ x j } Jj =

Original: but the low-frequency coefficients { x j } Jj =1 .
Revised: but the low-frequency coefficients { x j } Jj =1 .

Original: As a result, cascaded  models do not219 exploit the whitening properties of the wavelet transform.
Revised: As a result, cascaded 212 models do not exploit the whitening properties of the wavelet transform.

Original: We also point out the recent work of [16]which, while not using the cascading framework, drop subspaces from the noising process at differenttimes.
Revised: We also point out the recentwork of [14] which, while not using the cascading framework, drop subspaces from the noisingprocess at different times.

Original: This allows using only one SDE to sample approximately from the data distribution.
Revised: This allows using only one SDE to sample approximately from the datadistribution.

Original: However,the reconstruction is still computed with respect to { x j } Jj =1 instead of the  wavelet coefficients.
Revised: However, the reconstruction is still computed with respect to { x j } Jj =1 instead of the 216 wavelet coefficients.

Original: Finally, we highlight that our work could be combined with other acceleration techniques such asthe ones of [17, 26, 52, 41, 35, 42, 11, 23, 28, 40, 50] in order to improve the empirical results of
Revised: Finally, we highlight that our work could be combined with other acceleration techniques such asthe ones of [15, 23, 46, 36, 31, 37, 10, 20, 25, 35, 45] in order to improve the empirical results of

Original: WSGM.
Revised: WSGM.

Original: 3.2 Discretization and Accuracy for Gaussian Processes
Revised: 3.2 Discretization and Accuracy for Gaussian Processes

Original: We now illustrate Theorem 1 and the effectiveness of WSGM on Gaussian multiscale processes.
Revised: We now illustrate Theorem 1 and the effectiveness of WSGM on Gaussian multiscale processes.

Original: Weuse the whitening properties of the wavelet transform to show that the time complexity required inorder to reach a given error is linear in the image dimension.
Revised: Weuse the whitening properties of the wavelet transform to show that the time complexity required inorder to reach a given error is linear in the image dimension.

Original: The following result proves that the normalization of wavelet coefficients performs a preconditioningof the covariance, whose eigenvalues then remain of the order of 1 .
Revised: The following result proves that the normalization of wavelet coefficients performs a preconditioningof the covariance, whose eigenvalues then remain of the order of 1 .

Original: This is a consequence of atheorem proved by [32] on the representation of classes of singular operators in wavelet bases, see
Revised: This is a consequence of atheorem proved by [28] on the representation of classes of singular operators in wavelet bases, see

Original: Appendix S4.
Revised: Appendix S3.

Original: As a result, the number of iterations T/δ required to reach an error ε is independent ofthe dimension.
Revised: As a result, the number of iterations T/δ required to reach an error ε is independent ofthe dimension.

Original: Theorem 3.
Revised: Theorem 3.

Original: Let x be a Gaussian stationary process of power spectrum P ( ω )
Revised: Let x be a Gaussian stationary process of power spectrum P ( ω )

Original: =
Revised: =

Original: c ( ξ η + | ω | η ) − 1with η > 0 and ξ > 0 .
Revised: c ( ξ η + | ω | η ) − 1with η > 0 and ξ > 0 .

Original: If the wavelet has a compact support, q ≥ η vanishing moments and is C q ,then KL( p ∥ ˜ p 0 ) =
Revised: If the wavelet has a compact support, q ≥ η vanishing moments and is C q ,then KL( p ∥ ˜ p 0 ) =

Original: E T + E δ
Revised: E T + E δ

Original: + E δ,T with E δ,T a higher order term such that E T,δ
Revised: + E δ,T with E δ,T a higher order term such that E T,δ

Original: = o ( δ + e − 4 T )when δ → 0 and T → + ∞ .
Revised: = o ( δ + e − 4 T )when δ → 0 and T → + ∞ .

Original: Furthermore, for any ε > 0 , there exists C > 0 such that for any δ , T :
Revised: Furthermore, for any ε > 0 , there exists C > 0 such that for any δ , T :

Original: [Equation]
Revised: [Equation]

Original: To prove this result, we show that the conditioning number of the covariance matrix of the renormalized wavelet coefficients does not depend on the dimension using [32].
Revised: To prove this result, we show that the conditioning number of the covariance matrix of the renormalized wavelet coefficients does not depend on the dimension using [28].

Original: We conclude upon combiningthis result, the cascading property of the Kullback-Leibler divergence and an extension of Theorem 1to the setting with non-zero mean.
Revised: We conclude upon combiningthis result, the cascading property of the Kullback-Leibler divergence and an extension of Theorem 1to the setting with non-zero mean.

Original: The detailed proof is postponed to Appendix S6.
Revised: The detailed proof is postponed to Appendix S5.

Original: Numerical results We illustrate Theorem 3 on a Gaussian field x , with power spectrum P as (12).
Revised: Numerical results We illustrate Theorem 3 on a Gaussian field x , with power spectrum P as (11).

Original: In Figure 2, we display the sup-norm between P and the power spectrum
Revised: In Figure 2, we display the sup-norm between P and the power spectrum

Original: ˆ P of the samples obtainedusing either vanilla SGM or WSGM with uniform stepsize δ k
Revised: ˆ P of the samples obtainedusing either vanilla SGM or WSGM with uniform stepsize δ k

Original: = δ .
Revised: = δ .

Original: In the case of vanilla SGM, thenumber N ( ε ) of time steps needed to reach a small error ∥
Revised: In the case of vanilla SGM, thenumber N ( ε ) of time steps needed to reach a small error ∥

Original: P − ˆ P ∥ = ε increases with the size ofthe image L (Fig. 2, right).
Revised: P − ˆ P ∥ = ε increases with the size ofthe image L (Fig. 2, right).

Original: Equation (8) suggests that N ( ε ) scales like a power of the conditioningnumber κ of Σ , which, for multiscale gaussian processes, is κ ∼ L η .
Revised: Equation (8) suggests that N ( ε ) scales like a power of the conditioningnumber κ of Σ , which, for multiscale gaussian processes, is κ ∼ L η .

Original: In the WSGM case, we sampleconditional distributions ¯ p j of wavelets
Revised: In the WSGM case, we sampleconditional distributions ¯ p j of wavelets

Original: ¯ x j given low-frequencies x j .
Revised: ¯ x j given low-frequencies x j .

Original: At scale j , the conditioningnumbers ¯ κ j of the conditional covariance become dimension-independent (Appendix S4), removingthe dependency of N ( ε ) on the image size L as suggested by (16).
Revised: At scale j , the conditioningnumbers ¯ κ j of the conditional covariance become dimension-independent (Appendix S3), removingthe dependency of N ( ε ) on the image size L as suggested by (15). Figure

Original:  Acceleration with WSGM: Numerical Results
Revised:  Acceleration with WSGM: Numerical Results

Original: For multiscale Gaussian processes, we proved that with WSGMs, the number of time steps N ( ε ) toreach a fixed error ε does not depend on the signal size, as opposed to SGMs.
Revised: For multiscale Gaussian processes, we proved that with WSGMs, the number of time steps N ( ε ) toreach a fixed error ε does not depend on the signal size, as opposed to SGMs.

Original: This section shows thatthis result applies to non-Gaussian multiscale processes.
Revised: This section shows thatthis result applies to non-Gaussian multiscale processes.

Original: We consider a physical process near a phasetransition and images from the CelebA-HQ database [19].
Revised: We consider a physical process near a phasetransition and images from the CelebA-HQ database [17].

Original: 4.1 Physical Processes with Scalar Potentials
Revised: 4.1 Physical Processes with Scalar Potentials

Original: Gaussian stationary processes are maximum entropy processes conditioned by second order momentsdefined by a circulant matrix.
Revised: Gaussian stationary processes are maximum entropy processes conditioned by second order momentsdefined by a circulant matrix.

Original: More complex physical processes are modeled by imposing a constrainton their marginal distribution, with a so-called scalar potential.
Revised: More complex physical processes are modeled by imposing a constrainton their marginal distribution, with a so-called scalar potential.

Original: The marginal distribution of x is theprobability distribution of x ( u ) , which does not depend upon u if x is stationary.
Revised: The marginal distribution of x is theprobability distribution of x ( u ) , which does not depend upon u if x is stationary.

Original: Maximum entropyprocesses conditioned by second order moments and marginal distributions have a probability densitywhich is a Gibbs distribution p ( x ) =
Revised: Maximum entropyprocesses conditioned by second order moments and marginal distributions have a probability densitywhich is a Gibbs distribution p ( x ) =

Original: Z − 1 e − E ( x ) with:
Revised: Z − 1 e − E ( x ) with:

Original: [Equation]
Revised: [Equation]

Original:  where C is a circulant matrix and V : R → R is a scalar potential.
Revised: (16) where C is a circulant matrix and V : R → R is a scalar potential.

Original: Appendix S8 explains how toparametrize V as a linear combination of a family of fixed elementary functions.
Revised: Appendix S7.3 explains how toparametrize V as a linear combination of a family of fixed elementary functions.

Original: The φ 4 model is aparticular example where C = ∆ is a Laplacian and V is a fourth order polynomial, adjusted in orderto imposes that x ( u ) = ± 1 with high probability.
Revised: The φ 4 model is aparticular example where C = ∆ is a Laplacian and V is a fourth order polynomial, adjusted in order to imposes that x ( u ) = ± 1 with high probability.

Original: For so-called critical values of these parameters,the resulting process becomes multiscale with long range interactions and a power law spectrum, see
Revised: For so-called critical values of these parameters,the resulting process becomes multiscale with long range interactions and a power law spectrum, see

Original: Figure 3-(c).
Revised: Figure 3-(c).

Original: We train SGMs and WSGMs on critical φ 4 processes of different sizes; for the score model s θ ,we use a simple linear parametrization detailed in Appendix S8.2.
Revised: We train SGMs and WSGMs on critical φ 4 processes of different sizes; for the score model s θ ,we use a simple linear parametrization detailed in Appendix S7.2.

Original: To evaluate the quality of thegenerated samples, it is sufficient to verify that these samples have the same second order momentand marginals as φ 4 .
Revised: To evaluate the quality of thegenerated samples, it is sufficient to verify that these samples have the same second order momentand marginals as φ 4 .

Original: We define the error metric as the sum of the L 2 error on the power spectrum andthe total-variation distance between marginal distributions.
Revised: We define the error metric as the sum of the L 2 error on the power spectrum andthe total-variation distance between marginal distributions.

Original: Figure 3-(a) shows the decay of this erroras a function of the number of time steps used in an SGM and WSGM with a uniform discretization.
Revised: Figure 3-(a) shows the decay of this erroras a function of the number of time steps used in an SGM and WSGM with a uniform discretization.

Original: With vanilla SGM, the loss has a strong dependency in L , but becomes almost independent of Lfor WSGM.
Revised: With vanilla SGM, the loss has a strong dependency in L , but becomes almost independent of Lfor WSGM.

Original: This empirically verifies the claim that an ill-conditioned covariance matrix leads toslow sampling of SGM, and that WSGM is unaffected by this issue by working with the conditionaldistributions of normalized wavelet coefficients.
Revised: This empirically verifies the claim that an ill-conditioned covariance matrix leads toslow sampling of SGM, and that WSGM is unaffected by this issue by working with the conditionaldistributions of normalized wavelet coefficients.

Original: 4.2 Scale-Wise Time Reduction in Natural Images
Revised: 4.2 Scale-Wise Time Reduction in Natural Images

Original: Images are highly non-Gaussian multiscale processes whose power spectrum has a power law decay.
Revised: Images are highly non-Gaussian multiscale processes whose power spectrum has a power law decay.

Original: We now show that WSGM also provides an acceleration over SGM in this case, by being independentof the image size.
Revised: We now show that WSGM also provides an acceleration over SGM in this case, by being independentof the image size.

Original: We focus on the CelebA-HQ dataset [27] at a 128 × 128 resolution.
Revised: We focus on the CelebA-HQ dataset [24] at a 128 × 128 resolution.

Original: Though non-stationary, its powerspectrum still has a power law decay, as shown in Figure 4.
Revised: Though non-stationary, its powerspectrum still has a power law decay, as shown in Figure 4.

Original: We compare SGM [11] samples atthe 128 × 128 resolution with WSGM samples which start from the 32 × 32 resolution.
Revised: We compare SGM [10] samples atthe 128 × 128 resolution with WSGM samples which start from the 32 × 32 resolution.

Original: Thoughsmaller, the 32 × 32 resolution still suffers from a power law decay of its spectrum over several ordersof magnitude.
Revised: Thoughsmaller, the 32 × 32 resolution still suffers from a power law decay of its spectrum over several ordersof magnitude.

Original: The reason why we limit this coarsest resolution is because border effects becomedominant at lower image sizes.
Revised: The reason why we limit this coarsest resolution is because border effects becomedominant at lower image sizes.

Original: To simplify the handling of border conditions, we use Haar wavelets.
Revised: To simplify the handling of border conditions, we use Haar wavelets.

Original: Following [36], the global scores s θ ( x ) are parametrized by a neural network with a UNet architecture.
Revised: Following [32], the global scores s θ ( x ) are parametrized by a neural network with a UNet architecture.

Original: It has 3 residual blocks at each scale, and includes multi-head attention layers at lower scales.
Revised: It has 3 residual blocks at each scale, and includes multi-head attention layers at lower scales.

Original: Theconditional scores s θ (¯ x j | x j ) are parametrized in the same way, and the conditioning on the lowfrequencies x j is done with a simple input concatenation along channels [36, 39].
Revised: Theconditional scores s θ (¯ x j | x j ) are parametrized in the same way, and the conditioning on the lowfrequencies x j is done with a simple input concatenation along channels [32, 34].

Original: The details of thearchitecture are in Appendix S9.
Revised: The details of thearchitecture are in Appendix S8.

Original: We use a uniform discretization of the backward SDE to stay in thesetting of Theorem 2.
Revised: We use a uniform discretization of the backward SDE to stay in thesetting of Theorem 2.

Original: The generation results are given in Figure 4.
Revised: The generation results are given in Figure 4.

Original: With the same computational budget of 16 discretizationssteps at the largest scale (iterations at smaller scales having a negligible cost due to the exponentialdecrease in image size), WSGM achieves a much better perceptual generation quality.
Revised: With the same computational budget of 16 discretizationssteps at the largest scale (iterations at smaller scales having a negligible cost due to the exponentialdecrease in image size), WSGM achieves a much better perceptual generation quality.

Original: Notably, SGMgenerates noisy images due to discretization errors.
Revised: Notably, SGMgenerates noisy images due to discretization errors.

Original: This is confirmed quantitatively with the Fréchet
Revised: This is confirmed quantitatively with the Fréchet

Original: Inception Distance (FID) [10].
Revised: Inception Distance (FID) [9].

Original: The FID of the WSGM generations decreases with the number ofsteps, until it plateaus.
Revised: The FID of the WSGM generations decreases with the number of steps,until it plateaus.

Original: This plateau is reached with at least 2 orders of magnitude less steps for WSGM than SGM.
Revised: This plateau is reached with at least 2 orders of magnitude less steps for WSGMthan SGM.

Original: This number of steps is also independent of the image size for WSGM, thusconfirming the intuition given in the Gaussian case by Theorems 1-3.
Revised: This number of steps is also independent of the image size for WSGM, thus confirmingthe intuition given in the Gaussian case by Theorems 1-3.

Original: Our results confirm that vanilla[Figure] SGM on a wide range of multiscale processes, including natural images, suffers from ill-conditioning,in the sense that the number of discretization steps grows with the image size.
Revised: Our results confirm that vanilla SGM on awide range of multiscale processes, including natural images, suffers from ill-conditioning, in thesense that the number of discretization steps grows with the image size.

Original: WSGM, on the contrary,leads to uniform discretization schemes whose number of steps at each scale does not depend on theimage size.
Revised: WSGM, on the contrary,leads to uniform discretization schemes whose number of steps at each scale does not depend on theimage size.

Original:  We also stress that there exists many techniques [18, 17, 26, 52, 41, 35, 42, 23, 11, 28, 40, 50] toaccelerate the sampling of vanilla SGMs, with sometimes better FID-time complexity tradeoff curves.
Revised: [Figure] We also stress that there exist many techniques [16, 15, 23, 46, 36, 31, 37, 20, 10, 25, 35, 45] toaccelerate the sampling of vanilla SGMs, with sometimes better FID-time complexity tradeoff curves.

Original: Notably, the FID plateaus at a relatively high value of 20 because the coarsest resolution 32 × 32 isstill ill-conditioned and requires thousands of steps with a non-uniform discretization schedule toachieve FIDs less than 10 with vanilla SGM [36].
Revised: Notably, the FID plateaus at a relatively high value of 20 because the coarsest resolution 32 × 32 isstill ill-conditioned and requires thousands of steps with a non-uniform discretization schedule toachieve FIDs less than 10 with vanilla SGM [32].

Original: Such improvements (including proper handling ofborder conditions) are beyond of the scope of this paper.
Revised: Such improvements (including proper handling ofborder conditions) are beyond of the scope of this paper.

Original: The contribution of WSGM is rather to showthe reason behind this sampling inefficiency and mathematically prove in the Gaussian setting thatwavelet decompositions of the probability distribution allow solving this problem.
Revised: The contribution of WSGM is rather to showthe reason behind this sampling inefficiency and mathematically prove in the Gaussian setting thatwavelet decompositions of the probability distribution allow solving this problem.

Original: We believe thatextending this theoretical result to a wider class of non-Gaussian multiscale processes and combining
Revised: We believe thatextending this theoretical result to a wider class of non-Gaussian multiscale processes and combining

Original: WSGM with other sampling accelerations are interesting avenues for future work.
Revised: WSGM with other sampling accelerations are interesting avenues for future work.

Original:  Discussion
Revised:  Discussion

Original: This paper introduces a Wavelet Score-based Generative Model (WSGM) which applies an SGMto normalized wavelet coefficients conditioned by lower frequencies.
Revised: This paper introduces a Wavelet Score-based Generative Model (WSGM) which applies an SGMto normalized wavelet coefficients conditioned by lower frequencies.

Original: We prove that the number ofstepsizes in SGMs is controlled by the regularity of the score of the target distribution.
Revised: We prove that the number ofstepsizes in SGMs is controlled by the regularity of the score of the target distribution.

Original: For multiscaleprocesses such as images, it requires a considerable number of time steps to achieve a good accuracy,which increases quickly with the image size.
Revised: For multiscaleprocesses such as images, it requires a considerable number of time steps to achieve a good accuracy,which increases quickly with the image size.

Original: We show that a WSGM eliminates ill-conditioningissues by normalizing wavelet coefficients.
Revised: We show that a WSGM eliminates ill-conditioningissues by normalizing wavelet coefficients.

Original: As a result, the number of stepsizes in WSGM does notincrease with the image dimension.
Revised: As a result, the number of stepsizes in WSGM does notincrease with the image dimension.

Original: We illustrated our results on Gaussian distributions, physicalprocesses and image datasets.
Revised: We illustrated our results on Gaussian distributions, physicalprocesses and image datasets.

Original: One of the main limitations of WSGM is that it is limited to multiscale processes for which theconditional wavelet probabilities are close to white Gaussian distributions.
Revised: One of the main limitations of WSGM is that it is limited to multiscale processes for which theconditional wavelet probabilities are close to white Gaussian distributions.

Original: A promising directionfor future work is to combine WSGM with other acceleration techniques such as adaptive timediscretizations to handle such cases.
Revised: A promising directionfor future work is to combine WSGM with other acceleration techniques such as adaptive timediscretizations to handle such cases.

Original: In another direction, one could strengthen the theoretical study of
Revised: In another direction, one could strengthen the theoretical study of

Original: SGM and extend our results beyond the Gaussian setting, in order to fully describe SGM on physicalprocesses which can be seen as perturbations of Gaussian distributions.
Revised: SGM and extend our results beyond the Gaussian setting, in order to fully describe SGM on physicalprocesses which can be seen as perturbations of Gaussian distributions.

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See the end of Section
Revised: [Yes] See the end of Section

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] Like all
Revised: [No] Like all

Original: generative models, WCRG can reproduce or amplify biases present in the data, or could
Revised: generative models, WCRG can reproduce or amplify biases present in the data, or could

Original: be used to generate deceptive content.
Revised: be used to generate deceptive content.

Original: We stress that this work is theoretical in nature
Revised: We stress that this work is theoretical in nature

Original: and should not be used for practical applications without a careful consideration of the
Revised: and should not be used for practical applications without a careful consideration of the

Original: aforementioned issues.
Revised: aforementioned issues.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] Assumptions
Revised: [Yes] Assumptions

Original: are stated in the respective theorems.
Revised: are stated in the respective theorems.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] See Appendices S
Revised: [Yes] See Appendices S

Original: and S6.
Revised: and S5.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes] All the codeneeded to reproduce the experiments is provided in the supplemental material.
Revised: [Yes] All the code needed to reproduce the experiments is provided in the supplemental material.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes] See Appendix S9.
Revised: [Yes] See Appendix S8.

Original: [Table] (c) Did you report error bars (e.g., with respect to the random seed after running exper- [Table] iments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [No] Standard errors are negligible for the comparisons weperform.
Revised: [No] Standard errors are negligible for the comparisons we

Original: [Table]
Revised: perform.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?[No] We estimate the total amount of
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Table] compute used during the preparation of this paper, including preliminary experiments, [Table] at around 10k hours of NVIDIA A100 GPUs.
Revised: [No] We estimate the total amount ofcompute used during the preparation of this paper, including preliminary experiments,is around 10k hours of NVIDIA A100 GPUs.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: [Table]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [No]
Revised: [No]

Original: [Table]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Original: [Table] (d) Did you discuss whether and how consent was obtained from people whose data you’re[Table]
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [No]
Revised: [No]

Original: [Table] (e) Did you discuss whether the data you are using/curating contains personally identifiable[Table]
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [No]
Revised: [No]

Original: [Table]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: [Table]

Original: (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A]
Revised: [N/A]

Original: [Table]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: [Table]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: [Table]

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]