PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior

Denoising diffusion probabilistic models have been recently proposed to generate high-quality samples by estimating the gradient of the data density . The framework defines the prior noise as a standard Gaussian distribution , whereas the corresponding data distribution may be more complicated than the standard Gaussian distribution , which potentially introduces inefficiency in denoising the prior noise into the data sample because of the discrepancy between the data and the prior . In this paper , we propose PriorGrad to improve the efficiency of the conditional diffusion model for speech synthesis ( for example , a vocoder using a mel-spectrogram as the condition ) by applying an adaptive prior derived from the data statistics based on the conditional information . We formulate the training and sampling procedures of PriorGrad and demonstrate the advantages of an adaptive prior through a theoretical analysis . Focusing on the speech synthesis domain , we consider the recently proposed diffusion-based speech generative models based on both the spectral and time domains and show that PriorGrad achieves faster convergence and inference with superior performance , leading to an improved perceptual quality and robustness to a smaller network capacity , and thereby demonstrating the efficiency of a data-dependent adaptive prior . 1 INTRODUCTION . Deep generative models have been achieving rapid progress , by which deep neural networks approximate the data distribution and synthesize realistic samples from the model . There is a wide range of this type of approach , ranging from autoregressive models ( Oord et al. , 2016a ; b ) , generative adversarial networks ( Goodfellow et al. , 2014 ; Brock et al. , 2019 ) , variational autoencoders ( Kingma & Welling , 2013 ; Vahdat & Kautz , 2020 ) , and normalizing flows ( Rezende & Mohamed , 2015 ; Kingma & Dhariwal , 2018 ) . Denoising diffusion probabilistic models ( DDPMs ) ( Ho et al. , 2020 ) and score matching ( SM ) ( Song & Ermon , 2019 ) are recently proposed categories that can be used to synthesize high-fidelity samples with competitive or sometimes better quality than previous state-of-the-art approaches . Consequently , there have been a variety of applications based on DDPM or SM ( Saharia et al. , 2021 ; Kawar et al. , 2021 ) . Speech synthesis is one of the most successful applications , where the diffusion model can synthesize spectral or time-domain audio conditioned on text or spectral information , respectively , achieving a competitive quality but faster sampling ( Chen et al. , 2021 ; Kong et al. , 2021 ; Jeong et al. , 2021 ; Lee & Han , 2021 ) than autoregressive models ( Oord et al. , 2016b ; Kalchbrenner et al. , 2018 ) . ∗Work done during an internship at Microsoft Research Asia †Corresponding Authors However , although the diffusion-based speech synthesis models have achieved high-quality speech audio generation , they exhibit potential inefficiency , which may necessitate advanced strategies . For example , the model suffers from a significantly slow convergence during training , and a prohibitively large training computation time is required to learn the approximate reverse diffusion process . We investigate the diffuion-based models and observe the discrepancy between the real data distribution and the choice of the prior . Existing diffusion-based models define a standard Gaussian as the prior distribution and design a non-parametric diffusion process that procedurally destroys the signal into the prior noise . The deep neural network is trained to approximate the reverse diffusion process by estimating the gradient of the data density . Although applying the standard Gaussian as the prior is simple without any assumptions on the target data , it also introduces inefficiency . For example , in time-domain waveform data , the signal has extremely high variability between different segments such as voiced and unvoiced parts . Jointly modeling the voiced and unvoiced segments with the same standard Gaussian prior may be difficult for the model to cover all modes of the data , leading to training inefficiencies and potentially spurious diffusion trajectories . Given the previous reasoning , we assessed the following question : For a conditional diffusion-based model , can we formulate a more informative prior without incorporating additional computational or parameter complexity ? To investigate this , we propose a simple yet effective method , called PriorGrad , that uses adaptive noise by directly computing the mean and variance for the forward diffusion process prior , based on the conditional information . Specifically , using a conditional speech synthesis model , we propose structuring the prior distribution based on the conditional data , such as a mel-spectrogram for the vocoder ( Chen et al. , 2021 ; Kong et al. , 2021 ) and a phoneme for the acoustic model ( Jeong et al. , 2021 ) . By computing the statistics from the conditional data at the frame level ( vocoder ) or phoneme-level ( acoustic model ) granularity and mapping them as the mean and variance of the Gaussian prior , we can structure the noise that is similar to the target data distribution at an instance level , easing the burden of learning the reverse diffusion process . We implemented PriorGrad based on the recently proposed diffusion-based speech generative models ( Kong et al. , 2021 ; Chen et al. , 2021 ; Jeong et al. , 2021 ) , and conducted experiments on the LJSpeech ( Ito & Johnson , 2017 ) dataset . The experimental results demonstrate the benefits of PriorGrad , such as a significantly faster model convergence during training , improved perceptual quality , and an improved tolerance to a reduction in network capacity . Our contributions are as follows : • To the best of our knowledge , our study is one of the first to systematically investigate the effect of using a non-standard Gaussian distribution as the forward diffusion process prior to the conditional generative model . • Compared to previous non-parametric forward diffusion without any assumption , we show that the model performance is significantly improved with faster convergence by leveraging the conditional information as the adaptive prior . • We provide a comprehensive empirical study and analysis of the diffusion model behavior in speech generative models , in both the spectral and waveform domains , and demonstrate the effectiveness of the method , such as a significantly accelerated inference and improved quality . 2 BACKGROUND . In this section , we describe the basic formulation of the diffusion-based model and provide related studies , along with a description of our contribution with PriorGrad . Basic formulation Denoising diffusion probabilistic models ( DDPM ) ( Ho et al. , 2020 ) are recently proposed deep generative models defined by two Markov chains : forward and reverse processes . The forward process procedurally destroys the data x0 into a standard Gaussian xT , as follows : q ( x1 : T |x0 ) = T∏ t=1 q ( xt|xt−1 ) , q ( xt|xt−1 ) : = N ( xt ; √ 1− βtxt−1 , βtI ) , ( 1 ) where q ( xt|xt−1 ) represents the transition probability at the t-th step using a user-defined noise schedule βt ∈ { β1 , ... , βT } . Thus , the noisy distribution of xt is the closed form of q ( xt|x0 ) = N ( xt ; √ ᾱtx0 , ( 1− ᾱt ) I ) , where αt : = 1−βt , ᾱt : = ∏t s=1 αs . q ( xT |x0 ) converges in distribution to the standard Gaussian N ( xT ; 0 , I ) if ᾱT is small enough based on a carefully designed noise schedule . The reverse process that procedurally transforms the prior noise into data is defined as follows : pθ ( x0 : T ) = p ( xT ) T∏ t=1 pθ ( xt−1|xt ) , pθ ( xt−1|xt ) = N ( xt−1 ; µθ ( xt , t ) , Σθ ( xt , t ) ) , ( 2 ) where p ( xT ) = N ( xT ; 0 , I ) and pθ ( xt−1|xt ) corresponds to the reverse of the forward transition probability , parameterized using a deep neural network . We can define the evidence lower bound ( ELBO ) loss as the training objective of the reverse process : L ( θ ) = Eq [ KL ( q ( xT |x0 ) ||p ( xT ) ) + T∑ t=2 KL ( q ( xt−1|xt , x0 ) ||pθ ( xt−1|xt ) ) − log pθ ( x0|x1 ) ] . ( 3 ) As shown in Ho et al . ( 2020 ) , q ( xt−1|xt , x0 ) can be represented by Bayes rule as follows : q ( xt−1|xt , x0 ) = N ( xt−1 ; µ̃ ( xt , x0 ) , β̃tI ) , ( 4 ) µ̃t ( xt , x0 ) : = √ ᾱt−1βt 1− ᾱt x0 + √ αt ( 1− ᾱt−1 ) 1− ᾱt xt , β̃t : = 1− ᾱt−1 1− ᾱt βt . ( 5 ) By fixing p ( xT ) as a standard Gaussian , KL ( q ( xT |x0 ) ||p ( xT ) ) becomes constant and is not parameterized . The original framework in Ho et al . ( 2020 ) fixed Σθ ( xt , t ) as a constant β̃tI and set the standard Gaussian noise as the optimization target instead of µ̃t by reparameterizing x0 = 1√ ᾱt ( xt − √ 1− ᾱt ) from q ( xt|x0 ) to minimize the second and third terms in equation 3 . Based on this setup , in Ho et al . ( 2020 ) , the authors further demonstrated that we can drop the weighting factor of each term and use a simplified training objective that provides a higher sample quality : − ELBO = C + T∑ t=1 Ex0 , [ β2t 2σ2tαt ( 1− ᾱt ) ‖ − θ ( √ ᾱtx0 + √ 1− ᾱt , t ) ‖2 ] , ( 6 ) Lsimple ( θ ) : = Et , x0 , [ ‖ − θ ( xt , t ) ‖2 ] . ( 7 ) Related work Since the introduction of the DDPM , there have been a variety of further studies ( Nichol & Dhariwal , 2021 ; Song et al. , 2020 ) , applications ( Chen et al. , 2021 ; Jeong et al. , 2021 ; Kong et al. , 2021 ; Saharia et al. , 2021 ) , and a symbiosis of diffusion ( Ho et al. , 2020 ; Sohl-Dickstein et al. , 2015 ) and score-based models ( Song & Ermon , 2019 ; 2020 ) as a unified view with stochastic differential equations ( SDEs ) ( Song et al. , 2021 ) . From an application perspective , several conditional generative models have been proposed . Waveform synthesis models ( Chen et al. , 2021 ; Kong et al. , 2021 ) are one of the major applications in which the diffusion model is trained to generate time-domain speech audio from the prior noise , conditioned on a mel-spectrogram . A diffusionbased decoder has also been applied to text-to-spectrogram generation models ( Jeong et al. , 2021 ; Popov et al. , 2021 ) . PriorGrad focuses on improving the efficiency of training such methods from the perspective of a conditional generative model . We investigate the potential inefficiency of the current methods which require unfeasibly large computing resources to train and generate high-quality samples . Studies on formulating an informative prior distribution for deep generative model are not new , and there has been a variety of studies investigating a better prior , ranging from hand-crafted ( Nalisnick & Smyth , 2017 ; Tomczak & Welling , 2018 ) , autoregressive ( Chen et al. , 2017 ) , vector quantization ( Razavi et al. , 2019 ) , prior encoder ( Rezende & Mohamed , 2015 ) , and data-dependent approaches similar to ours ( Li et al. , 2019 ) . We tackle the problem of training inefficiency of diffusion-based models by crafting better priors in a data-dependent manner , where our method can provide a better trajectory and can reduce spurious modes , enabling more efficient training . Nachmani et al . ( 2021 ) used Gamma distribution as the diffusion prior . Note that there has also been a concurrent study conducted on leveraging the prior distribution on the acoustic model , Grad-TTS ( Popov et al. , 2021 ) , in which the effectiveness of using the mean-shifted Gaussian as a prior with the identity variance was investigated . Unlike the method in Popov et al . ( 2021 ) , which enforces the encoder output to match the target mel-spectrogram by using an additional encoder loss , our approach augments the forward diffusion prior directly through data and the encoder has no restriction on latent feature representations . In Popov et al . ( 2021 ) , the forward diffusion prior is jointly trained and may induce additional overhead on convergence as the prior changes throughout the training , whereas our method provides guaranteed convergence through the fixed informative prior .

Summary: This work builds on denoising diffusion probabilistic models (DDPM), and argues for to modify the forward and backward diffusion processes such that instead of using an uninformative prior $p(\mathbf{x}_T) = \mathcal{N}(0,\mathbf{I})$, they use a data-dependent prior $p(\mathbf{x}_T) = \mathcal{N}(\mu, \Sigma)$, where the mean and covariance are derived from training data, in a "pre-processing" step.  The authors show that, under restrictive conditions, the ELBO obtained with the proposed prior is smaller than that obtained with the uninformative prior. Under similar restrictive conditions, the authors argue that convergence rate of the parameter optimization is better conditioned, and as a consequence faster.  This work illustrates the benefits of the proposed method using two applications, one to a vocoder, and one to an acoustic model. Experiments rely on several very recent related work, especially concerning technical details of the architectures, and data pre-processing choices. Overall, two methods are compared, a baseline DDPM and the proposed DDPM that uses training data to build a better prior.