PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior

Denoising diffusion probabilistic models have been recently proposed to generate high-quality samples by estimating the gradient of the data density . The framework defines the prior noise as a standard Gaussian distribution , whereas the corresponding data distribution may be more complicated than the standard Gaussian distribution , which potentially introduces inefficiency in denoising the prior noise into the data sample because of the discrepancy between the data and the prior . In this paper , we propose PriorGrad to improve the efficiency of the conditional diffusion model for speech synthesis ( for example , a vocoder using a mel-spectrogram as the condition ) by applying an adaptive prior derived from the data statistics based on the conditional information . We formulate the training and sampling procedures of PriorGrad and demonstrate the advantages of an adaptive prior through a theoretical analysis . Focusing on the speech synthesis domain , we consider the recently proposed diffusion-based speech generative models based on both the spectral and time domains and show that PriorGrad achieves faster convergence and inference with superior performance , leading to an improved perceptual quality and robustness to a smaller network capacity , and thereby demonstrating the efficiency of a data-dependent adaptive prior . 1 INTRODUCTION . Deep generative models have been achieving rapid progress , by which deep neural networks approximate the data distribution and synthesize realistic samples from the model . There is a wide range of this type of approach , ranging from autoregressive models ( Oord et al. , 2016a ; b ) , generative adversarial networks ( Goodfellow et al. , 2014 ; Brock et al. , 2019 ) , variational autoencoders ( Kingma & Welling , 2013 ; Vahdat & Kautz , 2020 ) , and normalizing flows ( Rezende & Mohamed , 2015 ; Kingma & Dhariwal , 2018 ) . Denoising diffusion probabilistic models ( DDPMs ) ( Ho et al. , 2020 ) and score matching ( SM ) ( Song & Ermon , 2019 ) are recently proposed categories that can be used to synthesize high-fidelity samples with competitive or sometimes better quality than previous state-of-the-art approaches . Consequently , there have been a variety of applications based on DDPM or SM ( Saharia et al. , 2021 ; Kawar et al. , 2021 ) . Speech synthesis is one of the most successful applications , where the diffusion model can synthesize spectral or time-domain audio conditioned on text or spectral information , respectively , achieving a competitive quality but faster sampling ( Chen et al. , 2021 ; Kong et al. , 2021 ; Jeong et al. , 2021 ; Lee & Han , 2021 ) than autoregressive models ( Oord et al. , 2016b ; Kalchbrenner et al. , 2018 ) . ∗Work done during an internship at Microsoft Research Asia †Corresponding Authors However , although the diffusion-based speech synthesis models have achieved high-quality speech audio generation , they exhibit potential inefficiency , which may necessitate advanced strategies . For example , the model suffers from a significantly slow convergence during training , and a prohibitively large training computation time is required to learn the approximate reverse diffusion process . We investigate the diffuion-based models and observe the discrepancy between the real data distribution and the choice of the prior . Existing diffusion-based models define a standard Gaussian as the prior distribution and design a non-parametric diffusion process that procedurally destroys the signal into the prior noise . The deep neural network is trained to approximate the reverse diffusion process by estimating the gradient of the data density . Although applying the standard Gaussian as the prior is simple without any assumptions on the target data , it also introduces inefficiency . For example , in time-domain waveform data , the signal has extremely high variability between different segments such as voiced and unvoiced parts . Jointly modeling the voiced and unvoiced segments with the same standard Gaussian prior may be difficult for the model to cover all modes of the data , leading to training inefficiencies and potentially spurious diffusion trajectories . Given the previous reasoning , we assessed the following question : For a conditional diffusion-based model , can we formulate a more informative prior without incorporating additional computational or parameter complexity ? To investigate this , we propose a simple yet effective method , called PriorGrad , that uses adaptive noise by directly computing the mean and variance for the forward diffusion process prior , based on the conditional information . Specifically , using a conditional speech synthesis model , we propose structuring the prior distribution based on the conditional data , such as a mel-spectrogram for the vocoder ( Chen et al. , 2021 ; Kong et al. , 2021 ) and a phoneme for the acoustic model ( Jeong et al. , 2021 ) . By computing the statistics from the conditional data at the frame level ( vocoder ) or phoneme-level ( acoustic model ) granularity and mapping them as the mean and variance of the Gaussian prior , we can structure the noise that is similar to the target data distribution at an instance level , easing the burden of learning the reverse diffusion process . We implemented PriorGrad based on the recently proposed diffusion-based speech generative models ( Kong et al. , 2021 ; Chen et al. , 2021 ; Jeong et al. , 2021 ) , and conducted experiments on the LJSpeech ( Ito & Johnson , 2017 ) dataset . The experimental results demonstrate the benefits of PriorGrad , such as a significantly faster model convergence during training , improved perceptual quality , and an improved tolerance to a reduction in network capacity . Our contributions are as follows : • To the best of our knowledge , our study is one of the first to systematically investigate the effect of using a non-standard Gaussian distribution as the forward diffusion process prior to the conditional generative model . • Compared to previous non-parametric forward diffusion without any assumption , we show that the model performance is significantly improved with faster convergence by leveraging the conditional information as the adaptive prior . • We provide a comprehensive empirical study and analysis of the diffusion model behavior in speech generative models , in both the spectral and waveform domains , and demonstrate the effectiveness of the method , such as a significantly accelerated inference and improved quality . 2 BACKGROUND . In this section , we describe the basic formulation of the diffusion-based model and provide related studies , along with a description of our contribution with PriorGrad . Basic formulation Denoising diffusion probabilistic models ( DDPM ) ( Ho et al. , 2020 ) are recently proposed deep generative models defined by two Markov chains : forward and reverse processes . The forward process procedurally destroys the data x0 into a standard Gaussian xT , as follows : q ( x1 : T |x0 ) = T∏ t=1 q ( xt|xt−1 ) , q ( xt|xt−1 ) : = N ( xt ; √ 1− βtxt−1 , βtI ) , ( 1 ) where q ( xt|xt−1 ) represents the transition probability at the t-th step using a user-defined noise schedule βt ∈ { β1 , ... , βT } . Thus , the noisy distribution of xt is the closed form of q ( xt|x0 ) = N ( xt ; √ ᾱtx0 , ( 1− ᾱt ) I ) , where αt : = 1−βt , ᾱt : = ∏t s=1 αs . q ( xT |x0 ) converges in distribution to the standard Gaussian N ( xT ; 0 , I ) if ᾱT is small enough based on a carefully designed noise schedule . The reverse process that procedurally transforms the prior noise into data is defined as follows : pθ ( x0 : T ) = p ( xT ) T∏ t=1 pθ ( xt−1|xt ) , pθ ( xt−1|xt ) = N ( xt−1 ; µθ ( xt , t ) , Σθ ( xt , t ) ) , ( 2 ) where p ( xT ) = N ( xT ; 0 , I ) and pθ ( xt−1|xt ) corresponds to the reverse of the forward transition probability , parameterized using a deep neural network . We can define the evidence lower bound ( ELBO ) loss as the training objective of the reverse process : L ( θ ) = Eq [ KL ( q ( xT |x0 ) ||p ( xT ) ) + T∑ t=2 KL ( q ( xt−1|xt , x0 ) ||pθ ( xt−1|xt ) ) − log pθ ( x0|x1 ) ] . ( 3 ) As shown in Ho et al . ( 2020 ) , q ( xt−1|xt , x0 ) can be represented by Bayes rule as follows : q ( xt−1|xt , x0 ) = N ( xt−1 ; µ̃ ( xt , x0 ) , β̃tI ) , ( 4 ) µ̃t ( xt , x0 ) : = √ ᾱt−1βt 1− ᾱt x0 + √ αt ( 1− ᾱt−1 ) 1− ᾱt xt , β̃t : = 1− ᾱt−1 1− ᾱt βt . ( 5 ) By fixing p ( xT ) as a standard Gaussian , KL ( q ( xT |x0 ) ||p ( xT ) ) becomes constant and is not parameterized . The original framework in Ho et al . ( 2020 ) fixed Σθ ( xt , t ) as a constant β̃tI and set the standard Gaussian noise as the optimization target instead of µ̃t by reparameterizing x0 = 1√ ᾱt ( xt − √ 1− ᾱt ) from q ( xt|x0 ) to minimize the second and third terms in equation 3 . Based on this setup , in Ho et al . ( 2020 ) , the authors further demonstrated that we can drop the weighting factor of each term and use a simplified training objective that provides a higher sample quality : − ELBO = C + T∑ t=1 Ex0 , [ β2t 2σ2tαt ( 1− ᾱt ) ‖ − θ ( √ ᾱtx0 + √ 1− ᾱt , t ) ‖2 ] , ( 6 ) Lsimple ( θ ) : = Et , x0 , [ ‖ − θ ( xt , t ) ‖2 ] . ( 7 ) Related work Since the introduction of the DDPM , there have been a variety of further studies ( Nichol & Dhariwal , 2021 ; Song et al. , 2020 ) , applications ( Chen et al. , 2021 ; Jeong et al. , 2021 ; Kong et al. , 2021 ; Saharia et al. , 2021 ) , and a symbiosis of diffusion ( Ho et al. , 2020 ; Sohl-Dickstein et al. , 2015 ) and score-based models ( Song & Ermon , 2019 ; 2020 ) as a unified view with stochastic differential equations ( SDEs ) ( Song et al. , 2021 ) . From an application perspective , several conditional generative models have been proposed . Waveform synthesis models ( Chen et al. , 2021 ; Kong et al. , 2021 ) are one of the major applications in which the diffusion model is trained to generate time-domain speech audio from the prior noise , conditioned on a mel-spectrogram . A diffusionbased decoder has also been applied to text-to-spectrogram generation models ( Jeong et al. , 2021 ; Popov et al. , 2021 ) . PriorGrad focuses on improving the efficiency of training such methods from the perspective of a conditional generative model . We investigate the potential inefficiency of the current methods which require unfeasibly large computing resources to train and generate high-quality samples . Studies on formulating an informative prior distribution for deep generative model are not new , and there has been a variety of studies investigating a better prior , ranging from hand-crafted ( Nalisnick & Smyth , 2017 ; Tomczak & Welling , 2018 ) , autoregressive ( Chen et al. , 2017 ) , vector quantization ( Razavi et al. , 2019 ) , prior encoder ( Rezende & Mohamed , 2015 ) , and data-dependent approaches similar to ours ( Li et al. , 2019 ) . We tackle the problem of training inefficiency of diffusion-based models by crafting better priors in a data-dependent manner , where our method can provide a better trajectory and can reduce spurious modes , enabling more efficient training . Nachmani et al . ( 2021 ) used Gamma distribution as the diffusion prior . Note that there has also been a concurrent study conducted on leveraging the prior distribution on the acoustic model , Grad-TTS ( Popov et al. , 2021 ) , in which the effectiveness of using the mean-shifted Gaussian as a prior with the identity variance was investigated . Unlike the method in Popov et al . ( 2021 ) , which enforces the encoder output to match the target mel-spectrogram by using an additional encoder loss , our approach augments the forward diffusion prior directly through data and the encoder has no restriction on latent feature representations . In Popov et al . ( 2021 ) , the forward diffusion prior is jointly trained and may induce additional overhead on convergence as the prior changes throughout the training , whereas our method provides guaranteed convergence through the fixed informative prior .

Summary: This paper presents an extension to conditional denoising diffusion models for text-to-speech (TTS). The initial noise in the inference procedure is not sampled from a standard Gaussian but from a data-dependent conditional non-standard Gaussian, which is called PriorGrad. The motivation is to increase inference speed and training convergence. This method is applied to DiffWave used as a vocoder, and also applied to DiffWave used as an acoustic model (phonemes to mel spect).