WaveGrad: Estimating Gradients for Waveform Generation

1 INTRODUCTION . Deep generative models have revolutionized speech synthesis ( Oord et al. , 2016 ; Sotelo et al. , 2017 ; Wang et al. , 2017 ; Biadsy et al. , 2019 ; Jia et al. , 2019 ; Vasquez & Lewis , 2019 ) . Autoregressive models , in particular , have been popular for raw audio generation thanks to their tractable likelihoods , simple inference procedures , and high fidelity samples ( Oord et al. , 2016 ; Mehri et al. , 2017 ; Kalchbrenner et al. , 2018 ; Song et al. , 2019 ; Valin & Skoglund , 2019 ) . However , autoregressive models require a large number of sequential computations to generate an audio sample . This makes it challenging to deploy them in real-world applications where faster than real time generation is essential , such as digital voice assistants on smart speakers , even using specialized hardware . There has been a plethora of research into non-autoregressive models for audio generation , including normalizing flows such as inverse autoregressive flows ( Oord et al. , 2018 ; Ping et al. , 2019 ) , generative flows ( Prenger et al. , 2019 ; Kim et al. , 2019 ) , and continuous normalizing flows ( Kim et al. , 2020 ; Wu & Ling , 2020 ) , implicit generative models such as generative adversarial networks ( GAN ) ( Donahue et al. , 2018 ; Engel et al. , 2019 ; Kumar et al. , 2019 ; Yamamoto et al. , 2020 ; Bińkowski et al. , 2020 ; Yang et al. , 2020a ; b ; McCarthy & Ahmed , 2020 ) and energy score ( Gritsenko et al. , 2020 ) , variational auto-encoder models ( Peng et al. , 2020 ) , as well as models inspired by digital signal processing ( Ai & Ling , 2020 ; Engel et al. , 2020 ) , and the speech production mechanism ( Juvela et al. , 2019 ; Wang et al. , 2020 ) . Although such models improve inference speed by requiring fewer sequential operations , they often yield lower quality samples than autoregressive models . This paper introduces WaveGrad , a conditional generative model of waveform samples that estimates the gradients of the data log-density as opposed to the density itself . WaveGrad is simple to train , and implicitly optimizes for the weighted variational lower-bound of the log-likelihood . ∗Work done during an internship at Google Brain . †Equal contribution . WaveGrad is non-autoregressive , and requires only a constant number of generation steps during inference . Figure 1 visualizes the inference process of WaveGrad . WaveGrad builds on a class of generative models that emerges through learning the gradient of the data log-density , also known as the Stein score function ( Hyvärinen , 2005 ; Vincent , 2011 ) . During inference , one can rely on the gradient estimate of the data log-density and use gradient-based samplers ( e.g. , Langevin dynamics ) to sample from the model ( Song & Ermon , 2019 ) . Promising results have been achieved on image synthesis ( Song & Ermon , 2019 ; 2020 ) and shape generation ( Cai et al. , 2020 ) . Closely related are diffusion probabilistic models ( Sohl-Dickstein et al. , 2015 ) , which capture the output distribution through a Markov chain of latent variables . Although these models do not offer tractable likelihoods , one can optimize a ( weighted ) variational lower-bound on the log-likelihood . The training objective can be reparameterized to resemble deonising score matching ( Vincent , 2011 ) , and can be interpreted as estimating the data log-density gradients . The model is non-autoregressive during inference , requiring only a constant number of generation steps , using a Langevin dynamics-like sampler to generate the output beginning from Gaussian noise . The key contributions of this paper are summarized as follows : • WaveGrad combines recent techniques from score matching ( Song et al. , 2020 ; Song & Ermon , 2020 ) and diffusion probabilistic models ( Sohl-Dickstein et al. , 2015 ; Ho et al. , 2020 ) to address conditional speech synthesis . • We build and compare two variants of the WaveGrad model : ( 1 ) WaveGrad conditioned on a discrete refinement step index following Ho et al . ( 2020 ) , ( 2 ) WaveGrad conditioned on a continuous scalar indicating the noise level . We find this novel continuous variant is more effective , especially because once the model is trained , different number of refinement steps can be used for inference . The proposed continuous noise schedule enables our model to use fewer inference iterations while maintaining the same quality ( e.g. , 6 vs. 50 ) . • We demonstrate that WaveGrad is capable of generating high fidelity audio samples , outperforming adversarial non-autoregressive models ( Yamamoto et al. , 2020 ; Kumar et al. , 2019 ; Yang et al. , 2020a ; Bińkowski et al. , 2020 ) and matching one of the best autoregressive models ( Kalchbrenner et al. , 2018 ) in terms of subjective naturalness . WaveGrad is capable of generating high fidelity samples using as few as six refinement steps . 2 ESTIMATING GRADIENTS FOR WAVEFORM GENERATION . We begin with a brief review of the Stein score function , Langevin dynamics , and score matching . The Stein score function ( Hyvärinen , 2005 ) is the gradient of the data log-density log p ( y ) with respect to the datapoint y : s ( y ) = ∇y log p ( y ) . ( 1 ) Given the Stein score function s ( · ) , one can draw samples from the corresponding density , ỹ ∼ p ( y ) , via Langevin dynamics , which can be interpreted as stochastic gradient ascent in the data space : ỹi+1 = ỹi + η 2 s ( ỹi ) + √ η zi , ( 2 ) where η > 0 is the step size , zi ∼ N ( 0 , I ) , and I denotes an identity matrix . A variant ( Ho et al. , 2020 ) is used as our inference procedure . A generative model can be built by training a neural network to learn the Stein score function directly , using Langevin dynamics for inference . This approach , known as score matching ( Hyvärinen , 2005 ; Vincent , 2011 ) , has seen success in image ( Song & Ermon , 2019 ; 2020 ) and shape ( Cai et al. , 2020 ) generation . The denoising score matching objective ( Vincent , 2011 ) takes the form : Ey∼p ( y ) Eỹ∼q ( ỹ|y ) [ ∥∥∥sθ ( ỹ ) −∇ỹ log q ( ỹ | y ) ∥∥∥2 2 ] , ( 3 ) where p ( · ) is the data distribution , and q ( · ) is a noise distribution . Recently , Song & Ermon ( 2019 ) proposed a weighted denoising score matching objective , in which data is perturbed with different levels of Gaussian noise , and the score function sθ ( ỹ , σ ) is conditioned on σ , the standard deviation of the noise used : ∑ σ∈S λ ( σ ) Ey∼p ( y ) Eỹ∼N ( y , σ ) [ ∥∥∥∥sθ ( ỹ , σ ) + ỹ − yσ2 ∥∥∥∥2 2 ] , ( 4 ) where S is a set of standard deviation values that are used to perturb the data , and λ ( σ ) is a weighting function for different σ. WaveGrad is a variant of this approach applied to learning conditional generative models of the form p ( y | x ) . WaveGrad adopts a similar objective which combines the idea of Vincent ( 2011 ) ; Ho et al . ( 2020 ) ; Song & Ermon ( 2019 ) . WaveGrad learns the gradient of the data density , and uses a sampler similar to Langevin dynamics for inference . The denoising score matching framework relies on a noise distribution to provide support for learning the gradient of the data log density ( i.e. , q in Equation 3 , andN ( · , σ ) in Equation 4 ) . The choice of the noise distribution is critical for achieving high quality samples ( Song & Ermon , 2020 ) . As shown in Figure 2 , WaveGrad relies on the diffusion model framework ( Sohl-Dickstein et al. , 2015 ; Ho et al. , 2020 ) to generate the noise distribution used to learn the score function . 2.1 WAVEGRAD AS A DIFFUSION PROBABILISTIC MODEL . Ho et al . ( 2020 ) observed that diffusion probabilistic models ( Sohl-Dickstein et al. , 2015 ) and score matching objectives ( Song & Ermon , 2019 ; Vincent , 2011 ; Song & Ermon , 2020 ) are closely related . As such , we will first introduce WaveGrad as a diffusion probabilistic model . We adapt the diffusion model setup in Ho et al . ( 2020 ) , from unconditional image generation to conditional raw audio waveform generation . WaveGrad models the conditional distribution pθ ( y0 | Algorithm 1 Training . WaveGrad directly conditions on the continuous noise level √ ᾱ . l is from a predefined noise schedule . 1 : repeat 2 : y0 ∼ q ( y0 ) 3 : s ∼ Uniform ( { 1 , . . . , S } ) 4 : √ ᾱ ∼ Uniform ( ls−1 , ls ) 5 : ∼ N ( 0 , I ) 6 : Take gradient descent step on ∇θ ∥∥ − θ ( √ᾱ y0 +√1− ᾱ , x , √ᾱ ) ∥∥1 7 : until converged Algorithm 2 Sampling . WaveGrad generates samples following a gradient-based sampler similar to Langevin dynamics . 1 : yN ∼ N ( 0 , I ) 2 : for n = N , . . . , 1 do 3 : z ∼ N ( 0 , I ) 4 : yn−1 = ( yn− 1−αn√1−ᾱn θ ( yn , x , √ ᾱn ) ) √ αn 5 : if n > 1 , yn−1 = yn−1 + σnz 6 : end for 7 : return y0 x ) where y0 is the waveform and x contains the conditioning features corresponding to y0 , such as linguistic features derived from the corresponding text , mel-spectrogram features extracted from y0 , or acoustic features predicted by a Tacotron-style text-to-speech synthesis model ( Shen et al. , 2018 ) : pθ ( y0 | x ) : = ∫ pθ ( y0 : N | x ) dy1 : N , ( 5 ) where y1 , . . . , yN is a series of latent variables , each of which are of the same dimension as the data y0 , and N is the number of latent variables ( iterations ) . The posterior q ( y1 : N | y0 ) is called the diffusion process ( or forward process ) , and is defined through the Markov chain : q ( y1 : N | y0 ) : = N∏ n=1 q ( yn | yn−1 ) , ( 6 ) where each iteration adds Gaussian noise : q ( yn | yn−1 ) : = N ( yn ; √ ( 1− βn ) yn−1 , βnI ) , ( 7 ) under some ( fixed constant ) noise schedule β1 , . . . , βN . We emphasize the property observed by Ho et al . ( 2020 ) , the diffusion process can be computed for any step n in a closed form : yn = √ ᾱn y0 + √ ( 1− ᾱn ) ( 8 ) where ∼ N ( 0 , I ) , αn : = 1− βn and ᾱn : = ∏n s=1 αs . The gradient of this noise distribution is ∇yn log q ( yn | y0 ) = − √ 1− ᾱn . ( 9 ) Ho et al . ( 2020 ) proposed to train on pairs ( y0 , yn ) , and to reparameterize the neural network to model θ . This objective resembles denoising score matching as in Equation 3 ( Vincent , 2011 ) : En , [ Cn ∥∥ θ ( √ᾱn y0 +√1− ᾱn , x , n ) − ∥∥22 ] , ( 10 ) where Cn is a constant related to βn . In practice Ho et al . ( 2020 ) found it beneficial to drop the Cn term , resulting in a weighted variational lower bound of the log-likelihood . Additionally in Ho et al . ( 2020 ) , θ conditions on the discrete index n , as we will discuss further below . We also found that substituting the original L2 distance metric with L1 offers better training stability .

Summary: The work uses diffusion probabilistic models for conditional speech synthesis tasks, specifically to convert mel-spectrogram to the raw audio waveform. Results from the proposed approach match the state-of-the-art WaveRNN model. The paper is very well-written and it is quite easy to follow. The study of the total number of diffusion steps and two different ways (continuous and discrete) ways to feed it in the network is very interesting. It is quite relevant and important for speech synthesis tasks. Using this, authors are able to find a 6-step inference procedure that yields very competitive performance to WaveRNN while still being computationally feasible.