Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme

Abstract: Voice conversion is a common speech synthesis task which can be solved in different ways depending on a particular real-world scenario. The most challenging one often referred to as one-shot many-to-many voice conversion consists in copying target voice from only one reference utterance in the most general case when both source and target speakers do not belong to the training dataset. We present a scalable high-quality solution based on diffusion probabilistic modeling and demonstrate its superior quality compared to state-of-the-art one-shot voice conversion approaches. Moreover, focusing on real-time applications, we investigate general principles which can make diffusion models faster while keeping synthesis quality at a high level. As a result, we develop a novel Stochastic Differential Equations solver suitable for various diffusion model types and generative tasks as shown through empirical studies and justify it by theoretical analysis.

Reviews and author responses:

This paper proposes a method to perform voice conversion through recent methods in Diffusion Based Probability Modeling (DPM) through Scochastic Differential Equations (SDE), building upon recent work Grad-TTS and Glow-TTS. The architecture is an encoder-decoder setup, trained separately, described in upcoming paragraphs. Viewed at a high level, the encoder maps input to 'noise', and the decoder inverts noise to output, in keeping with the SDE formalism. To transform average features to output voice, speaker info is conditioned on the average voice features before feeding to decoder. 

The encoder learns to map input voice features to an average mel spectrogram. In order to do this, the Montreal Forced Aligner is used to align mel frames to phonemes. The average output voice (for encoder training) is obtained by aligning and averaging out corresponding phonemes in the dataset. 

The main novelty in this paper is claimed to be in the decoder setup solving the reverse SDE. The authors derive a modified SDE that does MLE inference on the original reverse SDE. Through this approach, the claims are that the likelihoods are better, and its estimation is 'faster' (ostensibly, owing to getting around step size limitations). 

Evaluations are carried out for VCTK and LibriTTS, and quite well done, and the samples provided show that the approach works. Comparisons are made against several other approaches, include subjective evaluations and FID scores, and look for similarity and naturalness metrics. They also compare against different flavors of SDE setups (Variance Exploding, Preserving, etc.).


 Post rebuttal
-----------------
My initial review was largely positive, but for a few issues with the motivation 
- what does the MLE solver give over the usual Euler-Mayurama integration setup?
- some checks on the formulation and intuitions behind proof.

The authors have sufficiently answered my concerns:
- The MLE solver gives an optimized trajectory for data, even though the overall scheme is still first order, as shown through a simple example. The authors argue that while second (or higher) order schemes are very much possible, they would need more function evaluations and therefore not necessarily faster (this point could use another demo). 
- The formulation as far as I can see is correct, as shown in the author rebuttal. Intuitions behind proof are also summarized.
- I think a lot more work could be done in exploring diffusion probability models on the lines of this paper, in regards to numerical integrators and formulation setup.

I am raising my score.

Initial Review 
-----------------
I reviewed this paper with great interest. It is rather remarkable how much progress has been made in the last few years in speech modeling. It is also interesting that the newer models (and this is one of them) compute the input-output alignment differently from the attention mechanism used in works such as Tacotron. Likewise, it is inspiring that generative modeling research (e.g. flow based, and DPMs) finds application in speech models quite rapidly, and with telling practical use (e.g. WaveGlow vocoder).

Strengths 
-------------
+ Builds upon current work in DPMs (Grad-TTS, Wave-Grad) to come up with a setup for voice conversion. 
+ Uses a sensible scheme involving computing an 'average voice' into which speaker information is added and output voice is decoded. 
+ Paper contains a good review of existing work, and it is possible to piece together developments in score based modeling and adaptations in speech. 
+ Maximum Likelihood solver formulation for reverse SDE is well motivated and gives good results. It produces a first order scheme - not very different from Euler-Mayurama - with three additional hyperparameters. Supposedly, this results in an improvement in sample quality through optimized likelihood.

Weaknesses
-----------------
- I found the mathematical derivations not very clearly explained. The development is generally easy enough to follow if we work out the slightly tedious derivations. However, I think it would benefit from a few lines of 'intuition'. What is the intuition behind the derivations, why does it work and in what types of SDEs does such a procedure work? 
- There seems to be an inexactitude (or mistake) in the derivations (equation 40). I think we get $ k^*_{t,h} = 1+ \hat{k}_{t,h} $ (i.e. the original derivation needs an extra 1 on the right hand side).

- The motivations behind the MLE derivations aren't very clear (at least to me). Is it that since we maximize likelihood, the setup is more optimized and therefore produces better samples, or are we talking about being able to take 'larger' step sizes (smaller N)? 

- As such, it seems to me that the scheme is O(h), and should not work well when timesteps are large. It would be nice to see some explanation or analysis of why the setup converges in such a small number of iterations. I would expect a second order scheme to converge faster. Likewise, a predictor-corrector scheme (as in the stem SDE works by Yang Song) improves performance. 


 On the whole, the work looks plausible for voice conversion, even if it only used the base Euler-Mayurama scheme. The derived scheme produces better samples, or does so allowing use of much larger timesteps (~ N = 6). Nevertheless, I am not totally convinced because
1) It is not explained why the optimization 'works' in an intuitive sense. I assume that it provides a path with better likelihood since it has been optimized
2) The derivations ought to be gone over very carefully, and it is crucial that the details are correct. I would like the other reviewers to take a careful look at the derivations, especially so as I feel that there is a mistake in equation (40), which could carry over to the other steps as well. 

I rule this work as a marginal reject, but can change my decision after discussion and clarification. 

 The responses from the authors provided detailed clarifications about the work.  I keep my recommendation to accept the paper.

 We would like to thank the reviewer for the comments and address the concerns in the following.

1. One of the main concerns is unclear motivation behind the proposed Maximum Likelihood (ML) SDE solver and the reasons why it should be better than general-purpose Euler-Maruyama (EM) one. As established in the Theorem 1, the ML solver optimizes log-likelihood of sample paths $X_{kh}$ under generative scheme $\hat{X}$ given by the formula (11). It means that the sequences $X_{kh}$ of real data points corrupted with Gaussian noise according to the forward SDE (6-F) are more likely to come as solutions of the reverse SDE (6-R) obtained with the ML SDE solver than with any other solver belonging to the class described in the Theorem 1 (we'd like to note it is only this class that is parameterized with three parameters $\kappa$, $\omega$ and $\sigma$ while the ML solver itself does not have hyperparameters other than step size).  
Thus, for any step size $h$:  
a) "real" sequences $X_{kh}$ are best described by the ML SDE solver $\hat{X}$;   
b) as a consequence, score-matching network $s_{\theta^{*}}$ is more likely to point to the correct directions (of increasing of the logarithm of noisy data density) because its inputs $\hat{X_{kh}}$ are "closer'" to the ones seen at training $X_{kh}$.   
Combined together, a) and b) suggest that the ML SDE solver relies on better approximations of likely data trajectories which can reduce the negative effect caused by SDE discretization and lead to samples of higher quality compared to other SDE solvers from the class.   
Although in principle it holds for every step size $h$, for small step sizes the ML SDE solver does not differ much from standard EM SDE solver as derived in Appendix B and discussed after the Theorem 1, so the benefits of the proposed ML solver in terms of higher quality are more clear when the step size is large. We demonstrate this feature of our ML solver in Table 3: only the ML solver allows us to use as few as 6 reverse diffusion steps while others seem to suffer from large discretization errors resulting in significant MOS degradation. Also, the fact that we can use larger step sizes with almost no loss of quality is illustrated by speech samples on our web demo page.

2. The middle part in eq.(40) implies that $\hat{\kappa_{t,h}}$ = $\frac{(1 - \gamma_{0,t}^{2}) \nu_{t-h,t}}{\gamma_{0,t}\beta_{t}h} - 1$ which equals right-hand side of the definition of $\kappa_{t,h}^{\star}$ in eq.(10). Thus, the equality $\hat{\kappa_{t,h}} = \kappa_{t,h}^{\star}$ in the right-hand side of eq.(40) is correct.

3. Our proposed ML solver is indeed a first-order scheme. Employing second-order sampling schemes could lead to acceleration in terms of the number of solver steps, but each step would be more computationally intensive, i.e. the number of function evaluations (the slowest part in a diffusion model) would be bigger. Moreover, the fact that we deal with SDEs rather than ODEs makes things even worse: estimating first-order differential of the drift term $s_{\theta}(x, t)$ in the reverse SDE would require second order derivatives of $s_{\theta}$ in $x$ as follows from Ito's formula (see second-order schemes derivation in [1]). Considering this, we decided to limit ourselves to solvers requiring only one function evaluation per timestep, so predictor-corrector and second-order schemes are out of the scope of our paper because of their potential computational inefficiency.

4. In contrast with unconditional models, it is typically possible to tune diffusion models with strong conditioning to generate high-quality samples with a few reverse diffusion steps (see [2], [3]) despite using a first-order sampling scheme rather than more sophisticated techniques. We believe it is because strong conditioning (as in our case) makes score-matching task easier and training results in score-matching networks of better quality, which means fewer reverse diffusion steps for a good output quality. E.g. the limiting case (ii) from the Theorem 1 shows that for simple data distributions and ideally-trained score-matching networks our ML solver provides exact data reconstruction even when one reverse diffusion step is used.

[1] Glasserman, P. (2004) Monte Carlo Methods in Financial Engineering, Chapter 6.  
[2] Chen, N. et al. (2021) WaveGrad: Estimating Gradients for Waveform Generation  
[3] Popov, V. et al. (2021)  Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech

 5. The proof of the Theorem 1 is rather technical but here is a short summary of the main steps. The key Lemma 1 helps us to express 
$s_{\theta}(x,t)$ as a linear combination of $x$ and conditional data expectation $E[X_{0}|X_{t}=x]$. It allows us to write down an explicit 
expressions for $Law(\hat{X_{t-h}}|\hat{X_{t}})$ in terms of $\hat{X_{t}}$ and $E[X_{0}|\hat{X_{t}}]$ (36) and the corresponding transition probability $\hat{p_{t-h|t}}(x_{t-h}|x_{t})$ (39). From the general properties of Gaussian distributions we derive similar expressions for $Law(X_{t-h}|X_{t},X_{0})$ in terms of $X_{t}$ and $X_{0}$ (42) and the corresponding transition probability $p_{t-h|t}(x_{t-h}|x_{t})$ (45). We then minimize cross-entropy between distributions with transition densities $\hat{p_{t-h|t}}(x_{t-h}|x_{t})$ and $p_{t-h|t}(x_{t-h}|x_{t})$ (46) subject to parameters $\hat{\mu}$, $\hat{\nu}$ and $\hat{\sigma}$ of (39). Finding optimal $\hat{\mu}$, $\hat{\nu}$ and $\hat{\sigma}$ lead us to optimal values in formula (10).

 We would like to thank the reviewer for the comments and for high appreciation of our work!

 We would like to thank the reviewer for the comments and for high appreciation of our work! We addressed small remarks in the revision.

The paper proposes a diffusion probabilistic model-based voice conversion method for one-shot voice conversion scenario.  The proposed method can generate high-quality converted speech compared to state-of-the-art approaches.  Furthermore, to improve the real-time factor, a novel stochastic differential equations solver is proposed which makes the diffusion model faster.  The proposed solver is also suitable for other generative tasks.
 Strengths:
(1) A diffusion probabilistic model-based voice conversion method has been proposed for the one-shot voice conversion scenario.  
(2) The proposed method can generate high-quality converted speech compared to state-of-the-art approaches. 
(3) To solve the disentanglement problem, a new approach in which the encoder predicts the "average voice" is proposed.  This idea is interesting and novel.
(4) For the decoder part, the diffusion-based method is adopted which reconstructs the converted voice from the "average voice".
(5) To solve the problem of slow inference of the diffusion probabilistic model, a novel inference scheme is developed which significantly reduces the number of iterations.
(6) A novel stochastic differential equations solver, named maximum likelihood SDE solver, is proposed with theoretical analysis and empirical studies.
(7) A lot of experiments have been conducted to validate the effectiveness of the proposed method.
(8) Besides the experiments on VC, the maximum likelihood sampling scheme is also validated by the CIFAR-10 image generation task.
 Based on the above main review (especially the strengths of the paper), the proposed method is novel enough and the experiments can well support the effectiveness of the proposed method.  think the paper could be accepted for publication on ICLR.

This paper tackles the problem of one-shot many-to-many voice conversion (with both unseen source and target speakers) using a diffusion model on mel-spectrograms. The authors propose a dedicated architecture able to generalize to unseen speakers naturally (without relying on Phoetic Posteriorgrams (PPGs) as in previous works) by conditioning the diffusion model on "average-phoneme" spectrograms together with the target speech. They also address the important problem of the errors arising from using the Euler-Maruyama scheme with large discretization steps by proposing a novel SDE solver, allowing to perform voice conversion with as few as 6 diffusion steps. They demonstrate the applicability of this solver on other modalities (CIFAR-10) using on unconditional model. Extensive and rigorous experiments are conducted on the voice conversion task and an accompanying web page is provided with numerous and convincing voice conversions. This paper is well-written and remarkable on many aspects:
- A novel SDE solver than can be of interest for anyone interested in Diffusion models is proposed. This solver is provably better than Euler-Maruyama discretization in the setting of a small number of diffusion steps,
- The idea to condition the diffusion model on the "average voice" is novel and fully exploited by the proposed architecture,
- There are extensive experiments that demonstrate the relevance of this approach,
- A website showcasing many voice conversions is available and the examples are convincing,
- The relevant literature is properly addressed,
- Code will be available.

Small remarks:
- p. 2: Chen et al. 2021a also reported results (for vocoding) using only as few as 6 diffusion steps.
- "most of them lack theoretical grounds" may be a bit harsh.



 A paper of high quality, featuring substantial contributions that can be valuable to anyone interested in diffusion models. Code and models are available and may have practical usages.

Meta-review: The paper is exceptionally well summarized by Reviewer QC5G which is difficult to improve up on. I will save the readers the effort of reading more text (without adding more substance). The reviewers unanimously rated this paper highly. The discussion has been robust,  enlightening and also has improved the revised paper.

Decision: Accept (Oral)