# CycleDiffusion: Voice Conversion Using Cycle-Consistent Diffusion Models
## 2 Related Works
### Diffusion Model-Based Voice Conversion  
A DM involves two processes: forward diffusion and reverse diffusion. The forward diffusion process progressively introduces Gaussian noise into the input data, whereas the reverse diffusion process attempts to eliminate this noise. In the context of voice conversion, these forward and reverse diffusions can be represented by the following stochastic differential equations (SDEs):  
\[\mathbf{dx}_{t}=\frac{1}{2}\beta_{t}(\mathbf{\bar{x}}-\mathbf{x}_{t})\mathbf{ d}t+\sqrt{\beta_{t}}\mathbf{dw}_{t},\text{forward diffusion} \tag{1}\]\[\mathrm{d}\widetilde{\mathbf{x}}_{t}=\left(\frac{1}{2}\left(\mathbf{\bar{x}}- \widetilde{\mathbf{x}}_{t}\right)-\
abla\mathrm{log}\,p\left(\widetilde{\mathbf{ x}}_{t}\right)\right)\beta_{t}\mathrm{d}t+\sqrt{\beta_{t}}\mathrm{d}\widetilde{ \mathbf{w}}_{t},\text{reverse diffusion} \tag{2}\]  
where \(\mathbf{x}_{t}\) is a forward-time stochastic process at time \(t\) with \(\mathbf{x}_{0}\) being an input speech feature vector, \(\mathbf{\bar{x}}\) is the linguistic information contained in \(\mathbf{x}_{0}\), \(\widetilde{\mathbf{x}}_{t}\) is a reverse-time stochastic process at time \(t\) with \(\widetilde{\mathbf{x}}_{0}\) being a reconstructed speech feature vector, \(\mathbf{w}\) and \(\widetilde{\mathbf{w}}\) are the forward- and reverse-time Wiener processes, respectively; \(\beta_{t}\) is a noise schedule, and \(t\in[0,T]\) is a continuous time variable (\(T\) is generally set to 1). When \(\widetilde{\mathbf{x}}_{T}\) is the linguistic information contained in \(\mathbf{x}_{0}\) without any speaker details, the reverse-time stochastic process, \(\widetilde{\mathbf{x}}_{t}\), gradually reconstructs the input speech feature vector, incorporating the input speaker characteristics. The reverse-time Wiener process, \(\widetilde{\mathbf{w}}\), mirrors the forward-time Wiener process, except that it operates with time flowing backward, from \(T\) to 0. A DM is trained to minimize the discrepancy between the trajectories of the forward and reverse diffusion processes, enforcing \(\widetilde{\mathbf{x}}_{0}\) similar to \(\mathbf{x}_{0}\), as an autoencoder. This can be accomplished as follows. First, note that the forward diffusion SDE, Equation (1), yields the following derivation:  
\[p(\mathbf{x}_{t}|\mathbf{x}_{0})=\mathcal{N}\Big{(}\mathbf{x}_{t};\alpha_{t} \mathbf{x}_{0}+(1-\alpha_{t})\mathbf{\bar{x}},\left(1-\alpha_{t}^{2}\right) \mathbf{I}\Big{)}, \tag{3}\]  
where \(\alpha_{t}\) is dependent on the noise schedule. Consequently, \(p(\mathbf{x}_{t}|\mathbf{x}_{0})\) evolves into \(\mathcal{N}(\mathbf{x}_{t};\mathbf{\bar{x}},\mathbf{I})\) for a noise schedule where \(\alpha_{t}\) approaches \(-\infty\), meaning that the final latent vector \(\mathbf{x}_{T}\) contains only linguistic information without any speaker information. Since \(p(\mathbf{x}_{t}|\mathbf{x}_{0})\) is a Gaussian distribution, the gradient of the log probability density with respect to the data (known as the score) can be easily computed. Then, \(\
abla\mathrm{log}\,p\left(\widetilde{\mathbf{x}}_{t}\right)\) in the reverse diffusion SDE, Equation (2), is replaced with an estimation obtained by minimizing the following loss function:  
\[\mathcal{L}_{\text{diffusion}}\Big{(}\mathbf{\bar{x}}_{0}^{\zeta},\zeta\Big{)} =\mathbb{E}_{t}\bigg{[}\left(1-\alpha_{t}^{2}\right)\mathbb{E}_{ \mathbf{x}_{0}^{\zeta}}\bigg{[}\mathbb{E}_{\mathbf{x}_{t}^{\zeta}|\mathbf{ \bar{x}}_{0}^{\zeta}}\bigg{[}\left\|\mathcal{S}_{\theta}\Big{(}\mathbf{x}_{t}^ {\zeta},\mathbf{\bar{x}},\zeta,t\Big{)}-\
abla\mathrm{log}\,p\left(\mathbf{x}_ {t}^{\zeta}|\mathbf{x}_{0}^{\zeta}\right)\left\|_{2}^{2}\right\|\right]\bigg{]} \bigg{]}, \tag{4}\]  
where \(\mathbf{x}_{0}^{\zeta}\) represents an input speech feature vector from speaker \(\zeta\), \(\mathcal{S}_{\theta}\) indicates the estimated score function implemented using a DNN with parameters \(\theta\), and \(\left\|\cdot\right\|_{2}\) denotes the \(\ell^{2}\) norm.