# CycleDiffusion: Voice Conversion Using Cycle-Consistent Diffusion Models
## 2 Related Works
### CycleGAN-Based Voice Conversion  
The CycleGAN-based voice conversion technique employs two GANs; i.e., two sets of generator and discriminator pairs. One generator transforms the speech feature vectors from the source speaker to those of the target speaker, while the other does the reverse. During training, the adversarial loss from the discriminators ensures the correct conversion of speaker characteristics in the feature vectors, while the cycle consistency loss between the input to the first generator and the output of the second generator ensures that the linguistic content of the speech feature vectors remains unchanged.  
For instance, an input speech feature vector \(\mathbf{x}^{\zeta}\) (omitting the time index subscript hereafter for brevity) from speaker \(\zeta\) is transformed into \(\mathbf{x}^{\zeta\to\xi}\) using the first generator \(\mathcal{G}_{1}\):  
\[\mathbf{x}^{\zeta\to\xi}=\mathcal{G}_{1}\Big{(}\mathbf{x}^{\zeta}\Big{)}. \tag{7}\]  
The first discriminator \(\mathcal{D}_{1}\big{(}\mathbf{x}^{\zeta\to\xi}\big{)}\) provides the adversarial loss for \(\mathbf{x}^{\zeta\to\xi}\) so that the speaker characteristic in the feature vector changes from speaker \(\zeta\) to speaker \(\xi\). The converted feature vector \(\mathbf{x}^{\zeta\to\xi}\) is then transformed back to \(\mathbf{x}^{\zeta\to\xi\to\xi}\) by the second generator \(\mathcal{G}_{2}\):  
\[\mathbf{x}^{\zeta\to\xi\to\zeta\to\xi}=\mathcal{G}_{2}\Big{(}\mathbf{x}^{\zeta \to\xi}\Big{)}, \tag{8}\]  
which reverts the speaker characteristic in the feature vector from speaker \(\zeta\) back to speaker \(\zeta\). The second discriminator \(\mathcal{D}_{2}\big{(}\mathbf{x}^{\zeta\to\xi\to\xi}\big{)}\) provides the adversarial loss for \(\mathbf{x}^{\zeta\to\xi\to\xi}\). The following cycle consistency loss between the original input vector \(\mathbf{x}^{\zeta}\) and the reconstructed vector \(\mathbf{x}^{\zeta\to\xi\to\zeta}\) maintains the preservation of linguistic content:  
\[\mathcal{L}_{\text{cycle}}\Big{(}\mathbf{x}^{\zeta},\mathbf{x}^{\zeta\to\xi \to\xi}\Big{)}=\big{\|}\mathbf{x}^{\zeta}-\mathbf{x}^{\zeta\to\xi}-\mathbf{x}^ {\zeta}\big{\|}_{1}, \tag{9}\]  
where \(\left\|\cdot\right\|_{1}\) represents the \(\ell^{1}\) norm.  
Similarly, an input speech feature vector \(\mathbf{x}^{\zeta}\) from speaker \(\xi\) is converted to \(\mathbf{x}^{\zeta\to\xi}\) and then back again to \(\mathbf{x}^{\zeta\to\zeta\to\xi}\) by the generators \(\mathcal{G}_{2}\) and \(\mathcal{G}_{1}\), respectively, providing another cycle consistency loss. The discriminators \(\mathcal{D}_{2}\) and \(\mathcal{D}_{1}\) provide the adversarial losses for \(\mathbf{x}^{\xi\to\xi}\) and \(\mathbf{x}^{\xi\to\xi\to\xi}\), respectively. By using both adversarial loss and cycle consistency loss together, a CycleGAN can directly learn the conversion path without the need for parallel training data.  
The recently proposed MaskCycleGAN-VC [12], which is a variant of CycleGAN-based voice conversion, and wav2 wav [16], which utilizes cycle consistency for wave-to-wave voice conversion, have demonstrated high performance. However, GANs are prone to training instability and mode collapse, and developing many-to-many voice conversion models with CycleGANs entails substantial computational complexity.