## 3 Proposed Method  
As discussed in the previous section, traditional voice conversion approaches train the spectral feature conversion models and vocoder models separately and use a fixed feature extraction scheme, leading to the degraded quality of converted speech. The proposed method, depicted in Figure 3, adopts a wave-to-wave approach, which uses waveforms directly as the input and output of the system and jointly trains the analysis, mapping, and reconstruction modules in an end-to-end manner, enabling more flexible feature analysis and natural speech generation. However, training the feature extraction module from scratch in this end-to-end learning may require too much data and training time. Since the mel-spectral features are widely used in various speech-processing tasks, we decided to improve their performance further by learning detailed parameters suitable for voice conversion based on them. We first reproduced the mel-spectral feature extraction process with a two-layer CNN initialized with a DFT matrix and mel-filterbank coefficients; then, each weight was adjusted during training. Figure 4 shows the feature extraction network. The output of the first convolution layer, which initially consisted of real and imaginary components extracted using conventional feature extraction methods, was converted to magnitude vectors via square, summation, and square root operations. The resulting magnitude vectors were fed into the second convolutional layer, followed by rectified linear unit (ReLU) and logarithm operations. The CNN parameters were shared between the two generators and updated during training. Figure 3: Overview of the proposed voice conversion model. Greyed objects contain trainable parameters. The generators include both a feature extractor and vocoder, which can be jointly trained in an end-to-end manner. By including the feature-extracting CNN and vocoder into a generator, the proposed voice conversion model enables waveform inputs and waveform outputs, resulting in wave-to-wave voice conversion, shown as follows:  
\[w^{A\to B}=G_{A\to B}\Big{(}w^{A}\Big{)}\, \tag{3}\]  
where \(G_{A\to B}\) is the generator model that converts the speaker identity of the input speech from speaker \(A\) to speaker \(B\), and \(w^{A\to B}\) is the converted waveform from speaker \(A\)'s waveform \(w^{A}\) to match speaker \(B\)'s voice characteristics. The loss function of the proposed method consists of adversarial loss [5], cycle consistency loss [35], identity-mapping loss [36], and feature-matching loss [34], similar to other CycleGAN-based methods, as shown below:  
\[\mathcal{L}_{\text{adv1}}(G_{A\to B},D_{B})=\mathbb{E}_{w^{B}}\Big{[}\log D_{ B}\Big{(}F\Big{(}w^{B}\Big{)}\Big{)}\Big{]}+\mathbb{E}_{w^{A}}\Big{[}\log \Big{(}1-D_{B}\Big{(}F\Big{(}G_{A\to B}\Big{(}w^{A}\Big{)}\Big{)}\Big{)} \Big{)}\Big{]}\, \tag{4}\]  
\[\mathcal{L}_{\text{adv2}}(G_{A\to B},D_{B})=\mathbb{E}_{w^{B}}\Big{[}\log D_{ B}\Big{(}F\Big{(}w^{B}\Big{)}\Big{)}\Big{]}+\mathbb{E}_{w^{B\to A}}\Big{[}\log \Big{(}1-D_{B}\Big{(}F\Big{(}G_{A\to B}\Big{(}w^{B\to A}\Big{)}\Big{)} \Big{)}\Big{)}\Big{]}\, \tag{5}\]  
\[\mathcal{L}_{\text{cyc}}(G_{A\to B})=\mathbb{E}_{w^{B}}\Big{[}\Big{\|}F \Big{(}G_{A\to B}\Big{(}G_{B\to A}\Big{(}w^{B}\Big{)}\Big{)}\Big{)}-F\Big{(}w ^{B}\Big{)}\Big{\|}_{1}\Big{]}\, \tag{6}\]  
\[\mathcal{L}_{\text{id}}(G_{A\to B})=\mathbb{E}_{w^{B}}\Big{[}\Big{\|}F \Big{(}G_{A\to B}\Big{(}w^{B}\Big{)}\Big{)}-F\Big{(}w^{B}\Big{)}\Big{\|}_{1} \Big{]}\, \tag{7}\]  
\[\mathcal{L}_{\text{fm}}(G_{A\to B})=\mathbb{E}_{w^{B}}\Bigg{[}\sum_{i}\Big{\|} D_{B}^{i}\Big{(}F\Big{(}G_{A\to B}\Big{(}G_{B\to A}\Big{(}w^{B}\Big{)} \Big{)}\Big{)}\Big{)}-D_{B}^{i}\Big{(}F\Big{(}w^{B}\Big{)}\Big{)}\Big{\|}_{1} \Bigg{]}\, \tag{8}\]  
where \(\mathbb{E}\) stands for the expectation operation, \(G_{B\to A}\) is the generator model that converts the speaker identity of the input speech from speaker \(B\) to speaker \(A\), and \(D_{B}^{i}\) represents the \(i\)-th layer of the discriminator.