## 1 Introduction  
Voice conversion aims to convert the speaker-specific characteristics of input speech into the target speaker's characteristics while preserving the linguistic content of the input speech. The application areas of voice conversion include entertainment [1], education [2], and medicine [3] domains. With advances in deep learning, many studies on voice conversion using deep neural network (DNN) models have been attempted. In particular, generative models, such as variational autoencoders (VAEs) [4] and generative adversarial networks (GANs) [5], have shown promising results in many voice conversion tasks [6, 7, 8, 9, 10, 11]. Recently proposed flow-based voice conversion [12] and diffusion model-based voice conversion [13] also showed good performance. Most of these works on voice conversion generally consist of three steps: analysis, mapping, and reconstruction [14]. In the analysis step, feature vectors are extracted, which are easy to process while retaining relevant information from the input waveform. The mapping step is the actual voice conversion process that changes only the identity of the speaker while preserving the linguistic content of the input feature vectors. Finally, the reconstruction step uses a vocoder to synthesize a waveform using the converted feature vectors from the previous step. Figure 1 illustrates this process. The analysis steps of the conventional approaches use fixed feature extraction methods, such as Fourier transform and mel-scale filtering. Traditionally, these feature vectors have been widely adopted in various speech-processing areas, such as speech recognition, speaker identification, and text-to-speech. However, it could be further optimized [15, 16] for voice conversion tasks, as the importance of each frequency can vary across conversion pairs. Additionally, most of the existing approaches focus on improving the feature converter model itself, as it performs the actual voice conversion task and provides its output to a pre-trained vocoder. This approach may lead to poor performance because the vocoder model is not explicitly trained to process the output feature vectors from the feature converter model. Therefore, some unnatural speech characteristics that arise from the feature converter model are not sufficiently compensated for by the vocoder model, resulting in poor speech quality. Recent studies [17, 18, 19, 20, 21, 22] that proposed to jointly train feature converter models and vocoder models showed promising results. In this study, we extend the work in [22] and propose an architecture that integrates all three stages of analysis-mapping-reconstruction to optimize the quality of converted speech. Table 1 summarizes the existing voice conversion methods. The proposed method enables wave-to-wave (wav2 wav) voice conversion by jointly training the entire analysis-mapping-reconstruction steps in an end-to-end manner. This allows the model to flexibly adjust model parameters to achieve high-quality speech output. However, training from scratch consumes too much training time and data, as it is difficult to find optimal feature representations. Furthermore, training the GAN-based models used in the mapping and reconstruction steps is known to be inherently difficult. Our proposed method overcomes these difficulties in two ways. The first is that the analysis step uses two-layer convolutional networks (CNNs) initialized with a discrete Fourier transform (DFT) matrix and mel-filterbank coefficients. This allows the analysis step to