## 5 Experiments and Results  
### Experiment Setup  
The evaluation was carried out using the CMU ARCTIC database [42] for US English and was sampled at 16 kHz. These databases were constructed as phonetically balanced, and originally designed for unit selection speech synthesis research. These databases consist of around 1150 utterances and includes US English male and female speakers as well as other accepted speakers. Among them, two male speakers, bdl and rms, and two female speakers, clb and slt were used. Four different voice conversion tasks were investigated including male-to-male (rms \(\rightarrow\) bdl), male-to-female (bdl \(\rightarrow\) slt), female-to-female (slt \(\rightarrow\) clb) and female-to-male (clb \(\rightarrow\) rms) conversion. To obtain the conversion rules, 200 utterances were used, and the remaining 100 utterances were prepared for evaluation. The order of the LPCC was 20. The speech data were analyzed pitch-synchronously, at the manually labelled pitch marks. For voiced regions, the frame length was set to two or three pitch periods depending on the pitch modification factor [6], whereas the frame length was set to be constant (=25 msec) for unvoiced regions. A pre-emphasis factor 0.95 was applied. Since it is impossible to mathematically determine the optimum number of hidden layers and optimum number of hidden nodes, we performed several experiments to investigate the relationship between the number of hidden layers and objective performance in terms of overall perceptual  
Figure 4: Examples of perceptual distances according to the pitch modification factors. **Left**: male-to-male. **Right**: male-to-female conversions. Figure 3: Block diagram of constructing of the conversion parameters. distances. No clear relationship between them was found. According to the experimental results, the best performance was achieved in case when the DNN had three hidden layers and the number of the nodes in the hidden layer was set to 100. To prevent the network from converging to poor local minima, a deep generative model of input features was adopted to initialize the network by a stacking of multiple restricted Boltzmann machines (RBMs) [43, 44]. The number of RBM pre-training epoches in each layer was 20. The learning rate of the RBM training was set as 0.0005. A fixed learning rate of 0.001 was applied for the fine-tuning of the baseline. The total number of epoches at the fine-tuning stage was 50. For both RBM pre-training and fine-tuning, The momentum was set to 0.05 for the first five epochs, then maintained at 0.07 thereafter. Mean and variance normalization was applied to the input and target feature vectors of the DNN. The performance of the DNN was expected to be improved by dropout regularization [45]. Hence, dropout regularization with a keep probability of 0.8 was employed. For comparison, the performance of the four conventional VC methods including the minimum mean square error (MMSE)-based joint Gaussian method (JGMM) [20], the maximum likelihood trajectory conversion method (JDGMM) [21], dynamic frequency warping with amplitude scaling (DFW) [22], and DNN-based conversion with independent pitch scaling (MLP-ind) were also evaluated. For all these methods, pitch modification with a fixed scale factor was employed to convert the spectral fine structures. For each conversion, the pitch modification factors were determined so that the statistical properties (mean and standard variation) of the converted pitch periods were matched with those of the target pitch periods [24]. Three measurements, Mel-cepstral distance (MCD), perceptual distance (8), and PESQ, were employed to evaluate the performance of each method objectively. Note that all three measurements are relevant to distortions perceived by the human auditory system. PD and PESQ, however, were newly adopted in this study. The listening tests were conducted to subjectively evaluate the validity of the proposed VC method.