## 3 Perceptual Distance  
Although the MSE has previously been shown to be a reasonably successful choice both for modifying speaker individuality and obtaining transformed voice with high quality, it was not guaranteed that the MSE necessarily reflected the perceptual differences. The objective of this study is to incorporate a distance metric that sufficiently reflects the perceptual differences into the conversion rules. Our assumption is that the usage of a perceptually relevant distance metric ensures that the resulting converted speech sounds perceptually closer to target speech, and it is hoped, would outperform the conventional MSE-based methods. There are several ways to implement a perceptually relevant distance metric. The properties of the human auditory system was mostly exploited in this kind of distance metric. Hence, frequency-selective emphasis, non-uniform frequency sampling, and loudness transformation were adopted to measure the perceptual distance. In the present study, the structure of the distance metric used in the PESQ, which quantitatively measured the degree of perceptual degradation, was employed to measure the distances between the converted and target speech signals. Accordingly, the traditional MSE-based objective function was modified by incorporating both a symmetrical disturbance, \(D^{(s)}\), and an asymmetrical disturbance, \(D^{(a)}\)[25]  
\[D_{\text{PD}}=\frac{1}{N}\sum_{n}\left(\text{MSE}_{n}+\alpha D_{n}^{(s)}+\beta D _{n}^{(a)}\right) \tag{8}\]  
where \(\text{MSE}_{n}\) is the MSE of the \(n\)-th spectrum  
\[MSE_{n}=\frac{1}{M}\sum_{m=0}^{M-1}\frac{1}{\sigma_{m}^{2}}\Bigg{(}\log\frac{ |X_{t,n}(m)|^{2}}{|\hat{X}_{t,n}(m)|^{2}}\Bigg{)}^{2} \tag{9}\]  
Figure 2: Examples of the time-aligned waveforms. Top: Reference speech. Middle: Target speech. Bottom: Time-aligned target speech. where \(|X_{t,n}(m)|^{2}\) and \(|\hat{X}_{t,n}(m)|^{2}\) are, respectively, the target and converted power spectra obtained by multiplying the spectral envelope derived from the LPCC, \(\mathbf{h}_{t,n}(m)\), and the spectral fine structure, \(\mathbf{e}_{t,n}(m)\), while \(\sigma_{m}\) is the standard deviation of \(|X_{t,n}(m)|^{2}\). Indices \(n\) and \(m\) denote, respectively, frame and frequency, while \(M\) is the number of frequency bins. The number of frequency bins was chosen according to the frame length and the sampling frequency, that was 256. In (8), \(\alpha\) and \(\beta\) are weighting factors for each disturbance. The symmetrical disturbance reflects the absolute difference between the converted and target loudness spectra when auditory masking effects are account for. When the symmetrical disturbance is applied to VC, it can be regarded as a distance function between the reference and target speakers measured in a domain that reflects human auditory system. There are two types of difference patterns in VC, one where the target value is greater than the reference value and vice-versa. Such difference patterns cannot be reflected on the distance metric such as MSE and the symmetrical disturbance. Whereas the signs of the loudness differences are considered in the asymmetrical disturbance. The negative differences (loss of target spectral component) and positive differences (residuals of reference spectral component) are differently perceived owing to masking effects. By using the asymmetrical disturbance, the differences between the two speakers can be described in more detail, which can lead to the improvement of the VC performance. The calculation of symmetrical and asymmetrical disturbances reflects the human auditory system and is composed of several steps, briefly described as follows [25,30]:  
1.