## 4 Estimation of the Conversion Parameters  
The overall procedure of constructing the conversion parameters is explained in Figure 3. Basically, the converted and target speech signals are represented in the frequency domain, since the perceptual distance is computed using the power spectra. The converted spectra were given by multiplying the spectral envelope derived from the LPCC and the pitch-scaled spectral fine structure. A supervised learning framework was adopted where DNN [31] are used to estimate the conversion rules for the LPCC, \(\mathcal{F}_{H}\). The power spectra necessary for calculation of the perceptual distance is obtained by  
\[\mathbf{x}_{t,n}=(\mathbf{G}\cdot\mathbf{h}_{t,n})\odot\mathbf{e}_{t,n},\ \mathbf{\hat{x}}_{t,n}=(\mathbf{G}\cdot\mathbf{\hat{h}}_{t,n})\odot\mathbf{ \hat{e}}_{t,n} \tag{15}\]  
where \(\mathbf{G}\) is the transformation matrix that transforms the LPCC vector into the power spectrum. The elements of the matrix \(\mathbf{G}\) are given by  
\[g_{ij}=\cos\Big{(}\pi\frac{i(j+1)}{M-1}\Big{)},\ \ 0\leq i\leq M-1,\ \ 0\leq j \leq N_{L} \tag{16}\]  
where \(N_{L}\) is the order of the LPCC. The updated estimate of the DNN weights \(\mathbf{W}\) with a learning rate \(\lambda_{W}\) is computed iteratively as follows:  
\[\mathbf{W}_{n+1}=\mathbf{W}_{n}-\lambda_{W}\
abla_{\mathbf{W}}D_{PD}(\mathbf{ X}_{t},\mathbf{\hat{X}}_{t}) \tag{17}\]  
The conversion rule for the spectral fine structure, \(\mathcal{F}_{E}\), was achieved by pitch modification wherein the time-domain pitch-synchronized overlap and addition (TD-PSOLA) [6] method was performed on the LP-residual of the reference speech. Note that pitch modification was adapted only to the voiced regions, and hence, the pitch locations of the unvoiced regions were not changed. Since TD-PSOLA was implemented in the time domain, the modified reference spectrum was obtained by discrete Fourier transform (DFT) of the pitch-scaled LP-residual signal. Estimation of the conversion rule \(\mathcal{F}_{E}\) is then formulated as finding the optimal pitch modification factor that minimizes the overall perceptual distortion with the given converted LPCCs, as shown in (4). Although there was no explicit relationship between \(D_{PD}\) and the pitch modification factor, the convexity of the perceptual distance function over the pitch modification factor was clearly observed for all conversion pairs, as shown in Figure 4. Accordingly, the gradient descent algorithm was employed to find the pitch modification factor as follows:  
\[\varphi_{n+1}=\varphi_{n}-\lambda_{\varphi}\
abla_{\varphi}D_{PD}(\mathbf{X}_ {t},\mathbf{\hat{X}}_{t}) \tag{18}\]  
where \(\varphi_{n}\) is the pitch modification factor that is estimated at the \(n\)-th iteration. A learning rate \(\lambda_{\varphi}\) was heuristically determined, that was 0.01. Note that the derivative term \(\
abla_{\varphi}D_{PD}\) cannot be computed mathematically. The mean value theorem was employed to approximate \(\
abla_{\varphi}D_{PD}\) as follows;  
\[\
abla_{\varphi}D_{PD}(\mathbf{X}_{t},\mathbf{\hat{X}}_{t})\approx\frac{D_{PD} (\mathbf{X}_{t},\mathbf{\hat{X}}_{t}|\varphi_{n})-D_{PD}(\mathbf{X}_{t},\mathbf {\hat{X}}_{t}|\varphi_{n-1})}{\varphi_{n}-\varphi_{n-1}} \tag{19}\]  
where \(D_{PD}(\mathbf{X}_{t},\mathbf{\hat{X}}_{t}|\varphi)\) is the perceptual distance in case when the pitch modification factor is given by \(\varphi\).