# Whispered Speech Conversion Based on the Inversion of Mel Frequency Cepstral Coefficient Features
## 2 Whispered Speech Conversion Process  
### GMM-Based Whispered Speech Conversion Model  
GMM-based whispered speech conversion reconstructs normal speech from whispered speech by estimating the acoustic parameters of normal speech. As shown in Figure 1, the method has two stages: the model establishment stage and the conversion stage. In the first stage, the MFCC parameters of each frame of normal speech and whispered speech were extracted, and then the reference normal speech and whispered speech were processed with dynamic time warping (DTW). The normal speech-whispered speech joint MFCC feature distribution model was then constructed using GMM. In the second stage, the MFCC features of whispered speech were converted to the MFCC features of normal speech using the established GMM from the first stage, and the normal speech was then synthesized using the MFCC features of normal speech. The probability density function of an M-order GMM was obtained by a weighted summing of M Gaussian probability density functions as follows:  
\[P(X/\lambda)=\sum_{i=1}^{M}w_{i}b_{i}(X) \tag{1}\]  
where \(X\) is a D-dimensional random vector; \(b_{i}(X_{t})\) is the subdistribution (\(i\) = 1,..., M); \(w_{i}\) is the mixed weight (\(i\) = 1,..., M). Each subdistribution is a D-dimensional joint Gaussian probability distribution, which can be expressed as follows:  
\[b_{i}(X)=\frac{1}{(2\pi)^{D/2}|\Sigma_{i}|^{1/2}}\exp\biggl{\{}-\frac{1}{2}(X -\mu_{i})^{t}\Sigma_{i}^{-1}(X-\mu_{i})\biggr{\}} \tag{2}\]  
where \(\mu_{i}\) is the mean vector, and \(\Sigma_{i}\) is the covariance matrix. Assuming that the source feature vector and the target feature vector conform to the joint Gaussian probability distribution, the GMM was used to model the mixed MFCC feature parameters as follows:  
\[p\biggl{(} \begin{array}{c}x\\ y\end{array} \biggr{)}=\sum_{m=1}^{M}a_{m}N\biggl{(}\biggl{(} \begin{array}{c} x\\ y\end{array} \biggr{)},\mu_{m},\Sigma_{m}\biggr{)} \tag{3}\]  
\[\mu_{m}=\biggl{(} \begin{array}{c}\mu_{m}^{x}\\ \mu_{m}^{y}\end{array} \biggr{)},\hskip 14.226378 pt\Sigma_{m}=\biggl{(} \begin{array}{ ccc}\Sigma_{m}^{xx}&\Sigma_{m}^{xy}\\ \Sigma_{m}^{yx}&\Sigma_{m}^{yy}\end{array} \biggr{)} \tag{4}\]  
Figure 1: Flow chart of whispered speech–normal speech conversion. where \(N\bigg{(}\bigg{(} \begin{array}{c}x\\ y\end{array} \bigg{)},\mu_{m},\Sigma_{m}\bigg{)}\) is the \(m\)th Gaussian component, where \(\mu_{m}\) and \(\Sigma_{m}\) are the mean and the covariance, respectively, of the \(m\)th Gaussian component of the GMM; \(\mu_{m}^{x},\mu_{m}^{y}\) are the source feature vector corresponding to the \(m\)th Gaussian component and the mean vector of the target feature vector, respectively; \(\Sigma_{m}^{xx},\Sigma_{m}^{yy}\) are the autocorrelation matrixes of the source target feature vector and the target feature vector, respectively, corresponding to the \(m\)th Gaussian component; \(\Sigma_{m}^{yy},\Sigma_{m}^{yx}\) are the covariance matrixes of the source target feature vector and the target feature vector, respectively, corresponding to the \(m\)th Gaussian component; \(a_{m}\) is the probability weight of the \(m\)th Gaussian component. The expectation-maximization (EM) algorithm was used to estimate the parameters of the Gaussian probability distribution. In each Gaussian component, a linear relation between the source feature vector and the target feature vector was developed as follows:  
\[\hat{y}_{t}=\sum\limits_{m=1}^{M}p(c_{m}|x_{t})(A_{m}x_{t}+B_{m}) \tag{5}\]  
where \(p(c_{m}|x_{t})\) is the posterior probability of the \(t\)th frame feature vector in the \(m\)th Gaussian component; \(A_{m}\) is the conversion matrix of the \(m\)th Gaussian distribution; \(B_{m}\) is the offset vector.