## 2 Related Work
### GMM-Based VC  
In VCC 2018, two different baseline methods (i.e., a traditional VC system based on a GMM [2], and a vocoder-free VC system based on a differential Gaussian mixture model (DIFFGMM) [22]) were officially released, which were used by the participants to improve their work on the tasks. However, few submissions beat these competitive baselines. GMM is a statistical model for feature mapping. As the most popular baseline, [PERSON] et al. proposed a conversion method with a Gaussian mixture model (GMM) that accomplished feature mapping via soft clustering [1]. However, the performance of the conversion was still insufficient. The quality of the converted voice is deteriorated by some factors, e.g., spectral movement with inappropriate dynamic characteristics caused by the frame-by-frame conversion process and over-smoothing of converted spectra [23; 24]. To address the problems of the time-independent mapping and over-smoothing, [PERSON] et al. proposed a conversion method based on the maximum likelihood estimation of a spectral parameter trajectory [2]. This method considered the global variance (GV) of the converted spectra over a time sequence as a novel feature, which was able to capture the characteristics of the parameter trajectory. This idea effectively complemented the conventional frameworks of statistical conversion. The feature conversion part of a VC system can be denoted using  
\[y=f(x), \tag{1}\]  
where \(f(\cdot)\) is a mapping function. The vectors \(x\) and \(y\) are features from the source and target speakers, respectively. Given the sequential nature of speech, the features are sequences of frame-level vectors, i.e.,  
\[x=\left[x_{1}^{\mathrm{T}},x_{2}^{\mathrm{T}},\cdots,x_{i}^{\mathrm{T}},\cdots,x_{1}^{\mathrm{T}}\right]^{\mathrm{T}}, \tag{2}\]  
\[y=\left[y_{1}^{\mathrm{T}},y_{2}^{\mathrm{T}},\cdots,y_{i}^{\mathrm{T}},\cdots,y_{1}^{\mathrm{T}}\right]^{\mathrm{T}}, \tag{3}\]  
where T denotes the transposition of the vector and \(x_{i}\) and \(y_{i}\) are row vectors with features of the \(i\)th frame. In [PERSON]'s approach, \(2D\)-dimensional joint static and dynamic feature vectors, \(X_{i}=\left[x_{i^{\prime}},\Delta x_{i}\right]\) and \(Y_{i}=\left[y_{i^{\prime}},\Delta y_{i}\right]\) are composed of \(D\)-dimensional static feature vectors \(x_{i}\) and \(y_{i}\) and \(D\)-dimensional dynamic feature vectors \(\Delta x_{i}\) and \(\Delta y_{i}\), respectively. Their joint probability density function modeled using GMM is given as Equation (4),  
\[P(X_{i},Y_{i}|\lambda)=\sum_{m=1}^{M}\alpha_{m}N\left(\left[ \begin{array}{c} X_{i}\\ Y_{i}\end{array} \right];\left[ \begin{array}{c}\mu_{m}^{(X)}\\ \mu_{m}^{(Y)}\end{array} \right];\left[ \begin{array}{c}\sum\limits_{m}^{(X \,X)}\\ \sum\limits_{m}^{(Y\,X)}\end{array} \right]\left[ \begin{array}{c}(X\,X)\\ \sum\limits_{m}^{(Y\,Y)}\end{array} \right]\right), \tag{4}\]  
where \(N(\mu\Sigma)\) denotes the normal distribution with mean \(\mu\) and covariance matrix \(\Sigma\). \(m\) is the index of the Gaussian mixture. \(M\) is the total number of Gaussian mixtures. \(\lambda\) is a GMM parameter set consisting of the mixture weight \(\alpha_{m}\), the mean vector \(\mu_{m}\), and the covariance matrix \(\Sigma_{m}\) of the \(m\)th mixture. The GMM is trained using joint vectors of \(X_{i}\) and \(Y_{i}\) in the parallel data set, which have been automatically aligned to each other by dynamic time warping (DTW) [4]. The detailed steps can be found in References [2; 25]. DIFFGMM is a differential Gaussian mixture model. The DIFFGMM is analytically derived from the GMM (in Equation (4)) used in the conventional VC. Let \(D_{i}=\left(d_{i}^{T},\Delta d_{i}^{T}\right)^{T}\) denote the static and dynamic differential feature vector, where \(d_{i}=y_{i}-x_{i}\) and \(\Delta d_{i}=\Delta y_{i}-\Delta x_{i}\). The \(2D\)-dimensional joint static and dynamic feature vector between the source and the differential features is given as:  
\[\left[ \begin{array}{c}X_{i}\\ D_{i}\end{array} \right]=\left[