# Spectral Envelope Transformation in Singing Voice for Advanced Pitch Shifting
## 4 Complete Pitch Shifting Scheme
### Feature Extraction Stage  
The first step is the extraction of the \(f0\) and the spectral envelope, according to the model previously described. The \(f0\) is used not only to determine the frequency displacement, but also to guide the process of the localization of pitch marks.  
The well-known YIN algorithm [32] is used as the pitch estimator, to extract \(f0\) at the frame level. This algorithm has proven to be efficient for pitch extraction in different transcription systems [33; 34].  
With the \(f0\) extracted, it is possible to determine the position of the pitch marks. However, some problems arise related to mis-displacements of the pitch mark locations identified, which lower the quality of the transformation. This issue is addressed by using the mean-based signal (MBS) method, which has proven to be an efficient approach for PSOLA [35] in terms of removing discontinuities. The MBS is defined by the following expression:  
\[MBS(n)=\frac{1}{2N+1}\sum_{m=-N}^{N}w(m)x(n+m) \tag{16}\]  
where \(x(n)\) is the voice signal, \(w(m)\) is a Blackman windowing function of length \(2N+1\) samples and \(N\) is a value between 1.5- and two-times the average pitch period \(T_{0,mean}\). Then, the maxima in MBS in each window are used to locate the pitch marks.  
On the other hand, glottal closure instants (GCIs) are related to instants of high energy in the voice signal, and they are also a good approach to locate the pitch marks in continuous speech; as a matter of fact, some authors [36] carried out several objective measures to support the usage of CGI in PSOLA transformations. However, in our context, it is reasonable to think that the main focus in this evaluation should be the psychoacoustic perception, which is a subjective feature.  
Figure 6: Pitch shifting scheme with envelope transformation.  
Figure 7 shows the result of the utilization of both MBS and GCI approaches to locate pitch marks. GCI is a more precise method locating the higher energy instants, but causes abrupt shifts between consecutive marks (third and fourth blue marks in the figure). This problem is diminished by detecting the global peaks of the MBS in every interval. In this way, the difference between pitch marks is smoothed, preventing errors in the later re-synthesis using PSOLA. Recall that our final objective is to perform pitch shifting of sustained sung notes. Then, avoiding abrupt shifts between consecutive pitch marks is of main importance. This is a main reason why we use MBS instead of GCI to locate the pitch marks.  
Moreover, an additional post-processing stage aimed at correcting pitch mark location errors is included. The objective is to eliminate false alarms by using a thresholding operation. To this end, the minimum distance allowed between two consecutive pitch marks is defined as:  
\[d_{min}=\frac{F_{s}}{f0_{max}} \tag{17}\]  
where \(F_{s}\) is the sampling rate and \(f0_{max}\) is the upper limit of the YIN algorithm, which is 900 Hz in our case.  
If the distance between two pitch marks is smaller than the threshold, the closest one to the adjacent mark is eliminated. In case of similar distances to the adjacent marks, a new mark is located halfway between them, and both of them are removed.  
Finally, in this feature extraction stage, the spectral envelope \(H_{1}(z)\) of the original sound is estimated by making use of the model described in Section 2. Note that FDPS performs the spectral transformation by shifting the central frequency of the formants, so only the resonator coefficients are needed in order to perform the transformation, as will be described next.