# Spectral Envelope Transformation in Singing Voice for Advanced Pitch Shifting
## 4 Complete Pitch Shifting Scheme
### Pitch Shifting Stage  
This stage performs the actual pitch shifting process with spectral envelope transformation; the pitch of an audio waveform will be increased or decreased by using the PSOLA technique, and also, the spectral envelope will be modified by means of formant shifting and filtering. Figure 7: Pitch marking using two different methods. MBS achieves smaller differences between consecutive pitch marks than the glottal closure instant (GCI). The pitch-shifting factor, \(\beta\) (shown in Figure 6), is used to define the strength of the shift of the pitch. This parameter can be larger than one (transposing up) or smaller than one (transposing down), and it is related to the number of semitones shifted through the following expression:  
\[\beta=\left(2^{\frac{1}{2}}\right)^{N} \tag{18}\]  
The PSOLA [9] algorithm is composed of two steps: in the first step, the input waveform is conveniently segmented, and in the second one, a time-stretched version of the original signal is synthesized by overlapping and adding the time segments extracted before [37]. The complete process is detailed below:  
* Once the \(f0\) and the pitch marks are determined, the algorithm extracts a signal segment at every pitch mark \(t_{i}\) by using a Hanning window of length \(L=2\cdot P(t_{i})\), with \(P(t_{i})\) the pitch period at the current pitch mark location. * Overlap and add the selected segment at the specified instant \(t_{k+1}\). \(t_{k+1}\) is used to center the next synthesis segment, which is obtained according to: \[t_{k+1}=t_{k}+\tilde{P}(t_{k})=t_{k}+\frac{P(t_{k})}{\beta}\] (19) where \(\tilde{P}(t_{k})\) is the new estimated period of the signal and \(\beta\) is the pitch shifting factor or skip, a multiple of the number of half steps \(N\) expressed as shown in Equation (18) [37]. At this point, a pitch-shifted version of the original signal has been generated by a formant preservation technique. However, the spectral envelope changes as pitch varies [10], so an additional step is performed in order to modify the spectral envelope of the synthesized signal to achieve a more natural result. The first three formants (\(F1\), \(F2\) and \(F3\)) are modified according to the expressions (13)-(15), resulting in a new spectral envelope \(H_{2}(z)\). The resulting resonator coefficients are stored and combined with the original ones, generating the filter response in charge of modeling the shifting of the formants. Finally, the pitch-shifted signal is inverse-filtered by \(H_{1}(z)\) and filtered by \(H_{2}(z)\). Figure 8 shows an example of the resulting filter response of the \(H_{2}/H_{1}\) filter for a \(F3\) note (172.6 Hz) to be pitch-shifted a fifth above (C3 at 261.63 Hz), sung by a male amateur singer. The filter corresponds to shifting the formants of the harmonic component in the range of [0,5000] Hz, with a sampling rate of 44,100 Hz. As can be observed, the first formant is the most strongly affected in terms of correction (note the valley-peak sequence with a difference of almost 6 dB), whereas Formants 2 and 3 are just slightly modified. Figure 8: Magnitude of the \(\frac{H_{2}}{H_{1}}\) filter response of the harmonic component of an \(F3\) note (172.6 Hz) of a male singer to be pitch shifted a fifth above. Due to our filtering stage and the way in which the envelopes are modeled, the low frequency band ([0,550] Hz approximately) is attenuated in the case of shifting to a higher pitch, so a compensation filter has been added. In Figure 9, the magnitude response of this filter when shifting to a higher pitch is shown, making up for the attenuation of the previous \(H_{2}/H_{1}\) filter. This compensation stage is a pre-defined second-order low shelf filter with a gain of 3 dB and a cut-off frequency at 500 Hz. In the case of shifting to a lower pitch, the result is exactly the inverse: the low frequency band needs to be attenuated; this is accomplished applying the inverse of the compensation filter defined as described above.