# A Review of Time-Scale Modification of Music Signals +
## 5 TSM Based on the Phase Vocoder (PV-TSM)
### The Phase Vocoder  
Each time-frequency bin \(X(m,k)\) of an STFT can be interpreted as a sinusoidal component with amplitude \(|X(m,k)|\) and phase \(\varphi(m,k)\) that contributes to the \(m^{\text{th}}\) analysis frame of the input signal \(x\). However, the Fourier transform yields coefficients only for a discrete set of frequencies that are sampled linearly on the frequency axis, see Equation (14). The STFT's frequency resolution therefore does not suffice to assign a precise frequency value to this sinusoidal component. The phase vocoder is a technique that refines the STFT's coarse frequency estimate by exploiting the given phase information. In order to understand the phase vocoder, let us have a look at the scenario shown in Figure 8. Assume we are given two phase estimates \(\varphi_{1}=\varphi(m,k)\) and \(\varphi_{2}=\varphi(m+1,k)\) at the time instances \(t_{1}=T_{\text{coef}}(m)\) and \(t_{2}=T_{\text{coef}}(m+1)\) of a sinusoidal component for which we only have a coarse frequency estimate \(\omega=F_{\text{coef}}(k)\). Our goal is to estimate the sinusoid's \"real\" instantaneous frequency IF(\(\omega\)). Figure 8 shows this sinusoidal component (gray) as well as two sinusoids that have a frequency of \(\omega\) (red and green). In addition, we also see phase representations at the time instances \(t_{1}\) and \(t_{2}\). The red sinusoid has a phase of \(\varphi_{1}\) at \(t_{1}\) and the green sinusoid a phase of \(\varphi_{2}\) at \(t_{2}\). One can see that the frequency \(\omega\) of the red and green sinusoids is slightly lower than the frequency of the gray sinusoid. Intuitively, while the phases of the gray and the red sinusoid match at \(t_{1}\), they diverge over time, and we can observe a considerable discrepancy after \(\Delta t=t_{2}-t_{1}\) seconds (blue oval). Since we know the red sinusoid's frequency, we can compute its _unwrapped phase advance_, _i.e._, the number of oscillations that occur over the course of \(\Delta t\) seconds:  
\[\omega\ \Delta t. \tag{19}\]  
Knowing that its phase at \(t_{1}\) is \(\varphi_{1}\), we can predict its phase after \(\Delta t\) seconds:  
\[\varphi^{\text{Pred}}=\varphi_{1}+\omega\ \Delta t. \tag{20}\]  
At \(t_{2}\) we again have a precise phase estimate \(\varphi_{2}\) for the gray sinusoid. We therefore can compute the _phase error_\(\varphi^{\text{Err}}\) (also called the _heterodyned phase increment_[17]) between the phase actually measured at \(t_{2}\) and the predicted phase when assuming a frequency of \(\omega\):  
\[\varphi^{\text{Err}}=\Psi(\varphi_{2}-\varphi^{\text{Pred}})\, \tag{21}\]  
where \(\Psi\) is the _principal argument function_ that maps a given phase into the range \([-0.5,0.5]\). Note that by mapping \(\varphi^{\text{Err}}\) into this range, we assume that the number of oscillations of the gray and red sinusoids differ by at most half a period. In the context of instantaneous frequency estimation, this means that the coarse frequency estimate \(\omega\) needs to be close to the actual frequency of the sinusoid, and that the interval \(\Delta t\) should be small. The unwrapped phase advance of the gray sinusoid can then  
Figure 8: Principle of the phase vocoder. be computed by the sum of the unwrapped phase advance of the red sinusoid with frequency \(\omega\) (red spiral arrow) and the phase error (blue curved arrow):  
\[\omega\ \Delta t+\varphi^{\text{Err}}. \tag{22}\]  
This gives us the number of oscillations of the gray sinusoid over the course of \(\Delta t\) seconds. From this we can derive the instantaneous frequency of the gray sinusoid by  
\[\text{IF}(\omega)=\frac{\omega\ \Delta t+\varphi^{\text{Err}}}{\Delta t}= \omega+\frac{\varphi^{\text{Err}}}{\Delta t}. \tag{23}\]  
The frequency \(\varphi^{\text{Err}}/\Delta t\) can be interpreted as the small offset of the gray sinusoid's actual frequency from the rough frequency estimate \(\omega\).