# Constraint-Based Time-Scale Modification of Music Recordings for Noise Beautification
## 5 Adaption Using Time-Scale Modification (TSM)  
We now describe how we apply time-scale modification (TSM) to adapt the source signal to run synchronously along the reference signal. As described in the introduction, TSM is a procedure for modifying the duration of a given audio signal while preserving other properties such as pitch and timbre. In a typical TSM approach, the input signal is decomposed into short _frames_ of a fixed length, typically in the range of 50 to 100 milliseconds. The frames are then relocated on the time axis to achieve the actual time-scale modification, while, at the same time, preserving the signal's pitch (see [20] for an overview). Many TSM algorithms allow for stretching or compressing the signal by a constant factor. In our scenario, however, the time axis needs to be modified in a non-linear fashion. Such a non-linear modification can be formally described by a _time-stretch function_, which is a strictly monotonously increasing function \(\tau:\mathbb{R}\to\mathbb{R}\) yielding a continuous mapping between input time points (given in seconds) and output time points. Based on such a continuous time-stretch function \(\tau\), as described in [20], one can convert basically all frame-based global TSM procedures into non-linear versions. The main idea of the conversion is to fix an equidistant time grid for synthesis frame positions that will be used to generate the output signal. Then, for every grid point, the inverse of \(\tau\) is used to identify an analysis frame position of the input signal. Finally, these non-linearly spaced analysis frames are used as linearly spaced synthesis frames to construct a modified signal that has the desired local tempo changes. For details, we refer to [20]. In our scenario, the source signal serves as input signal that needs to be modified. Recall that a valid mapping function \(\varphi\) defines a mapping between reference APs \(r_{m}\) and source APs \(\varphi(r_{m})\), \(m\in[1:M]\). We use this mapping to define the pairs \((\varphi(r_{m}),r_{m})\in\mathcal{S}\times\mathcal{R}\), which we call _anchor points_ (see Figure 2(a)). Based on these anchor points, we define a time-stretch function \(\tau\) within the intervall \([\varphi(r_{1}),\varphi(r_{M})]\) by applying linear interpolation between these anchor points (see Figure 2(b)). Finally, one can extend \(\tau\) to the left of \(\varphi(r_{1})\) and to the right of \(\varphi(r_{M})\) in a linear fashion with a slop as defined for the subsequent and previous interval, respectively. Based on this time-stretch function \(\tau\), one can then apply a non-linear TSM procedure as described in the previous paragraph to modify the source signal. In the resulting time axis of the output signal, the mapped source APs are now in sync with the reference APs (see Figure 2(c)). As summarized in [20], there are different types of frame-based TSM procedures including the waveform similarity overlap-add (WSOLA) approach [21] or the phase vocoder (PV) approach [22; 23; 24]. Some of the procedures are designed to preserve the perceptual quality of harmonic signal components, while introducing undesired artifacts in the presence of sudden transient-like signal components. Other TSM procedures can better preserve the sharpness of percussive elements, while having problems in preserving the characteristics of harmonic passages. To reduce unwanted artifacts on both sides,we use the combined TSM procedure introduced in [25]. The idea is to first separate the input signal into a harmonic and a percussive component, then to process each component with a different TSM procedure that preserves its respective characteristics, and finally to generate the output signal by superimposing the processed harmonic and percussive component.