# A Review of Time-Scale Modification of Music Signals +
## 7 Applications  
### Music Synchronization--Non-Linear TSM  
In scenarios like _automated soundtrack generation_[36] or _automated DJing_[1; 2], it is often necessary to synchronize two or more music recordings by aligning musically related beat positions in time. However, music recordings do not necessarily have a constant tempo. In this case, stretching or compressing the recordings by a constant factor \(\alpha\) is insufficient to align their beat positions. Instead, the recordings'time-scales need to be modified in a non-linear fashion. The goal of non-linear TSM is to modify an audio signal according to a strictly monotonously increasing _time-stretch function_\(\tau:\mathbb{R}\to\mathbb{R}\), which defines a mapping between points in time (given in seconds) of the input and output signals. Figure 14 shows an example of such a function. The first part, shown in red, has a slope greater than one. As a consequence, the red region in the input signal is mapped to a larger region in the output, resulting in a stretch. The slope of the function's second part, shown in blue, is smaller than one, leading to a compression of this region in the output. One possibility to realize this kind of non-linear modifications is to define the positions of the analysis frames according to the time-stretch function \(\tau\), instead of a constant analysis hopsize \(H_{\text{a}}\). The process of deriving the analysis frames' positions is presented in Figure 14. Given \(\tau\), we first fix a synthesis hopsize \(H_{\text{s}}\) as we did for linear TSM (see Section 2). From this, we derive the _synthesis instances_\(s_{m}\in\mathbb{R}\) (given in seconds), which are the positions of the synthesis frames in the output signal:  
\[s_{m}=\frac{m\ H_{\text{s}}}{F_{\text{s}}}. \tag{34}\]  
By inverting \(\tau\), we compute the _analysis instances_\(a_{m}\in\mathbb{R}\) (given in seconds):  
\[a_{m}=\tau^{-1}(s_{m}). \tag{35}\]  
Figure 14: Example of a non-linear time-stretch function \(\tau\). When using analysis frames indicated by the analysis instances for TSM with a procedure of our choice, the resulting output signal is modified according to the time-stretch function \(\tau\). To this end, all of the previously discussed TSM procedures can also be used for non-linear TSM. A very convenient way of defining a time-stretch function is by introducing a set of _anchor points_. An anchor point is a pair of time positions where the first entry specifies a time position in the input signal and the second entry is a time position in the output signal. The actual time-stretch function \(\tau\) is then obtained by a linear interpolation between the anchor points. Figure 15 shows a real-world example of a non-linear modification. In Figure 15b, we see the waveforms of two recorded performances of the first five measures of [PERSON]'s Symphony No. 5. The corresponding time positions of the note onsets are indicated by red arrows. Obviously, the first recording is longer than the second. However, the performances' tempi do not differ by a constant factor. While the eighth notes in the first and third measure are played at almost the same tempo in both performances, the durations of the half notes (with fermata) in measures two and five are significantly longer in the first recording. The mapping between the note onsets of the two performances is therefore non-linear. In Figure 15c, we define eight anchor points that map the onset positions of the second performance to the onset positions of the first performance (plus two additional anchor points aligning the recordings' start times and end times, respectively). Based on these anchor points and the derived time-stretch  
Figure 15: **(a)** Score of the first five measures of [PERSON]’s Symphony No. 5; **(b)** Waveforms of two performances. Corresponding onset positions are indicated by red arrows; **(c)** Set of anchor points (indicated in red) and inferred time-stretch function \(\tau\); **(d)** Onset-synchronized waveforms of the two performances.