# Automatic Segmentation of Ethnomusicological Field Recordings
## 2 Materials and Methods
### Segmentation  
The goal of segmentation is to split the recording into a set of consistent units, such as speech or singing. Unlike broadcast segmentation, we are not interested in each small change of contents (e.g., just a few spoken sentences), but aim to segment the recording into larger units (entire songs or longer spoken parts). This is also where the presented approach differs the most when compared to other speech-music segmentation approaches (e.g., [5; 6])--it is designed to segment audio into broader regions and to ignore small local changes. The presented segmentation algorithm is a modification of our approach first presented in [1]. To segment a recording, we first find a set of candidate segment boundaries and calculate the likelihood  
Figure 2: The network architecture. of splitting the recording at each boundary. We consider two criteria for boundary placement: A criterion based on signal energy, such as when performances are separated by regions of silence (or noise, since recordings may be quite noisy), and a criterion based on the change in signal content, such as when speech is followed by singing. The way the two criteria are calculated is the main distinction between the presented approach and [1]. To observe the presence of a signal (vs. silence or noise), we first calculate the RMS energy, \(e\), of the audio signal over 46 ms time frames. The energy is compared to two measures capturing the global and local amplitude thresholds. First, the 1 st percentile of the energy within 20 s time windows is calculated and the global noise floor, \(n_{g}\), is set to 5 dB over this value and lower bounded to \(-60\) dB. Such an adaptive estimate is needed, as field recordings are often noisy or recorded with varying dynamics, so an absolute threshold would not be suitable as a global noise floor estimate. The local noise floor, \(n_{l}\), captures local dynamics and is set to 15 dB below the energy median filtered with a 6 s time window. Based on both noise floors, the silence indicator function, \(e_{s}(t)\), is set to a value of one, when energy at time \(t\) falls below any of the two noise floors, and zero elsewhere. The likelihood of placing a boundary at time \(t\), \(p_{s}(t)\), is calculated by zero-phase filtering the indicator function with a second order low pass filter with 0.1 Hz cutoff. An example is given in Figure 3.  
Transitions between different kinds of signal content (e.g., speech to singing) are detected by calculating the symmetric Kullback-Leibler (KL) divergence, \(d(t)\), between probabilities of target classes (as calculated by the deep classifier) within 10 s windows to the left and right of each time frame. KL divergence will be large when the contents (class probabilities) on both sides will be different, and close to zero, when they will be similar. The transition boundary placement likelihood, \(p_{t}(t)\), is obtained similarly to the silence likelihood by thresholding and low-pass filtering the divergence function, \(d(t)\). Both likelihood curves are depicted in Figure 3, which shows an excerpt of a field recording, where content varies between speech and singing. The transition likelihood indicates segment boundaries due to transitions between different content types well, while the silence likelihood indicates boundaries between individual units separated by silence/noise, as well as short pauses, which often occur during speech segments. To segment a recording, we construct a probabilistic model, which integrates the data on segment boundary likelihoods with prior knowledge of typical segment lengths. First, a set of boundary candidates, \(B_{i}\), is obtained by finding peaks of both curves. Based on the boundary candidates, segmentation is defined as a sequence of segments, \(S_{i1},\ S_{i2},\ \ldots,\ S_{iN},0<i1<i2<\ \ldots\ <iN\), where  
Figure 3: Segment boundaries and the two boundary likelihood curves: Transition and silence.