# Automatic Segmentation of Ethnomusicological Field Recordings
## 3 Results
### Segmentation  
We evaluated our segmentation algorithm on a set of 160 field recordings from the Ethnomuse archive, which were manually segmented by ethnomusicologists and contained 3703 segments. When evaluating the accuracy of our segmentation approach on this dataset, we needed to consider the goal of segmentation, which is not to separate every small excerpt of a particular type (e.g., short speech between units), but larger units, such as songs. This also means that boundaries in manual annotations are not placed very precisely, but often quite arbitrarily. For example, if two songs that follow each other are separated by an 8 s break or a short spoken sentence, the boundary annotation may be placed anywhere within the pause. To account for that and provide a realistic measure of the algorithm's performance, we needed to use wide evaluation windows (up to 8 s) within which the boundaries were counted as correctly placed--much larger as is customary, e.g., in Mirex evaluations, where 0.5 s and 1 s windows are used. Note, since the average segment length is over 120 s, such large windows sizes are still acceptable. In Table 3, we provide results of our approach with two differently sized windows: 3 s and 8 s. We also provide a comparison of the presented probabilistic approach to a baseline based on thresholding the silence and/or transition likelihood curves at the value of 0.5.  
With larger window sizes, the algorithm can find a majority of boundaries (recall is at 76%). Precision is not so good, which means that the algorithm oversegments certain regions. This mostly occurs in long speech sections, which contain regions of silence that, for example, occur when people reflect on past events (consequently causing new boundaries), or in solo singing performances that are interleaved with reciting or spoken statements, such as \"this is repeated three times and we start dancing in a circle so and so...\" (causing high KL divergence and new boundaries). False negatives occur when performances follow each other without significant changes, for example, several songs sung in a row almost without interruptions or when the boundary was placed because of the change of topic in speech parts, which the algorithm is not designed to detect. Segment start or end points may also be missed, because they interleave with speech, so that the boundary is placed either too soon or too late in a recording.  
The accuracy of classification of correctly found segments into one of the four classes is similar to the overall classification accuracy of entire excerpts as presented previously--95%, errors are also similar to the ones described previously.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & \multicolumn{4}{c}{**Predicted**} \\  & & _solo_ & _chair_ & _instr._ & _speech_ \\ \hline \multirow{4}{*}{**true**} & _solo_ & 0.84 & 0.07 & 0.02 & 0.07 \\  & _chair_ & 0.05 & 0.91 & 0.03 & 0.01 \\  & _instr._ & 0.01 & 0.02 & 0.96 & 0.01 \\  & _speech_ & 0.05 & 0.01 & 0.01 & 0.93 \\ \hline \hline \end{tabular}
\end{table}
Table 2: The confusion matrix calculated from test results of all three cross validation folds.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Algorithm** & \multicolumn{3}{c}{**3s Window**} & \multicolumn{3}{c}{**8s Window**} \\ \hline  & **P** & **R** & **F1** & **P** & **R** & **F1** \\ \hline proposed approach & 0.47 & 0.62 & 0.52 & 0.56 & 0.76 & 0.63 \\ silence threshold & 0.51 & 0.38 & 0.40 & 0.58 & 0.44 & 0.46 \\ transition threshold & 0.45 & 0.42 & 0.40 & 0.56 & 0.54 & 0.51 \\ silence \& transition thr. & 0.44 & 0.60 & 0.48 & 0.52 & 0.70 & 0.57 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Precision, recall, and F1 measure for segmentation with different tolerance windows.