## 3 Evaluation
### Results  
We use five baseline methods for evaluation: [PERSON] et al. [4], which uses an adaptive spectral decomposition based on NMF; [PERSON] and [PERSON] [51], which selects candidates among spectral peaks, validating candidates through additional audio descriptors; [PERSON] [21], a PLCA model for multi-pitch detection from multi-singers, similar to the acoustic model of our proposed system, although it also includes a binary classifier to estimate the final pitch detections from the pitch activations; as well as two multi-pitch detection methods from the Essentia library [54]: Klapuri [49], which sums the amplitudes of harmonic partials to detect pitch presence; and [PERSON] [50], which uses melodic pitch contour information to model pitch detections. For all five of these methods, we also run the post-processing refinement step described in Section 2.1.1 on their output. We evaluate the above systems against two versions of our proposed model: VOCAL4-MP, using only the acoustic model described in Section 2.1; and VOCAL4-VA, using the fully-integrated model. From the multi-pitch detection results in Table 2, it can be seen that our integrated model VOCAL4-MP achieves the highest \(F_{mp}\) on both datasets. In fact, VOCAL4-VA outperforms VOCAL4-MP substantially, indicating that the music language model is indeed able to drive the acoustic model to a more meaningful factorisation. For voice assignment, using each baseline method above that does not output any voice assignment information ([PERSON] [49], [PERSON] and [PERSON] [50], [PERSON] et al. [4] and [PERSON] and [PERSON] [51]), we run our music language model once on its output with default settings and \(M=4\), after the post-processing refinement step. Meanwhile, for [PERSON] and [PERSON] [21], as well as VOCAL4-MP, the voice assignments are derived from each model's probabilistic voice assignment estimates (\(P_{t}(v|p)\) for [21] and \(P_{t}(p|v)\) for VOCAL4-MP). The voice assignment results are shown in Table 1, where it is shown that VOCAL4-VA outperforms the other models, suggesting that a language model is necessary for the task. It is also clear that integrating the language model as we have (rather than simply including one as a post-processing step) leads to greatly improved performance. Specifically, notice that the difference in performance between our model and the baseline methods is much greater for voice separation than for multi-pitch detection, even though we applied our language model to those baseline methods' results as post-processing. Also interesting to note is that our model performs significantly better on the bass voice than on the other voices. While this is also true of many of the baseline methods, for none of them is the difference as great as with our model. Overtones are a major source of errors in our model, and the bass voice avoids these since it is almost always the lowest voice. A further investigation into our model's performance can be found in Figure 5, which shows all of the VOCAL4-VA model's F-measures, averaged across all songs in the corresponding dataset after each EM iteration. The first thing to notice is the large jump in performance at Iteration 15, when the language model is first integrated into the process. This jump is most significant for voice assignment, but is also clear for multi-pitch detection. The main source of the improvement in multi-pitch detection is that the music language model helps to eliminate many false positive pitch detections using the integrated pitch prior. In fact, the multi-pitch detection performance improves again after the 16 th iteration and then remains relatively stable throughout the remaining iterations. The voice assignment results follow a similar pattern, though without the additional jump in performance after Iteration 16. In the Bach chorales, the voice separation performance even continues to improve until the end of all 30 iterations.