# Melody Extraction Using Chroma-Level Note Tracking and Pitch Mapping
## 3 Evaluation Results and Discussion
### Evaluation Results
#### 3.3.4 Performance on Different Collections  
The performance of the compared methods is first evaluated on ISMIR2004. To intuitively show the results on each recording, the RPAs, RCAs, and OAs for the first 10 excerpts are illustrated and discussed herein. The overall evaluation results on this dataset are provided in Table 2. The RPAs of all compared methods with respect to the excerpts are illustrated in Figure 6. It can be seen from Figure 6 that the RPAs of all methods vary a great deal among different excerpts. The HCT and MEA perform poorly on the 9 th excerpt, and BHWM achieves lower RPAs than others on the 10 th excerpt. Comparatively, the proposed method obtains \"smoother\" RPAs on these ten excerpts. The RCAs of different methods on the first 10 excerpts of ISMIR2004 are depicted in Figure 7. Similar conclusions can be got as from the RPA results, indicating the robustness of the proposed method among these excerpts. \begin{table}
\begin{tabular}{c c c c c} \hline
**Collection** & **Method** & **OA (\%)** & **RPA (\%)** & **RCA (\%)** \\ \hline \multirow{4}{*}{ISMIR2004} & HCT & 62.29 & 70.83 & 74.10 \\  & BHWM & 66.64 & 74.78 & 81.06 \\  & MEA & 58.77 & 63.59 & 68.64 \\  & Proposed & 59.35 & 64.23 & 69.12 \\ \hline \multirow{4}{*}{MIR1K} & HCT & 33.54 & 34.82 & 44.15 \\  & BHWM & 35.63 & 47.52 & 54.39 \\ \cline{1-1}  & MEA & 43.89 & 56.07 & 63.09 \\ \cline{1-1}  & Proposed & 44.58 & 53.89 & 61.93 \\ \hline \end{tabular}
\end{table} 
Table 2: Accuracies of different methods on ISMIR2004 and MIR1K. Figure 5: OAs of compared methods on Bach10 (%). Figure 6: Raw pitch accuracies of different methods on ISMIR2004. The OAs of all compared methods are illustrated in Figure 8. It can be seen that the proposed method performs the best or second best for recordings No. 1, 3, 5, 7, and 9. It can be concluded from Figures 6-8 that the proposed method works well on these excerpts of ISMIR2004. The overall evaluation results of the proposed and other reference methods on ISMIR2004 and MIR1K are listed in Table 2. It can be seen from this table that the proposed method outperforms the MEA on the ISMIR2004 dataset, and performs inferior than HCT and BHWM. Moreover, the proposed method gets the highest overall accuracy (OA), and obtains the second best RPA and RCA on the MIR1K dataset. Therefore, the proposed method can track the dynamic pitch changing within notes and achieves good performance comparing with the state-of-the-art methods. It is interesting to see that the proposed method's performance on Bach10 is much better than that on the other two datasets. When we look into the recordings, it is found there are deep vibratos and pitch glides in some excerpts of ISMIR2004, such as those opera and pop recordings. The proposed method could not track such fast and severe pitch variations. On the other hand, the melodies in MIR1K originate from the singing voice, where pitches are also modulated. However, the proposed method still works comparatively well on this dataset. That is because most pitches in this dataset do not vary so severely as in operas. Therefore, it can be concluded that the proposed method can track the ordinary pitch variation exhibited in the singing voice. Its performance on excerpts with deep vibratos and pitch glides still need to be improved. As an example, the results corresponding to the processing procedure, including spectrogram, HPCP, DP-based chroma tracking output, coarse mapped pitches, fine-tuned pitches, and the final melody output, are shown in Figure 9. This excerpt is taken from the MIR1K collection (Ani_4_05.wav). The melody originates from the singing voice with some vibrato as can be seen in Figure 9a. The chromagram constructed from the HPCP vectors is shown in Figure 9b. Then, it is  
Figure 8: Overall accuracies of different methods on ISMIR2004. Figure 7: Raw chroma accuracies of different methods on ISMIR2004. not difficult to obtain the DP-based chroma tracking output as illustrated in Figure 9c.