# Singing Transcription from Polyphonic Music Using Melody Contour Filtering
## 4 Evaluation
### Audio Melody Extraction
#### 4.1.3 Results  
The algorithms from the results of MIREX for the past two years are shown below, and systems are denoted by the initials of their authors: KN4 [16], AH1 [37], HZ4 (our submission), HLD1 (different variants from AH1), KD1 [38], ZC1 [39], and BH (who submitted two variants) [40].  
The overall accuracy of our algorithm (HZ4) was lower than that of KN4 and close to AH1 (Table 1). Both of the previous two algorithms were realized through neural network modeling. Our system focuses on the innovation of contour searching, in contrast to the traditional frame-to-frame filtering mode. Significantly, we directly analyzed the contours using existing datasets, especially the threshold values in each detection, which seemed to be acceptable. Due to the existence of a large number of thresholds in the experiment, there is still considerable room for progress in melody extraction. Owing to the limitations in the precision of MIREX, we only submitted the extension system.  
In addition to the estimation of accuracy, we also present the performance of each algorithm for different datasets, mainly including the statistical results for the overall accuracy (OA). As shown in Figure 9, in the ADC04 datasets, the algorithm proposed in this paper reached an optimal accuracy rate of 0.824; barring an accuracy rate of 0.815 for the KD1 algorithm, other algorithms did not reach a value of more than 80%. For the MIREX05 dataset, BH1 achieved the highest accuracy rate of 0.679, and the overall accuracy rate of this dataset was the lowest. For the MIREX08 and MIREX09 data sets, the highest accuracy rates were 0.793 and 0.828, respectively, and the results were within the applicable range.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Algorithm** & **OA** & **RPA** & **RCA** & **VR** & **VFA** \\ \hline KN4 & **0.724** & 0.698 & 0.702 & 0.764 & 0.124 \\ AH1 & 0.703 & 0.746 & 0.772 & 0.797 & 0.171 \\
**HZ4** & 0.701 & 0.721 & 0.751 & 0.879 & 0.32 \\ HLD1 & 0.686 & 0.739 & **0.774** & 0.777 & 0.177 \\ KD1 & 0.681 & **0.749** & 0.767 & 0.832 & 0.373 \\ ZC1 & 0.671 & 0.612 & 0.619 & 0.698 & 0.174 \\ BH2 & 0.657 & 0.734 & **0.774** & 0.714 & 0.292 \\ BH1 & 0.64 & 0.721 & 0.756 & 0.707 & 0.312 \\ HH4 & 0.444 & 0.615 & 0.633 & 0.956 & 0.92 \\ WJH2 & 0.433 & 0.599 & 0.621 & 0.955 & 0.914 \\ \hline \hline \end{tabular}
\end{table}
Table 1: MIREX correlative results in 2019 and 2020.  
Figure 10 shows line graphs for the three parameters--VR, VFA, and OA--for different values of \(\theta\) and \(\mu\). Comparing (a) and (b), we can see from the curve fluctuation that the values of \(\theta\) and \(\mu\) are inversely proportional to the two parameters of VR and VFA; however, we paid more attention to the results of OA. When \(\theta\) = 0.3, OA reaches its highest value. It can be seen from the figure that the curve in Figure 10a is steeper, which proves that \(\theta\) has a greater impact on the singing voice detection results. It can be seen from Figure 10b that, when \(\mu\) is selected as 0.5, the OA is higher. This also shows the process of selecting parameters in our experiments.