# Efficient Melody Extraction Based on Extreme Learning Machine
## 4 Experimental Results and Discussions
### Experimental Results on Test Sets
#### 4.3.2 Statistical Significance Analysis  
To determine if the proposed method achieves significantly higher OA and lower RPA compared with the reference methods, the statistical significance analysis was performed. A paired-sample t-test was conducted between the proposed and reference methods in terms of OA and RPA. The statistical significance results about OA are reported in Table 3. It can be seen from this table that the proposed method performed better than reference methods with respect to OA on ISMIR2004, but the differences were not significant. It outperformed the other methods significantly on both MIREX05 and MIR-1K. Figure 6: Accuracies on MIREX05. Figure 7: Accuracies on MIR-1K. The statistical significance results about RPA are listed in Table 4. It can be found from this table that Melodia and PFDP achieved significantly higher RPAs than the proposed method on ISMIR2004. The proposed method obtained comparable RPA with the other methods on this dataset. However, all methods, except MEA, outperformed the proposed method significantly on MIREX05. As only one recording of MIREX05 was covered in the training set, it can be inferred that training samples were important for ELM parameter training, similar to other machine learning-based methods. As far as MIR-1K is concerned, the proposed method surpassed BG1 and HCT significantly, was comparable with MEA, and was inferior to Melodia and PFDF significantly with respect to RPA. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Datasets** & **Pairs** & **Mean** & \(t\) **statistic** & \(p\)**-Value** \\ \hline \multirow{8}{*}{ISMIR2004} & Proposed-BG1 & 10.85 & 1.33 & 0.20 \\  & Proposed-HCT & 9.39 & 1.39 & 0.19 \\  & Proposed-MEA & 1.70 & 0.28 & 0.78 \\  & Proposed-Medonia & 13.53 & 2.35 & 0.03 \\  & Proposed-FFDP & 13.54 & 2.27 & 0.04 \\  & Proposed-PM & 10.72 & 1.26 & 0.23 \\ \hline \multirow{8}{*}{MIREX05} & Proposed-BG1 & 24.47 & 3.47 & 0.01 \\  & Proposed-HCT & 21.00 & 3.50 & 0.01 \\ \cline{1-1}  & Proposed-MEA & 8.74 & 0.94 & 0.37 \\ \cline{1-1}  & Proposed-Melodia & 21.08 & 3.42 & 0.01 \\ \cline{1-1}  & Proposed-PFDP & 23.77 & 4.04 & 0.00 \\ \cline{1-1}  & Proposed-PM & 26.92 & 5.28 & 0.00 \\ \hline \multirow{8}{*}{MIREX05} & BG1-Proposed & \(-\)22.93 & \(-\)28.23 & 0.00 \\ \cline{1-1}  & HCT-Proposed & \(-\)5.42 & \(-\)8.34 & 0.00 \\ \cline{1-1}  & MEA-Proposed & 0.74 & 0.99 & 0.32 \\ \cline{1-1}  & Melodia-Proposed & 12.74 & 20.23 & 0.00 \\ \cline{1-1}  & PFDP-Proposed & 7.63 & 12.77 & 0.00 \\ \cline{1-1}  & PM-Proposed & 6.01 & 11.08 & 0.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Statistical significance analysis of a paired-sample t-test (raw pitch accuracy (RPA))between the proposed and reference methods. \begin{table}
\begin{tabular}{c l c c c} \hline \hline
**Datasets** & **Pairs** & **Mean** & \(t\) **statistic** & \(p\)**-Value** \\ \hline \multirow{8}{*}{ISMIR2004} & Proposed-BG1 & 4.08 & 0.67 & 0.51 \\  & Proposed-HCT & 5.56 & 1.08 & 0.30 \\  & Proposed-MEA & 9.34 & 1.04 & 0.06 \\  & Proposed-Melodia & 2.20 & 0.52 & 0.62 \\  & Proposed-PFDP & 2.35 & 0.51 & 0.62 \\  & Proposed-PM & 6.45 & 0.97 & 0.35 \\ \hline \multirow{8}{*}{MIREX05} & Proposed-BG1 & 12.44 & 2.20 & 0.05 \\  & Proposed-HCT & 15.28 & 3.00 & 0.01 \\ \cline{1-1}  & Proposed-MEA & 20.22 & 2.79 & 0.02 \\ \cline{1-1}  & Proposed-Melodia & 15.97 & 3.27 & 0.01 \\ \cline{1-1}  & Proposed-PFDP & 11.39 & 2.60 & 0.01 \\ \cline{1-1}  & Proposed-PM & 13.57 & 2.89 & 0.02 \\ \hline \multirow{8}{*}{MIREX05} & Proposed-BG1 & 38.55 & 66.95 & 0.00 \\ \cline{1-1}  & Proposed-HCT & 28.17 & 56.31 & 0.00 \\ \cline{1-1}  & Proposed-MEA & 20.36 & 38.31 & 0.00 \\ \cline{1-1}  & Proposed-Melodia & 16.52 & 32.55 & 0.00 \\ \cline{1-1}  & Proposed-PFDP & 15.38 & 31.24 & 0.00 \\ \cline{1-1}  & Proposed-PM & 21.55 & 43.63 & 0.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Statistical significance analysis of a paired-sample t-test (OA) between the proposed and reference methods.