## 4 Initial Attempts at Lightweight Machine Learning
### Experimental Results  
For each classifier, we performed ten-fold cross-validation with pseudo-random, where each fold was trained using 90% of the data and tested with the rest 10%. Table 1 lists the average cross-validation accuracy of the binary classification for whether a song is traditional Chinese anhemitonic pentatonic folk, obtained by applying different combinations of features to the three classifiers, where the last two rows give the best leave-one-out cross-validation results with their corresponding feature combinations as references.  
Overall, all the classifiers show their ability to discriminate the traditional Chinese anhemitonic pentatonic folk songs. Note that the \(k\)-NN results are significantly affected by \(k\), and the value of 49 shows superiority in the ten-fold cross-validation, while the value of 3 leads the results in the leave-one-out cross-validation.  
From the feature combination perspective, applying the complete feature set produced satisfactory results but did not reach the optimal performance on the \(k\)-NN classifier. The \(NT\) features work viably on their own, even the top on the SVM classifier. The \(DoBell\) feature set outperforms other combinations on the \(k\)-NN classifier.  
Features extracted for both types of merged pitch histograms generally perform better than those of the not-merged histograms. However, the way merged pitch histograms help Chinese pentatonic folk songs enclose better bell-shaped envelopes incidentally benefits other songs' shapes, as Table 2 implies. For example, the merged pitch histogram in Figure 7(12) promotes the distribution to a bell shape.  
Anhemitonic pentatonic pitch histograms are not necessarily better than the original histograms and even work worse in the merged case. We witnessed that the anhemitonic pentatonic pitch histogram, like the merged histogram, may unexpectedly improve other songs' bell-shapedness while advantaging the bell shape of the Chinese pentatonic folk songs, as exemplified in Figure 7(4-6) and explained in Section 3.  
The fact that the merged operation and the anhemitonic-pentatonic operation make some non-anhemitonic-pentatonic songs challenging to distinguish is also reflected by the confusion matrices in Figure 11, where true positives outperform true negatives overall. The uncomplicated classifiers trained with our designed features perform well in correctly labeling the traditional Chinese anhemitonic pentatonic songs but are likely to misidentify some other types of songs.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Feature Combination** & **GaussianNB** & \(k\)**-NN (\(k\) = 49)** & **SVM** \\ \hline
4 \(NT\) features & 0.81 \(\pm\) 0.11 & 0.83 \(\pm\) 0.13 & **0.82 \(\pm\) 0.09** \\
4 \(DoBell\) features & 0.80 \(\pm\) 0.11 & **0.87 \(\pm\) 0.16** & 0.79 \(\pm\) 0.10 \\
4 features extracted from basic PH & 0.66 \(\pm\) 0.10 & 0.75 \(\pm\) 0.12 & 0.73 \(\pm\) 0.12 \\
4 features extracted from merged PH & 0.81 \(\pm\) 0.08 & 0.82 \(\pm\) 0.11 & 0.80 \(\pm\) 0.08 \\
4 features extracted from basic AP PH & 0.77 \(\pm\) 0.10 & 0.76 \(\pm\) 0.11 & 0.73 \(\pm\) 0.14 \\
4 features extracted from merged AP PH & 0.80 \(\pm\) 0.07 & 0.78 \(\pm\) 0.13 & 0.77 \(\pm\) 0.09 \\ All 8 features & **0.82 \(\pm\) 0.10** & 0.85 \(\pm\) 0.12 & **0.82 \(\pm\) 0.09** \\ \hline Best leave-one-out cross-validation result & 0.80 & 0.81 & 0.80 \\ Corresponding feature combination & All 8 features & 4 \(DoBell\), \(k=3\) & 4 \(NT\) features \\ \hline \hline \end{tabular}
\end{table}
Table 1: The mean and standard deviation values of the three classifiers’ ten-fold cross-validation accuracies applying different combinations of features. The bolded results indicate the optimal performance of each classifier. The last two rows give the best leave-one-out cross-validation accuracies and their corresponding feature combinations as references. PH: pitch histogram; AP: anhemitonic pentatonic.