# Automatic Evaluation of Piano Performances for STEAM Education
## 5 Experiments and Results
### Experimental Results
#### 5.5.4. Song for Kids by CNN Approach  
Next, we examined our piano performance assessment system for whole song performances. There were five kids' songs used in this experiment, including Beautiful Brown Eyes, Hot Cross Buns, Jingle Bells, The Cuckoo, and Twinkle Twinkle. For each song, we collected 117 piano performances from four teachers and nine students, including 39 performances played with left hands, 39 performances played with right hands, and 39 performances played with both left and right hands. Here, we only tested the CNN-based approach since it performs better than the other three approaches. Furthermore, we modified the configuration of CNN from Figures 6-17 in order to better handle the long duration of the whole song performance. Figure 17 shows the architecture of the 2-layer CNN with Filters. In the three phases of the CNN feature maps, the current input size is \(160\times 1\) on three sets of convolutional layers with kernel size \(3\times 1\). The kernel was used to deal with local features in the first convolutional phase. We used a dropout parameter value of 0.5 and Max pooling \(2\times 1\). Finally, the process transfers the operation into Flatten, Dense, Fully Connected layer, and SoftMax Activation. We determined the new hyperparameters appropriate for the evaluation of kids' songs performances by using the training data of 54 samples and then tested the other 63 data samples. Other parameters in this experiment follow the best parameters obtained from both the Legato and Staccato scale experiments. The resulting classification accuracies are shown in Table 7. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Song**} & **Left Hand** & **Right Hand** & \
\begin{tabular}{c} **Both Left and** \\ **Right Hands** \\ **Accuracy (\%)** \\ \end{tabular} & \multirow{2}{*}{
\begin{tabular}{c} **Overall** \\ **Accuracy (\%)** \\ \end{tabular} } \\ \cline{1-1} Hot Cross Buns & 82.33\% & 73.22\% & 75.91\% & 90.47\% \\ Beautiful Brown Eyes & 69.34\% & 68.22\% & 90.81\% & 90.32\% \\ Jingle Bell & 85.77\% & 61.17\% & 87.76\% & 88.70\% \\ The Cuckoo & 83.86\% & 61.46\% & 74.46\% & 83.87\% \\ Twinkle Twinkle & 100\% & 87.50\% & 75.00\% & 80.64\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: CNN Classification results for kids’ songs. Figure 17: Architecture of the 2-layers CNN with Filters. As shown in Table 7, we examined four types of piano performance, including playing with the left hand, playing with the right hand, playing with both left and right hands, and the overall playing (overall is the datasets combination of the left hand, right hand, and both left and right hands). For the overall classification, accuracies are higher than 80%. Among them, we found that the song \"Twinkle Twinkle\" had the lowest accuracy when compared to the other songs. Thus, we focused on the recordings of the song \"Twinkle Twinkle\", associated with the spectrograms, waveforms, and confusion matrices. We observed the main problem of Twinkle Twinkle in terms of beat and tempo in the spectrograms, and waveforms are shown in Figures 18-21. The beat is used as the placement of sounds in time, with repeated or regular patterns, while the tempo is the rate of speed for a piece of music [32; 33]. In this case, the classification accuracy for the piano performances is lower when played with both left and right hands and is higher when played with separate hands. When we divided the dataset for each hand, the overlapping of the beat and tempo in the spectrograms and waveforms was reduced. Hence, we speculate that this might be due to the changeable characteristics of beat and tempo in both left and right hands, making it difficult to classify. To summarize, the left hand in the song \"Twinkle Twinkle\" achieves 100% accuracy using the CNN classifier. The confusion matrices, shown in Figures 22-25, indicate the correct classification and misclassification into three categories: \"Good\", \"Normal\", and \"Bad\". From these matrices, the prediction of the left hand shows no classification error.