# Assessment of Student Music Performances Using Deep Neural Networks
## 5 Model Analysis  
In order to better explain the model predictions and explore the feature learning process further, we take a closer look at the PC-FCN model. We do not use the other larger models because they have high dimensionality in the feature space, which makes analysis using the tools mentioned below difficult. The PC-FCN model is suitable for our analysis because of its simpler architecture and smaller size, which enables easier and interpretable analysis. The 1 st convolutional layer (referred to as _conv1_) has a 1-dimensional kernel of size 7 and a channel width of 4. This weight matrix is flattened into a \(28\times 1\) dimensional vector and is extracted individually for the trained models for all four assessment criteria and for both student categories. Subsequently, pair-wise Cosine distance is computed between the eight vectors. The resulting distance matrix is shown as a plot in Figure 7. An interesting observation from the plot is that, for the Symphonic Band, _conv1_ weight vectors for the Musicality, Note Accuracy and Rhythmic Accuracy models (s-M, s-N, s-R) are very closely aligned (see the top \(3\times 3\) square in Figure 7), which indicates that even though these models are trained for different criteria, the features learned by _conv1_ are similar. In addition, the effectiveness of these weights is clear: PC-FCN is the best (or close to the best) performing model (based on the \(R^{2}\) metric) for the three criteria under consideration. However, the same cannot be said for the Middle School _conv1_ layers. This hints in favor of our earlier conjecture that the high level melodic encoding inherent in the PC representation might be more important for assessing students with higher musical proficiency and while performing more complex scores. Within a particular student category, the correlation amongst the ground truth ratings for different assessment criteria tend to indicate the same. For example, for Symphonic Band, the correlation between Musicality and Note Accuracy is 0.75, whereas, for Middle School, it is 0.60. A second observation from Figure 7 is that, for Musicality, _conv1_ layers of both student categories are closely aligned, which is not the case for the other assessment criteria. This suggests that there is potential for transfer learning for the Musicality criteria, i.e., a model trained on one set of students can be used to test another with suitable modifications. To test out this hypothesis, a short cross-category \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Symphonic Band**} & \multicolumn{4}{c}{**Middle School**} \\ \hline
**Models** & **M** & **N** & **R** & **T** & **M** & **N** & **R** & **T** \\ \hline SVR-BD & 0.612 & 0.547 & 0.649 & 0.628 & 0.619 & 0.595 & 0.418 & 0.587 \\ PC-FCN & **0.744** & 0.647 & **0.743** & 0.612 & 0.619 & 0.561 & 0.465 & 0.560 \\ M-CRNN & 0.733 & 0.582 & 0.700 & 0.624 & 0.723 & **0.666** & 0.602 & 0.573 \\ PCM-CRNN & 0.689 & **0.694** & 0.655 & **0.649** & **0.724** & 0.653 & **0.630** & **0.634** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Evaluation results showing the Pearson Correlation Coefficient \(\rho\) for all models and both student categories (M: Musicality, N: Note Accuracy, R: Rhythmic Accuracy, T: Tone Quality, SVR-BD: Baseline Model, PC-FCN: Fully Convolutional Pitch Contour Model, M-CRNN: Convolutional Recurrent Model with Mel Spectrogram, PCM-CRNN: Hybrid Model Combining Mel-Spectrogram and Pitch Contour Inputs)experiment is conducted where the PC-FCN models trained on Middle School are tested on Symphonic Band and vice versa. We obtain an \(R^{2}=0.193\) for testing the Middle school model on Symphonic Band and an \(R^{2}=-0.106\) the other way around. While these results are not impressive, it is worth noting that results of this cross category testing for Note Accuracy and Rhythmic Accuracy were far worse with an average \(R^{2}=-0.765\). This shows that Musicality as an abstract general concept can be potentially modeled using data from different student categories.