# Assessment of Student Music Performances Using Deep Neural Networks
## 6 Conclusions  
Music performance assessment is a broad research area that deals with understanding, modeling, and characterizing several aspects of a music performance. This article explores the use of DNNs for assessing performances of pitched wind instruments for two student categories at different proficiency levels. Specifically, we train DNN based regression models for the prediction of assessments given by expert human judges across several subjective assessment criteria. Two different DNN architectures are proposed for the task: (i) a fully convolutional model using pitch contours as input, and (ii) a convolutional recurrent model using Mel spectrogram as input. The two proposed input representations are chosen with the intent to respectively encode high level melodic content and low level information spanning multiple dimensions. While the melodic information encoded in pitch contours enables better characterization of Musicality and Note Accuracy, timbral information in Mel spectrograms aids assessment of Tone Quality. A third hybrid model combining the above architectures is also proposed, which tries to leverage features learned from both input representations. The proposed DNN models outperform a baseline model using low-level and hand-designed features across all four assessment criteria (Musicality, Note Accuracy, Rhythmic Accuracy and Tonality) for both evaluation metrics (\(R^{2}\) and \(\rho\)). This is observed for both student categories. Finally, in order to explain the model predictions and understand the feature learning process in a better way, model analysis techniques are used. These techniques help explain the effectiveness of pitch contours as input  
Figure 8: Saliency maps using PC-FCN model for four performances of Bb Clarinet (the two on the top are from 2014 Symphonic Band performances, the two on the bottom are from 2013 Middle School performances). The original pitch contour is shown in black and the salient sections are shown in red. The \(y\)-axis represents the normalized pitch values while the \(x\)-axis represents time in terms of blocks. representation for assessing students at a higher proficiency level and show that abstract concepts like Musicality share features across student categories. To summarize, our key contributions include:  
1. We introduce DNN based architectures for the music performance assessment task. The experimental results show the overall promise of using supervised feature learning methods for this task and the shortcomings of traditional approaches in describing music performances. 2. We compare and contrast input data representations at different levels of abstraction and show that information encoded by each is critical for different assessment criteria. 3. Compared to the baseline, we are able to show considerable improvement in model performance. For Musicality, which is arguably the most abstract assessment criterion, we demonstrate clear superior performance of the DNN models. 4. We use model analysis techniques such as filter distance measures and saliency maps to explain the model predictions and further our understanding of the performance assessment process in general. While the results are encouraging, they are not reliable enough for use in practical applications to provide feedback to students. The following avenues for future work might help improve the system:  
1. _Developing better input representations_: Constant Q-Transforms have shown good results with other MIR tasks [28] and are thus a potential candidate for a low level representation. In addition, augmenting the pitch contours with the amplitude variations or pitch saliency could be a simple and yet effective way to encode dynamics. Another potential direction could be to switch to a raw time-domain input representation. While input representations help condense information, which facilitates feature learning, information that might be otherwise useful is discarded during their computation process.