# Assessment of Student Music Performances Using Deep Neural Networks
## 1 Introduction  
Music is essentially a performing art. Western classical music in particular often requires musicians to perform an acoustic rendition of a written piece of music, referred to as a musical _score_. While the score provides essential information regarding the sequence of notes and the overall rhythm, several finer details of the performance such as dynamics and timing are often left for the performer to decide. During a musical performance, the performer has to understand the musical notation, incorporate these finer details into their thoughts, and convert these thoughts into complex and precise motor movements. This makes music performance amongst the most skill-intensive actions performed by human beings [1] spanning all the three domains of learning [2], i.e., _cognitive_ (e.g., the knowledge of music theory and understanding of musical symbols), _psychomotor_ (e.g., ability to move specific body parts in a coordinated way) and _affective_ (e.g., feel and express the emotions through performance). Learning to perform music is, therefore, a complex and time-consuming task at the core of music education. The average music student requires regular attention and feedback from a trained teacher to improve their performance skills in a holistic manner. The teacher is expected to provide a consistent and reliable assessment of multiple facets of a performance. These facets include, for example, pitch and rhythmic accuracy or general expressiveness. The ill-defined and subjective nature of these _criteria_ result in high variance of such assessments amongst music educators [3; 4]. In spite of these potential issues, subjective performance assessment is ubiquitous in music education: selection inensembles/bands is determined through competitive auditions and progress is monitored through recials. Hence, improving the reliability and consistency of music performance assessments can play a critical role in the music learning process. The focus of this article lies in developing data-driven computational models capable of understanding music performances so as to assess them along several criteria (see Figure 1). The key motivation is to provide reliable and reproducible feedback to students by introducing an element of objectivity to the process. A better understanding of both the objective parameters characterizing music performances and the main factors contributing to their assessment can also give us new insights into the production and perception of musical performance. The last two decades have seen significant progress and improvement in the field of music information retrieval (MIR) [5], which focuses on algorithms and methods for the analysis and extraction of information from music signals. Examples of typical MIR tasks are fundamental pitch detection [6; 7], music transcription [8; 9] and source separation [10]. Approaches to solve these tasks can be seen as critical building blocks for systems analyzing and assessing music performances [11; 12; 13; 14; 15; 16; 17; 18; 19; 20]. There exist commercial systems for performance assessment such as Yousician [21] and SmartMusic [22], which provide students with visual feedback during their individual practice sessions. However, the assessments are either simplistic (note correct/incorrect), cover only a small aspect of the performance, or are opaque with respect to their computation and meaning. Previous research on objective music performance assessment, utilizing traditional low-level audio features such as Mel frequency cepstral coefficients (MFCCs) or hand-crafted features characterizing pitch and rhythm could only show limited success [11; 14; 18; 19]: while the models did capture some aspects of the performance, they were unable to achieve high accuracy results. Recently, it has been hypothesized that this \"sub-optimal\" performance could stem from the limitation of hand-designed features and that feature learning might allow for capturing relevant information not represented in the hand-crafted feature sets [23].