DMRN + 11 : Digital Music Research Network

We discuss our work in modelling and generating music transcriptions using deep recurrent neural networks. In contrast to similar work, we focus on creating a rich evaluation methodology that seeks to address questions related to what a model has learned about the music, how useful it is for music practices, and its broader implications for music tradition. We engage with a specific homophonic music practice (session music), and present several examples of using our models for music composition in and out of the conventions of that idiom. We are currently exploring how these computer models can contribute to the tradition by engaging with its practitioners. The work in [1][2] describes our application of deep recurrent neural networks (specifically, long short-term memory) to modelling textual music transcriptions. Each model estimates the joint probability distribution over a set of symbols, either single characters in a large sequence of text, or higher-level tokens in single transcriptions. We build these models from crowd-sourced transcriptions of “session music,” e.g., Celtic and Morris. Our code and data are available: https://github.com/IraKorshunova/folk-rnn. Training a model entails adjusting its weights such that it predicts with high probability the “correct” symbol given a number of preceding symbols. The resulting models are generative in nature, and so can generate new transcriptions via sampling and feedback. This provides us with opportunities to discover what a model has actually learned, but also to incorporate them into a music practice “pipeline. ” In this work, we describe our approaches to evaluation, and how we are using the system to engage the session music community. The typical approach to quantitatively evaluating these models involves a comparison of descriptive statistics of the training data and generated data. This provides low-level sanity checks on the model output, but does not provide any indication of the music quality of the transcriptions. We find that our models are generating transcriptions that share many of the characteristics of the real ones, but are biased to generating transcriptions with specific meters and lengths. To move closer to evaluating the generated transcriptions, we look at them through the lens of music analysis, e.g., as a *This research is supported by AHRC Grant No. AH/N504531/1. BLS is with the School of Electronic Engineering and Computer Science, Queen Mary University of London, UK (corresponding author e-mail: b.sturm@qmul.ac.uk). OBT is with the Music Department, Kingston University, UK (e-mail: O.Ben-Tal@kingston.ac.uk). teacher of composition judging the creation of a student. We find that many generated transcriptions are plausible within the conventions of session music, e.g., many have correctly counted measures, and have a conventional AABB form with repetition and variation throughout. Many transcriptions suffer from poor harmonic implications, however, and lack coherence between sections. To test the general abilities of each model, we examine the response of the system to seed material that is musically valid, but outside the conventions of the training data (“nefarious testing”). For instance, we test whether the model can actually count time, or has learned to repeat and vary material. We see that the model can do these things, but only within a narrow context. Its knowledge lacks generality. To test how applicable these models are to composition, we use them in an iterative process of generation and curation, both in and out of the conventions of the training data. We have produced and posted online many examples of this, e.g., https://soundcloud.com/sturmen-1. We find particular interest in the “failures” of the models, which calls into question the compatibility of the training the systems to faithfully reproduce real transcriptions. We are expanding upon this work in several ways. We have generated many thousand transcriptions without curation and invited session music practitioners to play and comment, e.g., https://highnoongmt.wordpress. com/2016/09/ 12/folk-rnn-session-tunes-volume-1-of-10. This work is ongoing but interesting discussions have already occurred, e.g., https://thesession.org/discussions/39604. We are exploring different ways of sampling from the model output, and ways of involving a critic system in the model training. We also intend to train the models starting from their current weights on more specific subsets of the session practice, e.g., Irish jigs, or of other music traditions, e.g., gospel music.