# A Deeper Look at Sheet Music Composer Classification Using Self-Supervised Pretraining
###### Abstract  
This article studies a composer style classification task based on raw sheet music images. While previous works on composer recognition have relied exclusively on supervised learning, we explore the use of self-supervised pretraining methods that have been recently developed for natural language processing. We first convert sheet music images to sequences of musical words, train a language model on a large set of unlabeled musical \"sentences\", initialize a classifier with the pretrained language model weights, and then finetune the classifier on a small set of labeled data. We conduct extensive experiments on International Music Score Library Project (IMSLP) piano data using a range of modern language model architectures. We show that pretraining substantially improves classification performance and that Transformer-based architectures perform best. We also introduce two data augmentation strategies and present evidence that the model learns generalizable and semantically meaningful information. s +
Footnote †: journal: Applied sciences  
1  
e1e1e2021e387e38e4  
an N-gram model or Markov model is trained for each composer, and a query is classified by determining which model has the highest likelihood. The third approach is to feed the raw data into a neural network model and to learn meaningful feature representations directly from the data. While this approach is not new (e.g., [23]), most recent works on composer classification have adopted this approach by applying a convolutional neural network (CNN) to a piano roll-like representation of the data [24; 25; 26]. CNN models can be considered the current state-of-the-art in composer classification. This article differs from previous work in two significant ways. First, we study the composer classification task using raw sheet music images rather than symbolic music formats such as MIDI, MusicXML [27], **kern [28]**, or MEI [29]. This arguably makes the problem much more challenging, but it comes with one very significant advantage: data. Most previous works are highly constrained by the scarcity of symbolic music data, which leads to a reliance on manually designed features, overly simplistic model architectures, and/or a focus on narrowly defined classification tasks (e.g., [PERSON] vs. [PERSON] string quartet binary classification task [8; 10; 13; 26]). In contrast, there is an enormous quantity of sheet music image data available on IMSLP. Second, our approach seeks to leverage unlabeled data to improve classification performance. Rather than relying exclusively on supervised learning, we explore self-supervised pretraining approaches to take advantage of the vast quantities of data available online. In our experiments, we use all solo piano sheet music images in IMSLP. Our approach utilizes recent advances in transfer learning in natural language processing (NLP). Prior to 2018, transfer learning in NLP primarily consisted of using pretrained word embeddings such as word2 vec [30; 31] or GloVe [32]. Classifiers would use the pretrained word embeddings as the first layer in the model, and the weights in all remaining layers would be trained on task-specific labeled training data. In 2018, [PERSON] and [PERSON] proposed ULMFit [33], in which they first trained an LSTM-based language model on a large set of unlabeled data, and then added a classification head on top of the pretrained language model. While this paradigm was not novel (e.g., [34]), [PERSON] and [PERSON]'s study was significant in that they were able to achieve state-of-the-art results on a range of NLP tasks using this approach. Many other language model approaches were developed soon after that utilized the Transformer architecture [35], such as GPT-2 [36] and BERT [37]. These Transformer-based architectures have been the basis of many recent extensions and variants (e.g., [38; 39; 40]). Virtually all modern NLP systems utilize this type of self-supervised language model pretraining.