Aria-MIDI: A Dataset of Piano MIDI Files for Symbolic Music Modeling
Authors: Louis Bradshaw
Simon Colton
Venue: ICLR 2025 Poster (2025)
Keywords: music
symbolic music
piano transcription
dataset
midi
Primary area: datasets and benchmarks
TL;DR: We introduce a dataset of MIDI files comprising roughly 100,000 hours of transcribed piano recordings.

Abstract: We introduce an extensive new dataset of MIDI files, created by transcribing audio recordings of piano performances into their constituent notes. The data pipeline we use is multi-stage, employing a language model to autonomously crawl and score audio recordings from the internet based on their metadata, followed by a stage of pruning and segmentation using an audio classifier. The resulting dataset contains over one million distinct MIDI files, comprising roughly 100,000 hours of transcribed audio. We provide an in-depth analysis of our techniques, offering statistical insights, and investigate the content by extracting metadata tags, which we also provide. Dataset available at https://github.com/loubbrad/aria-midi.

Published as a conference paper at ICLR 2025
ARIA-MIDA A DATASET OF PIANO MIDI FILES FOR SYMBOLIC MUSIC MODELING
Louis Bradshaw, Simon Colton
Queen Mary University of London b.b.bbasahaw s. (oolton)@qmull
ABSTRACT
We introduce an extensive new dataset of MIDI files, created by transcribing audio recordings of piano performances into their constituent notes. The data pipeline we use is multi-stage, employing language model to autonomously crawl and score audio recordings from the internet based on their metadata, followed by a stage of pruning and segmentation using an audio classifier. The resulting dataset contains over one million distinet MIDI files, comprising roughly 100,000 hours of transcribed audio. We provide an in-depth analysis of our techniques, offering statistical insights, and investigate the content by extracting metadata tags, which we also provide. Dataset available at https//ggitub..coo/uubbraadd
INTRODUCTION
Central to the success of deep learning as a paradigm has been the datasets used to train neural networks. With the rapid technical advancements and ever-increasing availability of computational power, music has become a popular target for deep learning research, and deep learning in turn has had a notable impact on the study and creation of musical works (Briot et al., 2019). The progress of music-oriented deep learning depends heavily on access to diverse, well-structured datasets. Mu- sic is inherently structured and can be represented computationally 2016). In this work, we focus on symbolic representations of music, such as MIDI (Musical Instru- ment Digital Interface). which are widely used for encoding, analyzing, and facilitating the genera- tion of musical compositions by both humans and machines (Ji et al., 2023).
variety of forms (Wiggins,
In fields outside of computational music, the significance of comprehensive datasets is well estab- lished. For example, in computer vision, the ImageNet dataset (Deng et al., 2009) catalyzed research for almost a decade, providing both high-quality training data and robust benchmarks. Similarly, datasets such as Common Crawl (2024), C4 (Raffel et al., 2020), and the Pile (Gao et al., 2020) have been instrumental in advancing the field of natural language processing. These resources have en- abled the study of scaling-based based approaches for language modeling, enhanced available techniques, and provided foundation models for researchers with restricted resources (Zhou et al., 2024).
The situation for music-oriented deep learning research is mixed. While numerous publicly available audio-based datasets exist (Gemmeke et al., 2017; Hawthorne et al., 2018; Thickstun et al., 2016; Bertin-Mahieux al., 201 1), symbolic datasets, which represent music in formats such as MIDI, ABC, or MusicXML, are comparatively lacking in both quality and quamtity. The Lakh dataset (Raffel, 2016), comprising 176,581 MIDI files scraped from the internet, has been widely adopted in model training (Thickstun et al., 2023; Zeng et al., 2021) due to its scale. However, its files, pre- dominantly created through software sequencing or digital score conversion, often lack the nuances of expressive human performances, and vary significantly in quality. In contrast, the MAESTRO dataset (Hawthorne et al., 2018) offers high-quality Disklavier MIDI recordings from professional pianists, capturing the subtleties of human interpretation. However, its size and focus virtuosic classical piano performances limit its applicability across diverse musical genres and styles.
Underlying this is a common limitation: the manual transcription process creates bottlenecks for both the scale and quality. In recent years, researchers have turned to Automatic Music Transcription (AMT) (Benetos et al., 2019) to address these limitations, creating various large-scale symbolic datasets (Kong et al., 2020; Zhang et al., 2022: Edwards et al., 2023). Leading AMT techniques leverage neural networks to extract symbolic note-level information from audio (Sigtia al., 2016;

---

Published as a conference paper at ICLR 2025
Table 1: Comparison of various publicly available datasets of symbolic music.
Dataset
# Files
# Hours
Genre
Source
Multi-track
MAESTRO
1,276
199
Classical
Piano Competitions
No
Mutopia
1,862
69
Mixed
Lilypond
Yes
PiJAMA
2,777
223
Jazz
AMT
No
GiantMIDI
10,855
1,237
Classical
AMT
No
ATEPP
11,742
1,009
Classical
AMT
No
Lakh
176,581
9,567
Mixed
Web-scrape
Yes
Aria-MIDI 1,186,253
100,629
Mixed
AMT
No
https:/www.muttot www.ppotopiaprojcctorrgggg
2 Full dataset size, including corrupted files. The commonly used matched subset contains 45,129 files.
3 Reduces to 800.973 files and 66,650 hours after compositional deduplication described in Section 4.
Hawthorne et al., 2017; Kong et al., 2021), theoretically enabling symbolic datasets to match the scale found in other modalities. Nevertheless, several challenges persist:
Transcription Quality. Some musical formats, like solo-piano recordings, translate more accu- rately to MIDI than others. Additionally, training neural-AMT models relies on small number of specific high-quality datasets of aligned audio-MIDI, e.g., MAESTRO for solo-piano. This limita- tion restricts use cases outside the distribution of the training data, leading to degraded transcriptions of recordings different genres or with audio artifacts (Marták et al., 2024; Edwards et al., 2024).
Pre-processing. A dichotomy between quality and scale still presents itself. Current methods em- ployed for audio curation and pre-processing (e.g., selection, pruning, and segmentation) are insuf- ficient when applied to noisy and diverse audio corpora without human oversight. This is partially due to a lack of training data precisely labeled for the nuances this application. Datasets main- taining high-quality standards have utilized stage of machine-guided manual human verification to remove falsely identified audio (Zhang et al., 2022; Edwards et al., 2023), an approach that does not scale well.
In this work, we address these challenges, developing techniques for precise curation, pre- processing, and metadata attribution of publicly available piano recordings. We demonstrate that with strategic modifications to the data pipeline, AMT-based approaches can be effectively scaled, resulting in large, high-quality dataset of piano transcriptions. To this end, we leverage Aria-AMT (Bradshaw et al.), a piano transcription model designed to handle diverse timbres and recording qualities. Although this model important to our process, our primary focus on the techniques we develop and the analysis of the resulting dataset's 's content and quality.
1.1 CONTRIBUTIONS OF THIS PAPER
More specifically, our contributions are as follows:
1. We introduce scalable, language model-guided method for crawling and extracting meta- data from specific categories of videos. We analyze the effectiveness of this approach in the context of publicly available piano recordings.
2. We outline process for distilling audio source-separation model train classifier ca- pable of accurately identifying and segmenting diverse real-world piano recordings, which we open-source' This enabled an 8-fold improvement in identification of non-piano audio without human supervision when compared to previous work.
3. Using an existing taano-ttanscription model, we provide a new MIDI dataset of piano tran- scriptions, Aria-MIDI, one of the largest and cleanest to date.
We hope the dataset released alongside this work has positive impact the MIR research com- munity. We foresee several potential areas where may accelerate research. Firstly, pretrained generative models have had large impact on the textual and visual domains (Zhou et al., 2024).
2

---

Published as a conference paper at ICLR 2025
These models rely datasets typically in terabytes. Comprising approximately 20 gigabytes of MIDI files, Aria-MIDI isn't on this scale; however, may still be useful for research into pretrained music models. Secondly, piano audio classifier scores, which due to our training methods can act as a proxy for recording quality. This information is valuable for many MIR tasks (Choi al., 2017), as well as for making clean and compositionally deduplicated subsets.
releasing accurate compositional metadata for each file, as well as
1.2 RELATED WORK
The use of neural networks for automatic music transcription has its roots in the seminal work of Sigtia et al. (2015; 2016). This followed by various works experimenting with different approaches and neural architectures (Hawthorne et al., 2017; 2021; Yan et al., 2021; Toyama et al., 2023). The high-resolution piano transcription model introduced in Kong et al. (2021), trained using the MAESTRO (Hawthorne et al., 2018) and MAPS (Emiya et al., 2010) datasets, became the de facto benchmark for accuracy. More recently, AMT research has extended to other instruments (Riley et al., 2024) and multi-track transcription (Gardner et al., 2021; Chang et al., 2024), where it has seen success.
There are three predominant publicly available datasets of piano transcriptions, all of which utilized the transcription model introduced in Kong al. (2021). GiantMIDI (Kong et al., 2020) was the first, comprising transcriptions of piano recordings matching names of musical works taken from the IMSLP (2006). From 443 initial recordings, 10,855 were identified by a model trained to detect solo-piano recordings. The ATEPP dataset (Zhang et al., 2022) took different approach than GiantMIDI, focusing on repeat performances of standard classical piano repertoire, and using text- based techniques to determine opus and piece numbers. PiJAMA (Edwards et al., 2023), dataset of jazz piano transcriptions, spans 120 different pianists across 244 recorded albums. Recordings were curated by matching tracks from albums performed by manually curated list of pianists.
All three datasets utilized YouTube to match musical metadata with audio recordings and employed audio-based classifiers, trained using MAESTRO and AudioSet. identify piano recordings. For ATEPP and PiJAMA, these classifiers were also used to remove applause and speech. The level of human intervention varied across datasets: GiantMIDI relied solely automated processes, while ATEPP and PiJAMA incorporated manual checks. Table I presents a comparison of these datasets in context.
2 METHODOLOGY
In this section, we describe the methodology used to compile our dataset of MIDI files. Our approach consists of three distinct stages. First, we assemble comparatively large corpus of candidate piano recordings using low-overhead, text-based methods. Next, we employ audio-based techniques to refine our initial corpus through pruming and pre- pre-processing. Finally, conduct a computationally intensive stage of transcription and metadata extraction. This multi-stage approach allows us to efficiently process large volume of data while ensuring high-quality data in our final dataset.
2.1 CRAWLING
A common theme in previous work has been to compile candidate recordings by first obtaining corpus of metadata (e.g., composers, performers, album titles) via various means, and then using the APIs provided by Spotify² and YouTube to match piece titles to corresponding videos on YouTube. We take different approach: Our method begins with a small collection of manually curated seed videos and uses YouTube's 's API to crawl related content. The crawling priority is determined by a language model, which performs two tasks: 1) parsing the title and description of each video, and 2) scoring the likelihood of the video containing solo-piano content on scale of 1-5. More specifically, starting from fifty solo-piano seed videos which span a variety of genres and styles, we follow two-step procedure which we cycle through repeatedly:
3

---

Published as a conference paper at ICLR 2025
BB
Piano Spectrogranh
Piano RMS-Finergy
LABEL
Audio Segment
Other Spoctrogran
Other RMS-Finerg
Figure I: A visual representation of the pseudo-labeling process applied five-second excerpt from a piano concerto. As the non-piano component has a contiguous region above the energy threshold (dotted red line), the full audio segment is labeled non-piano.
1. For each unscored video, we provide system prompt to a language model along with the video's YouTube title and description. The model is tasked with assigning a score from one five, indicating the likelihood that the video contains solo-piano performance.
2. In order of priority determined by the score, we use the YouTube API to fetch related video URLs, titles, and descriptions.
By taking advantage of the related videos endpoint of the YouTube API, outsource the majority of the crawling process to YouTube's 's own recommendation algorithm. We used the 70B parameter version of Llama 3.1 (Dubey et al., 2024) for the language model, observing that smaller models made obvious mistakes more frequently. The system prompt we used can be found in Appendix A. Overall, found this process to be effective. Although initially this procedure tended to overrepresent recordings of well-known classical pieces, as these became less available later in the process, piano recordings representing diverse set of musical styles and genres were crawled.
2.2 AUDIO CLASSIFICATION AND SEGMENTATION
As we analyze in-depth in Section 3, relying solely on the score attributed to each recording during the crawling process results in an unacceptably high rate of misclassifications Following other work (Kong et al., 2020; Zhang et al., 2022), we address this by using an audio classification model (Dieleman and Schrauwen, 2014) in the next stage of our pipeline. We identified the following problematic situations which we aimed to mitigate with audio classifier:
Misclassifications due to logical mistakes by the language model or misleading/ambiiuuus YouTube data: The classifier identifies and removes such recordings while allowing to retain those with some ambiguity, which would otherwise have to be pruned.
Undesirable acoustic qualities in positively classified recordings: Despite positive classifi- cation by the language model, recordings can be inappropriate for transcription for vari- ety of reasons including incorrect instrumentation (e.g., harpsichord, organ, electric piano), low audio quality, or the presence of additional instruments. To mitigate this, we include representative examples from these categorics in the training data for our classifier.
Non-piano content in high-quality piano recordings: Many otherwise high-quality piano recordings contain segments of non-piano content, such as applause, commentary, or ex- tended periods of silence. Using the algorithm describe in Section 2.2.2, we adapt the audio classifier to segment recordings into contiguous regions of solo-piano performance, removing unwanted content.
A primary concern when building audio classifier is the quality, diversity, and accuracy of the labels used for training. Initial investigations revealed that relying on well-known datasets such as MAESTRO (Hawthorne et al., 2018) and AudioSet (Gemmeke et al., 2017) was insufficient, we display in Table 4. In an effort to achieve classification accuracy approximating human labels, we curated mixed training dataset, representative of our corpus of crawled recordings. We used novel approach, leveraging an audio cource-separation model to accurately generate pseudo-labels for the unlabeled parts of the training corpus.
4

---

Published as a conference paper at ICLR 2025
Table Overview of the supervised and pseudo-labeled audio corpora used to train the piano audio classifier. Prop. denotes the proportion of audio in each component labeled as solo piano. Notably, when pseudo-labeled using source separation, the raw audio from which the GiantMIDI dataset was transcribed contains 87.35% solo piano labels.
Component
Length (h)
Weight (%)
Pseudo lab.
Prop.
(%)
dB
mmin
(s)
GiantMIDI
1040
43.24
True
87.35
-25dB
1.5
Score-4
676
28. 11
True
44.81
-22dB
1.5
MAESTRO
198
8.23
False
99.62
N/A
N/A
Synthetic Data
143
5,95
False
98.34
N/A
N/A
Jazz Trio Database
139
5.78
True
3.61
-28dB
1.0
Piano Other
75
3.12
False
99.22
N/A
N/A
Non-Piano Other
2.95
False
0.00
N/A
N/A
Symphonies
40
1,66
False
0.00
N/A
N/A
Piano Concertos
0.96
True
14.98
-28dB
1.0
Total
2405
100.0
Given an audio file, we apply the MVSep Piano source-separation model (Uhlich et al., 2024; Fab- bro, 2024; Solovyev al., 2023) to decompose it into its constituent parts, isolating the piano component from the remaining audio (other). For each five-second clip, we compute separate spectrograms for both the piano and other components after resampling them to 22,050 Hz. Each spectrogram is computed with 2048 frequency bins, a frame length of 2048, and hop length of 512. The RMS energy then computed for each frame and converted to the dBFS scale (Zölzer, 2022). Given parameters dBmin and lmin- we label five-second clip non-piano if the other component contains a contiguous region longer than mii at an energy level exceeding dBmin- We also label a clip non-piano if the piano component has a contiguous silent region (below -20 dB) lasting more than four seconds. In all other cases, the clip is labeled as piano. Figure illustrates this process.
We applied this labeling procedure various collections of publicly available audio files, displayed in Table 2, the main constituent being 10,000 YouTTube videos from our corpus which were assigned score of four by the language model. We also used the GiantMIDI audio files, the Jazz Trio Database (Cheston et al., 2024), and smaller collections of piano and non-piano recordings which we curated manually. Including the pseudo-labeled audio allows us to distill (Hinton et al., 2015) the source-separaiee separast model, bypassing the high computational cost associated with applying source separation to our entire inference corpus, which we estimate at about 5,000 A A100 hours".
2.2.1 TRAINING
For our solo-piano classifier, we chose a CNN-based architecture (LeCun et al., 2015) with five con- volutional layers followed by two dense layers and single output neuron. The input the classifier consists of mel-spectrograms calculated from five-second audio clips. We used sample rate of 22,050 Hz, 2048 spectrogram frequency bins, 256 mel bins, and a hop length of 220 (corresponding to 10ms hops). We trained the model for ten epochs using the Adam W optimizer (Loshchilov and Hutter, 2019) with 31, .2 0.9. 0.95, le-6 and an weight decay 0.01. linear learning rate scheduler was used, decaying to 10% of the initial learning rate after warmup over the first 500 optimizer steps.
One consequence of training with pscudo-labels obtained using source separation having to use relatively sensitive energy thresholds in order to correctly label training examples with a quiet but notable non-piano component. These thresholds occasionally result in incorrect training la- bels for solo-piano recordings with significant but acceptable background audio artifacts like noise, distortion, and reverb. To mitigate this, trained with the corresponding audio augmentation in approximately 10% of batches, as well as randomly applying pitch shifting and bandpass filters. We also included labeled examples, representative of such piano recordings, as part of training data.
In comparison, classification of 100,000 hours of audio using model only took 20 A100 hours, I/O heing the main bottle-neck.
5

---

Published as a conference paper at ICLR 2025
2.2.2 INFERENCE
As well as per-file classification, we also use our classification model to segment audio recordings into their standalone components of contiguous piano performance. To do this, we employ a sliding- window based technique adapted from standard approaches (Keogh et al., 2004), aimed at accurately removing non non-piano content whilst being robust to short-lived classification mistakes.
Given an audio recording, we score each five-second interval, sampled with a one-second stride, by passing the inputs through our model. We classify region (n, m + 5). m 2 n + d, as non- piano if and only if all segments starting between n and m are scored below A. The parameters d and À control the sensitivity and minimum length of non-piano segments, which we set to 3 and 0.5 respectively. After excluding all non-piano segments, we classify the remaining contiguous segments as piano if they are longer than 45 seconds. Finally, we discard piano segments with an average score lower than 0.7. Our choice of algorithm and hyperparameters was motivated to reduce the chance of solo-piano segment being prematurely interrupted due to instability in scoring. Since our classifier designates intervals that are mostly silent as non-piano, this approach also separates segments of piano performances that are separated by at least d seconds of silence. In Section 3, we investigate the accuracy of both classification and segmentation of our proposed approach.
2.3 TRANSCRIPTION
We used a Whisper-based model, Aria-AMT (Bradshaw et al.), to transcribe the segmented audio recordings into MIDI files. This choice was informed by the model's robustness in transcribing audio from a diverse set of recording environments, compared to models used in previous work (Kong et al., 2020; Zhang et al., 2022) (See Appendix A.3). Transcription of the 100,629 hours of audio took 765 hours using an NVIDIA H100 GPU with a batch size of 128, representing an inference speed of roughly 131x real-time, transcribing approximately 2327 notes per second.
2.4 METADATA EXTRACTION
Access to per-file metadata labels enables flexible dataset splits for generative and MIR tasks. A central concern of ours was entity resolution (ER) (Christen, 2011), i.e., identifying the composi- tional source of each recording and using this information to mitigate overrepresentation of popular pieces
Inspired by our crawling methodology, we applied similar approach to metadata extraction. Using Llama 3.1 (70B), we processed YouTube titles and descriptions for files that passed language model and audio detection filters. The prompt (see Appendix A.2) extracted composer, opus numbers (c.g.. Op., BWV, K., D.). piece identifiers, performer, genre, and form labels when such information was present in the text.. These metadata labels facilitate control over compositional duplication and provide supervised labels for MIR. We analyze their accuracy and distribution in Sections and 4.
3 METHODOLOGICAL ANALYSIS
In this section, we evaluate the effectiveness of the components in our data pipeline. Where applica- ble, we compare our methods to those used in previous work, in particular the GiantMIDI, ATEPP, and PiJAMA datasets. For baselines and to determine ground truth, we relied on human labels obtained from two musically trained pianists familiar with popular classical and jazz repertoire.
Language Model Classification. We first analyze the ability of a language model to correctly clas- sify a video as solo-piano according to its YouTube title and description. In our experiment we chose random sample of 250 videos from those crawled, and calculated the accuracy of the labels provided by different language models, judged relative to the ground truth in the audio content. We also asked human participants to label the videos according to the same prompt given to the lan- guage models. Classification precision, recall, and F1 scores can be seen in Table 3. In comparison to derived labels, language models perform well at this text-based classification task. We attribute this to the depth of knowledge of different composers, performances, and pieces, which the
httpss//gihbb.co//llcteemiiiii com///tctthheeaaaaiaaan 'oor example, moonlight appears in 6,819 titles, likely referring Beethoven's Moonlight Sonata.

---

Published as a conference paper at ICLR 2025
Table Classification precision, recall, and Fl-scores for different models in the Llama 3.1 family, as well as human labels, across various score classification thresholds. Results suggest that Llama 3.1 70B with a score threshold of 4 provides a good balance between inference cost and accuracy.
Score
Score 2
Score 2
Model
(%%
R (%)
F1 (%)
(%)
(%)
F1 (%)
P (%)
R (%)
(%)
Ilama 3.1 8B
73.08
81.43
77.03
82.26
72.86
77.27
86.36
27.14
41.30
Ilama 3.1 70B
64.76
97.14
77.71
70.83
97.14
81.93
84.51
85.71
85.11
Llama 3.1 405B
77.01
95.71
85.35
80.49
94.29
86.84
94.44
72.86
82.26
Human labels
73.63
95.71
83.23
83.56
87.14
85.31
85.71
25.71
39,56
language models have access to. Despite this, there remains a clear discrepancy between the audio ground truth and the labels obtained from YouTube titles and descriptions alone.
Audio Segmentation. We evaluate the performance of our audio classification model in identifying and segmenting solo-piano content within audio recordings. For comparison, we used MVSep di- rectly to obtain binary labels, applying the same inference procedure described in Section 2.2.2. For ablation, we trained a model without the pseudo-labeled training data listed in Table 2. To mimic classifiers used for segmentation in other work, notably for the GiantMIDI, ATEPP, and PiJAMA datasets, we include various noise, applause, and speech from AudioSet (Gemmeke et al., 2017) as negative training examples for our ablation model.
For this analysis, a random sample of 250 audio recordings assigned language model scores greater than equal to 3 was selected, excluding those used during training. To establish ground truth, two participants were tasked with segmenting recordings into regions of solo-piano content and as- signing files into one of three categories: Not solo-piano, solo-piano with significant audio artifacts, or solo-piano with good to pristine recording quality. Human-labeled segments were post-processed in accordance with our inference algorithm: Non-piano segments shorter than eight seconds were ignored, and minimum length of 45 seconds was imposed on piano segments. Segmentation ac- curacy results can be seen Table 4. While the ablation model achieves accurate segmentation accuracy, being less likely to interrupt piano segments by misclassifying occasional noisy periods of extreme piano audio as non-piano content, mislabels non-piano audio as piano eight times more frequently than the proposed approach in absolute terms.
We next evaluated our model's classification performance. To assess this on a per-file basis, we imposed minimum thresholds the average audio score for predicted piano segments, negatively classifying files with no predicted piano segments after filtering. Table 5 reports the accuracy of this approach in identifying non-piano recordings well as solo-piano recordings with significant audio artifacts in our evaluation dataset. Additionally, we analyzed the audio files that constitute Giant- MIDI, ATEPP, and PiJAMA. Our human participants manually categorized files in these datasets that fell below an empirically determined average score threshold of 0.7, indicating issues with recording quality content. The resulting distributions of these categorizations are shown in Figure 2.
In both tasks, our approach performs well. For segmentation with A=0.5, we achieve a 96.38% overlap with the ground truth for high-quality piano recordings, while removing 98.83% of non- piano audio across the evaluation corpus. When applying a segment average-score threshold of T=0.7, we remove 100% of non- non-piano files on per-file basis while retaining 95.28% of high-qualit, piano files. Furthermore, increasing T to 0.9 removes most low-quality piano audio files, enabling us to curate a clean dataset split which we also provide.
Metadata extraction. To evaluate our approach to metadata extraction, we selected a random sam- ple of 200 files and cross-referenced the metadata labels assigned by the language model with the corresponding YouTube titles and descriptions. For each file and metadata category, manually checked for incorrect labels (e.g., misattributions) and missing labels where relevant information was present in the raw text. This process provides accuracy estimates for the metadata labels in the final dataset. Notably, the language model sometimes generated accurate labels that were not present in the raw text. The results of this evaluation shown in Table 6.

---

Published as a conference paper at ICLR 2025
Table Segmentation accuracy and overlap ratios for different techniques and hyperparameters. We consider predicted segment correct if its beginning and end match the reference within tolerances of +2 seconds and #5 seconds, respectively. Each reference segment is matched at most one predicted segment. Overlap ratios are calculated separately for piano and non-piano audio, each as ratio of the duration of correctly identified audio to total duration of respective ground truth audio type. dB and À denote the sensitivity to non-piano content as described in Section 2.2.
Segmentation Accuracy
Segment Overlap
Technique
P ((%) (%)
R
F1 (%)
Piano (%)
Non-Piano (%
Full corpus
MVSep, dB 22dB
65.62
70.47
67.96
91.96
97.22
MVSep, dBmin=-25dl
58.28
63.76
60.90
88.74
98.18
MVSep, -
49.38
53.69
51.45
82.23
98.66
Proposed, A=0.5
71.97
75.84
73.86
94.22
98.83
Proposed, A=0,6
70.70
74.50
72.55
92.67
98.89
Proposed, A=0.7
68.39
71.14
69.74
91.04
99.10
Ablation, X=0.5
71.18
81.21
75.86
97.05
91.10
All solo-lpppii sollo-piano recordings
MVSep, = m
68.18
70.47
69.31
91.96
89.50
MVSep, dBmin=
59.75
63.76
61.69
88.74
93.43
MVSep, -
50.00
53.69
51.78
82.23
94.12
Proposed, A=0.5
72.44
75.84
74.10
94.22
92.73
Proposed, A=066
71.15
74.50
72.79
92.67
93.18
Proposed, A=0.7
68.83
71.14
69.97
91.04
94.66
Ablation, A=0.5
77.56
81.21
79.34
97.05
73.33
Quality solo-piano recordings
MVSep, 22dB
70.00
75.97
72.86
94.71
85.13
MVSep. -25dB
61.64
69.77
65.45
92.06
87.04
MVSep, dB 228dB
50.33
58.91
54.29
87.41
88.22
Proposed, A=0.5
75.36
80.62
77.90
96.38
84.80
Proposed, A=0.6
73.91
79.07
76.40
95.33
86.13
Proposed, A=0.7
71.53
75.97
73.68
93.92
88.29
Ablation, A=055
82.09
85.27
83.65
97.15
83.13
Table 5: Classification performance for different segment average score thresholds, evaluated on our human-labeled dataset. Segments were calculated using X=0.5. All solo-piano refers to the classification performance for identifying files containing solo piano segments, regardless of audio artifacts. Quality solo-piano refers to identifying only recordings with good to pristine audio quality. FP False Positives.
All solo-piano
Quality solo-piano
Threshold
(%)
R (%)
F1
FP
P (%)
R (%)
F1 (%)
FP
20.50
99.28
93.24
96.17 17
88.49
96.85
92.48
16
T 20660
99.27
91.89
95.44
89.05
96.06
92.42
15
T 20.70
100.00
89.86
94.66
90.98
95.28
93.08
12
T 2.800
100.00
85.14
91.97
0
90.48
89.76
90.12
12
T 20.90
100.00
75.68
86.15
0
95.54
84.25
89.54
T 20.95
100.00
61.49
76.15
97.80
70.08
81.65
2
8

---

Published as a conference paper at ICLR 2025
Non Diano
Solo-piano /w audio artifacts
Solo- salo-piano//aa /wo audio artifacts)
800
100
20
600
75
15
Feequency
100
200
25
5
GiantMIDI
ATEPP
PiJAMA
Figure 2: Distribution of files with average audio classifier scores < 0.7 across datasets. Files were manually categorized. The differences in distribution between datasets may be attributed to human- in-the-loop pruning ATEPP and PiJAM,, which is absent in GiantMIDI..
Table 6: Analysis metadata presence and accuracy across different attributes. For each attribute, presence indicates the percentage of files with assigned metadata, accuracy shows the percentage of correct labels among present metadata, and missed labels represents the percentage of files where metadata omitted despite being inferrable from YouTTube titles and descriptions. Accuracy was verified following the criteria specified in the system prompt (see Appendix A.2).
Attribute
Presence (%)
Accuracy (%)
Missed Labels ( (%)
Composer
71.0
99.3
2.7
Performer
62.0
99.2
0.8
Opus Number
32.0
100.0
1.5
Piece Number
22.0
93.2
4.3
Key Signature
23.0
97.8
0.0
Genre
86.5
94.2
0.6
Music Period
63.0
92.9
12.5
DATASET STATISTICS
In this section, we present statistics on our methodology and the resulting dataset of MIDI files. Overall, when executing our data pipeline, we collected YouTube data for 3,290,453 videos, from which we further processed 1,713,650 using our audio classifier. We then transcribed over million audio segments into approximately 100,000 hours of transcribed solo-piano music. We present breakdown of scores ascribed during each stage of processing in Figure 3. Taken together with the experiments Section we conclude that the techniques we have developed work well at scale.
4
Classifirr score
0.8
0.6
7.11%
0.4
26
20.99
2
20
40
80
100
Datasel prup. (%)
(a) Language model
(b) Audio-classifier
Figure 3: Score breakdowns for different components of our data pipeline. Figure (a) shows the dis- tribution language model scores during crawling. Figure (b) illustrates the cumulative distribution of audio classifier scores for recordings with language model score of least three.

---

Published as a conference paper at ICLR 2025
10a
Frequency (log scale)
10200
11473
10f
5006
5300
1900
7000
1190
600
1200
1125
2570
10a
Chap
Bach Beetho
Ge GerATi
Debut TATTI Rachmia
Mend TREEE
FORD
Uan Shop Se
MOSS EICCIISO
pla aalaaoo
Tataa
Sari
M
Figure 4: Number of transcriptions in Aria-MIDI attributed the top 50 composers (log scale).
Genre
Form
Musical Perlod
Performer
1.0
Relative frequency
0.8
0.6
0.2
Classical
Soundtrack
Pap
Jazz
Rock
Sonata
Etude
Prelude
Waltz
Noctumee
Romantic
Classical
Contemporary
Hodern
Baroque
Richter
Yiruma
Ashkenazy
Gould
Horawitz
Figure 5: Relative frequency distribution of metadata across categories, normalized the most
frequent one. Common surnames in the performer category (e.g.. Lee, Kim) are not displayed.
Moreover, extrapolating from the results in Table 5, observe that the top-scoring 35,000 hours of MIDI files likely contain few transcriptions of non-solo-pian content.
To address compositional duplicates, we analyze metadata tags in three categories: composer, opus number, and piece number. To obtain a dataset split minimizing compositional duplicates within the text-based metadata constraints, we remove files that either match on all three tags (composer, opus number, and piece number) or match on both composer and opus number in cases where piece number tags are absent. For composers who appear more than 250 times across the dataset, we also prune all additional files that lack opus number and piece number tags. Overall, we identified 23,28 unique metadata triples, and after removing compositional duplicates using this procedure, 800,973 files remained. Figure 4 shows the frequency of performances of works by different composers in the complete dataset, illustrating their relative popularity.
Lastly, Figure 5 shows the distribution of metadata for other categories over the entire collection of MIDI files, without deduplication. Overwhelmingly. transcriptions of classical piano performances dominate; however, when accounting for the total size, many other genres are well represented.
5 CONCLUSION
We have introduced a new dataset of MIDI files, created by transcribing piano performances pub- licly accessible on the internet. In this paper, we provide an analysis of the components in our data pipeline and find them to be well-suited for our purposes. Going forward, we several areas for future work: Primarily, extending our approach to other instruments such as guitar, as well as the inntiiinsttumment case, could be approachable via variations of the source-separation based ap- proaches to audio pre-procssiing we have outlined. Secondly, further study into metadata attribution using language models, especially targeting improvements in compositional entity recognition.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This research utilized the Al Industrial Convergence Cluster supported by the Ministry of Science and ICT of Korea, and Gwangju Metropolitan City. We also thank Roman Sol for making the MVSep piano source separation model available to us, and Alex Spangher for his assistance with audio processing.
COPYRIGHT DISCLAIMER
The use of copyrighted works, and derivatives thereof such as MIDI transcriptions. for machine learning is complex issue. To mitigate potential harms, we do not release audio files or raw metadata under direct copyright. We distribute this dataset under CC-BY-NC-SA license (Creative Commons).
REFERENCES
Emmanouil Benetos, Simon Dixon, Zhiyao Duan, and Sebastian Ewert. Automatic music transcrip- tion: An overview. IEEE Signal Processing Magazine, 36(1):20-38) 30, 2019. doi: 10.1109/MSP. 2018.2869928.
Thierry Bertin-Mahieux, Daniel P.W. Ellis, Brian Whitman, and Paul Lamere. The million song dataset. In Proceedings of the 12th International Conference on Music Information Retrieval (ISMIR 2011), 2011.
Louis Bradshaw, Alexander Spangher, Xavier Riley, Zixun Guo, Stella Biderman, and Simon Colton. Aria-amt: Efficient and robust seq-to-seg automatic piano transcription. https: //github. cww/co////////. httm//eeeterrr/rrreeee. amt. Accessed: 2024.
Jean-Pierre Briot, Gaëtan Hadjeres, and François-David Pachet. Deep learning techniques for music generation survey. arXiv preprint arXiv:1709.01626. 2019.
Sungkyun Chang, Emmanouil Benetos, Holger Kirchhoff, and Simon Dixon. Yourmt3+: Multi- instrument music transcription with enhanced transformer architectures and cross-dataset stem augmentation. arXiv preprint arXivv2407.04822 2024.
Huw Cheston, Joshua L Schlichting. Ian Cross, and Peter M C Harrison. Jazz trio database: Au- tomated annotation of jazz piano trio recordings processed using audio source separation. 2024. doi: 10.5334/issir..1e URL https:://olt orgoor/////// 10. 5334/tismir 186.
Keunwoo Choi, György Fazekas, Kyunghyun Cho, and Mark Sandler. tutorial on deep learning for music information retrieval. arXiv preprint arXiv: 04396, 2017.
Peter Christen. survey of indexing techniques for scalable record linkage and deduplication. IEEE transactions on knowledge and data engineering. 1435511537 2011.
Common Crawl, 2024. URL CRRLhttps://cmooccaall..
Creative Commons. 4.0). URL http:://ccraateeecoooooos///////
Attrbution onnCCmmecccallSaaaaln 4.0 International (CC BY-NC-SA 0//.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale seale hi- erarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pages 248-255. leee, 2009.
Sander Dieleman and Benjamin Schrauwen. End-to-end learning for music audio. 2014 IEEE International Conference Acousties, Speech and Signal Processing (ICASSP), pages 6964- 6968. IEEF, 2014.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle. Aiesha Leuman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 2024.
11

---

Published as a conference paper at ICLR 2025
Drew Edwards, Simon Dixon, and Emmanouil Benetos. Pijama: Piano jazz with automatic midi annotations. Transactions of the International Society for Music Information Retrieval, 2023.
Drew Edwards, Simon Dixon, Emmanouil Benetos, Akira Maezawa, and Yuta Kusaka. A data- driven analysis of robust automatic piano transcription. IEEE Signal Processing Letters, 2024.
Valentin Emiya, Nancy Bertin, Bertrand David, and Roland Badeau. MAPS A piano database for multipitch estimation and automatic transcription of music. Research report, 2010. URL https: bttpps:/imme //irria. hal www.:ssceeeeeeeeeeeeeeee hatpe/iinraii0044111
Uhlich S. Lai C.-H. Choi W. Martinez-Ramírer M. Liao W. Gadelha I. Ramos G. Hsu E. Rodrigues H. Stöter F.-R. Défossez A. Luo Y. Yu J. Chakraborty D. Mohanty S. Solovyev R. Stempkovskiy A. Habruseva T. Goswami N. Harada T. Kim M. Lee J. H. Dong Y. Zhang X. Liu Mitsufuji Y Fabbro, G. The sound demixing challenge 2023-music demixing track. Trans. Int. Soc. Music.
Inf. Reir, 7(1):63-84, 2024.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.0000 10100027.0.00... 2020.
Josh Gardner, Ian Simon, Ethan Manilow, Curtis Hawthorne, and Jesse Engel. Mt3: Multi-task multitrack music transcription. arXiv preprint arXiv:21 2021.
Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: ontology and human-labeled dataset for audio events. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 776-780. IEEE, 2017.
Curtis Hawthorne, Erich Elsen, Jialin Song, Adam Roberts, Ian Simon, Colin Raffel, Jesse Engel, Sageev Oore, and Douglas Eck. Onsets and frames: Dual-objective piano transcription. arXiv preprint arXiv: 153, 2017.
Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dicleman, Erich Elsen, Jesse Engel. and Douglas Eck. Enabling factorized piano music modeling and generation with the maestro dataset. arXiv preprint w:Xh::110.122444 2018.
Curtis Hawthorne, Ian Simon, Rigel Swavely, Ethan Manilow, Jesse Engel. Sequence-to- sequence piano transcription with transformers. arXiv preprini arXiv:2107.09142 2021.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv preprint arXiv:1503.02533 2015.
IMSLP. IMSLP/Petrucci music library, 2006. URL https://mmspp.....
Shulei Ji, Xinyu Yang, and Jing Luo. survey on deep learning for symbolic music generation: Representations, algorithms, evaluations, and challenges. ACM Computing Surveys, 56(1):1-39, 2023.
Eamonn Keogh, Selina Chu, David Hart, and Michael Pazzani. Segmenting time series: A survey and novel approach. In Data mining in time series databases, pages 1-21, World Scientific, 2004.
Qiuqiang Kong, Bochen Li, Jitong Chen, and Yuxuan Wang. Giantmidi-piamn large-scale midi dataset for classical piano music. arXiv preprint arXiv:2010.07061, 2020..
Qiuqiang Kong, Bochen Li, Xuchen Song, Yuan Wan, and Yuxuan Wang. High-resolution piano transcription with pedals by regressing onset and offset times. IEEF/ACM Transactions on Audio, Speech, and Language Processing, 29:3707-3717, 2021.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436444 444, 2015.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv:1711005 2019.
arXiv preprint
12

---

Published as a conference paper at ICLR 2025
Lukáš Samuel Marták, Patricia Hu, and Gerhard Widmer. Quantifying the corpus bias problem in automatic music transcription systems. arXiv preprint arXiv:2408.04737, 2024.
Colin Raffel. Learning-Basee Methods for Comparing Sequences, with Applications to Audio-to-
MIDI Alignment and Matching. PhD thesis, Columbia University, 2016.
Colin Raffel, Brian Mcfee, Eric Humphrey, Justin Salamon, Oriol Nieto, Dawen Liang, and Daniel Ellis. mir_eval: A transparent implementation of common mir metrics. In ISMIR, 2014.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1-67, 2020.
Xavier Riley, Zixun Guo, Drew Edwards, and Simon Dixon. Gaps: A large and diverse classical guitar dataset and benchmark transcription model. arXiv preprint arXiv:2408 08653, 2024.
Siddharth Sigtia, Emmanouil Benetos, Nicolas Boulanger- Lewandowski, Tillman Weyde, Artur d' Avila Garcez, and Simon Dixon. A hybrid recurrent neural network for music transcription. In 2015 IEEE international conference acoustics, speech and signal processing (ICASSP), pages 2061-2065. IEEE, 2015.
Siddharth Sigtia, Emmanouil Benetos, and Simon Dixon. An end-to-end neural network for poly- phonic piano music transcription. IEEE/ACM Transactions Audio, Speech, and Language Processing, 24(5):927-939, 2016.
Roman Solovyev, Alexander Stempkovskiy, and Tatiana Habruseva. Benchmarks and leaderboards for sound demixing tasks, 2023.
John Thickstun, Zaid Harchaoui, and Sham Kakade. Learning features of music from scratch. arXiv preprint 1.09927, 2016.
John Thickstun, David Hall, Chris Donahue, and Percy Liang. Anticipatory music transformer.
arXiv preprint 23.6:v2306008622 2023.
Keisuke Toyama, Taketo Akama, Yukara lkemiya, Yuhta Takida, Wei-Hsiang Liao, and Yuki Mitsu- fuji. Automatic piano transcription with hierarchical frequency-tinn transformer. arXiv preprint arXiv:2307.04305. 2023.
Stefan Uhlich, Giorgio Fabbro, Masato Hirano, Shusuke Takahashi, Gordon Wichern, Jonathan Roux, Dipam Chakraborty, Sharada Mohanty, Kai Li, Yi Luo, al. The sound demixing challenge 2023-cinematic demixing track. Trans. Int. Soc. Music. Inf. Retr., 7(1):44-6) 62, 2024.
Geraint A Wiggins. Computer representation of music in the research environment. In Modern Methods for Musicology, pages 7-22. Routledge, 2016.
Yujia Yan, Frank Cwitkowitz, and Zhiyao Duan. Skipping the frame-level: Event-based piano transcription with neural semi-crfs. Advances Neural Information Processing Systems, 34: 00583-205990 2021.
Mingliang Zeng. Xu Tan, Rui Wang, Zeqian Ju, Tao Qin, and Tie-Yan Liu. Musicbert: Symbolic music understanding with large-scale pre-training. arXiv preprint arXiv:2106.056.0 2021.
Huan Zhang, Jingjing Tang, Syed Rifat Mahmud Rafee, and Simon Dixon György Fazekas. Atepp: A dataset of automatically transcribed expressive piano performance. In ISMIR 2022 Hybrid Conference, 2022.
Ce Zhou, Qian Li, Chen Li. Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, et al. comprehensive survey on pretrained foundation models: history from bert to chatgpt. International Journal of Machine Learning and Cybernetics, pages 1-65, 2024.
Udo Zölzer. Digital audio signal processing. John Wiley & Sons, 2022.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 CRAWLING SYSTEM PROMPT
Analyze the YouTube video title and description to determine if it's likely solo piano performance. Consider the folloving:
Is the content music-related?
2. Are there explicit mentions of solo piano or pianist names?
3. Does it mention other instruments, vocalists, or non non-musical elements?
4. Is it educational video (tutorial, lesson) rather than
performance?
5. a piece name is provided, is it typically for solo piano?
Pay special attention to these factors, which suggest the content is NOT pure solo piano performance:
11
12 Presence of other instruments or vocalists
13 Educational content (lessons, tutorials)
Non Non-piano keyboard instruments (e.g., organ, harpsichord) Sigmificant narration or spoken content
16
Orchestral accompaniment (e.g., piano concertos)
17 Audio content beyond solo piano
Repetitive tracks (e.g., loop videos)
20
The presence of any these elements should generally result a lower rating.
Assign score from 0-5 where:
24
Certainly a solo piano performance only
25
Clear indication of solo pianist performing
No signs of additional instruments, vocals, or non non-performance elements
27
28 Very likely solo piano performance, but not entirely certain
20
Strong indications of solo piano, but some minor ambiguity No clear signs of additional elements, but not explicitly ruled
32
Possibly a solo piano performance, but with significant uncertainty
33
Some indications of solo piano, but also hints of potential
additional elements
Gould be
piece with minimal additional content
35
36
= Likely includes elements other than solo piano
Clear indications of additional instrumente, educational content, pon-perfornance elements Still primarily piano maano-oouuee. but definitely not pure solo performance
38
30
= Mostly not solo piano performance Significant presence of other instruments, vocals, or non-musical content
42 Piano may be present but is not the sole or main focus
Definitely
solo piano performance not piano-related
65 No indication of solo piano content
56 Completely unrelated to piano performances
Examples:
14

---

Published as a conference paper at ICLR 2025
"Chopin Nocturne 0p. No. Arthur Rubinstein" " =>
"The Art Fugue Glenn Gould (Piano) =>
"Bohemian Rhapsody Piano Cover with Sheet Music" => 4
"Beethoven Ode To Joy I VERY EASY Piano Tutorial" => 3
"Mozart Piano Concerto No. 21 London Symphony Drcheatra" => "Top 10 Guitar Solos All Time" =>
56
Think step by step concisely, and then provide your score as a JSON string: {"score": X}
A.2 METADATA EXTRACTION SYSTEM PROMPT
Analyze the YouTube video title and description provided within XML tags. it contains information about a solo -piano performance, extract the following metadata and provide it as a JSON string:
composer: Last name of the composer, if applicable (string, omit if not present
uncertain)
opus: Opus number (e.g., Op., BWV, K., D.), if applicable (integer, omit if not present or uncertain)
piece_number number: Number or identifier within the opus, if applicable integer, omit if
present
uncertain)
genre: Primary genre of the piece (string: "classical", "jazz", pop", "blues", "ragtime", "atonal", "rock", "soundtrack", ambient", "folk", omit if uncertain)
form: Musical form (e.g., "sonata", "etude", "improvisation", fantasy", etc.) (string, omit unknown not applicable) performer: Last name of the pianist or performer, if known (string, omit if unknown or uncertain)
key_signature: Key signature of the piece (string, omit if not mentioned or uncertain)
difficulty: Estimated difficulty level (string: "beginner", intermediate", "advanced", "virtuoso", omit if uncertain) 11 music_period: Primary musical period (string: "claasical", romantic", "baroque", " "impressionist", "contemporary", "modern", omit if uncertain)
10
12
13
Rules:
14
1. Omit keys and values entirely for fields not present, unknown, or uncertain. Do not include empty strings placeholder values. include fields unless you reasonably
2. Be cautious
certain they are correct.
16
3. Provide opus and siece_number as integers only (e.g. don't include BWV, K., 8., or 0p.). Omit if not clearly number or if 2ero
4. In the case of well -nown pieces (e.g., Moonlight Sonata, Fantaisic Cantaisie-Improon Impromptu, etc.), add the opus and piece_numbes are certain, even
you 5. Provide form as a single word each, using very general and well- known terms.
it not in the raw text.
18
6. Infer difficulty and period from context when possible, but omit uncertain.
20
7. For all strings only provide a single word in lovercase ACCII.
8. For composer and performer, use only the last name. If unsure which name is the last name, omit the field.
Provide sey_stgna using standard ASCII musical notation: Use 'b' for flat, '2' for sharp, and 'g' for minor. Major keys should not have a suffix. Examples: 'c', 'I#m', "bb'.
10. Only include opus and piece_number number if the video is complete performance
single traditional opus number (0p., BWV, K., .) and its movements/variatton compilations, multiple works, ambiguous titles, or when using non -traditionalmmoden catalog numbers.
Omit both fields for
11. Dont confuse piece_number with other identifiers like sonata numbers (e.g., "Sonata No. 14") separate opus numbers (e.g.,
15

---

Published as a conference paper at ICLR 2025
in "Op. 37-38", neither 38 is piece_aumber) Only plece_number when part an opus and subordinate it.
25
26
Examples:
Input:
29
<title>Chopin Nocturne in E-flat major, 0p. No. title>
Rousseau </
<description>>rr d ric Chopin's Nocturne in E-flat major, Op. 9, No.
2. One of the most famous classical piano pieces from the Romantic era. Performed by Rousseau. #chopin #nocturne #classical /piaro//descriptionn>
35
Output:
55
" "composer": "chopin",
36
"opus": 9, "piece_number"; 2, "genre": "classical", "form": "nocturne", "performer": "rousseau", "key "key_signature"" "eb", "difficulty": "advanced", "music_period"; "romantic"
37
40
41
42
44
45
46
2. Input:
<title>Glenn Gould plays Bach Partita No. in C-minor (FULL) </title> dlescription>195. Studio recording DISCLAIMER: I not any material shown in this video. This for entertainment purposes ONLY. Unlavful distribution of this material can result in bad stuff, apparently, s0 DON 'T DO IT1</description>
50
Output:
51
52
"composer": "bach",
" "genre": "classical",
"form": "partita",
55
"performer"; "gould", "key "key_signature" "cm", "difficulty": "advanced", "music_period": "baroque"
56
60
61
3. Input: <title>Jazz Piano Bill Evans The Solo Sessions, Vol1 Full Album ]</title>
63
<decrripp1oo<<<eeerrppptta
64
65
Output:
"performer": "evans",
68
"genre": "jazz", "music_period": "modern"
69
70 }
72
4. Input:
73 <title>Martha Argerich plays Beethoven Sonata No. 31, 0p. 110</title>
76 <description>00:: 1. Moderato cantabile molto espressivo
06:12 2. Allegro molto
08:18 3. Adagio
troppo Allegro troppo
77 </description>
78
Output
16

---

Published as a conference paper at ICLR 2025
composer"; "beethoven",
82
" "opus": "110",
83
"genre": "classical",
84
"form": "sonata",
"performer": "argerich",
"difficulty"; "advanced",
87
"music_period": "classical"
88
Think step by step concisely, and then provide the metadata JSDN string.
A.3 TRANSCRIPTION ACCURACY
Table 7: Piano transcription note accuracy of the transcription model used for Aria-MIDI, evaluated on the MAESTRO (v3) and MAPS test sets. Results are calculated using the mir_eval library (Raffel et al., 2014) with default settings. We compare to the model introduced in Kong et al. (2021), which was used for GiantMIDI, ATEPP, and PiJAMA.
Note (%)
Note /w offiet
Note /w offiset & vel.
P (%)
F1 (%)
P (%)
(%)
(%)
(%)
R (%)
Fl (%)
Chosen model
MAESTRO
98.86
96.45
97.63
91.63
89.42
90.50
90.56
88.39
89.45
MAPS
91.78
89.47
90.58
Kong et al. (2021)
MAESTRO 98.82
95.53
96.82
86.51
84.21
85.33
84.97
82.72
83.82
MAPS 79.37
87.43
83.10
17


Reviews:
[{"review_id": "hy8mA4YppF", "reviewer": "Reviewer_VtYa", "summary": "This paper introduces a dataset of over one million MIDI-files containing piano music. The files are obtained by transcribing music recordings from YouTube using an existing AMT model. As part of the dataset creation pipeline, YouTube is repeatedly crawled and a language model is used to score candidate video descriptions. Furthermore, audio recordings are segmented into piano/non-piano segments using a classification model distilled from an existing source separation model. Finally, dataset statistics are provided.", "questions": "Major questions/comments:\n- Even though the provided MIDIs are the main contribution of the paper, the accuracy of the transcription model on the music segments that are ultimately transcribed by it is never evaluated. As such, it is unclear how clean the provided data actually is. \n- Following up on the previous question, how would the transcription model have fared for piano/non-piano classification and segmentation? \n- In lines 374ff, the accuracy of almost 98% for metadata extraction seems very, very high to me. Does this mean for 98% of songs, every single label (opus number, genre, performer...) is correct? Was this checked against the actual audio content? After all, a lot of YouTube videos are missing information about the exact piece being played or the performer in their description. How would a video that simply has the title \"Favorite Chopin Prelude\" (with no further information on which prelude exactly or which performer etc.) be evaluated in this context?\n- In Table 4, it seems like the \"Ablation\" (without the pseudo-labeled training data) works much better than the \"Proposed\", which calls into question the use of this data and the proposed distillation of MVSep. Why would one still want to use the proposed approach?\n\nMinor questions/comments:\n- Line 44: It is claimed that the files in the Lakh dataset are primarily transcoded from MusicXML. I don't think that's true (are they not rather collected from various MIDI repositories on the internet?). Do you have a source for this claim?\n- Table 1: The comparison in this table is misleading since the 1 million MIDIs in Aria-MIDI contain a lot of duplicates, whereas for example Mutopia and GiantMIDI contain unique compositions. Please clarify this in the table.\n- Line 71ff notes \"Instrumentation\" as a persisting challenge in AMT and existing MIDI datasets. Line 84 claims \"we address these challenges\". However, the provided dataset contains only piano music, so this claim seems wrong to me.\n- Line 174: It is stated that \"piano recordings representing diverse set of musical styles and genres were crawled thoroughly.\" However, the final dataset overwhelmingly contains classical music. Were the other styles and genres filtered out in subsequent steps or is this statement here wrong?\n- In Table 3: How can the Recall for Score >= 4 be higher than for Score >= 3 for the 70B model? The recall should decrease monotonically when changing the score threshold.\n\nAdditional comments:\n- Line 28f: \"the foundational work of Hawthorne et al. (2018), most music is highly structured and can be represented in a variety of forms\" --- The cited source is hardly the first work to realize that music is highly structured and has a variety of forms. Please find a better reference.\n- Line 318: \"labels obtained from participants very familiar with music and piano repertoire.\" --- Consider adding information on the number and experience level of human annotators and the amount of data annotated per annotator.\n- Lines 465f are superfluous.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The novel dataset provided in this paper is a welcome contribution, especially given its very large size. The steps of the dataset preparation pipeline make sense to me and the paper provides several interesting insights and comparisons, e.g., on the effectiveness of different LLMs for pre-filtering videos or numbers on piano/non-piano classification and segmentation accuracies.", "weaknesses": "Overall, I am uncertain whether the dataset provided is a contribution significant enough to warrant inclusion at ICLR. This is especially because the dataset preparation pipeline consists in large parts of plugging together existing systems in a straightforward way (notably, the LLMs and AMT model). I am also skeptical about some of the evaluations provided and claims made in the paper (see \"Questions\" below).\nThis leads to my recommendation to reject the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L85a4VkjTZ", "reviewer": "Reviewer_41nW", "summary": "The paper provides a dataset of one million MIDI files of expressive piano performances, and describes the process by which the dataset was constructed: auto-browsing YouTube for likely piano performances, downloading, auto-verifying and segmenting the piano audio, then transcribing the piano audio to MIDI.", "questions": "I do wonder why this wasn't submitted to ISMIR, as that community seems more suitable for this work.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "It's awesome that this dataset is orders of magnitude bigger than other similar datasets and is being made publicly available.\n\nThe process for constructing the dataset, filtering, segmenting, assigning metadata, transcribing, etc., and checking the accuracy of each of these steps seems painstaking and I very much appreciate the work that has gone into the dataset construction.", "weaknesses": "Not a weakness per se, but this paper really seems like it should be at ISMIR.\n\nI think the main thing I would have wanted to see but didn't is an effort to perform non-exact deduping on the *audio* (or MIDI).  I wouldn't at all be surprised if the same recording has been uploaded to YouTube multiple times with different titles, descriptions, etc., possibly including audio alterations.  Ideally if this did happen, only one instance would appear in the dataset.\n\nSomewhat related (and partially out of my own curiosity), what fraction of the MIDI files are just Moonlight Sonata?  In my experience crawling YouTube, an outrageously large fraction of piano recordings are amateur recordings of Moonlight Sonata, in the single digit percents.  It might be worth emphasizing that blindly training a generative model on the dataset might yield undesirable results due to some weird skews (though of course the provided metadata should be able to help construct an appropriate training set). \n\nWhile including only the MIDI files in the dataset and not the original audio may avoid some copyright issues, you might also want to remove *pieces* of music that are under copyright.  Since you have ostensibly accurate metadata for each performance, it shouldn't be too difficult to remove these, or at least to provide a version of the dataset that doesn't include these pieces.  Based on the statistics given in the paper, it seems likely that the *vast* majority of the pieces in the dataset are not under copyright and so this shouldn't reduce the size much.\n\nThe paper doesn't mention whether or not the transcribed performances include sustain pedal; some transcription models attempt to label sustain events and others don't.  Depending on the intended purpose of the dataset, it may be important to clarify.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "M7wjHttABx", "reviewer": "Reviewer_u1GW", "summary": "This article proposes a MIDI dataset obtained from automatic transcription and scrapping. The data can be used for training generative models and for musicological analysis. It uses an LLM-powered scrapping method and an automatic music transcription algorithm to find MIDI files for a diversity of piano pieces.", "questions": "- Where will the dataset be hosted, and how will free access be granted?\n- Will original creators be acknowledged?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The dataset is in fact extensive, and the data can be used to train generative music models. Together with the metadata, this could lead to a diversity of works related to music generation.\n\nOriginality: the idea of a dataset is not novel; however, a dataset this large is novel.\nQuality: the paper is well written and addresses all issues related to the problem\nClarity: The only issue I found is that the fact that the dataset contains solely piano pieces should be mentioned in the title.\nSignificance: Highly significant, yet for a niche.", "weaknesses": "The dataset is a great contribution, but it might be controversial to say that the dataset falls under \"fair use\". This is, first, because \"fair use\" is different in each country; also, nothing in the CC-BY-NC licence mentions the use of models derived from this data to generate new data, that is, although the files cannot be sold themselves, the models able to generate similar excerpts could be sold and used for commercial purposes.  In this sense, a CC-BY-NC-SA would be more adequate and more coherent with the \"fair use\" idea, or, alternatively, terms of licencing that prohibit usage for commercial models or models that might be used in commercial distributions (avoiding a situation in which a free model is released, but is used commercially)?\n\nAccording to https://www.copyright.gov/fair-use/, using creative work is less likely to support a \"fair use\" claim; also, using a large amount of copyrighted work for derivative creation could play agains the \"fair use\" claim. Could the authors add a discussion on how the dataset addresses each of the requisites for fair use?\n\nIt would be interesting to have a professional opinion - and maybe even community guidelines in the future - for this matter, as the liability could be large; if that issue is solved, the dataset is amazing. Could the authors include a detailed legal analysis - maybe derived from this consultancy with a copyright expert - and include their findings in the paper, particularly addressing potential liabilities and jurisdictional differences (remember, we have a worldwide community, so these rules might be different in each country).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GIGQz4bXgs", "reviewer": "Reviewer_NngF", "summary": "This paper proposes a large-scale piano MIDI dataset through an automatic pipeline, comprised of a metadata extraction module, a source-separation guided classifier, and a MIDI transcriber.", "questions": "1. Could the authors clarify if the proposed pipeline would be open-sourced, and if so, under which license?\n2. Can the authors provide baseline experiment results on generation using this dataset against previous datasets, for example, GiantPiano-MIDI?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper proposes a large-scale dataset, and details the creation process associated with it. With a large-scale dataset like this, the field of piano symbolic music generation can vastly benefit.", "weaknesses": "This paper lacks novelty, as the main work is centered around the creation of this dataset. The main process of dataset creation can be described as a cascade of open-source models and tools (such as llama, MVSep); these tools are widely available, and their combination is not novel. I aim to see some sort of generative experiments to demonstrate the superiority of this dataset (especially the scaling law effect the large-corpus should bring) or at least any neural networks trained on this proposed dataset, as without these experiments, I have no way of evaluating the validity of the dataset. As experiments are largely missing, I believe this paper still needs more work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)