LEGATO: Large-scale End-to-end Generalizable Approach to Typeset OMR
Authors: Guang Yang
Victoria Ebert
Nazif Can Tamer
Brian Siyuan Zheng
Luiza Amador Pozzobon
Noah A. Smith
Venue: ICLR 2026 Poster (2026)
Keywords: Optical Music Recognition
AI for Music
Multimodal Learning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: Legato is the first large-scale pretrained OMR model that achieves the state of the art on various datasets.

Abstract: We propose Legato, a new end-to-end  model for optical music recognition (OMR), a task of converting music score images to machine-readable documents. Legato is the first large-scale pretrained OMR model capable of recognizing full-page or multi-page typeset music scores and the first to generate documents in ABC notation, a concise, human-readable format for symbolic music. Bringing together a pretrained vision encoder with an ABC decoder trained on a dataset of more than 214K images, our model exhibits the strong ability to generalize across various typeset scores. We conduct comprehensive experiments on a range of datasets and metrics and demonstrate that Legato outperforms the previous state of the art. On our most realistic dataset, we see a 68\% and 47.6\% absolute error reduction on the standard metrics TEDn and OMR-NED, respectively.

Published as a conference paper at ICLR 2026
LEGATO: LARGE-SCALE END-TO-END GENERALIZABLE APPROACH TO TYPESET OMR
Guang Yang'
Victoria Ebert' Nazif Can Tamer'
Brian Siyuan Zheng' Luiza Amador Pozzobon' Noah A. Smith
Paal G. Allen School of Computer Science & Engineering. University of Washington
2 AAllen Institute for AI
(gyangl, nasmith (cs. washington. edu
ABSTRACT
We propose Legato, new end-to-end model for optical music recognition (OMR), a task of converting music score images to machine-readable documents. Legato is the first large-scale pretrained OMR model capable of recognizing full-page or multi-page typeset music scores and the first to generate documents in ABC notation, concise, readable format for symbolic music. Bringing together a pretrained vision encoder with an ABC decoder trained on a dataset of more than 214K images, model exhibits the strong ability generalize across various typeset scores. We conduct comprehensive experiments on range of datasets and metries and demonstrate that Legato outperforms the previous state of the On our most realistic dataset, we see a 68% and 47.6% absolute error reduction on the standard metries TEDn and OMR-NED, respectively.
INTRODUCTION
A substantial portion of written music exists only as photocopies of printed sheet music (c.g., IM- SLP:,Projec Petrucci LLC 2025). Digitalizing these images into modern, machine-readable formats would unlock data for wide range of music analysis and synthesis applications, at an unprecedented scale. With this goal in mind, we tackle the problem of optical music recognition (OMR) to effi ciently convert images of typeset scores to symbols.
The most successful approaches to this problem are end-to-en OMR systems (Rios- Vila al. 2024 Vils et al., 2024 Ríos- Vila et al. 2023 Mayer et al.) 2024 Calvo-Zaragoza & Rizo 2018b), focusing exclusively on formats for piano, monophonic music, or single-system scores. More gen- eralizable solutions require careful consideration of the diversity of inputs (i.e., complex layouts that contain multiple systems, staves and voices on a single page, as well as extensive text annotations such titles and lyrics), outputs (each output format-es e.g., MusicXML, ABC, **kern hhern-has its own advantages for OMR and current evaluation heavily relies on output format choice), and data for training. Our main contributions
follows.
We construct symbolic scores in PDMX (Long et al. 2025 Xu et al, 2024) with diverse rendering schemes ($3)
multi-page large-scale OMR dataset, PDMX-Synth. rendered from
We apply data-driven tokenization algorithm to symbolic music and find that is capable of learning composite music concepts ($4.1).
We introduce Legato, the first end-to-end OMR model built upon a pretrained vision en- coder, and capable of recognizing multi-page typeset scores ($4).
In comprehensive experiments, even those giving advantages to the baseline, we find that
Legato achieves state-of-the-are performance on multiple OMR datasets, including newly constructed sample from IMSLP (Project Petrucci LLC 2025), with a large im- provement the previous best model ($)).
We release the code to reproduce our work http::://gtthbb...mmmaaagggggy//

---

Published as a conference paper at ICLR 2026
Image Segments
Image Patches
cd)
2b)
\n
1:4 Crop
Resize Split
Vision Encoder
Transformer Decoder
<8>
<I>
(8>
0
cd)
30
10
Context Prefix
Autoregressive Decoding
Figure 1: Model architecture. The input image is first cropped into overlapping segments with an aspect ratio of 1:4 or less, then resized and divided into four patches per segment ($4.2.1) The image patches fed into vision encoder ($4.2.2) parameters are frozen during training). The resulting latent embeddings serve as cross-attention keys and values in a transformer decoder, which autoregressively generates ABC tokens ($4.2.3). Special tokens <B>, <I>, and <E> denote wleeiinnee _f abc >, < image >>, tion, here we use represent whitespace.
and < end of _abe >>, respectively. For better visualiza-
2 BASELINE AND DESIGN CONSIDERATIONS
2.1 MUL.TI-SYSTEN OMR
Our starting point is the success of Sheet Music Transformer++ (SMT++; Rios- Vila et al. 2024), which is, to our knowledge, the only model designed to handle multiple systems in a score rather than single-stafff single-system scores (Mayer al., 2024 Ríos-Vila et al. 2023). SMT+ is an encoder-decoder transformer model trained end-to-end on purely synthetic, full-page piano-form scores. It was trained on Fr-GrandStaff (688 pages). which was generated by randomly concate- nating single-system piano scores from GrandStaff (Rios- Vila et al. 2023). SMT+ is the baseline against which we compare our approach.
Besides specialized models like SMT++, we also investigate how state-of tftt-e-rtt general-purpose multimodal models perform on OMR tasks. Specifically, we evaluate GPT-5 (OpenAl 2025) under similar conditions as the other baseline.
2.2 OUTPUT SCORE REPRESENTATION
Many different symbolie score formats have been considered in past work on OMR; this paper considers **kern, ABC, and MusicXML.
**kern is musical representation designed within the Humdrum toolkit (Huron 1997 **kern ASCII-based and places pitch and relative duration as the main focal points of the format, with visual information coming second. **kern explicitly models concurrent notes using spaces for notes in the same staff, and tabs for notes in other staves. It was designed with music researchers in mind, and with some study can be read by humans. * **kern an OMR target in SMT++ (Rios- Vila et al. 2024) and other systems (Rios- Vila et al. 2024 Ríos- Vila et al. 2023).
The ABC music standard was introduced as an alternative ASsed-based form of musical nota- tion (Walshaw (011). Similar to #*kern, it can be written and read by any text editor. The main attraction of ABC notation the simple, concise format-a song that would take thousands of lines in other formats will take only tens in ABC, potentially reducing the computational cost for au- toregressive models like our decoder. ABC notation can encode lyrics, title, tempo, decorations, articulations, and even some kinds of typesetting parameters, making it nearly comprehensive for- mat. The more explicit structure for musical engravings such as barlines and linebreaks, as well notation more similar to sheet music than that of **kern, has led to ABC notation target for many systems (Wu et al. 2024] Wu & Sun 2023 Casini et al. 2024), but to our knowledge, iBage-to-ABC OMR models have yet been trained.
2

---

Published as a conference paper at ICLR 2026
MusicXML is tree-based music notation format introduced in 2001 (Good 2001). widely adopted by commercial software such as MuseScore [MuseScore Ltd. (2021) and Sibelius (Avid Technology, 2023). Its popularity has made it common target for OMR and music transcription tasks (Beyer & Dai 2024), sometimes with simplifications (Mayer et al., 2024). However, compared to ABC and *errn, MusicXML. is more verbose, harder to parse, and its hierarchical structure poses challenges for sequence models.
Each of these formats, despite their differences, carry much of the same information. They all include some intrinsic method of codifying visual score information such as barlines and phras- ing, and each encodes note length as an absolute value. similar typical numan-readabee sheet elements that other popular symbolic music formats, such as MIDI. lack. This makes conversion among the three formats relatively simple, and many tools exist that can convert be- tween formats (Vree et al.; 2018 Cuthbert & Ariza 2010, Canncio-Chacón et al., 2023 Pugin et al.) 2014 Sapp 2012). retaining core musical elements (notes, rests, measures). However, due dif- ferences in the scope of information encoded across formats, certain elements- as lyrics and articulations-may not be preserved during conversion.
We chose ABC as the output format for our model, as it is concise but nearly comprehensive and centers musical (rather than typesetting) elements. We believe the structure of ABC also lends itself well to usage with NLP techniques that help the model learn composite musical concepts, will see in the tokenizer ($441). Besides, we note that Markdown is widely used as the interface between general-purpose language models and humans, and some modern Markdown editors (e.g., StackEdit, JotterPad) are able to directly render scores from ABC code blocks, while **kern and MusieXML, our knowledge, are not supported.
Finally, we focus on the recognition of musical notation rather than textual features of a musical score (e.g.. lyrics), leaving the textual recognition future work.
Given these design decisions and the strong starting point of SMT+, our approach to building multi-system, multi-page, end-to-end OMR model includes constructing a large-scale dataset of score images with ABC-formatted representation (t3) and using it to train an encoder-decoder transformer model ( that builds on an existing pretrained image encoder.
DATASET
To train an end-to-end transformer-base based OMR system, a large quantity of paired data (images with symbolic scores in the target format, ABC) is required.
3.1 A LARGE-SCALE OMR DATASET BASED ON PDMX
While the ABC notation project offers 750K examples available for free download, in genres from medieval music to pop music [
we found this data to frequently monophonic. We believe these examples did not originate as full scores. We therefore turn to PDMX (Long al. 2025 Xu al.) 2024). dataset with 250K public domain MusicXML files collected from the online sharing fo- rum MuseScore. We construct our ABC dataset, PDMX-SSnth, by converting PDMX files from MusicXML to ABC format, and rendering images from a mixture of these two formats. During rendering, we exclude scores with aspect ratios greater than 10 (about 5% of the data). Training au- toregressive models on long sequences is computationally expensive, and since such cases are rare, our approach sacrifices them for efficiency. Also, canonicalize ABC format so that the model can more easily learn the format constraints, and also to simplify evaluation ($3.2). Due to filtering, conversion, and rendering loss, our final dataset contains 238,386 image-ABC pairs, about 93.8% of the original PDMX dataset.
3.2 CANONICAL ABC REPRESENTATION
We use the xml2abe script (Vree et al. 2018) to convert from MusicXML to ABC in batch mode. The following rules are applied to nearly canonilanniic ABC
https:: / /aabbncoatttt at iontioonee com/
Moor could be done; e.g., voice numbering can still be swapped.
3

---

Published as a conference paper at ICLR 2026
Lacrimosa (from the Requiem in D minor)
Wolfgang Amadeus Mozart (1756-1791)
Intro
0
La
cri
mo
Di
Cs il
la.
Qua
Rc
Sur
Get
X:1
T:< F:</text]
13
|pl"</text|>> (*cd) (ab) (dc) z (-c'b) | z (ad') (bg) (ef)
Te<]
(a*c) ||s [FA]3- [FA][AF][Fd] ([G-Bd]3 [GA<C]2) 2 [FA]3- [F]] ([G-Bd]3 [GA<]]2) [F,A,D]2 z [A,C [A, 2 [=CEG]2 |$ X5
C:</text|
XSscore ( 2
[|<[text/> </text|>> >> </text|> + </text|>> </text]>|<<] <<t]>///text]>>>>>> 

6
L:1/8
<]text]>> </ttxt|>> </text|>> [[text]><<]ees

 </text|>
M:12/8
text|>>
I:linebreak
9
K:F
16
[DF]2 z [FA]2 z [EG]2 [G*c]2 z [Fd]2 2[K:bass] [G,E]2 [F,0]2 [A,G]2 z ||s D, (*CD) (A,B)) (DC) A,, (B,A,,, (*CD) F, (A,B,)
18
V:1 treble nmt [
treble
(DC) A,, (B,A,) (D,(D,))s) (E,,(E)),))) (F,D), (,,G,, [$
Figure 2: An example of our canonical ABC representation (below) with a MusicXMMLreendrred image (above).
$. Line breaks are optional in ABC; explicitly marking them allows recovering original line breaks.
Transcribe real line breaks with
Force ABC files to break lines every 5 bars. Textual line breaks in ABC have mantics for scores. We enforce a fixed line length of 5 bars in the ABC text, except when fewer than 5 bars remain at the end of the score and retrain converter-generated comments & @number_of_ten
meeaauureee res] ] at the end of each line.
Fix the unit note to an 8th note. The ABC grammar allows customized unit note length with L: 1/:111, / L: different unit note lengths. To simplify learning, we set L: L:1/8. This does not change the expressive power of the representation.
so the same score could be transcribed differently with
Since PDMX-Synth designed for the OMR task, we replace all text contents ABC represen- tation with special < text >> tokens. This includes the titles, instrument names, lyries, and an- notation contents quoted in the ABC tune body. Figure 2 shows an example of our canonical ABC representation.
3.3 SCORE RENDERING
To construct an OMR training dataset from MusicXML or ABC files, is important to choose an appropriate renderer. It should be able faithfully generate score images and represent most information contained in MusicXML files (e.g., fonts and spacing). Moreover, the rendered images should be varied enough that the trained model will not overfit the default renderer parameters.
The FP- GrandStaff dataset used to train SMT+ Ríoss Vila et al. 2024) uses the Verovio tool (Pugin al. 2014) the default renderer, and generates the images from ** kern format. Since the **kern
4

---

Published as a conference paper at ICLR 2026
(a) Bd
(h) arpeggia
(c)
(3GAB
(d)
A>BA
Figure 3: Example vocabulary items from tokenization.
format they use does not encode any typesetting information, the software's default rendering pa- rameters are used, heavily limiting dataset diversity. To address this issue, we generate images with two rendering pipelines:
MuseScore 3.6.2 (MuseScore Ltd. 202 which takes in MusicXML and outputs PNG.
abcm2ps 8. 8.14.15 (Moine 2024), which takes in ABC and outputs SVG files. We further
use CairoS 2.7.1 (CourtBouillon, 2023) to convert SVG files into PNG files.
To prevent default rendering parameters from prevailing in the dataset, we apply these visual aug- mentations:
For (i), the final images augmented with randomized image resolution (i.e., randomly set the resolution parameter in MuseScore) and randomized margin cropping sampled from uniform distribution. When cropping, the main score remains untouched.
For (ii), since ABC formats carry less typesetting information, more augmentations applied:
1. render a score with a single image or multiple images concatenated;
2. with probability of 50%, render the images in landscape mode;
3. with probability of 70%, add the measure numbers with different numbering styles; uniformly set the left and right margins;
randomly scale the image by a fraction [0.9, 1].
Besides, for both renderers, the background color is sampled uniformly from the grayscale range [192, 255]. We release this synthesized ABC OMR dataset, PDMX-Synth, support future search.
4 END-TO-END MODEL
Our model, Legato, follows the architecture of multimodal Llama @ Meta| 2024 As shown in Figure [ the main components of our model are pretrained vision encoder and transformer decoder into ABC. The input score image is divided into segments, resized, and further split into four patches per segment, which are encoded into latent embeddings by the vision encoder. These embeddings are then used by the transformer decoder to autoregressively generate tokens in ABC format. We discuss our ABC tokenization method in $4.1 then turn to the architecture: details of image processing (42..1. the use of pretrained vision encoder (42.22 and the transformer decoder ($4 4.2.3..
4.1 TOKENIZATION
Tokenization schemes for language models split a stream characters into tokens from a fixed vocabulary. They be based cabulary contains all possible **kern symbols (Ríos-Vila et al. 2024)), or constructed in data-driven fashion. We take the latter approach, which allows composite musical concepts like chords to be represented directly in the vocabulary if they are sufficiently frequent.
expert-defined vocabulary, done with SMT++, whose vo-
We adopt the byte-pair encoding method (BPE: (SPEniic et al. 2016) for learning tokenizer, widely used in natural language processing research and known for effectively capturing diverse patterns within limited vocabulary, particularly when trained large corpora. We choose to apply BPE
5

---

Published as a conference paper at ICLR 2026
tokenization directly to the ABC representation of PEMMD-Myymii training set, with a vocabulary size of 4097, ensuring efficient representation and facilitating better model performance.
We find that our tokenizer captures phrases. For example, the major triad, represented as CEG, emerges as a discrete token. This token exhibits contextual flexibility: within square brackets ({)), it denotes simultaneous chord, whereas in the absence of brackets, it represents an arpeggiated sequence. When combined with duration tokens such as 2 4, CEG can represent major triad composed quarter notes or half notes, respectively. More examples are shown in Figure
composite musical concepts like chords and short melodic
4.2 MODEL ARCHITECTURE
Multimodal Llama effectively handles images with varying aspect ratios without significant distor- tion (AI@ Metal 2024). Given the complexity of the OMR task and the associated computational cost, we use the pretrained vision encoder from meta-1lann walllaammaaaaaaaaaaaaaaaaaaaaaaa LLtamaaammaammam (836M parameters, frozen in our system) and train a p-rrammrer transformer decoder from scratch on ABC representation, along with 5.9M-parameter linear multimodal projector to bridge between them. We refer to this model as Legato.
To control for model size when comparing with previous state-of-the-ar methods, we also introduce a smaller variant, LegatOsmall, which has 8.5M parameters the decoder and 2.5M the projector. This design has a comparable number of trainable parameters SMT++, though the pretrained and frozen vision encoder adds substantially more.
4.2.1 IMAGE PROCESSING
Our input score image / consists of the full score of a composition. Since a composition of music can be very long, the image I might have very large height, but the width stays relatively fixed (since scores are printed in portrait or landscape mode on standard paper sizes). We divided the image into multiple segments, each with an aspect ratio of 1:4 or less. Adjacent segments also have an overlap, ensuring that each segment retains contextual information.
We follow the approach used in multimodal Llama (Al@ Meta, 2024 to further process each image segment. After resizing and cropping, each segment is divided into four smaller image patches. Therefore, starting from the original image we get a tensor p E RSx4xCxDxD where the number of segments, C' = 3 is the number of color channels, and D 448 is the internal image size.
4.2.2 VISION ENCODER
The Llama vision encoder was pre-trained on general-purpose images. It maps an image segment embedding. We conjecture that vision encoder trained on diverse image data provides strong starting point for OMR on score images, so we keep the vision encoder frozen in our training and testing experiments. Finetuning this module specifically for scores is a promising direction for future work.
We refer the reader to AI Meta 2024 for details on the encoder, noting only that provides an output embedding in RR Sx4x where L is the sequence length (specifically + (22) and d, the internal visual embedding dimension. In the Llama checkpoint used, 1025, d, 1280.
4.2.3 TRANSFORMER DECODER
We adopt decoder architecture similar to that of multimodal Ilama, but with a smaller scale. A linear projection is first applied to the latent embedding to match the decoder's hidden dimension di enabling its use in cross-attention. The core of the decoder consists of a Læ-layer transformer, where cross- coossaattentinn is selectively applied at subset of layers denoted by r7, while the remaining layers use only self-attention to reduce computational cost. The MLP module in each layer first upscales the dimension to d, then downscales back to dj.
As illustrated in Figure | the next token in the sequence, and during inference, it autoregressively generates tokens at
the decoder takes a sequence tokens as input. It is trained to predict

---

Published as a conference paper at ICLR 2026
time. In Legato. d, 768, d, 1526, La 18 and r d, = 448, La = and r = {3, (3.5.7 7)
In Lmgalosmall- di 320,
4.3 PRETRAINING DETAILS
Both Legato and Legatosmall are trained for 10 epochs using a batch size of 32 and learning rate of 0.0003. Following standard language model practices, we use the AdamW optimizer (3) = 0.9, 82 = 0.99, = 10-6), linear learning rate scheduler, and warm-up ratio of 0.03. To improve efficiency, text sequences are truncated to 4096 tokens, and 6 precision is used.
Due to the high cost of inference and metric computation, evaluation is performed every 5000 steps on a subset of 800 validation samples. The checkpoint with the lowest symbol error rate (SER) on ABC is retained: 65,000 steps for Legato and 60,000 steps for Legato
EVALUATION METRICS
Evaluation of the end-to and-to-end end OMR task has not yet converged on single standard. Previous work (Ríos- Vila et al. 2024 Ríos- Vila et al.) 2024, Mayer et al., 2024 Calvo-Zaragoza & Rizo, 2018a b) report error rates in different score formats there is no unified output format that is most suitable for this task. Typically, the chosen format is the one the proposed model was trained on. This choice might unfairly penalize the performance of models with different output format and therefore hinder cross-model evaluation, as the conversion among formats is not always successful, and conversions of ill-formed outputs undefined.
We adopt MusicXML as a unifying evaluation format across different models as it is able to carry all information required to typeset music scores. It is also not the format used to train Legato or SMT++, which yields a more fair comparison. Moreover, MusicXML. is widely supported across music software, making it reasonable final target for use-cases leading editing of scores by humans. When necessary, we also provide error rates in **kern and ABC formats, which are the formats used in training SMT++ (Ríos-Vilk Vila al. (2024) and Legato, respectively. Currently, both TEDn and OMR-NED -ED evaluate textual elements. This means the final error rates include the edit distance between Legato's predicted < text token and the actual text in the ground truth.
5.1 TREE EDIT DISTANCE WITH NOTE FLATTENING (TEDN)
Hajič Jr. et al. (2016) investigate different evaluation metrics on MusicXML and reach the con- clusion that tree edit distance with <note> flattening is the best match to human evaluation. To elaborate, this metric first flattens all <note> elements in both XML trees and then uses normalized tree edit distance (edit distance between two trees divided by edit distance to recover the gold tree) as the final non non-negative score. Flattening is used to decrease the high cost to delete <not e> element as it always contains many child elements. We use the implementation from Mayer al.) (2024). The most efficient edit distance algorithm requires O(m²n²) time, where m and n are the number of nodes in the two trees (Zhang Shasha 1989). Because of this cost, we truncate Mus.cXMI. files so that < 6000 and use format-specific error rates to validate models during training. This truncation prevents the TEDn from capturing all errors.
5.2 FORMAT-SPECITCC ERROR RATES
A group of more widely-used metrics in OMR are character, symbol, and line error rates (CER, SER, and LER). They measure how much effort takes to correct the predicted content, but they simplify the scores to text sequences rather than structured encoding and therefore fail to capture the magnitude structural errors. These metrics are also relatively cheap, with ((m)) runtime (for string lengths and 71). Previous work (Rios-Vill et al. 2024) uses these metrics on #*kern, but our model trained on ABC. Error rates on these two formats are not comparable since characters, symbols, and lines in these formats represent different concepts. So, use converters to achieve an apples-to-apple comparison in both formats. However, these metrics still favor the model trained in the target format, as failed conversions for the other model result in empty outputs.

---

Published as a conference paper at ICLR 2026
5.3 OMR NORMALIZED EDIT DISTANCE (OMR-NED)
OMR-NED [Martinez-Sevii Sevilla et al., 2025a) is novel evaluation metric for OMR that achieves a balance between computational efficieccy and evaluation granularity. based on the sequence emoor rate computed between two sequences of musical measures, where the cost of transforming one measure into another is defined by the set edit distance between their constituent symbols. Insertion and deletion costs are specified for different categories of music symbols, enabling fine-grained assessment of model performance. This metric is both efficient and perceptually meaningful, with time complexity of O 5 log S), where M denotes the number of measures and S the average number of symbols per measure. Furthermore, OMR-NED is format-agnostic, provided that a robust parser is available extract symbolic representations from the model's output.
The current OMR-NED implementation is optimized for **ke through syntax correction, while ABC v2.1 lacks parser support; thus, ABC outputs are converted MusicXML. for symbol extrac- tion, which may introduce errors and disadvantage ABC.
EXPERIMENTAL EVALUATION AND RESULTS
One limitation of many prior end- and-to-end OMR models is that they are only evaluated on the test split of the same datasets used for training (Rios- et al., 2024 Ríos- Vila et al.] 2023 Mayer al., 2024). or continue training on evaluation datasets (Ríos- Vila et al. 2024). which raises the risk of overfitting narrow population of scores. General OMR inputs exhibit significant variability, while individual datasets often stem from limited sources. Such restricted datasets are easier to over- fit, artificially boosting evaluation metrics. We therefore evaluate Legato and the SMT++ baseline on a diverse set of OMR datasets, none of which are used for training or validation, ensuring more robust and unbiased assessment of generalization performance. These include: (i) OpenScore String Quartets (Gotham et al. 2023), using both realistic and rendered images, (ii) OpenScore String Quartets Violin parts, using Verovio-rendered images, (iii) OpenScore Lieder (Gotham & Jonas, 2022). using both realistic images and newly rendered ones, and (iv) newly manually converted Piano scores from IMSLP. Furthermore, we also evaluate Legato on JAZZMUS (Martinez-Sevillaa
ct al. 2025b), handwritten score dataset. As this dataset deviates from Legato's primary training distribution, the model demonstrates limited generalization to handwritten notation.
To provide a reference of current general VLM's s ability, also evaluate GPT-5's performance on these datasets. Results are shown in Table [ GPT-5 consistently outperforms SMT+ on TEDn but underperforms on OMR-NED. This is likely due to **kem syntax correction made by OMR-NED.
All evaluations for Legato are performed with beam search (beam size 10, max length 2048) and repetition penalty of 1.1, except PDMX-Synth, where use beam size of 3 due very large images. The baseline settings for both SMT++ and GPT-5 are detailed in appendix SA
Evaluation on PDMX-Synth We first evaluate the Legato models on 800 items from the PDMX- Synth test split. This evaluation establishes the quality of the Legato models" "in-domain" ability and on the output format they were trained to produce (ABC). Legato achieves (ABC) character, symbol and line error rates of 23.3%, 25.8%, and 31.7%, respectively, while Legatosmall achieves 36.4%, 39.2%, and 45.9%, respectively. We also evaluate the multi page performance (appendix (B6)).
Evaluation on OpenScore String Quartets. To compare fairly with SMT++, we use third-party dataset, OpenScore String Quartets (Gotham et al., 2023), mainly from the 19th century; this data was not used to train either model. The dataset provides MusicXML files, and for some entries, a scanned PDF is also available, from which the MusicXML was annotated. We extract a sub- set of the OpenScore String Quartets dataset containing scanned images of real scores, and render the corresponding clean images from the associated MusicXML files. We call these two differ- ent kinds of images "Camera" and "Rendered" and evaluate on both. The results are shown in Table [ (blocks 1-2). Note that for both SMT+ and Legato, the output is not guaranteed to be convertible to MML, a necessary step for calculating the TEDn evaluation metric. However, when favoring SMT++ by evaluating only on instances where it produces valid results (rows marked "TEDn "), Legato still outperforms it. Additional experiments further reducing the differences between these models presented in appendix B.5
8

---

Published as a conference paper at ICLR 2026
Table 1: Experimental results on various datasets and metrics. Lower is better for all metrics. represents the number of samples tested for the metric. TEDn is the primary metric, requiring outputs be converted MusicXML. TEDnett evaluated only on instances where SMT++ produces outputs that can be successfully converted to MusicXML; Legato outputs always converted to MusicXML successfully. OMR-NED is format agnostic, built on extraction of symbols from any format. For OMR-NED, **kern outputs are automatically corrected for syntax errors, while ABC first converted to MusicXML (again, always successful in practice) before symbol extraction. OpenScore String Quartets is the most challenging dataset, since it has much denser score images. All metrics are explained in
Metric
GPT-5
SMT++
Legato
LegatO
1. Camera OpenScore String Quartets (252 pages)
TEDn
252
90.5
98.6
60.4
84.1
TEDneonvert
18
98.1
80.2
58.6
84.1
OMR-NED 252
97.6
94.7
58.2
93.5
2. Rendered OpenScore String Quartets (252 pages)
TEDn
252
93.0
97.9
52.1
78.4
TEDDee
28
93.8
81.3
50.5
78.9
OMR-NED
252
97.8
94.3
32.9
88.5
3. Camera OpenScore Lieder (64 pages)
TEDn
64
91.4
98.7
47.0
82.7
TEDDeoncet
6
98.6
86.3
44.5
57.2
OMR-NED
64
97.0
86.4
55.8
85.1
4. Rendered OpenScore Lieder (64 pages)
TEDn
64
91.0
97.4
26.5
68.8
TEDneonvert
97.5
58.1
8.7
52.6
OMR-NED
64
97.2
78.0
34.3
69.4
5. IMSLP Piano Scores (32 poges)
TEDn
32
96.7
97.7
29.7
76.9
TEDDeonvert
3
90.3
75.2
3.8
43.7
OMR-NED
32
98.7
91.9
44.3
86.7
Evaluation on OpenScore Lieder. enable a more comprehensive evaluation, we assess both models the OpenScore Lieder dataset (Gotham & Jonas 2022). which contains songs by 19th- century composers. Similar to OpenScore String Quartets, we obtain the source PDFs to generate both camera and rendered images. Again favoring SMT++, retain only the piano part by mask- ing the vocal staff with white boxes. This approach similar to the one used in the OLiMPiC dataset (Mayer
2024), although we full-page images instead of single-system excerpts. As shown in Table (blocks 3-4), both Legato variants outperform SMT++ by a large margin.
Evaluation on IMSLP Piano Scores. Masking the vocal staff with white boxes still introduces artifacts such as large gaps between staves. For more realistic evaluation on piano-form camera scores, we manually annotated 32 full-page piano scores from IMSLP (Project Petrucci LLC 2025). We ensure there is overlap with PDMX-Synth and it includes only scans produced before the adoption of modern typesetting software such as MuseScore or Verovio. This allows us to eliminate biases introduced by synthetic typesetting and data source overlap. This small evaluation dataset is publicly available in our codebase. The results are presented in [bbock] 5).
Although the dataset is relatively small, it provides fair and realistic comparison- both models are designed to recognize piano scores, and the image sources reflect real-world scenarios, as many piano learners obtain their scores from IMSLP. We show some examples for comparing the outputs in the appendix BB.7
7 CONCLUSION
In this work, we propose Legato: a state-of-the-e end-to-end generalizable model for typeset OMR. Legato is capable of recognizing multi-page realistic typeset score images and outputs ABC

---

Published as a conference paper at ICLR 2026
representations. To achieve this, we leveraged a pretrained vision encoder (frozen), and trained tokenizer and a decoder model on more than 214K samples from HDMMX-Sythh a processed version of the PDMX dataset.
Legato achieves slate-of-the-e performance on all evaluated datasets, even when favoring previ- methods, and it represents significant step forward the first multi-page end-to-end OMR model for typeset scores. As future work, researchers can investigate how to finetune modern vision encoders to further adapt the specifie challenges of OMR.
ETHICS STATEMENT
This work has made use of large language model (LLM) tools, including experiments on GPT-5, as well as assistance with language polishing and the generation of acronyms. The conception of this paper, together with the complete codebase, PDMX-Sym dataset, and experiments related to Legato, are original contributions of the authors.
REFERENCES
Meta. Llama 3.2: Revolutionizing edge AI and vision with open, customizable models. Tech- nical report, Meta Platforms, Inc., September 2024. URL htops://ii meta. com/blog/ Ilama- 3-2 connect-
Avid Technology. Sibelius, 2025.
Tim Beyer and Angela Dai. End-to-end piano performance-MIDI to score conversion with trans- formers, 2024. URL
wtt.or//aab////000 00210
Jorge Calvo-Zaragoza and David Rizo. Camera-PrIMuSS Neural end-to-end optical music recogni- tion on realistic monophonic scores. In Proceedings of the 19th International Society for Music Information Retrieval Conference, pp. 248-255, 2018a. doi: 10.5281/zenodo. 1492395.
Jorge Calvo- Zaragoza and David Rizo. End-to-end neural optical music recognition of monophonic scores. Applied Sciences, 8(4):606, 2018b. doi: 10.3390/app0440000.
Carlos Eduardo Cancino-Chacón Silvan David Peter, Emmanouil Karystinaios, Francesco Foscarin, Maarten Grachten, and Gerhard Widmer. Partitura: A Python package for symbolic music pro- cessing. Music Encoding Conference 2022 Proceedings, pp. 16-26. Humanities Commons, 2023. doi: 10.17613/131v-k502
Luca Casini, Nicolas Jonason, and Bob T. Sturm. Investigating the viability of masked language modeling for symbolic music generation in abc-notation. In Colin Johnson, Sérgio M. Rebelo, and Iria Santos (eds.), Artificial Intelligence Music, Sound, Art and Design, pp. 84-96, Cham, 2024. Springer Nature Switzerland. doi: 10.10077978-3--33156999
CourtBouillon. CairoSVG 2.7.1, 2023. URL httpss//caarosvv.ccr
Mike Cuthbert and Christopher Ariza. Music21: A toolkit for computer-aided musicology and symbolic music data. In Proceedings of the 11th International Society for Music Information Retrieval Conference, pp. 637 642, 2010. doi: 10.5281/zenodo. http//ooooo..111111 1416114.
Michael D. Good. MusicXML: An internet-rrennly format for sheet music. In Proceedings of XML 2001, 2001.
Mark Gotham, Maureen Redbond, Bruno Bower, and Peter Jonas. The "OpenScore String Quartet" corpus. Proceedings of the 10th International Conference on Digital Libraries for Musicology, pp. 49-57, New York, NY, USA, 2023. Association for Computing Machinery, doi: 10.1145/ 3625135.3625155.
Mark Robert Haigh Gotham and Peter Jonas. The OpenScore Lieder corpus. In Music Encod- ing Conference Proceedings 2021, pp. 131-136. Humanities Commons, 2022. doi: 10.17613/ 1my2-dm23.
10

---

Published as a conference paper at ICLR 2026
Jan Hajié Jr., Jifí Novotný, Pavel Pecina, and Jaroslav Pokomý. Further steps towards standard testbed for optical music recognition. In Proceedings of the 17th International Society for Music Information Retrieval Conference, pp. 157-163, 2016. doi: 10.528
David Huron. Humdrum and Kerm: selective feature encoding. In Eleanor Selfridge-Fiee Fielld (ed.), Beyond MIDI: The Handbook of Musical Codes, pp. 375-401, MIT Press, Cambridge, MA, USA, 1997. ISBN 026000922902 193949.
Phillip Long, Zachary Novack, Taylor Kinkgairkattrick, and Julian McAuley. PDMX: A large-scale public domain MusicXML dataset for symbolic music processing. ICASSP 2025 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5, 2025.
doi: 10. 10.110000CASSSP444002200..5.. 202800001...
Juan C. Martinez-Sevilla Joan Cerveto-Serrano, Noelia Luna, Greg Chapman, Craig Sapp, David Rizo, and Jorge Calvo-Zaragoza Sheet music benchmark: Standardized optical music recognition evaluation, 2025a. URL httpt: http://aaxv..v/o wrg. abs abdeaabs.55/.. 10488
Juan Carlos Martinez-Sevilla, Francesco Foscarin, Patricia Garcia- lasci, David Rizo, Jorge Calvo- Zaragoza, and Gerhard Widmer. Optical music recognition of jazz lead sheets, 2025b. URL
https://axxiv.org// httpgg//b/////00 05329
Jifí Mayer, Milan Straka, Jan Hajič, and Pavel Pecina. Practical end-to-end optical music recognition for pianoform music. In Elisa H. Barney Smith, Marcus Liwicki, and Liangrui Peng (eds.), Document Analysis and Recognition ICDAR 2024, pp. 55-73, Cham, 2024. Springer Nature
Switzerland. ISBN 978-3-031-70552-66 doi: 10. 110100978-3-001-70552----
Jean-Francois Moine ahem2ps 8. 8.14.15, 2024.
URL https: com lewdll ime/
abcm2pa/tree//ve 14. 15
MuseScore Ltd. MuseScore 3.6.2, 2021. URL
OpenAl. Introducing GPT-5. Technical report, OpenAI, Inc., August 2025. URL https://
openai. htamm//ineex/ntttouuiiiiioo
Project Petrucci LLC. International music score library project (IMSLP). https://imss org. 2025.
Laurent Pugin, Rodolfo Zitellini, and Perry Roland. Verovio: A library for engraving MEI mu- sic notation into SVG. In Proceedings of the 15th International Society for Music Information Retrieval Conference, pp. 107-122, 2014. doi: 00.5281/zenodo.144
John Rink. Digital editions and the creative work of the performer. Nineteenth- Centrrr Music Review, 18(1):11-81, 2021. doi: 10.1017/$1479409911 0007794498190000033
Antonio Ríos-Vile David Rizo, José M lñestt,, and Jorge Calvo-Zaragoza, End-to-end optical music recognition for pianoform sheet music. International Journal on Document Analysis and Recog- nition (IJDAR), 26(3):347-362, Sep 2023. doi: 1110077/100032-22--004444
Antonio Ríos- Rila,Vila, Jorge Calvo-Zaragoza, and Thierry Paquet. Sheet music transformer: End- to-end optical music recognition beyond monophonic transcription. In Elisa H. Barney Smith, Marcus Liwicki, and Liangrui Peng (eds.), Document Analysis and Recognition ICDAR 2024, pp. 20-37, Cham, 2024. Springer Nature Switzerland. ISBN 9783031705526. doi: 10. 978-3-031-70552-6.2
Antonio Ríos- Vila, Jorge Calvo-Zaragoza, David Rizo, and Thierry Paquet. End-to-end full-page optical music recognition for pianoform sheet music, 2024. URL https::/arxiv.. org/
abs/2405.12103
Craig Stuart Sapp. hum2abc/
hum2abe, 2012. URI. https: /ttt:///extrrs... www.huu.wrg...ooooo..t..
Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Katrin Erk and Noah A. Smith (eds.), Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers). pp. 7725-172, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10. 10.188 16653/1//P1--.. 1766-1122 1162.

---

Published as a conference paper at ICLR 2026
W.G. Vree, M. Tarenskeen, N. Liberg, Paul Villiger, Janus Meuris, Larry Myerscough, Dick Jackson, Jan Wybren de Jong, and Mark Zealey, xml2abc 147, 2018. BRLLtttp::/// www. vree vrge.org/
html
Chris Walshaw. The abe music standard 2.1 (2011), 2011. URL SKK[sttps////nchheeeekkk com/abct oww.tolccoooooooooooooooooooolla
Shangda Wu and Maosong Sun. Exploring the efficacy of pre-trained checkpoints in text-to-music petttpss://axllvorgggg.o
generation task, 2023.
11216
Shangda Wu, Yashan Wang, Xiaobing Li, Feng Yu, and Maosong Sun. MelodyT5: A unified score- to-score transformer for symbolic music processing. In Proceedings of the 25th International Society for Music Information Retrieval Conference, pp. 642-650, 2024. doi: 14877419.
Weihan Xu, Julian McAuley, Taylor Berg- Kirkparikk,,rk Shlomo Dubnov, and Hao-War Dong. Gen- erating symbolic music from natural language prompts using llm-enhanced dataset, 2024. URL
https::/arxivy. www.oogg///ooo http:://///rxv..oor///////////// 042044
Kaizhong Zhang and Dennis Shasha. Simple fast algorithms for the editing distance between trees and related problems. SIAM Journal Computing, 1845):1245-1262. 1989. doi: 10.1137/
8082.
A BASELINE SETTINGS
A.1 SMT++
There are two available checkpoints for SMT++: antoniorvv/smttpp catop__moaatterr and antoniorv6/nan
the checkpoint antoniorv6/ httpeniirvossmpppppprpse https/ttpppoooootteeeeeeeee because its training data, the Mozarteum dataset (Rink) 2021. more similar to evaluation dataset. This choice is favorable toward SMT++.
scores. For evaluation,
+ at. 2024) claimed "we used a 5-fold cross-validation" for evaluation on Mozar- realistic evaluation since only shows the capability
teum and Polish Digital Scores. This of the model to overfit the distribution of these two datasets, and it's not the performance of pre-trained model on real-world data. We do not know which portion of the dataset their released checkpoints were trained on.
For generation hyperparameters,, we use the default setting from SMT++'s repository https:// and set max generation length to 2048.
/ httpu//.....nnttooooooo/////////
github.
A.2 GPT-5
Legato's performance was compared against the gpt-55-202500-000 of GPT-5. Due to the full- page setting, found that without additional prompting, GPT-5 would often avoid answering and resort to outputs such as
The notation in the provided image is not legible enough to transcribe accurately. Please upload a higher-resolution image or closer crop so the notes, rhythms, and accidentals can be read.
Therefore, we added the following system prompt to make sure GPT-5 always outputted valid ABC.
You will be given an image of sheet of music. Transcribe it into valid ABC 2.1 notation. Try your best to transcribe and make reasonable guess if the image is not clear.
Output only the ABC (no explanations), preferably inside single "aab fenced block.
IMPORTANT: NEVER give outputs like: Unable to transcribe from the provided image due to insufficient resolution/clarisy. If you can't tell, give your best guess.
12

---

Published as a conference paper at ICLR 2026
!! JALNAYS OUTPUT VALID ABC, DO NOT GIVE ANY ENGLISH OUTPUT. GIVE YOUR BEST GUESS YOU ARE NOT SURE!
This is the default setting and used for evaluation in Table | on purpose VLMs, we refer readers to appendix B.4
For more optimization and evaluation
B ADDITIONAL EXPERIMENTAL RESULTS
B.I ABLATION STUDY
To further quantify the improvements brought by different design consideration of Legato, duct an ablation study on IMSLP Piano Scores. In addition to the original SMT++ and Legato as shown in Table we train the following models:
SMT++ ABC Full-Page GrandStaff
SMT+ ABC Full-Pagee GrandStaff'
SMT++aage on PDMX-Synth
SaT++aagge is an enlarged version of SMT++ with in total 177M trainable parameters. In contrast, Legato has 107M trainable parameters. For the dataset, we converted the "kkem-bbsed Full-Page GrandStaff used by SMT+ into ABC, and follow the exact same training paradigm, including syetem-level pre-training and curriculum fine-tuning. We use the same training hyperparameters for SMT++, and for SMT++aarge we pick the best learning rate from (le - 4, 3e 5, le 5, 3e 6).
For generation, we follow the setting of SMT++. The evaluation results are shown in Table 2
Table 2: Ablation study on IMSLP Piano Scores. Lower better for all metrics. All metrics are explained in |5| f: Following SMT++ style of data processing, i.e., syetem-level pre-training and curriculum fine-tuning.
Architecture
SMT++ GrandStaft ** **kern
SMT++ GrandStaff ABC
SMT+haage GrandStaff ABC
SMT Haage
Legato PDMX- -ynth ABC
Training Dataset
PDMX-Synth ABC
Format & Tokenizer
IMSLP Piano Scores (32 pages)
TEDn OMR-NED
96.7
74.6
73.0
80.5
29.7
91.9
70.9
82.9
96.2
44.3
Comparing the columns of SMT++ GrandStaff of **kern and ABC, observe clear improvement. This improvement is brought by the usage of ABC format and our BPE tokenizer, verifying that ABC + BPE tokenization is practical setting for training auto-regressive models for OMR.
We then discovered that SMT++ failed to scale there is no improvement by either enlarging the model size or data size, even though we carefully tuned the hyperparameters. We think that SMT++aage has too many parameters and easy overfit the artifacts in the training data, as we observe a low validation loss but high testing errors. Legato, on the other hand, works on the robust features encoded by the pre-trained vision encoder, and thus is more generalizable to the test data, even if only trained synthetic scores. As a conclusion, the improvement arises from the synergy between the Legato pre-trained encoder, the Legato architecture, and the larger, more diverse PDMX-S -yytt-Sytthha dataset.
B.2 LEARNING CURVE OF LEGATO
To validate the effect of data scaling Legato Figure The curves show a consistent decrease in validation loss and error rates as the number of training steps increases, indicating effective learning and improved generalization. The metrics eventually plateau around 55K training steps, suggesting that Legato has largely converged by this point and that additional training yields diminishing returns.
model's performance, we present the training curves of
13

---

Published as a conference paper at ICLR 2026
vs Training Step
SER, Training Step
Enror Rate (%)
Loss
10000
22003
30020
42000
50320
C0002
70000
10200
20002
30200
40000
52000
00020
75000
Training Step
Training Step
Figure 4: Learning Curve of Legato. The validation loss and emrr rates are tested on the validation split of PDMX-Synth.
B.3 CONVERSION FAILURE RATES
We show in Table the conversion failure of **kern MusicXML SMT++'s outputs. Notice that all outputs of Legato can be successfully converted to MusicXML.
Table 3: Failure rates of #=kern-to-MusiiXMML. conversion on SMT++'s outputs.
Dataset
Total Files
Failed Files
Failure Rate
Camera OpenScore String Quartets
252
234
92.9%
Rendered OpenScore String Quartets
252
224
88.9%
Camera OpenScore Lieder
64
58
90.6%
Rendered OpenScore Lieder IMSLP Piano Scores
64
60
93.8%
32
29
90.6%
We observe that the failure rates for Mokerm-to-MusiccMMM conversion are very high. This is primar- well-formed. Moreover, the * **kern format is highly
ily because most outputs from SMT+ susceptible to errors - for example, the omission of a tab character can make difficult to repair the output. The ABC format, on the other hand, is more robust to such errors.
B.4 RESULTS ON GENERAL-PURPOR VLMS
To further investigate the capability of general-purposss VLMSs on the task of OMR, we conducted experiments with some prompt variants. This include: (i) adding in-context learning examples, and (ii) prompting models to output MusicXML directly. We include the results in Table
For in-context learning, we include three examples randomly picked from PDMX-Synth training split. Specifically, we prompt the VLMs with three rounds of conversations follows:
User: Transcribe this score to ABC. <In Context Example Image 1> AI: <Groundtruth for ICE 1>
User: Transcribe this score to ABC. <In Context Example le Tmage 2> AI: <Groundtruth for ICE 2>
User: Transcribe this score to ABC. <In Context Example Image 3>
AI: <Groundtruth for ICE
User: Transcribe this score to ABC. <Test Image>
The system prompt used to prompt VLMs to generate ABCs is the as the prompt in ap- pendix $A.2 The system prompt we used to prompt VLMs to generate ABCs when given in context examples given follows:
14

---

Published as a conference paper at ICLR 2026
You will be given an image of sheet of music. Transcribe it into valid ABC 2.1 notation. Try your best to transcribe and make reasonable guess if the image is not clear.
You will be given three in context examples of image-transcription pairs Output only the ABC (no explanaticns), preferably inside single aaac fenced block.
IMPORTANT: NEVER give outputs like: 'Unable to transcribe from the provided image due to insufficient resolution/clariey) If you can't tell, give your best guess.
!!!ALAYS OUTPUT VALID ABC, DO NOT GIVE ANY ENGLISH OUTPUT, GIVE YOUR BEST CUESS IF YOU ARE NOT SURE!!!
The system prompt we used to prompt VLMs to generate MusicXML is similar:
You will be given image of sheet of music. Transcribe into valid MusicXML netation. Try your best to transcribe and make reasonable guess the image is not clear.
Output only the MusicXML (no explanations), preferably inside a single "'fnnnced block.
IMFORIANT: NEVER give outputs like: 'Unable to transcribe from the provided image
to Insufficient resolution/clarity.. If you can't tell, give your best guess.
!!!AIWAYS OUTPUT VALID MusicXMI, DO NOT GIVE ANY ENGLISH OUTPUT. GTVR YOUR BRST GURSS IF YOU ARB NOT SURE!!!
For in-context learning, only tested it with ABC outputs because MusicXMLs orders of magnitude more verbose than ABCs (as they include specific rendering and metadata information well), and within budget only test in-context learning with ABC notation.
Table 4: General-purposs VLMs' ability on IMSLP Piano Scores.
Model
GPT-5
Gemini 2.5 Pro ABC MusicXMI.
Output Format
ABC
ABC
MusieXML.
ABC
In- In-Context Learning
IMSLP Piano Scores
TEDn
96.7
74.7
86.6
87.8
78.6
80.3
OMR-NED
98.7
93.2
94.3
95.4
93.7
94.3
From Table we observe that for both GPT-5 and Gemini 2.5 Pro, either in-context learning or prompting with MusicXMI. leads improved performance. This finding is consistent with the observation that MusicXML resources are more prevalent than ABC notation on the internet, and that providing few ABC examples further facilitates VLMs understanding of this less common format.
B.5 FOCUSED COMPARISON ON OPENSCORE STRING QUARTETS VIOLIN PARTS
We carry out a focused comparison to reduce the differences among models. We observe that SMT++ consistently produces two-staff outputs, as it is trained exclusively on piano scores. In this comparison, we manually retain only the two violin parts in the ground truth. Additionally, since SMT++ is trained on Verovio-rendered images, we also render the input images using Verovio. Fur- thermore, we report **kern error rates and assign empty strings to instances where Legato's output cannot be converted to **kern, further biasing the evaluation in favor of SMT++. As shown in Table 5 even under these evaluation settings extremely favorable to SMT++, Legato still outper- forms it. Legatogmall underperforms SMT+ on **kern CER and SER due to conversion failure. On TEDn, which we believe a more reasonable and comparable metric, it is far superior, though not strong as the larger Legato model.
15

---

Published as a conference paper at ICLR 2026
Table Results on OpenScore String Quartets violin parts. Lower better for all metrics. TEDn is the primary metric, requiring outputs to be converted to MusicXML. ToDneo evaluated only on instances where SMT++ produces outputs that can be successfully converted to MusicXML; Legato outputs always converted to MusicXML successully OMR-NED is format agnostic, built on extraction symbols from any format. For OMR-NED, **kern outputs are automatically corrected for syntax errors, while ABC is first converted to MusicXML (again, always successful in practice) before symbol extraction. t: 10.2% of Legato's 's outputs and 27.9% of LegatOsmall 's outputs are not convertible to **ern, challenging dataset, since it has much denser score images. All metrics are explained in $
empty prediction used. OpenScore String Quartets is the most
Metric
SMT+
Legato
LegalOamall
Rendered String Quartets Violin Parts (256 pages)
CERee
30.7
16.7
50.6
SERRkrn
42.6
19.1
55.5
1gE.R
75.2
29.2
73.5
TEDa
95.0
7.7
35.0
TEDnemwerr
64.5
8.7
35.9
OMR-NED
71.2
15.1
46.2
Error Rates vs. Aspect Ratio
1100000
Number of Samples in Training Set
0.8
Error Rate (Micro Average)
80000
60000
0.4
40000
CER
SER
LER
20000
Number of samples
0.0
1:0
1:1
1:2
1:3
1:4
1:5
1:6
1:7
1:8
1:9
1:10
Aspect Ratio
Figure 5: ABC error rates on PDMX-Synth test input with different aspect ratios. Error rates are reported by averaging over each bin. Legato is capable of recognizing multi-page scores.
B.6 MULTI-PAGE PERFORMANCE
bbreaks down the Legato етror rates (micro averaged) across different aspect ratios, showing that multi-page inputs are more challenging; it also shows they are much less frequent in the data. These inputs often result in long sequences that are truncated before reaching the decoder, limiting model performance.
B.7 QUALITATIVE EXAMPLES
Figure shows an example from IMSLP Piano Scores and output from Legato and SMT++, with errors marked in red. In this brief example-chosen chosen where the TEDneonvert to the average TEDn score for each model-we can see that Legato had issues distinguishing between 16th rest and an 8th rest, and 32nd rest and 16th rest, as well instance of
close
16

---

Published as a conference paper at ICLR 2026
Input Image
Legato
TEDn:
SMT++
TEDn: 66
Figure 6: Example (first system of Duetto No. I in E minor by Bach, BWV 802) from IMSLP Piano Scores (top), with output from Legato (middle) and SMT++ (bottom). Errors are marked in red boxes.
mistaking a natural sign for a sharp. The SMT++ output incorrectly detects the time signature and the bass clef key signature, and in these 4 measures, accidentals are either missing or incorrect- although we do note that in the final measure of our example, the initial C# is included in the key signature for that line, and the omitted sharp is likely a limitation of **kern rather than the system output.
We include more examples in Figure 7 These examples do not come from our evaluation dataset but instead were chosen as examples of famous piano compositions. We pick scanned typesetring version that was not rendered by MuseScore, abcm2ps or Verovio. To account for the fact that SMT++ was trained only on piano data, we select only piano works for this additional example set. Additionally, we isolate a single system from the score onto blank page for input. This provides better visualization, despite the fact that both SMT++ and Legato are capable of handling multi- system inputs.
Figure shows the results on the first line the first movement of Mozart's piano sonata K. 545. Like many beginner piano pieces, the input image very clean, with clearly separated voices. In this particular rendition, the trill symbol is rendered a different font than Verovio's 's default, and thus unrecognizable SMT++. In contrast, Legato can adapt various fonts because the multiple different typesetting options in PDMX can be rendered with MuseScore, and thus are contained in the training data. Moreover, Legato correctly identifies all the slurs, while SMT+ misses them. Both **kern and ABC represent slurs using paired parentheses (), indicating that the training data for Legato also includes diverse types of slurs. This enables Legato to recognize slurs in the input image, even though it was not rendered with MuseScore, abem2ps, or Verovio.
Figure 7b shows the results on the first system of Chopin's Op. 10 No. 4, widely regarded as an essential étude for advanced piano study. This score is more complex than the Mozart in Figure 7a and contains multiple voices. Furthermore, the version we selected contains distortions and artifacts from the print-and-scan process, making the input more difficult for OMR. SMT++ produces an output with excessive voices, broken beaming, and inconsistent measure lengths. Some quarter rests misaligned with eighth notes, and the clef change appears twice, making the output impossible to render with existing software. To visualize it, we manually recognized the symbols from SMT++'s output and annotated them in MuseScore. Although there are some errors in Legato's output (such as
17

---

Published as a conference paper at ICLR 2026
Figure 7: Qualitative amples of piano scores. Since SMT+ is trained only on piano data, we illustrate results on well- known piano works. All score images are sourced from IMSLP (not rendered by MuseScore, abcm2ps, or Verovio) and cropped to a single system for clar- ity, though both SMT++ and Legato can process full pages. The examples progress from easy (a), to difficult (b). ill-formed input (c). TEDn values are reported at system-leve level. Since the number of errors too large for some of the examples, we did not mark them with red boxes to pre- serve readability.
Input Image
0.00
Legato
SMT+
TEDm: 5.63
(a) Mozart: Piano Sonata No. 16 in C major, K. 545
Presto 2- 88
Input Image
of con froce ofy
TEDn: 1.
Legato
SMT++
TEDn: 55.34
(b) Chopin: Étude Op. 10, No. 4 in C2 minor
Input Image
lest
TEDn: 31.43
Legato
26.
24
TEDm: 14
SMT++
(c) Liszu Erlkönig (S. 558/4), arrangement Schubert's D. 328
18

---

Published as a conference paper at ICLR 2026
an ill-formed first measure), the overall quality is much better, and most fingerings and articulations are correctly detected.
Figure 7c shows system from Liszt's Erlkönig. Without the context of the full score, the excerpt appears "ill-formed" because the triplet symbols are omitted. The imput thus shows a quarter note in the upper staff aligned with three eighth notes in the lower staff an unusual case absent from the training data. In addition, the eighth rests in the lower staff are missing in the second through fourth measures. While an experienced pianist can easily interpret this, the models infer literal eighth notes and rests rather than tuplets, which accounts for most of the TEDn errors. Nevertheless, Legato still outperforms SMT++ by accurately capturing the pitch of most notes and the duration of the notes in the upper staff. Moreover, Legato can also predict the placement of lyrics and pedal symbols, which lie beyond SMT++'s 's capabilities.
19


Reviews:
[{"review_id": "0U5Z9TCtka", "reviewer": "Reviewer_H56V", "summary": "This paper presents a comprehensive solution for OMR on long and complex real-world music sheets, including the collection and preprocessing of a large-scale dataset, as well as an end-to-end recognition model, LEGATO. Experimental results demonstrate that LEGATO exhibits superior performance and generalization capabilities, outperforming both prior specialized OMR model SMT++ and general-purpose vision-language model GPT-5.", "questions": "1. When calculating the TEDn metric, did the authors attempt to have GPT-5 directly output MusicXML? If so, were there any changes in the results?\n2. Have the authors attempted more data augmentation strategies such as adjusting brightness, applying affine transformations, or adding noise—on the rendered score images in the training data to better align them with real-world scenarios?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The PDMX-Synth dataset curated in this paper addresses the scarcity of real-world, complex music score data for OMR. Its public release would significantly benefit research in this field.\n- Experimental results demonstrate that LEGATO significantly outperforms both SMT++ and GPT-5, with strong performance on real-world scenarios (camera versions of OpenScore String Quartets and OpenScore Lieder)—highlighting the effectiveness and practicality of the proposed approach.", "weaknesses": "1. Limited in performance analysis of general VLMs. \n    1. The paper relies solely on GPT-5 as the representative general-purpose vision-language model for comparison, which somewhat limits the persuasiveness of the evaluation. Including additional state-of-the-art multimodal models—such as those from the Gemini, Claude, and Qwen families—would provide a more comprehensive and robust assessment.\n    2. The evaluation of GPT-5 lacks sufficient detail. The paper does not specify the prompts provided to the model, nor does it indicate whether exemplars were used in the input to optimize the model's output. Furthermore, the absence of a comparative case study between the outputs of GPT-5 and LAGETO prevents a thorough analysis of GPT-5's deficiencies. This raises concerns about whether the authors have adequately explored the capabilities of the general VLMs.\n2. Lack of experimental analysis regarding the training data. The large-scale dataset introduced in this paper constitutes one of its core contributions, and considerable space is devoted to detailing the dataset’s collection and preprocessing pipeline. However, the paper does not include experimental comparisons evaluating the impact of the proposed dataset versus existing datasets on model performance (although the new dataset is strongly likely to yield better results). Moreover, the authors do not provide empirical ablation studies to assess the effectiveness of their data cleaning and data augmentation strategies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Mi6EhFa5r1", "reviewer": "Reviewer_aT3R", "summary": "The paper introduces Legato, an end-to-end OMR system that converts full-page/multi-page typeset score images into ABC notation using a frozen pretrained vision encoder (Llama-3.2-Vision) and a lightweight transformer decoder with a multimodal projector. Training uses a new 214K-image synthetic corpus (PDMX-Synth). Evaluation is standardized via MusicXML conversion and measured with TEDn and OMR-NED, where Legato reports large absolute error reductions (e.g., −68% TEDn, −47.6% OMR-NED) over prior work (e.g., SMT++). The paper also presents a data-driven tokenization that appears to learn composite musical concepts (e.g., triads, short phrases).", "questions": "Why better than SMT++?\nPlease quantify the contribution of (a) ABC output vs. MusicXML/kern, (b) data scale (learning curves), (c) using a pretrained vision encoder (frozen vs. finetuned), and (d) decoder capacity. A controlled ablation matrix would clarify attribution.\n\nCase studies & perceptual relevance.\nProvide score-level examples where Legato reduces TEDn/OMR-NED, with before/after renderings and (ideally) audio renderings to illustrate musically meaningful fixes (e.g., rhythm beaming, voice assignment, chord spelling).\n\nHandwritten & degraded inputs.\nHow does Legato perform on handwritten scores (e.g., MUSCIMA++, historical manuscripts) or noisy scans? Any domain-adaptation strategy planned?\nPlease include perturbation tests (resolution, blur, skew, lighting, JPEG artifacts) and transposition/key/time-signature shifts to probe symbol and structure invariance.\n\nData hygiene & licensing.\nClarify licensing of PDMX-Synth sources and safeguards against train-test leakage across editions/engraving variants of the same piece.\n\nExtending beyond rhythm/pitch.\nDo you plan to evaluate semantic elements (lyrics alignment, articulations, ornaments, dynamics) and non-Western notation to support broader claims of generalizability?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Clear problem focus & scope. Tackles practical, full-page OMR with multi-system/stave layouts—beyond monophonic or single-system settings.\n\nStrong empirical gains. Large improvements on TEDn/OMR-NED across several datasets, including an IMSLP sample.\n\nStandardized evaluation. Using MusicXML as a unifying target for comparison reduces format bias; reporting ABC/kern helps triangulate.\n\nScalable training recipe. Pretrained vision encoder + compact decoder yields a seemingly efficient path to high accuracy.\n\nTokenizer insight. Data-driven tokenization that captures chords/phrases is promising and could simplify decoding of frequent musical patterns.", "weaknesses": "Attribution of gains is unclear. It’s hard to disentangle where improvements come from: ABC target format, frozen VLM encoder, data scale (214K), or architecture. A dedicated ablation is missing.\n\nEvaluation confounds. Conversions among ABC/ kern/MusicXML may introduce asymmetric errors; conversion failure rates and their impact on metrics are not reported.\n\nMetric interpretability. TEDn/OMR-NED reductions are compelling but lack qualitative error analysis or audible case studies linking metric changes to musically meaningful differences.\n\nGeneralization gaps. Focus is on typeset scores; robustness to handwritten, degraded scans, complex lyrics/ornaments, and non-Western notation is not demonstrated.\n\nRobustness & reliability. No tests on scan noise, resolution changes, staff skew, lighting artifacts, transposition/key/time-signature shifts, or symbol vocabulary tails.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7ieTqqOlS3", "reviewer": "Reviewer_zT9i", "summary": "- This paper proposes a large-scale end-to-end generalizable approach to typeset OMR. The authors created paired image–ABC data from the PDMX dataset. The training system adopts a vision encoder and language decoder architecture. The proposed system is capable of recognizing full-page and multi-page typeset music scores into ABC notation. Although there is a lack of research on comparing different vision encoders and extending the LLM decoder to larger, the overall framework shows its effectiveness. The results are promising. I like this work.", "questions": "- How would different vision encoders affect the recognition performance?\n\n- As Table 1 shows, Legato outperforms Legato_small by a large margin. Would increasing the decoder size further improve system performance?\n\n- Will hyperparameters, such as temperature affect the prediction performance? Do authors choose the best or are there randomness in next token prediction?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- OMR is an interesting and important research topic in music information retrieval with strong ongoing research interest.\n\n- The proposed vision encoder and Transformer decoder form a well-designed architecture for OMR.\n\n- The system is trained on a large-scale dataset of 214k images, demonstrating strong generalization ability.\n\n- The procedure for creating the dataset is well-designed and systematic.\n\n- Comprehensive experiments are conducted across a range of datasets, showing state-of-the-art performance.\n\n- The training and testing datasets are separate, further validating the system’s generalization capability.\n\n- The recognition results shown in the Appendix are impressive and of high quality.", "weaknesses": "- The proposed PDMX-Synth dataset is derived from the PDMX dataset; therefore, it is effectively a subset and smaller in scale. Will this affect the performance?\n\n- The model architecture shares similarities with the design of SMT++, indicating limited novelty in architectural design.\n\n- There is limited comparison of different vision encoders and more decoder sizes, leading to the technical part a bit weak. \n\n- There is a lack of equations, such as autoregressive prediction, and loss functions to show how the LLM is trained. \n\n- The paper lacks discussion on human manuscript transcription — how would the system’s performance degrade in that case?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HRVwDey0ur", "reviewer": "Reviewer_qebB", "summary": "This paper proposes Legato, the first large-scale end-to-end optical music recognition (OMR) model capable of transcribing multi-page typeset scores into ABC notation. Legato combines a frozen pretrained vision encoder from Llama 3.2 with a trained transformer decoder, processing segmented images rather than requiring pre-split pages or systems. The authors construct PDMX-Synth, a dataset of 238,386 image-ABC pairs derived from the PDMX dataset. Evaluation on multiple held-out datasets shows that Legato substantially outperforms Sheet Music Transformer++ on the TEDn metric.", "questions": "No further question", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Legato is the first model to handle multi-page typeset scores end-to-end without the need of pre-splitting scores into pages or systems.\n* The evaluation results show substantial improvements across multiple datasets and metrics, particularly on realistic camera images.\n* Publicly releasing implementation code and PDMX-Synth dataset with 238k (multi-page typeset score image, ABC) pairs is beneficial to the community.\n* Multiple held-out datasets for the evaluation.", "weaknesses": "* Legato is essentially multimodal Llama with a smaller transforemer decoder for OMR task. It has very limited architectural novelty.\n* Applying BPE tokenization needs more justification than just providing 4 examples in Figure 3. and writing the tokenizer 'captures some composite musical concepts.'  Since ABC-notation have limited number of symbols and combinations, there could be a more efficient domain-knowledge-based method of tokenization. To support BPE tokenization, authors need to provide more systematic vocabulary analysis and comparison with expert-defined tokenization.\n* The comparion between Legato and SMT++ heavily relies on the format conversion, which cannot ensure intergrity of the output.\n* Legato and SMT++ have drastically different number of parameters. Additionally Legato includes pre-trained vision encoder compared to SMT++'s vision encoder, which is trained from scratch.\n* Since the main difference of Legato and SMT++ is the vision encoder part and SMT++'s implementation is publicly available, the authors could have applied an enlarged SMT++ model with a matching number of parameters in the decoder trained on the same PDMX dataset.\n* The GPT-5 evaluation's prompting strategy could be explored more thoroughly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)