# Optical Medieval Music Recognition--A Complete Pipeline for Historic Chants
## 1 Introduction  
Optical music recognition (OMR) is a technology that enables the automatic conversion of sheet music into a digital format. This technology is particularly useful for preserving and digitizing historical musical documents, which are often fragile and subject to deterioration over time. Since the majority of musical pieces in the Western culture have been composed in written form, their conservation and digitization through manual methods is a laborious time intensive and often error-prone process. OMR is one of the key technologies to accelerate and simplify this task automatically. To overcome these challenges, OMR systems typically incorporate a combination of image processing, machine learning, and music theory to accurately capture and decode the musical notation. When working with well-preserved documents, modern OMR systems can achieve high levels of accuracy. By contrast, in historical documents, those OMR systems still struggle to achieve such good results. The reasons are several challenges associated with OMR on historical documents. Firstly, the quality of the source material can greatly affect the accuracy of the recognition process. Historical documents may be faded, smudged, or torn, making it difficult to accurately capture and decipher the musical notation. Secondly, many historical musical documents are quite heterogeneous in layout and notation styles, which requires the development of different solutions. Due to thesechallenges, there are a limited amount of training data available for historical documents, which in turn constrains the range of potential solutions. Due to the complexity of optical music recognition (OMR), it is typically divided into multiple sub-steps in order to effectively address each component. The general approach to OMR involves the following stages: (1) Preprocessing and deskewing, which prepares the input image for further analysis, (2) staff line detection, which identifies the horizontal lines upon which musical symbols are placed, (3) layout detection, which identifies the spatial organization of music regions and other notations on the page, and (4) symbol detection, which locates and classifies individual musical symbols. For the transcription of historical charts, there are additional steps that are rarely addressed in the literature. These include (5) recognizing the text, (6) dividing it into syllables, and (7) assigning each syllable to its corresponding note or notes. Finally (8), using the previously extracted data, the document is reconstructed and the target encoding is generated. The main focus of the paper is the integration of a fully automated OMR pipeline covering all stages that can be utilized for mass transcription purposes on historical documents in the context of the Corpus Monodicum project ([[https://corpus-monodicum.de](https://corpus-monodicum.de)]([https://corpus-monodicum.de](https://corpus-monodicum.de)), accessed on 10 August 2024). The pipeline focuses on historical documents, e.g., manuscripts written in square notation, which was developed in the 11 th-12 th century [1]. This notation includes several staves on a page, with each staff typically containing four lines and a clef marker at the start. An example image of such documents can be seen in Section 4. The primary contributions of this paper are as follows:  
1. Development of a comprehensive automatic pipeline tailored for digitizing handwritten historical monophonic music depicted in squared notation. The pipeline processes input from a book and produces digitized chants (in formats such as MusicXML [2], MEI [3], Monodi [4]) corresponding to the content of the book. Additionally, position-related data are extracted, including the positions of the staff lines, layout, symbols, and syllables/lyrics on the page to support manual post-correction and training. 2. Extended layout analysis capabilities, facilitating the detection and extraction of chants within the book, such as individual chants, which may extend across multiple pages. 3.