Robust optical recognition of handwritten musical scores based on domain knowledge

For centuries, music has been shared and remembered by two traditions: aural transmission, common to folk and popular musical genres, and in the form of written documents normally called musical scores. Prior to music typographical systems, all music was copied manually including large scores and each and every part for the players and singers. Publishers have typeset a large body of historical musical scores, principally from what is known as classical music. However, there remains a substantial and important corpus of works that exist as original hand-written manuscripts (or facsimiles of these manuscripts such as photocopies). The problem is not restricted to historical documents: many contemporary compositions also exist in hand-written or facsimile format. These important cultural artifacts are in danger of being lost through the normal ravages of time. To preserve the music (rather than the documents themselves) requires some form of typesetting or, ideally, a computer system that can automatically decode the symbolic images and create new scores. Meaning, the documents must be digitized and transformed into machine-readable format. Programs analogous to optical character recognition systems called optical music recognition (OMR) systems have been under intensive development for many years. However, the results to date are far from ideal. Each of the proposed methods emphasizes different properties and therefore makes it difficult to effectively evaluate its competitive advantages. This thesis provides a study and an overview of the literature concerning the automatic analysis of images of printed and handwritten musical score including an introduction to OMR processing systems. After an image preprocessing stage, an OMR system is typical divided into three modules: (1) music symbol recognition, (2) musical notation reconstruction and (3) final representation construction. For the binarization operation, several options have already been proposed in the past. However, in OMR, researchers still prefer using standard binarization procedures, such the Otsu’s method. No goaldirected studies for music sheets have been carried out. In this thesis, a novel binarization algorithm is presented. The process is based in the knowledge of the content of the image. The method uses the estimation of the staff line thickness and the vertical distance between two staff lines extracted directly from the gray level music score to guide the binarization process. The next operation in the OMR is the detection and subsequent removal of staff lines. In this work, a general-purpose, knowledge-free method for the automatic detection of music staff lines based on stable paths approach is proposed. One of the final steps of an OMR system encompasses the segmentation and classification of the music symbols. For the detection of these various objects, two procedures are discussed: the first music symbols segmentation’s method is based in contextual information and music rules; the score is first split by staffs and the symbols are divided considering their graphical position and geometric features; the second method performs simultaneously the segmentation and the classification. In the music symbol classification, a comparative analyses divided into three distinct studies is described. Five different pattern recognition procedures are examined. In the first study, the database of scores is augmented with replicas of the existing patterns, transformed according to an elastic deformation