# Optical Medieval Music Recognition--A Complete Pipeline for Historic Chants
## 3 Related Work
SIMSSA utilizes a segmentation network for the layout approach, which segments staff lines, layout, symbols, and text in a single step. This has the advantage of requiring only one network for the segmentation task. However, the disadvantage is that compared to separate steps, more training data are needed, and pixel-perfect annotation is required, which is very time-consuming. Additionally, these models often need to be retrained for new books as they do not generalize well. In general, specialized algorithms typically yield better results and generalize better (e.g., staff line recognition generalizes well and usually does not need retraining when applied to other books). For symbol classification, SIMSSA uses a k-nearest neighbor approach. This method involves categorizing small patches of symbols into predefined classes based on their similarity to existing samples in the dataset. For text recognition, Calamari [10] is employed and aligned with texts from the Cantus database [11]. SIMSSA is more of a workflow than an automatic pipeline, as it connects various web tools to transcribe such documents. It is unclear to what extent the detection of chants in books or syllable note assignment is supported or if they need to be manually annotated. In the following, the individual steps of the OMR pipeline are examined in more detail. Table 1 gives an overview of existing publications that cover parts of the OMR pipeline. [PERSON] [12] segments staff line pixels by passing a square patch centered around the pixel to a Convolutional Neural Network (CNN). The CNN is trained by exposing both patches containing centered staff line pixels and those devoid of them. As a result, the CNN learns to differentiate between staff line pixels and non-staff line pixels (by learning distinctive features of patches with/without centered staffline pixels). Using this method, the authors receive a \(F_{1}\) score between 97% and 99%. Similar to this, [PERSON] et al. [13] used an fully convolutional neural network (FCN) with an encoder-decoder architecture like a U-Net [14] to generate a binarized image where only staff line pixels remain. Afterward, a post-processing algorithm is used to extract the staff lines detection (\(F_{1}^{D}\)) 99.7% of all staff lines. The length of the detected lines fitted the GT by 98.2% (\(F_{1}^{Lf}\)). [PERSON] et al. [15] analyzed the performance of an object detection neural network using a Mask-RCNN [16] on complex handwritten documents for the layout detection task. Therefore, several experiments were conducted on different datasets including datasets with musical manuscripts. Instances of three different region types (music, lyrics and drop capital) had to be recognized, resulting in a mean Intersection-over-Union (IoU) score of 89.5%. Similar to that in [17] a segmentation approach was used to classify each pixel in an image to a layout type and afterward extract the regions using a post-processing algorithm. [PERSON] [18] has also worked from an object detection perspective and conducted several experiments to test common layout metrics and architectures. It was found that commonly used object detection metrics such as mean average precision (mAP) [19] are not necessarily suitable, as they do not correlate with the final result of an OMR system. On the symbol level, there are comparatively more publications. On the whole, there are three different approaches to performing symbol recognition: as an object detection (recognition) task, as a segmentation task, or as a sequence-to-sequence task (similar to OCR). [PERSON] et al. [20] approached the problem as a segmentation task and employed Convolutional Neural Networks for the automation of historical music image document processing. The dataset was limited in size and comprised square notation. A classification network was implemented to assign one of four classes to each pixel in the image. Small patches extracted from the pages centered at the point of interest were used as input to the network. Similarly, the work of [PERSON] et al.