# Optical Medieval Music Recognition Using Background Knowledge
## 7 Layout Analysis  
Since the symbol post-processing pipeline partially uses the layout, e.g., drop capitals or lyric regions of a page to correct symbols, this section briefly presents the layout analysis of the system. The aim of the layout analysis is to split the page into music regions and text regions. Text regions can be further split into lyric, paragraph and drop capitals. However, this is not mandatory. Figure 5 shows an example of such a layout recognition. The pipeline consists of two simple steps and uses the encoded staff lines of the dataset:  
1. **Music-Lyric**: The bounding box for each encoded stave on a page is calculated and is padded with \(d_{SL}/2\) on the top and bottom side of the box. These are marked as music regions. Regions that lie between two music regions are marked as lyric regions. Since the bottom lyric line is not between two music regions, it is added separately. To do this, the average distance between two music regions (staffs) is calculated and afterwards used to determine the lowest music region. 2. **Drop-Capital**: Drop capitals (compare Figure 5) often overlap music regions, so these must be recognized separately. The pipeline uses a Mask-RCNN [21] with a ResNet-50 [19] encoder to detect drop capitals on the page. 1. **Training:** Since only limited data are available, the weights of the model trained on the COCO dataset [22] is used to fine-tune the model. For training, only horizontal flip augmentations are used. The network is then fine-tuned on the \"Latin 14819\" dataset. A total of 320 instances of drop capitals are included there. The model is trained with positive (images with drop capitals) and negative (images without drop capitals) examples. Inputs are the raw documents. SGD was used as the optimizer, with hyperparameters of learning rate = 0.005, momentum = 0.9 and weight decay = 0.0005. 2. **Prediction:** The raw documents are the input. A threshold of 0.5 is applied to the output of the model. Next, the output is used to calculate the concave hulls of the drop capitals on the document. If hulls overlap, all but one are removed and only the one with the smallest area is kept. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Id** & **Encoder** & **Decoder** & **Parameters** & **ImageNet-Weights** \\ \hline FCN & [12] & [12] & 603,649 & None \\ U-Net & [1] & [1] & 31,032,265 & None \\ MobileNet & [18] & Own & 6,584,033 & Encoder \\ ResNet & [19] & Own & 34,345,545 & Encoder \\ Eff-b1 ([[https://github.com/qubitvel/efficientnet](https://github.com/qubitvel/efficientnet)]([https://github.com/qubitvel/efficientnet](https://github.com/qubitvel/efficientnet)), accessed on 30 May 2022) & [20] & Own & 6,237,253 & Encoder \\ Eff-b3 ([[https://github.com/qubitvel/efficientnet](https://github.com/qubitvel/efficientnet)]([https://github.com/qubitvel/efficientnet](https://github.com/qubitvel/efficientnet)), accessed on 30 May 2022) & [20] & Own & 7,851,275 & Encoder \\ Eff-b5 ([[https://github.com/qubitvel/efficientnet](https://github.com/qubitvel/efficientnet)]([https://github.com/qubitvel/efficientnet](https://github.com/qubitvel/efficientnet)), accessed on 30 May 2022) & [20] & Own & 12,005,305 & Encoder \\ Eff-b7 ([[https://github.com/qubitvel/efficientnet](https://github.com/qubitvel/efficientnet)]([https://github.com/qubitvel/efficientnet](https://github.com/qubitvel/efficientnet)), accessed on 30 May 2022) & [20] & Own & 20,141,881 & Encoder \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Overview of the evaluated architectures. For each architecture, the publication is cited that describes it. Some architectures have been expanded to include a decoder part so that they reflect a U-Net. In addition, the number of parameters is specified and states whether there were pre-trained weights for the encoder part available. Figure 5: Example of the layout detection. Red regions are lyric regions. Green areas are music regions and purple areas are drop capital regions. We divided the drop capitals into two classes.