# Optical Medieval Music Recognition Using Background Knowledge
## 8 Results
Evaluation of the OMR pipeline similar to experiment 1 on a smaller dataset (Nevers Part 1-3 with 15,845 symbols) with 5-fold cross validation, repeating an evaluation from the literature (Table 5);
3. Evaluation of the OMR pipeline on a different, challenging dataset (Pa904 with 9025 symbols on 20 pages) as a real-world use case: 1. With pretrained mixed model and document specific fine-tuning: Starting with the mixed model of experiment 1 and using 16 pages of the dataset for fine-tuning while evaluating on the remaining 4 pages (5-fold cross validation; Table 6) 2. With document specific training without a pretrained model: Using solely the Pa904 dataset for training and evaluation in a 5-fold cross validation similar to experiment 1. Here 4 architectures (FCN, U-Net, Eff-b3, Eff-b7; Table 6) are compared;
3. With a mixed model without fine-tuning: Using the mixed model of experiment 1 and evaluation of the 20 pages of the dataset Pa904 (Table 7). \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Detection**} & \multicolumn{4}{c}{**Type**} & \multicolumn{2}{c}{**Position in Staff**} & \multicolumn{2}{c}{**Sequence**} \\  & **Archit.** & \(\mathbf{F_{1}^{all}}\) & \(\mathbf{F_{1}^{note}}\) & \(\mathbf{F_{1}^{def}}\) & \(\mathbf{F_{1}^{Acids}}\) & \(\mathbf{Acc_{note}}\) & \(\mathbf{Acc_{clcf}}\) & \(\mathbf{Acc_{note}}\) & \(\mathbf{Acc_{clcf}}\) & \(\mathbf{hSAR}\) & \(\mathbf{dSAR}\) & \(\mathbf{mAR}\) \\ \hline \multirow{4}{*}{FCN} & FCN & 99.1 & 99.2 & 93.2 & 22.2 & n/a & 96.9 & 95.8 & 1 & 93.3 & n/a & 84.8 \\  & Eff-b3 & 99.6 & 99.6 & 98.0 & 39.9 & n/a & 97.5 & 95.7 & 1 & 94.0 & n/a & 91.2 \\  & Eff-b7 & 99.4 & 99.5 & 97.4 & 23.8 & n/a & 97.6 & 95.7 & 1 & 93.8 & n/a & 91.4 \\  & U-Net & 99.3 & 99.3 & 96.0 & 36.2 & n/a & 96.5 & 95.5 & 1 & 93.3 & n/a & 87.8 \\ \cline{2-11}  & Mean & 99.3 & 99.4 & 96.2 & 29.5 & n/a & 97.1 & 95.7 & 1 & 93.6 & n/a & 88.3 \\ \hline \multirow{4}{*}{FCN} & FCN & 99.2 & 99.3 & 97.0 & 22.2 & n/a & 95.3 & 98.8 & 1 & 96.4 & n/a & 91.3 \\  & Eff-b3 & 99.6 & 99.7 & 98.0 & 39.9 & n/a & 97.0 & 98.8 & 1 & 97.0 & n/a & 94.0 \\  & Eff-b7 & 99.5 & 99.6 & 98.7 & 23.8 & n/a & 97.7 & 98.5 & 1 & 96.7 & n/a & 92.4 \\  & U-Net & 99.5 & 99.6 & 98.0 & 36.2 & n/a & 95.4 & 98.6 & 1 & 96.6 & n/a & 94.7 \\ \cline{2-11}  & Mean & 99.4 & 99.6 & 97.9 & 29.9 & n/a & 96.4 & 98.7 & 1 & 96.7 & n/a & 93.1 \\ \hline \multirow{4}{*}{FCN} & FCN & 99.1 & 99.2 & 96.0 & 0.03 & n/a & 98.7 & 98.8 & 1 & 93.8 & n/a & 88.8 \\  & Eff-b3 & 99.5 & 99.6 & 98.0 & 33.3 & n/a & 98.0 & 98.8 & 1 & 94.5 & n/a & 90.5 \\  & Eff-b7 & 99.5 & 99.6 & 98.7 & 34.7 & n/a & 95.3 & 98.5 & 1 & 94.4 & n/a & 90.3 \\  & U-Net & 99.5 & 99.5 & 97.0 & 32.3 & n/a & 97.0 & 98.6 & 1 & 94.4 & n/a & 90.3 \\ \cline{2-11}  & Mean & 99.4 & 99.5 & 97.4 & 25.1 & n/a & 97.3 & 98.7 & 1 & 94.3 & n/a & 90.0 \\ \hline \multirow{4}{*}{FCN} & FCN & 99.2 & 99.2 & 98.3 & 26.7 & n/a & 97.9 & 98.7 & 1 & 96.6 & n/a & 94.6 \\  & Eff-b3 & 99.5 & 99.6 & 98.8 & 33.3 & n/a & 98.0 & 99.0 & 1 & 97.4 & n/a & 95.8 \\ \cline{1-1}  & Eff-b7 & 99.5 & 99.6 & 98.0 & 34.7 & n/a & 95.3 & 98.9 & 1 & 97.3 & n/a & 93.0 \\ \cline{1-1}  & U-Net & 99.5 & 99.6 & 98.5 & 32.3 & n/a & 97.0 & 99.0 & 1 & 97.3 & n/a & 94.3 \\ \cline{1-1} \cline{2-11}  & Mean & 99.4 & 99.5 & 98.4 & 31.8 & n/a & 97.1 & 98.9 & 1 & 97.2 & n/a & 94.4 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Experiments on the Nevers dataset, which was used for evaluation in [12]. Using the post-processing pipeline, the dSAR has improved by about 1%. The melody accuracy rate has improved by 8%. Using more advanced encoders, the mAR could further be improved by about 3%. 4 Evaluation how effective the different parts of post-processing pipeline are for the mixed model with and without fine-tuning on the Pa904 dataset (Table 8);
5 Evaluation of the frequency of remaining error types with and without post-processing on the Pa904 dataset using the mixed model with fine-tuning (Table 9).