## 5 Experiments
### Symbol and Note Component Detection
#### 5.3.1 Metrics  
We use several metrics for the symbol detection evaluation. First, we measure both the detection rates of symbols in general (\(F_{1}^{\text{all}}\)) and also the subgroups of notes (\(F_{1}^{\text{note}}\)) and clets (\(F_{1}^{\text{clef}}\)). Thereto, we compare all GT and prediction symbols and match a pair as TP if two symbols are closer than 5 px, all other symbols in the prediction and GT are treated as FP and FN, respectively. In the subgroups, we only account for symbols that match a note or clef in the GT or prediction. In the following, accidentals are not evaluated separately because no accidental is predicated at all.  
Moreover, we measure how well types and staff locations of the note or clef subgroups are predicted. For each subgroup, our measure only takes TPs into account and counts the number of correctly predicted types (\(Acc_{\text{type}}^{\text{note}}\) and \(Acc_{\text{type}}^{\text{clef}}\)), that is the neume start and graphical connection for notes and the clef type (C or F) for clefs, or the staff positions (\(Acc_{\text{pos}}^{\text{note}}\) and \(Acc_{\text{pos}}^{\text{clef}}\)).  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & & \multicolumn{3}{c}{**No Data Augmentation**} & \multicolumn{3}{c}{**Data Augmentation**} \\  & \(N_{\text{train}}\) & \(F_{1}^{\text{D}}\) & \(F_{1}^{\text{LF}}\) & **Tot. F1** & \(F_{1}^{\text{D}}\) & \(F_{1}^{\text{LF}}\) & **Tot. F1** \\ \hline CT. & 0 & 0.993 & 0.987 & **0.980** & 0.982 & 0.986 & 0.968 \\ \hline \multirow{4}{*}{Default} & 1 & 0.983 & 0.969 & 0.953 & 0.991 & 0.980 & 0.971 \\  & 2 & 0.986 & 0.976 & 0.962 & 0.990 & 0.985 & 0.975 \\  & 4 & 0.994 & 0.979 & 0.973 & 0.994 & 0.984 & 0.978 \\  & All & 0.989 & 0.984 & 0.974 & 0.994 & 0.986 & 0.979 \\ \hline \multirow{4}{*}{PT} & 1 & 0.992 & 0.980 & 0.972 & 0.988 & 0.985 & 0.973 \\  & 2 & 0.992 & 0.981 & 0.973 & 0.993 & 0.984 & 0.977 \\  & 4 & 0.999 & 0.983 & **0.982** & 0.993 & 0.987 & 0.980 \\  & All & 0.995 & 0.984 & 0.980 & 0.998 & 0.986 & **0.984** \\ \hline \multirow{4}{*}{Inc.} & 1 & 0.992 & 0.988 & **0.980** & 0.985 & 0.988 & 0.973 \\  & 2 & 0.992 & 0.987 & **0.979** & 0.985 & 0.984 & 0.969 \\ \cline{1-1}  & 4 & 0.994 & 0.986 & 0.980 & 0.986 & 0.986 & 0.972 \\ \cline{1-1}  & All & 0.994 & 0.988 & 0.982 & 0.988 & 0.986 & 0.974 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Evaluation how the number of training instances influence the accuracy of pages with an unknown layout. All results are the averages of a five fold of each data set part. Only \(N_{\text{train}}\) pages are chosen for actual training. The major row grouping shows if pages of the remaining parts are included during training: Either no data is used (Default), a pretrained model on the data is used (PT), a large data set comprising all other data which is extended by \(N_{\text{train}}\) is used (Inc.) The first row shows the cross-training (CT.) results of Table 4 without any data of the target layout. The two major column sections show the results with and without data augmentation on the training data. The best values for each \(N_{\text{train}}\) are marked bold.  
Our last measure evaluates either the diplomatic or the harmonic transcription. We compare the produced sequence with the GT and count the number of insertions, deletions or substitutes in the sequence, which is similar to OCR tasks. As diplomatic transcription, we compare staff position and types of notes and clefs and their order; however, the actual horizontal position is ignored. Thus, only the resulting transcription is evaluated. The harmonic transcription is similar to the diplomatic one but ignores the type of notes, which is why only the harmonic information that is the melody is captured. We represent these two sequence accuracy rates by dSAR and hSAR.