# Applying Automatic Translation for Optical Music Recognition's Encoding Step
## 4 Results
### Error Analysis of Translation Categories  
In order to gain some insights on the performance of the proposed models in some of the translation categories discussed in Section 3.2.1, the positions of notes affected by the associated phenomena were identified and annotated for each of the three corpora,  
Figure 10: Mean symbol error rate (SER) and standard deviation, represented by vertical bars, obtained when training the statistical and the transformer neural approaches with subsets of the PRIMUS corpus of different sizes. \begin{table}
\begin{tabular}{l l c c c c} \hline \hline  & & \multicolumn{2}{c}{**Training**} & \multicolumn{2}{c}{**Translating**} \\
**Approach** & **Data Set** & **Time** & **Memory** & **Time** & **Memory** \\ \hline \multirow{3}{*}{SMT} & Camera-PrIMuS & 12 h & 31 MB & 2 h & 40 MB \\  & FMT & \(6^{\prime}\) & 31 MB & \(13^{\prime\prime}\) & 38 MB \\  & Zaragoza & \(6^{\prime}\) & 30 MB & \(10^{\prime\prime}\) & 37 MB \\ \hline \multirow{3}{*}{Seq2 Seq-Attn} & Camera-PrIMuS & 4 d & 317 MB & \(10^{\prime\prime}\) & 317 MB \\  & FMT & 3 h & 317 MB & \(9^{\prime\prime}\) & 317 MB \\  & Zaragoza & \(5^{\prime}\) & 317 MB & \(7^{\prime\prime}\) & 317 MB \\ \hline \multirow{3}{*}{The Transformer} & Camera-PrIMuS & 5 h & 385 MB & \(7^{\prime\prime}\) & 385 MB \\  & FMT & \(13^{\prime}\) & 385 MB & \(5^{\prime\prime}\) & 385 MB \\  & Zaragoza & \(5^{\prime}\) & 385 MB & \(3^{\prime\prime}\) & 385 MB \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Total time and peak memory consumed by each the three approaches evaluated (SMT, Seq2 Seq-Attn, and the Transformer) to train and to translate one fold of each of the three datasets used for evaluation (Camera-PrIMuS, FMT, and Zaragoza). based on ground truth data. Then, the SER was computed taking into account only these notes for each of the translation categories analyzed. For that purpose, we computed a _masked SER_, for which only the errors in the target notes were taken into account. For the masked version of SER, we aligned the translation hypothesis \(h\) and the ground-truth reference \(r\) as usual, but took into account only the symbols involved in the translation category studied to compute the error rate. Table 4 presents the masked SER for the translation categories of interest. Note, however, that due to how this metric is computed at the symbol level, results in each category were most probably influenced by errors due to phenomena from other categories. For example, an error predicting the pitch of a note would also compute as an error in the dotted notes, and the pitch altered by key categories. Hence, these results have to be taken as an inkling of which categories may contribute more to the overall SER, and deserve further study. For the Camera-PrIMuS corpus, it is evident that the availability of contextual information about the current clef and key signature of each note drastically improved the performance of the statistical (SMT) method, approaching the performance figures of neural models. The improvement for the SMT has occurred despite the increase of the vocabulary size of the corpus more than sixteen times. Pitch and pitch altered by key categories were expected to improve when introducing the tonal context in the input symbols. The fact that there was also a great improvement in the dotted notes category, which a priori should not be affected by contextual information, as it is irrelevant for computing the duration of notes, was most probably due to the fact that errors in this category were indeed due to errors related to pitch prediction. The results for the FMT corpus revealed that the pitch altered by key issue is the hardest to learn for all models except the Transformer. Note that the use of contextual information for the SMT approach does not provide a clear improvement of the performance. In fact, contextual SMT seems to perform worse than its non-contextual counterpart for some translation categories.