## 7 Results
### Diacritics Restoration  
The diacritics restoration results are presented in Table 3. Our ByT5 method results lay between the dictionary (a simple statistical Unigram model) and the state-of-the-art model [46]. The highest alpha-word accuracy is for French, Spanish, and Croatian, with results that were only 0.34%, 0.29%, and 0.56% behind the state of the art, respectively. These languages have the smallest percentage of diacritic words (see Table 1). The lowest scores are recorded for Vietnamese and Latvian at 94.25% and 96.33%, respectively. We also note that the Irish language, with the smallest dataset, has the highest standard deviation of 0.32%. The \"Raw\" column in Table 3 indicates the alpha-word accuracy of the uncorrected text for comparison. Naturally, the more diacritic-heavy the language is, the lower the number. An Approach with the Dictionary and the ByT5 models (Dict.+ByT5)  
We noticed that the dictionary method outperforms the ByT5 method for words that have only a single target translation in the dictionary. We grouped words by how many translation targets in the dictionary they have and we show the ratio of ByT5-to-Dictionary error rates in Table 4. The resulting values that are higher than 1 indicate the Dictionary outperforming the ByT5 model. This is the case for all languages at a word group with only a single translation. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**Language**} & \multirow{2}{*}{**Raw**} & \multirow{2}{*}{**Dict.**} & \multirow{2}{*}{**[**46**]**} & \multicolumn{2}{c}{**ByT5**} & \multicolumn{2}{c}{**Dict.+ByT5**} \\ \cline{5-8}  & & & & & **\#samples: \(\times\)1** & \(\times\)8** & \(\times\)**1** & \(\times\)**8** \\ \hline Croatian & 85.01 & 99.11 & 99.73 & 99.17 \(\pm\) 0.06 & & 99.42 \(\pm\) 0.03 & \\ Czech & 49.71 & 95.67 & 99.22 & 98.01 \(\pm\) 0.03 & & 98.38 \(\pm\) 0.04 & \\ French & 83.11 & 97.98 & 99.71 & 99.37 \(\pm\) 0.04 & & 99.49 \(\pm\) 0.03 & \\ Hungarian & 50.34 & 96.22 & 99.41 & 98.42 \(\pm\) 0.02 & 99.20 & 98.78 \(\pm\) 0.01 & 99.25 \\ Irish & 69.97 & 96.65 & 98.88 & 98.14 \(\pm\) 0.32 & & 98.40 \(\pm\) 0.16 & \\ Latvian & 50.14 & 90.59 & 98.63 & 96.33 \(\pm\) 0.12 & 97.78 & 96.62 \(\pm\) 0.09 & 97.66 \\ Lithuanian & 60.76 & 93.83 & — & 97.94 \(\pm\) 0.19 & 99.07 & 98.18 \(\pm\) 0.13 & 98.95 \\ Polish & 66.73 & 97.00 & 99.66 & 99.00 \(\pm\) 0.03 & & 99.16 \(\pm\) 0.02 & \\ Romanian & 70.37 & 96.09 & 98.64 & 97.99 \(\pm\) 0.03 & & 98.17 \(\pm\) 0.04 & \\ Slovak & 56.34 & 96.88 & 99.32 & 98.43 \(\pm\) 0.06 & & 98.77 \(\pm\) 0.02 & \\ Spanish & 87.97 & 99.11 & 99.62 & 99.33 \(\pm\) 0.04 & & 99.43 \(\pm\) 0.02 & \\ Turkish & 68.39 & 98.41 & 98.95 & 98.86 \(\pm\) 0.04 & & 99.03 \(\pm\) 0.02 & \\ Vietnamese & 15.88 & 73.53 & 98.53 & 94.25 \(\pm\) 0.07 & 97.53 & 94.29 \(\pm\) 0.07 & 97.54 \\ \hline Average & 62.67 & 94.70 & 99.19 & 98.10 & & 98.32 & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Alpha-word accuracy results (%) for the diacritics restoration task. We report means and standard deviations for three separate training runs with different initial model weights and dataset samplings trained for 524 288 sentences (#samples: \(\times\)1) and a single run for eight times more(\(\times\)8), cycling through the available training data (Table 1) as needed. Table 4 also portrays how the ratio of the ByT5-to-Dictionary error rates changes during half and full training. The trend is obvious: the transformer improves for all word groups with training. If our training was longer, the ByT5 model may even surpass the Dictionary model at a word group of one translation candidate. This is exactly what happened for the Latvian and the Lithuanian languages after eight times more training samples. Note that at half the training, the standard deviation of the Turkish ratio is abnormally high. This is due to one of three ByT5 training runs that temporarily fail. However, with further training, the run recovered up to the same accuracy level as the other two.