## 8 Experiments
### Error Analysis
\begin{table}
\begin{tabular}{l l c c c} \hline \hline
**Dataset** & **Metric** & **Statistical Model** & **LSTM** & **Transformer** \\ \hline GosVL & Corr [\%] & 96.58 & 97.03 & 97.50 \\  & Miss [\%] & 2.06 & 2.02 & 1.54 \\  & Wrng [\%] & 0.48 & 0.56 & 0.44 \\  & Unwrr [\%] & 0.88 & 0.40 & 0.50 \\  & W-ins [\%] & 0.00 & 0.00 & 0.00 \\  & W-del [\%] & 0.00 & 0.00 & 0.02 \\  & AR \(p\)-value & & 0.0003 & 0.0001 \\  & BR \(p\)-value & & 0.0002 & 0.0001 \\ \hline Gos & Corr [\%] & 94.74 & 94.87 & 95.13 \\  & Miss [\%] & 2.57 & 2.53 & 2.49 \\  & Wrng [\%] & 1.42 & 1.58 & 1.45 \\  & Unwrr [\%] & 1.28 & 1.01 & 0.92 \\  & W-ins [\%] & 0.00 & 0.00 & 0.00 \\  & W-del [\%] & 0.00 & 0.02 & 0.01 \\  & AR \(p\)-value & & 0.0886 & 0.0001 \\  & BR \(p\)-value & & 0.0828 & 0.0001 \\ \hline [PERSON] & Corr [\%] & 97.65 & 98.04 & 98.00 \\  & Miss [\%] & 1.16 & 1.21 & 0.93 \\  & Wrng [\%] & 0.74 & 0.63 & 0.71 \\  & Unwrr [\%] & 0.45 & 0.12 & 0.35 \\  & W-ins [\%] & 0.00 & 0.00 & 0.00 \\  & W-del [\%] & 0.00 & 0.00 & 0.01 \\  & AR \(p\)-value & & 0.0001 & 0.0001 \\  & BR \(p\)-value & & 0.0001 & 0.0001 \\ \hline [PERSON] & Corr [\%] & 97.27 & 97.79 & 97.71 \\  & Miss [\%] & 1.31 & 1.11 & 1.07 \\  & Wrng [\%] & 0.97 & 0.83 & 0.85 \\  & Unwrr [\%] & 0.44 & 0.26 & 0.33 \\  & W-ins [\%] & 0.01 & 0.01 & 0.01 \\  & W-del [\%] & 0.01 & 0.01 & 0.04 \\  & AR \(p\)-value & & 0.0001 & 0.0001 \\  & BR \(p\)-value & & 0.0001 & 0.0001 \\ \hline All & Corr [\%] & 96.12 & 97.16 & 97.01 \\  & Miss [\%] & 1.81 & 1.46 & 1.47 \\  & Wrng [\%] & 1.18 & 0.96 & 1.04 \\  & Unwrr [\%] & 0.90 & 0.42 & 0.47 \\  & W-ins [\%] & 0.00 & 0.00 & 0.00 \\  & W-del [\%] & 0.00 & 0.00 & 0.01 \\  & AR \(p\)-value & & 0.0001 & 0.0001 \\  & BR \(p\)-value & & 0.0001 & 0.0001 \\ \hline \hline \end{tabular}
\end{table} 
Table 14: Error analysis of character-based models on all datasets. of those errors is much smaller than for other types of errors. Examples of those errors are presented later in this section. Statistical tests show significance in the results of all neural network-based models compared to statistical-based models, with one exception--the LSTM model on Gos. Comparing the results among different datasets, we found the best results with GosVL, Artur_J, and Artur_P. These datasets all contain public speech, while Gos and Artur_N contain mostly private speech. Speakers in the public speech datasets might be more used to speaking in formal language and are more conscious of their speech. Additionally, public speech may contain a smaller number of different non-standard word forms, thus making it easier for the algorithm to train better models. It should also be noted that the results on GosVL are among the best, despite this dataset providing the smallest training dataset. In Gos, numerous different dialects are present, which poses a greater challenge for training models. Types of errors made by neural network-based models are depicted in Figures 1-3. The figures show excerpts from the HTML document generated by the evaluation tool. In each example, the first row presents a non-standard (in our case, literal) form, the second row presents a reference standard form, and the third row presents the evaluated segment. For each erroneous segment, the line in which the segment is found within the dataset is provided. Figures 1 and 2 are obtained on the GosVL dataset, while Figure 3 is from Artur_N. All results are from the character-based transformer model. Figure 1 shows the most common error types highlighted in different colors: red for missing conversions, green for wrong conversions, and blue for unwarranted conversions. The text is grey for all words in the segments, which are the same in literal and standard form, and black otherwise. Segments are numbered from the lines in the file. Line 93 is missing, as the system converted this line without errors. Analyzing the results, it is worth noting that the word _firmware_ is a foreign word and is in the literal form in the dataset written as it was pronounced, while the standard form is the standard English form.