BERT-like Models for Slavic Morpheme Segmentation
Authors: Dmitry Morozov, Lizaveta Astapenka, Anna Glazkova, Timur Garipov, Olga Lyashevskaya
Accepted at: acl25

BERT-like Models for Slavic Morpheme Segmentation

Dmitry Morozov1,2  Lizaveta Astapenka3  Anna Glazkova4,2  Timur Garipov1,2  Olga Lyashevskaya5,6

1The Artificial Intelligence Research Center of Novosibirsk State University
2Russian National Corpus  3St. Petersburg State University  4University of Tyumen
5HSE University  6Vinogradov Russian Language Institute RAS

Abstract
Automatic morpheme segmentation algorithms are applicable in various tasks, such as building tokenizers and language education. For Slavic languages, the development of such algorithms is complicated by the rich derivational capabilities of these languages. Previous research has shown that, on average, these algorithms have already reached expert-level quality. However, a key unresolved issue is the significant decline in performance when segmenting words containing roots not present in the training data. This problem can be partially addressed by using pre-trained language models to better account for word semantics. In this work, we explored the possibility of fine-tuning BERT-like models for morpheme segmentation using data from Belarusian, Czech, and Russian. We found that for Czech and Russian, our models outperform all previously proposed approaches, achieving word-level accuracy of 92.5-95.1%. For Belarusian, this task was addressed for the first time. The best-performing approach for Belarusian was an ensemble of convolutional neural networks with word-level accuracy of 90.45%.

1 Introduction
Morpheme segmentation is the process of dividing a word into substrings — morphemes — which are the smallest indivisible meaningful elements of a language: roots, prefixes, suffixes, and others. This process can be essential for language learning, particularly for languages with rich derivational capabilities. For example, many orthographic rules taught in Russian language school curricula rely on the ability to identify and analyze the internal structure of a word, such as the spelling of voiceless and voiced consonants at the end of prefixes and the verification of unstressed vowels in roots (Volskaya et al., 2018).

Another potential use case for morpheme segmentation is its application as a subword tokenizer for language models. Using a morpheme tokenizer as an alternative to the widely adopted Byte-Pair Encoding (BPE) (Gage, 1994) has been shown by several researchers (Matthews et al., 2018; Nzeyimana and Niyongabo Rubungo, 2022) to improve the quality of trained models. Finally, morpheme annotation is used in large text corpora (Savchuk et al., 2024), which are employed for linguistic research, to enhance user search capabilities.

For all three scenarios mentioned above, the existence of an algorithm for constructing morpheme segmentation — that is, mapping a word form or lemma to its morphemes — is necessary. In some cases, manually compiled and verified dictionaries of morpheme annotations are used for this purpose. This approach is often applied in school education. For example, for the Belarusian language, the School morpheme dictionary of the Belarusian language (Mormysh et al., 2005) is used, while for Russian, the Word Formation Dictionary of the Russian language (Tikhonov, 1990) is employed.

A significant drawback of this approach is the need for constant adaptation to the emergence of new words in the language. For Slavic languages, with their extensive derivational capabilities, expanding and maintaining such dictionaries requires regular, long-term work by expert linguists. Moreover, the lack of a unified interpretation of the term “morpheme segmentation” and clear criteria for identifying morphemes (Iomdin, 2019) makes it impossible to develop an analytical solution.

At the same time, machine learning-based morpheme segmentation algorithms have demonstrated high quality, including for Slavic languages such as Czech (Peters and Martins, 2022) and Russian (Sorokin and Kravtsova, 2018; Peters and Martins, 2022; Morozov et al., 2024). In particular, Morozov et al. (2024) found that for Russian, the quality of automatic annotation on a random sample, in terms of the number of fully correct segmentations, is on par with expert annotation. In
this case, the work of linguists in expanding morpheme dictionaries could be significantly accelerated by creating automatic draft annotations and subsequently correcting and validating them by the experts.

A significant obstacle to this approach, however, is the sharp decline in annotation quality when models encounter words containing roots not present in the training data (out-of-vocabulary roots, OOV roots) (Morozov et al., 2024). A potential solution to this problem could be the use of pre-trained language models. For example, Pranjic et al. (2024) proposed a binary classifier for detecting morpheme boundaries in words based on fine-tuning the Glot500 model (Imani et al., 2023). This approach, however, is likely unsuitable for annotating large dictionaries due to its computational complexity. Meanwhile, Sorokin (2022) used BERT-like models to enrich the feature representation of words, which improved segmentation quality.

Unlike previous work where BERT was used only to obtain word embeddings, in our work we test whether BERT-like models can be fine-tuned for morpheme segmentation, outperform a high-level baseline on Slavic language data, and address the issue of OOV roots. We utilized a CNN ensemble (Sorokin and Kravtsova, 2018), which outperforms other algorithms on Russian language (Morozov et al., 2024) as the baseline. We used four datasets with morpheme annotations: two for Russian and one each for Czech and Belarusian. Russian and Czech were selected as languages with sufficient representation and existing large BERT-like models, while Belarusian was included as a low-resource language. Unlike the problem statement at the SIGMORPHON competition (Batsuren et al., 2022), we considered the problem of surface segmentation, i.e. dividing a word into morphemes without restoring the original form of the morpheme. In addition, based on the potential application of the algorithm in school education we included in the problem the determination of the type of each morpheme.

Our main contributions are as follows:

1. For Czech and Russian, our fine-tuned BERT-like models outperformed the CNN ensemble. We managed to achieve a share of completely correct annotations of 92-95% in the case of a random test sample, and 72-77% in the case of testing on words with roots that were not found in the training sample. The proportion of erroneous segmentations on a random sample decreased by 30-45%, and on words with OOV roots by 9-15%. The results obtained exceed all previously presented results for these languages.

2. For Belarusian, the CNN ensemble showed better performance in both types of testing, achieving word accuracy of 90.5% in the case of random split and 74.8% in the split-by-roots case. To our knowledge, similar experiments have not been conducted for this language before, which allows us to consider the presented result as a new state-of-the-art baseline. We also publish the first publicly available Belarusian morpheme dataset with morpheme type annotation.

3. We found that when testing on words with OOV roots, almost all roots can be divided into two groups: “recognizable” and “completely unknown”. The first group includes roots for which words are annotated completely correctly in 100% of cases, while the second group consists of roots for which words are never annotated completely correctly by the model.

2 Related Work

2.1 Morpheme Segmentation

Research on automatic morpheme segmentation varies significantly in terms of problem formulation. This is because morpheme segmentation algorithms are typically developed for specific applications within larger tasks. One criterion that differentiates approaches is the type of segmentation: surface or canonical (Cotterell et al., 2016). Surface segmentation involves segmenting the original string, while canonical segmentation additionally restores the original form of the morpheme. This distinction becomes evident in cases where language rules cause changes at morpheme boundaries. For example, the Belarusian word абапенiцi ‘subscription (adj.)’ is formed by adding the suffix -ск- and the ending -i to абапен ‘subscriber’, with the resulting combination -мс- at the morpheme boundary transforming into -ц-. In this case, the surface segmentation of this word might be either абап-еnц-к-i or абап-еn-цк-i, depending on the adopted paradigm, while the canonical segmentation would be абап-еnц-ск-i.
Another difference lies in the definition of morpheme types. For many tasks, such as building morpheme tokenizers, defining morpheme types is not essential, whereas for others, such as language education, specifying morpheme types is mandatory.

Finally, a third important distinction is the composition of the dataset used. Typically, two types of datasets are considered: lemma datasets and word form datasets. Morpheme dictionaries compiled by linguists usually work with lemmata, while tokenization obviously requires algorithms that also handle word forms.

Most early, relatively effective morpheme segmentation algorithms belong to the Morfessor family (Creutz and Lagus, 2002). This family includes unsupervised and semi-supervised algorithms, which have been tested on a variety of languages, including English, Finnish, German, Turkish, and others. In the SIGMORPHON 2022 competition (Batsuren et al., 2022), which addressed the task of canonical segmentation of word forms without specifying morpheme types, the Morfessor2 algorithm (Smit et al., 2014) was used as one of the baselines and outperformed the other two baselines — ULM (Kudo, 2018) and WordPiece (Schuster and Nakajima, 2012) — for 8 out of 9 languages.

However, the quality of algorithms in this family remains relatively low. For instance, according to the SIGMORPHON 2022 results, Morfessor2 achieved an F-score for correctly predicted morphemes ranging from 9% to 41%, while the best solutions from competition participants exceeded 90% for each language. Among the solutions presented at the competition, the DeepSPIN team (Peters and Martins, 2022) achieved the best results for all 9 languages. Their models rely on LSTM networks with a specific loss function (DeepSPIN-1 and DeepSPIN-2) and the Transformer architecture (DeepSPIN-3). Among other approaches, the solution by the CLUZH team (Wehrli et al., 2022), an ensemble of neural character-level transducers, deserves mention, as it trailed the leader by only a small margin.

As in other areas of natural language processing, there is significant interest in exploring the potential of large language models for solving morpheme segmentation tasks. For example, Pranjic et al. (2024) presented a fine-tuned model for detecting morpheme boundaries in words, which demonstrated superior results for several low-resource languages. A drawback of this work, however, is the high computational complexity of the algorithm, which sequentially iterates through all possible morpheme boundary positions in a word.

2.2 Slavic Morpheme Segmentation

Among the three languages considered in this work, Russian has been the most extensively studied in terms of morpheme segmentation. Several research teams (Sorokin and Kravtsova, 2018; Bolshakova and Sapin, 2019, 2022; Morozov et al., 2024) have explored segmentation algorithms based on gradient boosting over decision trees, convolutional neural networks, LSTM networks, and Transformers networks. In most cases, the authors addressed the task of surface segmentation with morpheme type identification, using a dataset consisting of lemmata. The best results across various experiments involving different morpheme dictionaries were achieved using a CNN ensemble, with the proportion of fully correct annotations reaching 88-90% (Morozov et al., 2024).

In the case of the Czech language, most of the research is related to the DeriNet database1. Macháček et al. (2018) investigate the effectiveness of two linguistically uninformed subword construction methods (Byte Pair Encoding and Subword Text Encoder) in handling morphological variations in Czech. Svoboda and Sevcíková (2022) explore the possibility of automatic construction of word-formation chains.

Finally, we were unable to find any relevant studies for Belarusian.

Among the languages represented at SIGMORPHON, Czech and Russian were also included. For Czech, the best result was achieved using the DeepSPIN-2 model (F-score=93.88), while for Russian, the DeepSPIN-3 model yielded the highest performance (F-score=99.35). However, Morozov et al. (2024) demonstrated that the quality of DeepSPIN-3 drops sharply when transitioning to surface segmentation, particularly when dealing with OOV roots. Additionally, the dataset for Russian in SIGMORPHON 2022 consisted of only 10% lemmata and 90% word forms, and according to official results (Batsuren et al., 2022), the performance of the DeepSPIN-3 approach on Russian lemmata drops significantly below an F-score of 93 (a detailed analysis for Czech was not provided in the paper).

1https://ufal.mff.cuni.cz/derinet
Morozov et al. (2024) highlighted that the problem of recognizing OOV roots remains a key unresolved issue. While several algorithms achieve 85-90% fully correct annotations when tested on random words, this figure drops to 67-72% when tested on words containing OOV roots. In the case of DeepSPIN-3, testing without morpheme type identification showed 81% fully correct annotations for a random sample of words, with 12% of segmentations not matching the original word letter by letter. However, when tested on words with OOV roots, the proportion of fully correct annotations dropped to 14.5%, with 74% of words having segmentations that did not match the original word letter by letter.

The problem of “recognizing” OOV roots can be partially addressed by using embeddings from BERT-like models, as demonstrated by Sorokin (2022) and Morozov et al. (2024). However, it remains an open question whether it is possible to rely solely on BERT-like models for segmentation without external models, by fine-tuning a BERT-like model specifically for this task.

3 Data

3.1 Belarusian

Since no machine-readable annotated dataset for the Belarusian language could be found, we prepared such a dataset ourselves. As the source of analyses, we used the School morpheme dictionary of the Belarusian language (Mormysh et al., 2005). Since this dictionary does not include annotations for morpheme types, we conducted this annotation ourselves with the involvement of native Belarusian speakers with linguistic education. For annotation, we used five types of morphemes: root (ROOT), prefix (PREF), suffix (SUFF), ending (END), and linking vowel (LINK). We decided not to separate suffixes and postfixes, since the original dictionary contains only a few dozen examples of the use of postfixes. Zero endings and zero suffixes were excluded from consideration. The final dataset, Slounik, contains annotations for 31,057 words. The dataset is available for downloading under the CC-BY-NC-SA 4.0 license2.

3.2 Czech

For the Czech language, we utilized the DeriNet 2.1 database3. It contains annotations for 1,248,572 words, with three annotated morpheme types: root, prefix, and suffix. For the experiment, we excluded proper nouns from consideration. The final dataset contains annotations for 820,387 words.

3.3 Russian

For the Russian language, we used two datasets previously utilized by Morozov et al. (2024): Morphodict-T, based on the “Word Formation Dictionary of the Russian Language” (Tikhonov, 1990), and Morphodict-K, based on the “Dictionary of Morphemes of the Russian Language” (Kuznetsova and Efremova, 1986). These datasets contain annotations for 95,895 and 75,649 words, respectively, differing in both their vocabulary and their approach to morpheme segmentation. Morphodict-K uses an approach that emphasizes strong but not maximal splitting of morphemes and parallels to structurally similar words, while Morphodict-T uses the so-called Vinokur criterion, which requires the existence of a corresponding word-formation chain in modern Russian to isolate a morpheme. Both of these datasets use seven types of morphemes: root, prefix, suffix, ending, postfix (POST), linking vowel, and hyphen (HYPH).

3.4 Datasets Statistics

A brief numerical description of the datasets used is provided in Table 1.

4 Models and Experimental Setup

Our approach relies on character-level annotation. For this purpose, each letter in a word is assigned a label consisting of two elements: the position of the letter within the morpheme (B for beginning, M for middle, E for end, S for single-letter morpheme) and the type of the morpheme itself. Thus, a total of 22 different labels are possible during conversion (LINK and HYPH can only be single-letter). As a result, the Czech word šimlatost ‘shyness’ with the root šimlat- and the suffix -ost is mapped to the sequence of labels ['B-ROOT', 'M-ROOT', 'M-ROOT', 'M-ROOT', 'M-ROOT', 'E-ROOT', 'B-SUFF', 'M-SUFF', 'E-SUFF']. We considered two approaches. In the first case, the model was fed a sequence of letters, and a sequence of labels was expected as output. In the second case, we
<table>
  <tr>
    <th>Dataset</th>
    <th>Slounik</th>
    <th>DeriNet</th>
    <th>Morphodict-T</th>
    <th>Morphodict-K</th>
  </tr>
  <tr>
    <th>Language</th>
    <td>Belarusian</td>
    <td>Czech</td>
    <td>Russian</td>
    <td>Russian</td>
  </tr>
  <tr>
    <th>Unique words</th>
    <td>31,057</td>
    <td>820,387</td>
    <td>95,895</td>
    <td>75,649</td>
  </tr>
  <tr>
    <th>Unique morphemes</th>
    <td>6,350</td>
    <td>109,208</td>
    <td>15,899</td>
    <td>8,079</td>
  </tr>
  <tr>
    <th>Unique roots</th>
    <td>6,095</td>
    <td>102,889</td>
    <td>15,253</td>
    <td>7,148</td>
  </tr>
  <tr>
    <th>Avg morphemes per word</th>
    <td>3.74</td>
    <td>4.64</td>
    <td>3.86</td>
    <td>4.12</td>
  </tr>
  <tr>
    <th>Avg morpheme occurrence</th>
    <td>18.31</td>
    <td>34.89</td>
    <td>23.29</td>
    <td>38.56</td>
  </tr>
  <tr>
    <th>Avg root occurrence</th>
    <td>5.20</td>
    <td>7.97</td>
    <td>7.54</td>
    <td>12.24</td>
  </tr>
  <tr>
    <th>Avg characters in root</th>
    <td>4.18</td>
    <td>5.30</td>
    <td>5.52</td>
    <td>4.62</td>
  </tr>
  <tr>
    <th>Morpheme types</th>
    <td>PREF, ROOT, SUFF, END, LINK</td>
    <td>PREF, ROOT, SUFF</td>
    <td>PREF, ROOT, SUFF, END, LINK, POST, HYPH</td>
    <td>PREF, ROOT, SUFF, END, LINK, POST, HYPH</td>
  </tr>
</table>

Table 1: Brief characteristics of the datasets

prefixed the sequence of letters with the word itself, assigning it a special label ‘0’, that is, for the word šimlatost the input sequence was equal to the following: [‘šimlatost’, ‘š’, ‘i’, ‘m’, ‘l’, ‘a’, ‘t’, ‘o’, ‘s’, ‘t’], and the output one: [‘0’, ‘B-ROOT’, ‘M-ROOT’, ‘M-ROOT’, ‘M-ROOT’, ‘M-ROOT’, ‘E-ROOT’, ‘B-SUFFIX’, ‘M-SUFFIX’, ‘E-SUFFIX’].

We conducted two series of experiments: one with random dataset train-test splitting and another with splitting by roots. In both cases, we used 5-fold cross-validation. In the first case, each dataset was randomly divided into 5 nearly equal folds. In the second case, we divided all roots mentioned in the dataset into 5 nearly equal groups and then included all words with roots from group \( k \) in fold \( k \). In this scenario, words containing two or more roots were excluded from consideration. The sizes of the folds in the second case were quite close but not equal (see Table 2).

<table>
  <tr>
    <th>Dataset</th>
    <th>Min fold size</th>
    <th>Max fold size</th>
  </tr>
  <tr>
    <td>Slounik</td>
    <td>5,629</td>
    <td>6,712</td>
  </tr>
  <tr>
    <td>DeriNet</td>
    <td>157,380</td>
    <td>170,613</td>
  </tr>
  <tr>
    <td>Morphodict-T</td>
    <td>15,253</td>
    <td>16,013</td>
  </tr>
  <tr>
    <td>Morphodict-K</td>
    <td>12,401</td>
    <td>13,667</td>
  </tr>
</table>

Table 2: Size of the folds in the split-by-roots experiment (in words)

To evaluate the quality of annotation, we used metrics previously employed in (Sorokin and Kravtsova, 2018; Morozov et al., 2024): F-score, Precision, and Recall for morpheme boundaries; F-score, Precision, and Recall for root boundaries; character-level Accuracy; and the proportion of completely correct annotations (WordAccuracy). In the case where the word itself was prefixed to the sequence of letters, the label assigned to it was not taken into account when calculating the metrics.

In experiments with a CNN ensemble, we used the implementation by Sorokin and Kravtsova (2018)\footnote{https://github.com/AlexeySorokin/NeuralMorphemeSegmentation}. We used 3 models in the ensemble, with the window size set to 5. Each model was trained for a maximum of 25 epochs, incorporating early stopping with a patience of 10 epochs to prevent overfitting. Model training was performed on an AMD Ryzen 5 5600X CPU without the use of a GPU. Each training epoch took up to 90 seconds (in the case of the DeriNet dataset).

For implementation BERT-like models, we used the simpletransformers\footnote{https://simpletransformers.ai/} framework. For Russian and Czech, we selected models that have demonstrated high quality in other tasks: ruRoberta-large\footnote{https://huggingface.co/ai-forever/ruRoberta-large} (355M parameters) (Zmitrovič et al., 2024) and Czert-B-base-cased\footnote{https://huggingface.co/UWB-AIR/Czert-B-base-cased} (110M parameters) (Sido et al., 2021), respectively. For Belarusian, we were unable to find a model with similar characteristics. Therefore, we decided to conduct experiments with two models: a smaller one specifically tailored for Belarusian, roberta-small-belarusian\footnote{https://huggingface.co/KoichiYasuoka/roberta-small-belarusian} (16M parameters),
and SlavicBERT9 (180M parameters) (Arkhipov et al., 2019), which was trained on four other Slavic languages: Bulgarian, Czech, Polish, and Russian.

The batch size during training was set to 16, and the learning rate was set to 4e-6. The values of the remaining parameters were set to default. We fine-tuned the BERT-like models for 30 epochs. The models were trained on an Nvidia RTX 4090 GPU. Each training epoch took up to 18 minutes (in the case of the DeriNet dataset). The fine-tuned models are available on request from the corresponding author.

5 Results

The averaged cross-validation results are presented in Tables 3, 4. Full experimental results for each dataset, model, and fold can be found in Appendix E. The best results for each dataset are underlined. F stands for F-score, P stands for Precision, R stands for Recall, Acc stands for character-level accuracy, and WA stands for word-level accuracy. The “+lex” models correspond to models trained with the lemma added to the input sequence.

For three out of the four datasets, BERT-like models lead across all metrics. In the case of random split, the best results were achieved for Czech: 95.4% fully correct annotations and 98.8% character-level accuracy. For all datasets, a WordAccuracy greater than 90% and a character-level accuracy greater than 95% were achieved. When transitioning to testing on OOV roots, the annotation quality, as expected, decreases. However, similar to the random split scenario, BERT-like models, particularly those trained with the addition of the word lemma in the input sequence, outperformed others for Czech and Russian datasets. The best result was achieved on the Morphodict-T dataset: 77.2% fully correct segmentations and 92.82% character-level accuracy. The superiority over the CNN ensemble for Czech and Russian ranged from 2.2 to 4.5 percentage points in terms of WordAccuracy.

For Belarusian, the best result after the CNN ensemble in the random split scenario was achieved by the roberta-small-belarusian model, trained without lemmata in the input sequence. This demonstrates that a model 10 times smaller, trained on a low-resource language, can be more effective for the given task than a larger model trained on related languages. In the split-by-roots case, the SlavicBert model, similarly trained without lemmata, proved competitive: it outperformed CNN for 4 out of 8 metrics and trailed only slightly for the remaining four.

Four out of the five tested models showed improved quality in the split-by-roots scenario when lemmata were added to the training data. The exception was the SlavicBert model, which notably lacked the target language in its training data. Thus, it can be concluded that for models specifically trained on the target language, adding lemmata to the input sequence enhances the model’s ability to annotate OOV roots. However, for two of these four models, adding lemmata reduced recognition quality in the random split scenario.

Comparing Precision and Recall for all morphemes and for roots only, it can be observed that Recall for all morphemes is higher than Precision across all datasets and models. For root morphemes, a similar pattern is observed for the CNN ensemble, whereas for BERT-like models with added lemmata, the situation is reversed.

Finally, the analysis of WordAccuracy dynamics during training showed that for random split the value steadily increases, especially in the first 10 epochs. At the same time, for split-by-roots the changes are much less pronounced. An example of WordAccuracy dynamics for the Morphodict-K dataset and the RuRoberta+lex model is shown in Figure 1.

![WordAccuracy values during model training](page_1012_180_388_246.png)

Figure 1: WordAccuracy values during model training

6 Error Analysis

A selective analysis of errors made by the models revealed that two types of deviations from the target annotation occur most frequently: differences in suffix segmentation and differences in extract-
<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="3">All morphemes boundaries</th>
    <th colspan="3">Only roots boundaries</th>
    <th rowspan="2">Acc</th>
    <th rowspan="2">WA</th>
  </tr>
  <tr>
    <th>F</th><th>P</th><th>R</th>
    <th>F</th><th>P</th><th>R</th>
  </tr>
  <tr>
    <th colspan="9">Slounik</th>
  </tr>
  <tr>
    <td>CNN</td><td>98.41</td><td>98.10</td><td>98.73</td><td>95.26</td><td>95.30</td><td>95.21</td><td>96.95</td><td>90.45</td>
  </tr>
  <tr>
    <td>Roberta-bel</td><td>98.40</td><td>98.25</td><td>98.55</td><td>94.92</td><td>95.08</td><td>95.08</td><td>96.82</td><td>90.32</td>
  </tr>
  <tr>
    <td>Roberta-bel+lex</td><td>98.09</td><td>97.88</td><td>98.30</td><td>94.01</td><td>94.13</td><td>93.88</td><td>96.21</td><td>88.86</td>
  </tr>
  <tr>
    <td>SlavicBert</td><td>98.00</td><td>97.72</td><td>98.28</td><td>93.56</td><td>93.72</td><td>93.41</td><td>95.99</td><td>87.73</td>
  </tr>
  <tr>
    <td>SlavicBert+lex</td><td>97.96</td><td>97.68</td><td>98.24</td><td>93.33</td><td>93.58</td><td>93.08</td><td>95.89</td><td>87.58</td>
  </tr>
  <tr>
    <th colspan="9">DeriNet</th>
  </tr>
  <tr>
    <td>CNN</td><td>98.91</td><td>98.72</td><td>99.11</td><td>94.09</td><td>94.04</td><td>94.14</td><td>97.45</td><td>91.09</td>
  </tr>
  <tr>
    <td>Czert</td><td>99.40</td><td>99.27</td><td>99.53</td><td>96.63</td><td>96.64</td><td>96.63</td><td>98.69</td><td>95.12</td>
  </tr>
  <tr>
    <td>Czert+lex</td><td>99.44</td><td>99.33</td><td>99.55</td><td>96.78</td><td>96.79</td><td>96.78</td><td>98.76</td><td>95.39</td>
  </tr>
  <tr>
    <th colspan="9">Morphodict-T</th>
  </tr>
  <tr>
    <td>CNN</td><td>98.09</td><td>97.79</td><td>98.38</td><td>94.08</td><td>94.19</td><td>93.99</td><td>96.61</td><td>88.49</td>
  </tr>
  <tr>
    <td>ruRoberta</td><td>98.56</td><td>98.57</td><td>98.56</td><td>95.31</td><td>95.45</td><td>95.18</td><td>97.39</td><td>91.09</td>
  </tr>
  <tr>
    <td>ruRoberta+lex</td><td>98.76</td><td>98.69</td><td>98.84</td><td>96.07</td><td>96.13</td><td>96.00</td><td>97.78</td><td>92.47</td>
  </tr>
  <tr>
    <th colspan="9">Morphodict-K</th>
  </tr>
  <tr>
    <td>CNN</td><td>98.66</td><td>98.58</td><td>98.74</td><td>96.24</td><td>96.26</td><td>96.22</td><td>97.40</td><td>90.82</td>
  </tr>
  <tr>
    <td>ruRoberta</td><td>99.09</td><td>99.05</td><td>99.14</td><td>97.43</td><td>97.44</td><td>97.43</td><td>98.19</td><td>93.61</td>
  </tr>
  <tr>
    <td>ruRoberta+lex</td><td>99.10</td><td>99.04</td><td>99.17</td><td>97.36</td><td>97.37</td><td>97.35</td><td>98.19</td><td>93.54</td>
  </tr>
</table>

Table 3: Average metric value during cross-validation, random split

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="3">All morphemes boundaries</th>
    <th colspan="3">Only roots boundaries</th>
    <th rowspan="2">Acc</th>
    <th rowspan="2">WA</th>
  </tr>
  <tr>
    <th>F</th><th>P</th><th>R</th>
    <th>F</th><th>P</th><th>R</th>
  </tr>
  <tr>
    <th colspan="9">Slounik</th>
  </tr>
  <tr>
    <td>CNN</td><td>95.44</td><td>94.88</td><td>96.00</td><td>83.98</td><td>83.97</td><td>83.98</td><td>91.24</td><td>74.78</td>
  </tr>
  <tr>
    <td>Roberta-bel</td><td>95.36</td><td>95.51</td><td>95.21</td><td>83.60</td><td>83.67</td><td>83.53</td><td>90.80</td><td>73.55</td>
  </tr>
  <tr>
    <td>Roberta-bel+lex</td><td>95.48</td><td>95.48</td><td>95.49</td><td>83.71</td><td>83.79</td><td>83.64</td><td>91.12</td><td>74.59</td>
  </tr>
  <tr>
    <td>Slavic</td><td>95.59</td><td>95.29</td><td>95.89</td><td>84.28</td><td>84.49</td><td>84.06</td><td>91.17</td><td>74.74</td>
  </tr>
  <tr>
    <td>Slavic+lex</td><td>95.49</td><td>95.49</td><td>95.73</td><td>83.70</td><td>83.84</td><td>83.56</td><td>91.03</td><td>74.33</td>
  </tr>
  <tr>
    <th colspan="9">DeriNet</th>
  </tr>
  <tr>
    <td>CNN</td><td>96.30</td><td>96.42</td><td>96.18</td><td>80.07</td><td>79.98</td><td>80.16</td><td>91.38</td><td>70.35</td>
  </tr>
  <tr>
    <td>Czert</td><td>96.32</td><td>96.91</td><td>95.74</td><td>79.26</td><td>79.31</td><td>79.22</td><td>91.42</td><td>70.39</td>
  </tr>
  <tr>
    <td>Czert+lex</td><td>96.62</td><td>97.06</td><td>96.18</td><td>80.70</td><td>80.74</td><td>80.66</td><td>92.08</td><td>72.63</td>
  </tr>
  <tr>
    <th colspan="9">Morphodict-T</th>
  </tr>
  <tr>
    <td>CNN</td><td>94.71</td><td>94.46</td><td>94.46</td><td>81.97</td><td>81.96</td><td>81.98</td><td>90.16</td><td>70.53</td>
  </tr>
  <tr>
    <td>ruRoberta</td><td>94.98</td><td>94.67</td><td>95.31</td><td>82.08</td><td>81.91</td><td>82.26</td><td>90.15</td><td>71.10</td>
  </tr>
  <tr>
    <td>ruRoberta+lex</td><td>95.59</td><td>96.09</td><td>95.08</td><td>84.34</td><td>84.37</td><td>84.30</td><td>91.70</td><td>74.95</td>
  </tr>
  <tr>
    <th colspan="9">Morphodict-K</th>
  </tr>
  <tr>
    <td>CNN</td><td>95.19</td><td>95.35</td><td>95.04</td><td>82.50</td><td>82.46</td><td>82.54</td><td>91.30</td><td>72.63</td>
  </tr>
  <tr>
    <td>ruRoberta</td><td>95.26</td><td>95.78</td><td>94.74</td><td>82.32</td><td>82.37</td><td>82.28</td><td>91.39</td><td>73.32</td>
  </tr>
  <tr>
    <td>ruRoberta+lex</td><td>96.09</td><td>96.38</td><td>95.81</td><td>84.87</td><td>84.95</td><td>84.78</td><td>92.82</td><td>77.17</td>
  </tr>
</table>

Table 4: Average metric value during cross-validation, split-by-roots

ing suffixes and prefixes from the root, with the latter becoming more frequent when transitioning from random split to split-by-roots. Some errors can be explained by mistakes and inconsistencies in dataset annotation (Table 5). This aligns with previously obtained results for the Russian language (Sorokin and Kravtsova, 2018; Bolshakova and Sapin, 2019; Morozov et al., 2024).

The results of analyzing errors made by the models in the split-by-roots experiment proved to be
<table>
  <tr>
    <th>Dataset</th>
    <th>Lemma</th>
    <th>Dataset segmentation</th>
    <th>Corrected segmentation</th>
  </tr>
  <tr>
    <td>Slounik</td>
    <td>усынавіцель</td>
    <td>у-сын-ав-i:LINK-цель</td>
    <td>у-сын-ав-i:SUFF-цель</td>
  </tr>
  <tr>
    <td>DeriNet</td>
    <td>pedopsychiatricky</td>
    <td>pedop-sychiatrick-y</td>
    <td>pedo-psychiatrick-y</td>
  </tr>
  <tr>
    <td>Morphodict-T</td>
    <td>очевидный</td>
    <td>очевидин-ый</td>
    <td>очевид-н-ый</td>
  </tr>
  <tr>
    <td>Morphodict-K</td>
    <td>растение</td>
    <td>рас-т-ени-е</td>
    <td>раст-ени-е</td>
  </tr>
</table>

Table 5: Examples of mis-segmentation in datasets

particularly interesting. For each OOV root, we calculated the root recognition: the proportion of completely correctly segmented words among all words containing that root. We found that the recognition rate of the vast majority of roots is either 1 or 0 across all models and datasets (Figure 2). The roots of the first type we termed “recognizable”, while the roots of the second type were labeled “completely unknown”. We found two features that differed between these root groups:

1. Proximity to roots in the training sample.
For example, for the Morphodict-K dataset, the average minimal Levenshtein distance between recognizable roots and training roots was 1.1, versus 1.5 for completely unknown roots.

2. Shared derivation patterns. When masking roots in Morphodict-K (e.g. “игр:ROOT/a:SUFF/п:END” ‘to play’ → “[MASK]:ROOT/a:SUFF/п:END”), 90% of words with recognizable roots had a training-set word with an identical masked pattern, compared to only 73% for completely unknown roots. A similar trend holds for Belarusian – e.g., the erroneous segmentation ‘грукат:ROOT/a:SUFF/тб:SUFF’, ‘to rumble’ (vs. the correct ‘грук:ROOT/ат:SUFF/a:SUFF/тб:SUFF’) likely arises because the pattern “ат:ROOT/a:SUFF/тб:SUFF” appears 46 times in the training set, while “ат:SUFF/a:SUFF/тб:SUFF” occurs only 7 times.

Another interesting observation is that the segmentation behavior (including root recognizability) is stable across random seeds. For Morphodict-K, three model reruns produced identical segmentations in 90% of cases (76% of segmentations were fully correct for each of the models, 17% were incorrect for each of the models). The overlap of “recognizable root” sets across models reached 95% of each set’s size.

![Histograms showing root recognition rates for different datasets and models](page_1012_384_384_384.png)
Figure 2: Histogram for the root recognition

![Histogram showing the difference in root recognition for the Czert and Czert+lex models](page_1012_1012_384_384.png)
Figure 3: Histogram of the difference in root recognition for the Czert and Czert+lex models

When comparing this statistic for models trained without adding lemmata to the input sequence versus those trained with lemmata, it turns out that roots most often do not transition between categories. However, for each pair of models, there are roots that transition from “completely unknown” to “recognizable” and vice versa. An example of a histogram of such difference for the Czert models before and after adding a lemma to the input sequence is shown in Figure 3. Unfortunately, we were unable to identify clear patterns or dependencies between the roots and this dynamic.
7 Comparison with DeepSPIN-3

The models presented at the SIGMORPHON 2022 cannot be directly compared to our approach due to differences in task formulation: the competition focused on canonical segmentation, whereas our work addresses surface segmentation. Nevertheless, the exceptionally high performance achieved by the DeepSPIN-3 algorithm during the competition prompted our interest in conducting at least an approximate comparison. Beyond a direct performance comparison, this would also help assess the applicability of algorithms designed for canonical segmentation task to the surface one.

Since DeepSPIN-3 does not involve morpheme type labeling, types were removed. The results are presented in Table 6. In addition to the Levenshtein distance used in the competition, we included WordAccuracy — the proportion of fully correct segmentations.

<table>
  <tr>
    <th>Dataset</th>
    <th colspan="2">Levenshtein</th>
    <th>WordAccuracy</th>
  </tr>
  <tr>
    <th></th>
    <th colspan="2">random split</th>
    <th></th>
  </tr>
  <tr>
    <td>Slounik</td>
    <td>0.80</td>
    <td></td>
    <td>68.65</td>
  </tr>
  <tr>
    <td>DeriNet</td>
    <td>0.10</td>
    <td></td>
    <td>92.28</td>
  </tr>
  <tr>
    <td>Morphodict-T</td>
    <td>0.41</td>
    <td></td>
    <td>79.69</td>
  </tr>
  <tr>
    <td>Morphodict-K</td>
    <td>0.55</td>
    <td></td>
    <td>77.10</td>
  </tr>
  <tr>
    <th colspan="4">split by roots</th>
  </tr>
  <tr>
    <td>Slounik</td>
    <td>2.22</td>
    <td></td>
    <td>12.11</td>
  </tr>
  <tr>
    <td>DeriNet</td>
    <td>0.68</td>
    <td></td>
    <td>55.16</td>
  </tr>
  <tr>
    <td>Morphodict-T</td>
    <td>1.58</td>
    <td></td>
    <td>25.85</td>
  </tr>
  <tr>
    <td>Morphodict-K</td>
    <td>1.90</td>
    <td></td>
    <td>13.38</td>
  </tr>
</table>

Table 6: Results obtained using the DeepSPIN-3 algorithm

The results demonstrate that in most scenarios DeepSPIN-3 underperforms not only pretrained models but also the CNN ensemble baseline. The sole exception is DeriNet dataset in the random-split scenario, where DeepSPIN-3 achieved 92% word-level accuracy (vs. 91% for CNN ensemble and 95% for pretrained models). For other languages, DeepSPIN-3 is significantly inferior, which we attribute to the size of the training sets.

The performance drops drastically in the split-by-roots scenario — the average Levenshtein distance increases by 0.5-1.5, with word-level accuracy decreasing by more than 30 percentage points. Draft error analysis reveals that the algorithm consistently generates roots similar to training set instances. For example, in Morphodict-K, the word “отплатить” ‘to pay back’ with correct segmentation от:ПREF/плат:ROOT/н:СУFF/тъ:END receives incorrect segmentation от:ПREF/лат:ROOT/н:СУFF/тъ:END (missing the root letter -п-), where лат:ROOT appears in training set words like “подплатать” ‘to patch up’. Our analysis shows that generating Levenshtein-close training set roots accounts for two-thirds of all errors. A potential solution might involve restricting model generation to non-root morphemes while treating unmarked segments as roots, though this would require substantial algorithm modifications and warrants separate investigation.

8 Conclusion

In this work, we investigated the applicability of BERT-like models to the task of morpheme segmentation using the material of the Belarusian, Czech, and Russian languages. We used a CNN ensemble as a baseline. For Czech and Russian, our fine-tuned BERT-like models outperformed the baseline. We managed to achieve a share of completely correct annotations of 92-95% in the case of a random test sample, and 72-77% in the case of testing on words with roots that were not found in the training sample. The proportion of incorrect segmentations on a random sample decreased by 30-45%, and on words with OOV roots by 9-15%. The results obtained exceed all previously presented results for these languages.

For Belarusian, the CNN ensemble showed better performance in both types of testing, achieving word accuracy of 90.5% in the case of random split and 74.8% in the split-by-roots case. To our knowledge, similar experiments have not been conducted for this language before, which allows us to consider the presented result as a new state-of-the-art baseline. We also prepared the first publicly available Belarusian morpheme dataset with morpheme type annotation.

We also found that when testing on words with OOV roots, almost all roots can be divided into two groups: “recognizable” and “completely unknown”. The first group includes roots for which words are annotated completely correctly in 100% of cases, while the second group consists of roots for which words are never annotated completely correctly by the model.
9 Limitations

1. Significant computational power required for training and inference BERT-like models. Unlike the CNN ensemble, fine-tuning and inference of BERT-like models require the use of sufficiently powerful GPUs (albeit from the consumer segment). On the other hand, preliminary testing indicated that using large language models on this task would require significantly more computational time and resources, so we decided not to conduct full-scale experiments with LLMs. However, automatic expansion of morpheme dictionaries does not require frequent model runs, since new words appear in the language relatively rarely. In this case, the additional costs of BERT-like models can be compensated by higher quality of the labeling.

2. Separation of roots into “recognizable” and “completely unknown” requires additional linguistic research. We found that for both CNN and BERT-like approaches, when testing on words with OOV roots, two categories of roots emerge: “recognizable” and “completely unknown”. However, a detailed analysis of why certain roots fall into one category or another is planned for future work.

3. Lack of pre-trained Belarusian models and small size of the morpheme dataset. Although we were able to draw several conclusions about morpheme segmentation for the Belarusian language, the lack of relatively large pre-trained models for this language does not allow us to consider CNN as the unequivocal leader. On the contrary, the results obtained for Czech and Russian suggest that the absence of models and the small size of the dataset were the reasons for the failure of the BERT-like approach. With the emergence of larger Belarusian models and annotated datasets, our study should be replicated to draw more reliable conclusions.

Acknowledgments

This work was supported by a grant for research centers, provided by the Ministry of Economic Development of the Russian Federation in accordance with the subsidy agreement with the Novosibirsk State University dated April 17, 2025 No. 139-15-2025-006: IGK 000000C313925P3S0002.

References

Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A system for large-scale machine learning. In 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16), pages 265–283.

Mikhail Arkhipov, Maria Trofimova, Yuri Kuratov, and Alexey Sorokin. 2019. Tuning multilingual transformers for language-specific named entity recognition. In Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing, pages 89–93, Florence, Italy. Association for Computational Linguistics.

Khuyagbaatar Batsuren, Gábor Bella, Aryaman Arora, Viktor Martinovic, Kyle Gorman, Zdeněk Žabokrtský, Amarsanaa Ganbold, Šárka Dohnalová, Magda Ševčíková, Kateřina Pelegrinová, Fausto Giunchiglia, Ryan Cotterell, and Ekaterina Vylomova. 2022. The SIGMORPHON 2022 shared task on morpheme segmentation. In Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 103–116, Seattle, Washington. Association for Computational Linguistics.

Elena Bolshakova and Alexander Sapin. 2019. Bi-LSTM model for morpheme segmentation of Russian words. In Artificial Intelligence and Natural Language, pages 151–160, Cham. Springer International Publishing.

Elena I. Bolshakova and Alexander S. Sapin. 2022. Building a combined morphological model for russian word forms. In Analysis of Images, Social Networks and Texts, pages 45–55, Cham. Springer International Publishing.

Francois Chollet et al. 2015. Keras.

Ryan Cotterell, Tim Vieira, and Hinrich Schütze. 2016. A joint model of orthography and morphological segmentation. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 664–669, San Diego, California. Association for Computational Linguistics.

Mathias Creutz and Krista Lagus. 2002. Unsupervised discovery of morphemes. In Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning, pages 21–30. Association for Computational Linguistics.

Philip Gage. 1994. A new algorithm for data compression. C Users J., 12(2):23–38.

Ayyoob Imani, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, André Martins, François Yvon, and Hinrich Schütze. 2023. Glot500: Scaling multilingual corpora and language models to
500 languages. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1082–1117, Toronto, Canada. Association for Computational Linguistics.

B. L. Iomdin. 2019. How to define words with the same root? Russian Speech = Russkaya Rech’, (1):109–115.

Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 66–75, Melbourne, Australia. Association for Computational Linguistics.

A. I. Kuznetsova and T. F. Efremova. 1986. Dictionary of Morphemes of the Russian Language. Russkii yazyk, Moscow.

Dominik Macháček, Jonáš Vidra, and Ondřej Bojar. 2018. Morphological and language-agnostic word segmentation for NMT. In International Conference on Text, Speech, and Dialogue, pages 277–284. Springer.

Austin Matthews, Graham Neubig, and Chris Dyer. 2018. Using morphological knowledge in open-vocabulary neural language models. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1435–1445, New Orleans, Louisiana. Association for Computational Linguistics.

L. S. Mormysh, A. M. Bordovich, and L. M. Shakun. 2005. School morpheme dictionary of the Belarusian language [SHkol’ny marfemny slovnik belaruskaj movy]. Aversev, Minsk.

Dmitry Morozov, Timur Garipov, Olga Lyashevskaya, Svetlana Savchuk, Boris Iomdin, and Anna Glazkova. 2024. Automatic morpheme segmentation for Russian: Can an algorithm replace experts? Journal of Language and Education, 10(4):71–84.

Antoine Nzeyimana and Andre Niyongabo Rubungo. 2022. KinyaBERT: a morphology-aware Kinyarwanda language model. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5347–5363, Dublin, Ireland. Association for Computational Linguistics.

Ben Peters and Andre F. T. Martins. 2022. Beyond characters: Subword-level morpheme segmentation. In Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 131–138, Seattle, Washington. Association for Computational Linguistics.

Marko Pranjić, Marko Robnik-Šikonja, and Senja Poljak. 2024. LLMSegm: Surface-level morphological segmentation using large language model. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 10665–10674, Torino, Italia. ELRA and ICCL.

Svetlana O Savchuk, Timofey Arkhangelskiy, Anastasiya A Bonch-Osmolovskaya, Ol’ga V Donna, Yuliya N Kuznetsova, Ol’ga N Lyashevskaya, Boris V Orekhov, and Mariya V Podryadchikova. 2024. Russian National Corpus 2.0: New opportunities and development prospects. Voprosy Jazykoznaniya, (2):7–34.

Mike Schuster and Kaisuke Nakajima. 2012. Japanese and korean voice search. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5149–5152.

Jakub Sido, Ondřej Pražák, Pavel Přibaň, Jan Pašek, Michal Seják, and Miloslav Konopík. 2021. Czert – Czech BERT-like model for language representation. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 1326–1338, Held Online. INCOMA Ltd.

Peter Smit, Sami Virpioja, Stig-Arne Grönroos, and Mikko Kurimo. 2014. Morfessor 2.0: Toolkit for statistical morphological segmentation. In Proceedings of the Demonstrations at the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 21–24, Gothenburg, Sweden. Association for Computational Linguistics.

Alexey Sorokin. 2022. Improving morpheme segmentation using bert embeddings. In Analysis of Images, Social Networks and Texts, pages 148–161, Cham. Springer International Publishing.

Alexey Sorokin and Anastasia Kravtsova. 2018. Deep convolutional networks for supervised morpheme segmentation of Russian language. In Artificial Intelligence and Natural Language, pages 3–10, Cham. Springer International Publishing.

Emil Svoboda and Magda Sevcíková. 2022. Word formation analyzer for czech: Automatic parent retrieval and classification of word formation processes. The Prague Bulletin of Mathematical Linguistics, 118:55.

A. N. Tikhonov. 1990. Word Formation Dictionary of the Russian language [Slovoobrazovatel’nyi slovar’ russkogo yazyka]. Russkiy yazyk, Moscow.

Anastasiya S. Volskaya, Tatyana A. Korneyeva, and Tatyana D. Markova. 2018. Word-formation, morphemic, etymological analysis in school. Amazonia Investigata, 7(15):190–195.

Silvan Wehrli, Simon Clematide, and Peter Makarov. 2022. CLUZH at SIGMORPHON 2022 shared tasks on morpheme segmentation and inflection generation. In Proceedings of the 19th SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 212–219, Seattle, Washington. Association for Computational Linguistics.
Dmitry Zmitrovich, Aleksandr Abramov, Andrey Kalmykov, Vitaly Kadulin, Maria Tikhonova, Ekaterina Taktasheva, Danil Astafurov, Mark Baushenko, Artem Snegirev, Tatiana Shavrina, Sergei S. Markov, Vladislav Mikhailov, and Alena Fenogenova. 2024. A family of pretrained transformer language models for Russian. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 507–524, Torino, Italia. ELRA and ICCL.

A Risks

Our research is primarily foundational. Despite this, we can assume some risks associated with the integration of our algorithm into language learning processes. Although the obtained labeling quality is quite high, the probability of errors is still significant. The segmentations generated by the model must be validated by a human expert before being implemented in educational materials.

In addition, the datasets used may contain a small number of examples of obscene and offensive vocabulary. However, all the data used are based on academic dictionaries and projects, so we believe that such words are an integral part of the language, and morpheme segmentation models should be able to work with such words.

B Scientific Artifacts

Our work uses external datasets, pre-trained models and software libraries. We created one scientific artifact, the Slounik dataset10, publicly available under the CC-BY-NC-SA 4.0 license.

B.1 Datasets

The datasets we utilized do not contain any personal information. The datasets may contain a small number of examples of obscene and offensive vocabulary. However, all the data used are based on academic dictionaries and projects, so we believe that such words are an integral part of the language, and morpheme segmentation models should be able to work with such words.

1. Slounik. As a basis for preparing the Slounik dataset, we used the version of the School morpheme dictionary of the Belarusian language (Mormysh et al., 2005) available for non-commercial use in the repository of the Belarusian State Pedagogical University named after Maxim Tank (BSPU)11121314151617. According to the description of the repository, it is allowed to copy and quote materials exclusively for non-commercial purposes with the obligatory indication of the author of the work and a hyperlink to the BSPU Repository. We translated the dictionary into a machine-readable form. During the additional labeling, the annotators did not make changes to the morpheme segmentation, but only marked the types of segmented morphemes. The Slounik dataset18 is publicly available under the CC-BY-NC-SA 4.0 license, including mandatory mention of the authors of the original dictionary and a link to the BSPU repository.

2. DeriNet. Adapted from the DeriNet19 dataset, which is available in the LINDAT/CLARIAH-CZ digital library20 at the Institute of Farmal and Applied Linguistics, Faculty of Mathematics and Physics, Charles university under the terms of the CC-BY-NC-SA 3.0 license.

3. Morphodict-T and Morphodict-K. The datasets were provided to us for use exclusively for scientific purposes under a license agreement with the Russian National Corpus21 (Savchuk et al., 2024). A detailed description of the differences in the markup between the datasets can be found on the Corpus website22.

B.2 Pre-trained Models

We utilized four pre-trained models from Hugging-Face23:

1. roberta-small-belarusian24. This model is monolingual (Belarusian), has 16M parameters, and is available under the CC-BY-SA 4.0 license.

10https://huggingface.co/datasets/ruscorpora/morphodict-bel
11http://elib.bspu.by/handle/doc/30574
12http://elib.bspu.by/handle/doc/30575
13http://elib.bspu.by/handle/doc/30576
14http://elib.bspu.by/handle/doc/30577
15http://elib.bspu.by/handle/doc/30578
16http://elib.bspu.by/handle/doc/30579
17http://elib.bspu.by/handle/doc/30580
18https://huggingface.co/datasets/ruscorpora/morphodict-bel
19https://ufal.mff.cuni.cz/derinet
20https://lindat.mff.cuni.cz/repository/xmlui/handle/11234/1-3765
21https://ruscorpora.ru/en
22https://ruscorpora.ru/en/page/instruction-derivation
23https://huggingface.co/
24https://huggingface.co/KoichiYasuoka/roberta-small-belarusian
2. SlavicBERT\(^{25}\) (Arkhipov et al., 2019). This model is multilingual (Bulgarian, Czech, Polish, and Russian), has 180M parameters, and is available under the Apache 2.0 license.

3. Czert-B-base-cased\(^{26}\) (Sido et al., 2021). This model is monolingual (Czech), has 110M parameters, and is available under the CC-BY-NC-SA 4.0 license.

4. ruRoberta-large\(^{27}\) (Zmitrovich et al., 2024). This model is monolingual (Russian), has 355M parameters, and is available under the MIT license.

B.3 Software libraries

1. In experiments with a CNN ensemble, we used the implementation by Sorokin and Kravtsova (2018) from publicly available repository\(^{28}\). Unfortunately, there is no indication of licensing terms in the repository. The code in the repository utilizes the keras (Chollet et al., 2015) and tensorflow (Abadi et al., 2016) libraries. We used version 2.12.0 of both libraries to run it. We used the following parameters set by the configuration file: models number — 3, number of convolutional layers — 3, window size — 5, filters number — 192, dense output units — 64, validation split ratio — 0.2, dropout ratio — 0.2.

2. For implementation BERT-like models, we used the simpletransformers\(^{29}\) framework, which is available under Apache 2.0 license. We used the NERModel class to load and fine-tune the models. All models were loaded via HuggingFace API\(^{30}\). The batch size during training was set to 16, and the learning rate was set to 4e-6. The values of the remaining parameters were set to default.

C Computational Experiments

Training all 20 convolutional neural network ensembles together took less than 48 hours on an AMD Ryzen 5 5600X CPU.

BERT-like models fine-tuning was performed on a single Nvidia RTX 4090 GPU. Training time depended on the specific dataset and, to a lesser extent, the base model. Fine-tuning of a single roberta-small-belarusian model on the Slounik dataset took less than one hour, while fine-tuning of the Czert model on the DeriNet dataset required about nine hours. In total, 160 GPU hours were used for fine-tuning the models, including preliminary experiments.

D Human Annotation

We used human annotators only for additional labeling of the Slounik dataset. It should be noted that during this process we did not change the segmentation of morphemes in the source dictionary. The task of type labeling is not complex, since the main challenge in morpheme segmentation of Belarusian (as well as other Slavic languages) is precisely the division of the source string. In the vast majority of cases, the morpheme type is easily determined unambiguously: prefixes, suffixes, endings and connecting vowels can only be from a fixed set of strings, and the intersection between the set of roots and the rest of the morphemes is extremely small (no more than two dozen strings). In order to avoid potential discrepancies in labeling between annotators, we decided not to separate postfixes from suffixes, labeling all such morphemes as suffixes.

Preliminary labeling was carried out by a native Russian speaker (as a morphologically close language) with linguistic background, and validation of the labeling results was carried out by two Russian- and Belarusian-speaking annotators with linguistic education.

The annotators were instructed to assign each morpheme of the word one of five possible types: PREF (prefix), ROOT, SUFF (suffix), END (ending), LINK (linking vowel). The primary annotator received data as a json dictionary. Each lemma in the dictionary corresponded to a list of several morphemes. The concatenation of morphemes coincided with the original lemma. The annotator’s task was to add the type to each morpheme, separating it with the ":" character, for example, for the word абавязаць, the annotator received the list ["абавяз", "а", "ц"], to which the annotator matched the list ["абавяз:ROOT", "а:SUFF", "ц:SUFF"]. The annotators were informed that the labeled dataset would be used, among other
things, to train morpheme segmentation models.

E   Detailed Experimental Results

The results obtained during the experiments for each of the folds are presented in Tables 7, 11, and 12 for the Slounik dataset, in Tables 8 and 13 for the DeriNet dataset, in Tables 9 and 14 for the Morphodict-T dataset, and in Tables 10 and 15 for the Morphodict-K dataset. Additionally, the average value and standard deviation are presented.
<table>
  <tr>
    <th>Metric</th>
    <th>Fold 1</th>
    <th>Fold 2</th>
    <th>Fold 3</th>
    <th>Fold 4</th>
    <th>Fold 5</th>
    <th>Average</th>
    <th>StdDev</th>
  </tr>
  <tr>
    <th colspan="8">random split</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>97.92</td>
    <td>98.20</td>
    <td>98.24</td>
    <td>98.09</td>
    <td>98.04</td>
    <td>98.10</td>
    <td>0.13</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>98.85</td>
    <td>98.65</td>
    <td>98.47</td>
    <td>98.89</td>
    <td>98.79</td>
    <td>98.73</td>
    <td>0.17</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>98.39</td>
    <td>98.43</td>
    <td>98.36</td>
    <td>98.49</td>
    <td>98.41</td>
    <td>98.41</td>
    <td>0.05</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>95.03</td>
    <td>95.36</td>
    <td>95.12</td>
    <td>95.60</td>
    <td>95.39</td>
    <td>95.30</td>
    <td>0.23</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>95.08</td>
    <td>95.29</td>
    <td>95.03</td>
    <td>95.42</td>
    <td>95.26</td>
    <td>95.21</td>
    <td>0.16</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>95.05</td>
    <td>95.32</td>
    <td>95.08</td>
    <td>95.51</td>
    <td>95.32</td>
    <td>95.26</td>
    <td>0.19</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>96.85</td>
    <td>96.96</td>
    <td>96.93</td>
    <td>97.11</td>
    <td>96.90</td>
    <td>96.95</td>
    <td>0.10</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>90.41</td>
    <td>90.23</td>
    <td>90.18</td>
    <td>90.87</td>
    <td>90.55</td>
    <td>90.45</td>
    <td>0.28</td>
  </tr>
  <tr>
    <th colspan="8">split by roots</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>94.90</td>
    <td>94.62</td>
    <td>94.63</td>
    <td>94.66</td>
    <td>95.59</td>
    <td>94.88</td>
    <td>0.41</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>96.01</td>
    <td>96.14</td>
    <td>96.20</td>
    <td>95.73</td>
    <td>95.93</td>
    <td>96.00</td>
    <td>0.18</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>95.45</td>
    <td>95.37</td>
    <td>95.41</td>
    <td>95.20</td>
    <td>95.76</td>
    <td>95.44</td>
    <td>0.20</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>85.09</td>
    <td>83.19</td>
    <td>84.60</td>
    <td>82.52</td>
    <td>84.47</td>
    <td>83.97</td>
    <td>1.07</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>85.09</td>
    <td>83.19</td>
    <td>84.60</td>
    <td>82.52</td>
    <td>84.51</td>
    <td>83.98</td>
    <td>1.08</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>85.09</td>
    <td>83.19</td>
    <td>84.60</td>
    <td>82.52</td>
    <td>84.49</td>
    <td>83.98</td>
    <td>1.08</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>91.56</td>
    <td>91.12</td>
    <td>91.00</td>
    <td>90.80</td>
    <td>91.74</td>
    <td>91.24</td>
    <td>0.39</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>75.41</td>
    <td>73.56</td>
    <td>74.65</td>
    <td>73.78</td>
    <td>76.49</td>
    <td>74.78</td>
    <td>1.21</td>
  </tr>
</table>

Table 7: Segmentation quality of the Slounik dataset using a CNN ensemble

<table>
  <tr>
    <th>Metric</th>
    <th>Fold 1</th>
    <th>Fold 2</th>
    <th>Fold 3</th>
    <th>Fold 4</th>
    <th>Fold 5</th>
    <th>Average</th>
    <th>StdDev</th>
  </tr>
  <tr>
    <th colspan="8">random split</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>98.63</td>
    <td>98.80</td>
    <td>98.90</td>
    <td>98.59</td>
    <td>98.70</td>
    <td>98.72</td>
    <td>0.11</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>99.20</td>
    <td>99.07</td>
    <td>98.96</td>
    <td>99.17</td>
    <td>99.12</td>
    <td>99.11</td>
    <td>0.09</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>98.92</td>
    <td>98.94</td>
    <td>98.93</td>
    <td>98.88</td>
    <td>98.91</td>
    <td>98.91</td>
    <td>0.02</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>94.08</td>
    <td>94.18</td>
    <td>94.07</td>
    <td>93.80</td>
    <td>94.07</td>
    <td>94.04</td>
    <td>0.13</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>94.17</td>
    <td>94.27</td>
    <td>94.19</td>
    <td>93.90</td>
    <td>94.17</td>
    <td>94.14</td>
    <td>0.13</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>94.12</td>
    <td>94.22</td>
    <td>94.13</td>
    <td>93.85</td>
    <td>94.12</td>
    <td>94.09</td>
    <td>0.13</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>97.50</td>
    <td>97.50</td>
    <td>97.46</td>
    <td>97.38</td>
    <td>97.44</td>
    <td>97.45</td>
    <td>0.04</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>91.10</td>
    <td>91.28</td>
    <td>91.15</td>
    <td>90.86</td>
    <td>91.06</td>
    <td>91.09</td>
    <td>0.13</td>
  </tr>
  <tr>
    <th colspan="8">split by roots</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>96.10</td>
    <td>96.49</td>
    <td>96.17</td>
    <td>96.71</td>
    <td>96.63</td>
    <td>96.42</td>
    <td>0.24</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>96.68</td>
    <td>95.92</td>
    <td>96.26</td>
    <td>95.85</td>
    <td>96.21</td>
    <td>96.18</td>
    <td>0.29</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>96.39</td>
    <td>96.20</td>
    <td>96.22</td>
    <td>96.28</td>
    <td>96.42</td>
    <td>96.30</td>
    <td>0.09</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>80.07</td>
    <td>79.70</td>
    <td>80.36</td>
    <td>79.59</td>
    <td>80.20</td>
    <td>79.98</td>
    <td>0.30</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>80.13</td>
    <td>79.89</td>
    <td>80.57</td>
    <td>79.79</td>
    <td>80.40</td>
    <td>80.16</td>
    <td>0.30</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>80.10</td>
    <td>79.79</td>
    <td>80.47</td>
    <td>79.69</td>
    <td>80.30</td>
    <td>80.07</td>
    <td>0.30</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>91.47</td>
    <td>91.29</td>
    <td>91.36</td>
    <td>91.39</td>
    <td>91.38</td>
    <td>91.38</td>
    <td>0.06</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>70.52</td>
    <td>69.85</td>
    <td>70.16</td>
    <td>70.72</td>
    <td>70.51</td>
    <td>70.35</td>
    <td>0.31</td>
  </tr>
</table>

Table 8: Segmentation quality of the DeriNet dataset using a CNN ensemble
<table>
  <tr>
    <th>Metric</th>
    <th>Fold 1</th>
    <th>Fold 2</th>
    <th>Fold 3</th>
    <th>Fold 4</th>
    <th>Fold 5</th>
    <th>Average</th>
    <th>StdDev</th>
  </tr>
  <tr>
    <th colspan="8">random split</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>97.79</td>
    <td>97.76</td>
    <td>97.76</td>
    <td>97.64</td>
    <td>98.00</td>
    <td>97.79</td>
    <td>0.13</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>98.38</td>
    <td>98.33</td>
    <td>98.40</td>
    <td>98.46</td>
    <td>98.32</td>
    <td>98.38</td>
    <td>0.06</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>98.09</td>
    <td>98.04</td>
    <td>98.09</td>
    <td>98.05</td>
    <td>98.16</td>
    <td>98.09</td>
    <td>0.05</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>94.02</td>
    <td>94.17</td>
    <td>94.07</td>
    <td>93.98</td>
    <td>94.69</td>
    <td>94.19</td>
    <td>0.29</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>93.80</td>
    <td>93.97</td>
    <td>93.88</td>
    <td>93.93</td>
    <td>94.35</td>
    <td>93.99</td>
    <td>0.21</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>93.91</td>
    <td>94.07</td>
    <td>93.97</td>
    <td>93.95</td>
    <td>94.52</td>
    <td>94.08</td>
    <td>0.25</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>96.57</td>
    <td>96.55</td>
    <td>96.63</td>
    <td>96.53</td>
    <td>96.77</td>
    <td>96.61</td>
    <td>0.10</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>88.22</td>
    <td>88.35</td>
    <td>88.56</td>
    <td>88.39</td>
    <td>88.94</td>
    <td>88.49</td>
    <td>0.28</td>
  </tr>
  <tr>
    <th colspan="8">split by roots</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>94.45</td>
    <td>95.13</td>
    <td>93.92</td>
    <td>94.73</td>
    <td>94.05</td>
    <td>94.46</td>
    <td>0.50</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>95.46</td>
    <td>95.20</td>
    <td>94.78</td>
    <td>94.28</td>
    <td>95.09</td>
    <td>94.96</td>
    <td>0.45</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>94.95</td>
    <td>95.16</td>
    <td>94.35</td>
    <td>94.51</td>
    <td>94.57</td>
    <td>94.71</td>
    <td>0.34</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>83.03</td>
    <td>82.14</td>
    <td>81.89</td>
    <td>81.74</td>
    <td>81.01</td>
    <td>81.96</td>
    <td>0.73</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>83.04</td>
    <td>82.16</td>
    <td>81.90</td>
    <td>81.74</td>
    <td>81.07</td>
    <td>81.98</td>
    <td>0.72</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>83.03</td>
    <td>82.15</td>
    <td>81.90</td>
    <td>81.74</td>
    <td>81.04</td>
    <td>81.97</td>
    <td>0.72</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>90.53</td>
    <td>90.87</td>
    <td>89.52</td>
    <td>89.95</td>
    <td>89.95</td>
    <td>90.16</td>
    <td>0.53</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>72.04</td>
    <td>72.61</td>
    <td>68.24</td>
    <td>69.67</td>
    <td>70.07</td>
    <td>70.53</td>
    <td>1.79</td>
  </tr>
</table>

Table 9: Segmentation quality of the Morphodict-T dataset using a CNN ensemble

<table>
  <tr>
    <th>Metric</th>
    <th>Fold 1</th>
    <th>Fold 2</th>
    <th>Fold 3</th>
    <th>Fold 4</th>
    <th>Fold 5</th>
    <th>Average</th>
    <th>StdDev</th>
  </tr>
  <tr>
    <th colspan="8">random split</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>98.57</td>
    <td>98.61</td>
    <td>98.74</td>
    <td>98.50</td>
    <td>98.47</td>
    <td>98.58</td>
    <td>0.11</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>98.77</td>
    <td>98.80</td>
    <td>98.59</td>
    <td>98.76</td>
    <td>98.79</td>
    <td>98.74</td>
    <td>0.09</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>98.67</td>
    <td>98.71</td>
    <td>98.66</td>
    <td>98.63</td>
    <td>98.63</td>
    <td>98.66</td>
    <td>0.03</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>96.26</td>
    <td>96.33</td>
    <td>96.41</td>
    <td>96.25</td>
    <td>96.05</td>
    <td>96.26</td>
    <td>0.13</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>96.23</td>
    <td>96.40</td>
    <td>96.19</td>
    <td>96.23</td>
    <td>96.06</td>
    <td>96.22</td>
    <td>0.12</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>96.24</td>
    <td>96.37</td>
    <td>96.30</td>
    <td>96.24</td>
    <td>96.06</td>
    <td>96.24</td>
    <td>0.11</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>97.40</td>
    <td>97.45</td>
    <td>97.41</td>
    <td>97.38</td>
    <td>97.38</td>
    <td>97.40</td>
    <td>0.03</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>90.85</td>
    <td>91.01</td>
    <td>90.76</td>
    <td>90.66</td>
    <td>90.84</td>
    <td>90.82</td>
    <td>0.13</td>
  </tr>
  <tr>
    <th colspan="8">split by roots</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>95.23</td>
    <td>95.06</td>
    <td>95.25</td>
    <td>95.27</td>
    <td>95.92</td>
    <td>95.35</td>
    <td>0.33</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>95.36</td>
    <td>94.75</td>
    <td>94.72</td>
    <td>95.16</td>
    <td>95.22</td>
    <td>95.04</td>
    <td>0.29</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>95.30</td>
    <td>94.90</td>
    <td>94.98</td>
    <td>95.21</td>
    <td>95.57</td>
    <td>95.19</td>
    <td>0.27</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>84.46</td>
    <td>80.51</td>
    <td>80.38</td>
    <td>81.72</td>
    <td>85.23</td>
    <td>82.46</td>
    <td>2.26</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>84.53</td>
    <td>80.59</td>
    <td>80.44</td>
    <td>81.85</td>
    <td>85.30</td>
    <td>82.54</td>
    <td>2.25</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>84.50</td>
    <td>80.55</td>
    <td>80.41</td>
    <td>81.78</td>
    <td>85.26</td>
    <td>82.50</td>
    <td>2.25</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>91.50</td>
    <td>90.80</td>
    <td>90.92</td>
    <td>91.30</td>
    <td>91.98</td>
    <td>91.30</td>
    <td>0.47</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>73.33</td>
    <td>69.75</td>
    <td>71.14</td>
    <td>73.45</td>
    <td>75.48</td>
    <td>72.63</td>
    <td>2.22</td>
  </tr>
</table>

Table 10: Segmentation quality of the Morphodict-K dataset using a CNN ensemble
<table>
  <tr>
    <th>Metric</th>
    <th>Fold 1</th>
    <th>Fold 2</th>
    <th>Fold 3</th>
    <th>Fold 4</th>
    <th>Fold 5</th>
    <th>Average</th>
    <th>StdDev</th>
  </tr>
  <tr>
    <th colspan="8">random split</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>98.14</td>
    <td>98.24</td>
    <td>98.19</td>
    <td>98.32</td>
    <td>98.39</td>
    <td>98.25</td>
    <td>0.10</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>98.60</td>
    <td>98.55</td>
    <td>98.53</td>
    <td>98.56</td>
    <td>98.50</td>
    <td>98.55</td>
    <td>0.04</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>98.37</td>
    <td>98.39</td>
    <td>98.36</td>
    <td>98.44</td>
    <td>98.45</td>
    <td>98.40</td>
    <td>0.04</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>94.83</td>
    <td>95.18</td>
    <td>95.04</td>
    <td>95.18</td>
    <td>95.20</td>
    <td>95.08</td>
    <td>0.16</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>94.56</td>
    <td>94.91</td>
    <td>94.83</td>
    <td>94.74</td>
    <td>94.70</td>
    <td>94.75</td>
    <td>0.13</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>94.69</td>
    <td>95.04</td>
    <td>94.93</td>
    <td>94.96</td>
    <td>94.95</td>
    <td>94.92</td>
    <td>0.13</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>96.76</td>
    <td>96.74</td>
    <td>96.85</td>
    <td>96.87</td>
    <td>96.89</td>
    <td>96.82</td>
    <td>0.07</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>90.16</td>
    <td>90.20</td>
    <td>90.07</td>
    <td>90.53</td>
    <td>90.63</td>
    <td>90.32</td>
    <td>0.25</td>
  </tr>
  <tr>
    <th colspan="8">split by roots</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>95.42</td>
    <td>95.46</td>
    <td>95.28</td>
    <td>95.58</td>
    <td>95.80</td>
    <td>95.51</td>
    <td>0.19</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>95.19</td>
    <td>95.39</td>
    <td>94.98</td>
    <td>95.33</td>
    <td>95.13</td>
    <td>95.21</td>
    <td>0.16</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>95.31</td>
    <td>95.42</td>
    <td>95.13</td>
    <td>95.46</td>
    <td>95.46</td>
    <td>95.36</td>
    <td>0.14</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>84.55</td>
    <td>83.11</td>
    <td>84.04</td>
    <td>83.47</td>
    <td>83.18</td>
    <td>83.67</td>
    <td>0.62</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>84.35</td>
    <td>82.86</td>
    <td>84.07</td>
    <td>83.32</td>
    <td>83.05</td>
    <td>83.53</td>
    <td>0.65</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>84.45</td>
    <td>82.99</td>
    <td>84.05</td>
    <td>83.39</td>
    <td>83.11</td>
    <td>83.60</td>
    <td>0.63</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>90.82</td>
    <td>90.79</td>
    <td>90.41</td>
    <td>91.00</td>
    <td>90.99</td>
    <td>90.80</td>
    <td>0.24</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>73.37</td>
    <td>72.97</td>
    <td>72.99</td>
    <td>73.85</td>
    <td>74.57</td>
    <td>73.55</td>
    <td>0.67</td>
  </tr>
  <tr>
    <th colspan="8">random split (+lex)</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>97.77</td>
    <td>98.05</td>
    <td>97.75</td>
    <td>97.88</td>
    <td>97.96</td>
    <td>97.88</td>
    <td>0.13</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>98.34</td>
    <td>98.24</td>
    <td>98.18</td>
    <td>98.48</td>
    <td>98.24</td>
    <td>98.30</td>
    <td>0.12</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>98.05</td>
    <td>98.15</td>
    <td>97.97</td>
    <td>98.18</td>
    <td>98.10</td>
    <td>98.09</td>
    <td>0.08</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>93.94</td>
    <td>94.42</td>
    <td>93.88</td>
    <td>94.28</td>
    <td>94.15</td>
    <td>94.13</td>
    <td>0.23</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>93.87</td>
    <td>94.24</td>
    <td>93.73</td>
    <td>93.92</td>
    <td>93.62</td>
    <td>93.88</td>
    <td>0.24</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>93.90</td>
    <td>94.33</td>
    <td>93.81</td>
    <td>94.10</td>
    <td>93.88</td>
    <td>94.01</td>
    <td>0.21</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>96.10</td>
    <td>96.35</td>
    <td>96.07</td>
    <td>96.33</td>
    <td>96.20</td>
    <td>96.21</td>
    <td>0.13</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>88.39</td>
    <td>88.97</td>
    <td>88.57</td>
    <td>89.44</td>
    <td>88.92</td>
    <td>88.86</td>
    <td>0.40</td>
  </tr>
  <tr>
    <th colspan="8">split by roots (+lex)</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>95.49</td>
    <td>95.33</td>
    <td>95.51</td>
    <td>95.54</td>
    <td>95.51</td>
    <td>95.48</td>
    <td>0.08</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>95.67</td>
    <td>95.47</td>
    <td>95.47</td>
    <td>95.49</td>
    <td>95.36</td>
    <td>95.49</td>
    <td>0.11</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>95.58</td>
    <td>95.40</td>
    <td>95.49</td>
    <td>95.51</td>
    <td>95.44</td>
    <td>95.48</td>
    <td>0.07</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>84.63</td>
    <td>83.38</td>
    <td>84.45</td>
    <td>83.47</td>
    <td>83.04</td>
    <td>83.79</td>
    <td>0.70</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>84.29</td>
    <td>83.32</td>
    <td>84.47</td>
    <td>83.34</td>
    <td>82.78</td>
    <td>83.64</td>
    <td>0.71</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>84.46</td>
    <td>83.35</td>
    <td>84.46</td>
    <td>83.40</td>
    <td>82.91</td>
    <td>83.71</td>
    <td>0.70</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>91.45</td>
    <td>91.09</td>
    <td>90.95</td>
    <td>91.11</td>
    <td>91.01</td>
    <td>91.12</td>
    <td>0.20</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>75.64</td>
    <td>73.98</td>
    <td>74.30</td>
    <td>74.67</td>
    <td>74.34</td>
    <td>74.59</td>
    <td>0.64</td>
  </tr>
</table>

Table 11: Segmentation quality of the Slounik dataset using a fine-tuned roberta-small-belarusian model
<table>
  <tr>
    <th>Metric</th>
    <th>Fold 1</th>
    <th>Fold 2</th>
    <th>Fold 3</th>
    <th>Fold 4</th>
    <th>Fold 5</th>
    <th>Average</th>
    <th>StdDev</th>
  </tr>
  <tr>
    <th colspan="8">random split</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>97.69</td>
    <td>97.69</td>
    <td>97.74</td>
    <td>97.78</td>
    <td>97.69</td>
    <td>97.72</td>
    <td>0.04</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>98.22</td>
    <td>98.36</td>
    <td>98.30</td>
    <td>98.22</td>
    <td>98.31</td>
    <td>98.28</td>
    <td>0.06</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>97.95</td>
    <td>98.02</td>
    <td>98.02</td>
    <td>98.00</td>
    <td>98.00</td>
    <td>98.00</td>
    <td>0.03</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>93.21</td>
    <td>94.08</td>
    <td>94.03</td>
    <td>93.84</td>
    <td>93.42</td>
    <td>93.72</td>
    <td>0.38</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>92.70</td>
    <td>93.77</td>
    <td>93.75</td>
    <td>93.72</td>
    <td>93.10</td>
    <td>93.41</td>
    <td>0.48</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>92.96</td>
    <td>93.92</td>
    <td>93.89</td>
    <td>93.78</td>
    <td>93.26</td>
    <td>93.56</td>
    <td>0.43</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>95.79</td>
    <td>95.92</td>
    <td>96.10</td>
    <td>96.12</td>
    <td>96.01</td>
    <td>95.99</td>
    <td>0.14</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>87.22</td>
    <td>87.52</td>
    <td>87.92</td>
    <td>88.23</td>
    <td>87.73</td>
    <td>87.73</td>
    <td>0.38</td>
  </tr>
  <tr>
    <th colspan="8">split by roots</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>95.28</td>
    <td>95.20</td>
    <td>95.00</td>
    <td>95.17</td>
    <td>95.79</td>
    <td>95.29</td>
    <td>0.30</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>96.16</td>
    <td>95.72</td>
    <td>95.89</td>
    <td>95.75</td>
    <td>95.91</td>
    <td>95.89</td>
    <td>0.17</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>95.72</td>
    <td>95.46</td>
    <td>95.45</td>
    <td>95.46</td>
    <td>95.85</td>
    <td>95.59</td>
    <td>0.19</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>85.85</td>
    <td>83.94</td>
    <td>84.78</td>
    <td>83.55</td>
    <td>84.32</td>
    <td>84.49</td>
    <td>0.89</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>84.99</td>
    <td>83.73</td>
    <td>84.14</td>
    <td>83.28</td>
    <td>84.18</td>
    <td>84.06</td>
    <td>0.63</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>85.42</td>
    <td>83.83</td>
    <td>84.46</td>
    <td>83.42</td>
    <td>84.25</td>
    <td>84.28</td>
    <td>0.75</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>91.64</td>
    <td>90.93</td>
    <td>90.74</td>
    <td>90.86</td>
    <td>91.71</td>
    <td>91.17</td>
    <td>0.46</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>76.09</td>
    <td>73.75</td>
    <td>74.03</td>
    <td>73.60</td>
    <td>76.22</td>
    <td>74.74</td>
    <td>1.30</td>
  </tr>
  <tr>
    <th colspan="8">random split (+lex)</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>97.51</td>
    <td>97.64</td>
    <td>97.70</td>
    <td>97.73</td>
    <td>97.83</td>
    <td>97.68</td>
    <td>0.12</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>98.35</td>
    <td>98.14</td>
    <td>98.20</td>
    <td>98.27</td>
    <td>98.22</td>
    <td>98.24</td>
    <td>0.08</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>97.93</td>
    <td>97.89</td>
    <td>97.95</td>
    <td>98.00</td>
    <td>98.02</td>
    <td>97.96</td>
    <td>0.05</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>93.47</td>
    <td>93.39</td>
    <td>93.81</td>
    <td>93.58</td>
    <td>93.67</td>
    <td>93.58</td>
    <td>0.17</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>93.06</td>
    <td>92.88</td>
    <td>93.27</td>
    <td>93.12</td>
    <td>93.06</td>
    <td>93.08</td>
    <td>0.14</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>93.27</td>
    <td>93.13</td>
    <td>93.54</td>
    <td>93.35</td>
    <td>93.37</td>
    <td>93.33</td>
    <td>0.15</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>95.80</td>
    <td>95.72</td>
    <td>96.03</td>
    <td>95.96</td>
    <td>95.93</td>
    <td>95.89</td>
    <td>0.13</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>87.67</td>
    <td>86.85</td>
    <td>87.62</td>
    <td>87.88</td>
    <td>87.91</td>
    <td>87.58</td>
    <td>0.43</td>
  </tr>
  <tr>
    <th colspan="8">split by roots (+lex)</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>95.32</td>
    <td>94.99</td>
    <td>95.09</td>
    <td>95.16</td>
    <td>95.73</td>
    <td>95.26</td>
    <td>0.29</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>96.04</td>
    <td>95.73</td>
    <td>95.46</td>
    <td>95.81</td>
    <td>95.60</td>
    <td>95.73</td>
    <td>0.22</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>95.68</td>
    <td>95.36</td>
    <td>95.27</td>
    <td>95.48</td>
    <td>95.67</td>
    <td>95.49</td>
    <td>0.18</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>84.94</td>
    <td>82.98</td>
    <td>83.44</td>
    <td>83.81</td>
    <td>84.05</td>
    <td>83.84</td>
    <td>0.73</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>84.54</td>
    <td>82.85</td>
    <td>83.26</td>
    <td>83.32</td>
    <td>83.83</td>
    <td>83.56</td>
    <td>0.65</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>84.74</td>
    <td>82.92</td>
    <td>83.35</td>
    <td>83.56</td>
    <td>83.94</td>
    <td>83.70</td>
    <td>0.69</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>91.48</td>
    <td>90.89</td>
    <td>90.63</td>
    <td>90.88</td>
    <td>91.27</td>
    <td>91.03</td>
    <td>0.34</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>75.41</td>
    <td>73.31</td>
    <td>73.65</td>
    <td>74.01</td>
    <td>75.28</td>
    <td>74.33</td>
    <td>0.96</td>
  </tr>
</table>

Table 12: Segmentation quality of the Slounik dataset using a fine-tuned SlavicBert model
<table>
  <tr>
    <th>Metric</th>
    <th>Fold 1</th>
    <th>Fold 2</th>
    <th>Fold 3</th>
    <th>Fold 4</th>
    <th>Fold 5</th>
    <th>Average</th>
    <th>StdDev</th>
  </tr>
  <tr>
    <th colspan="8">random split</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>99.27</td>
    <td>99.26</td>
    <td>99.28</td>
    <td>99.25</td>
    <td>99.28</td>
    <td>99.27</td>
    <td>0.01</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>99.54</td>
    <td>99.53</td>
    <td>99.52</td>
    <td>99.53</td>
    <td>99.53</td>
    <td>99.53</td>
    <td>0.01</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>99.41</td>
    <td>99.40</td>
    <td>99.40</td>
    <td>99.39</td>
    <td>99.41</td>
    <td>99.40</td>
    <td>0.01</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>96.66</td>
    <td>96.64</td>
    <td>96.67</td>
    <td>96.61</td>
    <td>96.62</td>
    <td>96.64</td>
    <td>0.03</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>96.66</td>
    <td>96.62</td>
    <td>96.65</td>
    <td>96.60</td>
    <td>96.60</td>
    <td>96.63</td>
    <td>0.03</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>96.66</td>
    <td>96.63</td>
    <td>96.66</td>
    <td>96.61</td>
    <td>96.61</td>
    <td>96.63</td>
    <td>0.03</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>98.71</td>
    <td>98.68</td>
    <td>98.69</td>
    <td>98.67</td>
    <td>98.70</td>
    <td>98.69</td>
    <td>0.01</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>95.17</td>
    <td>95.14</td>
    <td>95.14</td>
    <td>95.03</td>
    <td>95.13</td>
    <td>95.12</td>
    <td>0.05</td>
  </tr>
  <tr>
    <th colspan="8">split by roots</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>96.96</td>
    <td>96.99</td>
    <td>96.78</td>
    <td>96.98</td>
    <td>96.87</td>
    <td>96.91</td>
    <td>0.09</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>96.02</td>
    <td>95.43</td>
    <td>95.63</td>
    <td>95.72</td>
    <td>95.87</td>
    <td>95.74</td>
    <td>0.22</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>96.49</td>
    <td>96.20</td>
    <td>96.20</td>
    <td>96.35</td>
    <td>96.37</td>
    <td>96.32</td>
    <td>0.12</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>80.18</td>
    <td>78.63</td>
    <td>79.49</td>
    <td>78.93</td>
    <td>79.29</td>
    <td>79.31</td>
    <td>0.59</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>80.01</td>
    <td>78.55</td>
    <td>79.50</td>
    <td>78.92</td>
    <td>79.11</td>
    <td>79.22</td>
    <td>0.56</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>80.10</td>
    <td>78.59</td>
    <td>79.50</td>
    <td>78.92</td>
    <td>79.20</td>
    <td>79.26</td>
    <td>0.57</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>91.78</td>
    <td>91.17</td>
    <td>91.35</td>
    <td>91.44</td>
    <td>91.36</td>
    <td>91.42</td>
    <td>0.22</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>71.43</td>
    <td>69.51</td>
    <td>70.01</td>
    <td>70.92</td>
    <td>70.08</td>
    <td>70.39</td>
    <td>0.77</td>
  </tr>
  <tr>
    <th colspan="8">random split (+lex)</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>99.32</td>
    <td>99.32</td>
    <td>99.33</td>
    <td>99.32</td>
    <td>99.34</td>
    <td>99.33</td>
    <td>0.01</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>99.56</td>
    <td>99.56</td>
    <td>99.54</td>
    <td>99.55</td>
    <td>99.57</td>
    <td>99.55</td>
    <td>0.01</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>99.44</td>
    <td>99.44</td>
    <td>99.43</td>
    <td>99.43</td>
    <td>99.45</td>
    <td>99.44</td>
    <td>0.01</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>96.79</td>
    <td>96.79</td>
    <td>96.77</td>
    <td>96.77</td>
    <td>96.80</td>
    <td>96.79</td>
    <td>0.01</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>96.79</td>
    <td>96.79</td>
    <td>96.76</td>
    <td>96.76</td>
    <td>96.78</td>
    <td>96.78</td>
    <td>0.01</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>96.79</td>
    <td>96.79</td>
    <td>96.77</td>
    <td>96.76</td>
    <td>96.79</td>
    <td>96.78</td>
    <td>0.01</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>98.76</td>
    <td>98.76</td>
    <td>98.75</td>
    <td>98.75</td>
    <td>98.79</td>
    <td>98.76</td>
    <td>0.02</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>95.36</td>
    <td>95.43</td>
    <td>95.34</td>
    <td>95.35</td>
    <td>95.48</td>
    <td>95.39</td>
    <td>0.06</td>
  </tr>
  <tr>
    <th colspan="8">split by roots (+lex)</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>97.20</td>
    <td>97.03</td>
    <td>97.04</td>
    <td>97.00</td>
    <td>97.03</td>
    <td>97.06</td>
    <td>0.08</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>96.52</td>
    <td>95.92</td>
    <td>96.13</td>
    <td>96.10</td>
    <td>96.21</td>
    <td>96.18</td>
    <td>0.22</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>96.86</td>
    <td>96.47</td>
    <td>96.58</td>
    <td>96.55</td>
    <td>96.62</td>
    <td>96.62</td>
    <td>0.15</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>81.88</td>
    <td>79.98</td>
    <td>81.16</td>
    <td>80.24</td>
    <td>80.43</td>
    <td>80.74</td>
    <td>0.77</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>81.79</td>
    <td>79.98</td>
    <td>81.11</td>
    <td>80.15</td>
    <td>80.26</td>
    <td>80.66</td>
    <td>0.77</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>81.83</td>
    <td>79.98</td>
    <td>81.13</td>
    <td>80.20</td>
    <td>80.34</td>
    <td>80.70</td>
    <td>0.77</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>92.51</td>
    <td>91.84</td>
    <td>92.25</td>
    <td>91.96</td>
    <td>91.85</td>
    <td>92.08</td>
    <td>0.29</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>74.40</td>
    <td>71.49</td>
    <td>73.04</td>
    <td>72.41</td>
    <td>71.81</td>
    <td>72.63</td>
    <td>1.15</td>
  </tr>
</table>

Table 13: Segmentation quality of the DeriNet dataset using a fine-tuned Czert model
<table>
  <tr>
    <th>Metric</th>
    <th>Fold 1</th>
    <th>Fold 2</th>
    <th>Fold 3</th>
    <th>Fold 4</th>
    <th>Fold 5</th>
    <th>Average</th>
    <th>StdDev</th>
  </tr>
  <tr>
    <th colspan="8">random split</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>98.72</td>
    <td>98.41</td>
    <td>98.77</td>
    <td>98.41</td>
    <td>98.54</td>
    <td>98.57</td>
    <td>0.17</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>98.76</td>
    <td>98.36</td>
    <td>98.71</td>
    <td>98.57</td>
    <td>98.38</td>
    <td>98.56</td>
    <td>0.18</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>98.74</td>
    <td>98.38</td>
    <td>98.74</td>
    <td>98.49</td>
    <td>98.46</td>
    <td>98.56</td>
    <td>0.17</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>95.91</td>
    <td>94.91</td>
    <td>95.88</td>
    <td>95.21</td>
    <td>95.31</td>
    <td>95.45</td>
    <td>0.44</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>95.78</td>
    <td>94.47</td>
    <td>95.74</td>
    <td>95.05</td>
    <td>94.87</td>
    <td>95.18</td>
    <td>0.57</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>95.85</td>
    <td>94.69</td>
    <td>95.81</td>
    <td>95.13</td>
    <td>95.09</td>
    <td>95.31</td>
    <td>0.50</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>97.71</td>
    <td>97.05</td>
    <td>97.74</td>
    <td>97.21</td>
    <td>97.25</td>
    <td>97.39</td>
    <td>0.31</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>92.12</td>
    <td>89.97</td>
    <td>92.19</td>
    <td>90.63</td>
    <td>90.52</td>
    <td>91.09</td>
    <td>1.01</td>
  </tr>
  <tr>
    <th colspan="8">split by roots</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>93.80</td>
    <td>94.80</td>
    <td>94.25</td>
    <td>95.89</td>
    <td>94.60</td>
    <td>94.67</td>
    <td>0.78</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>96.30</td>
    <td>95.68</td>
    <td>95.31</td>
    <td>93.85</td>
    <td>95.44</td>
    <td>95.31</td>
    <td>0.90</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>95.03</td>
    <td>95.24</td>
    <td>94.78</td>
    <td>94.86</td>
    <td>95.01</td>
    <td>94.98</td>
    <td>0.18</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>81.98</td>
    <td>81.49</td>
    <td>81.67</td>
    <td>82.71</td>
    <td>81.69</td>
    <td>81.91</td>
    <td>0.48</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>82.53</td>
    <td>82.29</td>
    <td>82.21</td>
    <td>82.47</td>
    <td>81.81</td>
    <td>82.26</td>
    <td>0.28</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>82.25</td>
    <td>81.89</td>
    <td>81.94</td>
    <td>82.59</td>
    <td>81.75</td>
    <td>82.08</td>
    <td>0.34</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>90.28</td>
    <td>90.63</td>
    <td>89.90</td>
    <td>89.66</td>
    <td>90.30</td>
    <td>90.15</td>
    <td>0.38</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>71.40</td>
    <td>72.85</td>
    <td>69.78</td>
    <td>69.68</td>
    <td>71.81</td>
    <td>71.10</td>
    <td>1.36</td>
  </tr>
  <tr>
    <th colspan="8">random split (+lex)</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>98.66</td>
    <td>98.67</td>
    <td>98.67</td>
    <td>98.70</td>
    <td>98.73</td>
    <td>98.69</td>
    <td>0.03</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>98.86</td>
    <td>98.79</td>
    <td>98.90</td>
    <td>98.85</td>
    <td>98.81</td>
    <td>98.84</td>
    <td>0.04</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>98.76</td>
    <td>98.73</td>
    <td>98.79</td>
    <td>98.77</td>
    <td>98.77</td>
    <td>98.76</td>
    <td>0.02</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>95.95</td>
    <td>96.13</td>
    <td>96.07</td>
    <td>96.18</td>
    <td>96.34</td>
    <td>96.13</td>
    <td>0.14</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>95.90</td>
    <td>95.94</td>
    <td>96.02</td>
    <td>96.08</td>
    <td>96.08</td>
    <td>96.00</td>
    <td>0.08</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>95.92</td>
    <td>96.03</td>
    <td>96.05</td>
    <td>96.13</td>
    <td>96.21</td>
    <td>96.07</td>
    <td>0.11</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>97.75</td>
    <td>97.70</td>
    <td>97.83</td>
    <td>97.82</td>
    <td>97.80</td>
    <td>97.78</td>
    <td>0.05</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>92.31</td>
    <td>92.35</td>
    <td>92.58</td>
    <td>92.64</td>
    <td>92.48</td>
    <td>92.47</td>
    <td>0.14</td>
  </tr>
  <tr>
    <th colspan="8">split by roots (+lex)</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>96.34</td>
    <td>96.37</td>
    <td>95.63</td>
    <td>96.30</td>
    <td>95.83</td>
    <td>96.09</td>
    <td>0.34</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>95.39</td>
    <td>95.17</td>
    <td>95.09</td>
    <td>94.63</td>
    <td>95.13</td>
    <td>95.08</td>
    <td>0.28</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>95.86</td>
    <td>95.77</td>
    <td>95.36</td>
    <td>95.46</td>
    <td>95.48</td>
    <td>95.59</td>
    <td>0.22</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>84.91</td>
    <td>84.31</td>
    <td>84.78</td>
    <td>84.50</td>
    <td>83.35</td>
    <td>84.37</td>
    <td>0.62</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>84.85</td>
    <td>84.28</td>
    <td>84.75</td>
    <td>84.32</td>
    <td>83.32</td>
    <td>84.30</td>
    <td>0.61</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>84.88</td>
    <td>84.29</td>
    <td>84.76</td>
    <td>84.41</td>
    <td>83.33</td>
    <td>84.34</td>
    <td>0.61</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>92.13</td>
    <td>91.99</td>
    <td>91.32</td>
    <td>91.49</td>
    <td>91.59</td>
    <td>91.70</td>
    <td>0.34</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>76.73</td>
    <td>76.01</td>
    <td>73.15</td>
    <td>74.19</td>
    <td>74.65</td>
    <td>74.95</td>
    <td>1.43</td>
  </tr>
</table>

Table 14: Segmentation quality of the Morphodict-T dataset using a fine-tuned RuRoberta-large model
<table>
  <tr>
    <th>Metric</th>
    <th>Fold 1</th>
    <th>Fold 2</th>
    <th>Fold 3</th>
    <th>Fold 4</th>
    <th>Fold 5</th>
    <th>Average</th>
    <th>StdDev</th>
  </tr>
  <tr>
    <th colspan="8">random split</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>99.07</td>
    <td>99.06</td>
    <td>99.10</td>
    <td>99.03</td>
    <td>98.97</td>
    <td>99.05</td>
    <td>0.05</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>99.12</td>
    <td>99.16</td>
    <td>99.18</td>
    <td>99.14</td>
    <td>99.11</td>
    <td>99.14</td>
    <td>0.03</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>99.10</td>
    <td>99.11</td>
    <td>99.14</td>
    <td>99.08</td>
    <td>99.04</td>
    <td>99.09</td>
    <td>0.04</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>97.42</td>
    <td>97.55</td>
    <td>97.65</td>
    <td>97.47</td>
    <td>97.12</td>
    <td>97.44</td>
    <td>0.20</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>97.45</td>
    <td>97.45</td>
    <td>97.56</td>
    <td>97.44</td>
    <td>97.23</td>
    <td>97.43</td>
    <td>0.12</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>97.43</td>
    <td>97.50</td>
    <td>97.61</td>
    <td>97.45</td>
    <td>97.17</td>
    <td>97.43</td>
    <td>0.16</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>98.18</td>
    <td>98.19</td>
    <td>98.25</td>
    <td>98.22</td>
    <td>98.09</td>
    <td>98.19</td>
    <td>0.06</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>93.52</td>
    <td>93.72</td>
    <td>93.76</td>
    <td>93.65</td>
    <td>93.38</td>
    <td>93.61</td>
    <td>0.16</td>
  </tr>
  <tr>
    <th colspan="8">split by roots</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>96.27</td>
    <td>95.44</td>
    <td>95.50</td>
    <td>95.47</td>
    <td>96.23</td>
    <td>95.78</td>
    <td>0.43</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>95.13</td>
    <td>94.90</td>
    <td>94.24</td>
    <td>94.53</td>
    <td>94.89</td>
    <td>94.74</td>
    <td>0.35</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>95.69</td>
    <td>95.17</td>
    <td>94.86</td>
    <td>95.00</td>
    <td>95.56</td>
    <td>95.26</td>
    <td>0.36</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>85.01</td>
    <td>81.29</td>
    <td>79.82</td>
    <td>80.92</td>
    <td>84.81</td>
    <td>82.37</td>
    <td>2.38</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>85.15</td>
    <td>81.26</td>
    <td>79.54</td>
    <td>80.82</td>
    <td>84.62</td>
    <td>82.28</td>
    <td>2.47</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>85.08</td>
    <td>81.28</td>
    <td>79.68</td>
    <td>80.87</td>
    <td>84.71</td>
    <td>82.32</td>
    <td>2.42</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>92.15</td>
    <td>91.25</td>
    <td>90.70</td>
    <td>90.96</td>
    <td>91.90</td>
    <td>91.39</td>
    <td>0.62</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>75.66</td>
    <td>71.74</td>
    <td>70.65</td>
    <td>73.03</td>
    <td>75.53</td>
    <td>73.32</td>
    <td>2.24</td>
  </tr>
  <tr>
    <th colspan="8">random split (+lex)</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>99.02</td>
    <td>99.14</td>
    <td>99.00</td>
    <td>99.00</td>
    <td>99.03</td>
    <td>99.04</td>
    <td>0.06</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>99.20</td>
    <td>99.21</td>
    <td>99.14</td>
    <td>99.18</td>
    <td>99.14</td>
    <td>99.17</td>
    <td>0.03</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>99.11</td>
    <td>99.17</td>
    <td>99.07</td>
    <td>99.09</td>
    <td>99.09</td>
    <td>99.10</td>
    <td>0.04</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>97.23</td>
    <td>97.53</td>
    <td>97.40</td>
    <td>97.25</td>
    <td>97.41</td>
    <td>97.37</td>
    <td>0.12</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>97.26</td>
    <td>97.46</td>
    <td>97.33</td>
    <td>97.29</td>
    <td>97.39</td>
    <td>97.35</td>
    <td>0.08</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>97.25</td>
    <td>97.49</td>
    <td>97.37</td>
    <td>97.27</td>
    <td>97.40</td>
    <td>97.36</td>
    <td>0.10</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>98.20</td>
    <td>98.29</td>
    <td>98.15</td>
    <td>98.17</td>
    <td>98.15</td>
    <td>98.19</td>
    <td>0.06</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>93.52</td>
    <td>93.80</td>
    <td>93.54</td>
    <td>93.34</td>
    <td>93.51</td>
    <td>93.54</td>
    <td>0.16</td>
  </tr>
  <tr>
    <th colspan="8">split by roots (+lex)</th>
  </tr>
  <tr>
    <td>Precision<sub>all</sub></td>
    <td>96.64</td>
    <td>95.72</td>
    <td>96.29</td>
    <td>96.33</td>
    <td>96.90</td>
    <td>96.38</td>
    <td>0.44</td>
  </tr>
  <tr>
    <td>Recall<sub>all</sub></td>
    <td>95.82</td>
    <td>95.99</td>
    <td>95.37</td>
    <td>95.83</td>
    <td>96.03</td>
    <td>95.81</td>
    <td>0.26</td>
  </tr>
  <tr>
    <td>F1<sub>all</sub></td>
    <td>96.23</td>
    <td>95.85</td>
    <td>95.83</td>
    <td>96.08</td>
    <td>96.46</td>
    <td>96.09</td>
    <td>0.27</td>
  </tr>
  <tr>
    <td>Precision<sub>root</sub></td>
    <td>85.74</td>
    <td>83.58</td>
    <td>82.84</td>
    <td>84.60</td>
    <td>87.96</td>
    <td>84.95</td>
    <td>2.01</td>
  </tr>
  <tr>
    <td>Recall<sub>root</sub></td>
    <td>85.64</td>
    <td>83.28</td>
    <td>82.84</td>
    <td>84.53</td>
    <td>87.64</td>
    <td>84.78</td>
    <td>1.93</td>
  </tr>
  <tr>
    <td>F1<sub>root</sub></td>
    <td>85.69</td>
    <td>83.43</td>
    <td>82.84</td>
    <td>84.57</td>
    <td>87.80</td>
    <td>84.87</td>
    <td>1.97</td>
  </tr>
  <tr>
    <td>Accuracy</td>
    <td>92.91</td>
    <td>92.50</td>
    <td>92.46</td>
    <td>92.77</td>
    <td>93.49</td>
    <td>92.82</td>
    <td>0.42</td>
  </tr>
  <tr>
    <td>WordAccuracy</td>
    <td>77.95</td>
    <td>75.05</td>
    <td>75.32</td>
    <td>77.85</td>
    <td>79.70</td>
    <td>77.17</td>
    <td>1.96</td>
  </tr>
</table>

Table 15: Segmentation quality of the Morphodict-K dataset using a fine-tuned RuRoberta-large model