M o V o C : Morphology-Aware Subword Construction for G e’ez Script Languages (2025)

Abstract: Subword-based tokenization methods often fail to preserve morphological boundaries, a limitation especially pronounced in low-resource, morphologically complex languages such as those written in the Ge‘ez script. To address this, we present MoVoC (Morpheme-aware Subword Vocabulary Construction) and train MoVoC-Tok, a tokenizer that integrates supervised morphological analysis into the subword vocabulary. This hybrid segmentation approach combines morpheme-based and Byte Pair Encoding (BPE) tokens to preserve morphological integrity while maintaining lexical meaning. To tackle resource scarcity, we curate and release manually annotated morpheme data for four Ge‘ez script languages and a morpheme-aware vocabulary for two of them. While the proposed tokenization method does not lead to significant gains in automatic translation quality, we observe consistent improvements in intrinsic metrics, MorphoScore, and Boundary Precision, highlighting the value of morphology-aware segmentation in enhancing linguistic fidelity and token efficiency. Our morpheme-annotated datasets and tokenizer dataset will be publicly available under the Open Data licenses to support further research in low-resource, morphologically rich languages.

## introduction tokenization is a fundamental preprocessing step in nlp, converting raw text into structured units such as bytes @xcite , characters @xcite , subwords @xcite , words @xcite , or multi-word expressions @xcite . subword tokenization, such as bpe @xcite , gained popularity for being language-independent and compressing vocabulary, enabling efficient and balanced token learning. however, subwords often fail to capture morphological structure, a problem that is especially clear in multilingual models using a shared vocabulary across languages. without careful and balanced data selection, low-resource languages typically receive fewer subwords, leading to a high token-to-word ratio @xcite @xcite . moreover, negative morphemes challenge llms with tokenizers lacking morphological sensitivity @xcite . morphological systems across languages vary regarding the relation between form and meaning @xcite and are harder to model and predict @xcite . furthermore, languages with greater morphological complexity may lead to worse language model performance, as morphologically rich languages tend to have a large number of very infrequent word forms produced by combinations of morphemes, which leads to data sparsity @xcite . morphologically rich languages also have less annotated data @xcite and are often written with non-latin scripts, which require more bytes to be represented in common encoding standards like utf-8 @xcite . additionally while subword tokenization consistently outperforms character and word level approaches by efficiently compressing text into shorter token sequences @xcite , compression alone is not always predictive of downstream task success, especially for morphologically complex or semantically dense languages @xcite . overall the standard algorithms lack morphology awareness @xcite . alternatively morphologicallyaware tokenization produces more meaningful tokens, often resulting in improved model performance @xcite @xcite . this claim is supported by evidence from several languages, e.g., korean @xcite , arabic @xcite , japanese @xcite , and hebrew @xcite . however, such efforts are constrained by the limited availability of morphologically annotated datasets, mostly found in high-resource languages @xcite . moreover, even with sufficient morphological annotations, relying solely on morpheme-based tokenization is suboptimal. as noted by @xcite , approaches that disregard subword tokenization struggle to leverage shared statistical information across related word forms, result in longer input sequences, and generally perform worse than hybrid methods that integrate morphological information with subword tokenization. accordingly, based on the three-stage tokenization framework proposed by @xcite and @xcite , we adopt a tokenization process comprising three distinct phases: pre-tokenization, vocabulary construction, and segmentation. concretely, our contributions are as follows: (i) we develop morphologically annotated datasets for four low-resource ge'ez script languages to support improved tokenization, morphological analysis, and downstream nlp tasks; (ii) we propose movoc (morpheme-aware vocabulary construction), a supervised approach that leverages linguistically informed bpe segmentations as an alternative to standard subword techniques. it integrates bpe-derived vocabulary with morphemes extracted through supervised methods to enhance morphological representation and improve token-to-morpheme alignment in morphologically rich low-resource languages; (iii) we perform thorough intrinsic and extrinsic evaluations to measure the effectiveness of our approach and validate its practical applicability. ## proposed method in this section, we describe three separate methodologies for 1) pre-tokenization and supervised morphological analyses, 2) vocabulary construction (movoc), and 3) morpheme-aware subword segmentation (movoc-tok). ## evaluation framework we incorporate both intrinsic and extrinsic evaluations to assess our approach. intrinsic evaluation focuses on morpheme boundary precision and vocabulary consistency (e.g., rényi entropy), while extrinsic evaluation measures downstream performance in machine translation using metrics like bleu and chrf++. ## result tokenization quality. movoc-tok achieves morphscores for all four languages (see tab. 2) that are substantially higher than the mean morphscore reported for fusional languages in the original paper of arnett and bergen (2025) (0.533). while movoc-tok does not score higher than all sentencepiece tokenizer variants, this indicates that our hybrid approach instills at least partial morpheme awareness into the tokenization process. our intrinsic evaluation results (see tab. 4) further underscore this general result: generating tokens via movoc-tok yields both better rényi entropy and morpheme boundary precision scores across all four languages. while the effect for amharic and tigrinya text is less pronounced, we can observe a clear performance boost when processing the less-represented low-resource languages, tigre and ge'ez. ## qualitative analysis while quantitative evaluation (e.g., bleu, chrf) shows only modest gains from movoc compared to standard bpe, these metrics alone do not fully capture the benefits of morphology-aware subword construction. to better demonstrate the practical impact of movoc, we present a qualitative analysis focusing on representative examples from tokenization and machine translation outputs. preservation of morphological integrity: standard bpe often fragments morphologically rich words into arbitrary subword units, leading to a loss of meaningful morphemes. for instance, the tigrinya word ኣይትከውንን ("do not do it") is split by bpe into ኣይ-ት-ከ-ውን-ን, obscuring its internal structure. movoc instead produces ኣይ-ትከውን-ን , preserving the negation prefix (ኣይ-), verb root (ትከውን), and suffix (-ን). this linguistically aligned segmentation provides the model with units that carry functional meaning. ## conclusion and future work in this work, we extend the processing of ge'ez script languages by (i) releasing morphologically annotated datasets for four languages, tigrinya, amharic, ge'ez, and tigre, and (ii) proposing a morpheme-aware tokenization approach as an alternative to conventional bpe. our method constrains subword segmentation to align with morpheme boundaries, resulting in vocabularies that better reflect the underlying linguistic structure and improve tokenization quality for morphologically rich languages. the annotated data will further serve for research and evolution in low-resource language processing, supporting improved linguistic analysis and more effective natural language models.