MorSeD : Morphological Segmentation of Danish and its Effect on Language Modeling (2025)

Abstract: Current language models (LMs) mostly exploit subwords as input units based on statistical co-occurrences of characters. Adjacently, previous work has shown that modeling morphemes can aid performance for Natural Language Processing (NLP) models. However, morphemes are challenging to obtain as there is no annotated data in most languages. In this work, we release a wide-coverage Danish morphological segmentation evaluation set. We evaluate a range of unsupervised token segmenters and evaluate the downstream effect of using morphemes as input units for transformer-based LMs. Our results show that popular subword algorithms perform poorly on this task, scoring at most an F1 of 57.6 compared to 68.0 for an unsupervised morphological segmenter (Morfessor). Furthermore, evaluate a range of segmenters on the task of language modeling.

## introduction although there is no exact consensus on the definition of morphemes (e.g. @xcite , they are commonly described as the smallest meaning-carrying units in natural language @xcite . morphemes are useful for linguistic analysis, language understanding, language learning and potentially as input units for nlp models. traditionally, characters or words were used as inputs for nlp models, but contextualized language models (lms) popularized subwords @xcite , which are often based on a trained vocabulary obtained with statistical methods. morphemes, however, are a promising 1 data and code are available on https: //bitbucket.org/robvanderg/morsed input: frakkeskåner laerte morsed: frakke-skån-er laer-te tinybert: fra-kk-es-kan-er l-ae-rte bpe: frakke-skå-ner laerte wordpiece: fra-kke-sk-åne-r laerte unigram: figure 1 : two examples from our dataset, with the input words, gold morpheme annotation (morsed), and the outputs of: a baseline english language model segmenter (tinybert), three danish statistical segmenters, and a danish unsupervised morphological segmenter (morfessor). although there have been large multilingual benchmarking efforts for morphological tagging @xcite and reinflection @xcite , data for morphological segmentation is more scarce, especially for mid-resource languages, like danish @xcite . therefore, we create a small yet high-coverage benchmark to evaluate unsupervised segmenters for danish morphological segmentation and provide an extensive evaluation of existing models. there has been some work that incorporating morphemes as input to lms. for english, @xcite showed that derivational segmentation aids lm interpretation of complex words, and @xcite showed that using units that closer resemble morphemes improves language modeling (although the mor- ## morsed here, we introduce morsed, to the best of knowledge the first publicly available dataset annotated for morphological segmentation of danish. we follow the guidelines and categories defined in @xcite . our main annotator (author of @xcite ) has 35 years of experience as a danish teacher, with a degree in teaching and a postgraduate diploma in adult literacy education. the dataset contains 800 words. 2 the words were selected by our main annotator, focusing on diversity and good coverage for each category. in a second native danish annotator without a linguistic background annotated 300 words from morsed by following the same guidelines. since inter-annotator scores (e.g., cohen's kappa) are challenging to compute for segmentation tasks, we use f 1 score on the morpheme level for comparison. the resulting f 1 is 0.991, indicating welldefined guidelines and a clear task definition. raw text data. for training the segmenters and the lms, we use raw text data. we collect data from 8 different resources ( dataset domain source bookshop books tiedemann (2012) cc-100 webscrape wenzek et al. (2020) culturax webscrape nguyen et al. (2023) gigaword mixed strømberg-derczynski et al. (2021) opensubtitles foot_3 subtitles lison and tiedemann (2016) reddit social chang et al. (2020) twitter social archive.org/details/twitterstream wiki wiki attardi (2015) language model evaluation due to computational constraints, we choose to train a model with the same architecture as tinybert @xcite . we did a hyperparameter search with its default tokenizer on the english data from the babylm challenge @xcite to find reasonable settings (details are available in the repository). 4 we use the adam optimizer, with a learning rate of 1 × 10 -3 , a batch size of 512, and 1 epoch over the mixed 320m dataset (section 2), of which we keep 1% separate for evaluation. we use a 15% masking strategy during training and evaluation, because perplexity is affected by the segmentation. we use bits per character (bpc) to evaluate the language models. bits per character represents the average number of bits needed to encode each character in the dataset. furthermore, we use accuracy on the token level. even though the accuracy is affected by the segmentation, it is highly interpretable, and since none of our models is tuned to optimize on this metric we expect it to correlate to language model quality. ## analysis quantitative. our results show that recall is higher than precision for all methods ( qualitative. to get a more fine-grained picture of the difficulties for the segmentation models, we spot-check cases where at least three of the segmenters were incorrect. our analysis reveals that tokenizers frequently missegment in the categories compounds and compounds with linking elements. the segmentation of morphemes such as "-e" and "-s" is especially challenging, underscoring tokenizers' difficulties with complex morphological structures such as "sygeplejeskole" (syg-e-plej-eskole; en: "nursing school"), "gulerod" (gul-erod; en: "carrot") and "landsholdstrup" (land-shold-s-trup; en: "national team"). furthermore, as morpheme length increases, the error rate increases, highlighting the tokenizers' limitations in handling more complex word formations. melfo data after our experiments, we managed to get access to morphological segmentation data from the melfo (mobil e-laering for ordblinde) project foot_5 . this data is not publicly available, but we used it to evaluate the robustness of each segmenter on another dataset with different guidelines and annotators. upon manual inspection, we found that the main difference between the datasets is the choice of words (there are 8 overlapping words) and that the segmentation of morsed leads to more splits and smaller elements (e.g. fri-tid-s-hjem versus fritid-s-hjem). the results show a similar trend (i.e. ranking of models), but lower performances overall, which is partially due to tuning (of vocabulary size) on morsed, but also due to the structure of the data: melfo has a longer average word length (12 characters versus 8) and a larger average amount of morphemes per word (2.6 versus 1.9). ## conclusion we introduced morsed, a broad-coverage, expert-annotated dataset for subword segmentation in danish. we used morsed to show that an unsupervised segmenter outperforms statisticalbased subword segmenters on the task of morphological segmentation for danish by 10.3 points absolute f 1 score on our novel danish benchmark. we also show that the tokenizer that performs best at morphological segmentation also performs well on language modeling (accuracy).