C o D e R oo M or: A new dataset for non-inflectional morphology studies of S wedish (2021)

Abstract: The paper introduces a new resource, CoDeRooMor, for studying the morphology of modern Swedish word formation. The approximately 16.000 lexical items in the resource have been manually segmented into word-formation morphemes, and labeled for their categories, such as prefixes, suffixes, roots, etc. Word-formation mechanisms, such as derivation and compounding have been associated with each item on the list. The article describes the selection of items for manual annotation and the principles of annotation, reports on the reliability of the manual annotation, and presents tools, resources and some first statistics. Given the”gold” nature of the resource, it is possible to use it for empirical studies as well as to develop linguistically-aware algorithms for morpheme segmentation and labeling (cf statistical subword approach). The resource will be made freely available.

## introduction linguistic complexity is a fascinating phenomenon that influences language perception, language learning and language production (cf. @xcite @xcite . it has been studied at different levels and with different intentions, for example from a typological perspective (e.g. @xcite or from a computational perspective (e.g. @xcite . linguistic complexity also varies between individual users of the same language, which makes it possible to use linguistic indicators to differentiate betweeen language typical of advanced language users as opposed to, for instance, children or beginner learners @xcite @xcite . from a second language (l2) perspective there is a need to be able to follow how the morphological complexity develops in the learner language (e.g. @xcite for instance by following how the learner acquires more inflectional forms in the language but also by seeing how their vocabulary growth can be related to the acquisition of rules of word formation. the latter is a rather underdeveloped research area and it is that which has been our focus in developing the coderoomor resource -we want to be able to follow how word families (cf. @xcite grow and how awareness of word-formation mechanisms develops in language learners. morphology, as one of the dimensions of linguistic complexity, covers word formation in terms of compounding and derivational morphology as well as inflectional morphology, such as grammatical affixes that words take to reflect number, definiteness, gender, etc. most publications on morphological complexity deal with studies of the inflectional dimension of morphology (e.g. @xcite , with a few rare exceptions (e.g. bolshakova and sapin, 2020), which is not surprising. while automatic text annotation pipelines are able to process inflectional morphology (cf. morphosyntactic descriptors available for corpora in the korp search interface @xcite ), there is a lack of corpora containing analysis of the morphemes consituting the word lemmas. this is due to the absense of gold standard resources that can be used for training automatic tools (e.g. @xcite . this is hypothetically also the reason why we rarely find lexical resources organized by word family principles (cf. @xcite , even though there is a clear interest in that kind of resources in connection to vocabulary testing (e.g. @xcite and psycholinguistic and cognitive research (e.g. @xcite . 3 in the currently pursued project, development of lexical and grammatical competences in immigrant swedish, foot_3 funded by riksbankens jubileumsfond, we are looking for ways to characterize the language typical of second language (l2) learners of swedish from different perspectives based on the analysis of two learner-specific corpora (see section 3). based on those corpora, we have generated a sense-based wordlist, sen*lex, manually segmented each item on the list into morphemes and labeled those for their morpheme categories (section 4). the intention is to use this resource for empirical studies as well as for the development of automatic morphological segmentation and consequent morpheme classification for swedish. we expect this type of annotation to facilitate deeper studies into lexical and morphological complexity, language acquisition patterns, associative learning mechanisms and the like. the resource can also be of interest in pedagogical studies and applications. ## related work morphemic segmentation is an important nlp task which is applied to machine translation, cognate identification, linguistic typological studies, and the like @xcite . the task of morpheme segmentation consists of the identification of morpheme boundaries within a word, and classifying them by their category. most work has been focused on inflectional morphology and on classification of the endings by their syntactic and grammatical functions, such as gender, number, tense indicators (e.g. @xcite . identification of word formation morphemes (roots, suffixes, prefixes) and their subsequent classification is a more complicated task, and until recently most approaches have been targeting only morpheme boundary identification using unsupervised or semi-supervised approaches, for example a language independent approach taken in morfessor @xcite or sub-word identification techniques (e.g. @xcite . only recently have datasets with labeled data started to appear, and depending on their size, neural networks are used for experimentation with more complicated tasks including both morpheme segmentation and labeling of word formation morphemes (e.g. @xcite . morphology has not been one of the major strands of research on swedish, neither as an l1 (native speaker language) nor as an l2 (second language learners). there also has not been a lot of interest in the development of tools and resources in relation to swedish morphology except for saldo morphology @xcite which is used in annotation of swedish texts and which primarily includes inflectional paradigms. due to its language independence, morfessor @xcite offers a possibility to annotate words morphologically in any language and works relatively well on concatenative languages, including swedish. the output consists of several suggestions for word segmentation into morpheme constituents. in recent years interest has increased in finding ways to study different forms of complexity in connection to second language acquisition and learner corpora @xcite . however as housen et al. say, morphological complexity has not been at the centre of attention. when studies have looked at morphological complexity they have also tended to focus primarily on inflectional morphology. the resource we present in this paper is aimed at non-inflectional morphology of swedish and can be used in a variety of nlp and linguistic tasks, including within the second language acquisition domain, and is filling a gap by offering a richly annotated dataset for morphological studies. ## item selection to limit the annotation work to only the most relevant items, which in our context means items of relevance for second language learners of swedish, we have used two source corpora: both corpora have indications of levels of proficiency according to the common european framework of reference, cefr (council of europe, 2001), and contain texts and essays at five out of six defined levels: a1 (beginner), a2, b1, b2, c1 (advanced). from the two corpora, two lists of lemgrams (i.e. baseforms of the words + their corresponding parts of speech, pos) have been generated, namely: the approach used in the generation of the above lists has been reused by us to generate a new list based on senses (i.e. a list where each entry corresponds to a unique combination of baseform+pos+sense) once the pipeline for swedish could assign word senses @xcite based on saldo senses @xcite . this work resulted in sen*lex (a sense-based variant of swellex and svalex in one), a publication on which is currently under preparation. the non-problematic items of this latter list have been used for the morphological annotation. 5 figure 1 shows the basic information about the two source corpora and the three vocabulary lists. sen*lex includes both single-word items and multi-word expressions (mwes), and contains word senses coming from both learner essays and course books. a certain amount of items overlap, i.e. occur in both corpora; whereas some items are homographs within the same part of speech (cf. vara, verb -eng. 'be' and eng. 'last'), but have several distinct senses. these latter items may have identical morphological analysis despite having several entries in the list, but it is also possible that they have different morphological annotation as is the case with the verb vara, where the root is var-in both lemmas but the final -a is seen as derivational in one sense and inflectional in the other, since vara (eng. 'be'), has the imperative form var! and the verb vara, (eng. 'last') has the stem and imperative form vara!, not that you are ever likely to use it in the imperative. the coderoomor morphological dataset that we are presenting is, thus, not all-covering for modern swedish. however, given the nature of second language learning, the most central items should be represented in the list, therefore making it relatively comprehensive. 6 tomatically been assigned multiple lemgrams or failed to be assigned a lemgram. these items are left for future work. 6 we would also like to note that the set includes c. 500 triplets consisting of lemgrams which are verbs and part-ofspeech-tag participle, e.g. cykla 'to ride a bike' + pc (participle). since we annotate lemgrams these have then been annotated as the lemma of the verb, rather than one of the participles. we did look into annotating them as participles, but in fact each of these items can include occurrences in the data which are a combination of present participles or past participles, or even supine forms (a form etymologically re- ## annotation principles the items were analysed at the lemgram level and hence the work did not include annotation of inflectional forms/morphemes, with a few exceptions (see below). for example, oändlighet, noun ('infinity') was 1. segmented into four morphemes o-änd-lig-het 2. each morpheme received a label: o: prefix änd: root lig: derivational suffix het: derivational suffix 3. the word formation of the item was labeled as derivation the taxonomy of morphemes is presented in in most cases later comparisons helped resolve these issues and enabled the classificiation of the morpheme into one of the main morpheme categories. the taxonomy of word formation mechanisms follows from ## annotation workflow and visualization a team of three highly qualified annotators performed the annotation under the supervision of a project researcher. during the first month the three annotators went through a training period where they worked in parallel with the project researcher and annotated 100 new items per week plus reannotated items from previous weeks when need be. based on the parallel items, comparisons were run on both the morphological analysis and the word formation assignment of each item. the guidelines were refined to take care of any remaining unclarities or disagreements. the 400 items that were annotated by the 4 members of the morphology group during the training period have been used for calculating inter-annotator agreement (iaa) which we report using krippendorff's alpha in after annotating 400 items in parallel, the rest of the items were divided between the 3 annotators with weekly meetings to monitor progress and discuss problematic cases. before each meeting the project researcher got a morpheme-based comparison where disagreements and partial disagreements were identified, and these could then be checked by the project researcher and discussed as needed at the meeting. the team came up with a solution and amended the guidelines to ensure systematic annotation in the future. after each meeting the annotators were expected to correct any items that had been picked up in the comparison to adhere to the agreed or revised principles. the guidelines have been a living document all through the process. an article with a more detailed description of the linguistic principles of segmentation and labeling is under preparation @xcite . once the annotation was completed the project researcher once again checked disagreements, partial disagreements and also searched through the data consistently for certain strings to find possible inconsistencies. based on this some further corrections were done according to the guidelines, e.g. if a suffix had been annotated as an inflectional suffix but should be a derivational suffix according to the guidelines this was corrected. to ensure consistency of the annotation work, a tool for lexicographic annotation legato @xcite was implemented within the framework of the project, see appendix a. the tool requires annotators to log in to save their annotations. the functionality of the tool allows the annotator to see in addition, the tool offers possibilities to open guidelines, check a list of previously "skipped items" or click on supportive links (among others, coctaill corpus hits for the current item and saol/so hits). to navigate between the items, it is possible to "jump" to another item at a certain numeric index, search for some specific items or filter items. furthermore, the tool also allows each annotator to download their own annotated words with time stamps for inspection of the results. the project researchers can, in addition, download the annotations from all annotators, to generate several types of comparisons and statistics, and download a full set of annotated words. ## coderoomor dataset description the coderoomor dataset (version 1.0) contains 16 230 analyzed lemgrams foot_5 representing 4 429 unique roots, 259 unique derivational suffixes, 155 unique prefixes and 12 unique binding morphemes (infixes), see the five most frequent root morphemes in the sen*lex items on the coderoomor are: examples of other frequent root morphemes are • liv with 73 family members, e.g. affärsliv, livmoder, leva livet, ('business life', 'uterus', 'live the life'); and on inspection we can see that these family groupings need to be refined to be separated further into proper "families", so that words containing unrelated homographic roots do not accidentally end up in the same family. to give one example, the sam-family at the moment contains both samisk ('sami', adj.) and samhälle ('society'), which should be separated into two different families since a morpheme is the smallest meaningful unit in language and therefore each root should have only one meaning and homographs should be separated. taking a look at the most frequent derivational morphemes (prefixes and suffixes), we can see that in the annotated wordlist • the most common prefixes in the wordlist are för-(380 words), be-(299 words), o-(256 words), re-(112 words), pro-(85 words) as in förälder, besök, odjur, reagera, problem ('parent', 'visit', 'beast', 'react', 'conference', 'problem') there are several prefixes that only occur once in the dataset (wordlist and corpora), e.g. abs-, fysio-, ko-as in abstrakt, fysiologisk, koefficient ('abstract', 'physiological', 'coefficient'). in cases such as koefficient it would be good to consider comparison to allomorphs such as kon, but this currently needs to be done manually. some of the least common suffixes are -ej, -enn as in pastej, persienn ('paté', 'venetian blind') which only occur once in the dataset (wordlist and corpora). in the dataset it is also possible to access the frequency in relation to the number of occurrences in the l2 corpora we work with. from the initial exploration of the word formation mechanisms in the two source corpora, we can see that root lexemes clearly dominate (figures 3 and 4), followed by derivation and compounding. abbreviation is hardly represented, nor are lexicalized forms that we haven't even included into the graphs. the hypothetical reason for the overrepresentation of root lexemes can be the fact that most frequent words in the language, namely prepositions, particles and conjunctions, are root lexemes and therefore add to the running statis using coderoomor, it is possible to trace the morphemic complexity of the words at different stages of language development. from figure 5 we can see that the morphemic word structure is getting more complex as proficiency develops, with the average number of morphemes per new word (based on words which first occur at that cefr-level in our data) growing from 1.79 at the beginner level till 2.89 at the advanced level. the dataset can be downloaded as an excel file or as a file with comma separated values (csv file format). the information can be organized in several ways: the associated information per lemgram consists of: • lemgram • sense indicator (saldo-based) • part of speech • analysis by morpheme • real root (if applicable) • word segmentation boundaries • word morpheme patterns • word formation mechamism • the cefr level (level of first occurence) • frequency information from coc-taill, by level and in total (if applicable) • frequency information from swellpilot, by level and in total (if applicable) the legato annotation tool can compile some statistics and tables for overviews and visualization, which currently is only available for project researchers. in the future, we plan to make these functions open to all users, together with making this dataset available not only for download, but also for browsing (cf. english vocabulary profile, @xcite . the coderoomor dataset can be freely downloaded from språkbanken-text. 11 ## future work the coderoomor resource offers promising possibilities for several types of research. research questions with linguistics and second language acquisition domain are described in detail in lindström tiedemann et al. @xcite and are mentioned briefly in the introduction to this article. with regards to pedagogical and applied research prospects, we are currently exploring how the items can best be linked together and presented to the public as a word family resource for use both in research and in teaching. the plan is that since swedish uses both derivation and compounding frequently the resource will show all words which have a common root as a family and there will be information about how this relates to cefr levels based on the corpora that we mentioned above. the dataset can be effetively used for intelligent computer-assisted language learning research, 11 https://spraakbanken.gu.se/en/resources#refdata for example for exercise generation or text complexity analysis. to visualize the resource and support research into the non-inflectional morphology, we are working on a user interface for swedish similar to the english vocabulary profile (evp) 12 and pearson gse teacher toolkit. 13 the interface has a working title swedish l2 profile (swel2p) and is integrated into the lärka platform foot_9 @xcite , at språkbanken text (gothenburg, sweden). the gui will provide possiblilites to search, filter, browse and download various l2 swedish datasets (lexical, morphological, grammar, including coderoomor) generated as an output of the project. we are currently also experimenting with automatic morpheme segmentation based on the coderoomor dataset which is showing promising results and we hope that this might result in a new functionality in the sparv pipeline @xcite allowing automatic segmentation and labeling of morpheme categories for swedish. the ultimate aim is to analyze learner language in a more nuanced way, where analysis of word formation morphemes could help us to look deeper into lexical and morphemic complexity and to understand language acquisition and processing better. type token ratio (ttr) has been often used as a way to measure lexical diversity, i.e. how varied the vocabulary in a text is (see e.g. @xcite . however recently ttr has also been used as a means of studying morphological complexity @xcite . gutierrez et al. also explore the possibility of studying morphological complexity through entropy and crf in relation to typological comparisons of languages. our intention is to apply similar techniques for analysis of learner language.