M orphy N et: a Large Multilingual Database of Derivational and Inflectional Morphology (2021)

Abstract: Large-scale morphological databases provide essential input to a wide range of NLP applications. Inflectional data is of particular importance for morphologically rich (agglutinative and highly inflecting) languages, and derivations can be used, e.g. to infer the semantics of out-of-vocabulary words. Extending the scope of state-of-the-art multilingual morphological databases, we announce the release of MorphyNet, a high-quality resource with 15 languages, 519k derivational and 10.1M inflectional entries, and a rich set of morphological features. MorphyNet was extracted from Wiktionary using both hand-crafted and automated methods, and was manually evaluated to be of a precision higher than 98%. Both the resource generation logic and the resulting database are made freely available and are reusable as stand-alone tools or in combination with existing resources.

## introduction despite repeated paradigm shifts in computational linguistics and natural language processing, morphological analysis and its related tasks, such as lemmatization, stemming, or compound splitting, have always remained essential components within language processing systems. recently, in the context of language models based on subword embeddings, a morphologically meaningful splitting of words has been shown to improve the efficiency of downstream tasks @xcite @xcite . in particular, the reintroduction of linguistically motivated approaches and high-quality linguistic resources into deep learning architectures has been crucial for dealing with morphologically rich-highly inflecting, agglutinative-languages more efficiently @xcite @xcite . in response to such needs, and as simple and convenient substitutes for monolingual morphological analyzers, multilingual morphological databases have been developed, indicating for each word form entry one or more corresponding root or dictionary entries, as well as analysis (features) @xcite @xcite . the precision and recall of these resources vary wildly, and there is still a lot of ground to cover with respect to the support of new languages, the modelling of the inflectional and derivational complexity of each language, as well as the richness of the information (features, affixes, parts of speech, etc.) provided. as a further step towards extending online morphological data, we introduce morphynet, a new database that addresses both derivational and inflectional morphology. its current version covers 15 languages and has 519k derivational and 10.1m inflectional entries, as well as a rich set of features (lemma, parts of speech, morphological tags, affixes, etc.). similarly to certain existing databases, morphynet was built from wiktionary data; however, our extraction logic allows for a more exhaustive coverage of both derivational and inflectional cases. the contributions of this paper are the freely available morphynet resource, the description of the data extraction logic and tool, also made freely accessible, as well as its evaluation and comparison to state-of-the-art multilingual morphological databases. due to the limited overlap between the contents of these resources and morphynet, we consider it as complementary and therefore usable in combination with them. section 2 of the paper presents the state of the art. section 3 gives details on our method for generating morphynet data. section 4 presents the resulting resource, and section 5 evaluates it. section 6 concludes the paper. ## state of the art ever since the early days of computational linguistics, morphological analysis and its related taskssuch as stemming and lemmatization-have been part of nlp systems. earlier grammar-based systems used finite-state transducers or affix stripping techniques, and certain of them were already multilingual and were capable of tackling morphologically complex languages @xcite @xcite . however, due to the costliness of producing the grammar rules that drove them, many of these systems were only commercially available. more recently, several projects have followed the approach of formalizing and/or integrating existing morphological data for multiple languages. uder (universal derivations) @xcite integrates 27 derivational morphology resources in 20 languages. unimorph @xcite and the wikinflection corpus (metheniti and neumann, 2020) rely mostly on wiktionary from which they extract inflectional information. beyond the data source, however, the two last projects have little in common: unimorph is by far more precise and complete, and being used as gold standard for nlp community @xcite (recently covering 133 languages (mccarthy et al., 2020)), while wikinflection follows a naïve, linguistically uninformed approach of merely concatenating affixes, generating an abundance of ungrammatical word forms (e.g. for hungarian or finnish). morphynet is also based on extracting morphological information from wiktionary, extending the scope of unimorph by new extraction rules and logic. the first version of morphynet covers 15 languages, and it is distinct from other resources in three aspects: (1) it includes both inflectional and derivational data; (2) it extracts a significantly higher number of inflections from wiktionary; and ## morphynet generation morphynet is generated mainly from wiktionary, through the following steps. below we explain the non-trivial wiktionary extraction and enrichment steps, while section 4 provides details on the generated resource itself. ## the morphynet resource morphynet is freely available for download, both as text files containing the data and as the source code of the wiktionary extractor. foot_3 two text files are provided per language: one for inflections and one for derivations. the structure of the two types of files is illustrated in tables 1 and 2 , respectively. as shown, morphynet covers all data fields provided by unimorph for inflections and by uder for derivations. in addition, it extends unimorph by indicating the affix and the immediate source word that produced the inflection. such information is useful, for example, to nlp applications that rely on subword information for understanding out-of-vocabulary words. morphynet also extends the uder structure by indicating the affix and the semantic category for the target word when it can be inferred from the morpheme. such information is again useful for subword regularization of derivationally rich languages, such as english. ## evaluation we evaluated morphynet through two different methods: (1) through comparison to ground truth and (2) through manual validation by experts. uder @xcite is a collection of individual monolingual resources of derivational morphology. most of them have been carefully evaluated against their own datasets and offer high quality. we evaluated morphynet derivational data against uder over the nine languages covered by both resources: french @xcite , @xcite , czech @xcite , german @xcite , russian @xcite , italian @xcite @xcite , latin @xcite , and english @xcite . statistics and results are shown in manual evaluation was carried out by language experts over sample data from five languages: english, italian, french, hungarian, and mongolian. the sample consisted of 1,000 randomly selected entries per language, half of them inflectional and the other half derivational. the experts were asked to validate the correctness of source-target word pairs, of morphemes, as well as inflectional features and parts of speech (the latter for derivations). ## conclusions and future work we consider the resource released and described here as an initial work-in-progress version that we plan to extend and improve. we are currently working on increasing the coverage to 20 languages. we also plan to extend morphynet data with additional features and the semantic categories of words (e.g. animate or inanimate object, action) inferred from derivations. we are planning to conduct a more in-depth study of our evaluation results, especially with respect to uder where it is not yet clear whether the occasional lower precision figures (87% for finnish, 88% for russian) are due to mistakes in morphynet, in the uder resources, or are caused by other factors. a major piece of ongoing work concerns the representation of morphynet derivational data as a lexico-semantic graph, as it is done in wordnets @xcite where derivationally related word senses are interconnected by associative relationships. this effort, justifying the -net in the name of our resource, will allow us to address completeness issues in existing wordnets by extending them by morphological relations and derived words. we are happy to offer the morphynet extraction logic to be reused on a community basis. as extending the tool with new wiktionary extraction rules is straightforward, we hope that the availability of the tool will allow language coverage to grow even further. we also hope that the morphynet data and the extraction logic can serve existing high-quality projects such as unimorph and uder.