Recycling and Comparing Morphological Annotation Models for A rmenian Diachronic-Variational Corpus Processing (2020)

Abstract: Armenian is a language with significant variation and unevenly distributed NLP resources for different varieties. An attempt is made to process an RNN model for morphological annotation on the basis of different Armenian data (provided or not with morphologically annotated corpora), and to compare the annotation results of RNN and rule-based models. Different tests were carried out to evaluate the reuse of an unspecialized model of lemmatization and POS-tagging for under-resourced language varieties. The research focused on three dialects and further extended to Western Armenian with a mean accuracy of 94,00 % in lemmatization and 97,02% in POS-tagging, as well as a possible reusability of models to cover different other Armenian varieties. Interestingly, the comparison of an RNN model trained on Eastern Armenian with the Eastern Armenian National Corpus rule-based model applied to Western Armenian showed an enhancement of 19% in parsing. This model covers 88,79% of a short heterogeneous dataset in Western Armenian, and could be a baseline for a massive corpus annotation in that standard. It is argued that an RNN-based model can be a valid alternative to a rule-based one giving consideration to such factors as time-consumption, reusability for different varieties of a target language and significant qualitative results in morphological annotation.

## introduction so far rule-based (rb) approaches prevailed in the annotation of the armenian varieties which proved to show very good results provided that the system is sufficiently complete and refined (see @xcite for modern eastern armenian [henceforth mea]), or more modest ones if the system is perturbed by certain factors (see @xcite for classical armenian). however, rb systems have the drawback of being considerably time-consuming and not sufficiently reusable for other varieties of the target language. the current research aims at exploring an alternative recurrent neural network (rnn) approach to annotate armenian varieties favored for its flexibility and application rapidity on linguistically and structurally various datasets, as well as for the possibility of making predictions on unknown tokens (predominantly on very different corpora) and contextual disambiguation @xcite . rnn approach has already been applied to some armenian varieties [(vidal-gorène and kindt, 2020) for classical @xcite for mea], highlighting competitive advantages for tagging armenian data. trained on more modest (universal dependencies [ud]) or specialized (gregori project) datasets, the results described are equivalent in lemmatization to eastern armenian national corpus (eanc) rule-based approach and more precise in pos-tagging. the experiments (currently limited to pos tagging and lemmatization) are extended to three armenian dialect varieties and to the two modern armenian standards and the results are compared to eanc rule-based tools. the article is structured as follows: armenian language preliminaries give a highlight to armenian variation in diachronic and synchronic perspectives; the chapter on the armenian ressources online make a state of the art of existing armenian online open-access corpora and databases. the chapter on the datasets focuses on the target datasets designed and used for current research experiment, whereas the chapters on methodology and results, lemmatization results, pos-tagging results spotlight the lemmatisation and pos-tagging results which are furthermore compared in rnn and rb approaches. finally, the last chapter on mwa model explores the feasibility of mwa tagging with a mea model. ## armenian language preliminaries armenian is an indo-european language with a nominative-accusative alignment, predominantly with an agglutinative nominal system and with a more fusional verbal one. it is a left-branching language with flexible word order (svo/sov). the periodization of the armenian language includes: classical armenian (henceforth ca) 1 (5 th -10 th cen. a.d), middle armenian (11 th -17 th cen.) and modern armenian (17 th cen. -up to present). modern armenian includes two standards: modern eastern armenian and modern western armenian, both standardized in the 19 th century. mea is the official language of the republic of armenia and it is also spoken by the armenian communities of iran and ex-soviet republics. mwa is spoken by traditional armenian communities in europe, americas and middle east originating mainly from ottoman empire. aside from the two standards the armenian language continuum includes various dialects as well as vernacular forms (figure 1 ). classical armenian is preserved exclusively for canonical uses. the variation in armenian continuum can vary from light to significant with or without mutual intelligibility. in particular, mea can vary from mwa less than from certain armenian dialects which sometimes lack mutual intelligibility [for more details on armenian varieties and variation see @xcite and @xcite ]. different classifications exist for the armenian dialects depending on the criteria applied (e.g. areal @xcite , morphological (adjarian, 1909), phonological @xcite , typologicalstatistical @xcite , etc.). in our research the morphological criteria prevail due to their importance in annotation processing. one of the main distinguishing morphological features for armenian dialects is the formation of the present indicative according to which three main groups (-um, kə and -l branches) 2 can be outlined as shown in figure 1 [for more details on the armenian dialects see @xcite as well as greppin and khachaturian (1986)]. figure 1 : armenian diachronic and synchronic varieties with d i corresponding to a dialectal variety one of the important issues for dialect corpora is how to transcribe the recordings and how to annotate 1 classical armenian traces back to the creation of the armenian alphabet attributed to the monk mashtots at the beginning of the 5 th century a.d. currently, the armenian alphabet is composed of 39 graphemes and it is the only alphabet used for armenian. at the beginning of the 20 th century an orthography reform was carried out in (eastern/soviet) armenia alongside with her sovietization. currently, two spellings exist for armenian, since the traditional armenian diaspora (both western and eastern armenian speakers) continued to preserve the traditional orthographic standard. technically it requires a conversion system to avoid annotation "noise". several conversion tools have been developed. the transcripts. except for rare specialized dialect corpora, it is usually very difficult (often impossible in case of big corpora) and time-consuming to get a reliable phonetic transcription. an alternative to a phonetic transcription is either a complete standardized (orthographized) transcription (with the condition to have sound alignment) or a semi-standardized (orthographized) one with certain adjustments proper to the target dialect. the most important advantage of the standardized transcription would be the possibility to apply the nlp resources of the standard variety to dialects [for more details on different approaches see @xcite and von @xcite ]. ## armenian resources online heterogeneous texts representing various armenian varieties can be found in a number of online resources which vary in their accessibility, formatting and linguistic background. ## datasets five datasets were set up to conduct experiments (see figure 2 and ## methodology and results a number of tests were carried out to develop an rnn annotation model for three dialects (with manually annotated corpora available) and mwa (no annotated corpus available). three sets of neural networks have been trained and evaluated: the rnn relies on pie @xcite , which offers a highly modular architecture particularly designed to process historical (cf. classical armenian) and non-standard languages (cf. armenian dialects). the rnn model adopted in this research was successfully tested on classical armenian [for more details on the rnn model used see vidal-gorène and kindt (2020)]. generally, pie learning ability exploits fully sentence context to increase the lemmatization accuracy and pos-tagging tasks, particularly in case of ambiguous tokens @xcite . however, in our experiments pie learning ability has been limited because of the unresolved ambiguity of the annotations in d-ab, d-ga and d-shn. consequently, the rnn preserves either all possible categories or only the most probable one. although the linear decoder showed better results for classical armenian pos-tagging, it was compared with the crf decoder provided by marmot and lemming @xcite , which obtained convincing results on equivalent datasets at the last evalatin evaluation campaign @xcite . the model of the lemmatizer and pos-tagger has been trained jointly using a single multitask architecture. finally, the relevance of the architecture was evaluated for a standard language (mea) on the basis of two datasets (d-mea and d-ud). combo @xcite trained with d-ud (v. 2.3) is at 88.05% for lemmatization and 85.07% for pos-tagging @xcite . the present architecture (m-ud) trained with d-ud (v. 2.5) obtains 91.56% in lemmatization (74.35% for the ambiguous tokens and 61,85% for the unknown tokens) and 92.54% in pos-tagging (87,81% ambiguous tokens and 83,56% unknown tokens). ## conclusion different experiments were carried out to illustrate for the first time the automatic morphological annotation of armenian dialect varieties, and the possible reuse of non-specialized models for rapid corpus processing. the first results are more than relevant with very accurate models specialized on the dialects and mea showing more than 92% accuracy in lemmatization and 95% in pos-tagging. the mixed and non-specialized models prove to be insignificant for the annotation rate improvement. these models lack interoperability between the target databases which is detrimental to the models and results in producing many false negatives. the experiments show considerable relevance in model reuse for armenian diachronic and variational data, as illustrated more particularly by mwa target test (88.79% in lemmatization and 87.33% in postagging). the standardization and harmonization of annotation conventions is one of the further challenges. the upcoming experiments will be extended to context-based and full morphological annotation. the experimental data show that parallel to the rule-based approaches rnn models can be a sound alternative to process armenian diachronic and variational corpora. no precise estimations are available for the models trained on rb tagged corpora to assess rnn timeconsumption. however, current tagging results for mwa and ca allow to take into account an iterative and mixed approach which can partially cover the annotation of non-specialized models reducing the time-consumption for new corpora design. language varieties have usually fragile vitality when lacking the "standard" status (cf. dialects) and/or natural regenerating native speakers' rotation and evolution (cf. mwa). the first is true for the armenian dialects which have always "secondary" status as compared to the standard language with which they coexist. mwa is a standard language and yet it has become mainly a diaspora/heritage language for more than a century. therefore, the documentation of these armenian varieties as well as the processing of the documented data is of foremost importance not only in nlp but also and especially in linguistic, anthropological and social perspectives.