The Use of Khislavichi Lect Morphological Tagging to Determine its Position in the E ast S lavic Group (2023)

Abstract: The study of low-resourced East Slavic lects is becoming increasingly relevant as they face the prospect of extinction under the pressure of standard Russian while being treated by academia as an inferior part of this lect. The Khislavichi lect, spoken in a settlement on the border of Russia and Belarus, is a perfect example of such an attitude. We take an alternative approach and study East Slavic lects (such as Khislavichi) as separate systems. The proposed method includes the development of a tagged corpus through morphological tagging with the models trained on the bigger lects. Morphological tagging results may be used to place these lects among the bigger ones, such as standard Belarusian or standard Russian. The implemented morphological taggers of standard Russian and standard Belarusian demonstrate an accuracy higher than the accuracy of multilingual models by 3 to 15%. The study suggests possible ways to adapt these taggers to the Khislavichi dataset, such as tagset unification and transcription closer to the actual sound rather than the standard lect pronunciation. Automatic classification supports the hypothesis that Khislavichi is a border East Slavic lect that historically was Belarusian but got russified: the algorithm places it either slightly closer to Russian or to Belarusian.

## introduction automatic classification of lects that are both closely related and low-resourced has been the target of dialectology studies for the last two decades, because it provides insights on the linguistic variation, used both for developing language tools and language studies @xcite @xcite . however, the morphological tagging results were rarely considered to be the basis for automatic classification. the main goal of this research is to develop a morphological tagger for a low-resourced east slavic lect of khislavichi with the help of the lects that possess significantly more resources, standard russian and standard belarusian. after the tagger is built, the morphological variation becomes the main subject of study: what differences between standard russian, standard belarusian, and khislavichi stop the tagger from the correct cross-prediction between these lects? the automatic classification of standard russian, standard belarusian, and khislavichi lects with distance-tree matrix @xcite demonstrates how it is possible to specify the position of a low-resourced lect in the context of much bigger lects that are phylogenetically connected to it. the neutral term lect is used instead of dialect and/or language because the latter often imply a hierarchy of subjugation and one lect being considered as an inferior part of the other. the distinction between a language and a dialect is significantly more connected to the sphere of sociolinguistics rather than pure linguistic variation @xcite . as language classification studies operate in terms of language distance and not language hierarchy, all three lects are studied as equal. avoiding the language/dialect distinction, particularly in the study of the khislavichi lect, is a requisite, as its current research is heavily influenced by extralinguistic factors. by now, it is safe to assume that khislavichi is an east slavic lect, phylogenetically more closely connected to standard belarusian while being heavily influenced by standard russian as a part of the process of an intense russification @xcite . this process continues now and affects lesser russian lects @xcite , east slavic lects (naza-accent, 2023), and the lects of ethnic groups of diverse origins in russia @xcite . even the very name khislavichi is transcribed into english as khislavici (ipa: [hisëavitci]), with the letters ch representing the russian voiceless alveolo-palatal affricate c and not the native voiceless postalveolar affricate č which is more similar to @xcite in this paper, which deals with the issue of linguistic variation and classification in relation to morphological tagging, khislavichi lect is treated as a separate entity equally connected to standard belarusian and standard russian. therefore, automatic language classification will provide data on its possible grouping with the other two, not its position within the language hierarchy or its inclusion into either standard russian or standard belarusian. however, while making conclusions, we should not ignore the historical context, mainly the intensive russification the khislavichi lect has undergone. treating khislavichi as a separate lect opens the road to its fully independent study, as it will no longer be considered a part of the russian language (zaharova and orlova, 2004), contrary to its placement within the russian lect in the earlier research. this study requires the development of natural language processing tools (nlp). the first tool in this pipeline is generally a morphological tagger. morphological tagging is a process (and a product of this process) that includes assigning universal partof-speech tags and morphological features to the tokens @xcite . morphological tagging is employed to get basic information on the grammar structure of the lect under study. afterwards, it is utilised in both further processing, such as lemmatisation or masked language modelling, and the research of a lect, for instance, in the creation of a lect grammar. some studies suggest that the results of lect automatic processing may also be used for its classification @xcite . morphological tagging was not considered to be the best candidate in comparison with perplexity @xcite . however, morphological tagging seems to be useful for the preliminary classification that presents general information on the relationship between a small lect and the larger ones that surround it or influence it. the classification of lects is a process of grouping lects by some meaningful characteristics. among such characteristics may be the historical differentiation @xcite or typological similarities @xcite @xcite . the classification may give some insights into the development of a language or signal of a currently occurring intense language change. in this paper, the suggestion is to classify the lects based on the differences that cause problems in the work of a morphological tagger. the optimal algorithm is a distance-tree matrix @xcite . to build a tree from a triangular distance matrix collected from the models accuracy scores we use a statistical method, upgma @xcite , implemented via biopython package @xcite . it is generally used in evolutionary biology, and probably may be successfully adapted for language study, as the whole idea of tree classification had been (schleicher, 1863). the second section is dedicated to the previous research on the topics of morphological tagging, khislavichi lect studies, and classification methods, including the automatic ones. the third section details the methods of morphological tagging (bi-lstm neural network) and automatic classification (a distance matrix-based tree) that are going to be implemented in the experiments. the fourth section contains information about the datasets used for training, evaluation, and tests of taggers. the fifth section presents experiments and their analysis, performed on russian, belarusian, and khislavichi material. the conclusion wraps up the research with the final analysis of the morphological tagger prediction efficiency and provides an outline for future research of the khislavichi lect and the classification methods. ## related work the variation within the low-resourced closelyrelated territorially close lects, often joined under the term dialect, has been intensely studied for the last two decades @xcite . different methods have been used to study the lect variation. the most frequent, though heavily criticised @xcite , is the edit distance group of methods @xcite , mainly represented by using levenshtein distance on a certain list of words @xcite @xcite , generally the swadesh list items @xcite . recent years, however, witnessed some changes in this situation. the swadesh list items are no longer the ultimate solution, some other, topic-restricted, wordlists are used (saxena and borin, 2013) @xcite . the methods changed as well: phonetical approach @xcite , the perplexity of large language models on the task of masked language modelling @xcite , sequence alignment approach @xcite , linguacultural approach (lewandowska-tomaszczyk, 2021), information theory approach @xcite , and interdisciplinary approach @xcite . morphological taggers generally were not used to measure language variation, but most were claimed to benefit from it @xcite . this article inquires about the possible reverse situation when language variation is measured by the results of morphological tagging. automatic morphological tagging is an nlp task that has existed for a long time @xcite ) @xcite ) @xcite @xcite . there are different approaches to it, especially when low-resourced lects are considered. currently, the dominating approaches are the rule-based, adjusted for the needs of a particular language (gambäck, 2012), and the more universal one based on recurrent neural networks @xcite . the current shift into the direction of languageindependent morphological tagging @xcite leads to the development of taggers that can deal with close lects @xcite , which is an essential problem, for instance, for arabic @xcite . low-resourced morphological tagging is gaining increasing recognition (imanigooghari et al., 2022) @xcite . now a lot of attention is paid to the selection of data to train, evaluate, and test a tagger on @xcite . the old models @xcite are adjusted @xcite to meet the new requirements of efficient training on low-resourced closely-related lects. low-resourced closely-related east slavic lects are currently undergoing extinction @xcite , with khislavichi being no exception @xcite . the khislavichi lect is a lect of the khislavichi settlement, which is located on the border between russia and belarus. it used to be a part of belarusian territories until the beginning of the xx century, but became a part of russia in 1924 @xcite . its study began at the beginning of the xx century when it was characterised as a northern belarusian dialect @xcite (though it is important to state that the belarusian language itself was then considered a dialect of russian by the "colonial scientists" from the rus-sian empire). since the 1960s khislavichi was considered to be a russian dialect with belarusian elements becoming less and less prominent @xcite . the current consensus is that the khislavichi lect is a borderline lect between russian and belarusian, sharing key features with both these languages @xcite ). yet the russian features are mostly borrowed or inflicted upon this lect, and belarusian features form its historical core @xcite . the question of its classification remains uncertain. for a review of the historical and contemporary state of the khislavichi lect and its overall linguistic description, one should refer to @xcite . there are different ways to produce a classification of lects @xcite . the idea of splitting lects into non-hierarchical groups by performing hierarchical clustering became prevalent during the last decade @xcite . the clustering is made automatically, as recent years have witnessed an increasing use of quantitative methods for classification @xcite . the algorithms that provide visualisations for the clusters were developed as well @xcite . some clustering solutions are distance-based @xcite . ## method the research is split into three parts. the models for russian and belarusian are prepared via training and evaluation on the respective language datasets to get a general picture of their performance. after that, a cross-evaluation is performed to see the ability of both models to generalise based on the knowledge they have previously acquired. next, the predictions on the khislavchi lect material are made and evaluated manually. the final stage includes the automatic classification of the lects based on the results of the tagging evaluation. there are different models, including the multilingual pre-trained ones, that are used for the morphological tagging of heterogeneous lects @xcite @xcite . however, for this experiment, the model for training should be as unaware of the potential structure of russian, belarusian, or khislavichi as possible. at the same time, after training on one lect, it should still not know the others while possessing the ability to generalise its previous findings for their tagging. one more requirement is that the model should preserve some level of consistency while being trained on the corpora that are not significantly low-resourced but yet do not achieve the old national corpus standard of 1 million words. the models stated previously are either universal or underprepared for the lowresource scenario. one possible pick that satisfies all the requirements is the stanza tagger, developed at stanford for universal dependencies tagging @xcite . it was recently modified to use bidirectional character-level lstm by default, and specifically adjusted to the aims of part-of-speech tagging, the starting point for lowresource nlp @xcite . this fork is used in this paper. after the model is chosen, the training phase begins. it consists of two subsequent runs of the code, yielding two trained models, one for russian, and one for belarusian respectively. these models should satisfy the requirements for overall accuracy, overcoming the basic threshold of 50%, and, optimistically, getting close to the threshold of 85 -90% overall accuracy. to exclude overfitting, the additional evaluation of the model on the previously chosen part of the dataset is performed. the models are also compared to the previous results of morphological tagging for the russian and belarusian languages to check whether their ability to tag is not significantly lower. in the latter case, the shift to the other language model is probably going to be necessary. when the conditions are met, the models are cross-evaluated. the model that was trained on the russian material is evaluated on the belarusian material, and vice versa. this helps to evaluate the ability of both models to generalise before the final run is performed. there can be no expectations at this stage, as both the russian and the belarusian models are going to be trained on the monolingual corpora. however, as both lects are east slavic, the models are probably going to demonstrate at least a 20 to 30 per cent level of accuracy. at this stage, a manual analysis by the researcher must be performed to highlight some common mistakes that can be made by the model that switches from russian to belarusian tagging and the other way around as well. the final run of both models is going to be performed on the khislavichi material. as it is with cross-evaluation runs, there is no particular threshold that the models are expected to overcome. and for khislavichi tagging there is an additional obstacle of the gold dataset absence. so, the evaluation is performed only by overall accuracy, and both models are getting the easiest possible treatment: they may not guess all the tags, however, if the tags they assign are correct, they get a point. their performance, thus, may seem to get significantly boosted, though, in fact, it is going to remain at the same level as in the cross-evaluation stage. the main expectation here is that predictions of both models demonstrate a close accuracy score, as khislavichi lect is generally supposed to be located just in the middle of the spectre between standard russian and standard belarusian. if the expectation is not met, the reasons should be provided. this leads to the analysis of the errors the models make on the khislavichi material, as well as to possible explanations of the most common mistakes. and if one of the models performs abnormally well (getting close to the monolingual evaluation level of accuracy, or dealing particularly well with some classes of units), or abnormally bad (getting close to the bilingual evaluation level of accuracy, or coping particularly badly with some classes of units) the rationale is also going to be given. if the results of the two models contrast in some meaningful manner, clarification is expected. the analysis should provide recommendations for developing future datasets and taggers for the khislavichi dataset. after all the accuracy scores are acquired, the overall analysis for the whole picture of russian-belarusian continuum morphological tagging should be provided. the research finishes with an attempt to automatically classify the three lects of standard russian, standard belarusian, and khislavichi. for this, the standard method of distance-tree matrix @xcite is applied. this method has been previously successfully used in phylogenetic studies in biology @xcite @xcite . the borrowing of the automatic classification method from biology is justified, as the evolution of language and the evolution of life share a lot of similarities @xcite , which have been highlighted recently @xcite , and the concept of linguistic phylogeny tree is borrowed from biology @xcite . this may lead to a new potential way of lect classification, a possibility to clusterise lects with the same computational methods as species @xcite . when the classification is performed, the resulting trees are drawn by a python script that employs these methods. the resulting trees demonstrate the clusterisation of the standard russian, standard belarusian, and khislavichi lects predicted through morphological tagging. ## data three datasets are employed for the experiments. the first one is the corpus of the khislavichi lect @xcite . the second one is the belarusian-hse corpus, the belarusian universal dependencies one @xcite . the third one is the taiga corpus, one of the russian universal dependencies datasets @xcite . as the khislavichi lect and its relationship to standard russian and standard belarusian form the centre of the research, the entirety of the currently available data should be investigated. these data in the corpus have been collected and digitised by a. ryko and m. spiricheva @xcite . these are transcribed recordings of the interviews with the native speakers of this lect, all born between the late 1920s and the late 1960s. the khislavichi data is heterogeneous. not all texts are presented as transcriptions, most of them are edited into a cross between a transcription and a standard russian text: only the differentiating lexemes are given in parentheses. for instance, in kak (k) 'how', kak is a standard russian form, and k is a khislavichi form. for some texts, however, transcription is available as well. additional complications arise from the fact that the lect was under the process of intense russification during the soviet period, which manifests in the speakers born in the late 1920s and the late 1960s speaking in different manners. the common features persist (such as using č, more similar to belarusian, and not c, more similar to russian, in words like hislaviqah 'khislavichi'), however, some radical changes in lexis start manifesting (for example, using luk instead of cybul for onion). the texts are interviews, with interviewers speaking in standard russian. with these issues in mind, the khislavichi dataset is additionally preprocessed. the latter issue is resolved by the exclusion of the interviewers' lines from the final dataset. the issue of an inlect heterogeneity is treated as a matter of fact, no additional splits are performed. where the transcriptions are available, they are taken. if a pair of standard russian lexemes and a differentiating khislavichi lexeme in parentheses is met in the texts made to resemble standard russian, only the differentiating khislavichi lexeme is taken. the resulting set of texts is transferred into the conll-u format, which turns it into a corpus of nearly 100 000 tokens. this corpus is split into the training, evaluation, and test datasets (80 000, 10 000, and 10 000 tokens respectively). as the research does not imply training the model for the khislavichi lect, only the test dataset is going to be used for the later evaluation of the models trained on the belarusian and russian material. the belarusian-hse corpus @xcite is chosen to get the model able to perform morphological tagging for belarusian. while there are some much larger corpora, for instance, belarusian n-corpus (n-corpus, 2023), their tagging is not disambiguated and thus is impossible to be used for training. additionally, these corpora are not in open access. the belarusian-hse corpus, in turn, is available in conll-u format from the start and was designed with the tagger training in mind, as this is the requirement for the universal dependencies corpora. this corpus consists of different text genres, from the newspapers to the telegram messages and community posts, so it may safely be called a balanced representation of the modern belarusian language @xcite . the size of the corpus is 305 000 tokens, which makes it sufficient for the modern taggers to be trained on, especially for the ones that are well-adjusted for the universal dependencies low-resourced datasets @xcite . the corpus is split into the training, evaluation, and test parts in 80/10/10% proportion, as is generally the case with the universal dependencies datasets. in contrast to the khislavichi dataset, training and evaluation parts are going to be used in the training phase to get the model for tagging belarusian texts. the test part is going to be used for the final evaluation of this model, as well as for testing the ability of the model trained for the tagging of russian texts, which subsequently will aid the automatic classification of standard russian, standard belarusian, and khislavichi lects. the russian corpus, in its turn, should meet one, but a very strict condition. it should match the belarusian-hse corpus in size, the precision of manual tagging, and adjustment for the taggers that are designed for the data in universal dependencies (conll-u) format. again, there are giant corpora, consisting of billions of words, such as the russian national corpus (corpus, 2023), but they are not in open access, and, what is much more important, their tagging is not fully disambiguated. there is even a big universal dependencies corpus, 1.5 million words syntagrus @xcite . however, its use is going to give an advantage to the russian model. it is going to train better on a significantly bigger corpus. so, a smaller corpus should be used. the taiga corpus @xcite , with an overall size of 197 000 tokens, is proposed as a suitable candidate. this corpus is prepared in a universal dependencies format and designed specifically for tagging tasks. it is smaller, though not greatly, than the belarusian-hse corpus. it is also balanced, and quite representative of modern russian, containing blog texts, news texts, fiction (including poetry) texts, wikipedia articles, as well as different texts from social media @xcite . it is split into the training (80%), evaluation (10%), and test (10%) parts. as with belarusian, the model is going to be trained with the use of the training and the evaluation parts of the dataset. after this, the test part of the dataset will be used for the evaluation of the trained model as well as the model trained on the belarusian dataset, supplying the data for the automatic classification of the lects. both belarusian-hse and taiga contain a significant amount of texts from social media, a genre that is as close to the main khislavichi corpus genres, everyday talks and events retelling. this should eliminate genre elements from affecting accuracy scores of the models. ## experiments and analysis the starting point is testing the models in the languages they were trained on. thus, the first experiment includes testing the model that was trained on the standard russian taiga corpus on the test subset of this corpus, and testing the model that was trained on the standard belarusian corpus on the test subset of its corpus. after that, cross-evaluation is performed: the model that was trained in standard russian is tested on the test dataset from the belarusian corpus, and vice versa. the evaluation highlights the main dif- @xcite and modified stanza @xcite . the best results, here and afterwards, are given in bold. in the last experiment, both these models are tested on the restricted test dataset from the khislavichi lect, with an in-depth analysis of the reasons why each of the models succeeds in tagging of particular language units and fails in others. some preliminary predictions on how the classification may look like are made at this stage. the final analysis includes the comparison and the discussion of the experiments results, putting each of them in the general context of the research. two possible ways for the following automatic classification of lects, based on the morphological tagging evaluation results, are suggested and realised. the pro et contra for both of them is given. ## conclusion the models for morphological tagging of russian and belarusian, based on the architecture provided in @xcite , beat the previous results set by the multilingual models by a significant margin for both belarusian and russian. both models demonstrated the ability to perform a moderately successful tagging of the closely-related lects. the cross-evaluation results and the results of evaluation on the khislavichi dataset show that the russian and belarusian models may be used for the preliminary tagging of closely related lowresourced east slavic lects. the classification by the results of morphological tagging retains the same uncertainty level of the khislavichi lect position among the other east slavic lects as the classifications reviewed in @xcite . in the current dataset orthography state, the reasonable conclusion is the khislavichi dataset being classified as borderline between russian and belarusian datasets, not the khislavichi lect -as borderline between russian and belarusian (the same notion for italian presents @xcite ). we are going to modify and implement the presented automatic classification method for the bigger number of lects, and use the transformed and tagged khislavichi corpus for the further khislavichi lect processing. the corpus probably will later become a universal dependencies part.