The Application of Corpus-Based Language Distance Measurement to the Diatopic Variation Study (on the Material of the Old Novgorodian Birchbark Letters) (2025)

Abstract: The paper presents a computer-assisted exploration of a set of texts, where qualitative analysis complements the linguistically-aware vector-based language distance measurements, interpreting them through close reading and thus proving or disproving their conclusions. It proposes using a method designed for small raw corpora to explore the individual, chronological, and gender-based differences within an extinct single territorial lect, known only by a scarce collection of documents. The material under consideration is the Novgorodian birchbark letters, a set of rather small manuscripts (not a single one is more than 1000 tokens) that are witnesses of the Old Novgorodian lect, spoken on the territories of modern Novgorod and Staraya Russa at the first half of the first millennium CE. The study shows the existence of chronological variation, a mild degree of individual variation, and almost absent gender-based differences. Possible prospects of the study include its application to the newly discovered birchbark letters and using an outgroup for more precise measurements.

## introduction this article discusses the complexities of studying the variation with the low-resourced data on the material of the corpus of the east slavic birchbark letters, dated from 1020 to 1500 ce, found in the territories of modern russia (among others, staraya russa, pskov, moscow, smolensk) and belarus (viciebsk foot_0 , mscislaȗ foot_1 ); the most wellknown site with the most manuscripts, where they were originally discovered, is novgorod @xcite . the research investigates three distinct types of variation: variation within a collection of doculects from the same place and time, time variation, and gender variation. the latter two are impossible to study without the first one (with a high level of individual variety, it is not possible to produce more effective research with more approximation) and are crucial for the study of old novgorodian, allowing one to understand the social dynamics within the society and the reflection of its development on its language. they are also irreplaceable for the building of the old novgorodian lect resources as they help to capture its everchanging state until its extermination in the xv -xvi centuries ce. birchbark letters are usually small fragmented texts, so there is no way to use a more traditional lectometry @xcite or a corpus-based @xcite methodology. the study requires a method designed for small raw corpora. the method relies on the combination of frequency-based metrics, string similarity measures, and a set similarity coefficient and their application to the subtoken-level units. the research is based on three hypotheses: h1. the differences detected by the proposed method among the individual documents are insignificant. h2. the differences among the chronological periods of old novgorodian are significant. h3. genderlects are present in old novgorodian, there were significant differences between the style of writing between men and women. to test the hypotheses, the article uses a combination of quantitative and qualitative analysis, aimed at differentiating between random distributional skewings and regular significant differences. the important constraint is that the proposed method is intended to be preliminary, its results are not set in stone and require subsequent exploration by a human scholar, which this article is going to perform. however, it is necessary to state that a thorough qualitative analysis will require a detailed close reading of hundreds of texts @xcite , so the study concentrates on the method application and the exploration of its results. the structure of the study is as follows. section 2 expounds on the history of the old novgorodian studies and defines the present research gap. section 3 provides detailed information on the utilised data. section 4 explains the method and the means of analysis. section 5 reports on the results of the experiments. section 6 is a conclusion that outlines the future research prospects. ## related work the east slavic birchbark letters have been known in the field of slavic studies since the second half of the xx century @xcite , however, for a long time they failed to gain recognition, as scholars perceived them as erroneous and illiterate, thus having little to contribute to the language study @xcite , which is a common misconception in traditional and generative studies @xcite . only during the last two decades have the linguistic features of birchbark letters received acceptance as a full-fledged resource of information on lects spoken at the corresponding territory @xcite . since then, a significant body of work has been produced, with topics ranging from the language of these manuscripts @xcite @xcite , including the genderlect variation @xcite and sociolinguistics @xcite , to the creation of a network of linguistic databases that includes birchbark letter database foot_2 (bld), and russian national corpus foot_3 (rnc). old novgorodian is part of a large group of his-torical and contemporary lects, generally called fragmented languages, which are attested only partially and by rather low-resourced corpora (in the best-case scenario, less than 100 000 tokens, in the worst-case scenario, less than 100 tokens) @xcite . these lects present a significant challenge to the nlp methods due to their low-resourcedness and heterogeneity @xcite @xcite . old novgorodian and the cases akin to it @xcite add a new layer to the complexity of the task, as the texts themselves frequently lack significant parts due to the damage to the original manuscript. despite the relative well-studiedness of the old novgorodian @xcite and a high awareness of the low-resourcedness problem in nlp (dione, 2019), there are crucial lacunae in the current research. some types of language variation in the birchbark letters gained attention @xcite , but not all of them: for example, the chronological division remains understudied @xcite . the 2010s advancements in computational methodology @xcite were not applied to the language variation within old novgorodian. at the same time, low-resourced nlp rarely problematises the features of the analysed lects from the linguistic perspective @xcite , but rather declares these features as obstacles to be overcome via strictly mathematical algorithm enhancement @xcite and only rarely with language-aware methods @xcite . the current study aims to become the first step in the direction of a language-aware computer-assisted study. ## data the research corpus consists of 1249 documents available in the bld as of february 2025. the distribution is heavily skewed in favour of the novgorod letters which comprise most of the dataset. it complicates the comparison between different regions. at the same time, some of the non-novgorod charters are still going to influence the results of comparison by any other criterion (gender or time frame), especially given the number of tokens in some of them. for instance, mosk_3, the third of the charters found in moscow, has 470 tokens. as one of the biggest manuscripts in the dataset, it may quantitatively outweigh a hundred other charters. to eliminate this noise in measurements that use other criteria, the study data is restricted to charters from staraya russa and novgorod that represent old novgorodian in the strict sense @xcite . for the study, these letters undergo several stages of preprocessing foot_4 . the birchbark letters suffer from being very fragmented, and it is barely possible to use them either in their raw form preserving only the fully visible characters (there is not enough information), or in the processed form containing all the reconstructed characters (which may lead to the researcher bias interfering with the existing variation). thus, preprocessing starts with creating the middle ground. the initial step is to exclude all completely non-reconstructible tokens, marked with .... the next stage is the deletion of string breaks, marked @xmath0. following this, each of the charters is joined into a single string. after this, the non-reconstructible parts of the existing tokens (... joined to the tokens in the existing forms) undergo replacement with ^signs. the same applies to the parts of the tokens that may be inferred from the context but are not present in the charter in any shape or form, originally surrounded by (). if such reconstruction spans between two or more tokens, both the end of the first token before the reconstruction and the beginning of the second token after the reconstruction receive the ^sign. the present misspellings, originally designated by {}, are excluded from the texts. the parts of the tokens that are not fully visible but reconstructible with a high degree of certainty, surrounded by [], are taken as is; only the designating signs [] are excluded from the resulting text. the final step is to merge the consequent break signs ^that appear before the token in cases when the break and/or unrecognisable symbols go before the token that contains a non-reconstructible part. the further modifications to the dataset have the purpose of adjusting it for the clusterisation: the letters from novgorod and staraya russa still suffer from an imbalance between the size of different charters: some are too small, consisting only of one token, and some are too big, containing hundreds of tokens. figure 1 shows the distribution. for a more straightforward comparison, the next preprocessing step excludes each letter that consists of less than two tokens and five symbols. the letters that consist of more than 60 tokens (an approximate value of q3 foot_5 + 1.5 * (q3 -q1), with @xmath1.00 and @xmath2.00) are shortened to the first 45 largest and the first 15 smallest tokens to preserve their features in the set while partially eliminating imbalance. figure 2 shows the final distribution: here, the only remaining letters are the ones that are more than two tokens or five symbols and less than 60 tokens in length. this contrasts the original distribution, where there was a significant number of letters containing one short token, which are not very useful for comparison purposes, and a dozen of letters that contain several hundred tokens, which significantly overweight the other letters, rendering comparison meaningless. during the following step, each remaining letter receives two metadata tags, based on the existing analysis: time period and author gender. three periods are in the focus of the research: 1020 -1140 (the early stage @xcite ), 1180 -1240 (marked by the intensive contacts within the circumbaltic region @xcite ), and 1300 -1360 (one of the latter stages of the old novgorodian development, also marked by the dissolution of the east slavic area @xcite ). the texts assigned to these periods in rnc acquire the corresponding tag, the others get tag x. for most of the texts, there is no possibility to deduce the gender of the author. in such cases, they receive the tag unk. otherwise, m (for authors referred to with masculine gender), and f (for authors referred to with feminine gender). the resulting data frame containing 1158 letters consists of 7 columns, excluding index: text (the processed text of the charter), charter_number (the index of the letter in the database), num_token (number of tokens, excluding punctuation marks), text_len (length in symbols, excluding punctuation marks and breaks symbols ^), author_gender -the gender of the author, date -the estimated period of text creation, place -the estimated place of text creation. further preprocessing, required for specific language distance measurement methods, will be discussed in the corresponding section. ## method this section consists of three parts that elaborate on the preprocessing of the data for the experiments, applied quantitative methodology, and qualitative analysis. the implementation is available via github foot_6 . ## experiments and analysis this part provides the summary of the experiments and the subsequent discussion of the linguistic differences detected by the method. ## conclusion the paper employed a new method to study individual, chronological, and gender variation within old novgorodian. it supported the hypothesis h2 of chronological variation, showing the similarity between earlier and later periods of the old novgorodian development. at the same time, no signs of gender-based variation are present (hypothesis h3 is thus rejected): from the existing material only it is impossible to claim that old novgorodian had genderlects, which supports the primary qualitative work on the topic, @xcite . yet the amount of the available material may be misleading: it is possible that there is not enough data. the method statement on the variation within the different time periods highly depends on the letter size, supporting the idea of balancing the corpus before the method application @xcite ; hypothesis h1 is thus supported only partially. one of the key elements that helped the method to distinguish between different chronological periods and played an important role in other tasks is the contrast between the symbols that denote reduced and full vowels. this is not the only found contrast, as the method was able to find other factors, such as lexical differences. it is also paramount to note that all the components of the combined metric were analysed, and partly proved, during the final qualitative analysis. this affirms the necessity of using lectometry methods for computer-assisted and not computer-driven research. the acquired classification and the method itself, especially 3-shingle-based representation, will aid the analysis of the newly discovered documents and the exploration of how they fit the existing picture. it will facilitate expert judgment about the period of their creation, aiding theoretical paleographic analysis (janin and @xcite . the found similarities and dissimilarities may be included as linguistic features in the existing network of old novgorodian databases. the results require further attention and exploration, especially the ones that did not provide any satisfactory conclusions, such as the ъ$ and о distribution differences between the letters authored by men and women. the study shows that the quality of the resources is of the utmost importance for computational methods, especially for language distance measurement. one possible further research direction is using an outgroup (for example, old east slavic legal charters) to provide additional linguistic context to the clusterisation trees @xcite .