Unified Annotation of the Stages of the B ulgarian Language. First Steps (2024)

Abstract: The paper reports on an ongoing work on a proposal of guidelines for unified annotation of the stages in the development of the Bulgarian language from the Middle Ages to the early modern period. It discusses the criteria for the selection of texts and their representation, along with some results of the trial tagging with an existing tagger which was already trained on other texts.

## introduction so far, research on middle bulgarian and early modern bulgarian texts has not followed systematically applied methods of corpus linguistics, and no attempts have been made to integrate the corpora with electronic descriptions (and editions) of texts and manuscripts. separate corpora of texts reflecting different stages of the history of the bulgarian language are part of other large corpora which also contain old russian and old serbian sources 1 , as well as historical material of other indo-european languages (greek, latin, armenian, among others 2 ). numerous texts have 1 http://www.manuscripts.ru/ 2 http://dev.syntacticus.org/#annotation-principles 3 https://histdict.uni-sofia.bg/textcorpus/list 4 https://github.com/proiel been gathered in the historical corpus of the bulgarian language (from 10 th to 17 th century) 3 . in addition, the annotation schemes followed by the research efforts (if available at all) differ quite a lot as they reflect specific research purposes and cannot be uniformly applied. this means that the results of indexing and data analysis cannot be based on uniform criteria and cannot be used for comparison purposes. in fact, some textual collections contain only searchable texts (concordances can be made using other external tools) without any further linguistic annotation, active links to dictionaries or other useful tools. the present paper reports on the development of an ongoing project which aims at offering a proposal for a unified annotation for the bulgarian texts of all stages in the history of the bulgarian language. we have started from the annotation principles of the proiel project 4 to extend the linguistic annotation to be further used for middle bulgarian and early modern bulgarian texts while considering other recent efforts in this direction @xcite . a dictionary of words and their wordforms is generated, and an applicable model for description of texts and manuscripts 5 is to be integrated to allow for electronic publication of the texts along with applicable metadata. a similar project 6 for middle bulgarian dealing with the translation of philip monotrop's work dioptra in the 14th century, is being developed at the university of innsbruck. in this project, the slavonic translation and the greek original are lemmatized, morphologically annotated and aligned on the word level, thus yielding a searchable bilingual corpus. ## the approach the goal of the project a unified annotation of the stages of the bulgarian language (ausbul) is to create a model for infrastructure which will made the texts and the annotated data online accessible and user-friendly for researchers and other potential users. the infrastructure integrates several components: the project draws upon the idea that only such an integrated approach, devoted to sources from one era and resting on standardized practices and solutions, can provide new insights into the history of language and literature, to be further compared with similar phenomena from other cultures. the emphasis of our effort is to test for the possibility of applying the principles of corpus and computational linguistics to a selection of different types and genres of medieval texts. the result will be a unified model for describing the language and the texts of the middle bulgarian (13 th -14 th centuries) and the early modern bulgarian (17 th -18 th centuries) periods with respect to both archaic and vernacular samples. by following uniform principles of electronic linguistic annotation, we may compare and analyze different phenomena in the development of the grammatical system of the bulgarian language. at this first stage of the project, work is being done in several directions: this part of the project requires a considerate analysis of the history of the texts. for example, some texts must be selected to make possible the comparison of the earlier (or archaic) variants with the early modern bulgarian variants that reflect a more vernacular version of the language. ## the texts we will illustrate the approach with one of the texts currently being worked on, the acts of the apostle thomas in india @xcite @xcite . the preliminary selection of the witnesses (sources) was made after first comparing the earlier witnesses from the 14 th and 15 th centuries with the copies from the 17 th -18 th centuries. one representative of the text was selected -a manuscript from the dragomirna monastery no. 700 from the 15 th century @xcite . the manuscript contains a different, hitherto unknown redaction of the text @xcite , which would complement the observations made up to this point @xcite . transcripts from the 17 th -18 th centuries are divided into two groups. the first group involving the archaic version of the text, dating back to the 14 th -15 th century witnesses, which, is represented by the damaskin of kostenets (chai 503, second half of the 17 th century, @xcite . the second group contains the variants of the text according to the new bulgarian damaskini and miscellanies from the 17 th -18 th centuries. this second group is heterogeneous, with the following representatives: all these witnesses are representatives not only of different early modern bulgarian versions of the text, but also reveal different directions in the development of the bulgarian language in the 17 th -18 th centuries. ## resources and tests we have experimented with applying linguistic annotation on two different versions of the textson the (archaic) damaskin of kostenets (kosten) and the (vernacular) koprivštitsa damaskin (kopriv). before starting with the automatic linguistic annotation, we had to apply some preprocessing to our texts. the first step consisted of segmentation on the sentence level where we relied on the segmentation in the respective editions, i.e., we split the sentences wherever we found a punctuation mark indicating a sentence boundary (such as full-stop, colon, georgian paragraph separator). the problem of word segmentation (and tokenization) was more intricate as word segmentation in the manuscripts does not regularly correspond to modern practices. we adopt a method developed by @xcite for the edition of the pop-punčov sbornik that allows us both to keep information on word boundaries in the manuscript and to provide the taggers with linguistic input coherent with modern practices. special signs were added to the text indicating word boundaries: • a vertical line means that a token is written together with the following token in the manuscript, but the tokens are analyzed as two units for the morphological annotation (e.g., ца р | же lit. 'king thus', where же is a discourse particle, stands for ца рже in the manuscript but it is given as ца р же in the annotated version). as we are currently in the starting phase of the linguistic annotation and aim at finding out the best way to tag our data, we tested two different models for the tagging process. at first, we used the damaskini texts annotated by i. @xcite to train a model using them as training data. the tagging was performed using the stanza tagger version, which was modified to use bidirectional character-level lstm by default and specifically adjusted for parts-of-speech (for low-resource languages) by y. @xcite . as this tagger does only perform part-of-speech tagging (postagging) and morphological annotation but no lemmatisation, we had to use another tool for this purpose -lemming @xcite . the annotated texts are stored in the conll-u format and follow the conventions for universal dependencies @xcite . before we could use the model based on the data of @xcite to tag our texts, we had to apply one further step of preprocessing. as all the training data was in the latin alphabet, we created a script that transcribes the cyrillic letters to their latin counterparts and strips the texts of all the superscripts and diacritics. we, thus, performed the linguistic annotation on a graphically simplified version of the texts that matched the training data by @xcite . the obtained results will henceforth be referred to as tag1. as the number of tokens in the training data was rather small (around 60.000 tokens), we performed a second round of tagging using other data from another source. in this second round, we used annotated old church slavonic texts from the proiel @xcite and the torot @xcite corpora. the data is linguistically less similar to our texts than the data by @xcite but contains much more tokens (around 357.000). when we trained our model, we did not use the original data from the proiel and the torot corpora but adapted it to some linguistic peculiarities of the bulgarian language. prior research has shown that data in such an adapted format provides better results for middle bulgarian @xcite . the result of this second round of annotation will be referred to as tag2. ## results the results from the tagging using the two annotated datasets differ in elements that may have different linguistic interpretation depending on the purpose of the intended corpus (considering further annotation). the pop-punčov dataset (tag1) follows the multext-east annotation guidelines with a stricter focus on morphology while the dioptra dataset (tag2) closely (although not entirely) follows the proiel/torot annotation principles (which were directed toward the next step in the syntactic annotation for the purpose of building the proiel treebank). results with tag1 and tag2 differ in those elements (and parts-ofspeech) which may have different syntactic functions -pronouns, adverbs, particles, auxiliaries. the accuracy of the tagger trained with the respective datasets on the two texts is given in applied pos-tags were not considered erroneous when calculating the accuracy in the following cases: a. when tags are not part of the dataset as in: propn that are marked as noun if there is no propn in the training dataset; aux that are marked as verb if there is no aux in the training dataset. b. possessive pronouns that are marked as pron or adj if there is such marking in the training dataset. the tagger achieves the greatest accuracy with tagging the vernacular korpiv when trained with the pop-punčov dataset (tag1) and the lowest accuracy with kopriv and trained with the dioptra dataset (tag2). the pos-tagging of the archaic kosten was better when the tagger was trained on the dioptra dataset (tag2) than with the (vernacular) pop-punčov dataset (tag1). when the tagger was trained with the pop-punčov dataset (tag1) comprising texts from the same period, its results on both texts were much closer than when it was trained with the dioptra dataset (tag2). most errors on pos-level are found when the vernacular kopriv text was tagged with the tagger trained with the dioptra dataset (tag2) -in the example below 4 of 9 wordforms are wrong. и 'and' cconj непрѣста нно 'ceaselessly' adv б҃ ꙋ 'to-god' verb noun се̏ 'self' (reflexive) det pron мл҃ ꙗха 'prayed' noun verb и 'and' cconj сла вѣха 'praised' adj verb ст҃ аа 'saint' adj трⷪ ица 'trinity' noun the results for morphological annotation are lower (and for lemmatisation are even lower) but they are also linked to the accuracy of the postagging. our trial tagging has shown results that are similar to those from previous attempts at tagging early slavic texts but are still lower due to the character of the texts (they are bulgarian and from a later period). except for the normalization method with statistical crf-tagger marmot and a neural network tagger, @xcite experimented with applying modern russian resources to pre-modern data to show that transfer experiments did not improve tagging performance significantly, but state-of-the-art taggers still reached between 90% and more than 95% tagging accuracy even without normalization. j. besters-dilger (2021) applied neural network tagger clstm to the old russian žitie evfimija velikogo (gim, chud. 20), a copy of the second half of the 14 th century. the tagger was successfully applied on non-normalised text with high accuracyhowever, unknown words (which means those that had not been "seen" by the tagger before) still showed a higher error rate. ## ongoing work the next step in our effort is the development of a tagset and annotation principles. at this point, we have decided to keep morphology oriented marking, with some additions that can be beneficial for the further mark-up levels. we have decided to keep propn for proper names, and to mark all verbal forms as verb (as in the dioptra corpus) and all pronouns as pron. adverbials including pronominal adverbials will be marked as adv while the pronominal adjectives formed with an adjectival suffix (as in въсѣкъ 'each', оногози 'that', etc.) are marked as adj. the forms with the article-like suffixes (such as жената 'woman.def' and цр҃ ꙋтомꙋ 'king.def') will be marked as definite forms. after correcting the results for pos, we expect to achieve better results with the morphological annotation but also with other texts that will be included in the database.