Using Modern Languages to Parse Ancient Ones: a Test on O ld E nglish (2023)

Abstract: In this paper we test the parsing performances of a multilingual parser on Old English data using different sets of languages, alone and combined with the target language, to train the models. We compare the results obtained by the models and we analyze more in deep the annotation of some peculiar syntactic constructions of the target language, providing plausible linguistic explanations of the errors made even by the best performing models.

## introduction the performance of dependency parsing models for high-resource languages (hrls) has improved significantly in recent years due to the availability of large annotated corpora and the advancement of deep learning techniques. among others, models such as stanza @xcite and ud-pipe @xcite can achieve very high accuracy, with f1 scores approaching or even exceeding 0.90 on some treebanks datasets. this is true for some models for parsing data of (both modern and ancient) languages that have plenty of annotated resources, upon which is possible to train the models, while dependency parsing of low-resource languages (lrls) is more problematic. the challenges that dependency parsing for lrls has to face can be summarized as follows: a) data scarcity: lrls often have limited annotated text corpora, which makes it difficult to train high-quality models and b) transfer learning limitations: transfer learning approaches that rely on models pre-trained on hrls may not work well for lrls due to the language-specificity of syntactic constructions. as thoroughly discussed in section 2.1, for what concerns dependency parsing, old english (henceforth oe) can be considered a lrl, since the amount of annotated data available for this historical variety is scarce. given these premises, we attempted an automatic parsing of oe, starting from the automatic conversion of the york-toronto-helsinki parsed corpus of old english prose foot_0 (henceforth ycoe) into a conllu file, in which, however, the annotation is restricted to the sole morphological features retrievable from the ycoe annotation. taking this as a starting point, we manually annotated 292 sentences, following the standards of universal dependencies (de @xcite . then we tested the results obtained training uuparser v2.4 @xcite on data coming from our set of annotated sentences in oe and a set of treebanks of three related languages, following meechan-maddon and @xcite methodology, also followed by @xcite to test the performances of crosslingual transfer learning for parsing latin. the paper is structured as follows: in section 2 we introduce old english providing a brief description of its history, developments, and typological features. in addition, we provide a brief survey of the main available resources for this language and introduce some issues that an automatic parsing of oe may face. in section 3 we present our data and methodology. in section 4 we overview the results of the parsing of oe data and discuss them. finally, section 5 concludes the paper and summarizes our findings. ## old english old english is a west-germanic language, classified with old frisian and old saxon among the socalled ingvaeonic languages. it was the language spoken in england after angles, saxons, jutes and frisians came to britain and settled in the island in the 5 th century. it is attested from the 7 th century, except for some older brief runic inscriptions, whereas its ending point is conventionally established in 1066, date of the norman conquest of england @xcite . typologically, oe shows a nominativeaccusative alignment. like other indo-european languages, oe is a fusional language with inflectional word classes. nouns are inflected by number and case, and follow three inflectional classes, depending on their original proto-germanic stem. after some merging processes, only four of the eight original indo-european cases are found in oe: nominative, accusative, genitive, and dative. some traces of the instrumental are present, but residual. depending on the class, different cases can show syncretism. as other germanic languages, oe has two main conjugational system: the so-called strong and weak verbs, the former building the preterit by means of apophony, i.e. the vowel alternation found in present-day english (pde) irregular verbs, the latter with a dental suffix, just as pde regular verb, whose past form is constructed with the -ed suffix. finite oe verbs inflect for mood (indicative, subjunctive, imperative), tense (present and past), number, and person. some forms show syncretism, in particular the plural in all moods and tenses, and the first and third person singular in the subjunctive @xcite . although some regularities may be found, word order in oe is not as rigid as in pde @xcite , and it is still debated whether the basic word order was (s)vo or (s)ov. like other ancient and modern germanic languages, oe also exhibits v2, i.e. the tendency of the finite verb to follow the first constituent, regardless of its type. concerning the order of other constituents, nouns are generally preceded by modifiers, e.g. demonstratives, adjectives, genitive complements. however the latter can follow the noun if another preceding modifier is present. in pps, adpositions tend to precede a noun, but generally follow a pronoun; however, the opposite is also attested @xcite . contrary to pde, oe allowed discontinuous constituents, above all in relative constructions. ## results and discussion considering the metrics, the best-performing models were the ones trained with icelandic and oe data, which achieved the best unlabeled attachment score (uas) and labeled attachment score (las), together with the one trained with icelandic, german and oe data, which achieved the best label accuracy (la). 8 considering the uas and the las achieved by the models, it is surprising to notice that the model that performed best was the one trained upon only icelandic and target language data, since the icelandic monolingual model was the one which obtained significantly worse results than the other monolingual models. for what concerns the la, it seems reasonable to see a model trained on german data performing better that the others considering that the monolingual model trained upon german was the one that achieved the best scores among the monolingual models trained without oe data, even though such multilingual model was trained also upon icelandic data. finally, all models trained including the target language data achieved better results than their counterparts trained without having the target language data in the training set, even though the best performances are achieved combining icelandic and german with oe data. this seems reasonable in light of what discussed in section 3.2. in the following sections we will analyze more in detail the output of the parsing phase of the two models which scored the highest metrics (is+target and de+is+target) and the monolingual model trained only upon oe data. we will focus on the deprels advmod and obl for the following reasons: the former showed unexpectedly low results for the oe model (as shown in ## conclusion in this paper we tested the dependency parsing performances of four monolingual models and seven multilingual models on old english data. we showed that the model trained just using data of the target language achieved far better results than the models (both monolingual and multiliguals) trained without target language data and that, out of the three support languages we selected, icelandic and german combined better than swedish according to the scores reached parsing oe test data. as discussed in 3.2, we expected this result given the fact that modern icelandic and modern german retained many morphosyntactic features similar to those of the target language. then, we also discussed some cases of problem-atic annotation: in sections 4.1, 4.2, 4.3 we gave some linguistic explanations of the errors made by the best models, which include advmod, obl and acl:relcl showing that some poor results might be due to the peculiarity of such constructions in oe. finally, in 4.4, we discussed the impact which the correction of the dependency relation annotation using some rules based on the word forms, the universal parts-of-speech and the language-specific parts-of-speech had on the results achieved by the best models. this errors might have been avoided if we had used the option to force the models to learn embeddings for the parts-of-speech during the training phase, which would have made the parsing process aware of the already annotated parts-ofspeech. the situation in which the poss are annotated, though, is not so usual for oe, except for the above-mentioned ycoe and ycoep treebanks. our test, following the methodology described in meechan-maddon and @xcite , led to the same conclusions in terms of the benefits that support languages have on the parsing scores when combined to oe data during the training phase. in particular this is true when the support languages are related to the target language or, at least, share a significant number of features with the target language. this approach has proven useful for our broader twofold aim: a) having an alternative to a rulebased conversion of the ycoe(p) treebanks and b) developing a tool to annotate other oe texts, which are not included in the above-mentioned treebanks. despite the challenges, using this approach to parse historical languages can accelerate the process of creating new resources and produce outputs that, while not perfect, are satisfactory in terms of dependency parsing.