Lemmatisation & Morphological Analysis of Unedited G reek: Do Simple Tasks Need Complex Solutions? (2025)

Abstract: Fine-tuning transformer-based models for part-of-speech tagging of unedited Greek text has outperformed traditional systems. However, when applied to lemmatisation or morphological analysis, fine-tuning has not yet achieved competitive results. This paper explores various approaches to combine morphological features to both reduce label complexity and enhance multi-task training. Specifically, we group three nominal features into a single label, and combine the three most distinctive features of verbs into another unified label. These combined labels are used to fine-tune DBBERT, a BERT model pre-trained on both ancient and modern Greek. Additionally, we experiment with joint training – both among these labels and in combination with POS tagging – within a multi-task framework to improve performance by transferring parameters. To evaluate our models, we use a manually annotated gold standard from the Database of Byzantine Book Epigrams. Our results show a nearly 9 pp. improvement, demonstrating that multi-task learning is a promising approach for linguistic annotation in less standardised corpora.

## introduction the development of automatic linguistic annotation for ancient greek began in the 1970s, initially driven by pedagogical needs @xcite , and soon expanded to accommodate the language's diverse dialects @xcite . since then, researchers in natural language processing (nlp) have continuously improved automatic annotation systems, addressing the limitations of earlier approaches. why, then, develop yet another tagger more than thirty years after the first? the answer lies in the evolving nature and availability of the data. most existing systems have been optimised for standardised ancient greek -texts that reflect critical editions rather than original, unedited sources. however, the increasing availability of 'imperfect' or fragmentary texts introduces new challenges that traditional taggers struggle to handle. example 1 illustrates some of these challenges: example 1a presents the diplomatic transcription of a book epigram from manuscript vat. gr. 169, whereas example 1b shows the edited, readable version of that same epigram provided by the database of byzantine book epigrams (dbbe) @xcite . even without knowledge of ancient greek, it is clear that the orthography of example 1a deviates from the standardised version in example 1b. 1 a second challenge in example 1 is the irregular use of diacritics, which are essential for disambiguating tokens with identical orthography. however, in example 1a, these diacritics appear inconsistently, making disambiguation more difficult. a final challenge concerns corpus uniformity. the dbbe, for instance, expands manuscript abbreviations and, when necessary, applies the correct diacritics to the expanded parts. this raises broader questions about how other datasets handle abbreviations and how automatic transcription systems approach this issue. the rise of automatic systems for optical character recognition (ocr) and hand-written text recognition (htr) has led to a growing influx of unedited greek texts, where existing annotation systems generally fail to perform reliably. to address the challenges presented above, we move beyond minor tweaks and reconsider how linguistic annotation can be approached. first, we examine previous methods and their limitations, then turn to multi-task learning to assess its potential for improving annotation quality. building on recent research in part-of-speech tagging for unedited greek texts @xcite , we extend the annotation process by enriching part-ofspeech tags with lemma information and relevant morphosyntactic features. these include case, gender, and number for nominal tokens, as well as voice, mood, and tense for verbal tokens. through targeted experiments, we evaluate whether combining features improves accuracy and where the approach still struggles. finally, we reflect on the broader implications for computational philology and suggest directions for future work, including the potential of multi-task learning to fine-tune a transformer model pretrained on greek. ## literature review unlike raw-text corpora, linguistically annotated resources for ancient greek remain sparse. the largest annotated corpus, thesaurus linguae graecae @xcite , is subscription-based and not available for research. however, two comprehensive annotated corpora have recently been released: the glaux corpus @xcite and the opera graeca adnotata (oga) @xcite . the glaux corpus consists of 20m tokens, spanning from the 8 th c. bc. to the 4 th c. ad. it is primarily annotated automatically with part-of-speech tags, morphological features, lemmas, and syntactic information, while a smaller subset has been manually annotated, using data from several smaller treebanks: proiel @xcite , the ancient greek dependency treebanks @xcite , pedalion trees @xcite , gorman trees (gorman, 2020), harrington trees foot_2 and the treebank of aphthonius' progymnasmata foot_3 . the oga corpus, in contrast, contains 40m tokens, all annotated automatically. early systems for ancient greek linguistic annotation were rule-and dictionary-based taggers @xcite . while still widely used, these approaches face persistent issues: they do not resolve ambiguous tokens, fail to process out-of-vocabulary tokens, and lack mechanisms to handle crasis (the contraction of two words into one). the emergence of transformer-based language models @xcite has addressed two of these challenges by incorporating contextual information when tagging tokens. however, crasis remains problematic for automatic solutions. @xcite introduced the first transformerbased language model for ancient greek, ancient greek bert, which was initialised using a modern greek bert model (koutsikakis et al., 2020) and subsequently trained on monolingual data from the first1kgreek project foot_4 , the perseus digital library foot_5 , the proiel treebank and gorman's treebank. since the base model was pretrained on modern greek, the fine-tuned model for ancient greek can only handle tokens without diacritics. nonetheless, it achieves state-of-the-art results for part-of-speech tagging and lemmatisation. swaelens et al. (2023) pretrained a bert model on greek texts ranging from homeric poetry (800 bc.) to byzantine literature and modern greek wikipedia, then fine-tuned it for partof-speech tagging and lemmatisation, with a focus on unedited greek texts. while its part-of-speech tagging performance was competitive to existing taggers, its lemmatisation results were unsatisfactory @xcite . similarly, @xcite demonstrated that transformer-based morphological annotation requires further refinement, as its performance has yet to reach competitive levels. a comparative study evaluating stateof-the-art models for morphosyntactic annotation and lemmatisation @xcite found that the dithrax model yielded best results for morphosyntactic annotation, while greta, a multilingual llm @xcite , performed best on lemmatisation. this multilingual transformer model trained on greek, latin, and english was fine-tuned for multiple downstream tasks, including part-of-speech tagging and lemmatisation @xcite . in this paper, we investigate a multi-task learning approach to predict part-of-speech, lemma and fine-grained morphological analysis. multi-task learning in machine learning is inspired by human learning, where knowledge transfers between related tasks @xcite . assigning part-of-speech tags, morphological features, and lemmas are highly interdependent tasks, making them well-suited for experimentation with multitask learning. ## data we fine-tuned dbbert, a bert model trained for classification tasks of ancient greek. this language model is pretrained on a dataset of 127.413.536 tokens which spans nearly 3,000 years, ranging from homeric poetry to medieval literature and modern greek wikipedia data (cf. supra); this language model is described in greater detail by @xcite . the data used for finetuning is extracted from the pedalion trees. from the xml-files, we extracted the attributes part-ofspeech tag (e.g. v-pppanm-) and lemma for each token. this part-of-speech tag is the combination of nine slots: part-of-speech, person, number, tense, mood, voice, gender, case, and degree. an example sentence is shown in token lemma postag ταῦτα οὗτος p-p---naδέ δέ d-------εἶπε λέγω v3saia--πρός πρός r-------ἀπάτην ἀπάτη n-s---fa-. ## methodology building on previous attempts to achieve competitive results for both automatic morphological analysis and lemmatisation, we conducted two categories of experiments: single-task and multi-task learning. the single-task experiments differ from previous approaches in that they neither predict all features as a single label, nor treat each feature separately. initially, all nine morphological features were combined into a single label as illustrated in ## results this section presents the quantitative and qualitative evaluation results for the lemmatisation (section 5.1) and morphological analysis (section 5.2) tasks. ## conclusion this study examined the potential of multi-task learning for the linguistic annotation of unedited greek texts, focusing on part-of-speech tagging, lemmatisation, and morphological analysis. by introducing a novel approach that groups nominal and verbal features into combined labels, and finetuning a transformer-based language model within both single-and multi-task frameworks, we demonstrated significant improvements in tagging accuracy. our results suggest that multi-task learning enhances feature prediction, particularly in challenging cases involving dialectal and orthographic variation. despite these advances, challenges remain present. while verbal feature tagging achieved a very low error rate, nominal feature tagging still exhibits inconsistencies, particularly in handling ambiguous case forms and uninflected words. additionally, lemmatisation errors persist due to orthographic variation and closely related cognates. addressing these challenges may require further label design, incorporation of external lexical resources, or integrating post-processing techniques. future research should explore extending this approach by integration of character-level models or phonological representations, which may improve the system's ability to process orthographic inconsistencies more effectively. as digital access to non-standardised greek texts expands, improving automatic annotation techniques will be crucial for facilitating linguistic analysis and broader applications in computational philology. furthermore, we will evaluate this approach to latin book epigrams, to test whether this methodology is transferable across (ancient) languages.