Graecia capta ferum victorem cepit. Detecting L atin Allusions to A ncient G reek Literature (2023)

Abstract: Intertextual allusions hold a pivotal role in Classical Philology, with Latin authors frequently referencing Ancient Greek texts. Until now, the automatic identification of these intertextual references has been constrained to monolingual approaches, seeking parallels solely within Latin or Greek texts. In this study, we introduce SPhilBERTa, a trilingual Sentence-RoBERTa model tailored for Classical Philology, which excels at cross-lingual semantic comprehension and identification of identical sentences across Ancient Greek, Latin, and English. We generate new training data by automatically translating English into Ancient Greek texts. Further, we present a case study, demonstrating SPhilBERTa’s capability to facilitate automated detection of intertextual parallels. Intertextual allusions hold a pivotal role in Classical Philology, with Latin authors frequently referencing Ancient Greek texts. Until now, the automatic identification of these intertextual references has been constrained to monolingual approaches, seeking parallels solely within Latin or Greek texts. In this study, we introduce SPhilBERTa, a trilingual Sentence-RoBERTa model tailored for Classical Philology, which excels at cross-lingual semantic comprehension and identification of identical sentences across Ancient Greek, Latin, and English. We generate new training data by automatically translating English into Ancient Greek texts. Further, we present a case study, demonstrating SPhilBERTa’s capability to facilitate automated detection of intertextual parallels.

## introduction the study of intertextuality and allusions to literary sources has a longstanding tradition in classical philology, highlighting complex interconnections between different literary works. during the 1960s, the concept of intertextuality was shaped by a comprehensive theoretical framework developed by scholars such as julia kristeva, ferdinand de saussure, and michail bakhtin. the term "intertextuality" itself was introduced by kristeva during this pivotal era @xcite @xcite . intertextuality proves particularly crucial when examining roman literature's relationship with ancient greek texts. many latin authors consciously mirrored elements of greek classics, making intertextuality an essential concept for understanding this cultural literary exchange. 1 the importance of intertextuality, especially given the considerable attention it has received, is beyond dispute. while there exists a plethora of theoretical work exploring specific forms of intertextuality, our focus in this work is on the general occurrence of textual resemblances, specifically within latin and greek texts. traditionally, the identification of such parallels has largely relied on scholars' close reading. however, recent years have seen the development of statistical nlp tools -driven especially by the tesserae project @xcite at the forefront of this movement -that are able to automatically uncover a considerable number of textual parallels. these approaches, however, typically rely on string-level parallels and are grounded in carefully designed rules and scoring functions. notably, these systems are generally restricted to detecting parallels in the same language, as they rely on identifying identical tokens or stems. recently, the breakthrough in self-supervised training of powerful pre-trained language models (plms) has also led to a surge of diverse plms for classical philology @xcite @xcite @xcite . in fact, two recent case studies in @xcite and @xcite have shown that contextualized embeddings produced by such models can indeed identify texts bearing similar content. while a rigorous quantitative evaluation of these findings still remains to be conducted, the perceived potential of using these models for finding intertextual relations is clearly sparking widespread interest. however, research into modern language analysis tasks has demonstrated that sentence embeddings derived solely from standalone bertor roberta-based models generate suboptimal and inefficient embeddings. this insight led to the creation of sentence-bert (sbert) models @xcite . among the latest language models introduced in the field of classical philology is philberta (riemenschneider and frank, 2023), a robertabased model pre-trained on ancient greek, latin, and english language data. building upon this model, we present sphilberta, a model tailored to the discovery of intertextual parallels across latin, ancient greek, and english texts. in this work, our goal is to move away from systems relying on hand-crafted rules, and instead to employ state-of-the-art tools for identifying intertextual relations that are easy to adapt to a wide variety of languages from classical philology and beyond. most importantly, we probe the feasibility of uncovering intertextual parallels across languages, an area that has been largely neglected in the automatic identification of intertextual allusions until this point. this novel capability will considerably enlargen the space for new findings, by being able to compare texts directly across languages. we show that sphilberta is proficient in recognizing direct translations of sentences in ancient greek, latin, and english, thereby demonstrating comprehensive cross-lingual competence. applying our model directly to texts of philological significance not only underlines its practical applicability but also highlights areas for improvement, suggesting promising avenues for future exploration. in summary, our contributions are as follows: existing research has explored matching words or stems @xcite as well as methods that focus on semantics @xcite . additionally, techniques that combine both lexical and semantic elements have been examined, where semantic understanding is established through word embeddings @xcite or via the (ancient greek) wordnet @xcite . while the majority of preceding studies have concentrated on detecting text reuse in the bible and various religious texts, @xcite focus on classical latin literature. however, to our knowledge, no efforts have been undertaken to automatically detect intertextual similarities across languages, specifically between greek, latin, and english texts. this lack is likely due to the inherent complications of inducing cross-language mappings, a difficulty that arises both with surface form-based strategies and with techniques utilizing word embeddings. notwithstanding, this gap is of significant importance, as it overlooks the frequent appearance of such allusions, especially from latin to greek literature. language models for classical philology. bamman and burns (2020) and mercelis and keersmaekers (2022) introduced latin bert and electra models, respectively. for ancient greek, @xcite and @xcite provided bert models, initialized from modern greek bert and subsequently trained on ancient greek data. similarly, the ugarit project has successfully explored the usage of the xlm-r model @xcite for ancient @xcite , even though xlm-r has not been pre-trained on ancient greek texts. recently, riemenschneider and frank (2023) have complemented the encoder-only landscape with encoder-decoder models and developed trilingual models using ancient greek, latin, and english texts. moreover, @xcite and @xcite have developed odycy and la-tincy, respectively, as dedicated spacy pipelines foot_2 for ancient greek and latin. sbert embeddings. @xcite have shown that vanilla bert embeddings are not suitable for creating sentence embeddings, and instead proposed the s(entence)-bert models, which are based on siamese and triplet network structures. building on their work, reimers and gurevych (2020) introduced a method to learn multilingual sentence embeddings via multilingual knowledge distillation. this method realizes knowledge transfer from a monolingual teacher model to a student model, by training the student model to align the original sentence and its translation to the same location in the embedding space. ## methodology we closely follow reimers and gurevych's (2020) multilingual knowledge distillation recipe. their method requires a monolingual teacher model m and parallel sentences in the given source language and the target language(s) ((s 1 , t 1 ), ..., (s n , t n )). the teacher trains a student model m such that m (s i ) ≈ m (s i ) and m (t i ) ≈ m (s i ). for a given mini-batch b, the mean-squared loss is minimized: in other words, the student model is trained to map a given sentence to the same vector across languages, i.e., the translation of a given sentence should be mapped to the same vector as the source sentence. notably, this method is not restricted to a bilingual setup. instead, the student can be trained to map sentence vectors stemming from multiple languages to the same vector, namely the one provided by the teacher model. in our work, the teacher and student sbert models to be used for cross-lingual knowledge transfer will be initialized from strong transformer language models for the respective languages. for the english teacher model, we build on the mp-net model of @xcite , an encoder-only model that has been pre-trained using a combination of masked language modeling and permuted language modeling. specifically, we use different sentence transformer variants induced from mp-net, as provided by the sbert library @xcite . for the student model, we experiment with initializing it from different multilingual models: xlm-r @xcite , a multilingual model based on roberta that covers 100 languages, including modern greek and latin, in contrast to philberta (riemenschneider and frank, 2023), a recent trilingual model that has been pre-trained on ancient greek, latin, and english texts. more detail about our models and the specific experimental setup is provided in section 5. ## parallel data as outlined in section 3, the knowledge distillation method of reimers and gurevych (2020) crucially depends on the availability of parallel sentences between the relevant source and target languageshere, the source language english for the teacher model, and english, ancient greek, and latin for our student model. we collect this data from various sources: from the perseus digital library, foot_3 from parallel bible data, foot_4 parallel english-to-greek sentences from the opus corpus @xcite , and an extensive collection of parallel english and latin sentences available on the huggingface hub. 6 we refer to the latter dataset as "rosenthal", named after its associated account. the perseus project features a large collection of ancient greek and latin texts, many of them with corresponding translations. however, the alignment of the provided data is not always fine-grained enough for our purpose. therefore, we align individual lines with their corresponding translation, and discard lines that we cannot align successfully. to generate additional parallel data for enhanced knowledge transfer, we experiment with translating the english portions of the rosenthal dataset, ## experiments our first aim is to compare different model configurations. we test the following configurations: we use the all--mpnet-base-v2 8 and the multi--qa-mpnet-base-dot-v1 9 model from the sbert library (reimers and gurevych, 2019) as teacher models. while the former is fine-tuned on a variety of tasks, the latter is optimized for semantic search. in order to transparently evaluate our models, we first measure their ability to correctly detect translations of a sentence. for each parallel dataset, we hold out 1 000 sentences as test sets. given a query, i.e., the embedding of a specific sentence in the source language, we compute the cosine similarity to the embeddings of all 1 000 sentences in the target language. following reimers and gurevych (2020), we measure the success of our models by determining translation accuracy: we count a translation to be correctly identified if the model computes the highest cosine similarity between the query and its correct translation, and vice versa. this evaluates the student model's ability to align a source language sentence with an equivalent target language sentence. however, our primary interest is whether the model can effectively link ancient greek and latin texts. regrettably, the volume of parallel data available in ancient greek and latin is severely constrained. consequently, we utilize bible data, which is accessible in ancient greek, latin, and english. again, we examine the model's performance on 1 000 test sentences, given in ancient greek or latin. we ensure that the model has not encountered any of these sentences in its training data, either in english or latin, or in ancient greek. in addition, we use the philta @xmath0. we are aware that the task of identifying intertextual allusions poses a much greater challenge than merely recognizing translations, as allusions typically exhibit more subtlety and may extend beyond sentence or verse boundaries. however, we consider this evaluation a transparent method for comparing the effectiveness of different model configurations and an approximate measure to evaluate the potential success of our models in identifying intertextual allusions across languages. experiment details. we train all models with the exact same configurations. we fine-tune all models for 30 epochs, using a batch size of 32, the adamw optimizer with a learning rate of 2e-5, and 10 000 warmup-steps. the best-performing model is selected based on the translation accuracy derived from a total of 2 000 held-out validation examples, comprised of 1 000 english-greek and 1 000 english-latin sentence pairs. ## results we present our results for the different configurations in tables 2 to 4 . specifically, we evaluate: i) the performance of different teacher models (the more general all-mpnet-base-v2 sbert model in comparison to the multi-qa-mpnet sbert fine-tuned for semantic search), ii) different student models (xlm-r versus the philberta model), and iii) augmenting the parallel data for training sphilberta using philta @xmath1. employing xlm-r as the student model leads to catastrophic performance. specifically, the model never surpasses the 1% mark in test set performance. we observed this trend consistently, regardless of the model configuration or the random seed employed. this outcome is, to some degree, to be expected, as xlm-r is not pre-trained on ancient greek data. still, it is surprising that xlm-r performs so badly also on latin data, as its pre-training corpus contained a latin portion. moreover, the ugarit project @xcite has successfully adapted xlm-r to ancient greek. we hypothesize that the effectiveness of a broadly multilingual but unspecialized model may be task-dependent, and continuing selfsupervised pre-training on ancient greek texts may be required for xlm-r to adapt adequately. these findings highlight the importance of initializing the student model with a model that is proficient in the target languages. initializing the student model with philberta yields strong performance, often surpassing 95% translation accuracy. generally, employing multi-qa-mpnet as a teacher model contributes to a slight performance improvement over all-mpnet-base-v2. yet, when testing the model on the ancient greek rosenthal corpus, using the multi-qa-mpnet teacher model results in a performance decline. importantly, the greek part of this dataset has been translated by philta @xmath2, which could possibly have affected the quality of the dataset. indeed, while we see this negative trend when testing on the generated data, the inclusion of the philta-generated ancient greek rosenthal corpus as additional training data leads to a notable enhancement for the greek datasets, while the performance for latin translation retrieval remains largely unaffected. the results for latin-to-greek and greek-to-latin translations are shown in ## case study: the aeneid and homer's odyssey examinations of the intertextual allusions in virgil's aeneid to both the iliad and the odyssey have a long history, dating back to antiquity. structurally, the aeneid's initial six books mirror the narrative of the odyssey, while the concluding six books correspond more closely to the iliad. in the second book of the aeneid, the protagonist aeneas attempts to escape from the ravaged city of troy with his family. tragically, his wife, creusa, is lost amidst the chaos. creusa's ghost consoles him and bids him goodbye before receding into thin air: "this speech uttered, while i wept and would have said many a thing, she left me and retreated into thin air. thrice there was i fain to lay mine arms round her neck; thrice the vision i vainly clasped fled out of my hands, even as the light breezes, or most like to fluttering sleep." to evaluate our model's proficiency in identifying these intertextual allusions, we employ each verse of the aeneid passage (i.e., 5 verses) as a query, which we then compare to the verse embeddings (approx. 11 000 verses) of the complete odyssey. we note that these verses do not share a direct one-to-one relationship and they are not translations of each other, the scenario in which our model was trained. even so, we observe that verse 793 ("thrice the vision i vainly clasped fled out of my hands") is correctly paired with the best corresponding greek verse ("and thrice she flitted from my arms like a shadow or a dream"). in the majority of cases, our model accurately captures crucial concepts, such as weeping, departing, triplicity, wind, and sleep, linking them reasonably to different verses. however, our verse-to-verse mapping, which precludes longer texts, results in a lack of a cohesive concept of extended intertextually alluding passages. still, our case study demonstrates the proficiency of our models in recognizing sentence structures and translating them to a different language (as in "this speech uttered" → "so she spoke"), and in identifying common topics or concepts across languages, even locating verses where multiple relevant concepts exist within the same verse ("thrice", "the vision", "out of my hands" → "thrice", "a shadow or a dream", "from my arms"). despite these remarkable results, our case study also reveals the need for a more sophisticated retrieval mechanism that extends beyond verse boundaries to search for broader patterns. yet, already in the present state, our sphilberta model can serve as a useful tool for automatic first-pass exploration of potential cross-lingual intertextual allusions, and in this way can support philologists in the search for intertextual allusions. ## conclusion we introduce sphilberta, a multilingual philberta-derived sentence transformer model, specifically adapted to classical philology. our model represents a pioneering effort in detecting intertextual allusions between ancient greek and latin texts, which is characteristic of many roman writers who used greek literature for juxtaposition. sphilberta displays impressive performance across various datasets, confidently identifying direct translations among english, latin, and ancient greek. we have illustrated that sphilberta holds strong potential in revealing intertextual allusions; however, additional research is needed to fully exploit the model's capabilities. our multilingual sphilberta and the similarity-driven retrieval settings built upon it offer, for the first time, the option to study intertextuality cross-lingually on a broader scale in original classical literature.