Profiling of Intertextuality in L atin Literature Using Word Embeddings (2021)

Abstract: Identifying intertextual relationships between authors is of central importance to the study of literature. We report an empirical analysis of intertextuality in classical Latin literature using word embedding models. To enable quantitative evaluation of intertextual search methods, we curate a new dataset of 945 known parallels drawn from traditional scholarship on Latin epic poetry. We train an optimized word2vec model on a large corpus of lemmatized Latin, which achieves state-of-the-art performance for synonym detection and outperforms a widely used lexical method for intertextual search. We then demonstrate that training embeddings on very small corpora can capture salient aspects of literary style and apply this approach to replicate a previous intertextual study of the Roman historian Livy, which relied on hand-crafted stylometric features. Our results advance the development of core computational resources for a major premodern language and highlight a productive avenue for cross-disciplinary collaboration between the study of literature and NLP.

## introduction in "lonesome day blues," bob dylan sings, "i'm gonna spare the defeated...i am goin' to teach peace to the conquered / i'm gonna tame the proud." this lyric echoes a passage from vergil's ancient latin epic, the aeneid, as translated by allen mandelbaum: "to teach the ways of peace to those you conquer, / to spare defeated peoples, tame the proud" @xcite . such allusions or "intertexts" transmit ideas across space and time, diverse media, and languages. although researchers focus on those intertextual connections felt to have special literary significance for the works at hand, in principle intertextuality refers to any verbal or semantic resemblance within the literary system, ranging from direct quotation to topical similarities @xcite . given the importance of intertextual criticism to literary study, computational identification of text reuse in literature is an active area of research @xcite . classical latin literature is a highly influential tradition characterized by an extraordinary density of allusions and other forms of text reuse @xcite . the most widely used tools for the detection of latin intertextuality, such as tesserae and diogenes, rely on lexical matching of repeated words or phrases @xcite @xcite . in addition to these core methods, other research has explored the use of sequence alignment @xcite , semantic matching @xcite , and hybrid approaches @xcite for latin intertextual search, complementing related work on english @xcite @xcite . much nlp research on historical text reuse, including previous applications of latin word embeddings, has focused on the bible and other religious texts @xcite @xcite . as such, there is a clear need for enhanced computational methods for classical latin literature. we describe the optimization of word embedding models for latin and their application to longstanding questions about literary intertextuality. ## evaluation and optimization of word embedding models for latin as is typical for many low-resource and premodern languages, development of core nlp technologies for latin remains at an early stage. following attempts to train word2vec models on unlemmatized corpora of latin literature shortly after the method's introduction @xcite and inclusion of latin in large-scale multilingual releases of fasttext and bert @xcite , in the past year there has been increased interest in systematic optimization and evaluation of latin embeddings. spurred by the recent evalatin challenge @xcite , a number of latin models have been trained for use in lemmatization and part-ofspeech tagging @xcite @xcite , complementing new literary applications to biblical text reuse and neo-latin philosophy @xcite . in addition, @xcite recently introduced a synonym selection dataset, based on the toefl benchmark for english, which they used to evaluate word2vec and fasttext models trained on the lasla corpus of latin literature. to the best of our knowledge, there have been no attempts to compare the performance of these models on standard evaluation tasks. to establish a baseline for further language-specific optimization and to inform our research on intertextuality, we evaluate five latin models for which pretrained embeddings are publicly available. these models encompass a variety of training corpora and methods, including word2vec, fasttext, and nonce2vec (appendix). we consider two tasks involving synonym matching. the first is the selection task introduced by @xcite ; the task is to distinguish the true synonym of a latin word from three distractors (@xmath0, 759). the second task, which is modeled on one of the english evaluation datasets from @xcite , involves unrestricted search for the synonyms of 1,910 words found in an online dictionary of latin near-synonyms (appendix). in addition, we train word2vec embeddings on a large corpus of latin compiled from the internet archive @xcite , which we first lemmatize using either the classical language toolkit (johnson, 2021) or treetagger @xcite . the results of the comparative evaluation are summarized in ## construction of benchmark intertextuality dataset despite the enormous number of latin intertextual parallels recorded in the scholarship, computational research on literary text reuse is hampered by a lack of benchmark datasets. existing benchmarks tend to focus either on binary comparisons, such as between vergil and lucan @xcite , or on specialized forms of religious intertextuality @xcite . to enable validation testing of general nlp methods for intertextual search, we assemble a new benchmark dataset based on valerius flaccus' argonautica, an epic poem dating from the 1st century c.e. which recounts the myth of jason and the argonauts. for book 1 of the argonautica we record 945 verbal intertexts with four major epics (vergil's aeneid, ovid's metamorphoses, lucan's pharsalia, and statius' thebaid) that are noted in the commentaries of spaltenstein (2002), kleywegt (2005), and zissos (2008). our dataset thus contains a substantial number of intertexts of established literary interest with coverage across book 1. ## conclusions we present an empirical analysis of latin intertextuality using word embedding models. in addition to its specific contributions to literary criticism and the digital humanities, our work makes several methodological advances of interest to the broader nlp community. we conduct a comparative evaluation of latin word embedding models for two synonym matching tasks and report an optimized model that achieves state-of-the-art performance, agr de or mur de rep gal cat iug vitr inst 1 ger ann conf ps lucr g hf theb 0.0 0.2 0.4 0.6 0.8 1.0 fraction classified as livian republican post-republican verse word embeddings stylometric features which we apply to intertextual search of latin poetry. by capturing similarities other than exact repetition of words and phrases, our method complements existing search tools, such as diogenes and tesserae. given the diversity and complexity of references employed by latin authors, taking a multifaceted approach is essential to the computational study of latin intertextuality. although our initial work focuses on static embeddings, one potential avenue for improving our search method would be to leverage context-aware embeddings such as multilingual or latin bert @xcite . in addition, we illustrate how intertextuality can be studied using anomaly detection, and we replicate previous stylometric research about the roman historian livy, which was informed by domain knowledge, using an unsupervised approach. we hope that this work will strengthen cross-disciplinary collaboration between classics, the digital humanities, and nlp.