Venue: NAACL 2021

# Profiling of Intertextuality in Latin Literature Using Word Embeddings

Patrick J. Burns,\({}^{1}\) James A. Brofos,\({}^{2}\) Kyle Li,\({}^{3}\) Pramit Chaudhuri,\({}^{1}\) Joseph P. Dexter\({}^{4}\dagger\)

\({}^{1}\) Department of Classics, University of Texas at Austin

\({}^{2}\) Department of Statistics and Data Science, Yale University

\({}^{3}\) Department of Computer Science, Columbia University

\({}^{4}\) Data Science Initiative and Department of Human Evolutionary Biology, Harvard University

\({}^{\dagger}\) Corresponding author: jdexter@fas.harvard.edu

###### Abstract

Identifying intertextual relationships between authors is of central importance to the study of literature. We report an empirical analysis of intertextuality in classical Latin literature using word embedding models. To enable quantitative evaluation of intertextual search methods, we curate a new dataset of 945 known parallels drawn from traditional scholarship on Latin epic poetry. We train an optimized word2vec model on a large corpus of lemmatized Latin, which achieves state-of-the-art performance for synonym detection and outperforms a widely used lexical method for interstellar search. We then demonstrate that training embeddings on very small corpora can capture salient aspects of literary style and apply this approach to replicate a previous intertextual study of the Roman historian Livy, which relied on hand-crafted stylometric features. Our results advance the development of core computational resources for a major premodern language and highlight a productive avenue for cross-disciplinary collaboration between the study of literature and NLP. 1

Footnote 1: Code and data are available at [https://github.com/QuantitativeCriticismsLab/NAACL-HLT-20](https://github.com/QuantitativeCriticismsLab/NAACL-HLT-20) 21-Latin-Intertextuality.

## 1 Introduction

In "Lonesome Day Blues," Bob Dylan sings, "I'm gonna spare the defeated...I am goin' to teach peace to the conquered / I'm gonna tame the proud." This lyric echoes a passage from Vergil's ancient Latin epic, the _Aeneid_, as translated by Allen Mandelbaum: "to teach the ways of peace to those you conquer, / to spare defeated peoples, tame the proud" Thomas [20]. Such allusions or "intertexts" transmit ideas across space and time, diverse media, and languages. Although researchers focus on those intertextual connections felt to have special literary significance for the works at hand, in principle intertextuality refers to any verbal or semantic resemblance within the literary system, ranging from direct quotation to topical similarities Kristeva [18, 7]. Given the importance of intertextual criticism to literary study, computational identification of text reuse in literature is an active area of research Bamman and Crane [6, 1].

Classical Latin literature is a highly influential tradition characterized by an extraordinary density of allusions and other forms of text reuse Hinds [19]. The most widely used tools for the detection of Latin intertextuality, such as Tesserae and Diogenes, rely on lexical matching of repeated words or phrases Coffee et al. [20, 12, 13]. In addition to these core methods, other research has explored the use of sequence alignment Chaudhuri et al. [21, 1], semantic matching Scheirer et al. [16], and hybrid approaches Moritz et al. [15, 17] for Latin intertextual search, complementing related work on English Smith et al. [20, 21]. Much NLP research on historical text reuse, including previous applications of Latin word embeddings, has focused on the Bible and other religious texts Lee [20, 16, 1]. As such, there is a clear need for enhanced computational methods for classical Latin literature. We describe the optimization of word embedding models for Latin and their application to longstanding questions about literary intertextuality.

## 2 Evaluation and optimization of word embedding models for Latin

As is typical for many low-resource and premoform languages, development of core NLP technologies for Latin remains at an early stage. Following attempts to train word2vec models on unlemmatized corpora of Latin literature shortly after the method's introduction Bamman; Bjerva and Praet [

2015) and inclusion of Latin in large-scale multilingual releases of FastText and BERT (Grave et al., 2018; Devlin et al., 2019), in the past year there has been increased interest in systematic optimization and evaluation of Latin embeddings. Spurred by the recent EvaLatin challenge (Sprugnoli et al., 2020), a number of Latin models have been trained for use in lemmatization and part-of-speech tagging (Bacon, 2020; Celano, 2020; Straka and Strakova, 2020; Stoeckel et al., 2020), complementing new literary applications to Biblical text reuse and neo-Latin philosophy (Manjavacas et al., 2019; Bloem et al., 2020). In addition, Sprugnoli et al. (2019) recently introduced a synonym selection dataset, based on the TOEFL benchmark for English, which they used to evaluate word2vec and FastText models trained on the LASLA corpus of Latin literature.

To the best of our knowledge, there have been no attempts to compare the performance of these models on standard evaluation tasks. To establish a baseline for further language-specific optimization and to inform our research on intertextuality, we evaluate five Latin models for which pretrained embeddings are publicly available. These models encompass a variety of training corpora and methods, including word2vec, FastText, and nonce2vec (Appendix). We consider two tasks involving synonym matching. The first is the selection task introduced by Sprugnoli et al. (2019); the task is to distinguish the true synonym of a Latin word from three distractors (\(N=2,759\)). The second task, which is modeled on one of the English evaluation datasets from Mikolov et al. (2013), involves unrestricted search for the synonyms of 1,910 words found in an online dictionary of Latin near-synonyms (Appendix). In addition, we train word2vec embeddings on a large corpus of Latin compiled from the Internet Archive (Bamman and Crane, 2011; Bamman and Smith, 2012), which we first lemmatize using either the Classical Language Toolkit (Johnson, 2021) or TreeTagger (Schmid, 1994).

The results of the comparative evaluation are summarized in Table 1. For the synonym search task, we consider the number of correct matches found in the top 1, 10, and 25 results by cosine similarity, as well as the mean reciprocal rank (MRR). We find that our models achieve state-of-the-art performance on both tasks compared to the five published models. The improvement in performance may be due to the combination of training on lemmatized text, which Sprugnoli et al. (2019) identified as an important optimization for Latin, and use of a lower-quality but much larger training corpus (1.38 billion tokens, compared to 1.7 million tokens in the curated LASLA corpus).

## 3 Construction of benchmark

intertextuality datasetDespite the enormous number of Latin interstellar parallels recorded in the scholarship, computational research on literary text reuse is hampered by a lack of benchmark datasets. Existing benchmarks tend to focus either on binary comparisons, such as between Vergil and Lucan (Coffee et al., 2012), or on specialized forms of religious intertextuality (Moritz et al., 2016; Manjavacas et al., 2019). To enable validation testing of general NLP methods for intertextual search, we assemble a new benchmark dataset based on Valerius Flaccus' _Argonautica_, an epic poem dating from the 1st century C.E. which recounts the myth of Jason and the Argonauts. For Book 1 of the _Argonautica_ we record 945 verbal intertexts with four major epics (Vergil's _Aeneid_, Ovid's _Metamorphoses_, Lucan's _Pharsalia_, and Satius' _Thebaid_) that are noted in the commentaries of Spaltenstein (2002), Kleywegt (2005), and Zissos (2008). Our dataset thus contains a substantial number of intertexts of established literary interest with coverage across Book 1.

## 4 Analysis of intertextuality in Latin literature

### Enhanced intertextual search

Several widely used computational search methods for Latin intertextuality rely on lexical matching of related words. We present an alternative approach in which potential intertextual phrases are ranked using word embeddings.

According to this method, we compare a bigram of interest to all bigrams in another text subject to the constraint that the distance between the words does not exceed a fixed interval. The interval parameter is determined by the number of words occurring between the words comprising the bigram of interest and is usually, but not exclusively, between 0 and 2. The choice of bigrams as the basic unit conforms to ancient poetic practice, in which allusive phrases frequently consist of two words (although they can also be single words or longer phrases), and hence also conforms to modern intertextual search methods such as Tesserea (Coffee 

et al., 2012, 2013). A key difference in our approach, however, is that bigram pairs may share only one or even zero words in common. The bigrams are drawn from the dataset of commentators' annotations; in cases where commentators note only a single-word intertext or a phrase longer than a bigram, we supplement or select words on a case-by-case basis, giving preference to those words that bear a semantic or syntactic similarity to one or more words in the interact.

The similarity score for a bigram pair is calculated by taking the cosine similarities of the embeddings of the four possible pairs of words across both bigrams, and averaging the highest cosine similarity and the score for the remaining pair of words. The bigram pair _flammifero Olympo_ ("fiery Olympus") and _flammifera nocte_ ("firery night"), for example, generates the four lemmatized pairs _flammifer_\(\sim\)_flammifer_, _flammifer_\(\sim\)_nox_, _Olympus_\(\sim\)_flammifer_, and _Olympus_\(\sim\)_nox_. Hence, the similarity score for the bigram pair is the average of 1.0 for the exact match, _flammifer_\(\sim\)_flammifer_, and 0.35 for the other remaining word pair, _Olympus_\(\sim\)_nox_ (i.e., 0.67). In this way, the similarity score for an intertext noted by the commentators is ranked against all other bigrams in the relevant text, the size of which we set at a single book of poetry (i.e., equivalent to the text on which the dataset is based). Although the choice to use one unit of text rather than another is somewhat arbitrary-one could consider complete works rather than constituent books, for example-the use of single books has several advantages, notably provision of a large but not overwhelming number of comparison phrases while maintaining ancient textual units with distinct episodes and themes.

Following this approach, we compute a ranking for each of the 945 parallels in the Valerius Flaccus benchmark. For embeddings we use our word2vec model trained on CLTK-lemmatized text, which by MRR performs best in the synonym ranking task (Table 1). The precision\(@k\) and recall\(@k\) for \(k=1\), 3, 5, 10, 25, 50, 75, 100, and 250 are summarized in Fig. 1.

We next compare our method and the Tesserae search tool, which is regarded as state-of-the-art for Latininterxtual search Bernstein et al. (2015); Forstall and Scheirer (2019). Using their public web-based interface, we run Tesserae searches comparing Book 1 of the _Argonautica_ with each of the four texts in the benchmark dataset. Tesserae produces lists of repeated bigrams ranked according to a hand-crafted scoring formula that considers the rareness and proximity of the words in each bigram. For the complete set of Tesserae results, the recall is 33.9%, and the precision is 0.97%; with \(k=250\), our method achieves a comparable precision (1.4%) but higher recall (82.4%). An important advantage of the Tessera tool, however, is that it searches for similar phrases in parallel and does not require a list of specific queries as input. As such, the results aggregated for this c

Figure 1: Precision and recall for Latin intertextual search using an optimized word2vec model.

 

smaller number of Tesserae searches than the 945 embedding-based searches we run. For this reason, Tesserea is likely to be more suitable than our method for applications in which the user does not have predetermined phrases of interest.

A minority of intertexts in the dataset contain no shared lemma and hence present a challenge for existing detection methods based on lexical matching but are recoverable using our search method. The phrases _e clausis [antris]_ ("from the enclosed [cave]," _Arg._ 1.417) and _circum cloastra [fremunt]_ ("[they roar] around the gate," _Aen._ 1.56), for example, contain no words in common but have similar syntax (the prepositions \(e\) and _circum_) and semantics (words indicating enclosure). Similarly, the phrases _Phlegethontis operti_ ("hidden Phlegethon," _Arg._ 1.735) and _Acherontis aperti_ ("open Acheron," _Theb._ 11.150) both refer to rivers in the underworld and contain near-antonymic adjectives. Word embeddings can thus be used to identify intertexts of literary interest in a way that complements existing methods.

### Anomaly detection

Computational analysis of literary intertextuality is typically treated as an information retrieval problem, as in the previous section. Here we consider an alternative framework of studying intertextuality through anomaly detection (Forstall et al., 2011). For this approach, we train word embeddings on highly restricted corpora, so that the resulting models capture aspects of authorial style. We use those restricted embeddings as features to predict instances of similarity between authors, which can indicate intertextuality. To illustrate this approach we describe a case study involving Latin historiography and the development of prose style.

In particular, we examine patterns of stylistic influence between the Roman historian Livy, his source material, and other Latin prose literature. As assessment of similarities in literary style is inherently subjective, we consider the task of replicating two experiments from a previous computational study of Livy, which employed a hand-crafted set of Latin stylometric features such as syntactic markers and function words, using word embeddings. Our approach to evaluation of a subjective task is thus similar to that of Bamman et al. (2014), who tested a set of preregistered hypotheses about literary characters.

Like most historical writing, Livy's monumental history of Rome drew on a wide range of source material, such as earlier historiography and political speeches, most of which is no longer extant. The extent to which Livy cited these earlier sources, and their influence on Livy's compositional practice, remain important open questions for ancient historians. Dexter et al. (2017) demonstrated previously that anomaly detection could be used to distinguish a database of 439 putative citational passages from the remainder of Livy. To replicate this analysis, we train a word2vec model on all of Livy's surviving history and use the embeddings as input for a one-class support vector machine (SVM). Following Dexter et al. (2017), we set the detection rate of the one-class SVM to 20% and train on a random selection of 30,000 5-sentence passages of Livy. We find that the one-class SVM labels \(38.2\pm 0.8\)% of passages from the citation database as anomalous, compared to \(18.4\pm 2.0\)% of a validation set with 439 passages of general Livy (mean and standard deviation from \(N=3\) runs). These results provide further evidence that citational passages of Livy exhibit an anomalous writing style, whether due to source use or stylistic modulation, corroborating the earlier analysis.

Finally, we consider the stylistic similarity of Livy to 17 other works of Latin literature analyzed by Dexter et al. (2017). Again using a one-class SVM trained on Livy, we predict the "Livianess" of each work (Fig. 2). Our results confirm the major trends identified by the prior stylometric analysis, including the expected dissimilarity to Livy of the verse texts and the consistent similarity of contemporary and early imperial historiography. The primary difference between the two sets of results is that the stylometric features indicate greater similarity between Livy and non-historographical prose, such as Augustine's _Confessions_ and Vitruvius' _De architectura_, than do word embeddings, which may reflect a relative lack of shared diction.

## 5 Conclusions

We present an empirical analysis of Latin intertextuality using word embedding models. In addition to its specific contributions to literary criticism and the digital humanities, our work makes several methodological advances of interest to the broader NLP community. We conduct a comparative evaluation of Latin word embedding models for two synonym matching tasks and report an optimized model that achieves state-of-the-art performance, 

which we apply to intertextual search of Latin poetry. By capturing similarities other than exact repetition of words and phrases, our method complements existing search tools, such as Diogenes and Tesserae. Given the diversity and complexity of references employed by Latin authors, taking a multifaceted approach is essential to the computational study of Latin intertextuality. Although our initial work focuses on static embeddings, one potential avenue for improving our search method would be to leverage context-aware embeddings such as multilingual or Latin BERT (Devlin et al., 2019; Bamman and Burns, 2020). In addition, we illustrate how intertextuality can be studied using anomaly detection, and we replicate previous stylometric research about the Roman historian Livy, which was informed by domain knowledge, using an unsupervised approach. We hope that this work will strengthen cross-disciplinary collaboration between classics, the digital humanities, and NLP.

## Acknowledgements

This work was conducted under the auspices of the Quantitative Criticism Lab (www.qcrit.org), an interdisciplinary group co-directed by P.C. and J.P.D. and supported by an American Council of Learned Societies Digital Extension Grant and a National Endowment for the Humanities Digital Humanities Advancement Grant (Grant No. HAA-271822-20). P.C. was supported by a Mellon New Directions Fellowship, and J.P.D. by a Neukomhip and a Harvard Data Science Fellowship. The material contributed by J.A.B. is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No. 1752134. Any opinion, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. We thank Adriana Casarez, James Patterson, and Ariane Schwartz for assistance with compiling the Valerius Flaccus interextuality dataset, and Tommaso Spinelli for sharing the dictionary of Latin near-synonyms.

## References

* 1st Workshop on Language Technologies for Historical and Ancient Languages_, pages 111-113, Marseille. European Language Resources Association (ELRA).
* Bamman (11K Latin texts) David Bamman. 11K Latin texts.
* Bamman and Burns (2020) David Bamman and Patrick J. Burns. 2020. Latin BERT: A contextual language model for classical philology.
* Bamman and Crane (2008) David Bamman and Gregory Crane. 2008. The logic and discovery of textual illusion. In _Proceedings of the 2008 LREC Workshop on Language Technology for Cultural Heritage Data_.
* Bamman and Crane (2011) David Bamman and Gregory Crane. 2011. Measuring historical word sense variation. In _Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries_, page 1-10, New York, New York. Association for Computing Machinery.
* Bamman and Smith (2012) David Bamman and David Smith. 2012. Extracting two thousand years of Latin from a million book library. _Journal on Computing and Cultural Heritage_, 5.

Figure 2: Comparison of Livy to 17 other Latin works. The stylometric data is reprinted from Dexter et al. (2017), which used passages of 35 sentences each.

 

David Bamman, Ted Underwood, and Noah A. Smith. 2014. A Bayesian mixed effects model of literary character. In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 370-379, Baltimore, Maryland. Association for Computational Linguistics.
* Barbu and Trausan-Matu [2017] Miruna-Stefania Barbu and Stefan Trausan-Matu. 2017. Intertextuality detection in literary texts using word2vec models. In _2017 21st International Conference on System Theory, Control and Computing (ICSTCC)_, pages 262-265. Institute of Electrical and Electronics Engineers.
* Bernstein et al. [2015] Neil Bernstein, Kyle Gervais, and Wei Lin. 2015. Comparative rates of text reuse in classical Latin hexameter poetry. _Digital Humanities Quarterly_, 9.
* Bjerva and Praet [2015] Johannes Bjerva and Raf Praet. 2015. Word embeddings pointing the way for late antiquity. In _Proceedings of the 9th SIGHUM Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities (LaTeCH)_, pages 53-57, Beijing. Association for Computational Linguistics.
* the case of Cassiodorus.
* 1st Workshop on Language Technologies for Historical and Ancient Languages_, pages 84-93, Marseille. European Language Resources Association (ELRA).
* 1st Workshop on Language Technologies for Historical and Ancient Languages_, pages 119-123, Marseille. European Language Resources Association (ELRA).
* Chaudhuri and Dexter [2017] Pramit Chaudhuri and Joseph P. Dexter. 2017. Bioinformatics and classical literary study. _Journal of Data Mining & Digital Humanities_, Special Issue on Computer-Aided Processing of Intertextuality in Ancient Languages.
* Chaudhuri et al. [2015] Pramit Chaudhuri, Joseph P. Dexter, and Jorge A. Bonilla Lopez. 2015. Strings, triangles, and go-betweens: Interstellar approaches to Silius' Carthaginian debates. _Dicryma_, 12.
* Coffee et al. [2012] Neil Coffee, Jean-Pierre Koenig, Shakhti Poornima, Roeland Ossewaarde, Christopher Forstall, and Sarah Jacobson. 2012. Interpretability in the digital age. _Transactions of the American Philological Association_, 142:383-422.
* Coffee et al. [2013] Neil Coffee, Jean-Pierre Koenig, Shakhti Poornima, Roeland Ossewaarde, Christopher Forstall, and Sarah Jacobson. 2013. The Tesserae Project: Interval analysis of Latin poetry. _Literary and Linguistic Computing_, 28:221-228.
* Devlin et al. [2019] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
* Dexter et al. [2017] Joseph P. Dexter, Theodore Katz, Nilesh Tripuraneni, Tahagata Dasgupta, Ajay Kannan, James A. Brofos, Jorge A. Bonilla Lopez, Lea A. Schroeder, Adriana Casarez, Maxim Rabinovich, Ayelet Haimson Lushkov, and Primit Chaudhuri. 2017. Quantitative criticism of literary relationships. _Proceedings of the National Academy of Sciences_, 114:E3195-E3204.
* Forstall et al. [2011] Christopher W. Forstall, Sarah L. Jacobson, and Walter J. Scheirer. 2011. Evidence of intertextuality: investigating Paul the Deacon's _Angustae Vitae. Literancy and Linguistic Computing_, 26:285-296.
* Forstall and Scheirer [2019] Christopher W. Forstall and Walter J. Scheirer. 2019. _Quantitative Intertextuality: Analyzing the Markers of Information Reuse_. Springer International Publishing, Cham.
* Grave et al. [2018] Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018. Learning word vectors for 157 languages. In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_, pages 3483-3487, Miyazaki. European Language Resources Association (ELRA).
* Heslin [2019] Peter Heslin. 2019. Diogenes.
* Hinds [1998] Stephen E. Hinds. 1998. _Allusion and Intertext: Dynamics of Appropriation in Roman Poetry_. Cambridge University Press, Cambridge.
* Johnson [2021] Kyle P. Johnson. 2021. CLTK: The Classical Language Toolkit.
* Juvan [2009] Marko Juvan. 2009. _History and Poetics of Intertextuality_. Purdue University Press, West Lafayette, Indiana.
* Kleywegt [2005] A. J. Kleywegt. 2005. _Valerius Flaccus, Argonautica, Book I_. Mnemosyne, Bibliotheca Classica Batava. Supplementum. Brill, Leiden.
* Kristeva [1980] Julia Kristeva. 1980. Word, dialogue, and novel. In Leon S. Roudiez, editor, _Desire in Language: A Semiotic Approach to Literature and Art_, pages 64-91. Columbia University Press, New York, New York.
* Lee [2007] John Lee. 2007. A computational model of text reuse in ancient literary texts. In _Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics_, pages 472-479, Prague. Association for Computational Linguistics 

Enrique Manjavacas, Brian Long, and Mike Kestemont. 2019. On the feasibility of automated detection of allusive text reuse. In _Proceedings of the 3rd Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature_, pages 104-114, Minneapolis, Minnesota. Association for Computational Linguistics.
* Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space.
* Moritz et al. (2016) Maria Moritz, Andreas Wiederhold, Barbara Pavlek, Yuri Bizzoni, and Marco Buchler. 2016. Non-literal text reuse in historical texts: An approach to identify reuse transformations and its application to Bible reuse. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_, pages 1849-1859, Austin, Texas. Association for Computational Linguistics.
* Scheirer et al. (2016) Walter Scheirer, Christopher Forstall, and Neil Coffee. 2016. The sense of a connection: Automatic tracing of intertextuality by meaning. _Digital Scholarship in the Humanities_, 31:204-217.
* Schmid (1994) Helmut Schmid. 1994. Probabilistic part-of-speech tagging using decision trees. In _Proceedings of the International Conference on New Methods in Language Processing_, Manchester.
* Smith et al. (2014) David A. Smith, Ryan Cordell, Elizabeth Maddock Dillon, Nick Stramp, and John Wilkerson. 2014. Detecting and modeling local text reuse. In _Proceedings of the 14th ACM/IEEE-CS Joint Conference on Digital Libraries_, page 183-192. Institute of Electrical and Electronics Engineers.
* Spaltenstein (2002) Francois Spaltenstein. 2002. _Commentaire des "Argonautica" de Valerius Flaccus (livres 1 et 2)_. Ed. latomuns, Bruxelles.
* Ist Workshop on Language Technologies for Historical and Ancient Languages_, pages 105-110, Marseille. European Language Resources Association (ELRA).
* Sprugnoli et al. (2019) Rachele Sprugnoli, Marco Passarotti, and Giovanni Moretti. 2019. _Vir_ is to _modertatus_ as _mulier_ is to _interperans_: Lemma embeddings for Latin. In _Proceedings of the Sixth Italian Conference on Computational Linguistics_, Bari.
* Ist Workshop on Language Technologies for Historical and Ancient Languages_, pages 130-135, Marseille. European Language Resources Association (ELRA).
* Ist Workshop on Language Technologies for Historical and Ancient Languages_, pages 124-129, Marseille. European Language Resources Association (ELRA).
* Thomas (2012) Richard F. Thomas. 2012. The streets of Rome: The classical Dylan. In William Brockliss, Pramit Chaudhuri, Ayelet Haimson Lushkov, and Katherine Wasdin, editors, _Reception and the Classics: An Interdisciplinary Approach to the Classical tradition_, pages 134-159. Cambridge University Press, Cambridge.
* Zhang et al. (2014) Qi Zhang, Jihua Kang, Jin Qian, and Xuanjing Huang. 2014. Continuous word embeddings for detecting local text reuses at the semantic level. In _Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval_, page 797-806, New York, New York. Association for Computing Machinery.
* Zissos (2008) Andrew Zissos. 2008. _Valerius Flaccus' Argonautica Book I, Edited with Introduction, Translation, and Commentary_. Oxford University Press, Oxford.

 

Appendices
* [14] Links to published Latin word embedding models
* Bamman: [https://www.cs.cmu.edu/~dbamman/latin.html](https://www.cs.cmu.edu/~dbamman/latin.html)
* Grave et al. (2018): [https://fasttext.cc/docs/en/crawl-vectors.ht](https://fasttext.cc/docs/en/crawl-vectors.ht) ml Sprugnoli et al. (2019) (word2vec CBOW): ("allLASLAlemmi-vector-100-nocase-w5-CBOW.vec"), [https://embeddings.ilia-erc.eu/sampl](https://embeddings.ilia-erc.eu/sampl) es/download/word2vec
* Sprugnoli et al. (2019) (FastText SG): ("allLASLA-lemmi-fast-100-SKIP-win5-min5.vec"), [https://embeddings.lia-erc.eu/samples/download/fasttext](https://embeddings.lia-erc.eu/samples/download/fasttext)
* Bloem et al. (2020): ("bamman-c50-d100"), [https://github.com/bloemj/nonc](https://github.com/bloemj/nonc) e2vec/tree/nonce2vec-latin

### Training details for new Latin word2vec models

* word2vec (CLTK) is trained using Gensim word2vec with the following parameters: size = 300, min_count = 100, iter = 1. The model is trained on plaintext files from Bamman ([https://www.cs.cmu.edu/~dbamma](https://www.cs.cmu.edu/~dbamma) n/latin.html) lemmatized with CLTK BackoffLatinLemmatizer Johnson (2021).
* word2vec (TT) is trained using Gensim word2vec with the following parameters: size = 50, min_count = 100, iter = 1. The model is trained on plaintext files from Bamman ([https://www.cs.cmu.edu/~dbaman/latin.html](https://www.cs.cmu.edu/~dbaman/latin.html)) lemmatized with TreeTagger Schmid (1994).

### Links to publicly available datasets

* Synonym selection dataset from Sprugnoli et al. (2019): [https://embeddedings.lila-erc.eu/samples/syn/syn-s](https://embeddedings.lila-erc.eu/samples/syn/syn-s) election-benchmark-Latin.tsv
* Dictionary of Latin near-synonyms from Spinelli (2018): [https://github.com/tommasospinelli/Online-Dictionary-of-Latin-Near-Synonyms/blob/master/Latin%20Near-Synonyms%20dataset.txt](https://github.com/tommasospinelli/Online-Dictionary-of-Latin-Near-Synonyms/blob/master/Latin%20Near-Synonyms%20dataset.txt)
* Database of possible citational passages in Livy from Dexter et al. (2017): [https://github.com/qcrit/PNAS_2017_Qua](https://github.com/qcrit/PNAS_2017_Qua) nititativeCriticism/blob/master/Code/LivyPassages.csv

### Parameters for Tesserae searches

We run four Tesserae searches comparing Book 1 of Valarius Flaccus' _Argonautica_ to Vergil's _Aeneid_, Ovid's _Metamorphoses_, Lucan's _Pharsalia_, and Statius' _Thebaid_, respectively. For these searches, we use version 3 of the online Tessera tool (http s://tesserae.caset.buffalo.edu/) with all parameters set to their default values: unit = line, feature = lemma, number of stop words = 10, stoplist basis = corpus, score basis = word, frequency basis = corpus, maximum distance = 10 words, distance metric = frequency, drop scores below = 6.

### Latin texts

The texts of the Latin epic poems included in our intertextual search analysis are from the Tesserae corpus ( [https://github.com/tesserae/tree/master/texts](https://github.com/tesserae/tree/master/texts)), which is derived from the Perseus Digital Library ( [http://www.perseus.tufts.edu/hopper/](http://www.perseus.tufts.edu/hopper/)).

Following Dexter et al. (2017), we consider 17 texts for the stylistic analysis of Livy, abbreviated in Fig. 2 according to the _Oxford Classical Dictionary_: Agr, Cato's _De agri cultura_; Ann, Tacitus' _Annals_; Conf, Augustine's _Confessions_; De or, Cicero's _De ordore_; De rep, Cicero's _De republicia_; Cat, Sallus' _De coniuratione Catilinae_; G, Vergil's _Georgics_; Gal, Caesar's _Bellum Gallicum_; Ger, Tacitus' _Germania_; HF, Seneca's _Hercules Furens_; Inst 1, Quintimian's _Instituto Ontario_ 1; Iug, Sallust's _Bellum Iugurthinum_; Lucr, Lucretius' _De errum natura_; Mur, Cicero's _Pro Murenca_; Ps, Plautus' _Pseudolus_; Theb, Statius' _Thebaid_; Vitruvius' _De architectura_. Further information about this corpus is available in Dexter et al. (2017).