BERT in Plutarch’s Shadows (2022)

Abstract: The extensive surviving corpus of the ancient scholar Plutarch of Chaeronea (ca. 45-120 CE) also contains several texts which, according to current scholarly opinion, did not originate with him and are therefore attributed to an anonymous author Pseudo-Plutarch. These include, in particular, the work Placita Philosophorum (Quotations and Opinions of the Ancient Philosophers), which is extremely important for the history of ancient philosophy. Little is known about the identity of that anonymous author and its relation to other authors from the same period. This paper presents a BERT language model for Ancient Greek. The model discovers previously unknown statistical properties relevant to these literary, philosophical, and historical problems and can shed new light on this authorship question. In particular, the Placita Philosophorum, together with one of the other Pseudo-Plutarch texts, shows similarities with the texts written by authors from an Alexandrian context (2nd/3rd century CE).

## introduction authorship attribution through some form of statistical inference dates at least half of a century back, when mosteller and wallace used statistics of short, frequent words to estimate authorship of federalist papers, disputed by @xcite . the physicist @xcite was the first who systematically developed the methods that relied on statistical patterns like word or sentence length that have been found to be able to distinguish between different authors. while undeniably successful in many cases, they do not require any deeper understanding of the texts in question and therefore have their natural limitations. for a detailed review of various authorship attribution methods developed further, we refer the reader to @xcite . these methods rely on statistical patterns like word or sentence length that have been found to be able to distinguish between different authors. while undeniably successful in many cases, they do not require any deeper understanding of the texts in question and therefore have their natural limitations. transformer artificial neural networks have shown spectacular success in machine translation and answering search queries by internally extracting, after extensive pretraining, abstract patterns, and long-range dependencies in human language samples through encoder hierarchies @xcite . therefore, it seems natural to apply such schemes to other tasks that traditionally depended on human language understanding. thus, this paper uses versions of bert to investigate questions of authorship in ancient greek literature. such models have already been used for authorship attribution, but the actual number of cases is still limited. @xcite demonstrate that fine-tuning of a pretrained bert language model with an additional dense layer and a softmax activation allows performing authorship classification. @xcite use bert for author profiling in social media and conclude that despite encouraging results in terms of reliability, the computational power required for running such a model is too demanding for the task. these results show that transformers could be successfully used for authorship attribution, yet the application of these models to historical texts is still limited. specifically, bamman and burns (2020) develop bert for latin and @xcite train an lstm language model of ancient greek. there is also a char-bert implementation foot_0 , but we do not know of any public full bert model trained to work with ancient greek. we are also unaware of any examples when bert was used successfully for authorship attribution of historical documents. for this paper, we focus on plutarch of chaeronea (ca. 45-120 ce), a greek philosopher and biographer. his parallel biographies (one greek and one roman) and his philosophicalethical writings, which he wrote in the tradition of plato, intending to establish a coherent system, have been widely read in ancient and modern times. this paper addresses the authorship attribution of three manuscripts attributed in antiquity to plutarch: "de fluviis", "de musica", and "placita philosophorum" (abstracts and quotations from the now lost works of the ancient philosophers or schools). classicists argue that these three texts were not written by plutarch himself, yet the actual author(s) of these manuscripts is/are not known, and in particular, the question of authorship of the placita philosophorum has been widely discussed @xcite @xcite . so far, no decisive philological proof could resolve this question, and therefore evidence achieved through modern language processing techniques constitutes a valuable addition to this debate. the contributions of this paper are as follows: the data were taken from the digital history projects at the chair of ancient history of the university of leipzig. the plutarch texts were trans-formed into a digital representation under professional supervision. data can be obtained from the following free repositories perseus digital library foot_1 and first thousand years of greek foot_2 as part of open greek and latin foot_3 . the resulting representation was stored in xml format (tei guidelines) and enriched with metadata. the xml structure and metadata were removed. the strings were transferred into lowercase letters. the diacritics were removed. we did not touch hyphenation, punctuation, or any multilingual remains, nor did we apply any special language-related transformations. the resulting data set consists of 1 244 documents with 199 809 paragraphs or 14 373 311 words. ## ancient greek bert the resulting amount of data was too small to train ancient greek bert from scratch. however, data sets of smaller sizes are routinely used for transfer learning and fine-tuning of transformers. thus, we suggest obtaining bert for ancient greek via transfer learning on a masked language modelling (mlm) task. one could either use multilingual bert foot_4 or greek bert foot_5 as a starting model for knowledge transfer. the resulting model could then further be fine-tuned for the task of authorship attribution in ancient greek. ## conclusion this paper develops bert for ancient greek. it demonstrates that modern greek bert after transfer learning via mlm on ancient greek texts could be further fine-tuned as an authorship attribution classifier for ancient texts. the validation accuracy of authorship attribution is shown to be 80%. the model is then used to analyze text attributed to pseudo-plutarch. it shows that three documents have distinctly different statistical properties, and while de musica and de fluviis might originate with the same author, placita philosophorum has a different authorship and regional profile. thus, the classification of authorship allows the search for the authors of the 3 works gathered under pseudo-plutarch to be narrowed down to the 1st-3rd century ce, suggesting that at least one of the authors may have come from the vicinity of alexandria