Logion: Machine-Learning Based Detection and Correction of Textual Errors in G reek Philology (2023)

Abstract: We present statistical and machine-learning based techniques for detecting and correcting errors in text and apply them to the challenge of textual corruption in Greek philology. Most ancient Greek texts reach us through a long process of copying, in relay, from earlier manuscripts (now lost). In this process of textual transmission, copying errors tend to accrue. After training a BERT model on the largest premodern Greek dataset used for this purpose to date, we identify and correct previously undetected errors made by scribes in the process of textual transmission, in what is, to our knowledge, the first successful identification of such errors via machine learning. The premodern Greek BERT model we train is available for use at https://huggingface.co/cabrooks/LOGION-base.

## introduction ancient texts have been passed down by scribes over hundreds of years, in a process known as textual transmission. scribes occasionally make mistakes, some of which lie undiscovered to this day. as unchecked errors have the potential to change the meaning of a text, finding and correcting scribal errors is a central aim in greek philology. in a proof-of-concept paper, we presented the first scribal mistakes detected by contextual language models (graziosi et al., 2023). in this paper, we describe and study the approaches used to arrive at those results and evaluate the algorithm's effectiveness on artificially generated errors. prior to graziosi et al. (2023), to the best of our knowledge, scribal errors were found only by hand-that is, with domain experts carefully reading the text until they find potential errors, and then using database searches to assess textual problems and propose solutions. these errors include simplifying difficult expressions, omissions, replacing one word for another with a similar sound, shape, or function, etc. discovery of such errors typically requires a sophisticated understanding of an author's writing tendencies and the context of a particular text. this motivates the use of contextual language models for the detection of scribal errors. in this paper, we propose logion, a framework for detecting scribal errors based on contextual language models. foot_0 logion consists of three stages: in the first stage, a contextual language model learns conditional word distributions for a selected corpus; in the second stage, potential errors are identified according to statistics derived from the learned distribution; lastly, in the third stage, corrections are proposed for the words identified as potentially erroneous. while not all words flagged by the algorithm will be genuine scribal errors, a "shortlist" of potential scribal errors can point philologists to previously undetected errors which, after being corrected, restore the original meaning. to summarize, our main contributions are as follows: in this paper, we concentrate on the discovery of scribal errors in the works of the byzantine author michael psellus, who is a convenient choice at a proof-of-concept stage for philological reasons. however, we remark that these methods may be applied to any premodern text passed down by scribes, provided sufficient data is available. ## methodology logion is a three-stage framework for the discovery and emendation of textual errors in a given corpus. @xcite the initial stage involves training a bert model, which undergoes broad pre-training on premodern greek text followed by subsequent fine-tuning on specific works of interest, as outlined in subsection 3.1. the second and third stages harness the learned distributions of the fine-tuned bert to detect and emend errors, respectively. before describing the later stages in full, we briefly recount the conditional distributions which bert learns. given a sequence of tokens w 1 , . . . , w n , consider a single token w i and denote the surrounding (bidirectional) context by w -@xmath0, . . . , w i-1 , w i+1 , . . . , w n ). from the masked-language model (mlm) training task, the model learns the distribution in the second and third stages, described in subsection 3.2 and subsection 3.3, respectively, existing statistical theory is applied to the learned distribution p to determine the tokens which are most likely to contain errors, and subsequently to propose emendations. the stages are illustrated together in figure 1 . ## experiments in this section, we study the effectiveness of the proposed approach at finding artificially generated errors, while noting that the proposed approach has already resulted in the discovery of real errors, as outlined in graziosi et al. (2023). a sample of that work is shown in section 5. ## philologically significant results the metrics presented here have successfully identified errors that were previously undetected, ranging from scribal errors in the manuscripts, typographical errors in printed editions, and errors caused by digitization. these findings underwent philological peer review and have been published in tapa, the research journal of the society for classical studies. in graziosi et al. (2023), we show at proofof-concept stage how the approaches introduced here improve on previous knowledge of premodern greek texts by identifying and sometimes solving several different philological problems. for detailed examples and further discussion, please see graziosi et al. (2023). below, we offer a single example to illustrate one type of error which may be detected (in this case a misreading of the manuscript on the part of modern scholars rather than an actual error in the manuscript itself). in psellus's hist. brev. at lines 81.89-90, aerts' edition reads: οὗτος δὶς βασιλεύσας ηὔχετο καὶ τρὶς καὶ τετράκις• ἦ δὲ γάρ, φησι, μετὰ νέφος ὁ ἥλιος. houtos dis basileusas ēucheto kai tris kai tetrakis: ē de gar, fēsi, meta nephos, ho hēlios. 'this man, having been king twice, prayed for a third and fourth term. for indeed, he said, there is sun after clouds.' when thresholding for confidence and scribal distance, the token δε was one of the lowest chance tokens in the test set. the algorithm output depicted in figure 4 and the subsequent examination of the manuscript on which this edition is based led to the realization that the manuscript actually reports "ἡδὺ", not "ἦ δὲ". the sentence can now be translated as follows: "this man, having been king twice, prayed for a third and fourth term. for, he said, 'sun after clouds is sweet'." in this case, then, the error turned out to be not in the early manuscript but in subsequent readings of it. ## future work one major line of future work concerns developing an application which is adopted by domain experts and used to assist their work. given any text, such an application would be capable of generating shortlists of suspected errors and proposed emendations for review. future research directions in this area include developing efficient and linguistically motivated sub-word tokenization schemes and the capability to include or exclude sections of the dataset from consideration at inference time: this is relevant when one is interested in performing error detection on a section of text which was included in the training set without retraining the model entirely. in working towards the latter goal, one promising architecture is demix, which enables dynamic expert mixtures at inference time (gururangan et al., 2021). another idea for future work, and one which sets scribal error detection apart from traditional error detection, concerns treating scribal modifications as diffusion processes. as scribal errors are often contextually driven, text altered by scribes may paradoxically evaluate to having higher probability than the original text. @xcite on this view, then, the text evolves over time as a diffusion process with a transition kernel derived from p (for example, one option is to model the trajectory of the text by gibbs sampling according @xcite in philology, this is the principle known as lectio difficilior potior. because "the normal tendency is to simplify, to trivialize, to eliminate the unfamiliar word or construction," the more difficult reading (i.e., lectio difficilior) should in some circumstances be taken to be the authentic one (west, 1973). 7 figure 4 : algorithm output that led to the discovery of a scribal error in the words η δε. top line: words are given a grayscale color according to their ccr, as in equation 2; the word δε was flagged because it obtained the smallest ccr of all words in its given context (the surrounding 512 tokens, not all of which are pictured here). below top line: algorithm-generated suggestions, given a grayscale color according to their likelihood. in each case, the algorithm suggests merging two words by deleting the space before δε. the third suggestion, ηδυ, is, in fact, transmitted in the relevant manuscript and must be what was originally written by the author (graziosi et al., 2023). the small probability awarded here reflects the complexity of scribal errors. some are trivial, including the ones we generate artificially; others, including this one, are harder to emend. to the conditionals p(w i |w -i )). diffusion models are designed to recover original data from diffused data, so it may be fruitful to apply such models for recovery of original text from scribally-modified text (sohl-dickstein et al., 2015). while not itself a diffusion model, electra is a promising architecture for such future work (clark et al., 2020). ## conclusion in this study, we have trained a bert model to support philological work on premodern greek texts: in particular, we use statistical and machinelearning-based approaches to identify scribal errors that accrue in the process of textual transmission and to propose emendations. in a broader sense, this research aims to contribute to the future of philology, understood as a discipline concerned with preserving, elucidating, and making publicly accessible the global archive of premodern texts. some of what we have presented here is of relevance also for authors and languages we have not considered, as well as for modern text editing in general.