Dynamic embedded topic models and change-point detection for exploring literary-historical hypotheses (2024)

Abstract: We present a novel combination of dynamic embedded topic models and change-point detection to explore diachronic change of lexical semantic modality in classical and early Christian Latin. We demonstrate several methods for finding and characterizing patterns in the output, and relating them to traditional scholarship in Comparative Literature and Classics. This simple approach to unsupervised models of semantic change can be applied to any suitable corpus, and we conclude with future directions and refinements aiming to allow noisier, less-curated materials to meet that threshold.

## introduction characterizing and interpreting linguistic novelty has a long tradition in both humanistic scholarship. a foundational study in comparative literature @xcite hinges empirically on shifts in the meanings of particular words, most notably figura. he claims that figura went from particularly abstract (as a translation of plato's schema), to concrete (due in part to several particularly novel authors), and finally was usable in both senses in the writing of early church fathers. we refer to this type of semantic shift as bimodality, the degree to which a word makes a sharp transition between having one or two senses. 1 in our attempts to reproduce and extend this humanistic hypothesis, we make the following contributions: our goals and methods in this paper have connections to research tracking semantics across time in embedding spaces @xcite , which we differ from in our focus on modality rather than the geometric position. also related is the longstanding task of word sense disambiguation (wsd) @xcite , which we differ from by not operating with a gold standard sense inventory to target, or other supervised task. the dynamic embedded topic model (detm) @xcite extends topic models @xcite to operate over word embeddings, and capture topic evolution over time. change-point detection @xcite considers the problem of determining if and where the distribution generating a sequence of observations changes, typically w.r.t. time. the simplest approach, employed here, uses dynamic programming to find the optimal piecewise-linear fit to the observations. the jensen-shannon divergence (jsd) @xcite is a symmetric distance measure based on the kullback-leibler divergence. the perseus project @xcite is a longstanding database and interface to a curated corpus of primary sources from the classical and early medieval world. ## materials and methods 2 we derive our corpus from the perseus project by extracting all xml documents from the underlying repositories, and extract all text marked as latin along with the name of the purported author. we then manually assign years to the set of unique authors based on rough scholarly consensus, and keep materials that fall between 250 bce and 500 ce. this leads to a corpus of 574 documents from 101 authors. we use the classical language toolkit @xcite to lemmatize each token and filter non-latin vocabulary. we group documents into 75-year windows, and split documents into sub-documents of at most 500 tokens. we initialize a 50-topic detm with skip-gram embeddings @xcite trained on the corpus. ## future work deeper scrutiny of model output, involving scholars from classics, would benefit from in situ examples drawn automatically from the underlying sources. having established its ability to surface historically distinctive authors and vocabulary, we are augmenting the pipeline in this direction, in anticipation of implementing a frontend for humanists to apply and explore their own diachronic corpora. a critical facet of auerbach's arguments is the permeability and comingling of the languages, and a suitable greek lexicon and lemmatizer would make it a straightforward to include prior and contemporary greek writing. the greatest barrier to extending our methodology to arbitrary languages and time periods is imperfect and low-coverage data. in parallel with this research we have applied the same method to a noisy latin corpus derived from the hathitrust (hathitrust foundation, 2023), which offers considerably higher coverage of sources. unfortunately the level of noise (from ocr, commentary, etc) and redundancy renders it challenging to use credibly without extensive post-processing. we plan to use this research as a case study for developing a core set of methods for gathering, deduplicating, and rectifying an arbitrary htc-based corpus such that it approaches the fidelity of a manuallycurated resource like perseus.