Word Vectors in the Eighteenth Century

This talk explores how new vector-based approaches to computational semantics both afford new methods to digital humanities research, and raise interesting questions for eighteenth-century literary studies in particular. New semantic models known as “word embedding models” have generated excitement recently in the natural language processing and machine learning communities, due to their ability to represent and predict semantic relationships as complex as analogy. “Man” is to “woman” as “king” is to what? , one can ask of the model; “queen,” it will most likely reply. These models formulate analogical and other semantic relationships by computing mathematical vectors for words, such that, if V(x) denotes the vector for the word x, then the above analogy can be expressed as V(woman) – V(man) + V(king) ≈ V(queen). Although these models have a longer history– vector space semantics dates from the ‘70s, having been first developed for the SMART information retrieval system (Salton, 1971) by Gerard Salton and his colleagues (Salton et al, 1975)” (Turney and Pantel, 2010)– new innovations in their speed and accuracy (see Note [1]) have renewed researchers’ interests—a development begun, in part, by Google, when researchers there unveiled newly efficient algorithms in 2013, packaged in software they released called word2vec. (The word2vec algorithm was originally described by Mikolov et al, 2013. It introduced the neural network to vector space semantics, providing an efficient means by which to compute word vectors. The GloVe algorithm from the Stanford NLP Group eschews the neural network approach, instead performing a novel method of dimensionality reduction on word collocation counts). “Word vectors,” as these new methods are sometimes informally called, have already enabled published research into questions relevant to humanistic research, such as a recent landmark paper from researchers in the Stanford NLP Group into patterns of semantic change across centuries of discourse (Hamilton et al). However, unfortunately, word vectors have so far rarely appeared in research from the digital humanities community itself. Moreover, what work that does exist has so far been primarily circulated through blogs, rather than through published proceedings or articles. Ben Schmidt, for instance, has written an influential introduction to word vectors in his blog post “Vector Space Models for the Digital Humanities” (2015a), which also includes a documented R package for computing them. Also notable is his post, “Rejecting the gender binary” (Schmidt, 2015b), which uses word vectors to dissect the polysemy of words; as well as Michael Gavin’s post, “The Arithmetic of Concepts” (2015), which explores the conceptual implications of adding and subtracting word vectors. On the whole, the current research landscape of word vectors in the digital humanities resembles the landscape of topic modeling years ago, when the original LDA algorithm (published in 2003 [Blei et al]), before appearing in landmark published DH studies such as Matt Jockers’ Macroanalysis (2013), was employed for humanistic research as early as 2006 by researchers working outside or tangentially to the digital humanities (Newman and Block). Given this scarcity of digital-humanities research on word vectors, work that seeks equally to explain, interpret, and demonstrate their potential seems particularly useful. With these goals in mind, this paper attempts first to unpack for a digital-humanities audience how word vectors work, with reference to the canonical analogy cited above: “man is to woman as king is to queen. ” Second, in order to interpret word vectors’ conceptual implications for eighteenth-century literature, I move away from this canonical analogy to one central to a particularly influential argument in the period: “Learning is to Genius as Riches are to Virtue. ” Lastly, I turn from this close reading of word vectors to methods of distant-reading analogies that lie implicit in eighteenth-century literature.