Machines in the media: semantic change in the lexicon of mechanization in 19th-century B ritish newspapers (2022)

Abstract: The industrialization process associated with the so-called Industrial Revolution in 19th-century Great Britain was a time of profound changes, including in the English lexicon. An important yet understudied phenomenon is the semantic shift in the lexicon of mechanisation. In this paper we present the first large-scale analysis of terms related to mechanization over the course of the 19th-century in English. We draw on a corpus of historical British newspapers comprising 4.6 billion tokens and train historical word embedding models. We test existing semantic change detection techniques and analyse the results in light of previous historical linguistic scholarship.

## introduction started in the 18 th century in great britain, the industrial mechanization saw a dramatic acceleration in the 19 th century. new machines were introduced in different industries at a rapid pace and the ever more pervasive automation of manufacture meant large-scale reorganization and movement of the workforce throughout the territory. this had profound repercussions on many aspects of daily life from a cultural, political, and social perspective. the english language, particularly its lexicon, used by 19 th -century sources to describe these changes reflected the same rapid pace at which the objects and the societal landscape had been shifting, making it an important, and yet understudied, research topic. previous studies on the english language in the 19 th century have focussed on how the changes observed in the lexicon of the period often reflect 'new interpretations given to older words in a time of changing societal values' in victorian britain @xcite , as in the shift in the usage of words describing men and women @xcite , or have highlighted the plethora of neologism and new loanwords introduced as a result of the industrial revolution @xcite . as @xcite also notes, meaning change (besides mere new word formations) 'is best illustrated from semantic fields relating to the new technologies that rapidly became part of everyday experience, such as the field of vehicles/transport/traffic'. in this paper we investigate the issue of tracing these subtle shifts at scale using computational methods. drawing from examples of lexical semantic change in 19 th -century english from previous literature, we train diachronic word embedding models on a very large collection (4.6 billion tokens) of digitized 19 th -century british newspaper articles. we then compare these data-driven analyses with previous qualitative studies, to verify the extent to which historical language models reflect expert knowledge. in addition to validating the computational models, we assess how these methods can be employed to answer new complex questions on the linguistic effects of mechanization and other historical events. using historical newspapers as a data source presents specific methodological challenges, and in particular historical (issues of representativeness, @xcite ) complexities. however, given the size of newspaper archives and the possibility to sample them by variables of interest (e.g. time period, political leaning, place of circulation or publication), these sources are a very good fit for large-scale analyses of lexical change in periods of on-going deep societal changes. this is also shown by the growing number of projects which use historical newspapers as sources for large-scale semantic processing and data-driven historical analysis, including news-eye, 1 translantis, 2 impresso, 3 and living with machines. 4 this work is the first to provide a large-scale analysis of the english lexicon of mechanisation in the 19 th century. from a methodological point of view, our dataset presents challenges that are shared by other historical newspaper archives and thus our research can inform similar studies on other languages. from the point of view of historical linguistics and historical research, we present the first study of the english lexicon of mechanisation that employs computational techniques, which allows us to compare automatically detected semantic changes with those identified by close-reading methods in previous literature. ## previous work according to @xcite and @xcite , the 19 th century was a pivotal period in the history of english, when its lexicon underwent a significant transformation in both spoken and written sources, although the academic literature has paid less attention to late modern english compared to other periods in the history of the english language @xcite . in recent years a number of nlp studies have proposed algorithms for the automatic detection of lexical semantic change from historical texts using word type and token embeddings @xcite . algorithms based on type embeddings have been shown to perform best in the 2020 semeval shared task @xcite and they typically consist of the following steps: the corpus of interest is divided into timedependent slices; then word embedding models are trained from each subcorpus and their spaces aligned. finally, the cosine similarity between a word's embedding in the first (or last) space and its embedding in each of the spaces is computed. if the similarity is below a predefined threshold (i.e. the two embeddings are sufficiently different), the word is marked as a potential candidate for semantic change. in few cases these algorithms have been applied in real-world digital humanities research: wevers and koolen (2020), for instance, present a study on word embeddings trained on a 500,000 digitized dutch newspaper corpus for the purpose 1 https://www.newseye.eu/ 2 https://translantis.wp.hum.uu.nl/ 3 https://impresso-project.ch/ 4 livingwithmachines.ac.uk of studying the evolution of concepts. ## data and methods two newspaper collections were used for this experiment. a selection of titles from the british library's heritage made digital digitization project, foot_0 comprising 12 titles and around 2.3 billion tokens, and a collection specifically digitized for the living with machines project, comprising 107 titles and also around 2.3 billion tokens. jointly, the collections span the period between 1801 and 1920. to prepare the corpora for training diachronic word embeddings, we first split them into time slices of 10 years each. we preprocessed the articles for each decade by removing word breaks resulting from ocr, newlines, and punctuation, by lowercasing the text and removing the stop words provided by the nltk library for english. 6 we trained word2vec @xcite ) models as implemented in the gensim library @xcite . to choose the optimal hyperparameters for training, we performed a grid search comparing the skip-gram and the continuous-bag-of-words architectures, as well as different number of epochs ({5,10}), vector dimensions ({200,300}), context windows ({3,5,10}) and minimum word counts ({1,5,10}). we evaluated the quality of the models resulting from all combinations of these parameters on one decade (all articles published between 1821 and 1830) calculating the cosine similarity between pairs of synonyms 7 in each model and choosing the model that returned the highest average score for all pairs. the final models were trained using the skip-gram architecture, 5 epochs, 200 dimensions, a context window of 3 and a minimum count of 1. since the models for each decade are trained independently, the resulting word vectors in different decades are not aligned along the same coordinate axes. to allow for comparison between the representation of the same word across different decades, we aligned the semantic spaces on the basis of the or-thogonal procrustes problem @xcite . given w (d) ∈ r n×m , denoting the matrix of the vectors in decade d, the orthogonal procrustes problem consists in finding the orthogonal matrix @xmath0. this is done by: we compiled a list of words drawing from those indicated by @xcite as having undergone semantic change at some point during the 19 th century. 8 for each word we calculated the cosine similarity between its vector in the semantic space for the most recent decade (the 1910s) and its vector in each of the previous decades. we followed @xcite , who found that comparing the embeddings to the last time period leads to better results in semantic change detection. we then analysed the resulting scores in the following way. any time point t with a cosine similarity significantly higher than the one in the time point t -1 was considered a potential changepoint in the meaning of a word. significant changepoints were detected using the pruned exact linear time (pelt) algorithm @xcite , a penalized-cost method for detecting multiple changepoints in time-series data. we ran the algorithm with a jump parameter of 1 and comparing results with penalty set to 0.25 and 0.5. 9 we then extracted the nearest neighbours of each word for each decade to establish what type of semantic change might have occurred at each potential changepoint. we evaluated the accuracy of the models at detecting semantic change for a word against its entry in the oxford english dictionary (oed). 10 using the oed api, 11 for each word we extracted the list of its senses, their definition and first record in writing, and selected all senses that had a first recorded year later than 1800 and earlier than 1920. to identify whether the detected potential changepoint for a word corresponded to one (or several) of its selected senses from the oed, we extracted the nearest neighbours of the word in each time period and compared those from the relevant decade(s) with the oed senses. we can immediately see that fellow, indicated by @xcite as having a stable semantics in the 19 th century, is included among the words in both tables. two changepoints were also detected by the model trained with the lower penalty for wheel, another word cited by @xcite as semantically stable. if we compare the trajectories of wheel and train (figure 1 ), for example, it is not surprising to see that a changepoint detection model trained with stricter parameters may detect a change for the latter but not for the former, even though the plot suggests that a change in usage, albeit more gradual, occurred for wheel as well. ## quality control since large digitized newspaper collections are frequently not created with a specific criterion in mind, but rather following specific policies of the digitizing institution, we needed to be particularly wary that the likely biassed content of our data (cf. @xcite would not significantly interfere with our research questions. the quality of our models and validity of our method were checked in several ways. first, to make sure that potential detected changepoints were not simply the result of a biassed dataset, we ran our changepoint detection method individually on all the words in the list of synonym pairs which were also used to optimize the embedding hyperparameters, since they were indicated by the oed as semantically stable throughout our period of interest. with a jump parameter of 1 and a penalty of 0.5 (the safer, stricter option), no changepoint was detected for any of the words, with the exception of identical, suggesting an overall good reliability for our models. second, throughout the analysis we used two external sources to validate our results. a history of the english language in the nineteenth century by @xcite , specifically its chapter on lexical change, was used to draw examples from the language of mechanization that the scholar indicated as having undergone some type of semantic change. we also included words which he mentioned as seeming semantically stable throughout the century (namely fellow and wheel, the former not in the lexical field of mechanization) as a further form of comparison with non-digital scholarship on the subject. finally, throughout the analysis we employed the oed as a benchmark to check whether a changepoint coincided with a newly recorded senses, as well as to identify definition of new senses and usages, especially in highly polysemous or ambiguous contexts. ## conclusions in this paper we presented a first attempt at a largescale computational study of semantic change of terms related to the lexical field of mechanisation in 19th-century english. our main goal was to find out whether vector space models trained on very large (4.6b tokens) digitized, hence noisy, his-torical newspapers were able to stand the test of expert knowledge on the topic. we showed that using changepoint detection methods on the diachronic word embeddings that we trained gave results most often matching the observations made by traditional scholarship. through a combination of changepoint detection and neighbour analysis it was possible to provide explanations for mismatches between previous literature and our findings, in some cases noticing that our results were able to capture features of semantic change not identified by the expert sources (see, for example, the analysis of coach above). our analysis provides the bases for new datadriven investigations on the lexical field of mechanization that do not rely so closely on external knowledge bases as in our study. 1. define three or four decades around which a semantic shift appears to have taken place for a word w. this is established through a combination of automatic changepoint detection and close reading of the neighbours of w. the selected decades should be adjusted across different runs to achieve the clearest visual rendition of a semantic shift (if any).