SC o T : Sense Clustering over Time: a tool for the analysis of lexical change (2021)

Abstract: We present Sense Clustering over Time (SCoT), a novel network-based tool for analysing lexical change. SCoT represents the meanings of a word as clusters of similar words. It visualises their formation, change, and demise. There are two main approaches to the exploration of dynamic networks: the discrete one compares a series of clustered graphs from separate points in time. The continuous one analyses the changes of one dynamic network over a time-span. SCoT offers a new hybrid solution. First, it aggregates time-stamped documents into intervals and calculates one sense graph per discrete interval. Then, it merges the static graphs to a new type of dynamic semantic neighbourhood graph over time. The resulting sense clusters offer uniquely detailed insights into lexical change over continuous intervals with model transparency and provenance. SCoT has been successfully used in a European study on the changing meaning of ‘crisis’.

## introduction most real-world networks change over time. so do dynamic networks of word similarities that can be used to infer the meanings of a word. the noun 'crisis', for example, used to be strongly linked to the religious word 'doom' in english-language books in the early modern period. however, in the modern age 'crisis' has become more closely associated with terms denoting economic problems such as 'unemployment', 'depression' or 'inflation' @xcite . in the recent decade, the interest in dynamic networks has increased. @xcite . this has also stimulated new graph-based approaches to analysing vocabulary change @xcite . such research is a key interest of linguists @xcite and scholars in the humanities @xcite @xcite . traditionally, scholars have determined such changes through close reading. however, the growing availability of ever larger digital corpora @xcite and the increasing speed of sense changes in social media @xcite have boosted the significance of new research @xcite . of particular importance in the research on lexical change is the unsupervised approach of word sense induction (wsi). wsi enables the development of data-driven hypotheses. the approach induces meaning from the bottom upwards and can be used with a diachronic angle. several implementations for diachronic wsi exist @xcite . while many of them represent word meaning by dense vector embeddings, sparse models with network representations still play an important role. the use of sparse, human-readable models enables a better interpretation of meaning hypotheses by linguists and other researchers. there are two main approaches to implement diachronic network-based wsi. discrete approaches compare several networks that relate to discrete points in time. continuous approaches analyse when specific nodes, edges or clusters appear in a single dynamic network that changes continuously over time @xcite . many applications in the field of diachronic network-based wsi fall into the discrete category. @xcite , for example, reduce the number of measuring points to single intervals, build one graph per discrete interval, cluster it and track the resulting sense clusters over intervals. while this approach is considered as less complex than the continuous one @xcite , it brings up complexity problems of its own. the clustering of graphs can namely lead to different solutions. thus, the number of clustering combinations in a time-series of sense graphs can grow unpredictably. another issue is the identification of corresponding clusters across time points. continuous representations are more fine-grained, but can lead to other issues. since the clustering in such scenarios is mostly done in an incremental way, problems of costly reclusterings or very large clusters can emerge. the application sense clustering over time (scot) offers a new hybrid approach to network-based wsi that reduces complexity. scot works in two steps. in a first, 'discrete' step, the time-stamped documents are aggregated into intervals. static graphs are built per interval. then scot merges the static graphs to a new type of dynamic neighbourhood graph over time (ngot). the encoded time-based information from the underlying continuous series of graphs enables a time-coloring of the sense clusters. haase (2020) has shown that there are different approaches to constructing such semantic ngots. the best known method for creating such a dynamic network consists of the merging of a series of equally-sized graphs from each interval, but it is also possible to aggregate nodes and links in different ways. these approaches exhibit different strengths and are explained in more detail below. figure 1 shows how such a ngot looks like. the graph shows sense clusters for the target word bar. it shows that words such as "button", "desktop" and "icon" became increasingly similar to each other and to the target bar in the 1990s, thereby forming the new sense of 'computer-menu'. scot can be used for various tasks such as linguistic studies of polysemic words or research into the history of concepts, but also offers a general and new solution to the analysis of dynamic networks with the neighbourhood graph over time. ## system description the system enables an analysis of the lexical change of words in an interactive web-interface based on metrics calculated from large time-sliced corpora. this requires a division of the system into a web-front-end and a back-end that accesses the databases with the similarity scores. in addition, the system offers the user additional information on the syntagmatic features that have been used to calculate the similarity scores. ## conclusion and future directions this article describes scot, a new tool for the analysis of the changes of sense clusters in dynamic networks. scot reduces the complexity of this task through interval-aggregation and neighbourhood graphs over time. the dynamic network retains the time-based information. this enables advanced analyses that can be well visualised. the usage of a sparse approach to distributional semantic modeling provides model transparency and provenance. we have demonstrated the applicability of the solution in the domain of lexical and conceptual change. however, the general nature of the application make it transferable to other domains that use dynamic networks for analysis. future directions in the development of scot lie in the further refinement of neighbourhood graphs over time, the broadening of the usage of scot in various domains, including conceptual change, as well the research on the wider implications of the application for diachronic distributional semantics. scot is available open source under the mit license foot_0 and as an online demo foot_1 . a video demonstrating many of the functionalities can be found at https://youtu.be/sbmfa4hkjvg .