Anchors in Embedding Space: A Simple Concept Tracking Approach to Support Conceptual History Research (2023)

Abstract: We introduce a simple concept tracking approach to support conceptual history research. Building on the existing practices of conceptual historians, we use dictionaries to identify “anchors”, which represent primary dimensions of meaning of a concept. Then, we create a plot showing how a key concept has evolved over time in a historical corpus in relation to these dimensions. We demonstrate the approach by plotting the change of several key concepts in the COHA corpus.

## introduction conceptual history is the study of the abstract, sociopolitical concepts that are used to describe and understand history. the purpose of our work is to complement the computational methods that are available for research in conceptual history by introducing an approach specifically designed to be easily used by conceptual historians. conceptual historians are interested in the evolution over time of "key concepts" that have social or political relevance. our approach follows the work of reinhart koselleck, a pioneer of conceptual history. while key concepts are necessarily expressed as words, not all words are concepts in the sense of @xcite in his introduction to geschichtliche grundbegriffe, a collaborative multivolume lexicon of key concepts during the period 1750-1850, koselleck states, "a word becomes a concept when a single word is needed that contains-and is indispensable for articulating-the full range of meanings derived from a given sociopolitical context" @xcite . analysis of sociopolitical concepts aims to bring out the coexistence of meaning layers ("temporal strata") in a given concept @xcite . one approach used by koselleck is to analyze how definitions in dictionaries change over the years, e.g., in @xcite . a definition of a word in a dictionary explicitly expresses primary dimensions of meaning. here, we do not analyze historical dictionaries, but rather investigate changes in the relative importance of primary dimensions of meaning derived from a contemporary dictionary. as such, our approach represents a way to base concept tracking on known, and explicitly expressed, dimensions of meaning, without relying on the existence and availability of multiple historical dictionaries. note that dictionaries are not the only source that conceptual historians use to identify dimensions of meaning that are interesting for investigation. however, due to their importance and easy accessibility, we focus on them here. our approach uses two primary dimensions of meaning represented by words we refer to as "anchors". we use the contextual word embeddings of the two anchors to create a plot that visualizes how the meaning of a key concept has changed over time within a diachronic corpus of historical texts. we argue that this simple concept tracking approach is useful for conceptual history research. first, it yields a concise plot that is easily interpretable for historians. second, contextual embeddings do not necessarily have to be trained on the specific data being analyzed, which in the case of conceptual history research might be quite limited. ## our anchor-based approach for each key concept to be tracked, we retrieve its dictionary definition and look at the different meanings (sub-definitions) there. we select the two major meanings and, for each, choose an "anchor", a term that captures that meaning (i.e., a keyword from the sub-definition). in the online miriam-webster dictionary that we used, such words are often bolded. if more than two major meanings are present, we choose the meanings that are most related to the research interests of the conceptual historian. once we have two anchors, we plot the difference over time between the similarity of the key concept to one anchor and its similarity to the other. the data and the details of our implementation are described in this section. ## tracking key concepts: three examples we illustrate our approach with the key concepts "privacy" (anchors: "seclusion"/"freedom"), "peace" (anchors: "tranquility"/"safety") and "fairness" (anchors: "equality"/"honesty"). results are shown in ## connecting to conceptual history in this section, we present an example illustrating how our anchor-based approach might connect to existing conceptual history research. specifically, we look at work by @xcite on how "climate" has emerged as a key concept. before the rise of climate science, "climate" was undifferentiated from geography and weather associated with places. in early modern geography, "climate" was roughly identical to geodetic position. however, over the years "climate" has become globalized, i.e., associated with future weather conditions of the entire planet. we can explain the shift from "lo-cal" to "global" in terms of the difference between meteorology and climate science. the latter deals with weather patterns averaged over long periods of time and on a planetary scale. figure 2 shows a graph of the key concept "climate". we see "climate" moving further from "local" and closer to "global" over time. the mann-kendall test gave an increasing trend (@xmath0.42×10 -4 , @xmath1.001) so the shift was significant. in this time span, "climate" appeared an average of 48 times per year in the corpus, with a median standard deviation of the subtracted average cosine similarities of all years of 0.045. particularly in the last quarter of the 20 th century, our analysis shows "climate" became increasingly associated with "global". however, before that time it was already evolving away from "local" and towards "global". these observations are consistent with the ideas and insights of @xcite . we note that @xcite point to the oxford english dictionary as a source of support for older "local" meanings of "climate", but the choice of the anchors here is also based on other considerations, such the rise of climate science, mentioned above. finally, we emphasize that our anchor-based approach is not intended to replace concept graphs, such as those also used by @xcite , but rather complements them. ## conclusion and outlook in this study we have proposed a "definitions as anchors" approach to tracking the evolution of "key concepts", i.e., abstract sociopolitical concepts, which makes use of "anchors" drawn from dictionary definitions. our approach maps key con- key concept anchor concept(s) years slope m k p-value m k peace tranquility 1900-2000 -2.32×10 -4 <0.001 peace safety 1900-2000 -2.10×10 -4 <0.001 peace tranquility -safety 1900-1945 -3.66×10 -4 <0.001 peace tranquility -safety 1945-2000 1.12×10 -4 0.032 we have argued for pre-defined anchors because it builds on conceptual historians' established practices. however, we also note that using pre-defined anchors may help to address our concern that the neighbors of a key concept within a time window are determined more by the dominant topics in that time window, rather than by an actual shift in the semantics of the key concept. the importance of this concern should be investigated in future work. future work should also investigate the advantage that contextual word embeddings offer in leveraging more training data that non-contextual embeddings. our word embedding model was pre-trained on the order of 10 9 running words. in contrast, if the coha collection is split into year-length windows and a static word embedding model is built on each window, i.e., the approach of @xcite , each model is trained on only on the order of 10 6 words, three orders of magnitude fewer words. in sum, our study enriches conceptual history with an approach that can statistically confirm monotonic changes of abstract sociopolitical concepts over time in a diachronic text corpus. it contributes to the practical understanding of how and over what time periods conceptual shifts occur.