Towards Detecting Lexical Change of Hate Speech in Historical Data (2023)

Abstract: The investigation of lexical change has predominantly focused on generic language evolution, not suited for detecting shifts in a particular domain, such as hate speech. Our study introduces the task of identifying changes in lexical semantics related to hate speech within historical texts. We present an interdisciplinary approach that brings together NLP and History, yielding a pilot dataset comprising 16th-century Early Modern English religious writings during the Protestant Reformation. We provide annotations for both semantic shifts and hatefulness on this data and, thereby, combine the tasks of Lexical Semantic Change Detection and Hate Speech Detection. Our framework and resulting dataset facilitate the evaluation of our applied methods, advancing the analysis of hate speech evolution.

## introduction the present research landscape on lexical change in nlp predominantly focuses on generic language evolution, targeting shifts in meaning for a set of words that span a wide spectrum of vocabulary @xcite . this approach falls short of modeling meaning shifts in specific domains or dimensions of meaning (e.g. hatefulness), which is often of interest when applying language change detection in disciplines beyond linguistics, i.e. in social sciences and humanities. for instance, historians investigating religious conflicts between protestants and catholics during the english reformation may be particularly interested in the dynamics of polemical expressions @xcite , which exist within a limited subset of the lexicon. in this paper, we present a first step towards the detection of meaning shifts within a particular subdomain. specifically, our focus is on the domain of hate, aiming to uncover, for instance, change in hateful usage of the term 'foxes' during the 16 th century as illustrated in figure 1 . lexical semantic change detection (lscd) is currently the predominant approach to modeling meaning shift in nlp. lscd methods are typically designed to observe shifts in word usage, targeting a word's denotative meaning within evaluation data encompassing general language sources such as newspapers and books @xcite . target words are selected from the full vocabulary, often guided by etymological and historical dictionaries. following this, well-developed techniques detect semasiological (from term to concept) variation by determining to what extend a word has shifted its meanings somehow. while certain more interpretable methods could offer deeper insights into the nature of individual shifts, by e.g. looking into usage clusters or word substitutes @xcite , current lscd approaches have not demonstrated the ability to detect shifts within specific semantic subdomains, such as hate, which could be considered as an onomasiological perspective (from concept to term). the development of hate speech detection (hsd) systems, on the other hand, does address the identification of lexical items used to convey hateful meanings (e.g. @xcite @xcite . however, the evolution of these expressions often remains unexplored @xcite being a rare exception). although hate speech lexicons are frequently integrated into these systems, their application is most prevalent within limited temporal scopes, such as short-term social media data sets. our study introduces the task of detecting lexical semantic change of hate speech in historical texts. such changes can involve an increase or decrease in hatefulness, or even the acquisition of an entirely new hateful sense. to address this task, we present an interdisciplinary framework, that brings together nlp and history. more specifically, we use and combine methods, annotation and evaluation procedures for lexical semantic change detection (lscd) and hate speech detection (hsd) in the context of historical data. the resulting dataset, consisting of 16 th century early modern english religious writings in the context of the protestant reformation, is enriched with annotations of both lexical semantic changes and lexical hatefulness. in conclusion, our paper presents a 1) task, 2) dataset and 3) methodological framework facilitating the evaluation of computational approaches for identifying shifts in hateful word meanings. ## discussion & conclusion our study introduces the novel task of detecting changes of hateful word meanings in historical texts. our interdisciplinary approach combines lexical semantic change detection and hate speech detection. we leverage historical expertise to generate a pilot dataset with two-aspect annotations, a valuable resource for the evaluation of computational methods. while our methods showed effective precision in detecting hateful words that changed their meaning throughout the 16 th century, they also underscored the complexity of (the combination of) the tasks, as evidenced by the human interrater agreement scores. the exploration of hate speech within historical discourse poses particular challenges. most importantly, we acknowledge inherent limitations as we may never achieve a perfect reflection of historical connotations. still, our framework aims at a closer grasp of the past by combining historical research and linguistic analysis. the bounded text selection and the limited annotated data (for reasons of feasibility) pose challenges to the robustness and generalizability of our findings, pertaining to the efficacy of the employed methods as well as the outcomes we have presented. therefore, our conclusions should be further validated in followup research that incorporates more diverse textual sources and enhances the quantity (and quality) of the annotated data. we further propose to broaden the scope beyond nouns, as verbs and adverbial phrases can also convey hate in the form of devaluation of action. moreover, our error analyses highlighted the prevalence of metaphors for expressing hateful meanings, suggesting another direction further research. finally, expanding the focus to longer time-spans or conducting cross-language comparisons could also yield valuable insights. in conclusion, our paper lays foundations for advancing the analysis of lexical change of a specific domain in historical data. particularly, our interdisciplinary framework paves the way to an expanded dataset and the development of better computational methods for detecting the evolution of historical hate speech.