Can political dogwhistles be predicted by distributional methods for analysis of lexical semantic change? (2024)

Abstract: We show that meaning shifts in political dogwhistle expressions (DWEs) are explained by the expressions changing with regard to their “hidden” (in-group) and “public” (out-group) dimensions. We study the association between computational measures of Lexical Semantic Change (LSC) and the In-group/Out-group Ratio (IOR) of four Swedish DWEs. We use a combination of distributional modeling of DWEs in the online discussion forum Flashback and data collected from a lexical replacement survey of Swedish residents. We explore several vector-space meaning representation approaches and demonstrate that distributional methods can be used to identify semantic shifts relevant to dogwhistle development, particularly contextual representations from Swedish BERT, SBERT, and multilingual T5.

heading: 1 Introduction
text: Online media is important for political communication, but its fast pace makes it very susceptible to meaning manipulation and deceptive communication strategies. Analyzing communicative patterns in such large quantities of data requires computational methods (Theocharis and Jungherr, 2021). In the context of political discourse, this form of data analysis has been used to combat hate speech and related problems for online moderation.
A key challenge for automated analysis of text is identifying implicit meanings (Magu and Luo, 2018). In this work, we explore computational approaches for modeling the temporal dynamics of political dogwhistles. Following Lo Guercio and Caso (2022, p. 203), political dogwhistles can be defined as “speech acts that explicitly convey a certain content to an audience, while simultaneously sending a different, concealed message to a specific subset of that audience” (Saul, 2018; Howdle, 2023; Witten, 2023). Henceforth, we refer to the explicit meaning of dogwhistles as their out-group
meaning, and the concealed meaning as their ingroup meaning. We define a dogwhistle expression (DWE) as a linguistic form that encodes this dual function and carries both in-group and out-group meanings (Henderson and McCready, 2018).
Dogwhistles that secretly convey racist or otherwise derogatory attitudes are ethical problems for democratic society (Åkerlund, 2022; Lindgren et al., 2023; Bhat and Klein, 2020; Saul, 2018; Stanley, 2015; Haney-López, 2014). Independent of content, dogwhistles have been discussed as problems for democracy by obscuring political mandate and democratic legitimacy (Goodin and Saward, 2005; Howdle, 2023).
Previous work includes theoretical accounts of how dogwhistles work semantically (Breitholtz and Cooper, 2021; Henderson and McCready, 2018; Stanley, 2015; Khoo, 2017; Lo Guercio and Caso, 2022), experiments that test the consequence of dogwhistle communication for the acceptance of policies and attitudes (White, 2007; Wetts and Willer, 2019), and content analyses of how dogwhistles are used online (Bhat and Klein, 2020; Åkerlund, 2022). Less attention has been devoted to the distributional modeling of dogwhistle meaning (but see, e.g., Hertzberg et al., 2022; Mendelsohn et al., 2023; Boholm and Sayeed, 2023; Xu et al., 2021). In particular, while semantic change is essential to the concept of dogwhistle, it has only recently been systematically addressed (Boholm and Sayeed, 2023; Sayeed et al., 2024).
Our aim is to combine established methods of lexical semantic change (LSC) detection (Kutuzov et al., 2018; Tahmasebi and Dubossarsky, 2023; Tahmasebi et al., 2021; Tang, 2018) and survey data from linguistic replacement tests (Arefyev et al., 2022; Lindgren et al., 2023) to model the temporal dynamics of dogwhistle meaning over time. The research questions are (1) to what extent are computational measures of LSC associated with shifts in the in-group and out-group meanings of DWEs.
144
Moreover, we ask (2) how different approaches to modeling meaning compare with respect to the relationship between LSC and shifts in in-group and out-group meaning over time.
We analyse the relationship between rate of LSC and the in-group–out-group dynamics of dogwhistles through four ways of modeling meaning: (i) skip-gram with negative sampling (SGNS) (Mikolov et al., 2013), (ii) Bidirectional Encoder Representations from Transformer BERT (Devlin et al., 2019), (iii) Sentence-BERT (SBERT) (Reimers and Gurevych, 2019), and (iv) massively multilingual Text-to-Text Transfer Transformer (mT5) (Raffel et al., 2020; Xue et al., 2021). These methods are sensitive to the dynamic meaning changes of DWEs, suggesting that they can be developed for the detection and analysis of dogwhistle communication online. We also show that the pipelines with the large language models (LLMs) are better at predicting dogwhistle meaning shifts than the SGNS-based pipelines.

heading: 2 Related work
text: Methods of distributional semantics have recently been applied to the long-standing study of semantic change (Bréal, 1904). Advances include the development and validation of approaches for studying when, how, and how much words change. (Kutuzov et al., 2018; Tahmasebi and Dubossarsky, 2023; Tahmasebi et al., 2021; Tang, 2018). To study how much and when words change, the features of the vector representations can be compared. Formally, the semantic change of a word w in a transition from ti to tj can be defined as the distance of w’s vector at ti (−→w ti) and its vector at tj (−→w tj ):
∆ti,tj (w) = distance( −→w ti ,−→w tj )
Diachronic word embeddings have been built as static word embeddings trained at time periods t1, . . . tn (Hamilton et al., 2016b; Kim et al., 2014), such as SGNS (Mikolov et al., 2013), PPMI (Levy et al., 2015) and GloVe (Pennington et al., 2014); by averaging over contextualized token embeddings at t1, . . . tn (Martinc et al., 2020a; Kutuzov and Giulianelli, 2020), using, for example, BERT (Devlin et al., 2019) and ELMo (Peters et al., 2018); and as probability distributions over clusters of contextualised token embeddings at t1, . . . tn (Giulianelli et al., 2020; Kutuzov and Giulianelli, 2020; Martinc et al., 2020b; Vani et al., 2020).
To investigate how words change, we can analyze how words’ positions change in the vector space (Hamilton et al., 2016a,b). By measuring the distance between the vector of a word w and those of other words, the nearest neighbors of w at time ti can be compared with its neighbors at tj (Charlesworth et al., 2022; Vylomova and Haslam, 2021; Tripodi et al., 2019). With predefined concepts (or dimensions) of interest (Caliskan et al., 2017), w’s distance to those “concepts” can be tracked over time (Mendelsohn et al., 2020). This latter approach enables exploration of conceptual shifts in large datasets, possibly over long time spans (Garg et al., 2018). For example, Mendelsohn et al. (2020) studied the dehumanization of LGBTQ people in US media by tracking over time the distance between the words for these groups and the vocabulary relevant for the analytical dimensions investigated (e.g., disgust and power). Other work has tested the theory of “concept creep” (Haslam, 2016) by analyzing the semantic shift of harm-related (Vylomova and Haslam, 2021) and health-related concepts (Baes et al., 2023).
The present work analyses dogwhistles and how their in-group and out-group dimensions of meaning change over time. Previously, philosophers of language and linguists have tried to explain the dual meanings of dogwhistles (Breitholtz and Cooper, 2021; Henderson and McCready, 2018). The role of convention versus pragmatic inference is one of the main theoretical issues addressed in this discussion (Breitholtz and Cooper, 2021; Henderson and McCready, 2018; Stanley, 2015; Khoo, 2017; Lo Guercio and Caso, 2022). Few attempts have been made to use distributional semantics to study dogwhistles, but notable exceptions exist. Hertzberg et al. (2022) partitioned in-group and outgroup interpretations of DWEs in a word replacement experiment, using SBERT. Xu et al. (2021) built an annotated data set for Chinese dogwhistles. Similarly, Mendelsohn et al. (2023) presented an extensive database of dogwhistle definitions in a US context. In addition, they illustrated the ability of GPT-3 to identify dogwhistles, based on prompts with definitions from their database.
We expand on these efforts to study dogwhistles by combining LSC techniques and suvey data for modeling in-group–out-group dynamics of DWEs. Although time is essential for dogwhistles, since the in-group meaning evolves in parallel to an existing (out-group) meaning (Sayeed et al., 2024), only recently have the temporal aspects of dogwhistles
been systematically studied. Boholm and Sayeed (2023) used computational methods of LSC analysis to model the rate of change of DWEs in different online discussion forums and found that the rate of semantic change of DWEs observed in the highly politically polarized online community diverged from the rate of semantic change of the same terms (at the same period of time) in the less polarized community, suggesting that dogwhistle evolution is community dependent (Quaranto, 2022; Clark, 1996). However, they did not systematically test whether the rate of change observed for the DWEs was explained by systematic variation in the ingroup and out-group meaning of the expressions.

heading: 3 Data

heading: 3.1 Replacement survey
text: We use data from a word replacement test implemented via a survey of Swedish residents. The aim of this test was to quantify variability in how individuals understand the meaning of dogwhistles. In the first step, we collected potential dogwhistle words from political messaging in Swedish media. Twelve words were included in the replacement test (February and March 2021). The sample (n=1780) consisted of self-recruited panelists, pre-stratified to reflect the Swedish population in terms of age, gender and education.
Panelists were asked to read sentences and instructed to replace a potential DWE in each sentence with one or more words so that the meaning of the sentence remains largely the same. The replacement test was completed by 1,045 panelists, with a participation rate of 51%.
The test was followed by manual coding of responses. A coding manual was drafted and refined by the research group. Coders classified the replacement words into three categories: 1) the implicit dogwhistle meaning, 2) the explicit literal meaning, or 3) word(s) that could not be coded as 1 or 2. In this study, we take DWEs that had high inter-annotator agreement (Krippendorff’s α > 0.6) and acceptable corpus frequency (at least 10 instances per year when mentioned). We discuss these in the next section.

heading: 3.2 Four Swedish DWEs
text: The in-group meanings of the DWEs analyzed can be listed at a general level. With the out-group meaning of ‘suburban gang’, the in-group meaning of the dogwhistle förortsgäng is that of ‘immi-
grant gang’. As such, this DWE works by a biased place-for-person metonymy, similar to inner city discussed in US context (Saul, 2018). The DWE återvandring (‘re-migration’) has in-group and outgroup meanings based on the (in)voluntariness of the process, with a voluntary act as the out-group meaning, while ‘deportation’ is the in-group meaning. The DWE of berika (‘enrich’) is the result of malevolent irony, in response to positive opinions on multiculturalism, where the in-group meaning is the opposite of enrichment, namely criminal and destructive activities (by immigrants). In a Swedish context and elsewhere, globalist is used with several different in-group meanings, including an antiSemitic reference to Jews, a nationalistic reference to anti-nationalists (i.e., opponents of nationalism), and a populist reference to elitism.

heading: 3.3 Corpus
text: Flashback is a discussion forum with over 1.5 million users and more than 80 million posts, as of 13 March, 2024 (according to the website’s own claim). The topics of discussions are organized in “threads” under 15 general sections (e.g., drugs, economy, lifestyle and politics). With anonymous users, Flashback is known for discussion of controversial topics and the expression of controversial opinions, including discrimination and racism (Åkerlund, 2021; Blomberg and Stier, 2019; Malmqvist, 2015). Although hate speech and threats are not allowed by the rules, the website clearly contains offensive language. We here analyze Flashback data from 2000 to 2022, on the topic of politics. The corpus, which in total contains 49M sentences (posts) and 785M words, was collected from the Swedish national language data processing infrastructure Språkbanken Text.1 On average, there are 2.1M sentences (SD = 1.4M) and 34.1M words (SD = 21.7M) per year.
There is considerable variation in frequency of the four DWEs analyzed in the corpus (Table 1). In particular, förortsgäng is much less frequent than the other terms. Moreover, term frequencies are very different in different years, which is reflected in the high values of the standard deviation.
The corpus has been preprocessed for all pipelines (SGNS, BERT, SBERT and mT5) by lower-casing and removing URLs and emojis. Corpus data for the SGNS approach have been further processed by removal of numbers and punctuation;
1https://spraakbanken.gu.se/en/resources/ flashback-politik
separation of compounds that contain the DWEs under analysis as their left-hand element, e.g., “globalistelit” is replaced by “globalist elit” (with space); and lemmatization of the DWEs analyzed, for example, “globalisten” (definite form of globalist) is replaced by “globalist” (lemma form). Regular expressions were used for lemmatization and splitting of compounds. For the other approaches, there was no additional step of preprocessing to the steps listed above, but some minor changes were made to facilitate mapping of input words and tokenisation for BERT and mT5.

heading: 4 Semantic modeling
text: Below we introduce four pipelines to test the relationship between the LSC and the in-group/outgroup dynamics of DWEs. The piplelines have two basic steps: (a) modeling of the rate of semantic change of DWEs in the corpus; and (b) modeling of the degree of in-group vs. out-group meaning of the DWEs based on the replacements observed in the survey. The key difference between the four pipelines is the algorithm used for modeling meaning: SGNS, BERT, SBERT and mT5.2

heading: 4.1 LSC modeling
text: The semantic change of a word w in a transition from ti to tj , i.e., ∆ti,tj (w), is defined as the angular distance of w’s vector at ti (i.e., −→w ti) and its vector at tj (i.e., −→w tj ) (Kim et al., 2014; Noble et al., 2021):
∆ti,tj (w) = arccos(cossim(−→wti ,−→wtj ))
π
We apply four approaches to build time-indexed word vectors in the diachronic corpus C, which is a collection of sentences from the consecutive set of time periods, T = ⟨2000, . . ., 2022⟩. Thus, C = ⟨c2000, . . ., c2022⟩. Vectors are trained only for words at t with a minimum frequency of 10.
2Code for running experiments can be found at https: //github.com/mboholm/dogwhistle-lsc-predicition.

heading: 4.1.1 The SGNS approach
text: A SGNS model is trained for each sub-corpus in C, in the sorted order of T, from first to last. The weights of the model are randomly initialized for the first time period, M2000, but for every other model, Mti , where ti > 2000, the weights of Mti are initialized with the trained weights of Mti−1 . For every consecutive pair in T, i.e. the set of transitions R = ⟨⟨t1, t2⟩, . . .⟨tn-1, tn⟩⟩ = ⟨⟨2000, 2001⟩, . . .⟨2021, 2022⟩⟩, and for every word w existing in both models Mti and Mti+1 , the vectors −→wti and −→w ti+1 are compared for ∆ti,tj (w). We train six SGNS variants for 100 and 200 dimensions and window sizes of 5, 10, and 15.

heading: 4.1.2 The BERT approach
text: The diachronic corpus B is a subset of C, such that it covers the same consecutive time periods in T, but where every sub-corpus bt = {sentence s: s is in ct∧ at least one the analyzed DWEs is in s}. Sentences in B are encoded by Swedish BERT (Malmsten et al., 2020).3 A word vectors of a DWE w at t is built in two steps: first, contextualised token embeddings of w in sentences from bt, are built by averaging over the token embeddings of the last hidden layer of BERT that correspond to w in the input. Next, the mean vector of the contextualized token embeddings for w in t constitutes −→w ti .4

heading: 4.1.3 The mT5 approach
text: The third approach uses the mT5 model (Xue et al., 2021), a multilingual variant of T5 (Raffel et al., 2020) trained on the multilingual extension of the Colossal Clean Crawled Corpus (C4), mC4, which in total contains 6.3T tokens. Swedish is among the 101 languages in mC4. With T5, every NLP task is generalized as text-to-text problem. The model is similar to the original transformer model in Vaswani et al. (2017), with some alternations of, for example, normalization of layers and position embeddings (Raffel et al., 2020; Xue et al., 2021). T5 was originally developed to test, in a unified and controlled way, the effectiveness of transfer learning on a variety of NLP tasks (Raffel et al., 2020). However, our implementation does not fine-tune the pre-trained model. Rather, our main motive for
3https://huggingface.co/KB/ bert-base-swedish-cased
4For some compound words, the tokenization for BERT or mT5 does not perfectly match the DWE part of the compound. We then use the embeddings of tokens that maximize the similarity of the two strings by the Ratcliff et al. (1988) algorithm implemented as SequenceMatcher in Python.
using mT5 is to test a recent large-scale transformer. Here we use the 3.7 billion parameter version of the model, named XL.5
We build word vectors at t as in the BERT approach: contextualized token embeddings are built by averaging token embeddings of the last hidden layer, corresponding to w in the input sentence; −→w ti is the mean vector of the contextualised token embeddings at t.

heading: 4.1.4 The SBERT approach
text: The fourth and final approach uses Swedish SBERT (Rekathati, 2021).6 SBERT (Reimers and Gurevych, 2019) is BERT (Devlin et al., 2019) fine-tuned for predicting the semantic similarity of two sentences. SBERT has a bi-encoder architecture to reduce the computational cost of sentence pair-regression in original BERT. Reimers and Gurevych (2019) show that a bi-encoder with fine-tuning reaches state-of-the-art performance on sentence similarity. Swedish SBERT is trained with transfer learning in Reimers and Gurevych (2020), where the objective is to make a student model7 (of an under-resources language, here: Swedish) match the sentence embeddings of a high-performing teacher model8 (developed for a well-resourced language, here: English) in a parallel corpus.
The implementation of the SBERT approach is in most respects similar to the implementation of the other transformer models, but does not require mapping between token embeddings and the DWE of the input, nor selection of layer, since SBERT output 1 × 768-dimensional vectors that serve as the contextualized token embedding. The mean vector of the contextualized embeddings for w at t constitutes −→w t.

heading: 4.2 In-group and out-group modeling
text: We modeled the semantic dimensions of in-group and out-group meaning of a DWE w at time t by measuring the similarity between (a) the embedding for w at t trained on online community data (as defined above, sect. 4.1) and (b) the (averaged) embedding for text replacements Rw = {rw1 , ..., rwn } for w in the replacement survey, annotated as “ingroup” (Iw) or “out-group” ( Ow). Details on how
5https://huggingface.co/google/mt5-xl 6https://huggingface.co/KBLab/
sentence-bert-swedish-cased 7https://huggingface.co/KB/ bert-base-swedish-cased 8https://huggingface.co/sentence-transformers/ paraphrase-mpnet-base-v2
the in-group and out-group embeddings, −→ Iw and−→ Ow, are built from Iw and Ow are presented in the following (sect. 4.2.1 - 4.2.2); each approach parallels those defined above for the analysis of LSC.
Once in-group and out-group embeddings for DWE w are derived, we use cosine similarity to calculate an in-group score (IS) and an out-group score (OS) at each time t:
ISt(w) = cossim( −→wt, −→ Iw)
OSt(w) = cossim( −→wt, −→ Ow)
Next, we define the In-group/Out-group Ratio (IOR) of DWE w, reflecting a normalized measure of w’s in-group meaning relative to its out-group meaning (Kapron-King and Xu, 2021):
IORt(w) = ISt(w)
ISt(w) +OSt(w)
To measure the change in IOR for w over time, we define the absolute difference in IOR as:
∆IORti,tj (w) = abs(IORtj (w)− IORti(w))
This study uses linear regression to test whether the difference in IOR (i.e., ∆IORti,tj (w)) is a predictor of the LSC of DWEs (i.e., ∆ti,tj (w)). Regression models are described in more detail below (sect. 5.1), but first vectorization of in-group and outgroup dimensions is addressed.

heading: 4.2.1 SGNS
text: For the SGNS approach, vectorization of in-group and out-group dimensions is based on the word embeddings trained for the diachronic corpus data (sect. 4.1.1). A bag-of-words (BOW) approach was implemented to build in-group and out-group embeddings from the SGNS models.9
The steps for building in-group and out-group embeddings from the BOW-sets are as follows:10 first, stopwords were removed. Second, the top 3 words of each BOW set were selected based on
9An alternative approach could have been to build token vectors of multi-word inputs (replacements) by pooling SGNS word vectors of the input and then averaging over those token vectors (similar to the approaches described below). The BOW approach implemented here has lower computational cost than a pooling of multi-word inputs would have had.
10We have tested different strategies for selection. Other examples of strategies include selecting the top 3 most frequent words in Iw and Ow without overlap.
their keyness (Gabrielatos, 2018), using the odds ratio, which is an asymmetric measure of the probability of a word in a target corpus relative to a reference corpus (e.g., the probability of finding a word x in the in-group replacements relative to the outgroup replacements). Third, we generalize from the selected words, by adding related word forms of the same lexeme, using existing resources for Swedish morphology (Borin and Forsberg, 2009). For example, in the replacement survey participants were asked to replace the plural form of the DWE globalist, i.e., “globalister” (plural). Consequently, the replacements for globalist are dominated by plural forms of nouns, e.g., “elitister” (plural). However, embeddings for other wordforms than the exact ones used in the replacement survey might be relevant for modeling the in-group and out-group dimension. Once the word forms of each lexeme were identified, to minimize the influence of infrequent words, word forms that were not frequent enough to account for at least 20% of the frequency of the lexeme were removed. Finally, after selection and expansion, for each word in a remaining set, its SGNS embedding was collected. The ingroup and out-group embeddings are defined as the average vector of the collected embeddings for each set (see Appendix A for examples of words).

heading: 4.2.2 BERT, mT5, and SBERT
text: For BERT and mT5, we represent each replacement r by the average embedding of the last hidden layer (Ni et al., 2021).11 Since SBERT is designed to represent sentences, there is no need for (additional) pooling of token embeddings. Replacements are represented by sentence embeddings. We define the in-group ( −→ Iw) and out-group embeddings ( −→ Ow) as the mean vectors of the contextualized token embeddings for the replacements in Iw and Ow.
For examples of sentences from the Flashback training data, with high and low scores of IOR, see Appendix B.

heading: 5 Analysis

heading: 5.1 Regression models
text: The relationship between IOR and LSC is modeled by linear regressions (OLS), implemented in Python through statsmodels package. We try to predict the rate of semantic change of DWEs
11For BERT we also tested the embedding of the CLS token, which resulted in slightly higher R2 scores. Here we focus on the mean pooling approach for comparability with the pipeline for mT5-XL, which lacks a CLS token (Ni et al., 2021).
(∆ti,tj (w)) from their change in IOR (∆ IOR ti,tj (w)). If the coefficient for the (independent) variable is significant, the semantic change observed for the DWEs is explained by their shifting meaning with regard to in-group and out-group meanings. In addition to the significance of the coefficent for ∆IOR, the pipelines defined above can be compared with respect to the total variance explained (R2). In total, there are 64 DWE-time pairs in the data.
Previous research has shown that semantic change is strongly correlated with term frequency (Dubossarsky et al., 2017; Hamilton et al., 2016b). To avoid having term frequency as a confounding factor between ∆ti,tj (w) and ∆ IOR ti,tj (w), we control for the effect of term frequency by having term frequency per million (FPM) (at ti, log2transformed) and proportional change in FPM from ti to tj as predictors (control variables).
Thus, we model the following relationship:
∆ti,tj (w) = β0 + β1 ×∆IORti,tj (w)+ β2 × log2(FPMti(w)) + β3 ×∆FPMti,tj (w)
For comparability, the model variables are normalized by z-scores. We assess there being no problem with multicollinearity, since the variance inflation factor (VIF) for independent variables is close to 1 (below 2) in all models. For all regression models, except the ones based on the pipeline for 200-dimensional SGNS models, the residuals are not normally distributed, as measured by the Jarque-Bera test. Under the assumption of the central limit theorem, we proceed with the regression model proposed above, despite nonnormal residuals, relying on our sample size being sufficiently large (N = 64, with three predictors) (Weisberg, 2013; Schmidt and Finan, 2018). However, we did test transformations of variables to meet the assumption of normal residuals, see Appendix C. The overall patterns are the same.

heading: 5.2 Results
text: For most models, shifts in IOR (∆IORti,tj (w)) is a significant predictor of rate of semantic change (∆ti,tj (w)). That is, the rate of change observed for DWEs using common methods for LSC-modeling is related to shifts in in-group and out-group meaning. Overall, these findings suggest that the established computational methods of LSC detection are, in fact, sensitive to the emergence and decline
of dogwhistles. Exceptions to this general observation are found among variants of the SGNS models, where the coefficient for ∆IORti,tj (w) is not significant (at α = 0.05); namely those with window size = 10. We return to this pattern below.
Out of the pipelines, the LLM-based pipelines explain more variability of the data and have larger coefficients for ∆IORti,tj , than the SGNS-based models. Thus, in predicting semantic change, these models rely more on the semantic variability related to the IOR, than the SGNS models do. When comparing LLM-based pipelines, the SBERT-based approach shows higher R2 and a stronger effect of ∆IOR than the BERT and mT5 approaches. For SBERT, the strong correlation between LSC and IOR is illustrated in Figure 1. These observations suggest that sentence embeddings are beneficial for explaining the semantic
change of dogwhistles (SBERT), compared with averaging over the embeddings of input tokens mapping to the DWE (BERT, mT5). Note that these findings derive from pipelines that contain both the rate of change and the IOR. Thus, the different observed can be a consequence of how replacements are represented, how LSC is modeled, or both.
An explanation for why SBERT explains more variability in the data might be that SBERT is finetuned for a task that has a similar structure as the one implemented in our pipeline for modeling ingroup and out-group scores, namely to predict the similarity of embeddings (Reimers and Gurevych, 2019). It might also be the case that in-group and out-group meanings of DWEs are best captured holistically by sentence representations that give more prominence to the full context of DWEs.
The pipelines with BERT and mT5 are very similar in terms of R2 and effect of ∆IOR. On the one hand, the large computational overhead of mT5-XL compared to BERT does not result in stronger predictions, as modeled in the present context. On the other hand, the multilingual transformer performs on par with the language-specific one.
For the SGNS models, both the window size and the number of dimensions of the vectors matter. With higher dimensionality of the vectors, more variation in ∆ti,tj (w) is explained. When different window sizes are compared, a U-shaped pattern emerges. For both 100- and 200-dimensional models, the strongest effect of ∆IOR and the highest values of R2 are observed for window size = 5. However, almost as strong effects are found for
window size = 15, but smaller effect sizes for window size = 10. These observations indicate that words used both in close proximity and far away from the DWE are relevant to communicate ingroup messages. This U-shaped pattern may be related to the fact that we model different DWEs. That is, for some DWEs, words in close context may be central to the in-group meaning, but for other DWEs, a wider context is important.
As in previous studies, term frequency (at ti) explains the rate of semantic change (Hamilton et al., 2016b; Dubossarsky et al., 2017). For LLMs, the relationship is negative: the more frequent a word is, the less it changes, which is in line with “law of conformity” (Hamilton et al., 2016b). However, for the SGNS models, the relation between term frequency and semantic change is in most cases not significant; and when significant, the relationship is, unlike for the LLM pipelines, positive.The change in frequency from ti to tj have no effect on ∆ti,tj (w), besides the case of SGNS, where window size = 5 and d = 100.12 In both models for the BERT-pipelines, ∆IORti,tj (w) has a stronger effect on ∆ti,tj than term frequency, while for mT5 pipeline, the effect of IOR and term frequency are in the same magnitude (though the latter is negative).

heading: 6 Discussion
text: We find that the observed meaning shifts for DWEs using distributional methods are explained by their in-group and out-group dimensions. That is, the methods for detecting LSC are sensitive to the dynamic meaning of DWE, suggesting that the measures of LSC could be used to detect dogwhistles online. However, it could have been the case that LSC measures did pick up on contextual drifts of DWEs, which were not directly related to their function as dogwhistles. After all, as an implementation of the distributional hypothesis, meaning is in LSC detection modeled as statistical correlation over context words.
But context can vary for various reasons, not all of which are straightforward cases of change in meaning (Bender and Koller, 2020). Words can be used in the same sense in relation to different topics of discussion at different times, which poses challenges for modeling meaning change (Hengchen et al., 2021; Tang, 2018). For example, previous
12Other operationalisation of change in term frequency (than percental difference) were tested: (non-proportional) raw change in frequency and absolute difference of frequency, but the overall pattern persists: no effect for predicting ∆ti,tj (w).
work has showed that distributional methods for LSC sometimes overgeneralizes due to “referential effects”, i.e., the observed change of word usage is explained by reference to different persons or events at different times (Del Tredici et al., 2018). In such cases, “the meaning of the word stays the same, despite the change in context” (Del Tredici et al., 2018, 2073). These types of “semantic” (or contextual) shifts are not clear examples of meaning change or differentiation of senses that have been mainly discussed in theoretical linguistics (Traugott and Dasher, 2002). But from the point of view of distributional semantics, it is difficult to distinguish these different aspects of variable usage (Geeraerts et al., 2024). Given a strict interpretation of the distributional thesis, a change in context is a change of meaning.
In the context of these potential challenges that have been raised for the interpretation of distributional LSC detection results, our results are notably interpretable. The rate of change of the DWEs is, in fact, related to changes in the in-group vs. outgroup “senses” of these words. From the geometric viewpoint that defines distributional modeling of meaning, the shifting positions of DWEs in semantic space over time (as identified by LSC) are repositioning along the in-group vs. out-group axes (as identified by ∆IOR). Given the high values of R2, for many of the pipelines tested here, the IOR of the DWEs is a key factor in explaining their semantic variability over time.
The above findings suggest that the pipelines with LLMs are better than the SGNS models at the relevant meaning variation of DWEs. This finding is in line with the general trend, with transformer models having substantially improved the stateof-the-art for NLU tasks. The nuanced semantic representation enabled by these models seems to be important also for the related challenge of modeling dogwhistle meaning.
Future research should attempt to scale up the present approach for the analysis of a wider range of DWEs. A key challenge in doing so is inferring and representing the in-group and out-group dimensions of the DWEs. This study used a survey methodology to develop an independent basis for defining the in-group and out-group dimensions of DWEs, but such an approach is costly, especially at a large scale. Another possibility for future research is using definitions of dogwhistles in existing online databases to represent in-group and out-group embeddings (Mendelsohn et al., 2023).
Limitations
This work applies to the political media context of Sweden. Although we believe that the general methodologies developed should also apply to other national, linguistic, and political contexts, this must be tested in other work.
Since DWEs emerge and disappear on the basis of politically relevant current affairs, it is not possible to develop a sample of relevant DWEs that allows analysis of DWEs themselves as a general category. As a result, our work shows our hypothesis for an admittedly limited set of dogwhistles from which we cannot make global generalizations. However, the fact that the effects are strong is a contribution that calls for future testing of the methodology at a larger scale, with additional terms, and in other national contexts.
Ethics Statement
When creating a system that detects potentially negative social phenomena, there is always a risk of malicious use of the system. In principle, the developed technology can be used for evaluating, for example, attempts to manipulate political discourse. However, we believe that actors motivated to do so can do so anyway and that public research should not avoid the analysis of harmful communication for this reason. Rather, tools should be developed to detect and combat these harmful phenomena. In addition, this work is part of the foundational work that contributes to understanding dogwhistle communication; it does not enable full detection on its own.
The corpus data used in this project were obtained from a national repository given responsibility for archiving Swedish documents of political and cultural significance. The replacement test survey was approved by the Swedish Ethical Review Authority.

heading: Acknowledgements
text: Funding for this work was provided by the Gothenburg Research Initiative for Politically Emergent Systems (GRIPES) supported by the Marianne and Marcus Wallenberg Foundation grant 2019.0214 as well as a Swedish Research Council (VR) grant (2014-39) for the Centre for Linguistic Theory and Studies in Probability (CLASP). We wish to thank the anonymous reviewers for their constructive comments.

heading: B Example sentences
text: Warning: the following examples may be upsetting or offensive to some readers.
Examples are selected from the training corpus (Flashback) to illustrate high and low IOR values from years with high and low general IOR values, as measured by BERT. To identify examples, IOR values for individual sentences were computed. That is, we compute IORt(w), as defined above, but where −→w is not the diachronic embedding of t, but the embedding of a word instance from the time bin t. We show examples from the top (“high IOR”) and bottom (“high IOR”) five of the sentences of a year, measured by their individual IOR value.
B.1 berika
1. (low IOR, 2007) den typen av invandring är bra och berikande och bör uppmuntras (that kind of immigration is good and enriching and should be encouraged)
2. (high IOR, 2010) kålsvart hår och mörkt hudpigment, troligen hemmavarande i Iran eller Irak, varför måste vi skandinaver berikas med detta drägg? (coal black hair and dark skin pigment, probably native to Iran or Iraq, why do we Scandinavians have to be enriched with this dreg?)
B.2 globalist
3. (low IOR, 2006) jag är alltså globalist, frihandelsförespråkare, demokrat och kapitalist för att detta är det bästa sättet att göra fattiga
människor rikare och utvecklar alla länder som ingår i handelsutbytet (so I am a globalist, free trade advocate, democrat and capitalist because this is the best way to make poor people richer and develop all countries that are part of the trade exchange)
4. (high IOR, 2008) globalist-maffian med judarna i spetsen har ju mer eller mindre full kontroll över Amerika, och därmed har dom tillgång till världens starkaste armé (the globalist mafia with the Jews at the head has more or less full control over America, and thus they have access to the world’s strongest army)
B.3 återvandring 5. (low IOR, 2011) de flesta invandrar p.g.a
studier, arbete, återvandring eller för att de har anhöriga i Sverige (most people immigrate due to studies, work, re-migration or because they have relatives in Sweden)
6. (high IOR, 2018) återvandring och utvisning nu, det är enda lösningen (re-migration and deportation now, that is the only solution)
B.4 förortsgäng 7. (low IOR, 2014) kan tillägga att vi var ett
helsvenskt förortsgäng med 50 % skinnskallar
och 50 % fotbollshuliganer (can add that we were an all-Swedish suburban gang with 50 % skinheads and 50 % football hooligans)
8. (high IOR, 2015) ett passivt / slappt invandrarflöde orsakar sånt, och man måste aktivt minska folkvandringen som bosätter sig i förorterna om man vill bli av med förortsgäng (a passive / slack immigrant flow causes that, and you have to actively reduce the migration of people settling in the suburbs if you want to get rid of suburban gangs)

heading: C Transformations
text: To maximize the number of models having normal distribution of residuals, we tested combinations of log transformation of variables. The log transformation of the dependent variable and of the ∆IORti,tj resulted in normally distributed residuals for all models but BERT and mT5-XL. No combination of transformed variables was found that makes the error term normally distributed for all models. The regression models for the transformed data are shown in Table 4.

heading: Future Work
text: Future research should attempt to scale up the present approach for the analysis of a wider range of DWEs. A key challenge in doing so is inferring and representing the in-group and out-group dimensions of the DWEs. This study used a survey methodology to develop an independent basis for defining the in-group and out-group dimensions of DWEs, but such an approach is costly, especially at a large scale. Another possibility for future research is using definitions of dogwhistles in existing online databases to represent in-group and out-group embeddings (Mendelsohn et al. , 2023). Limitations
This work applies to the political media context of Sweden. Although we believe that the general methodologies developed should also apply to other national, linguistic, and political contexts, this must be tested in other work. Since DWEs emerge and disappear on the basis of politically relevant current affairs, it is not possible to develop a sample of relevant DWEs that allows analysis of DWEs themselves as a general category. As a result, our work shows our hypothesis for an admittedly limited set of dogwhistles from which we cannot make global generalizations. However, the fact that the effects are strong is a contribution that calls for future testing of the methodology at a larger scale, with additional terms, and in other national contexts