Silent Signals, Loud Impact: LLM s for Word-Sense Disambiguation of Coded Dog Whistles (2024)

Abstract: A dog whistle is a form of coded communication that carries a secondary meaning to specific audiences and is often weaponized for racial and socioeconomic discrimination. Dog whistling historically originated from United States politics, but in recent years has taken root in social media as a means of evading hate speech detection systems and maintaining plausible deniability. In this paper, we present an approach for word-sense disambiguation of dog whistles from standard speech using Large Language Models (LLMs), and leverage this technique to create a dataset of 16,550 high-confidence coded examples of dog whistles used in formal and informal communication. Silent Signals is the largest dataset of disambiguated dog whistle usage, created for applications in hate speech detection, neology, and political science.

## related work hate speech prior work categorizes abusive language with two dimensions: target specificity (directed or generalized) and explicitness (explicit or implicit) @xcite . in addition to detecting of explicit language @xcite , recent work also labels, detects, and explains the latent meaning behind implicitly abusive language @xcite @xcite @xcite , but these works do not primarily focus on dog whistles at scale. ## conclusion dog whistles are used to promote discrimination in both formal and informal environments. the use of this coded language allows speakers to maintain plausible deniability and bypass hate speech detection systems when used online. this work presents the largest, to date, survey of llm capabilities with respect to the automatic resolution of dog whistles. experimental results demonstrate that llms remain unreliable in the dog whistle resolution task. a hindrance to research in this space has been the unavailability of large datasets of coded dog whistles examples. we show that despite the overall inconsistencies of llms on the automatic dog whistle resolution task, with the proper methodology, they are adept at disambiguating coded dog whistles from standard language. we leverage this capability to create the silent signals dataset which contains 16,550 dog whistle examples and 298 distint dog whistles. we believe that this resource will be integral to the continued study of dog whistles with applications in content moderation, computational social science, and political science, on tasks such as analysis of trends in dog whistle use, dog whistle resolution, hate speech detection, and identification of emergent dog whistles.