Hiding in Plain Sight: Tweets with Hate Speech Masked by Homoglyphs (2023)

Abstract: To avoid detection by current NLP monitoring applications, progenitors of hate speech often replace one or more letters in offensive words with homoglyphs, visually similar Unicode characters. Harvesting real-world hate speech containing homoglyphs is challenging due to the vast replacement possibilities. We developed a character substitution scraping method and assembled the Offensive Tweets with Homoglyphs (OTH) Dataset (N=90,788) with more than 1.5 million occurrences of 1,281 non-Latin characters (emojis excluded). In an annotated sample (n=700), 40.14% of the tweets were found to contain hate speech. We assessed the performance of seven transformer-based hate speech detection models and found that they performed poorly in a zero-shot setting (F1 scores between 0.04 and 0.52) but normalizing the data dramatically improved detection (F1 scores between 0.59 and 0.71). Training the models using the annotated data further boosted performance (highest micro-averaged F1 score=0.88, using five-fold cross validation). This study indicates that a dataset containing homoglyphs known and unknown to the scraping script can be collected, and that neural models can be trained to recognize camouflaged real-world hate speech.

## introduction disclaimer: this paper includes language that some readers might find offensive. hate speech, discriminatory language against individuals or groups based on race/ethnicity, nationality, gender, religion, lgbtq+ identity, or disability status, is banned by facebook, youtube, and other major platforms. a common strategy to mask hate speech is replacing one or more letters in offensive words with homoglyphs, unicode characters that are visually homogeneous @xcite . for instance, the latin "a" (u+0061) and the cyrillic "а" (u+0430) are nearly indistinguishable to the human eye, yet they belong to different unicode character families. currently, there are almost 150,000 unicode characters, foot_1 which presents extensive substitution possibilities. despite the prevalence of online hate speech containing homoglyphs, the collection of organically generated data of this type is rare. to remedy this scarcity, we developed a dataset of real-world offensive text containing homoglyphs and used this data to train existing hate speech detection models. in particular, our contributions are: ## related work previous work that explored human-generated text with homoglyphs includes the anthro algorithm, which evaluated the phonetic properties of words to locate character-based perturbations, including homoglyphs @xcite . @xcite used homoglyphs, letter reordering, and letter deletion to test the detection capabilities of nlp models. additionally, @xcite artificially produced offensive text containing homoglyphs to simulate adversarial attacks. @xcite investigated obfuscation mitigation using a siamese convolutional neural network to convert homoglyphs into the characters they resembled. similarly, @xcite proposed a "visual scanning" method to detect and predict homoglyphs. other work has inventoried homoglyphs @xcite and identified previously unknown homoglyphs @xcite . finally, several studies have evaluated phishing attacks in which homoglyphs were used to imitate corporate domain names to deceive users and extract personal information @xcite @xcite . these studies involved training models using artificially created data containing homoglyphs and did not evaluate real-world content. ## discussion our character substitution scraping method yielded 90,788 tweets containing more than 1.5 million occurrences of 1,281 non-latin characters (emojis excluded). the search strategy used cyrillic homoglyphs for latin letters to assemble a broader collection of non-latin characters. as expected, the bulk of the oth dataset (82.82%) was comprised of cyrillic homoglyphs. however, 262,319 occurrences of 1,254 other non-latin characters were also captured, including 72 additional cyrillic characters and 1,182 characters from other unicode character families. the most common cyrillic characters were "а" (u+0430), "е" (u+0435), and "о" (u+043e), which are homoglyphs for the latin letters "a", "e", and "o." the most common non-cyrillic characters were "α" (u+03b1), "í" (u+00ed), and "σ" (u+03c3), which are homoglyphs for the latin letters "a", "i", and "o". these results may reflect malicious users' preference for homoglyphs that mimic latin vowels. the noise produced by the homoglyphs impeded the performance of all seven hate speech detection models tested. fully normalizing the data by replacing all non-latin characters with the corresponding latin characters dramatically improved model performance. most notably, model 1's f1score jumped from 0.04 to 0.60. in the five-fold cross validation, five models achieved f1-scores that exceeded 0.85. conversely, model 2 performed poorly (f1-@xmath0.41). this may be related to the model's original training data, which included only hate speech labeled as lgbtq+ identity and nationality-categories of hate speech which were rare within the annotated sample in the present study. the performance of the other six models demonstrates that neural classifiers can be trained to recognize real-world hate speech masked by homoglyphs. this is an exciting result, considering that the dataset contains many homoglyphs unknown to the scraping script, and thus these perturbations could not be addressed through deterministic normalization. the annotated sample analyzed in the present study included a 40%-60% split between tweets with and without hate speech. the large volume of hate speech tweets with misogynistic content (87.54%) found in the annotated sample is notable. this result is consistent with a prior survey of twitter that found 419,000 female slurs were posted on average each day @xcite . homoglyphs also appeared to interfere with twitter's internal system that classifies the body text language of tweets. in the annotated sample, twitter assigned the english language identifier to only 21.00% of tweets, but human annotators found that 92.43% of the tweets were written in english. twitter classified the majority of tweets in the annotated sample (60.50%) and the oth dataset (62.04%) as czech. these results are especially interesting because the czech language does not include cyrillic characters, which were used to generate the query terms. ## conclusion infusing hate speech with homoglyphs was found to be an effective strategy to avoid detection. in the present study, existing hate speech detection models were ineffective against obfuscation of this type. the oth dataset offers training data to build hate speech detection models inoculated against real-world homoglyphs.