Modeling Profanity and Hate Speech in Social Media with Semantic Subspaces (2021)

Abstract: Hate speech and profanity detection suffer from data sparsity, especially for languages other than English, due to the subjective nature of the tasks and the resulting annotation incompatibility of existing corpora. In this study, we identify profane subspaces in word and sentence representations and explore their generalization capability on a variety of similar and distant target tasks in a zero-shot setting. This is done monolingually (German) and cross-lingually to closely-related (English), distantly-related (French) and non-related (Arabic) tasks. We observe that, on both similar and distant target tasks and across all languages, the subspace-based representations transfer more effectively than standard BERT representations in the zero-shot setting, with improvements between F1 +10.9 and F1 +42.9 over the baselines across all tested monolingual and cross-lingual scenarios.

## introduction profanity and online hate speech have been recognized as crucial problems on social media platforms as they bear the potential to offend readers and disturb communities. the large volume of usergenerated content makes manual moderation very difficult and has motivated a wide range of natural language processing (nlp) research in recent years. however, the issues are far from solved, and the automatic detection of profane and hateful contents in particular faces a number of severe challenges. pre-trained transformer-based @xcite language models, e.g. bert @xcite , play a dominant role today in many nlp tasks. however, they work best when large amounts of training data are available. this is typically not the case for profanity and hate speech detection where few datasets are currently available @xcite @xcite with moderate sizes at most. in addition, these tasks are known to be highly subjective @xcite . annotation protocols for hate speech and profanity often rely on different assumptions that make it non-trivial to combine multiple datasets. in addition, such datasets only exist for few languages besides english @xcite @xcite . for such low-resource scenarios, few-and zeroshot transfer learning has seen an increased interest in the research community. one particular approach, using semantic subspaces to model specific linguistic aspects of interest @xcite , has proven to be effective for representing contrasting semantic aspects of language such as e.g. positive and negative sentiment. in this paper, we propose to learn semantic subspaces to model profane language on both the word and the sentence level. this approach is especially promising because of its ability to cope with sparse profanity-related datasets confined to very few languages. profanity and hate speech often co-occur but are not equivalent, since not all hate speech is profane (e.g. implicit hate speech) and not all profanity is hateful (e.g. colloquialisms). despite being distantly related tasks, we posit that modeling profane language via semantic subspaces may have a positive impact on downstream hate speech tasks. we analyze the efficacy of the subspaces to encode the profanity (neutral vs. profane language) aspect and apply the resulting subspace-based representations to a zero-shot transfer classification scenario with both similar (neutral/profane) and distant (neutral/hate) target classification tasks. to study their ability to generalize across languages we evaluate the zero-shot transfer in both a monolingual (german) and a cross-lingual setting with closely-related foot_0 (english), distantly-related (french) and non-related (arabic) languages. we find that subspace-based representations outperform popular alternatives, such as bert or word embeddings, by a large margin across all tested transfer tasks, indicating their strong generalization capabilities not only monolingually but also cross-lingually. we further show that semantic subspaces can be used for word-substitution tasks with the goal of generating automatic suggestions of neutral counterparts for the civil rephrasing of profane contents. conversely, @xcite show that the methods in @xcite are not able to identify and remove the gender bias entirely. following this, @xcite argue that semantic features such as gender are encoded nonlinearly, and suggest an iterative approach to identifying and removing gender features from semantic representations entirely. addressing the issue of data sparseness, @xcite use ultradense subspaces to generate task-specific representations that capture semantic features such as abstractness and sentiment and show that these are especially useful for lowresourced downstream tasks. while they focus on using small amounts of labeled data of a specific target task to learn the subspaces, we focus our study on learning a generic profane subspace and test its generalization capacity on similar and distant target tasks in a zero-shot setting. zero-shot transfer, where a model trained on a w (profane) ŵ (neutral) arschloch [asshole] mann [man] fotze @xcite frau [woman] hackfresse @xcite mensch @xcite ## method: semantic subspaces a common way to represent word-level semantic subspaces is based on a set p of so-called minimal pairs, i.e. n pairs of words (w, ŵ) that differ only in the semantic dimension of interest @xcite . we diverge from this approach in four ways: normalization we note that there is no convincing justification for the normalization step. as our experiments in the following sections show, we find that the profanity subspace is better represented by p than by p . for our experiments, we thus distinguish three different types of representations: here, projecting a vector representation r onto a subspace is defined as the dot product r • s(p ). ## word-level subspaces before moving to the lesser explored sentence-level subspaces, we first verify whether word-level semantic subspaces can also capture complex semantic features such as profanity. ## sentence-level subspaces in section 5, we identified profane subspaces on the word-level. however, abuse mostly happens on the sentence and discourse-level and is not limited to the use of isolated profane words. therefore, we move this method to the sentence-level, exploring the two subspace-based representation types pca-raw and pca-norm. concretely, we learn sentence-level profane subspaces that allow a context-sensitive representation and thus go beyond isolated profane words, and verify their efficacy to represent profanity. similarly to the word-level experiments, we focus our analysis on the ability of the subspaces to generalize to similar (neutral/profane) and distant (neutral/hate) tasks. we compare their performance with a bert-encoded base representation, which does not use a semantic subspace. ## conclusion and future work in this work, we have shown that a complex feature such as profanity can be encoded using semantic subspaces on the word and sentence-level. on the word-level, we found that the subspacebased representations are able to generalize to previously unseen words. using the profane subspace, we were able to substitute previously unseen profane words with neutral counterparts. on the sentence-level, we have tested the generalization of our subspace-based representations (pca-raw, pca-norm) against raw bert representations (base) in a zero-shot transfer setting on both similar (neutral/profane) and distant (neutral/hate) tasks. while the base representations failed to zero-shot transfer to the target tasks, the subspace-based representations were able to perform the transfer to both similar and distant tasks, not only monolingually, but also to the closelyrelated (english), distantly-related (french) and non-related (arabic) language tasks. we observe major improvements between f1 +10.9 (pca-raw on de-dt) and f1 +42.9 (pca-norm on fr-dt) over the base representations in all scenarios. as our experiments have shown that the commonly used mean-shift normalization is not required, we plan to conduct further experiments using unaligned significant words/sentences. the code, the fine-tuned models, and the list of minimal-pairs are made publicly available foot_4 .