Features or Spurious Artifacts? Data-centric Baselines for Fair and Robust Hate Speech Detection (2022)

Abstract: Avoiding to rely on dataset artifacts to predict hate speech is at the cornerstone of robust and fair hate speech detection. In this paper we critically analyze lexical biases in hate speech detection via a cross-platform study, disentangling various types of spurious and authentic artifacts and analyzing their impact on out-of-distribution fairness and robustness. We experiment with existing approaches and propose simple yet surprisingly effective data-centric baselines. Our results on English data across four platforms show that distinct spurious artifacts require different treatments to ultimately attain both robustness and fairness in hate speech detection. To encourage research in this direction, we release all baseline models and the code to compute artifacts, pointing it out as a complementary and necessary addition to the data statements practice.

## introduction hate speech in online social communities is a serious and pervasive concern, which requires fair and robust automated approaches to be tackled at scale. however, despite the great progress in natural language processing for detecting hate speech, current models have shown to be brittle when applied to real-world data, exhibiting limited out-ofdistribution (ood) robustness @xcite and perpetuating and amplifying harmful social biases @xcite . noticeably, hate speech detection systems are typically trained on data from limited language varieties such as individual platforms, which inevitably exhibit differences in writing norms, language use, and hate targets, hampering generalization @xcite . one of the main reasons for limited robustness and fairness of mainstream hate speech detection fair robust what have jews done to you? rt @xcite : i'm mad at this all black people literally go there previous work has shown that state-of-the-art models overly rely on identity words (e.g., "jews", "gay") to predict hateful content @xcite , further demoting voices of people from already marginalized groups @xcite . however, non identity-related lexical items -such as "sport", "announcer", and "football" in @xcite -are also often spuriously associated with hate speech due to a biased data collection process @xcite , undermining ood robustness. despite the recent trend in minimizing topic bias in data sampling, we show that some spurious lexical artifacts still remain highly-predictive on certain distributions even if data has been sampled in a more attentive fashion (e.g., artifacts that are potentially data-or platform-specific -figure 1 , highlighted in gray). we argue that disentangling artifacts into finegrained categories by means of a cross-platform analysis may be beneficial to drive a broader debiasing of current hate speech models, ultimately improving both fairness and robustness to out-ofdistribution data. to this purpose, we critically analyze artifacts in hate speech detection across multiple platforms and propose simple yet effective data-centric baselines exploiting spurious lexical items. we show that although we achieve substantial improvements in ood fairness by exploiting spurious identity-related artifacts, this comes at the cost of robustness. this confirms that fairness and robustness are strictly interrelated dimensions that should be studied together in future research. contributions to the best of our knowledge, we are the first to (i) conduct a thorough investigation of lexical artifacts across online platforms; (ii) disentangle artifacts into fine-grained categories; and (iii) propose a viable data-centric approach based on masking that consistently improves fairness over all baselines across all platforms. to foster future research on the topic, we also release (iv) code to reproduce all experiments, and (v) disaggregated lexical artifact annotations, more broadly (vi) suggesting the inclusion of dataset artifacts in data statements @xcite , which can be easily revealed using our codebase. ## lexical artifacts are not all the same we conceptualize dataset artifacts at the lexical level as emergent correlations between tokens and labels in input data, consistently to lexical annotation artifacts in nli @xcite . as such, given a target class c, we formally define lexical artifacts l c as the set of highly-discriminating foot_1 tokens for c, which comprise authentic artifacts a c -items that potentially carry useful information for the class at hand -and spurious artifacts s c -items that are spuriously (or undesirably) associated to the target class -such that l @xmath0. in the context of hate speech detection, we consider the hateful class as c unless otherwise specified and simplify the notation (i.e., from "• c " to "•"). we build our definitions on top of the categories of lexical biases by @xcite , which originally identify three bias groups: i) minority identity mentions which are not offensive, ii) minority identity mentions which are potentially offensive, and iii) non-identity mentions which are possibly offensive. we enrich this categorization by introducing a high-level separation into spurious (i.e., group i) in @xcite ) and authentic artifacts (i.e., group ii) and iii)), and including an additional spurious, non identity-related category (section 2.2). indeed, given the broad nature of authentic and spurious artifacts, we further categorize them in section 2.1 and 2.2 (see figure 1 for an overview). ## data in this work we focus on hate speech, i.e., messages whose content spreads hatred or incites violence, or threatens people's freedom, dignity and safety, and whose target is a protected group, or an individual targeted for belonging to such a group and not for his/her individual characteristics @xcite . hate speech typically encompasses serious cases of offense with severe moral and legal implications, i.e., those cases that are of primary importance for content moderation. we collect hate speech corpora that meet the following criteria: (i) they minimize topic and author biases in data collection @xcite , using alternatives to keyword and user searches such as pure or boosted random sampling, (ii) they pertain to different social media platforms, and (iii) they follow similar annotation guidelines, where hate speech is clearly defined and separated from other types of offensive language. for each corpus we create hateful and non-hateful examples. all datasets follow consistent preprocessing, deduplication, and anonymization (appendices a.1 and a.2). ## disentangling lexical artifacts in order to disentangle lexical artifacts, we first compute the correlation between each token and the hateful class for each dataset (section 4.1), then assessing the cross-distribution indicativeness of each token (section 4.2). for segmenting texts into tokens, we rely on the training portion of each dataset only (section 5) and employ the wordpiece (schuster and nakajima, 2012) subword tokenizer as used in bert @xcite . 5 finally, we perform lexical artifacts annotation (section 4.3) following the categories defined in section 2. ## experiments we investigate the impact of spurious lexical artifacts on fairness and robustness in hate speech detection. similarly to previous studies @xcite , we cast hate speech detection as a binary classification problem, where the two classes to be predicted are hateful and non-hateful, as defined in section 3. we carry out in-distribution and ood experiments, namely training and testing all models on the same or different platform data, respectively. we evaluate performance of models using macro f 1 score, whereas for fairness we use false positive rate (fpr) on test instances containing s i mentions, consistently to previous work @xcite . we outline the experimental setup in section 5.1, whereas data-centric baselines are presented in section 5.2. lastly, we present results and a thorough discussion in section 5.3. ## towards artifacts documentation the practice of data statements @xcite has been recently adopted by the nlp community as a way to include relevant information about the creators, the methodology and possible biases when a dataset is released. this should in turn have a positive impact on systems trained on such data, contributing to a better evaluation of models' generalization and fairness. we propose that an artifacts statement should be added to this documentation as a way to contribute to diagnosis (and thus mitigation) of pre-existing bias, which is also one of the goals of data statements. in particular, we propose a template for lexical artifacts documentation and publicly release code to easily compute ranked correlations between tokens and target classes of interest for a given annotated corpus. to ensure the process of documenting lexical artifacts will be as smooth as possible -and thus allows widespread adoption of artifacts statement in the future -our code automatically generates outputs in different formats, from raw text to latex code for seamless inclusion in publications. we present the artifacts statement template below, and provide a full example in appendix d. i) top lexical artifacts. which are the k most informative tokens in the corpus for the class(es) of interest? this can be a ranked list of (@xmath1, optionally along with associated scores. if there are multiple classes of interest, top k lexical artifacts for each class should be included. ii) class definitions. different definitions for the same class may exist across datasets. this impacts the annotation, which in turn has an effect on resulting lexical artifacts. an explicit definition of the target class(es) for which the top lexical artifacts are computed should be provided here. ## related work the problem of models' generalizability related to hate speech detection has been extensively discussed in recent works @xcite @xcite . indeed, it has been shown that state-of-the-art performance on this task overestimates the capability of models to yield the same results over time @xcite or across different domains @xcite . possible mitigation strategies include domain adaptation techniques @xcite , augmenting smaller datasets with a larger dataset from a different domain @xcite , the use of a domain lexicon to transfer knowledge across domains (pamungkas and patti, 2019) and the fine-tuning of hatebert @xcite on the target corpus @xcite , among others. concerning bias and fairness, several works have pointed out the presence of bias in hate and abusive language datasets @xcite @xcite . this issue has been addressed in different ways, including functional tests for hate speech detection models @xcite and post-hoc explanations to measure models' bias towards identity terms @xcite . as regards bias mitigation, the task has been addressed through a number of approaches, e.g., via adversarial feature learning @xcite , by using debiased word embeddings and gender swap data augmentation @xcite or by adding non-toxic examples to better balance the data @xcite . the work probably most related to ours is @xcite , which presents an analysis of lexical and dialectal biases in the dataset by @xcite . the authors propose lexical bias categories which we extend in this work (see section 2). however, they focus only on one dataset and on indomain bias reduction. moreover, they start from a list of "bad words", whereas we compute it from data. to our knowledge, this is the first work advocating for a joint view on fairness and robustness, both identified as critical aspects related to the classification of hate speech @xcite . ## limitations our work is a step forward towards a better understanding of the bias that can be encoded in hate speech detection corpora @xcite . however, we are aware of some limitations. first, all findings in this work are related to hate speech datasets in english. with the increasing availability of hate speech data in languages other than english, we aim to investigate our methods on other languages too. second, annotated data from multiple platforms may not be available for some languages, and this can limit the cross-distribution computation of artifacts. lastly, we acknowledge spurious statistical correlations may go beyond the token level. we believe our study is a first step towards contextual debiasing from spurious lexical artifacts, and thus can be of inspiration for future studies. ## conclusion and future directions this paper investigates the impact of lexical artifacts on out-of-distribution fairness and robustness in hate speech detection, raising awareness on the interplay between the two dimensions that should be studied together in future work. we propose a fine-grained categorization of lexical artifacts and simple yet effective data-centric baselines, show-ing that while robustness calls for model-centric approaches, masking spurious identity artifacts is a viable approach that we argue should be used as strong baseline for fairness assessment in future research. in future work we aim to investigate the role of dialectal biases and non-lexical artifacts, extending the study on languages other than english. we release all baseline models, resources, and the code to compute lexical artifacts, broadly suggesting the inclusion of "artifacts statement" as a way to document potential lexical biases when a dataset is released, to provide a complementary view to data statements @xcite .