Challenges in Automated Debiasing for Toxic Language Detection (2021)

Abstract: Biased associations have been a challenge in the development of classifiers for detecting toxic language, hindering both fairness and accuracy. As potential solutions, we investigate recently introduced debiasing methods for text classification datasets and models, as applied to toxic language detection. Our focus is on lexical (e.g., swear words, slurs, identity mentions) and dialectal markers (specifically African American English). Our comprehensive experiments establish that existing methods are limited in their ability to prevent biased behavior in current toxicity detectors. We then propose an automatic, dialect-aware data correction method, as a proof-of-concept. Despite the use of synthetic labels, this method reduces dialectal associations with toxicity. Overall, our findings show that debiasing a model trained on biased toxic language data is not as effective as simply relabeling the data to remove existing biases.

## introduction current hate speech or toxic language detection 1 systems exhibit problematic and discriminatory behavior that causes them to have disparate negative impact on minority populations @xcite @xcite . tweets simply containing a minority identity mention are commonly flagged as toxic by current systems, in contrast to those containing majority identity mentions, as illustrated in figure 1 . at the core of the issue are dataset biases, i.e., spurious correlations between surface patterns and annotated toxicity labels ( §2), which stem from the data creation process @xcite . previous work has outlined two such biases for hate 1 we use hate speech and toxic language interchangeably in this work, though their definitions do not perfectly align. detected toxicity score i identify as a black gay woman. i identify as a straight white man. fucking love this. adolf hilter is a great person. identity bias (lexical) swear word bias (lexical) what's up, bro! wussup, n*gga! dialect/ racial bias pers. api pers. ## biases in toxic language detection we test the use of debiasing foot_1 methods for the task of toxic language detection, which aims to flag rude, offensive, hateful, or toxic language on the internet, with the goal of moderating online communities @xcite . this task differs in several ways from the natural language understanding (nlu) tasks that debiasing methods have been successful on, such as textual entailment (e.g., snli, mnli; @xcite or reading comprehension (e.g., squad; @xcite . first, compared to these nlu tasks where there is one correct label, the toxicity of language is inherently more nuanced, subjective, and contextual, which causes toxic language datasets to have lower agreement in general @xcite . second, the dataset biases in nlu are predominantly artifacts introduced during data creation (e.g., negations, exaggerations; @xcite , whereas those in toxic language detection are grounded in the social dynamics of the world @xcite . for example, viewing aae as a more toxic or less proper variety of english is a form of linguistic discrimination that upholds racial hierarchies in the united states @xcite . in this work, we consider two broad categories of toxic language dataset biases-lexical ( §2.1) and dialectal ( §2.2). our experiments focus on a single, widely used dataset ( §2.3) from @xcite . ## debiasing methods we consider two types of debiasing methods from current literature. the first type addresses known, pre-defined biases-such as lexical and dialectal biases for hate speech detection, via a modelbased approach involving additional training objectives ( §3.1). in contrast, the second type is agnostic to prior knowledge about biases, and instead filters out examples that appear "too easy" and might hence contain spurious correlations ( §3.2). ## experiments: lexical biases we investigate the effect of debiasing approaches ( §3) on removing lexical biases in hate speech detection. first, we discuss the evaluation framework for measuring bias reduction ( §4.1). we present quantitative ( §4.2) and qualitative ( §4.3) results on lexical bias removal for all debiasing approaches, and ood evaluation for debiased training methods ( §4.4). see appendix a.3 for hyperparameters and other experimental settings. ## experiments: dialectal and racial biases we test the efficacy of the bias reduction methods from §3 for dialectal bias ( §2.2) reduction. ## towards data relabeling based on our quantitative and qualitative analyses, we believe there still is room for improvement in debiasing hate speech detection. therefore, we turn our attention to the role of label noise in datasets. partly inspired by our qualitative analyses of debiased models' predictions, we design a proof-of-concept study where we automatically correct the label of tweets using a(n automatic) dialectal translation of the tweet, inspired by previous work showing that highlighting aae tweets' dialect led them to be labeled as less toxic @xcite . we conclude this study by discussing the limitations and ethical implications of the synthetic data, and cautioning against its real-world application. focusing on dialectal bias, our key assumption is that an aae tweet and its corresponding wae version should have the same toxicity label, therefore toxic aae tweets whose wae versions are non-toxic are candidates for label correction. 14 however, gold-standard translations of aae to wae would require qualified translators, and automatic aae-to-wae translation systems do not exist, to the best of our knowledge. therefore, we create a proof-of-concept study-we set up a aae to wae "translation" system using the fewshot capabilities of the gpt-3 language model @xcite . under this mechanism, we prompt gpt-3 with four translation pairs (taken from spears, 1998) and an aae tweet from our training data, and generate its wae "translation". the list of prompts, as well as further details, are provided in appendix c. note that we do not recommend this approach to build large scale parallel data for dialects, as discussed under ethical implications and limitations. next, as per our heuristic, we only relabel toxic aae tweets whose wae translation is predicted as non-toxic by either our vanilla classifier trained on the original @xcite dataset, or an identical classifier trained on the wae translated tweets. the resulting dataset (aae-relabeled) is the same size as the original dataset, but with 954 (12%) out of 8260 toxic aae tweets relabeled as non-toxic (examples in then, we evaluate the dialectal bias of aaerelabeled and quantify the dialect and racial prediction biases from a roberta-large classifier trained on aae-relabeled, following §5. as shown in the last row of these results highlight that debiasing methods are much less effective at mitigating dialectal dataset biases compared to data relabeling. for future investigations, we recommend obtaining human-written aae-wae pairs (e.g., as done by @xcite . additionally, to ensure less biased toxicity labeling, we recommend recruiting aae speakers or experts for avoiding over-identification of aae-markers as toxic @xcite . alternatively, we recommend exploring more holistic representations of social biases or toxicity (e.g., social bias frames; @xcite . ## related work debiasing toxicity detection as the popularity of hate speech and toxic language detection systems has grown, several biases have been found in dataset and models, spurring various debiasing efforts to mitigate these individual biases (e.g., gender bias, racial bias; @xcite @xcite . some work tackles identity-based biases, e.g., using data re-balancing @xcite , or adversarial feature learning @xcite . less work has tackled racial or dialectal bias. notably, xia et al. (2020) use adversarial training to prevent the model from associating toxicity with aae, showing only small improvements in fairness. based on those results, we do not explore adversarial methods, opting instead for ensemble-based methods of predefined bias reduction. in contemporary work, mozafari et al. (2020) use a re-weighting mechanism, which shows some effects in debiasing racial bias. we leave it for future work to evaluate this method in our setting. in contrast to all previous work, our experiments also measure the effectiveness of bias-agnostic methods. other general debiasing methods several approaches for debiasing nlu tasks have been proposed lately. some approaches rely on adversarial training to remove protected attributes (e.g. gender or race), from a model's internal representations @xcite @xcite . other approaches include confidence regularization (utama et al., 2020), as well as other product of expert approaches @xcite similar to the debiased training approach from @xcite , which is the only debiased training we employ due to its relatively strong performance. ## conclusion we investigate whether toxic language detection systems can be debiased using recently introduced methods for debiasing text classification in nlu tasks. focusing on two types of biases, lexical and dialectal, our experiments show that these methods face significant challenges in reducing the biased behavior in toxicity detectors. this indicates that biases in toxic language detection might be different in nature compared to spurious associations studied in typical nlu settings. we studied a synthetic scheme for relabeling examples with potential dialectal biases; our results indicate that correcting noisy labels results in better bias reduction. our findings suggest that instead of solely relying on development of automatic debiasing for existing, imperfect datasets, future work focus primarily on the quality of the underlying data for hate speech detection, such as accounting for speaker identity and dialect. indeed, such efforts could act as an important step towards making systems less discriminatory, and hence safe and usable.