A Comprehensive Taxonomy of Bias Mitigation Methods for Hate Speech Detection (2025)

Abstract: Algorithmic hate speech detection is widely used today. However, biases within these systems can lead to discrimination. This research presents an overview of bias mitigation strategies in the field of hate speech detection. The identified principles are grouped into four categories, based on their operation principles. A novel taxonomy of bias mitigation methods is proposed. The mitigation strategies are characterized based on their key concepts and analyzed in terms of their application stage and their need for knowledge of protected attributes. Additionally, the paper discusses potential combinations of these strategies. This research shifts the focus from identifying present biases to examining the similarities and differences between mitigation strategies, thereby facilitating the exchange, stacking, and ensembling of these strategies in future research.

## introduction hate speech classification plays a crucial role in moderating online discourse, yet existing machine learning (ml) models often exhibit significant bias. these biases can lead to performance degradation @xcite , discrimination @xcite , erosion of trust in automated systems @xcite , and even violations of laws and regulations @xcite . despite the advancements of hate speech detection methods @xcite , their uncritical application can further exacerbate harm @xcite . addressing these challenges requires effective bias mitigation strategies. current research (e.g. @xcite ) typically classifies mitigation techniques based on specific bias types, making it difficult to identify methods that address multiple biases simultaneously. additionally, most studies treat bias mitigation as a single-step process @xcite , without considering the complexities involved in combining multiple strategies within an ml pipeline. these limitations complicate the selection and application of effective bias mitigation techniques. this study introduces a novel perspective by shifting the focus from bias types to mitigation strategies. instead of asking which methods can mitigate a given bias, the central question is: which types of bias can a specific method mitigate? this research systematically evaluates mitigation strategies to assess their effectiveness against multiple bias types. this perspective is particularly relevant for ml practitioners who encounter multiple biases within a single pipeline and need holistic solutions. by mapping these techniques to different stages of an ml pipeline, this research provides a more actionable and systematic approach. this study conducts a structured literature review to systematically analyze existing bias mitigation strategies in hate speech classification. the analysis identifies key characteristics of these strategies. the key contributions are: through adopting a method-centered approach and systematically structuring bias mitigation strategies, this research facilitates more effective and informed bias mitigation in hate speech classification. taxonomy of bias mitigation: @xcite offer an overview of various mitigation strategies tailored to each of the bias types they define. they identify general approaches for mitigating specific types of bias and present individual algorithms that fit within these broader strategies. while their work compiles a comprehensive range of methodologies, it does not include a taxonomy of these methods due to its focus on bias types. @xcite provide a general overview of bias mitigation methods specifically applied before model training (preprocessing). their aim is to reduce the model's reliance on protected attributes by transforming the underlying dataset according to the principles of the preprocessing method used. further research has surveyed and compared various fairness-aware classification algorithms. @xcite review definitions of fairness and fair machine learning methods across a wide range of applications and problem settings. they list algorithms previously used in specific fairness-aware learning scenarios, noting that these approaches can be categorized by their application stage into 'pre-processing,' 'inprocessing,' or 'post-processing,' but they do not provide a formal taxonomy. in contrast, jones et al. ( 2020 ) compare 28 different model pipelines across seven datasets, evaluating them based on both performance and fairness metrics. combining mitigation methods: park et al. ( 2018 ) examine the impact of three bias mitigation methods on models that exhibit disparities in handling different gender identity terms. their selection of mitigation algorithms allows them to assess both the individual effectiveness of each method and the combined effects when using multiple methods together. they discover that the most significant improvements in fairness metrics occur when all three mitigation methods are combined. similarly, feldman and peake (2021) propose an 'end-to-end' mitigation framework that integrates three bias mitigation algorithms, each targeting a different stage of the learning pipeline. this 'fusion model' demonstrates strong performance across all test metrics, generally outperforming models that rely on a single debiasing method. research contribution: previous research is focused on two tasks. firstly, examining individual types of bias. secondly, which methods have been applied to address bias in the hate speech detection domain. these approaches limit the ability to generalize findings across different types of bias. this research diverges from previous work by focusing on the bias mitigation methods themselves. it provides a comprehensive overview of the existing methods and strategies for mitigating bias. this research address the gaps in current research by creating a taxonomy of bias mitigation methods and further categorizing them based on underlying concepts. this framework will support the development of more effective individual methods or even a combination of methods to combat one or multiple types of bias. ## methodology through a structured literature research, key bias mitigation concepts were identified as foundation for the taxonomy and further research. this approach is similar to the works of @xcite and @xcite . first, a set of general keywords related to the domain of hate speech and toxic speech is collected ('task names'). given the absence of a consistent, operationalized standard and the scientific broadness and ambiguity of existing definitions, previous research may have been categorized differently depending on the author's understanding. thus a single all-encompassing definition would have limited the scope of the study and excluded relevant literature. next, a set of keywords related to the specific topic of investigation is defined. for this study these keywords were synonyms for 'bias mitigation' ('mitigation names'). these two sets were then combined into several queries, with the goal of identifying existing studies of mitigation methods for the problem of hate speech detection. afterwards a second wave of queries was created. here the names of possible mitigation strategies ('mitigation strategies') were combined with the 'task names' to consider research that did not explicitly aim for bias mitigation. these 'mitigation strategies' keywords were sourced from existing literature, with the goal to extend the literature collection and review current developments in the area. all prepared queries were then handed to google scholar as the primary search engine to discover relevant research. an overview of the utilized keywords can be found in appendix b. the literature research for bias mitigation studies ended in october 2023, but individual searches for specific methods continued until february 2024. starting from these sources, citations, and crossreferences were utilized to extend the collection of literature. the publications were evaluated according to the journal they were published in, the year of publication, the amount of citations and the relevance of the abstract, introduction and conclusion. if the research passed these initial hurdles, further investigation was undertaken. after this process 83 publications were utilized in this research. the extracted mitigation strategies were combined into a taxonomy of bias mitigation. additionally both the framework by @xcite and the extracted bias mitigation principles are combined in the work. ## framework of bias to position the mitigation strategies an explicit, shared understanding of bias needs to be defined. @xcite define seven potential sources of harm. theoretically, each source is aligned with a distinct kind of bias. however, in practice these types may not necessarily be mutually exclusive. the bias types proposed by @xcite are: 1. historical bias, 2. representation bias, 3. measurement bias, 4. aggregation bias, 5. learning bias, 6. evaluation bias and 7. deployment bias. a brief introduction to each bias type can be found in appendix c. the relation between bias types and the theoretical representation of the machine learning life cycle (data collection, data preparation, model development, model evaluation, model postprocessing, model deployment) can be seen in appendix a figure 2 . ## strategies principles of bias mitigation to organize the bias mitigation strategies based on the bias they address, this section outlines all different types of identified bias mitigation approaches. it differentiates them from each other, based on their conceptual approach, and delineates if the model is model-agnostic (independent of the underlying machine learning model). a complete overview of all identified mitigation methods, along with examples from applied research, is provided in appendix d. ## on combining mitigation methods bias mitigation is often treated as a single-step intervention, where methods are applied in isolation @xcite . while this simplifies implementation, it limits effectiveness, as no single method optimally balances fairness and performance across all bias types. research suggests that combining strategies can improve fairness @xcite , yet there is little systematic guidance on how to structure such combinations. the challenge lies in understanding interaction effects: some methods reinforce each other, while others may act independently or even neutralize each other. the order of application plays a crucial role. for example, assume that counterfactuals are first created by replacing words with their opposite-gender counterparts. if additional word manipulation methods are then applied to replace all gendered words with a token word, the resulting dataset will contain duplicate sentences, one from the original sample after tokenization and another from its counterfactual version, which is tokenized into the exact same output. it is important to understand, how different bias mitigation techniques interact, both within and across ml pipeline stages, to identify effective multi-stage interventions. another challenge is evaluating combined approaches: many fairness metrics only assess individual methods rather than their collective impact @xcite . developing benchmarks that quantify trade-offs between performance and fairness in multi-method settings is a crucial step forward. additionally, practical concerns, such as computational efficiency and deployment constraints, remain underexplored. to address these challenges, the introduced conceptual groups help organize bias mitigation by clarifying their roles within the ml pipeline. this structured approach enables more effective multimethod interventions, reducing conflicts and improving scalability. ## discussion this research categorized the identified bias mitigation methods based on their intended target within the ml pipeline. it defined four principle groups: text-oriented, sample-oriented, modeloriented, and meta information-oriented methods. the classification is based on the operational similarities of the methods within the ml workflow. while it is possible that other classes of mitigation strategies cannot be represented within this grouping, all identified strategies presented in the research could be positioned here. it would be possible to introduce different groupings based on other aspects, but especially with the goal of making the strategies exchangeable or stackable, this grouping provides the best identified option. this classification also highlights an uneven distribution of methods across ml pipeline stages, with more methods focused on pre-processing and in-processing, and fewer applied during postprocessing. this might be due to the field of research being relatively new and pre-processing being the first step, as it is often a necessity to preprocess the datasets before training an algorithm. therefore, pre-processing might also be the first step to be scrutinized by research. ## conclusion and future work this research investigates bias mitigation methods, specifically for hate speech detection. all identified mitigation strategies were presented and organized based on their principles of operation into a newly developed taxonomy for bias mitigation, categorizing them into four 'conceptual groups' of operation principles: 'text-oriented, @xcite this research shifts the focus away from the individual biases present in a system and onto the available mitigation strategies, creating a comprehensive overview of existing strategies, introducing a novel grouping and taxonomy, and highlighting which biases and stages these strategies cover in current research. it identifies methods that require protected attributes to function and discusses factors influencing the combination of various mitigation methods. therefore, making it easier for future research to understand which methods exist, function similarly and which could bear potential for replacing, stacking, or combining. in the next step, exploring how to best combine these different methods is a promising avenue for further research. future research could also consider adaptive bias mitigation frameworks that dynamically adjust mitigation strategies based on dataset properties. additionally, integrating new mitigation strategies from adjacent fields could enhance the effectiveness of bias reduction techniques.