D-CALM: A Dynamic Clustering-based Active Learning Approach for Mitigating Bias
Year: 2023

Abstract:
Despite recent advancements, NLP models continue to be vulnerable to bias. This bias often originates from the uneven distribution of real-world data and can propagate through the annotation process. Escalated integration of these models in our lives calls for methods to mitigate bias without overbearing annotation costs. While active learning (AL) has shown promise in training models with a small amount of annotated data, AL’s reliance on the model’s behavior for selective sampling can lead to an accumulation of unwanted bias rather than bias mitigation. However, infusing clustering with AL can overcome the bias issue of both AL and traditional annotation methods while exploiting AL’s annotation efficiency. In this paper, we propose a novel adaptive clustering-based active learning algorithm, D-CALM, that dynamically adjusts clustering and annotation efforts in response to an estimated classifier errorrate. Experiments on eight datasets for a diverse set of text classification tasks, including emotion, hatespeech, dialog act, and book type detection, demonstrate that our proposed algorithm significantly outperforms baseline AL approaches with both pretrained transformers and traditional Support Vector Machines. DCALM showcases robustness against different measures of information gain and, as evident from our analysis of label and error distribution, can significantly reduce unwanted model bias.

heading: 1 Introduction
text: While NLP models have experienced groundbreaking advancements in performance and functionality in recent years, they have been under scrutiny for exhibiting bias (Lu et al., 2020; Ahn and Oh, 2021; Kiritchenko and Mohammad, 2018). As noted by Davidson et al. (2019), classifier bias can stem from distribution in training data rather than the classifier itself. This bias is complex and can manifest in various forms, including racial, gender-based, and other types of discrimination. For example,
in a hatespeech dataset, hatespeech against Persons of Color might be underrepresented, leading to a model biased against Persons of Color (Figure 1). Since the true distribution of data is unknown prior to labeling, ridding these models of such unwanted bias would require annotating a large number of samples to ensure that minority groups are well-represented in the data, incurring much higher cost, time, and effort. As such, we are in need of methods that can mitigate unwanted bias without overwhelming annotation costs. We address the problem of bias with a novel clustering-based active learning approach.
Although active learning (Settles, 2009) is regarded as an efficient method for training models, generic active learning methods can induce bias (Krishnan et al., 2021) rather than mitigate it. Al-
5540
though there have been numerous works aimed at mitigating bias of active learning methods by the machine learning community (Farquhar et al., 2021; Gudovskiy et al., 2020), these approaches often necessitate an in-depth comprehension of machine learning and active learning theories. We hypothesize that infusing clustering with active learning will allow us to overcome bias issues of both generic active learning and traditional annotation approaches while leveraging the annotation efficiency of active learning.
To this end, we propose a novel dynamic clustering-based algorithm that can substantially improve performance and mitigate bias —D-CALM (Dynamic Clustering-based Active Learning for Mitigating Bias)1. D-CALM leverages the distance between a classifier’s predictions and true labels in dynamically-adjusted subregions within the data. As opposed to existing active learning methods (Bodó et al., 2011; Berardo et al., 2015) that utilize static clustering of data, our proposed algorithm adapts the clustering in each iteration of active learning. As the classifier gets updated in each iteration, the classifier’s error rate changes in different regions. By calibrating the boundaries of clusters iteratively, D-CALM focuses annotation effort in updated regions with the evolving classifier’s error-rate. As D-CALM dynamically adapts its regions for obtaining samples, we hypothesize that our approach will result in reduced bias. Similar to Hassan et al. (2018), we expect bias reduction to be reflected in improved performance metrics and more balanced label and error distribution. We test our hypothesis across eight datasets, spanning a diverse range of text classification tasks (e.g., fine-grained hatespeech, dialog act, emotion detection) and a case study of fine-grained hatespeech detection. Our algorithm is model agnostic, showing substantial improvement for both pretrained models and lightweight Support Vector Machines. Our experiments also demonstrate robustness of D-CALM with respect to different measures of information gain.

heading: 2 Related Work
text: Active learning is a well-studied problem in machine learning (Settles, 2009) with numerous scenarios and query strategies (Section 3). Although active learning has shown promise in many
1Our code is available at: https://github.com/ sabithsn/DCALM
tasks, susceptibility to bias, particularly for neural networks, is a concern raised by several works (Yuan et al., 2020). There are existing works that aim to mitigate this bias. Farquhar et al. (2021) proposes using corrective weights to mitigate bias. Gudovskiy et al. (2020) propose selfsupervised Fischer-Kernel for active learning on biased datasets. These approaches, however, often require a deep understanding of active learning and neural networks. Our approach is tailored for the NLP community and can easily be deployed.
In recent years, there has been a renewed interest in active learning within the NLP community (Zhang et al., 2022). Some recent works have applied active learning with BERT models for specific tasks such as intent classification (Zhang and Zhang, 2019), sentence matching (Bai et al., 2020), parts-of-speech tagging (Chaudhary et al., 2021) or named entity recognition (Liu et al., 2022). Margatina et al. (2022) propose continued pretraining on unlabeled data for active learning. Rotman and Reichart (2022) adapt active learning to multi-task scenarios for transformer models. Ein-Dor et al. (2020) perform a large-scale empirical study of existing active learning strategies on binary classification tasks. In comparison, we target a diverse range of binary and multi-class classification tasks.
Some other works in the NLP domain have adapted advanced active learning approaches. Yuan et al. (2020) adapt the BADGE (Ash et al., 2020) framework for active learning with BERT. While BADGE computes gradient embedding using the output layer of a neural network and then clusters the gradient space, Yuan et al. (2020) computes surprisal embeddings by using Masked Language Model loss. Margatina et al. (2021) use acquisition functions to obtain contrastive samples for BERT. Our algorithm is comparatively straightforward, not requiring in-depth understanding of surprisal embeddings or acquisition functions. Our algorithm is also model-agnostic and can be applied to neural networks such as BERT, and traditional models such as SVMs. In addition, our clustering step relies on feature representation independent from the learner’s representation which may induce bias during the learning process. While some of the aforementioned works (Ein-Dor et al., 2020; Yuan et al., 2020; Margatina et al., 2021) compute diversity in selected samples, our work is the first to analyze and address bias in active learning from a socio-cultural perspective.

heading: 3 Background
text: This section presents the relevant background of generic active learning followed by a discussion of adapting clustering-based active learning framework for text classification. Within the scope of this paper, we focus on creating the train data. We assume that the dev and test data are already created. Literature of active testing (Kumar and Raj, 2018; Hassan et al., 2018) can be referred to for efficiently creating the dev and test set.

heading: 3.1 Active Learning Framework
text: Due to the expanse of active learning literature, it is important to define the generic active learning framework within the scope of this paper. To do so, we need to define the labeling scenario and query-strategy.

heading: 3.1.1 Labeling Scenario
text: In our work, we assume there is a large pool of unlabeled dataset U but only a small set of labeled dataset L that can be obtained. L is iteratively constructed by querying label for the most-informative instance. We focus on pool-based active learning because of its relevance to many recent NLP tasks (e.g., hatespeech detetion), for which, a large amount of unlabeled data is scraped from the web and then a subset of it is manually annotated.

heading: 3.1.2 Query-Strategy
text: Many types of query-strategies have been proposed for active learning over the years, including, but not limited to: uncertainty sampling (Lewis and Gale, 1994), expected model change (Settles et al., 2007), expected error reduction (Roy and McCallum, 2001), and variance reduction (Hoi et al., 2006). In our work, we focus on uncertainty sampling because of its popularity and synergy with pool-based sampling (Settles, 2009). Settles (2009) lists three measures of uncertainty to identify most informative sample:
Least Confident: Query the instance whose prediction is the least confident.
x∗LC = argmax x 1− Pθ(ŷ|x) (1)
In Eq. 1, ŷ = argmaxyPθ(y|x), or the class label with the highest probability.
Smallest Margin: Query the sample with minimum difference between two most likely classes:
x∗MS = argmin x Pθ(ŷ1|x)− Pθ(ŷ2|x) (2)
Entropy: The most commonly used measure of uncertainty is entropy:
x∗E = argmax x
− ∑
i
Pθ(yi|x)logPθ(yi|x) (3)
In Eq. 3, i ranges over all possible labels. It should be noted that, in binary classification, all the above measures become equivalent. The active learning framework, within the scope of this paper, is summarized in Figure 2

heading: 3.1.3 Challenges
text: Bias Induction: Since the active learning framework relies on the model’s uncertainty to choose samples, the framework may never query samples that the model is confident on. The active learning classifier can become confidently wrong on certain samples, leading to an accumulation of bias.
Effective Batch Selection: In a real-world setting, it is not feasible to obtain annotations one by one and queries need to be done in batches. The most straightforward approach would be to choose the N most informative samples (Citovsky et al., 2021). The limitations of this approach can be easily seen. Particularly when N is large, it can amplify the bias discussed earlier.

heading: 3.2 Clustering-based Framework
text: To address the challenges outlined earlier, we approach the problem with clustering-based framework for active learning under pool-based uncertainty sampling settings. Within this framework, the first step is to obtain vector representation of the unlabeled data. This can be done using SentenceBERT (Reimers and Gurevych, 2019) or more traditional Doc2Vec (Le and Mikolov, 2014).
The next step is to cluster the data. This can be done using any clustering algorithm such as KMeans. Then, informative samples are chosen from each cluster, and are added to the training data. The classifier is retrained after each round and the process is repeated until the annotation budget runs out. Figure 3 summarizes this framework.

heading: 3.3 D-CALM
text: Within the clustering-based framework of active learning, we propose a novel algorithm, D-CALM, that dynamically adjusts clusters in the data based on estimated classifier error rate.
Algorithm 1 D-CALM: Dynamic Clusteringbased Active Learning for Mitigating Bias D,T ← dev data, test data U,L← unlabeled data, labeled data G← bootstrapped classifier B ← labeling budget N ← annotation batch size m← initial number of clusters Cluster U into {C1, C2, ... Cm} Partition D into {C ′1, C ′ 2, ... C ′ m}
while B ≥ 0 do for i=0,1,...m do
Estimate accuracy Ai in C ′i end for for i=0,1,...m do
Allocate li = N ∗ 1−Ai∑ j (1−Aj) Cluster Ci into {Ci1 , Ci2 , ...Cili} for j=0,1,...li do
x∗ij ← most infor. sample in Cij y∗ij ← query true label for x∗ij Add (x∗ij , y ∗ ij) to L
end for end for G← retrain on L B = B −N
end while Evaluate G on T
In our proposed algorithm, the cluster C ′i is used to dynamically partition Ci. Our algorithm first observes how the classifier behaves in C ′i. For cluster Ci, it allocates samples proportional to the error rate in C ′i. Then the cluster Ci is split into subclusters according to the number of samples allocated to Ci. Most informative sample from each subcluster is then added to training data. The subclusters are dynamically updated in each iteration to account for the classifier’s new state. This prevents the classifier from repeatedly sampling from any particular region. It is worth noting that error-rate can be substituted with different metrics to account for specific needs. For example, in scenarios where it is more important to reduce false negative rate reduce compared to false positive rate, the error rate can be substituted with false negative rate. In this paper, we focus on the general case of error-rate.

heading: 4 Experiment Setup
text: In this section, we outline our experimental setup.

heading: 4.1 Active Learning Approaches
text: For all the following approaches, total number of samples range from 100-300, initial allocation for bootstrapping is set to 50, and annotation batch size is 50. Similar to (Ein-Dor et al., 2020), the classifiers are retrained in each round.
Random: The allocated number of samples are picked randomly from the unlabeled pool.
TopN: The classifier is bootstrapped with 50 samples. In each iteration N most informative samples are labeled and added to training data until labeling budget runs out. TopN is a widely used baseline (Yuan et al., 2020; Ash et al., 2020).
Cluster-TopN: The classifier is bootstrapped in the same way. The unlabeled pool is first clustered into 10 clusters and in each iteration, N/10 most informative samples are chosen from each cluster. Cluster-TopN combines TopN and stratified sampling (Qian and Zhou, 2010). We choose Cluster-TopN as a baseline due to its similarity with multiple existing methods (Xu et al., 2003; Zhdanov, 2019).
D-CALM: The classifier is bootstrapped in a similar fashion. While D-CALM is not sensitive to the initial number of clusters because of its dynamic splitting into subclusters, we set initial number of clusters to 10 to be consistent with Cluster-TopN.

heading: 4.2 Models
text: Transformers We fine-tune the widely-used bertbased-cased (Devlin et al., 2019). We observed that the models stabilize on the dev data when finetuned for 5 epochs with learning rate of 8e-5 and batch size of 16. The same setting is used across all experiments.
Support Vector Machine (SVM) We choose SVM as our alternate model as it is completely different from transformers and because SVMs are still in use for practical purposes due to speed and lightweight properties (Hassan et al., 2021, 2022). We use Tf-IDF weighted character [2-5] grams to train SVMs with default scikit-learn settings2.

heading: 4.3 Datasets
text: We evaluate our proposed algorithm on eight diverse datasets, among which two are binary classification datasets and the rest are multiclass.
BOOK32 (Iwana et al., 2016) contains 207K book titles categorized into 32 classes such as Biographies & Memoirs. We take a subset that contains 20K random samples from 10 most frequent classes for runtime efficiency. Random sampling ensures the subset respects original distribution.
CONAN (Fanton et al., 2021) contains 5K instances annotated for hatespeech targets: Disabled, Jews, LGBT+, Migrants, Muslims, Person of Color (POC), Women, and Other.
CARER (Saravia et al., 2018) is an emotion detection dataset that contains six basic emotions in the released version: Anger, Fear, Joy, Love, Sadness, and Surprise. 3. The released version consists of 16K training, 2K dev and 2K test instances.
CoLA (Saravia et al., 2018) contains 9.5K sentences expertly annotated for acceptability (grammaticality) in the public version. We use the indomain set as dev and out-of-domain as test set.
HATE (Davidson et al., 2017) contains a total of 24.7K tweets that are annotated as: Offensive, Hatespeech, and Neither.
MRDA (Shriberg et al., 2004) contains 117K instances annotated for dialog acts. We consider the five basic labels: Statement, BackChannel, Disruption, FloorGrabber, and Question. We limit the
2https://scikit-learn.org/stable/modules/ generated/sklearn.svm.LinearSVC.html
3https://huggingface.co/datasets/emotion
data to 20K randomly chosen samples for runtime efficiency.
Q-Type (Li and Roth, 2002) contains 5.5K train and 0.5K test instances annotated for question types. We take the first level of annotation, containing six classes: Entity, Description, Abbrebivation, Number, Human, and textitLocation.
Subjectivity (Pang and Lee, 2004) contains 10K snippets from Rotten Tomatoes/IMDB reviews automatically tagged as Subjective or Objective.

heading: 4.4 Data Preparation
text: Splits We use default train-dev-test splits if they are provided. If they are not provided, we split the data into 70-10-20 splits. The train data is treated as unlabeled pool of data, dev data is used for tuning purposes and test data is used to report results. Table 1 shows summary of data used.
Vector Representation We use MiniLM (Wang et al., 2020) sentence-transformer to transform text instances into 384 dimensional vectors. These vectors are then used to cluster the unlabeled data.
Clustering We use KMeans to cluster the unlabeled pool of data. We use scikit-learn4 implementation of KMeans with default parameters.

heading: 5 Results and Case Study
text: We first discuss findings of our experiments, followed by a case study of fine-grained hatespeech detection. Figures 4,5 and 6 summarize the results across the eight datasets, different measures of information gain, and different models. Table 2 summarizes relative performance across all experiments. For each experiment, we report Macro-F1
4https://scikit-learn.org/stable/modules/ generated/sklearn.cluster.KMeans.html
score averaged across 3 runs. We choose MacroF1 as our metric since it provides a more holistic measure of a classifier’s performance across classes. Thus, reduction of bias is more likely to be reflected in metrics such as F1 compared to other metrics such as accuracy.

heading: 5.1 Experiment Results
text: D-CALM consistently outperforms baselines: From Figures 4, 5, 6 we can observe that DCALM consistently outperforms TopN, random and cluster-TopN across all datasets. From Table 2, we observe that D-CALM beats TopN in 32 out of 40 data points for BERT, among which, the difference in F1 score is greater than 5 in 15 cases. DCALM beats the nearest algorithm, Cluster-TopN in 28/40 (p value 0.003) for BERT and Random Sampling in 26/40 cases (p value 0.0073) for SVMs (Table 2). Both of these are statistically significant according to 2 population proportion test at significance level of 0.01.
D-CALM is more robust against critical failures: We observe from Figure 4 that on occasions such as in the case of Subjectivity and MRDA, TopN can have critical failures where the model ends up with an extremely low F1 score. Although on a few occasions, we witness dips in the curves of D-CALM, in general, the curves are much more stable, indicating its robustness.
D-CALM is robust across different measures of information gain: From Figure 5, we see that D-CALM outperforms random, TopN, and cluster-TopN for different measure of information gain. Figure 5 does not contain the Subjectivity and CoLA because these are binary datasets and Entropy (reported in Figure 4), Least Confident and Smallest Margin become equivalent in the case of binary classification (Section 3).
D-CALM is model-agnostic: From Figure 6, we observe similar patterns in improvement when the learner model is SVM instead of BERT. Although the degree of improvement is smaller for SVMs compared to BERT, it is a limitation of active learning rather than D-CALM’s, as we can see others showing smaller improvement as well. Improvement over the baselines for SVMs in addition to BERT suggests D-CALM is model-independent.
D-CALM is more robust against bias: Since D-CALM’s focus on enforcing diversity in dynamically adjusting clusters separates it from the other methods, we can deduce that it is the bias reduction that is resulting in improved performance metrics. This is further supported by a study of label and error distribution in the following section.

heading: 5.2 Case Study
text: We choose hatespeech detection as our case study because hatespeech and abusive content detection has been one of the most studied text classification tasks in recent years (Zampieri et al., 2019; Hassan et al., 2020). Due to its importance in many avenues, whether for AI moderation of online content (Ye et al., 2023) or filtering language model data (Abdelali et al., 2021), racial and gender bias in hatespeech and abusive content detection tasks have been a particular concern for the NLP community (Davidson et al., 2019; Ahn and Oh, 2021).

heading: 5.2.1 Label and Error Distribution
text: To understand the effect of the underlying distribution in data, we observe the label distribution of samples chosen for annotation after one round of active learning. We also observe the distribution when the same number of samples are chosen randomly. From Figure 7 we can observe that random sampling may result in small number of samples chosen for minority classes in the data such as Persons of Color and Disabled. The distribution of labels obtained by TopN can become particularly skewed. We see that on average, TopN samples 25+ hatespeech targeting Jews while mostly ignoring groups such as Person of Color (POC) with <5 samples on average. While Cluster-TopN mitigates this problem to an extent, the best results are obtained by D-CALM, with samples for POC doubling compared to TopN and samples for Disabled doubling compared to random sampling after just one iteration. The error distribution in Figure 7, reflects the effects of this as we see errors for POC
and Disabled are greatly reduced by D-CALM. It’s important to note, if we had access to a large pool of labeled data, we could obtain a more balanced dataset for training. However, in a real-world scenario, before the annotation process, we only have access to unlabeled pool of data. As such, we cannot identify low-frequency classes and balance the training set. D-CALM, however, can obtain more samples from the underrepresented classes without knowing their true labels beforehand.

heading: 5.2.2 Error Analysis
text: To understand the limitations of D-CALM, we manually annotated 100 errors made by the best run with BERT on the CONAN dataset after one iteration of active learning. Our key observations are listed below:
• The model can be confused on closely related classes such as Jews and Muslims as the hatespeech in both cases target religions.
• Some errors can be attributed to the limitation of annotation design. For example, CONAN contains the class Persons of Color (POC), but does not contain a separate class for racism against Asians. These instances are labeled as Other in the data but are predicted as POC by the model.
• In some cases, the error is in the original annotation, rather than the model’s prediction.
Examples of these errors are listed in Table 3. While the first type of error can possibly be reduced with the addition of more data close to boundary regions between closely related classes, the last two types of errors need to be addressed during the design and annotation phase of the task.

heading: 6 Conclusion and Future Work
text: In this paper, we presented a novel dynamic clustering-based active learning algorithm, DCALM, that can be easily adopted by the NLP community for training models with a small set of annotated data. We have shown that by focusing annotation efforts in adaptive clusters where
the learner model has higher error rates, the performance can be improved substantially while reducing bias against underrepresented groups in unlabeled data. Our experiments also show that DCALM is robust across different datasets, different measures of information gain, and completely different model types. In the future, our approach can be adapted for creating less biased test sets for evaluating classifiers. An exciting future direction for our approach is to adapt it for natural language generation tasks such as style-transfer (Atwell et al., 2022) or counterspeech generation (Ashida and Komachi, 2022).
Limitations
It’s important to note that, in this paper, we focus on bias resulting from underlying distribution of training data. Bias that may result from pretraining of transformer models (Li et al., 2021) is not within the scope of this paper.
Although we conduct a case study of finegrained hatespeech detection task, a collective effort from the research community is required to better quantify bias mitigation of our approach across multiple tasks and different types of bias.
Another limitation of our work is that our proposed algorithm requires dynamic adjustment of clusters. For very large datasets, this may be computationally expensive.
Ethics Statement
Although our proposed algorithm shows more stability and reduced bias compared to existing approaches and random sampling, it’s important to observe the behavior of active learner as the algorithm may not completely eliminate bias, specifically when the annotation budget is small. This can be achieved by observing label and error variance on the evaluation data. It is also important to take into consideration the necessities of practical scenarios. In scenarios where certain type of bias is desired (e.g., higher precision), the algorithm needs to be adapted as outlined in Section 3.3

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Limitations sections at the end

heading: 3 A2. Did you discuss any potential risks of your work?
text: Ethical considerations sections at the end

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Sections 3-5
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4,5

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 4,5
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? The models (e.g. BERT) are free-to-use for researchers.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
7 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.
C 3 Did you run computational experiments? Section 4,5

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? 4.2, 4.3
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 4

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 5.1
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. Left blank.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: In the future, our approach can be adapted for creating less biased test sets for evaluating classifiers. An exciting future direction for our approach is to adapt it for natural language generation tasks such as style-transfer (Atwell et al. , 2022) or counterspeech generation (Ashida and Komachi, 2022). Limitations
It’s important to note that, in this paper, we focus on bias resulting from underlying distribution of training data. Bias that may result from pretraining of transformer models (Li et al. , 2021) is not within the scope of this paper. Although we conduct a case study of finegrained hatespeech detection task, a collective effort from the research community is required to better quantify bias mitigation of our approach across multiple tasks and different types of bias. Another limitation of our work is that our proposed algorithm requires dynamic adjustment of clusters. For very large datasets, this may be computationally expensive