D - CALM : A Dynamic Clustering-based Active Learning Approach for Mitigating Bias (2023)

Abstract: Despite recent advancements, NLP models continue to be vulnerable to bias. This bias often originates from the uneven distribution of real-world data and can propagate through the annotation process. Escalated integration of these models in our lives calls for methods to mitigate bias without overbearing annotation costs. While active learning (AL) has shown promise in training models with a small amount of annotated data, AL’s reliance on the model’s behavior for selective sampling can lead to an accumulation of unwanted bias rather than bias mitigation. However, infusing clustering with AL can overcome the bias issue of both AL and traditional annotation methods while exploiting AL’s annotation efficiency. In this paper, we propose a novel adaptive clustering-based active learning algorithm, D-CALM, that dynamically adjusts clustering and annotation efforts in response to an estimated classifier error-rate. Experiments on eight datasets for a diverse set of text classification tasks, including emotion, hatespeech, dialog act, and book type detection, demonstrate that our proposed algorithm significantly outperforms baseline AL approaches with both pretrained transformers and traditional Support Vector Machines. D-CALM showcases robustness against different measures of information gain and, as evident from our analysis of label and error distribution, can significantly reduce unwanted model bias.

## introduction while nlp models have experienced groundbreaking advancements in performance and functionality in recent years, they have been under scrutiny for exhibiting bias @xcite @xcite . as noted by @xcite , classifier bias can stem from distribution in training data rather than the classifier itself. this bias is complex and can manifest in various forms, including racial, gender-based, and other types of discrimination. for example, in a hatespeech dataset, hatespeech against persons of color might be underrepresented, leading to a model biased against persons of color (figure 1 ). since the true distribution of data is unknown prior to labeling, ridding these models of such unwanted bias would require annotating a large number of samples to ensure that minority groups are well-represented in the data, incurring much higher cost, time, and effort. as such, we are in need of methods that can mitigate unwanted bias without overwhelming annotation costs. we address the problem of bias with a novel clustering-based active learning approach. although active learning @xcite is regarded as an efficient method for training models, generic active learning methods can induce bias @xcite rather than mitigate it. al-though there have been numerous works aimed at mitigating bias of active learning methods by the machine learning community @xcite , these approaches often necessitate an in-depth comprehension of machine learning and active learning theories. we hypothesize that infusing clustering with active learning will allow us to overcome bias issues of both generic active learning and traditional annotation approaches while leveraging the annotation efficiency of active learning. to this end, we propose a novel dynamic clustering-based algorithm that can substantially improve performance and mitigate bias -d-calm (dynamic clustering-based active learning for mitigating bias) 1 . d-calm leverages the distance between a classifier's predictions and true labels in dynamically-adjusted subregions within the data. as opposed to existing active learning methods @xcite that utilize static clustering of data, our proposed algorithm adapts the clustering in each iteration of active learning. as the classifier gets updated in each iteration, the classifier's error rate changes in different regions. by calibrating the boundaries of clusters iteratively, d-calm focuses annotation effort in updated regions with the evolving classifier's error-rate. as d-calm dynamically adapts its regions for obtaining samples, we hypothesize that our approach will result in reduced bias. similar to @xcite , we expect bias reduction to be reflected in improved performance metrics and more balanced label and error distribution. we test our hypothesis across eight datasets, spanning a diverse range of text classification tasks (e.g., fine-grained hatespeech, dialog act, emotion detection) and a case study of fine-grained hatespeech detection. our algorithm is model agnostic, showing substantial improvement for both pretrained models and lightweight support vector machines. our experiments also demonstrate robustness of d-calm with respect to different measures of information gain. ## related work active learning is a well-studied problem in machine learning @xcite with numerous scenarios and query strategies (section 3). although active learning has shown promise in many 1 our code is available at: https://github.com/ sabithsn/dcalm tasks, susceptibility to bias, particularly for neural networks, is a concern raised by several works @xcite . there are existing works that aim to mitigate this bias. @xcite proposes using corrective weights to mitigate bias. @xcite propose selfsupervised fischer-kernel for active learning on biased datasets. these approaches, however, often require a deep understanding of active learning and neural networks. our approach is tailored for the nlp community and can easily be deployed. in recent years, there has been a renewed interest in active learning within the nlp community @xcite . some recent works have applied active learning with bert models for specific tasks such as intent classification @xcite , sentence matching @xcite , parts-of-speech tagging @xcite or named entity recognition @xcite . @xcite propose continued pretraining on unlabeled data for active learning. @xcite adapt active learning to multi-task scenarios for transformer models. ein-dor et al. (2020) perform a large-scale empirical study of existing active learning strategies on binary classification tasks. in comparison, we target a diverse range of binary and multi-class classification tasks. some other works in the nlp domain have adapted advanced active learning approaches. @xcite adapt the badge @xcite framework for active learning with bert. while badge computes gradient embedding using the output layer of a neural network and then clusters the gradient space, @xcite computes surprisal embeddings by using masked language model loss. @xcite use acquisition functions to obtain contrastive samples for bert. our algorithm is comparatively straightforward, not requiring in-depth understanding of surprisal embeddings or acquisition functions. our algorithm is also model-agnostic and can be applied to neural networks such as bert, and traditional models such as svms. in addition, our clustering step relies on feature representation independent from the learner's representation which may induce bias during the learning process. while some of the aforementioned works @xcite @xcite compute diversity in selected samples, our work is the first to analyze and address bias in active learning from a socio-cultural perspective. ## background this section presents the relevant background of generic active learning followed by a discussion of adapting clustering-based active learning framework for text classification. within the scope of this paper, we focus on creating the train data. we assume that the dev and test data are already created. literature of active testing @xcite can be referred to for efficiently creating the dev and test set. ## experiment setup in this section, we outline our experimental setup. ## results and case study figure 7 : label and error distribution after one iteration of active learning on the conan (fine-grained hatespeech) dataset (averaged across 3 runs). d-calm doesn't suffer from strong bias toward particular groups as topn does and is more equitable toward underrepresented groups compared to random sampling. we observe from figure 4 that on occasions such as in the case of subjectivity and mrda, topn can have critical failures where the model ends up with an extremely low f1 score. although on a few occasions, we witness dips in the curves of d-calm, in general, the curves are much more stable, indicating its robustness. d-calm is robust across different measures of information gain: from figure 5 , we see that d-calm outperforms random, topn, and cluster-topn for different measure of information gain. figure 5 does not contain the subjectivity and cola because these are binary datasets and entropy (reported in figure 4 ), least confident and smallest margin become equivalent in the case of binary classification (section 3). d-calm is model-agnostic: from figure 6 , we observe similar patterns in improvement when the learner model is svm instead of bert. although the degree of improvement is smaller for svms compared to bert, it is a limitation of active learning rather than d-calm's, as we can see others showing smaller improvement as well. improvement over the baselines for svms in addition to bert suggests d-calm is model-independent. d-calm is more robust against bias: since d-calm's focus on enforcing diversity in dynamically adjusting clusters separates it from the other methods, we can deduce that it is the bias reduction that is resulting in improved performance metrics. this is further supported by a study of label and error distribution in the following section. ## conclusion and future work in this paper, we presented a novel dynamic clustering-based active learning algorithm, d-calm, that can be easily adopted by the nlp community for training models with a small set of annotated data. we have shown that by focusing annotation efforts in adaptive clusters where the learner model has higher error rates, the performance can be improved substantially while reducing bias against underrepresented groups in unlabeled data. our experiments also show that d-calm is robust across different datasets, different measures of information gain, and completely different model types. in the future, our approach can be adapted for creating less biased test sets for evaluating classifiers. an exciting future direction for our approach is to adapt it for natural language generation tasks such as style-transfer @xcite or counterspeech generation @xcite .