Leveraging Prototypical Representations for Mitigating Social Bias without Demographic Information (2024)

Abstract: Mitigating social biases typically requires identifying the social groups associated with each data sample. In this paper, we present DAFair, a novel approach to address social bias in language models. Unlike traditional methods that rely on explicit demographic labels, our approach does not require any such information. Instead, we leverage predefined prototypical demographic texts and incorporate a regularization term during the fine-tuning process to mitigate bias in the model’s representations. Our empirical results across two tasks and two models demonstrate the effectiveness of our method compared to previous approaches that do not rely on labeled data. Moreover, with limited demographic-annotated data, our approach outperforms common debiasing approaches.

## introduction and background the presence of social bias in training data presents a significant challenge in the development of language models for real-world applications. while these models possess remarkable capabilities, biases within the data can lead to unfair outcomes. mitigating these biases is crucial, but it becomes particularly challenging when acquiring or accessing sensitive attribute labels is costly or unfeasible. studies showed that language models have the ability to capture demographic information about the writer, including race or gender, within their representations @xcite . however, this capability can introduce unintended biases, leading to discriminatory outputs @xcite . common approaches for social bias mitigation require explicit annotation of biases for each sample in the data @xcite . recent concept removal methods @xcite @xcite 1 our code is available at https://github.com/ technion-cs-nlp/dafair figure 1: our debiasing method consists of defining task-specific representations for each social attribute, measuring similarity in the representation space for each example, and utilizing the kl loss to encourage uniform probabilities across social groups. a promising line of research has emerged that aims to mitigate bias without relying on explicit information about the biases present in the data. for instance, just train twice (jtt) @xcite employs a two-step training process. in the second step, a second model is trained on up-weighed training examples that were misclassified by the first model. another method is blind @xcite , which introduces a success detector and down-weighs examples for which the detector accurately predicts the outcome. in this paper, we propose dafair: demographics-agnostic fairness, a novel approach for mitigating social bias during the fine-tuning process of language models, without relying on demographic information. our approach aims to ensure equal similarity between the representation of a text and prototypical representations of different demographic groups. for instance, when classifying a biographical text of a person into their profession, our method aims to make the representation of the text equally similar to the representations of both males and females. more concretely, dafair first defines prototypical texts, such as "this is a biography about a male" and "this is a biography about a female". it then adds a regularization term that makes the representation of a training example equally similar to the representations of each of the prototypical texts (figure 1 ). furthermore, we extend our approach to scenarios where limited demographic-annotated data is available. in such cases, we obtain the prototypical representation by averaging the sample representations corresponding to each social attribute. we evaluate the effectiveness of dafair and its extension on two tasks: occupation prediction and sentiment analysis of twitter posts. in these tasks, we investigate the performance of our approach under the settings of limited demographic labels or no labels at all, reflecting real-world scenarios where labeled data is challenging to obtain. the experimental results with two base models demonstrate that our approach outperforms previous approaches that do not rely on demographic information, as well as common approaches with limited data. ## methodology assume a dataset @xmath0, y i , z i } n @xmath1, main task labels y @xmath2, and sensitive attributes z @xmath3, such as race. this sensitive attribute can either be unobserved during training or available in a small subset of the data. our aim is to learn a model f : @xmath4. ## conclusion we introduced dafair, a novel approach for mitigating social bias in language models without explicit demographic information. our method leverages semantic similarity to manipulate the model's text representations during finetuning to promote fairness. experimental results on two tasks and under different settings demonstrated the effectiveness of dafair in reducing bias and improving fairness while maintaining competitive downstream task performance, even with limited or no labeled demographic data. with its focus on social bias, dafair offers a flexible framework adaptable to address other forms of bias through the modification of prototypical texts. in conclusion, our approach offers a practical and flexible solution for bias mitigation in realworld applications, contributing to the development of fairer language models.