Fair Text Classification with Wasserstein Independence
Authors: Thibaud Leteno
Antoine Gourru
Charlotte Laclau
Rémi Emonet
Christophe Gravier
Venue: EMNLP 2023 Main (2023)
Keywords: Text Classification
Fairness
Wasserstein

Abstract: Group fairness is a central research topic in text classification, where reaching fair treatment between sensitive groups (e.g. women vs. men) remains an open challenge. This paper presents a novel method for mitigating biases in neural text classification, agnostic to the model architecture. Considering the difficulty to distinguish fair from unfair information in a text encoder, we take inspiration from adversarial training to induce Wasserstein independence between representations learned to predict our target label and the ones learned to predict some sensitive attribute. 
Our approach provides two significant advantages. Firstly, it does not require annotations of sensitive attributes in both testing and training data. This is more suitable for real-life scenarios compared to existing methods that require annotations of sensitive attributes at train time. Secondly, our approach exhibits a comparable or better fairness-accuracy trade-off compared to existing methods.

Fair Text Classification with Wasserstein Independence
Thibaud Leteno" Antoine Gourru 1+
Charlotte Laclau²
Rémi Emonet Christophe Gravier'
Laboratoire Hubert Curien, UMR CNRS 5516, Saint-Etienne, France
Télécom Paris, Institut Polytechnique de Paris, Paris, France
[thibaud.leteno, www.ine.gourrrenoe remi.emone,, christophe.grave gravier @univ-st-etiemne.f laarlotte.laclau telecom-paris..r
Abstract
Legend
0: Sensitive attributes
Group fairness is a central research topic in text classification, where reaching fair treat- ment between sensitive groups (e.g. women vs. men) remains an open challenge. This paper presents novel method for mitigating biases neural text classification, agnostic to the model architecture. Considering the diffi- culty to distinguish fair from unfair informa- tion text encoder, we take inspiration from adversarial training induce Wasserstein in- dependence between representations learned to predict our target label and the ones learned to predict some sensitive attribute. Our approach provides two significant advantages. Firstly, it does not require annotations of sensitive at- tributes in both testing and training data. This is more suitable for real-life scenarios compared existing methods that require annotations of sensitive attributes at train time. Secondly, our approach exhibits a comparable or better fairness-accurae trade-off compared to exist- ing methods. Our implementation available Github'.
Target Tabels
Dernonis Tiwd
Procksor
Figure 1: Our method, WFC, modifies the representation space of documents such that it fairer when training a classifier top. To do that, it makes it independent of "demonic" model that predicts the sensitive attribute.
Liang et al., 2021; Bender et al., 2021), these is- sues become even more significant with the advent of public-ready Al-powered NLP systems such as ChatGPT (OpenAl) or Google Bard (Pichai), mak- ing the need for fair NLP solutions pelling. As more researchers work to overcome these shortcomings, the first problem is define what fairness is. Such a definition may hardly be consensual or is at least difficult to establish, as it depends on situational and cultural contexts (Fiske, 2017). In this work, we adopt the most common definition of group faimess, and the one adopted by laws in several countries, which is based on the notion of disparate impact: prediction model is considered to have a disparate impact if its sults disproportionately harm (or benefit) people with certain sensitive attribute values (e.g., women, black people).
com-
1 Introduction
Machine learning algorithms have become increas- ingly influential in decision-making processes that significantly impact our daily lives. One of the major challenges that has emerged in research, both academic and industrial, concerns the fair- ness of these models, i.e. their ability to prevent predictions related to individuals to be based on sensitive attributes such gender or ethnicity. In this article, focus on the problem of fairness in the domain of Natural Language Processing (NLP) (Bender et al., 2021: Osoba and Welser IV, 2017; Schwemmer et al., 2020). While many stud- ies already report biases in NLP systems (Sun et al., 2019; Hutchinson et al., 2020; Tan and Celis, 2019;
In this work, we focus on group fairness for neu- ral text classification as it is one of the most ubiqui- tous tasks in our socicty, with prominent examples in medical and legal domains (Demner-Fushman et al., 2009) human resources (Jatobá al., 2019), to name few. Neural text classification relics on text encoders, which are parameterized and learned functions that map tokens (arbitrary
'Equal contribution.
'tttps: // //gitttttbt/t/ http./////.tte//tiaaa/////eeeeee fair_classificatte ficatioo

---

text chunks) into latent space of controllable di- mension, usually followed by classification layer. Built upon the Transformers architecture ( (Vaswani et al., 2017), popular Pre-trained Language Mod- els (PLMs) such as BERT (Devlin et al., 2019), GPT3 (Radford et al., 2019) or Llama (Touvron et al., 2023) leverage self-supervised learning to train the text encoder parameters. In modern NLP pipelines, these PLMs are further fine-tuned the supervised task at hand. Ultimately, PLMs accumulate uncontrolled levels of unfairness due to unbalanced learning data or algorithmic biases, for instance. This results in observable biases in predictions but also in the latent space as studied in (Zhao et al., 2019; May et al., 2019).
2 Related Works
Numerous studies have been conducted on how to tackle bias in machine learning systems. Ap- proaches to reinforce fairness can be divided be- tween pre-, in- and post-processing methods. In the NLP literature, common pre-processing tech- niques consist of data rebalancing (Park et al., 2018) or embedding debiasing (Wang et al., 2020; Bolukbasi et al., 2016). Yet, despite the efficiency of those methods, Kaneko et al. (2022) and Tokpo et al. (2023) showed that other biases can be learned during the training or fine-tuning processes. On the other hand, post- prot-processing procedures aim at cor- recting biases after the learning step. through model calibration (Zhao et al., 2017; Jia et al., 2020) or data projection (Ravfogel et al., 2020) (INLP). We refer to (Sun et al., 2019; Blodgett et al., 2020) for more exhaustive surveys of bias in NLP.
We propose a novel approach (see Figure 1), to mitigate bias in text encoders, that aims to tackle bias directly in the latent space on which docu- ments are projected, thus making our model appli- cable to any text encoder or decoder (e.g. BERT or LLAMA). To proceed, we disentangle the neural signals encoding bias from the neural signals used for prediction. The proposed architecture is based on three components. First, two Multi-Layer Per- ceptrons (MLPs): the first one whose objective is to predict the sensitive attribute, and the second one is dedicated to the prediction task hand. Then, third MLP, referred to as a critie, approximates the Wasserstein distance that acts as regularizer in our objective function. Our proposition overcomes a major shortcoming of prior studies: they rely on the availability of the sensitive attributes at train time. A constraint that is incompatible with recent regulations as the new European ones, that enforce more stringent requirements for the collection and utilization of protected attributes. Prior studies thus more difficult use in practical settings. In the following, we will show that our approach can address this limitation by avoiding the use of this information during both testing and training.
Recently, adversarial methods (Beutel et al., 2017; Zhang et al., 2018; Elazar and Goldberg, 2018) have been investigated to mitigate biases, Han et al. (2021c.b) respectively suggest using sev- eral discriminators where each learns different hid- den representations and applying an adversarial ap- proach in- and cross-domain train the adversary on a different dataset, with methods called Adv and GATE Differently, recent contributions focus on directly constraining the objective to improve fairness (Shen et al., 2022b; Han et al., 2021a). For instance, by adding some fairness metric, such as the Equal opportunity that we will define later in this paper, to the objective function.
Our approach is at the crossroad of these two philosophies: on the one hand, we propose to train a biased model whose sole purpose is to predict the sensitive attribute and use this latter to enforce fair- ness in our main prediction model. On the second hand, we minimize classifier loss with regular- ization term measuring the dependence between the latent representations of the classifier and some unfair representations, using Wasserstein distance. While many works use the Kullback-Leibler (KL) divergence to measure the mutual information be- tween representations, Ozair et al. (2019) show sev- eral limitations: the KL-diiergence is sensitive to small perturbations in the data, and exploiting for estimating the mutual information does not scale up properly. Thus, they suggest an improvement thanks to the Wasserstein distance. Other methods based on this distance suggest focusing on the dis- tance between the distributions of predictions to en-
The rest of this paper organized as follows. Section 2 presents recent advances regarding fair- ness in NLP. Section argues about our motivation and provides the background knowledge to under- stand our contribution. Section 4 proceeds with the description of the proposed approach, its the- oretical analysis, and algorithmic implementation. Section introduces the setting of our experiments, and Section presents the results and interpretation. The last section concludes the paper and gives a
couple of hints for possible future research.

---

force fairness (Jiang et al., 2020; Risser et al., 2022),
processed by two different encoders to make the predictions independent of the sensitive attributes. Moreover, their approach depends on identifying biased attribute words, limiting its applicability to cases where substitute words are accessible. This is constraint we avoid. Lastly, while previous methods primarily targeted classification issues in images or categorical and numerical data, we intro- duce an approach well-suited for Natural Language Processing. It can be upplied less-explored sce- narios, including continuous sensitive attributes and regression tasks.
Finally, most approaches aforementioned depend on the availability of sensitive attribute annotations in the training data, and as Kenfack et al. (2023) recently emphasized, employing proxy-sensitive at- trade-
tributes often worsens the fairness-accuran off. They also propose proxy model to retrieve the missing sensitive attributes, adapted to improve the model's fairness.
Limits of existing approaches and positioning Compared to the adversarial approaches previously mentioned, ours is conceptually closer to (Nam al., 2020), while their methodology is conducted on images rather than textual data. We also distin- guish from their research by the use of the Wasser- stein distance to evaluate the mutual information between the two models' representations instead of focusing the learning of the main model on samples going against the prejudice of the biased network. Like Ozair et al. (2019), we exploit the Wasserstein distance as estimator of the mutual information. However, while they use measure mutual infor- mation improve representational learning on im- ages, we consider sensitive attributes the mutual information estimation and use to improve the model fairness. Our proposition related to Risser et al. (2022), yet, we do not use the model out- puts directly as we use hidden representations, of fixed-size and task-independent dimensions, of Pre- trained Language Models that encode information on sensitive attributes. Additionally, we do not use the Wasserstein distance to compute the distance between each group's prediction probability but to enforce the independence of the representation from unfair representations. By using those latent representations in the Wasserstein-regulliizzli term, the model is encouraged not to encode the sensitive information in the representation during inference. Similarly, in the field of NLP, a re- lated approach proposed by Cheng et al. (2021). Their method maximizes the mutual information between pairs of sentence representations and their augmented versions, which vary based on the sen- sitive attribute. These representations go through the same encoder, ensuring that the input is inde- pendent of the sensitive information. However, this does not ensure independence between the predic- tion and the sensitive attribute (Shen al.. 2022a; Cabello al., 2023). In contrast, our theoretically grounded approach minimizes the mutual informa- tion between representations of the same sentence
Preliminaries
In this section, we introduce the notations used throughout this paper. We also present the defi- nitions of the necessary fairness metrics, and the main concepts, mostly related the Wasserstein distance which are essential for understanding the rest of the paper.
3.1 Motivation
We consider a corpus triplets ( y/, s;) H:) where 2; E X is a short document or a sentence, Y/ E y is label and s; E S corresponds to one or multiple variables, referred to as sensitive attributes, such as gender, ethnicity age. Let us consider binary classification for illustrative purposes. The goal is to predict the outcomes yı by estimating the conditional probability p(y 1/2 = x;) through a scoring function ʃ X + {0, 1}. The prediction associated with f is noted ÿ. For example, the context of social network, classifier can use the descriptors of a message (e.g., bag of word representation), 4;, to predict whether message is toxic or not, leading to the decision to ban the message and/or the user who wrote it from the social platform.
3.2 Measuring Fairness
In this context, of particular relevance is group- based fairness, which examines how well outcome (i) consistency is preserved across sensitive groups (s). Returning to our example, when determining whether a message is toxic, fairness here implies that the decision is consistent for all users, regard- less of gender or ethnicity.
A commonly used group-based metric used quantify the (un)fairness of a given classifier is the Equality of Opportunity EO (Hardt et al., 2016) that satisfied the prediction made by the clas-

---

sifier is conditionally independent of the protected attribute, given that the true value positive (e.g. y = 1). In effect, it means that the same proportion each group receives a positive outcome. For bi- nary sensitive attribute (s C {a, a}) and multi-class objectives, the consensual way of aggregating EO score over classes is the GAP score (De-Arteaga et al., 2019; Ravfogel et al., 2020) defined as fol- lows
the ground-truth not), and s, the sensitive tribute, will make the prediction and the sen- sitive attribute less and less dependent. Neverthe- less, MI is generally intractable for most real-life scenarios and has strong theoretical limitations as outlined by Ozair et al. (2019). Notably, it requires number of samples exponential in the value of the Mutual Information to build high confidence lower bound and it is sensitive small perturba- tions in the data sample. Consequently, Ozair al. (2019) proposed a theoretically sound dependency measure, the Wasserstein Dependency Measure, based on Wasserstein- distance:
1
GAP
F.O.²,
(1)
where the EO for given class e E C defined by
EO. =p( = cly = C, = a) p = cly C, == a).
Mlw(x, y) Wi y), p(z)p(v)). (4)
(2)
A key feature of the Wasserstein distance is that can act as a smooth objective function, as shown in the WGAN approach (Arjovsky et al., 2017). More precisely, the Cantorovvic-Rubinseti duality expresses Wi (p(z, v), p(x)p(U)) as:
Additionally, as faimess often requires deter- mining a trade-off such that reaching equity does not degrade the general classification performance. Han et al. (2021a) proposed the Distance To Op- timum (DTO) score. This latter measures the Securacy-fairness trade-off by computing the Eu- clidean distance from model to Utopia point (point corresponding the best Accuracy and best Fairness values across all the baselines). The goal is to minimize the DTO.
sup E Buy Buy-pla.y y)] ||C||
(5)
-E ( [C(,, v)],
where ||C||r < is the set of all I-Lipschitz func- tions. Arjovsky et al. (2017) propose to approxi- mate this measure by using a parameterized func- tion, defined as follows:
Finally, we consider the Leakage metric that cor- responds to the accuracy of a classification model trained to predict the sensitive attribute from the la- tent representations. It measures the fairness the latent representations themselves and demonstrates unfairness when close 100 of accuracy.
max E Eay-play) (z, w.I[Cuollsst
(6)
-a--p())) (a, y)],
3.3 Fairness Mutual Information minimization
where C. is called the critic and is usually neural network. Wasserstein distance has been efficiently used in many machine learning fields (Frogner et al., 2015; Courty al., 2014: Torres et al., 2021) and a particularly interesting application that of fair machine learning (Jiang et al., 2020; Silvia et al., 2020; Gordaliza et al., 2019; Laclau et al., 2021). See Appendix A. for further theoretical details on the Wasserstein Distance. The role of this measure in our contribution is detailed in the subsequent sections.
Mutual Information (MI) is an information-theory. based metric meant to measure the statistical depen- dence or the amount of information shared between two variables. In our context, given the class of a document predicted by our model along with the value of the sensitive attribute of the document, one MI evaluate the strength of their relation- ship. Formally, the mutual information defined as the Leibbec (KL) divergence between the joint distribution p(a, y) and the product of the marginal distributions:
4 Our contribution
We are now ready show how one can cast the problem of group fairness as an independence con- straint in the intermediate latent space of the ML.Ps and derive a theoretically sound approach based on the Wasserstein distance to solve it.
MI(x, y) (L(p(a,))ll((<)p(o))
(3)
Fairness can therefore be cast as MI minimiza- tion between j. our prediction (conditioned on

---

pre-trained
p Eay [x)
p(ay) (p(2)
critic
Zyy
2,1
Zzy
Z
Zy/
Z,2
Z
Zu
x,
n/2
7
Zyd
7x
Zyl
7,2
L 9)
( 2l)
PLM
Figure 2: Architecture for batch of size at train time. The data representation the left shows how create dependency or independence between by and Zg. At inference, only the trained classifier (green) is kept to predict y.
4.1 Definition of the Objective Function
min MI 2.) 2 MI(Ú, 8).
(8)
most recent NLP applications, deep classifi- cation performed as a two-step approach: the scoring function f a composition two param- eterized functions such that f = o h, where g(z) = 2 E RRd projects in low dimensional space and h is simple, usually one linear layer neural network with softmax activation followed by an argmax referred as classification layer. The objective of g is produce an embedding linearly separable with respect to the target of in- terest. For the deep model predicting y. we write fo = Ic he = (z()), where the index c stands for classification.
Lastly, we replace MI with MI for the reasons explained earlier. We propose to optimize simulta- neously the Wasserstein dependency measure and the more traditional classification loss (e.g. the cross-entropy). Our objective function writes follow
arg min L(y, h(2y)) +B Wi(p(2ys 22), p(2))(())))
(9)
where the loss function aiming at maximizing the accuracy of the model for predicting while the role of the second term is to constrain the en- coder part of the language model to learn fair rep- resentations. The hyper-parametter 3 R is weight allowing some control on the impact of the penalty as the speed of convergence of the sub- objectives may be different. In the following, we refer to the approach minimizing Equation 9 as WFC for Wasserstein Fair Classification (WFC).
As stated earlier, fairness can be defined min- imizing MI(j, As is neither observable nor allowed to be observed in most real-life scenarios, we make use of surrogate for s that we call the de- monic model. This deep model is a composition of f. 9. hr hz(2)), where stands for sensitive attribute. Therefore, in the absence of the sensitive attribute, use
4.2
Optimization of WFC
min MI 8).
The overall architecture of WFC is presented in Fig- ure 2. Given batch of documents along with their sensitive attribute, start by generating repre- sentation of each document using PLM. Then, taking these vectors as input, we train two MLPs to predict s and y, respectively. The former re-
As the argmax operation producing the hard pre- dictions following the classification layer is not differentiable, we propose minimize the MI be- tween the latent representations instead of the net- work final output, leading to optimizing an upper bound of the latter equation (see details in Ap- pendix):
ferred to as the demonic model in the remained this paper. Now, assuming that the MLPs consist

---

one hidden layer, for the sake of simplicity, we can extract two embedding vectors for all documents, denoted by 2g and Ey. Note that the prediction ø made by the second MLP (in green in Figure 2) can be directly used to compute the first part of our objective function (see Equation 9).
can leverage generated datasets, like the EEC dataset (Kiritchenko and Mohammad, 2018). Thus, we transfer this knowledge from one dataset to an- other, working towards faimess autonomy regard- less of the inclusion of sensitive attributes within the data.
Now for the second term of the loss, which is given by Wi (p(zy, 2s), p( p(2y)) )p())) we use the ap- proximation proposed by Arjovsky al. (2017):
Experimental Protocol
In our experiments, we intensively use the FairLib package (Han et al., 2022), which provides cess to many state-of-the-art methods and datasets.
max w/llCol aS1 E p (.) (Zy zx)]
(10)
5.1
Dataset
-E -e (Zy 2.)].
We employ two widely-used datasets evaluate fairness in the context of text classification, build- ing upon prior research (Ravfogel al., 2020; Han et al., 2021c; Shen et al., 2022a). Both datasets are readily available in FairLib.
In this context, Cl is MLP, referred to as the critic in Figure To enforce the Lipschitz constraint, we clamp the weights to given values ([-0.01, 0.01]) at each optimization step². For a batch of documents, the critic takes as input the concatenation of zy and 2, on the one hand, and the concatenation of Zy and 2a randomly drawn from the dataset (equivalent to By >(zz)) 2, p(2,)), on the other hand. We then follow the training pro- cedure introduced by Arjovsky et al. (2017) which alternate maximizing Equation 10 in the cri tritic parameters for ne iterations and minimizing Equa- tion 9 for ny iterations in the fy classifier param- eters. The overview of the training process de- tailed in the appendix B.3
Bias in Bios (De-Arteaga et al., 2019). This dataset, referred to as 'Bios dataset' in the rest of the paper, consists of brief biographies associated with occupations (a total of 28) and genders (male or female). As per the partitioning prepared by (Ravfogel al., 2020), the training, validation, and test sets comprise 257, 000, 10, 000 and 99, 000 samples, respectively.
Moji (Blodgett et al., 2016). This dataset con- tains tweets written in either "Standard American English" (SAE) or "African American English" (AAE), annotated with positive negative polarity. We use the dataset prepared by (Ravfogel et al., 2020), which includes 100, 000 training examples, 8, 000 validation examples, and 8, 000 test exam- ples. The target variable y represents the polarity, while the protected attribute corresponds to the eth- nicity, indicated by the AAE/SAE attribute.
Training the demonic model We pre-train the demonic model to predict the sensitive attributes and do not update the demonic weights during the training phase of the main model. The benefits are twofold. Firstly, unlike previous works (Caton and Haas, 2020). we require only limited access to sensitive attributes label at training and we do not need access the labeling of sensitive attributes in the inference regime. As result, WFC is highly compatible with recent regulations (e.g.. US Con- sumer Financial Protection Bureau). Secondly, the demonic model can now be trained in a few-shot fashion if some examples of the training set are annotated with sensitive attributes. However, when no sensitive attributes are available in the training set, we replace the training data of the demonic part of the architecture with data from another domain (e.g. another dataset) containing sensitive informa- tion for the same attribute. For example, for gender,
5.2 Baselines
Except for the classical cross-entropy loss without fairness constraint (CE) that we run ourselves, we report the results from (Shen et al., 2022a; Han et al., 2022) these two datasets. The consid- ered baselines are INLP (Ravfogel et al., 2020), the ADV method (Han al., 2021c), FairBatch (Roh al., 2021), GATE (Han al., 2021a) and Con, displaying the dy and eo versions (Shen et al., 2022a).
5.3 Evaluation Tasks
WW also tested some more recent improvements of Lips- chitz constraint enforcement (Gulrajani et al., 2017; Wai et al., 2018). Interestingly, all lead to poor performance
For training a vanilla text classification model, we follow the protocol proposed by Han et al. (2022):

---

frozen BERT encoder followed by 3-layer MLP. We use accuracy to assess the classification perfor- mance. Fairness for all models is evaluated against three metrics presented earlier: GAP, referred to as "Fairness" in previous works (Han et al., 2022;
Methods
90
INLP
Adv
Gate
Shen et al., 2022a), and the Distance To Optimum (DTO) proposed by Han et al. (2021a) (we follow the methodology of Shen et al. (2022a) and evalu- ate the DTO on the average fairness and accuracy of the best empirical results for each metric over all models build the utopia point). Finally, we consider the Leakage score.
Faimess
FairBatch
EOGLB
Condp
75
WFC
70
65
71
73
74
75
76
Accuracy
Task 1: Fair Classification We first compare our method against state-of-theeart approaches. We use the representation generated by a base BERT model an input to the MLPs. For Bios, the demonic MLP is trained 1% of the training set and obtains 99% accuracy for predicting the sensitive attributes the test Similarly, the demonic MLP obtains 88.5% accuracy on Moji.
91
90-
x
Methads
Fairness
INLP
Adv
FairBatch
Task Demonic transfer We conduct these experiments for Bios and train a demonic MLP either on the EEC dataset (Kiritchenko and Mo- hammad, 2018) or the Marked Personas dataset (Cheng et al., 2023). We then evaluate the perfor- mance of the demonic MLP to predict gender the Bios test dataset. When training the EEC dataset obtain 98.1% % of accuracy, and on the Marked Personas dataset, we obtain 98.4% of ac- curacy. We repeat Task with those variants of the demonic MLP. We focus on Bios in this exper- iment. For Moji, it would require to have access to other datasets with the same protected attribute (ethnicity).
86
EDGLB
Condp
85
Coneo
WFC
84
82.5
83.0
Accuracy
Figure 3: Visualization of the fairness- trade- off on (a) Moji and (b) Bios. Values correspond to the average results.
the representation 2s- reproduce Task 1, but replace 2, with the sensitive attributes predicted by the demonic MLP: and concatenate 2y and & dependently and independently when computing the Wasserstein distance. Note that we do not en- counter a problem with the non-differentiability for 9 (with the argmax operation as for & as men- tioned in Section 4.1) since the demonic model is pre-trained.
Use of representations from different layers In the previous experiments, following approaches presented in (Han et al., 2022), the Wasserstein distance is approximated using the last hidden representations of the 3-layer MLP. We com- pare this approach, on both datasets, with the use of the first hidden representations of the MLP and with the output logits. For the latter, the Wasser- stein is estimated between distributions of different dimensions: for example, in the of Bios, for the demonic MLP corresponding to the sensi- tive attributes and 28 for the classification MLP corresponding to the labels.
Task 3:
5.4 Implementation Details
Our architecture is composed of three components: two classifiers and a critic. The details of the MLPs used to parameterize each component are given in Appendix B. We find the best hyperparameters for our models by grid-search coss-validation over the MLP and Critic learning rates, the value of na (number of batches used to train the main MLP), the layers producing 2, and zy, the value of B and the value used to clamp the weights to enforce the Lipschitz constraint. The values allowing us
Task 4: Independence with predicted hard sen-
sitive attributes To evaluate the impact of using

---

Table 1: Results on Moji (top) and Bios (bottom). For baselines, results are drawn from (Shen et al., 2022a). We report the mean + standard deviation over 5 runs. indicates the model without fairness consideration.
DTO (INLP, FairBatch, and Adv), WFC improves either the performance or the fairness. Especially, WFC has the second-best accuracy compared to baselines. Finally, we note that WFC is more sta- ble in terms of Fairness compared with other ap- proaches having on average the best results for this metric (along with smaller standard deviation). Eventually, even when our method does not out- perform the baselines (e.g., Bios dataset), it still exhibits noteworthy properties, particularly its abil- ity achieve competitive performances without access to the sensitive attributes in the training set. 86.9 0.2
Model
Accuracy
Fairness 1
DTO
Leakage
"CE
0.5
61.2 1.4
31.0
87.9 3.3
INLP
7.0
85.6 0.0
7.02
86.7 + 0.6
Adv
7..6
9.
1.71
78.8 =
Gate
7.3
90.1 11:::5
1.90
100.0 1: 0.0
FairBatch
7..1
90.6+ 0.5
1.78
88.4 0.4
EOGLB
0.2
9.4
2.15
85.7 = 1.2
Condp
0.3
8.6
3.92
54.2 + 0.9
Coneo
7.71
8..1+3.0
8.17
80.1 + 4.2
WFC
0.1
91.4
1.17
Model
Accuracy
Fairness
DTO
Leakage
We evaluate this capability in the next subsection.
"CE
0.2
0.8
5.67
98.0 =.0
We also explore the ability of our proposition to improve the leakage. We initially aim at improving the fairness while maintaining the accuracy of the model. Yet, our method allows to improve leakage by increasing the value of | in Equation 9, in other words, we give more importance to the Wasserstein regularization the loss. We note in Table 2 that with a higher 3, the leakage decreases. However, on both datasets, the accuracy, that we want to pre- serve, decreases and the trade-off worsens we get better for the leakage. To sum up, reducing leak- age makes it more challenging to retrieve sensitive attributes but could result in unintended informa- tion loss needed for the classification task affecting the performance. Ultimately, we want to enhance fairness while kceping a good performance and this objective may not necessarily match with a strong leakage improvement.
INLP
82.3 81.9
8.0
2.44
97.6
Adv
0.5
88.6 4.6
Gate
83.7 0.2
9.94
0.20
100.0
FairBatch
82.2 0.1
1.3
1.86
98.0 0.3
EOGLB
81.7 0.4
8.
2.97
97.2 0.5
Condp
82.1 0.2
0.8
6.50
76.3 1 1.5
Conco
818 0.3
0.4
5.72
84.9 3.4
WFC
$2.4
89.0
96.5
Table 2: Evaluation of the impact of parameter 3 (Equa- tion 9) on the model's leakage. Results on Moji (top) and Bios (bottom).
B
DTO
Accuracy f Fairness
Leakage
1
1.2
75.2
91.4
86.9
4.2
72.1
93.4
81.1
10
5.8
70.5
92.1
81.6
20
7.6
68.6
92.1
84.1
DTO
Accuracy t
Fairness t
Leakage +
1
2.1
82.4
89.0
96.5
5
2.3
81.8
89.3
87.4
10
2.8
81.8
88.5
88.8
6.2
Task 2: Demonic Transfer
20
3.6
81.3
88.0
81.8
For this task, we pre-train the demonic model on other datasets. Table 3a shows that we achieve sim- ilar results as when the pre-training is done using the same dataset. The average loss of accuracy and fairness are not significant. These results are promising for improving faimesss, especially situ- ations where collecting sensitive data is not feasible or when only partial information is accessible.
obtain the lower DTO during this process are presented in Appendix B. The architecture details of the MLP for the leakage are provided in (Shen et al., 2022a) as we use the same configuration.
6 Results
6.1 Task 1: Fair Classification
6.3 Task 3: Use of representations from different layers
We compare WFC with text classification baselines. For Moji, (Table 1 and Fig. 3), accuracy of WFC is higher than the accuracy of CE and it is equiv- alent to competitors. On the fairness metrics, outperform all other baselines and obtain the best DTO. For Bios (Table 1 and Fig. 3), our method is competitive with the other baselines and ranks out of terms of accuracy-fainne fairness trade-off. In comparison, with equivalent methods in terms of
On both datasets (Table 3b), accuracy is rather sta- ble regardless of the layers used to compute the Wasserstein distance. Still, the best results are ob- tained using the last hidden representations. How- ever, while we note slight decrease in fairness on Bios when using representations from other layers, the decrease becomes much more significant on Moji. Using the last hidden layer is the best option.

---

Table 3: Summary the results for tasks 2, and 4. We report the mean = standard deviation over rums for all tasks. Boldface numbers the best results, and indicates that the difference is statistically significant according to signed-rank Wilcoxon test (i.c. with a p-value lower than 0.01).
Data
Accuracy
Fairness
DTO
Leakage +
Bios 1%
82.4 +: 0.1
89.0 = 0.3
2.06
96.5 + 0.5
EEC
82.2 + 0.4
88.9 = 0.4
2.26
97.5 + 0.3
MP
82.4 + 0.3
88.9 = 0.4
2.14
96.4 + 0.5
(a) Task 2: comparison between several scenarios for training the demonic model for prediction on Bios.
Layer
Accuracy 1
Fairness f
DTO 4
Leakage
Bios
Last hid.
82.4 + 0.1*
89.0 + 0.3*
2.06'
96.5 0.5
First hid.
81.9 1: 0.2
86.7 1: 0.4
4.29
96.5 0.6
Last lay.
82.1 + 0.6
87.5 + 0.3
3.49
87.0 + 1.1"
Moji
Last hid.
75.2 - 0.1*
91.4 + 0.3*
1.17'
86.9 + 0.2
First hid.
74.3
80.8 + 1.0
11.4
85.6 0.6
Last lay.
73.5 0.0
70.2 0.2
21.9
64.5
0.1*
(b) Task 3:
comparison between the of representations of different MLP layers to compute the Wasserstein.
Labels
Accuracy 1
Fairness
DTO
Leakage |
Bios
Representations
82.4 1: 0.1
89.0 i 0.3"
2.06
96.5 1 0.5
Hard labels
82.6 0.2
87.5 1 0.2
3.28
0.2
Moji
Representations
75.2 0.1*
91.4 - 0.3"
1.17"
86.9 0.2
Hard labels
72.2 0.1
65.0 + 0.0
27.3
81.0 1 0.8"
(c) Task 4: comparison between the use representations 2, and hard sensitive attributes to compute the Wasserstein distance.
6.4 Task 4: Independence with predicted hard sensitive attributes
both training and inference time. We obtain com- petitive results compared to baselines on the Bios dataset and outperform them on the fairness score with comparable accuracy on Moji dataset. Further- more, we present solution for our algorithm to be trained when sensitive attributes are not available for given dataset, paving the way for its under realistic applications. In further studies, we will focus on applying this method using different encoders or decoders, datasets, and downstream tasks, as this method can generalize to tasks out of the text classification scope, notably, regression and even unsupervised objectives.
We replace by the predicted & to compute the Wasserstein distance and report the results in Table 3c. We observe, on average, a slight improvement of the accuracy Bios, and a slight decrease in accuracy on Moji. However, while the decrease in fairnsss is not significant for Bios, we observe a result, using instead n seems to have a neutral impact at best, this may also result, in some cases, in a reduction of both accuracy and fairness.
a substantial drop for Moji.
7 Conclusion
Limitations
We presented WFC a novel method that enforces fair- ness constraints using pre-trained neural network the sensitive attributes and Wasserstein regular- ization. Our model is theoretically well-motivated and has interesting properties over existing mod- cls. The most important one is the fact that does not require annotation of the sensitive attribute at
The proposed approach is rather flexible as it can handle various types of sensitive attributes. How- ever, due the lack of available datasets, we were not able assess our performance for continuous sensitive attributes, e.g. age. addition, we are aware that gender may embrace a n-ary definition, in all our experiments, we were limited to consider-

---

ing only men vs women classification, due to data availability.
Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bis"" in nip. In Proceed- ings of ACL, pages 5454-5476.
For Task 2 defined in the experiments section, we were able to show empirically that our method works well when the demonic model is pre-trained on a different dataset when no sensitive attributes or very few of them are available on the main training dataset. However, we do not provide sufficient generalization guarantees to consider out-of-the- box large-scale deployment. The next step will be derive theoretical guarantees inspired by the results of domain adaptation to assess how well this idea can be generalized to other data and under which conditions it might fail or succeed.
Su Lin Blodgett, Lisa Green, and Brendan O'Connor. 2016. Demographic dialectal variation in social me- dia: A case study of african-american engliss arXiv preprint arXiv: 160Xi
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to home- maker? debiasing word embeddings. NewrlPS, 29.
Laura Cabello, Anna Katrine Jørgensen, and Anders Søgaard. 2023. On the independence of association bias and empirical fairness in language models. Proceedings of the 2023 ACM Conference on Fair- ness, Accountability, and Transparency, FAceT '23, page 370-378, New York, NY, USA. Association for Computing Machinery.
Finally, for Task 1 we did not perform statisti- cal test to assess the significance of the observed differences. Indeed, most of the results were re- ported from (Shen et al., 2022a) and we were un- able to retrieve the scores for each run.
Simon Caton and Christian Haas. 2020. Fairness in machine learning: A survey, arXiv preprint arXiv:2010.0405..
Ethics Statement
Myra Cheng, Esin Durmus, and Dan Jurafsky. 2023, Marked personas: Using natural language prompts to measure stereotypes in language models. arXiv preprint arXiv:2305. 18189.
We acknowledge the following concerns about our work. First, biases present in the data are US- centered, and concerning the Bias in Bios dataset genders are binary. Furthermore, to conduct research need to have access to the sensitive attributes contrary to what privacy measures rec- ommend, and the latter are annotated with risk of subjectivity.
Pengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si, and Lawrence Carin. 2021. Fairfil: Contrastive neu- ral debiasing method for pretrained text encoders.
Nicolas Courty, Rémi Flamary, and Devis Tuia. 2014. Domain adaptation with regularized optimal trans- port. In ECML PKDD, pages 274-289. Springer.
Acknowledgements
Maria De-Arteaga, Alexey Romanov, Hanna Wal- lach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In FaceT, pages 120-128.
This work was funded by the french National Agency for Research (ANR) in the context of the Diké project JANR-21-CE23-0000
Our experiments utilize the previously mentioned Fairlib framework. We would like to express our gratitude to Xudong Han for his availability and assistance in using it.
Dina Bemner-Fushnen Wendy W Chapman, and Clement McDonald. 2009. What can natural lan- guage processing do for clinical decision support? Journal of biomedical informatics, 42(5):760-772.
References
Jacob Devlin, Ming-Wo Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language under- standing. In Proceedings of NAACL-HLT, pages 4171-4186.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein generative adversarial networks. In ICML, pages 11--222..2222 PMLR.
Emily M Bender, Timmit Gebru, Angelina McMillan- Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In FAceT, pages 610-623.
Yanai Flazar and Yoav Goldberg. 2018. Adversarial removal of demographic attributes from text data. In Proceedings of EMNLP. Association for Computa- tional Linguisties.
Alex Beutel, Jilin Chen, Zhe 7hao, and Ed H Chi, 2017. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv: 170::0777700075
Susan T Fiske. 2017. Prejudices in cultural contexts: Shared stereotypes (gender, age) versus variable stereotypes (race, ethnicity, religion). Perspectives on psychological science, 12(5):711779.

---

Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya, and Tomaso A Poggio. 2015. Learn ing with wasserstein loss. NeurIPS, 28.
Masahiro Kaneko, Danushka Bollegala, and Naoaki Okazaki. 2022. Debiasing isn't enough!-on on the ef- fectiveness of debiasing mims and their social biases in downstream tasks. In Proceedings the Inter- national Conference on Computational Linguistics, pages 131--1110.
Paula Gordaliza, Eustasio Del Barrio, Gamboa Fabrice, and Jean-Michel Loubes. 2019. Obtaining fairness using optimal transport theory. In /CML, pages 2357- 2365. PMLR.
Leonid Kantorovich. 1942. On the translocation of masses. In C.R. (Doklady) Acad. Sci. URS)(N.S.), volume 37(10), pages 199-201.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky. Vin- cent Dumoulin, and Aaron C Courville. 2017. Im- proved training of wasserstein gans, Advances in neural information processing systems, 30.
Patrik Joslin Kenfack, Samira Ebrahimi Kahou, and Ulrich Aivodji. 2023. Fairness under demographic scarce regime. arXiv preprint arXiv:2307. 1308).
Xudong Han, Timothy Baldwin, and Trevor Cohn. 2021a. Balancing bias: Achieving fair- ness through balanced training. arXiv preprint arXiv:2109.085
Svetlana Kiritchenko and Saif M. Mohammad. 2018. Examining gender and race bias in two hundred sen- timent analysis systems.
Charlotte Laclau, Ievgen Redko, Manvi Choudhary, and Christine Largeron, 2021. All of the fairness for edge prediction with optimal transport. In A/STATS, pages 1782-1782. PMLR.
Xudong Han, Timothy Baldwin, and Trevor Cohn. 2021b. Decoupling adversarial training for fair NLP.
In Findings of the Association for Computational Lin- guisties: ACL-IJCNLP 2021. pages 471-477, Online. Association for Computational Linguistics.
Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021. Towards understanding and mitigating social biases in language models. ICML. pages
Xudong Han, Timothy Baldwin, and Trevor Cohn. 2021c. Diverse adversaries for mitigating bias in training. Proceedings of the 16th Conference of the European Chapter of the Association for Compu- tational Linguistics: Main Volume, pages 2760-2765, Online. Association for Computational I. Linguistics.
PMLR.
Chandler May, Alex Wang. Shikha Bordia, Samuel R Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. In Proceedings of NAACL-HLT, pages
Xudong Han, Aili Shen, Yitong Li, Lea Frermann, Tim- othy Baldwin, and Trevor Cohn. 2022. Fairlib: A unified framework for assessing and improving fair- ness. Proceedings of the The 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 60-71.
Gaspard Monge. 1781. Mémoire sur la théorie des déblais et des remblais. Histoire de l'Académie Royale des Sciences, pages 666-7044
Junhyun Nam, Hyuntak Cha. Sungsoo Ahn, Jacho Lee, and Jinwoo Shin. 2020. Learning from failure: De- biasing classifier from biased classifier. Advances in Neural Information Processing Systems, 33:20673- 20684.
Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equal- ity of opportunity in supervised learning. NeurIPS, 29.
OpenAl. Chatgpt. https: https://openai. combbo/g// chatgpt. Accessed: 2022-03-06.
Ben Hutchinson, Vinodkumar Prabhakaran, Emily Den- ton, Kellie Webster, Yu Zhong, and Stephen Denuyl. 2020. Social biases in NLP models barriers for persons with disabilities. In ACL, pages 5491-5501.
Osonde A Osoba and William Welser IV. 2017. An intelligence our image: The risks of bias and errors in artificial intelligence. Rand Corporation.
Mariana Jatobá, Juliana Santos, Ives Gutierriz, Danicla Moscon, Paula Odete Fernandes, and João Paulo Teixeira. 2019. Evolution of artificial intelligence research in human resources. Procedia Computer Science, 164:137-142.
Sherjil Ozair, Corey Lynch, Yoshua Bengio, Aaron Van den Oord, Sergey Levine, and Pierre Sermanet. 2019. Wasserstein dependency measure for repre- sentation learning. Advances in Neural Information Processing Systems, 32.
Shengyu Jia, Tao Meng, Jieyu Zhao, and Kai-Wei Chang. 2020. Mitigating gender bias amplification in distribution by posterior regularization. In Proceed- ings of the Annual Meeting of ACL.
Ho Park, Jamin Shin, and Pascale Fung. 2018. Re- ducing gender bias in abusive language detection. In Proceedings of EMNLP, pages 2799-2804.
Ray Jiang, Aldo Pacchiano, Tom Stepleton, Heinrich Jiang, and Silvia Chiappa. 2020. Wasserstein fair classification. In Uncertainty in artificial intelligence, pages 862-872. PMLR.
Sundar Pichai.
Google bard.
https:
//blog. satle--i-sserrc-uupttteeee bard- hord-google-and 2022-03-06.
Accessed:

---

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, liya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 188):
Luis Caicedo Torres, Luiz Manella Pereira, and M Hadi Amini. 2021. A survey on optimal transport for machine learning: Theory and applications. arXiv preprint arXiv:2 106.01963.
Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020. Null out: Guard- ing protected attributes by iterative nullspace projec- tion. In Proceedings the Annual Meeting of ACI., pages 7237-7256. ACL.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Maric-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro. Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint arX1v.230..13971
Laurent Risser, Alberto González Sanz, Quentin Vin- cenot, and Jean-Michel Loubes. 2022. Tackling al- gorithmic bias in neural-network classifiers using wasserstein-2 regularization. Journal of Mathemati- cal Imaging and Vision, 64(6):672-66 689.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017, Attention is all you need. NeurlPS, 30.
Tianlu Wang, Victoria Lin, Nazneen Fatema Ra- jani, Bryan McCann, Vicente Ordonez, and Caiming Xiong. 2020. Double-hard debias: Tailoring word embeddings for gender bias mitigation. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5443-5444 343-5553.
Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh. 2021. Fairbatch: Batch selection for model fairness. In International Conference on Learning Representations.
Carsten Schwemmer, Carly Knight, Emily D Bello- Pardo, Stan Oklobdzija. Martijn Schoonvelde, and Jeffrey W Lockhart. 2020. Diagnosing gender bias in image recognition systems. Socius, 6.
Xiang Wei, Boqing Gong, Zixia Liu, Wei La, and Liqiang Wang. 2018. Improving the improved train- ing of wasserstein gans: A consistency term and its dual effect. In International Conference Learning Representation (ICLR).
Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, and Lea Frermann. 2022a. Does representational fairness imply empirical fairness? In Findings of the Association for Computational Linguistics: AACI- IJCNLP 2022, pages 81-95.
Raymond W Yeung. 1991. A new outlook on shan- non's information measures. IEEE transactions information theory, 37(3):466 -744
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell, 2018. Mitigating unwanted biases with adversarial learning. In Proceedings the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335- 340.
Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, and Lea Frermann. 2022b. Optimising equal oppor- tunity fairness in model training. In Proceedings of the Conference of the North American Chapter of the A Association for Computational Linguistics: Human Associa-
Language Technologies. pages tion for Computational Linguistics.
Jieyu Zhao, Tianlu Wang. Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In Proceed- ings of NAACL-HLT, pages 629-634.
Chiappa Silvia, Jiang Ray, Stepleton Tom, Pacchiano Aldo, Jiang Heinrich, and Aslanides John. 2020. A general approach to fairness with optimal transport. In Proceedings of AAAI, volume 34, pages 3633- 3640.
Jieyu Zhao, Tianlu Wang. Mark Yatskar, Vicente Or- donez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification us- ing corpus-level constraints. In Proceedings of the 2017 Conference on Empirical Methods Natural Language Processing, pages 2979-2989.
Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jicyu Zhao, Diba Mirza, Elizabeth Belding, Kai-We- Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 1630-1640.
A
Theoretical background and Proof
A.1 Wasserstein Distance
Finding correspondences between two sets of points is longstanding issue in machine learn- ing. The optimal transport (OT) (Monge, 1781) problem offers an efficient solution to this issue by calculating an optimal one-to-one transport map between the two sets. This map is determined by considering the geometrical proximity of the
Yi Chem Tan and L Elisa Celis. 2019. Assessing so- cial and intersectional biases in contextualized word representations. NewrIPS, 32.
Ewoenam Tokpo, Pieter Delobelle, Bettina Berendt, and Toon Calders. 2023. How far can it go?: On intrinsic gender bias mitigation for text classification. arXiy e-prints, pages ar ariiy-2011.

---

Algorithm 1: WFC Algorithm
Data: D {( Vi, 8:)]=1 the training set, ne the number epochs, ne and ny the number of training iterations per epoch for the critic and the classifier respectively, a batch size nt, two neural networks (x) and he; 0), Critic Co. weight on the regularization |
for I... ?ee do
for 1 = 1. ..., Tee do
Sample {zi, Mi, 8o 8-ji=1
Rb
Encode 2a t (x, 1= 2y +- (ho(2;) )
Concatenate vectors get Zed (20,t0 Fy. 1=1
Shuffle the ž vectors.
Concatenate vectors get Zind t (20,55 Zy,i i=1
grad(w) + V ni 1 Cw (Zeep.) - -Iie Cw iidd,)) w + Adamn(w; grad(wr)))))) end for 1 = I.., nld do
Sample {zi, Vi, nh
Encode 2s + {hg(2i))<_1. By t {ho(Ti)} 1m
i=l
Concatenate vectors to get Zdep = [2,,4, i=
7ib
Shuffle the Zn,i vectors.
Concatenate vectors to get Zind [Z 2yyi 11b
L i== L(vi, (2zy)))
[ C
B Ca( (Zdep)) -i=1 Cw (Zind)))
Adam Vo L)
end
end
points in the sets. In practice, OT can be expressed problem of aligning two empirical distribu- tions Px, and Px, supported on two point sets N2 Xi = (x1 c N and X₂ = [x(() c N We consider the
A.2 Proof that MI 2z) 2 MI 8)
We have
p(ù, Zy, 8) = p(il ays 6)p(zy16)p(S) (12)
formulation of this problem (Kantorovich, 1942) where the goal is search for coupling y defined as joint prob- ability distribution over X1 x X2 with marginals Px, and F Xx, This amounts to minimizing the cost of transport w.r.t. some metric X1 x X2 > R!:
Recall that $ = ho(ze) and á ho(zy): therefore, à is fully determined by Zy. hence, p(ùl kyy 8) = p(0lzy). Then:
p(ù Zu. 8) p(1 by (pz))3))((S). (13)
Therefore, the three variables follow the markov property
min (M
(11)
PEII(PX x2)
where ( is the Frobenius dot product, M is dissimilarity matrix, i.., Mij 1(z))) (1)
$ 8 zy -> ho(Zy).
(14)
In such context, it was shown that (Yeung, 1991):
(9) with .r and [IPxx x, Px,) is set of doubly stochastic matrices. This problem admits unique solution 7 and de- fines metric on the space of probability measures called the Wasserstein distance (also known as the Earth-Mover Distance) as follows:
defining the cost of associating क
MI Z1zy 8) 2 MI(j. 8).
(15)
Following the same logic, we have
MI( 20. 2 MI(2y, 8)
(16)
hence
W M (P,,, Px,)= =
min
(M, 7) p-
Ten((xx,Px,))
MI(2,, 2.) MI(J,S) 8).
(17)

---

Dataset
BiasInBios
Moji
B.3
Algorithm of WFC
input dimension
768
2304
In Algorithm 1, we provide the detailed algorithm for WFC used in our experiments.
hidden layers
hidden dimension
300
300
learning rate
0.0001
0.00001
batch size
128
128
epochs max
10000
10000
activation
TanH
TanH
1
Tee
5
nld
20
5
clamp value
0.01
0.01
layer used
last
last
Table Details on hyperparameters used for the classi- fying MLP.
number hidden layer hidden dimension
512 ReLU Root Mean Square Propagation
activation
optimizer
learning rate
5e-5
Table 5: Details on hyperparameters used for the Critic ML..
Experimental details
In this section, we provide additional experimental details, notably, detail the MLP architectures. give the optimal hyperparameters, and describe the full algorithm of WFC.
B.1 MLP architectures
In Table 4, we present the architectural details of the classifier MLP. We grid searched over the learn- ing rate (Ir (1e-5, 5, le 4, 1e 1e-3,, 3, 5e 5, 5e 4, - 3), the number of training batches for clas- sification per epoch nd (5, 10, 20). the value used to clamp the weights enforce the Lipschitz constraint clamp value E (0.001, 0.01, 0.1). the parameter 3 E {0.5, 1, 2}. the layer used between the first hidden, last hidden, or last layer.
B.2 Critic architecture
Table we present the architectural details of the Critie, which a simple multi-layer perceptron. We grid searched over the learning rate Ir E (5e 5, 5e 4, 5e 3).


Reviews:
[{"review_id": "V5EKjqHbEA", "reviewer": "Reviewer_Xr2p", "summary": "", "questions": "See weaknesses", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Authors : First of all, we thank you for the care and attention you have given to our document, and your appreciation of our efforts in testing various settings. Similarly, we thank you for pointing out the interest of using the Wasserstein distance to enhance model fairness.\n\nWe also value your specific comments that will help in improving our work. Below, we provide clarifications to address some of your remarks and hope they answer your questions.\n\n---\n\n***Reviewer : I struggled to draw clear conclusions from the experiments. Of the two datasets considered, while GAP and DTO are improved on one of them, the approach is not able to outperform many of the baselines (in accuracy or fairness).***\n\nA : Indeed, our approach does not outperform all baselines on the Bias in Bios dataset (4th/9 in terms of accuracy fairness trade-off), but it does outperform baselines on the Moji dataset on both fairness and DTO (accuracy/fairness trade-off).\n\nAdditionally, on top of being very competitive, our method has strong advantages over prior works:  \n- it does not require annotation of the sensitive attribute at training time (matching GRPD regulations, for instance, that enforces more stringent requirements for the collection and utilization of protected attributes).\n- it can generalize to tasks out of the text classification scope.\n- it can be deployed in the case of continuous sensitive attributes.\n\nThose abilities afford a wide deployment of our approach in real-life scenarios and open the way to further work in this direction.\n\n---\n\n***R : Further, leakage results are very very poor. The representations basically leak the sensitive attributes nearly perfectly in both sets of experiments. With the motivation of the method being minimizing dependence between the label and sensitive attribute, shouldn't leakage results be better? There is no discussion provided for this behavior either.***\n\nA : While the method's motivation initially suggests better leakage, empirical fairness (Fairness) and representational fairness (Leakage) have been found to be uncorrelated and even opposing in various studies [1, 2]. We observe a similar phenomenon in INLP, which aims to remove sensitive information from the representations. Conversely, Condp significantly improves leakage but ranks poorly in terms of Fairness.  \n\nOur primary goal is to improve empirical fairness while considering the trade-off with accuracy working on the representations. As you noted, we want to reduce the dependence between the label and sensitive attribute.\nHowever, optimizing for empirical fairness may not necessarily ameliorate representational fairness. Furthermore, reducing leakage makes it more challenging to retrieve sensitive attributes but could result in unintended information loss needed for the classification task without significantly enhancing empirical fairness, as previously mentioned.\n\nStill, we provide the leakage results for baseline comparison. Moreover, adjusting experiment parameters, such as the weight put on the Wasserstein regularization, can improve leakage. If we keep only the Wasserstein regularization in the loss, we can get a leakage of 53.9 for the Bias in Bios dataset and 77.1 for the Moji dataset.\n\nWe conducted an extra experiment where we give more weight to the Wasserstein regularization in the loss. With a higher weight, the leakage decreases at the cost of accuracy. Considering $\\alpha$ and $\\beta$, respectively the weight on the classification loss and the Wasserstein regularization, we get the following results:\n\n| $\\alpha$ | $\\beta$ | DTO (Trade-off) | Accuracy | Fairness | Leakage |\n|---|---|---|---|---|---|\n| Bios |\n| 1 | 1 | 2.1 | 82.4 | 89.0 | 96.5 |\n| 1 | 5 | 2.3 | 81.8 | 89.3 | 87.4 |\n| 1 | 10 | 2.8 | 81.8 | 88.5 | 88.8 |\n| 1 | 20 | 3.6 | 81.3 | 88.0 | 81.8 |\n| Moji |\n| 1 | 1 | 1.2 | 75.2 | 91.4 | 86.9 |\n| 1 | 5 | 4.2 | 72.1 | 93.4 | 81.1 |\n| 1 | 10 | 5.8 | 70.5 | 92.1 | 81.6 |\n| 1 | 20 | 7.6 | 68.6 | 92.1 | 84.1 |\n\nOn the Bias in Bios dataset, when leakage decreases, accuracy, that we want to preserve, immediately decreases and the trade-off worsen, even tho we note an improvement of empirical fairness. However, for a certain leakage both accuracy and fairness decrease, matching the observation of previously mentioned papers. On the Moji dataset, while we note an improvement of empirical fairness, the impact on accuracy is too strong to preserve a good trade-off.\n\n---\n\n***R : The paper makes a claim over and over that their method is applicable in cases where sensitive attributes might not be available at test time. To my knowledge, this is true for all the baselines they considered.***\n\nA : Thank you for pointing out this mistake, the claim in question appears in line 97. Regrettably, this is a typographical error that has caused confusion for both you and R2. We sincerely apologize for any misunderstanding caused.\n\nOur approach, unlike current baselines, does not need sensitive attribute annotations in the *training data*, as detailed in line 420 (under \"Training the demonic model\") and empirically validated in Task 2 using the code provided on GitHub.\n\n---\n\n***R : Even if they were not available at train time, in most of the baselines considered, you can still train an attribute classification and apply their method the same as this paper does (as has been done in previous works like this: https://arxiv.org/abs/2307.13081). Using W-distance is orthogonal to this claim and the former seems to have mixed results anyway.***\n\nThank you for this additional reference that we can add to our related work. We were not able to acknowledge it since it was published after the submission deadline (24th of July only available on Arxiv).\n\nHowever, as the authors emphasize, employing proxy-sensitive attributes often worsens the fairness-accuracy trade-off, a key problem we address.\n\nFurthermore, this extension, although unrelated to the use of the Wasserstein distance as you mentioned, yields similar results to our original approach (Table 2.a) with additional benefits mentioned in the answer of the first remark. Thanks to this addition and unlike other baselines, we propose a version of our method compatible with recent regulations on the use of sensitive data.\n\n---\n\n***R : Many baselines related to INLP seem to be missing such as all the follow-up work from the same authors***\n\nA : We appreciate the recommendation of additional baselines and acknowledge that there may be more options due to the extensive work in the field. In our experiments, we made efforts to compare our work with eight baselines, including recent ones from 2022, following established protocols used in ACL conference papers (e.g., [1]).\n\n---\n\n[1] Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, and Lea Frermann. 2022. Does Representational Fairness Imply Empirical Fairness?. In Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022, pages 81–95, Online only. Association for Computational Linguistics.  \n[2] Laura Cabello, Anna Katrine Jørgensen, and Anders Søgaard. 2023. On the Independence of Association Bias and Empirical Fairness in Language Models. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23). Association for Computing Machinery, New York, NY, USA, 370–378.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Good motivation in using Wasserstein distance for improving fairness, as has been studied in image classification. It has shown to work better with GANs than adversarial learning as well so it makes sense to implement it for this setting. \n2. The experiment setup is thorough with many different settings considered in terms of the availability of sensitive attributes, what \"representations\" to compute the loss on, and so on. Good fairness improvements on some of the datasets considered.", "reasons_to_reject": "1. I struggled to draw clear conclusions from the experiments. Of the two datasets considered, while GAP and DTO are improved on one of them, the approach is not able to outperform many of the baselines (in accuracy or fairness). Further, leakage results are very very poor. The representations basically leak the sensitive attributes nearly perfectly in both sets of experiments. With the motivation of the method being minimizing dependence between the label and sensitive attribute, shouldn't leakage results be better? There is no discussion provided for this behavior either. \n\n2. The paper makes a claim over and over that their method is applicable in cases where sensitive attributes might not be available at test time. To my knowledge, this is true for all the baselines they considered. Even if they were not available at train time, in most of the baselines considered, you can still train an attribute classification and apply their method the same as this paper does (as has been done in previous works like this: https://arxiv.org/abs/2307.13081). Using W-distance is orthogonal to this claim and the former seems to have mixed results anyway. \n\n3. Many baselines related to INLP seem to be missing such as all the follow-up work from the same authors: https://proceedings.mlr.press/v162/ravfogel22a/ravfogel22a.pdf", "questions_to_authors": "See weaknesses", "paper_topic_and_main_contributions": "This paper presents a training method for improving group fairness in text classification. Posing fairness as minimizing the mutual information between the label and the sensitive attribute represented in a piece of text (such as age, gender, or race), the authors propose to operationalize this optimization as minimizing the Wasserstein-1 distance between their representations' joint probabilities and product of marginals and implement it using Kantorovich-Rubinstein duality as in WGANs. \n\nWith experiments on two datasets (with gender and race as sensitive attributes), the paper shows experiments with two cases: one where the sensitive attribute is available at training time, and the other where it is not (in which case they train an attribute classifier on a separate dataset). They report fairness metrics such as GAP, DTO, and Leakage, and show that their proposed method overall matches performance of previous methods while being fairer in terms of GAP and DTO.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jhEiYctLWQ", "reviewer": "Reviewer_nGpp", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Authors : We genuinely value your thorough review and recognition of the problem's significance. Your acknowledgment of our efforts in evaluations and the advantage of a sensitive attribute-independent training method is appreciated. Below, we address your valid concern.\n\n***Reviewer : Many of the existing debiasing approaches do not need sensitive attributes at the inference time, including the baseline methods INLP, Adv, FairBatch, etc.***\n\nAuthors : The remark you made is common with other reviewer's, regrettably, this is a typographical error line 97 that has caused confusion for both you and R1. We sincerely apologize for any misunderstanding caused.\n\nOur approach, unlike current baselines, does not need sensitive attribute annotations in the training data, as detailed in line 420 (under \"Training the demonic model\") and empirically validated in Task 2 thanks to the code provided on GitHub.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper solves an important problem\n- The paper is well-written and easy to follow\n- The proposed approach is well-motivated and evaluated \n- The proposed approach can be used without observing protected attributes during training\n- The source code has been provided, which can be used in future research", "reasons_to_reject": "- Many of the existing debiasing approaches do not need sensitive attributes at the inference time, including the baseline methods INLP, Adv, FairBatch, etc. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper focuses on group fairness in text classification and proposes to improve fairness by introducing Wassertein independence. The proposed method is extensively examined over two standard fairness benchmark datasets, demonstrating compatible results compared to SOTA debiasing methods. More importantly,  the demonic model can be pre-trained and used for the debiasing purpose, enabling semi-supervised bias mitigation. ", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7KDBhaXSfv", "reviewer": "Reviewer_6C1r", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Authors : We thank you for your review and the interest you had in our paper.\n\n***Reviewer : Since the objective is to minimize the mutual information, why adopting a low-bound for mutual information estimation? In [1], they also aim at improving fairness via minimizing mutual information, but adopt an upper-bound for estimation which is more plausible.***\n\nA : It seems that this remark may arise from a misunderstanding. As shown in equation (6), we do minimize an **upper bound** of the mutual information between the classification labels and the sensitive attribute, making the classification results independent to the sensitive attribute.\n\n***R : I think the novelty is limited, since [1] has proposed a similar idea and has a better methodology design (i.e., by using an upper-bound estimation). More importantly, [1] is neither compared or cited in the paper. The \"Wasserstein Dependency Measure\" in the paper follows previous works and is not appropriate to be used in this setting (as a lower-bound estimation).***\n\nA : Regarding the reference you mentioned, thank you for pointing it; we will include it in the related work.\n\nAccording to our understanding, this paper aims to maximize the mutual information between pairs of representations of a sentence and its augmented version, varying according to the sensitive attribute *s*, and going through the same encoder, making the input *x* independent of *s*.  \nHowever, this does not ensure independence between the prediction *ŷ* and *s* (cf. [1, 2]). In contrast, our theoretically grounded approach minimizes the mutual information between representations of the same sentence processed by two different encoders to make *ŷ* independent of *s*.\n\nAdditionally, their approach relies on identifying biased attribute words, it is limited to the case where substitute words are available. This is a constraint we avoid.\n\n---\n\n[1] Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, and Lea Frermann. 2022. Does Representational Fairness Imply Empirical Fairness?. In Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022, pages 81–95, Online only. Association for Computational Linguistics.   \n[2] Laura Cabello, Anna Katrine Jørgensen, and Anders Søgaard. 2023. On the Independence of Association Bias and Empirical Fairness in Language Models. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23). Association for Computing Machinery, New York, NY, USA, 370–378.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is written clearly.", "reasons_to_reject": "Since the objective is to minimize the mutual information, why adopting a low-bound for mutual information estimation? In [1], they also aim at improving fairness via minimizing mutual information, but adopt an upper-bound for estimation which is more plausible. \n\nI think the novelty is limited, since [1] has proposed a similar idea and has a better methodology design (i.e., by using an upper-bound estimation). More importantly, [1] is neither compared or cited in the paper. The \"Wasserstein Dependency Measure\" in the paper follows previous works and is not appropriate to be used in this setting (as a lower-bound estimation).\n\n\n[1] https://arxiv.org/pdf/2103.06413.pdf", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper propose to improve fairness for text classification via minimizing the mutual information between the task representation and representation of the sensitive attribute. The mutual is estimated with the Wasserstein distance following previous works. Experiments show that the proposed objective is effective in improving fairness.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents an approach to mitigating bias in text classifiers by inducing Wasserstein independence between representations learned to predict target labels and sensitive attributes. The paper is well-motivated and the experiment setup is thorough. The rebuttal addresses most of concerns.
Decision: Accept-Main