Everybody Needs Good Neighbours: An Unsupervised Locality-based Method for Bias Mitigation
Authors: Xudong Han
Timothy Baldwin
Trevor Cohn
Venue: ICLR 2023 poster (2023)

Abstract: Learning models from human behavioural data often leads to outputs that are biased with respect to user demographics, such as gender or race. This effect can be controlled by explicit mitigation methods, but this typically presupposes access to demographically-labelled training data. Such data is often not available, motivating the need for unsupervised debiasing methods. To this end, we propose a new meta-algorithm for debiasing representation learning models, which combines the notions of data locality and accuracy of model fit, such that a supervised debiasing method can optimise fairness between neighbourhoods of poorly vs. well modelled instances as identified by our method. Results over five datasets, spanning natural language processing and structured data classification tasks, show that our technique recovers proxy labels that correlate with unknown demographic data, and that our method outperforms all unsupervised baselines, while also achieving competitive performance with state-of-the-art supervised methods which are given access to demographic labels.

Published as a conference paper at ICLR 2023
EVERYBODY NEEDS GOOD NEIGHBOURS: AN UNSU- PERVISED LOCALITY-BASED METHOD FOR BIAS MIT- IGATION
Xudong 2.2
Timothy Baldwin
Trevor Cohn'
The University of Melbourne
2 Mohamed bin Zayed University of Artificial Intelligence (MBZUAD)
xudongh1@studenn unime lb. . edu. au (tbaldwin, t. cohn @unimelb, edu. au
ABSTRACT
Learning models from human behavioural data often leads to outputs that are bi- ased with respect to user demographics, such as gender or race. This effect can be controlled by explicit mitigation methods, but this typically presupposes access to demographically-kan
training data. Such data often not available, motivat- ing the need for unsupervised debiasing methods. To this end, we propose a new meta-algorithm for debiasing representation learning models, which combines the notions of data locality and accuracy of model fit, such that supervised debiasing method can optimise fairness between neighbourhoods of poorly vs. well mod- elled instances identified by method. Results over five datasets, spanning natural language processing and structured data classification tasks, show that our technique recovers proxy labels that correlate with unknown demographic data, and that our method outperforms all unsupervised baselines, while also achiev- ing competitive performance with state-of-the-ar supervised methods which are given access to demographic labels.
INTRODUCTION
It is well known that naively-trained models potentially make biased predictions even if demographic information (such as gender, age, or race) not explicitly observed in training, leading discrim-
ination such as opportunity inequality (Hovy & Søgaard 2015; Hardt et al. 2016). Although
range of fairness metrics (Hardt et al. 2016 Blodgett et al. 2016) and debiasing methods (Elazar & Goldberg 2018 Wang et al. 2019 Raviogel et al.) (0200) have been proposed to measure and improve fairness in model predictions, they generally require access to protected attributes during training. However, protected labels often not available (e.g., due privacy security con- cerns), motivating the need for unsupervised debiasing methods, i.e., debiasing without access to demographic variables. Previous unsupervised debiasing work has mainly focused improving the worst-performin groups, which does not generalize well to ensuring performance parity across all protected groups (Hashimoto et al. 2018 Lahoti et al. 2020).
In Section 3 we propose new meta-algorithm for debiasing representation learning models, named Unsupervised Locality-based Proxy Label assignment (ULPL). As shown in Figure to minimize performance disparities, ULPL derives binary proxy labels based on model predictions, indicating poorly- vs. well-modelled instances. These proxy labels can then be combined with any supervised debiasing method to optimize fairness without access to actual protected labels. The method is based on the key observation that hidden representations are correlated with protected groups even protected labels not observed in model training, enabling the modelling of unobserved protected labels from hidden representations. We additionally introduce the notion of data locality to proxy label assignment, representing neighbourhoods of poorly- well-modelled instances in nearest- neighbour framework.
In Section 4. compare the combination of ULPL with ofate-of-the-a supervised debiasing meth- ods on five benchmark datasets, spanning natural language processing and structured data clas- sification. Experimental results show that ULPL outperforms unsupervised and semi-supervised

---

Published as a conference paper at ICLR 2023
Grabl-unlabeled data
Encoder
Classifier
x
N/A
Loss
Neighborhood Smoothing
Proxy Label Assignment
Over-
Under-
performing
performing
Loss
Figure I: An overview of ULPL. Given model trained to predict label y from z by optimizing a particular loss, we derive binary proxy labels over- vs. under- paddrrperforming within each target class based training losses. These proxy labels are then smoothed according the neighbourhood in latent space. Finally, the group-unlabeled data is augmented with =', enabling the application of supervised bias mitigation methods.
baselines, while also achieving performance competitive with hate-of-the-eee supervised techniques which have access to protected attributes at training time.
In Section S we show that the proxy labels inferred by our method correlate with known demo- graphic data, and that it is effective over multi-class intersectional groups and different notions of group-wise fairness. Moreover, we test our hypothesis of locality smoothing by studying the pre- dictability of protected attributes and robustness hyperparameters in finding neighbours.
RELATED WORK
Representational fairness One line of work in the fairness literature is protected information leakage i.e., bias in the hidden representations. For example, it has been shown that protected information influences the geometry of the embedding space learned by models (Caliskan et al, 2017 May et al., 2019). Previous work has also shown that downstream models learn protected information such authorship that unintentionally encoded in hidden representations, even if the model does not have access to protected information during training (Li et al.) 2018 Wang et al., 2019 Zhao et al. 2019 Han al. (021b). Rather than reduce leakage, in this work, make use of leakage as robust and reliable signal of unobserved protected labels and derive proxy information from biased hidden representations for bias mitigation.
Empirical fairness Another line work focuses on empirical fairness by measuring model per-
formance disparities across protected groups. e.g., via demographic parity (Dwork al. 2012
equalized odds and equal opportunity (Hardt al. 2016), or predictive parity (Chouldechova) 201
Based on aggregation across groups, empirical fairness notions be further broken down into group-wise fairness, which measures relative dispersion across protected groups (Li et al., 2018 Ravfogel al. 2020 Han al. 2022a Lum al. 2022). and per-group fairness, which reflects extremum values of bias (Zafar et al. 2017 2017|Feldman Feldman et al. 2015 Lahoti et al.] 2020). We follow previous work (Ravfogel al. 2020 Han et 202Ib Shen et al. (022) focusing primarily on improving group-wise equal opportunity fairness.
Unsupervised bias mitigation Recent work has considered semi-supervise bias mitigation, such as debiasing with partilll protected attributes (Han et al. 2021a),, noised protected la- bels(Awasthi et al. 2020 Wang et al. 202T Awasthi al.] (021), domain adaptation of protected attributes (Coston et al. 2019 Han et al.] 2021a). However, these approaches are semi- sumi-sspee true protected labels still required for optimizing faimess objectives.
Although Gupta et al. (2018) has proposed to use observed features as proxies for unobserved pro- tected labels, the selection of proxy features handcrafted and does not generalize unstructured
2

---

Published as a conference paper at ICLR 2023
imputs (e.g., text images). Therefore, there is no guarantee of correlation between proxy labels and unobserved protected labels (Chen et al. 2019).
The most relevant line of work focuses the notion of Max-Min faireess (Rawls) 2001), which aims to maximize the minimum performance across protected groups. [Hahimoto et al. (018)) optimize worst-perfomiing distributions without access to actual protected labels, but suffer from the risk of focusing on outliers, reducing the effectiveness of bias mitigation. Adversarially reweighted learn- ing (ARL) (Lahoti et al. 2020) extends the idea by employing additional adversary in training to prevent the optimization from focusing on noisy outliers, based on the notion of computational iden- et al., 2017). However, adversarial training is notoriously non-convex, and there is no guarantee that the adversary will learn contiguous regions rather than identifying outliers. In contrast, our proposed neighbourhood smoothing method is memory-based, does not require adversarial training, and one can explicitly adjust the smoothness of neighbourhood search.
tifiability (Hébert-Johnsee
Unsupervised fairness evaluation To access fairness without demographics, recent work (Kallus et al. 2020) has proposed to measure fairness w.r.t. auxiliary variables such as surname and geolo- cation different datasets, which a different research topic and beyond the scope of this paper. In this paper, we use protected labels for tuning and evaluation, and in practice, one can employ our unsupervised debiasing methods together with unsupervised fairness evaluation approaches to perform hyperparameter tuning for better fairness.
Dataset cartography Training instances are also grouped based on predictability in the literature on dataset cartography, which is similar to the assignment of proxy labels in this paper. (Swayamdipta) ct al. (2020) propose to visualize training instances according to variability and confidence, where a higher-confidenee indicates the instance label can be predicted more easily. Le Bras et al. (2020) also group training instances by their predictability, measured by training simple linear discriminators. Such methods focus improving in- and catto--distribution performance without taking fairness into consideration. In comparison, our proposed method aims to mitigate bias by assigning proxy protected group labels to training instances based on their losses within particular class.
3 METHODS
3.1 PROBLEM FORMULATION
Consider a dataset consisting of n instances D the classifier, y; E [1...., C] represents target class label, and [1...., G] is the group label, such as gender. For unsupervised bias mitigation, protected labels are assumed to be unobserved at training and inference time. Nec. denotes the number of instances in subset with target label с and protected label g. i.., De.g = {(I,, Y:: 2)ly; = c, Z; = = 1- A vanilla model (m = fo e) consists of two connected parts: the encoder is trained to compute the hidden representation from input, h = e(x),, and the classifier makes prediction, y [(h).
Y;, y;, where æ; is an input vector to
Let ( De.s E(..,,2)ED. {mm(x:). y.) be the average empirical risk for De.g where e is loss function such as cross-entropy. Similarly, L denote the average for instances with target label c (Dc), and L denote the overall empirical risk.
Fairness measurement We follow previous work measuring group-wise performance dispari- ties (Ravfogel et al., [2020] Roh et al., [021] [Shen et al.) (022). Specifically, for particular utility metric U, e.g., the true positive rate, the results for each protected group are C-dimensional vectors, one dimension for each class. For the subset of instances Deg, we denote the corresponding evalua- tion results as Uc.g. Let Ug denote the overall utilities of class c, then group-wise fairness is achieved if the utilitics of all groups are identical, Ue.g Uc, Vc, g |Uog Ue = 0, Vc, g. In addition to the overall performance metric U, we denote the fairness metric F, as a measurement of group-wise utility disparities.
3.2 UNSUPERVISED LOCALITY-BASER PROXY LABEL ASSIGNMENT
Proxy label assignment To mitigate bias, the first question is how to minimize disparities of non-differentiable metrics in model training. Previous work has shown that empirical risk-based
3

---

Published as a conference paper at ICLR 2023
objectives form a practical approximation of expected fairness, measured by various met- ries including AUC-ROC (Lahoti et al., 2020), positive rate (Roh et al. (021), and true positive rate (Shen al. 2022). Without loss of generality, we illustrate with the equal opportunity faireess. Note that our method generalizes to other fairness criteria, see Appendix D for detailss.
By replacing the utility metrics U with empirical risks w.r.t. appropriate loss function 4, the group-wise faimess metrics are reformulated as E Σ - _ = 0, which an approx- imation of the desired fairness measurement Ee=1 Eي= Uveg - Ue| O. However, protected labels (z) we optimize fairness objectives with unobserved protected labels?
observed in unsupervised dehiasing settings, which raises the question: how can
Based on the fairness objective Ec= Ex= - -9 O, we propose to focus on groups within each target class that are systematically poorly modelled. To this end, we binarize the numerous unobserved group labels into two types based on their training losses: Z 1 where z denotes the augmented proxy group label. The two types of protected labels indicate that the loss of an instance is either greater than the mean (an "under-represented" group) or S the mean (an "over-representtd" group). Each instance can now be assigned with a binary proxy label and used with existing debiasing methods, resulting in augmented datasets D {(x,, Yo, 2))
Neighbourhood smoothing Simply focusing worse-performing instances can force the clas- sifier to memorize noisy outliers (Arpit al. 2017), reducing the effectiveness of bias mitigation. To address this problem, we find the neighbourhood that is likely to be from the same demographic based on the observation of protected information leakage (introduced in Section 2 Section 5.2 and smooth the proxy label of each instance based on its neighbours.
and justified in
Specifically, we use the notion of data locality and adopt a k-Nearest --Nerret--Niighbour classifier (k--NN to smooth the proxy label. Given hidden representations {h1, , h2 ................................ hn), where h, e(x;), and a query point hj. k-NN searches for the h points [h, hj....., j J, s.l. Y, = y/, is closest in distance to h,, and then makes predictions through majority vote among proxy lábels (7/ ...,..,,,,,. 2 J. Unlike the standard setting of k-NN, where the query point is excluded from consideration, we include the query instance. As a result, the smoothing process degrades to us- ing naive proxy labels when k = I, where the discriminator prediction is the original proxy label without smoothing. For 1, on the other hand, neighbourhood smoothing comes into effect.
Proxy label assignment and neighbour smoothing can be applied at different granularities, such as calculating the loss at steps vs. iterations; see Appendix C.I for details.
3.3 THEORETICAL JUSTIFICATION
Approximating Fairness Criteria In multi-class classification settings, the equal opportunity fair- ness is achieved if y I zly, Vy, i.e., the true positive rates (TPR) of each target class are equal for all partitions of the dataset, where partitioning based on z.
Using the definition of cross-entropy of the i-th instance, Ee_ 1 1 (c) log(p(8; = c)) where ý,
is a function of æ; the loss for the subset of instances with target label can be simplified as:
Σ
- -og(pp(; =c)
= Te (=)))
Σ - log(p(s = cly, = c))
(1)
Notice that L is calculated on the subset D. = {(x,, z:)ly, = c) making L an unbiased estimator of - log p(y = cly; = c), which approximates loo TPR of the c-th class. As such, it can be seen that TPR can be empirically replaced by cross-entropy loss when measuring EO fairness.
Fairness Lower Bound Consider the worst case in fairness, e.g. p(S = 1,y = 1/z 1) v 0 and p( 1,y 1/z 2) s3 1, where the TPR gap between the two groups is 1. Such unfairness in training is shown as the minimum training loss of instances in group being larger than the maximum loss of instances in group 0. Taking the proxy label assignment into consideration, this example results in the strong correlation between the gold-class group label and proxy group labels, i.e, p(z' = 1|= 1) p(z' 0f= 2) x 1. Therefore, the correlation between proxy labels
4

---

Published as a conference paper at ICLR 2023
and gold-class group labels is positively correlated with unfairness, and the optimization w.r.t. proxy labels increases the lower bound of fairness.
4 EXPERIMENTAL RESULTS
This section demonstrates the effectiveness of proposed method through experiments against various competitive baselines and across five widely-used datasets. We report evaluation results of all models based on average values over five runs with different random seeds for each dataset.
4.1 EXPERIMENT SETUP
Datasets We consider the following benchmark datasets from the fairesss literature: (1) Moji (Blodgett et al., 2016, Elazar & Goldberg 2018), sentiment analysis with protected attribute race; (2) Bios (DeAArteage et al, ( |Subramanian et al. (202T), biography classification with pro-
tected attributes gender and economy: (3) TrustPilot (Hovy et al.] (015), product rating prediction with protected attributes age, gender, and country; (4) COMPAS (Flores et al. 2016), recidivism prediction with protected attributes gender and race; and (5) Adult (Kohavi 1996), income predic-
tion with protected attributes gender and race.
To enable thorough comparison and explore correlation with unobserved protected attributes, for datasets with more than one protected attribute, we treat each protected attribute as a distinct task, e.g., Bios-gender, and Bios-economy are treated as two different tasks. As result, there different tasks in total.
Baselines We employ the following baselines: (1) Vanilla, which trains the classifier without ex-
plicit bias mitigation; (2) FairBatch (Roh al. (021), which adjusts the resampling probabilities of each protected group for each minibatch to minimize loss disparities; (3) GDCLA (Shen et al. 2022). which adjusts the weights of each protected group to minimize loss disparities; (4) GDGLB (Shen et al. 2022), which is variant of GDCLA that additionally minimizes loss differences across tar- get classes; (5) Adv (Li et al., 2018), which trains the adversary to identify protected information from hidden representations, and removes protected information through unlearning adversaries; (6) SemiAdv (Han et al. 2021a), which trains the adversary with partially-observed protected la- bels; and (7) ARL (Lahoti et al. 2020), which employs an adversary to assign larger weights to pnppuuaaiionaly-ddeniiiia
underrepresented instances. Besides the Vanilla model, methods (2)- (5) are supervised debiasing baselines, SemiAdv is semi-supervised debiasing baseline method, and ARL is the baseline for unsupervised bias mitigation.
In terms of our proposed method, we examine its effectiveness in combination with several super- vised debiasing methods, GDCLA. GDGLB. and Adv, denoted ULPL+GD ULPL+GD and ULPL+Adv, respectively. To be clear, the supervision in each case based the proxy labels z learned in an unsupervised manner by ULPL.
This paper is generalizable to different metrics by varying the objectives of the debiasing methods. For illustration purposes, we follow RRvvogeel (202)) and Han et al. (2021a) in measuring the overall accuracy and equal opportunity fairness, which measures true positive rate (TPR) disparities across groups. Consistent with Section 3.1 we measure the sum of TPR gap across subgroups to capture absolute disparities. We focus on less fair classes by using root mean square aggregation for class-wise aggregation. Overall, the fairness
Evaluation Metries
(2020); Shen et al.
metric is F = v t E
TPRR. - TPR.)) For both metrics, larger is better. DE
Model comparison Previous work has shown that debiasing methods suffer from performance- fairness trade-offs in bias mitigation (Shen et al. 2022).. Most debiasing methods involve trade-off hyperparameter to control the extent which the model sacrifices performance for fairness, such as λGD the strength of additional regularization objectives of GD and our proposed method ULPL+GD As shown in Figure 2a given the serformmance-firrnes, trade-offs, selecting the
Key characteristics of the datasets, including dataset statistics, are provided in Appendix
5

---

Published as a conference paper at ICLR 2023
Faimess
2'40Dcy
Vanilia Perfomance
2'r Perforancce
1.00
0.8
Falmess
0.95
Performenc
0.6
Faimees
0.90
0.90
z'+ 2'+GDcu
0.2
0.85
GDcu
0.0
0.0
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Performance
(a) Tuning ULPL+GDOLA trade-off hyperparameler. Shaded areas = 95% CI estimated over 5 runs.
(b) Performance-fairnes, curve of GDcLA (red dashed line), and ULPL+GDOLA (black solid line).
Figure 2: ULPL+GDOLA degrades Vanilla performance (dotted lines) when setting À 0. As increase the weight of the fairness objective, fairness (orange dashed line) improves at the cost of performance (blue solid line). Figure 2 focuses on the Pareto frontier, and presents AUC-PFC as shaded area over the Bios
Bios-gender gender dataset.
Dataset
Moji
Bios
TrustPilot
Adult
COMPAS
Attribute
R
G
E
G
A
C
R
G
R
Vanilla
0.172
0.471
0.490
0.130
0.128
0.125
0.092
0.082
0.111
0.101
GDCLA
0.249
0.498
0.507
0.133
0.131
0.132
0.095
0.084
0.124
0.100
GDeL.
0.230
0.480
0.495
0.132
0.131
0.131
0.096
0.085
0.124
0.102
FairBatch
0.245
0.482
0.495
0.130
0.131
0.130
0.094
0.083
0.119
0.106
Adv
0.247
0.484
0.494
0.132
0.132
0.129
0.094
0.084
0.120
0.102
SemiAdv
0.247
0.483
0.495
0.131
0.132
0.131
0.094
0.082
0.121
0.097
ARL
0.193
0.461
0.484
0.129
0.132
0.129
0.097
0.086
0.123
0.096
ULPL+GDDLA
0.209
0.485
0.503
0.133
0.132
0.131
0.093
0.082
0.126
0.105
ULPL+GDGLB
0.190
0.474
0.495
0.133
0.132
0.131
0.094
0.085
0.127
0.105
ULPL+Adv
0.185
0.472
0.493
0.132
0.131
0.131
0.094
0.084
0.125
0.101
Table 1: AUC-PFC on five datasets w.r.t. different protected attributes: G: gender; E: economy;
A: age; C: country; Rtrace. Debiasing methods are introduced in Section AUC-PFC scores are calculated based on trade-off curves averaged over 5 repeated runs with different random seeds.
best performance degrades to vanilla training and choosing the best faimess results in random pre- dictions. As such, performance and faimess must be considered simultaneously in model com- parisons, such as early stopping and hyperparameter tuning. We use protected attributes for early stopping over validation set and report results the test In practice, model selection should be made in a domain-specific manner, where the best method varies. To make quantitative com- parisons based on the performance-Fairees trade-offss we follow Hann et al. (2023) in reporting the area under the performance-fairnes. trade-off curves (AUC-PFC) of each method. As shown in Fig- ure 2b] the performance fairness trade-off curve (PFC) of particular method consists of Pareto frontier, which represents the best results that can be achieved in different scenarios, and the area under the curve based on PFC (AUC-PFC) reflects the overall goodness of method. In partieular, the AUC-PFC score of GD (red) and ULPL+GD (blue) are 0.498 and 0.485, respectively, and their difference (i.e., area between the two curves) 0.013. See Appendix for further details.
4.2 CAN WE MITIGATE BIAS WITHOUT ACCESS TO PROTECTED ATTRIBUTES?
Table I compares our proposed ULPL based methods against baselines.
Vanilla and supervised debiasing baselines: Compared with the Vanilla model, supervised de- biasing basclines (GD GD FairBatch, and Adv) substantially improve fairness with rela- tively little performance cost, resulting in larger AUC-PEC scores. Among the four supervised de-

---

Published as a conference paper at ICLR 2023
Dataset
Moji
Bios
TrustPilot
Adult
COMPAS
Attribute
R
G
G
A
G
R
G
R
F
0.636
0.837
0.915
0.963
0.971
0.960
0.951
0.751
0.894
0.672
Fyr
0.996
0.656
0.214
0.056
0.045
0.034
0.600
0.691
0.983
0.855
Table 2: Fairnsss (F) and Pearson's ry between proxy labels and signed gaps for Vanilla.
biasing methods, GDCLA outperforms other methods, which is consistent with previous work (Shen et al. 2022).
Similar to the baseline debiasing methods, debiasing w.r.t. proxy labels (ULPL+*) also improves fairness over Vanilla, and achieves results that are competitive with supervised debiasing methods.
Semi-supervised debiasing baselines: We examine the effectiveness of SemiAdv by removing 50% of the protected labels, i.e., the adversary is trained over a subset of training instances. Observe that SemiAdv achieves almost identical results Adv, consistent with Han al. (2021a). Al- though SemiAdv uses protected labels in training, it is substantially outperformed by the proposed unsupervised method ULPL+GDeIA.
Unsupervised debiasing baselines: ARL (Lahoti et al. 2020 is also unsupervised debias- ing method that trains an adversary to predict the weights of each instance such that the weighted empirical risk is maximized. The training objective of ARL does not match with the definition of group-wise fairness, and as such, it results in lower AUC-PFC scores than our proposed methods, that explicitly optimize for performance parity across protected groups.
In terms of excluding outliers, the adversary of ARL is intended to predict smooth weights in- stances from (ær, y), such that the trained model focuses more on worse- perse-eerfomingg instances and is discouraged from memorizing noisy outliers. However, our results show that our ULPL method is more robust and effective in implementing these notions.
Different ULPL methods: Among the proxy label-based methods, ULPL+GD consistently outperforms other methods. ULPL+GDoLA calculates the loss differences separately before aggre- gation, which eliminates the influence of group size in debiasing and treats each group and class equally in optimization, which better aligned with the evaluation metric.
Adv is a popular method for achieving representational fairness, and differs from ULPL+GD and ULPL+GDGLB in that it directly optimizes for empirical fairness. Removing protected informa- tion from hidden representations requires accurate perceptions of the global geometry of particular protected groups. However, proxy labels are based on local loss differences within each class, mean- ing that the same proxy label in different classes may conflate different protected groups. As such, the combination of ULPL with Adv is less effective than the two other combinations
ANALYSIS
5.1 PROXY LABEL ASSIGNMENT
We first investigate if the ULPL labels are meaningful through the lens of Pearson's 's correlation (rx) between proxy labels and signed performance gaps. Given that is optimized En.g [TPR..g
TPR_1 = 0, an instance (z;,y;) should be assigned z = 0 if TPR,,. - TPR,, < 0, i.e., the unobserved group z under-performi mnder-performing in class y;, and z! = otherwise. We calculate ry between P(z' = 00y: z) and TPRy,, TPR,, and presents the results of Vanilla over each dataset in Table 2
can be seen that there exists strong correlation rz' for all datasets other than TrustPilot, indicat- ing that the unsupervised proxy label recovers demographic data and provides strong signal for bias mitigation. We also observe that better fairness results in smaller Tz. for example, for TrustPilot and Bios-E, which is not surprising as the gaps (TTPR.,g TPR_)) close to O.
See Appendix F.6 for further discussion.

---

Published as a conference paper at ICLR 2023
Dataset
Moji
Bios
TrustPilot
Adult
COMPAS
Attribute
G
E
G
A
C
G
R
G
R
(a) Leakage
83
79
84
40
52
26
80
25
83
43
UL ULPL+GDCLA CLA
13
(b)
ULPL+GDGLB
1
13
7
9
13
ULPL+Adv
1
9
13
3
5
Table 3: (a) Leakage (%) of protected attributes. (b) Best & assignments of each method.
5.2 EFFECTIVENESS OF THE NEIGHBOURHOOD SOAHHHBOOUOOOO-SMMOOINNNO
In smoothing ULPL labels, we hypothesise that an instance's neighbours are likely from the same protected group. Except for instance itself smoothing, the remaining nearest neighbours are essentially the results of a standard n-rnarest-neiihbouur (KNN) model. Therefore, we perform analysis based on standard KNN models and investigate if the remaining nearest neighbours are helpful for label smoothing, i.e., are they from the same protected group the target instance.
Protected information predictability: Proxy label smoothing is based on the hypothesis that there is a strong correlation between the hidden representations and the protected labels, even protected labels not observed during training. To test this hypothesis, protected label prediction based on Vanilla hidden representations within each target class, using leave-one-out cross-valiees vols-validation over each batch evaluate the predictability of protected attributes.
employ 1-NN for
Table Ba presents the results (macro Fl score) for each protected attribute, from which we can see a strong correlation between unobserved protected labels and hidden representations.
Furthermore, Appendix 2 we show that although debiasing methods successfully reduce per- formance disparities in downstream tasks, leakage of protected attributes in debiased hidden repre- sentations is still high, consistent with previous work HHan et al. 2021b).
Sensitivity to k-NN hyperparameters: Es-
timations of protected labels be affected by k-NN's hyperparameters, including: (1) p. the norm of distance; (2) whether nearest- neighbours must match the target class versus all classes; and (3) k, the number nearest neighbours. We explore the sensitivity to these hyperparameters for the Moji dataset in Fig- ure 3
class wise False
class Lise True
0.86
ficcore
0.84
0.82
15
10
15
k
k
First, we can see that k-NN highly robust to varying values of p. In terms of whether la- bel smoothing should be class-specific or in- specific, there is slight empirical advantage to performing it a class-specific manner.
Figure 3: Hyperparameter sensitivity analysis over the Moji dataset.
In terms of k, for Moji, higher values result in better estimations of protected labels, although there is a clear plateau. we explore this effect over the other datasets in terms of the value that achieves the highest AUC-PFC score, (k > 1) improving results for Moji, Adult, and COMPAS and the best results being achieved for values from 3 to 13, whereas for Bios and TrustPilot,
be seen in Table (3b] there is clear trend, with
neighbourhood smoothing (k: 1) performs best. Although the optimal value of & varies greatly across datasets and debiasing methods. it is possible perform ane-grained tuning to reduce computational cost. In Appendix C.4] we discuss situations where label smoothing succeeds or fails, and an effective tuning strategy for the value of k:
5.3 DEBIASING FOR INTERSECTIONAL GROUPS
8

---

Published as a conference paper at ICLR 2023
Dataset
Moji
Adult
COMPAS
Attribute
R
G
R
GxR
G
R
GxR
Vanilla
0.173
0.087
0.076
0.073
0.109
0.097
0.094
GDCLA
0.253
0.091
0.081
0.079
0.121
0.094
0.085
GDGLB
0.233
0.091
0.081
0.077
0.121
0.098
0.091
Adv
0.254
0.091
0.077
0.074
0.117
0.094
0.945
ARL
0.197
0.092
0.081
0.078
0.120
0.092
0.092
ULPL+GDCLA
0.214
0.090
0.082
0.080
0.123
0.101
0.095
ULPL+GDGLR
0.192
0.090
0.081
0.078
0.124
0.100
0.097
ULPL+Adv
0,186
0.091
0.083
0.080
0.122
0.099
0.092
Table 5: AUC-PFC based on demographic parity faimess.
We also investigate the robustness of binary proxy labels to non-binary intersectional groups (i.e. the cross product of values across different protected attributes).
Dataset
Bios Gx E
TrustPilot GxAxC
Adult GxR
COMPAS
Attribute
GxR
Vanilla
0.455
0.115
0.072
0.098
GDCLA
0.487
0.126
0.072
0.091
GDGLB
0.453
0.122
0.075
0.100
Table 4 presents debiasing results for intersectional groups over those datasets that labelled with more than one protected attribute. Com- pared to the single protected tribute results, the AUC-PFC scores of Vanilla are consistently smaller, indicating greater bias across inter- sectional groups, consistent with the findings Subramanian et al. (2021) For ULPL models, on the other hand, the results are competitive with supervised debiasing methods, consistent with the single attribute setting.
Adv
0.456
0.121
0.072
0.098
ARL
0.426
0.120
0.075
0.097
ULPL+GDOLA
0.470
0.105
0.076
0.098
ULPL+GDGLB
0.459
0,101
0.079
0.101
ULPL+Adv
0.456
0.122
0.071
0.099
Table 4: AUC-PFC w.r.t. intersectional groups.
5.4 OTHER FAIRNESS METRICS: DEMOGRAPHIC PARITY
Finally, we investigate the robustness of ULPL to other notions of fairness. For illustration pur- poses, we focus demographic parity faimess (DP) (Blodgett et al. 2016). which requires model predictions to be independent protected attributes. Again, we aggregate accuracy and DP fairness trade-offs AUC-PFC scores. Since DP is sensitive to class imbalance and there is standard way of generalizing DP to multi-class classification tasks, we only conduct experiments over binary classification tasks, namely Moji, Adult, and COMPAS.
Table 5 shows the results w.r.t. demographic parity fairness. The overall trend is similar to our original results for equal opportunity faimess, indicating that ULPL is robust to different fairness metrics when combined with a range of debasing methods.
CONCLUSION
Much of previous work in the faimess literature has the critical limitation that assumes access to training instances labelled with protected attributes. To remove this restriction, we present novel way deriving proxy labels, enabling the adaptation of existing methods to unsupervised bias mitigation. We conducted experiments over five widely-used NLP and ML benchmark datasets, and showed that, when combined with different debiasing strategies, our proposed method consistently outperforms naively- naively-trained models and unsupervised debiasing baselines, achieving results which are competitive with supervised debiasing methods. Furthermore, we showed our proposed method to be generalizable to multi-class intersectional groups and different notions of fairness.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENTS
We thank the anonymous reviewers for their helpful feedback and suggestions. This work was funded by the Australian Research Council, Discovery grant DP200102519. This research was un- dertaken using the LIEF HPC-GPGPPU Facility hosted at the University of Melboume. This Facility was established with the assistance of LIEF Grant LE1 L0170100000.
ETHICS STATEMENT
This work focuses on learning fair models without observation of protected labels at training time. Demographics are assumed be available only for evaluation purposes, and used for model training or inference. We only use attributes that the user has self-identified in our experiments. All data and models this study are publicly available and used under strict ethical guidelines.
REPRODUCIBILITY STATEMENT
All baseline experiments are conducted with the FairLib library (Han et al. 2022b) Source code available |attps:/ Locality-based hatt-baaaa_MMehhhhc for Bias Mitigation Appendix [ reports relevant statis- ties, details of train/test/dev splits, etc. for the five benchmark datasets that are used in this paper. Appendix B reports implementation details of evaluation metrics and corresponding aggregation approaches. Appendix([presents details of computing infrastructure used our experiments, com- putational budget, hyperparameter search, etc. Appendix P reports PFC and full dis-aggregated results, i.., mean + std over 5 random runs with different random seeds.
wwm/HanXudongg//aaaaaaaaaa Unsupervil sed
REFERENCES
Devansh Arpit, Stanislaw Jastrzundefinedbsii, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Lacoste-Julien. A closer look at memorization in deep networks. In Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, pp. 233-242. JMLR.org, 2017.
Pranjal Awasthi, Matthäus Kleindessner, and Jamie H. Morgenstern. Equalized odds postprocessing under imperfect group information. In AISTATS, 2020.
Pranjal Awasthi, Alex Beutel, Matthaeus Kleindessner, Jamie H. Morgenstern, and Xuezhi Wang- Evaluating fairness of machine learning models under uncertain and incomplete information. Pro- ceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 2021.
Su Lin Blodgett, Lisa Green, and Brendan O'Connor. Demographic dialectal variation in social media: A case study of African-American English. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 1119 1130, November 2016. doi: 10. 18653/v1/D16-1 1/20.
Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. Semanties derived automatically from language corpora contain human-like biases. Science, 356: 186. 2017.
Jiahao Chen, Nathan Kallus, Xiaojie Mao, Geoffry Svacha, and Madeleine Udell. Fairness under unawareness: Assessing disparity when protected class is unobserved. Proceedings of the Con- ference on Fairness, Accountability, and Transparency, 2019.
Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism
prediction instruments. Big data, 5(2):153-163, 2017.
Amanda Coston, Karthikeyan Natesan Ramamurthy, Dennis Wei. Kush R. Varshney, Skyler Speak- man, Zairah Mustahsan, and Supriyo Chakraborty. Fair transfer learning with missing protected attributes. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, 2019.
10

---

Published as a conference paper at ICLR 2023
Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in bios: A case study of semantic representation bias in high-stakes setting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 120-128, 2019.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chupter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Pupers), pp. 4171 4186, 2019.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Con- ference, pp. 214-226, 2012.
Yanai Elazar and Yoav Goldberg, Adversarial removal of demographic attributes from text data. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp- 11 11-21, 2018.
Bjarke Felbo, Alan Mislove, Anders Sógaard, Iyad Rahwan, and Sune Lehmann. Using millions of emoji occurrences to learn any any-domain representations for detecting sentiment, emotion and sarcasm. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2017.
Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubra- manian. Certifying and removing disparate impact. In proceedings of the 21th ACM SICKDD international conference on knowledge discovery and data mining, pp. 259-268, 2015.
Anthony W. Flores, Kristin A. Bechtel, and Christopher T. LLownkkmp. False positives, false nega- tives, and false analyses: A rejoinder to "machine bias: There's software used across the country to predict future criminals. and it's biased against blacks". Federal Probation, 80:38, 2016.
Maya R. Gupta, Andrew Cotter, Mahdi Milani Fard, and Serena Wang. Proxy faimesss. ArXiv, abs/1806.1 1212, 2018
Xudong Han, Timothy Baldwin, and Trevor Cohn. Decoupling adversarial training for fair NLP. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 471-477,
August aa. doi: 10.18653/v1/200 1tt.///...niiiaaaa.....o findings-acl.41.
Xudong Han, Timothy Baldwin, and Trevor Cohn. Diverse adversaries for mitigating bias in train- ing. In Proceedings of the 16th Conference of the European Chapter of the Association for Com- putational Linguistics: Main Volume, pp. 2760-207 2765,2775,40 April 2021b.
Xudong Han, Timothy Baldwin, and Trevor Cohn. Balancing out bias: Achieving fairness through balanced training. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 11335-11350, Abu Dhabi, United Arab Emirates, December 2022a. Association for Computational Linguistics.
Xudong Han, Aili Shen, Yitong Li, Lea Frermann, Timothy Baldwin, and Trevor Cohn. FairLib: A unified framework for assessing and improving fairness. Proceedings of the The 2022 Confer- ence on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 60-71, Abu Dhabi, UAE, December 2022b. Association for Computational Linguistics.
Xudong Han, Timothy Baldwin, and Trevor Cohn. Fair enough: Standardizing evaluation and model selection for fairness research in nlp. arXiv preprint 21X1::2302.0571,, 2023.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning, Advances in Neural Information Processing Systems, 29:3315-3323, 2016.
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning, pp. 1938. PMLR, 2018
Úrsula
Michael P. Kim, Omer Reingold, and Guy N. Rothblum. Calibration for the (comptaatonally-ieetiiabbll masses, ArXiv, abs/1711 1111111.055 1100513,,, 2017.
11

---

Published as a conference paper at ICLR 2023
Dirk Hovy and Anders Søgaard. Tagging performance correlates with author age. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Interna- tional Joint Conference on Natural Language Processing ( (Volume 2: Short Papers), pp. 483-488, July 2015. doi: 10.31 1/.111/////15-2007.
Dirk Hovy, Anders Johannsen, and Anders Søgaard. User review sites resource for large-scale sociolinguistic studies. Proceedings of the 24th International Conference on World Wide Web, 2015.
Nathan Kallus, Xiaojie Mao, and Angela Zhou, Assessing algorithmic fairness with unobserved protected class using data combination. In FAT*s 2020.
Diederick P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
Ron Kohavi. Scaling up the accuracy of naive-bayes classifiers: decision-tree hybrid. KDD, 1996.
Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographics through adversarially reweighted learning. In Advances in Neural Information Processing Systems, volume 33, pp. 728-740, 2020.
Ronan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew Peters, Ashish Sabharwal, and Yejin Choi. Adversarial filters of dataset biases. International Confer- ence on Machine Learning, pp. 1078-1088 PMLR, 2020.
Yitong Li, Timothy Baldwin, and Trevor Cohn. Towards robust and privacy-preserving text rep- resentations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 25-30, July 2018. doi: 10.18653/v1/P18-2200
Marco Lui and Timothy Baldwin. langid. py: An off- shellf language identification tool. In Proceedings of the ACL 2012 system demonstrations, pp. 25-30, 2012.
Kristian Lum, Yunfeng Zhang, and Amanda Bower. De-biasing "bias" measurement. In 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22), Scoul, Republic of Korea, June 2022. ACM. doi: 10. 0..144/3331146635333
Timothy Marler and Jasbir S Arora. Survey of multi-ohjective optimization methods for engineer- ing. Structural and Multidisciplinary Optimization, 26(6):369-399 2004.
Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. On mea- suring social biases in sentence encoders. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long and Short Papers), pp. 622-628, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1/N19-11
Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. Null it out: Guarding protected attributes by iterative nullspace projection. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pp. 7237-7256. July 2020. doi: 10.18653/v1/20221 11
John Rawls. Justice Fairness: Restatement. Harvard University Press, 2001.
Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh. Fairbatch: Batch selection for model fairesss. In Proceedings of the 9th International Conference on Learning Representations, 2021.
Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, and Lea Frermann. Optimising equal opportunity fairness in model training. In Proceedings of the 2022 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Seattle, United States, July 2022, Association for Computational Linguistics. doi: N0.18853/v1/2000
pp. naall-maanc 2999
12

---

Published as a conference paper at ICLR 2023
Shivashankar Subramanian, Xudong Han, Timothy Baldwin, Trevor Cohn, and Lea Frermann. Eval- uating debiasing techniques for intersectional biases. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 2492-2498 Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
Swabha Swayamdipta, Roy Schwartz. Nicholas Lourie, Yizhong Wang, Hannanch Hajishirzi, Noah A. Smith, and Yejin Choi. Dataset cartography: Mapping and diagnosing datasets with training dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLPP), pp. 9275-9295 Online, November 2020. Association for Com- putational Linguistics. doi: 122665//1122220..mmmmmmmmiii
Jialu Wang, Yang Liu, and Caleb C. Levy. Fair classification with group-dependent label noise. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021.
Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. Balanced datasets not enough: Estimating and mitigating gender bias in deep image representations. In Pro- ceedings of the IEEE International Conference on Computer Vision, pp. 5310-5319, 2019.
Forest Yang, Mouhamadou Cisse, and Sanmi Koyejo. Fairness with overlapping groups; proba- bilistic perspective. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 4067-4078. Curran Asso- ciates, Inc., 2020.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fair- ness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics, pp- 962-970. PMLR, 2017.
Han Zhao, Amanda Coston, Tameem Adel, and Geoffrey J Gordon. Conditional learning of fair representations. International Conference Learning Representations, 2019.
13

---

Published as a conference paper at ICLR 2023
A DATASETS AND PRE-PROCESSING
A.1
MOJI
Following previous studies (Ravfogel al. 2020 Han al. 2021b), the original training dataset
is balanced with respect to both sentiment and ethnicity but skewed in terms of sentiment-etnniiity combinations (40% happy-AAE, 10% happy-SAt SAp. 10% sad-AAE. and 40% sad-SAE, respec- tively). The dev and test sets are balanced in terms of sentiment-ethnicity combinations. The dataset contains 100K/8K/8K train/dev/test instances.
When varying training set distributions, we keep the 8k test instances unchanged. We use DeepMoji Caliskan et al. 2017) to obtain twitter representations, where DeepMoji is model pretrained over 1.2 billion English tweets and DeepMoji is fixed during model training.
A.2 BIOs
Bios experiments are based on biog- raphy classification dataset (De-Arteaga) et al. 2019, |Ravfogel et al.] 22000). where biographies were scraped from the web, and annotated for the protected attribute binary gender and target la- bel of 28 profession classes.
Male
Female
Profession
Total
$
professor
21715
46
9
37
physician
7581
42
41
attorney
6011
51
10
33
photographer
4398
Besides the binary gender attribute, we additionally consider economic status a second protected attribute. (Subrama- nian et al. (222)) semi-automatically la- belled economic status based on the in- dividual's home country (wealthy vs. rest of world), as geotagged from the first sentence of the biography. For bias evaluation and mitigation, we con- sider the intersectional groups, i.e., the Cartesian product of the two protected attributes, leading to 4 intersectional classes: female-wealthy, femalc-rest, weal-wealthy, and male-rest.
journalist
3676
41
41
nurse
3510
76
15
psychologist
3280
31
52
11
teacher
2946
10
dentist
52
30
surgeon
2465
73
12
13
architect
1891
64
12
21
painter
1408
47
36
model
1362
13
poet
1295
46
39
software engineer
1289
70
14
14
filmmaker
1225
56
10
29
composer accountant
1045
70
1012
dietitian
730
82
12
comedian
499
69
19
Since the data is not directly available, in order to construct the dataset, we use the scraping scripts of [Ravfogel et al. (202)), leading to a dataset with 396k biographies Following Ravfogel et al. (2020). we randomly split the dataset into train (65%), dev (10%), and test (25%).
chiropractor
474
62
14
21
pastor
453
paralegal
330
12
70
15
yoga teacher
305
13
3
71
12
interior designer
267
16
4
67
12
personal trainer
264
42
DJ
rapper
221
75
15
9
Total
72578
7
39
The augmentation for economic at- tributes follows previous work (Subra- manian al. 202T), which results in approximate 30% instances that are la- belled with both protected attributes.
Table 6: For each profession in Bios, the table shows the number of individuals and the breakdown across demo- graphics percentage. $ and $ denote the economic status (high vs. low, respectively).
A.3 TRUSTPILOT
We fellow previous work Li et al.
2018 in using the TrustPilot dataset derived from Hovy et al. (2015). where each review is an-
There are slight discrepancies in the dataset composition due to data attrition: the original dataset (De- Arteaga 2019] had 399k instances, while 393k were collected by Ravfogel 2020
14

---

Published as a conference paper at ICLR 2023
Female
Male
Score
Total
UK
US
UK
US
Over45
Under35
Over45
Under35
Over45
Under35
Over45
Under35
5051
13
17
3
3
23
31
2
1783
15
15
2
5
24
29
4
3 2001
14
15
3
4
25
26
4877
13
23
25
5
29489
17
11
5
4
26
22
9
Table 7: For each score in TrustPilot, the table shows the number of instances and the breakdown across demographics as a percentage.
notated with the target rating variable and associated with three protected labels gender (male vs. female), age (under-35-year-ol) vs. over-45-year-old), and, location (UK vs. the US). The original dataset contains different countries (US, UK, Germany, Denmark, and France), and Li et al. (2018) discard non-English reviews after automatic language classification (Lui & Baldwin 2012). Despite this, there are some non-English reviews in the filtered dataset, and there, we further drop instances from Germany, Denmark and France, resulting in dataset with 54k instances in total.
A.4 ADULT AND COMPAS
Except for race features, we use the same pre- pre-processing as in Lahoti et al. (2020) for COM- PAS (Flores et al. 2016) and Adult (Kohavi] 1996) datasets with 5,278 and 43,131 examples, respectively. Lahoti et al. (2020) considers binary race groups (white vs. black). However, there are more than two protected groups in the original dataset. Specifically, there are 3 race groups in COMPAS: Amerrcan, Caucasian, and Other; and 5 race groups in Adult: White, Asian- Pac- Palandlan,er, Amer-Indian Indiin-Eskioooo, Other, Black.
B EVALUATION METRICS
Besides the absolute gap metric (/Uc.e - Uo| 0), a broad range of formats of metrics have been introduced in previous studies capture different assumptions about the nature of fairness. For example, Lum et al. (2022) propose to measure the variability of performance across demographic
groups
Yang et al. (2020) only focus on the largest gap LmaxylUe.g-Ue (.)) and
Feldman et 2015 measure performance ratio rather than gap in measuring fairness max,U/sa) min, Although, different aggregation methods have been applied to measure group-wise fairness, the optimization of any one of them sufficient condition for the optimization of other methods, as the optimization conditions of these metrics are identical, Ue.8 U.Ve,
For fair comparison consistently. Previous work has used different criteria for model selection. including: (1) minimum
different debiasing approaches, should select the hyperparameters
loss (Hashimoto et al. 2018 Li 2018); (2) maximum utility (Lahoti al. 2020) e.g., based on
accuracy or --easure: (3) manual selection based on visual inspection of the trade-off curve (Elazan
& Goldberg 2018
Ravfogel et al. 2020); and (4) constrained selection (Han et al.) 2021b Subra-
manian et al. 2021 by selecting the best faimeess constrained to a particular level of performance,
and vice versa. Each selection criterion reflects the performance at particular situation, making very hard to rigorously compare methods.
Instead, the AUC-PFC score the integral of performance-fairee mance interval [0, 1]. For a particular dataset, by the definition of fairness metrics, random classifier achieves the best fairness. Therefore, the integration from 0 the random prediction accu- racy is dataset-specific and is identical to different methods. In this paper, normalize AUC-PFC scores for each dataset by ignoring the performance worse than random guess. Table summaries the lowest accuracy scores w.r.t. each dataset.
curves with respect to perfor-
When calculating AUC-PFC scores, for these methods that are not flexible to achieve best fairness, manually add the random model the calculation. Taking the Vanilla model on Moji as an
15

---

Published as a conference paper at ICLR 2023
Moji 50%
Bios 30%
TrustPilot
Adult 76%
COMPAS
68%
56%
Table 8: Majority label proportion, i.e., lowest accuracy of each dataset.
example, the performance and faimesss 0.7109 + 0.0110 and 0.6358 + 0.1331, respectively. The random model corresponds to 0.5 accuracy and 1.0 fairness. Given these two points, the PFC the line form (0. (0.7109, 0.6358) to (0.5, 1), and the AUC-PFC score
(0.7109 - 0.5) x 0.6358 + 0.5 x (0.7109 0.5) x (1 0.6358) 0.172.
which is consistent with T
However, we still need to select model for early stopping before model selection. Instead of considering performance and fairness metries separately, we use the distance to the optimal point ("DTO"), which quantifies the accuracy-faimess tradeoff (Marler & Arora 2004: Han et al. 2022a). DTO measures the normalized Euclidean distance for a given combination accuracy and fairness to the optimal point which denotes the ideal result, e.g., accuracy and fairness of 1.0. It is typically unachievable in practice.
B. INTERPRETATION OF AUC-PFC RESULTS
The main motivation for using AUC-PFC CFC is for ease of comparison between approaches, as ag- gregates the performance fairness trade-off oof curve (PFC) of each model to single number, enabling systematic comparison across different tasks. The two common questions related to AUC-PFC are:
The magnitude of AUC-PFC differs from single metric, and a 0.0001 improvement in the AUC-PFC score is equivalent to a percentage point (pp) improvement in both perfor- mance and fairness (0.01 0.01). In the paper, numbers rounded to decimals, and minimum difference in AUC-PFC (0.001) is roughly equivalent to a 3 improvement in both performance and fairness in PFC plot.
The calculation AUC-PFC scores is normalized by the worst performance, which is the majority label proportion when using the accuracy metric. Therefore, AUC-PFC scores represent to what extent model improves the performance faimesss the random model.
There is no doubt that using AUC-PFC comes with certain limitations. To address the major con- cerns related to AUC-PFC scores, we present additional results in Appendix B including disaggre- gated results for each dataset.
In particular, we provide the PFC of each method (e.g., Figure 6 in Appendix F. representing the best fairness that can be achieved at different performance levels, and vice versa.
One limitation of a PFC plot is that it is hard to make quantitative conclusions based on the plot itself, and we cannot conclude that one method is better than another if any intersection exists between their PFCs.
To address this problem, we additionally conduct quantitative comparisons across different debias- ing methods by model selection w.r.t. two different criteria. and then compare both the performance and fairness of the selected models (e.g., Table 14 in Appendix E). For each method, we report the evaluation results averaged over 5 random with standard deviation for both the development set and test set.
As stated in Appendix F present disaggregated results (including PFC plot and table) for all 15 settings on GitHub
C EXPERIMENTAL DETAILS
We conduct our experiments on an HPC cluster instance with CPU cores, 32GB RAM, and one NVIDIA V100 GPU.
16

---

Published as a conference paper at ICLR 2023
C.I ASSIGNING AND SMOOTHING PROXY LABELS
Assigning proxy labels In the current experiments, proxy labels are assigned based on the losses of each minibatch, i... the loss per instance is taken from particular training iteration. We ac- knowledge that there are other ways of extracting training losses, c.g. taking losses from the final model or averaged over multiple iterations as the reviewer suggested, and we leave as a future work.
Smoothing proxy labels The proxy label assignment and smoothing happen simultaneously at each iteration. By doing so, our method can be incorporated into existing systems with only a few lines of changes to replace the actual protected labels with our proxy labels.
At each minibatch, the actual protected labels are replaced with smoothed proxy labels. All debias- ing methods will be on the proxy labels in the later process.
During label smoothing, unsmoothed labels are used for voting avoid inconsistency in smoothing decisions for other examples. We first collect the nearest neighbours of each instance and then do the voting for all of them.
C.2 MODELS AND PARAMETER TUNING
approaches presented in this paper share the same dataset-specific hyperparameters as the stan- dard model. Hyperparameters are tuned using grid-search, in order to minimize distance to the optimal.
Best assipament
Hyperparameter
Search space
Moji
Bios
TrustPilot
Adult
COMPAS
number of epochs
100
patience
encoder
DeepMoji (Felbo al
2017
BERT
Devlin et al. 2019
embedding size
2304
101
hidden size
300
number hidden layers
choice-intteerr1...
batch size
loguniform--ng integerritt..... 2018]
1024
1024
1024
512
1024
optimioer
Adam (Kingma
B012015
leaming rate
logumiformm-finnt fioat] 10
10
x 10-3
10
3 x 10-3
x 10
10
leurning
on plateon
LRS patience
2 epochs
LRS reduction
Trainable Parameter
782k
329k
323k
122k
225k
Table 9: Search space of dataset-specific hyperparameters.
All debiasing methods this paper does not introduce extra parameter to the main task model, and will not need to considered the inference time. As such, provide method-specific hyper- parameters separately, and the search space for method-specific hyperparameters are shared across difference datasets.
GDCLA tunes the strength of the additional loss for minimizing absolute loss difference within each class. (gguiiform-floar1100- 10 11 40 times.
GDGLB tunes the strength of the additional loss for minimizing absolute loss difference. loguniformm-lout(10- 10-01, times.
FairBatch tunes the adjustment rate for resampling probabilities. loguniform,flouu(100 10-9] 40 times.
Adv tunes the weights of unlearning adversaries in training. loguniform-ffaa1100 10 40 times.
SemiAdv tunes the weights of unlearning adversaries in training. loguniform,ffoa(110 10+2], 40 times.
17

---

Published as a conference paper at ICLR 2023
Dataset
Moji
Bios
TrustPilot
Adult
COMPAS
Attribute
E
GxE
GxAXC
G
GxR
GxR
Vanilla
:ST
GDCLA
55
131
131
149
43
36
36
42
26
26
27
10
11
12
GDar
33
FairBatch
54
46
48
47
26
26
27
24
24
21
21
11
10
11
Adv
48
29
SemiAdv
51
57
G3
42
30
29
30
25
24
20
29
11
ARL
71
73
40
25
27
22
21
23
11
12
ULPL
148
355
3T
40
34
51
55
54
18
ULPL+GDoLe
148
35
S7
38
34
19
54
54
16
17
ULPL
179
50
41
46
62
61
17
Table 10: Average computational budget, measured in seconds.
ARL tunes the learning rate of learning adversaries in training. loganiformm-flot(10 (10 10+ 40 times.
ULPL methods tunes the k from to 15, and p-norm from 2 6.
Notice that, this paper report the AUC-PFC, which eliminate the requirement for model selection, i.., there is no best-found trade-off hyperparameters w.r.t. bias mitigation.
C.3 COMPUTATIONAL BUDGET
Table 10 shows average GPU time of model training. Noticing that debiasing components will not be used for inference, i.e., different methods have identical inference cost.
C.4 PARAMETER TUNING FOR LABEL SMOOTHING
For Bios, class-specifie neighbourhood smoothing degrades to naive proxy labels when there is only small number of instances in particular class. For example, there are 28 distinct target classes in the Bios dataset, with highly skewed distribution. As such, there can be only one instance target class in minibatch, and the neighbourhood smoothing does work in this case.
For TrustPilot, we hypothesise that it is due to the leakage of protected information being very low, and accordingly the neighbourhoods of instances being noisy.
The selection of for label smoothing. As observed in Section 5.2, the optimal value of & varies greatly across datasets and debiasing methods, and in the context of this paper, we deal with this through simple grid search over different values of k, which is computationally expensive.
Although we do not currently have an algorithm for efficiently optimizing k at this time, we have observed that the value of k positively correlated with model leakage and unfairness. Therefore, we could start tuning the value of & from large value if the model is significantly biased, as the instances from the same protected group are likely be close each other. Otherwise, we can use the proxy labels without smoothing if the results are reasonably fair.
D THEORETICAL JUSTIFICATION
D.1 FROM EMPIRICAL LOSSES TO UTILITY METRICS
For illustration purposes, we assume binary settings for both target class and protected attribute labels. Section 3.3 have shown that the proposed method be used improve the equal opportunity fairness.
Demographic parity (DP) For DP fairness, the predictions are expected to be independent from protected attributes (ý 1. 2), and the faimess satisfied if the differences in positive prediction rate
18

---

Published as a conference paper at ICLR 2023
between demographic groups are zero: p(ý = 1/2 0) ply = 1/2 1). Thus,
p( = 1, - 0/ 0) + p 1,y - 1|z =0) 0)
=p(ÿ 1,y = 0 1)+ p( = 1, 1|z = =).
Since p(} 0,y/z) + p(y = 1,yl2) = 1, vy, 7 by replacing p(ý = 1, = 0/z) with 1 - -p(s- 0,y = 0/2),
(1 - pl- = y,y = 0/z = + 1, 1 = 0)
=(1 ply o, y 0/z = 1 = 1,y 1 = 1).
An equivalent condition the DP fairness is that
r = 1, = 1/z 0) - p(y = 0,y 0/z 0)
=plý = 1, = 1|z = 1) - p(ÿ = 0,y = 0/z = 1).
A sufficient condition for DP is, both pf} = 1,y = 1/. = 0) = plý = 1,y = 1/2 1) and
p(ý o,y 0/z 0) p(}- 0,y 0/z 1) are satisfied. Next, we show how map the
conditional joint probability to training losses. As for the = 1, recall that, L is an unbiased
estimator of log(p(} 1 y, 1) (Equation (I )
L log(p(} 1ly, = 1)
L = log(p() 1ly, = 1) - log(p(yy = 1)) + log(ply, = 1))
L = log(p() = 1ly, - 1)p(y, 1)) + log(p(y, 1))
C = log(p(ý = 1, -1) + +og((pyy; = 1))
By substituting the joint probability with losses,
p(y = 1,y = 1/2 = 0) = p(ý = 1,y 1,y 1/z = 1)
- log(ply = 1,y 1|z = 0)) log(p() 1.y 1/z 1))
_ log(p(y, L|z = -0)) = 1..1-lo((pd log(p(y; -1/z, 1))
L - LLd =log(P/) ply; = 1z, = 0),
ply, 1 =
Similarly, _ is an unbiased estimator of log(p() 0,y, 0) + log(p(y; 0)). and the DP condition for 0, p(ý = L,y 1/z = 0) p(ý = 1,y = 1 = 1). is equivalent to -0.0-_0.11 log( ply,=0
p(y,=U==0)
Notice that p(y|z), Vy, are dataset-specific constant numbers, and if p(ylz = 0) p(ylz= Vy. the DP conditions identical Equalized Odds fairness (Hardt al. 2016). and be approx- imated by _ = LL. and -00 = _ Last but not least, recall that the nature of DP assumes and z are independent, therefore, p(ylz 0) p(ylz 1) 0, Vy generally holds when the DP fairness is desired.
Confusion-matrii based metries So far, we have shown that minimizing loss differences can approximate the optimization of the two most wildly used notions of faimess: EO and DP faimesss Since model predictions and target labels are observed during training, such approximation can also be applied to other confusion- -ww--m-aaaiiateaeee based metries. For example, the cross-entropy loss of instances w.r.t. predictions as and 1 are approximations of the positive predictive value (p(y = 1ly = 1)) and negative predictive value (p(y 0ly 0)), respectively.
19

---

Published as a conference paper at ICLR 2023
D.2 OPTIMIZING FAIRNESS WITH PROXY LABELS
Bias mitigation for different fairness criteria Overall, ULPL only assigns proxy labels to train- ing instances, and the optimization for fairesss achieved using existing supervised debiasing meth- ods, by learning uniform representations across proxy groups (such as Adv) or minimizing loss dis- parities in training (such as GDCLA and GDGLB). Based on ULPL, different fairness criteria can be optimized by employing different variants of a particular debiasing method. Taking Adv as an example, discriminator is trained recover the protected information from hidden representations, and the main task model is optimized to remove protected information from hidden representations through unlearning the discriminator. By doing so, the hidden representations and corresponding predictions expected be independent of the protected attribute, ensuring DP fairess. To adopt Adv for EO fairmess, the discriminator takes target labels into consideration, e.g. training a specific discriminator for instances with positive target class only, and the removal of protected information is then class-dependent, aligning with the definition of EO fairness.
E
ADDITIONAL RESULTS
E.1
PROXY LABEL ASSIGNMENT MOJI
Sentiment
Race
P(z' =
PPRt
TPR
FPR
PPV1
NPV
SAE AAE
1.2
79.9
91.2
35.0
91.3
64.8
sad
1.75
1..3
45.7=2.6
6.4
64.2#3.2
0.6
SAE
3.0
2.8
6.10
6.. 0.7
91.3 0.7 3.2
happy
AAE
1..8
8.7
93.6
2.6
87.3
Table 11: Proxy label assignment without smoothing and evaluations for the Vanilla model over Moji. Evaluation results + standard deviation (%) are averaged over 5 runs with different random seeds. + P(z' 1) refers the proportion of instances being assigned with indicating worse- performed groups. Evaluating metrics include: (1) positive predictive rate (PPR), corresponding the demographic parity fairness (Blodgett al. 2016). (2) true positive rate (TPR) and false positive rate (FPR), corresponding to equalized odds and equal opportunity faimees (Hardt et al.) 2016), and (3) positive predictive value (PPV) and negative predictive value (NPV), corresponding to test fairness (Chouldechoval 2017).
We first investigate if the ULPL labels are meaningful through the lens of training examples in the Moji dataset. Table II presents the results of the Vanilla model.
can be seen that, AAE tweets are more likely to be classified as happy, while SAE tweets are more likely to be classified sad, resulting in consistent trend in gaps with respect PPR, TPR, FPR, PPV, and NPV. Based on loss-disparities, AAE instances with sad target labels are more possible to be assigned with z' (68.5% vs. 17% for SAE and AAE. respectively), encouraging debiasing methods to focus more on sad-AAE instances in training. Similarly, happy-SAE instances are more likely to be assigned with z' 1, indicating that happy-SAE are upweighted in training.
For the dataset distribution perspective of view, as introduced in Appendix A. Moji is balanced with respect to both sentiment and ethnicity but skewed in terms of sentiment- etthicict combi- nations (40% happy-AAE, 10% happy- 10% sad-AAE, and 40% sad- RAd- respectively), which closely related to the DL assignments that minority groups within each target class are assigned with z' 1. Le., our proposed proxy label differentiates minority groups with majority groups within each target class without observing demographic labels.
E.2 PROTECTED LABEL PREDICTABILITY AFTER DEBASING
Neighbour smoothing requires protected information in hidden representations during the whole training. requiring encoded protected information in hidden representations, Han et al. (202 show that although supervised debiasing methods have shown success in reducing performance disparities in downstream tasks, the predictability of protected attributes in debiased hidden representations is
20

---

Published as a conference paper at ICLR 2023
0.85
0.9
Predictabiltty of
0.84
Fairness
0.8
0.83
0.82
9
0.7
0.81
13
0.66
0.68
0.70
0.72
Performance
Figure 4: Predictability after debiasing.
still well above the ideal value. We take ULPL+GDOLA as an example and explore the protected label predictability across different debiased models (i.e., different trade-offs).
As seen from Figure faimesss scores (the blue line) improve at the cost of performance. However, the predictability of protected labels is quite stable at a high level, indicating that protected infor- mation still encoded in debiased models, and our proposed neighhourhood smoothing method robust to bias mitigation.
E.3 EFFECTIVENESS OF THE NEIGHBOUR SMOOTHING
In smoothing z' labels, we hypothesis that the nearest neighbours of an instance are likely to from the same protected group. Except the instance itself, the remaining nearest neighbours essentially the results of a standard d-nearest-neiihhourr (k-NN) model. Therefore, we perform analysis based standard N--N models, and investigate if the remaining nearest neighbours helpful for label smoothing, i.c., from the same protected group as the target instance.
E.4 OTHER FAIRNESS METRICS, DP
We investigate the robustness ULPL methods to other notions of fairness. For illustration pur- poses, we focus demographic parity faireess (DP) (Blodgett et al. (016) in this experiment, which requires model predictions to independent with protected attributes. Again, we aggregate accuracy and DP fairness trade-off's the AUC-PFC scores. Since DP is sensitive to class imbalance and there is no standard way generalizing DP multi-class classification tasks, we only conduct experiments over binary classification tasks, including Moji, Adult, and COMPAS. T2 results w.r.t. demographic parity fairness.
Trends for different methods similar to the results of equal opportunity fairness, indicating that debasing methods are robust to different fairness metrics.
Dataset
Moji
Adult
COMPAS
Attribute
G
R
GxR
G
R
GxR
Vanilla
0.173
0.087
0.076
0.073
0.109
0.097
0.094
GDCLA
0.253
0.091
0.081
0.079
0.121
0.094
0.085
GD
0.233
0.091
0.081
0.077
0.121
0.098
0.094
FairBatch
0,254
0.091
0.079
0.073
0.117
0.103
0.094
Adv
0.254
0.091
0.077
0.074
0.117
0.094
0.945
SemiAdv
0.255
0.090
0.077
0.074
0.119
0.093
0.945
ARL
0.197
0.092
0.081
0.078
0.120
0.092
0.092
ULPL+GD
0.214
0.090
0.082
0.080
0.123
0.101
0.095
ULPL+GDGLB
0.192
0.090
0.081
0.078
0.124
0.100
0.097
ULPL+Adv
0.186
0.091
0.083
0.080
0.122
0.099
0.092
Table 12: AUC-PFC based on demographic parity fairness.
21

---

Published as a conference paper at ICLR 2023
Dataset
T
Vanilla
GDcLA
FairBatch
SemiAdv
ARL
ULPL+GDOLA
G
E
-0.0011
0.0003
0.0054
0.0021
0.0018
0.0002
Bios
E
G
-0.0041
0.0131
0.0166
0.0188
0.0028
-0.0120
G
0.0025
0.0021
0.0007
0.0018
0.0018
0,0006
G
0.0047
0.0012
0.0010
0.0015
0.0004
0,0008
G
-0.0017
-0.0032
-0,0006
0.0001
0.0009
-0.0020
TrustPilot
A
C
0.0018
-0,0009
-0.0001
0.0000
0.0012
-0,0006
C
G
-0.0041
-.0023
-0,0001
0.0003
0.0007
-0.0023
C
-0.0027
0.0011
0.0011
0.0008
0.0023
-0,0010
G
R
0.0053
0.0098
0.0045
0.0037
0.0080
-0.0060
Adult
0..023
0.0010
0.0040
0.0033
00.012
-0.0020
G
R
0.0012
0.0036
0.0036
0.0002
0,0014
0.0011
COMPAS
0.0058
0.0026
0.0025
0.0064
-0.0083
-0.0035
Average
0.0002
0.0026
0.0027
0.0026
-0.0019
0.0022
Table 13: AUC-PFC score differences between debiasing w.r.t target protected attributes (T), and source protected attributes (S). Larger numbers indicate better generalizations to unobserved pro- tected groups.
1.000
0.975
0.950
Faimess
0.925
Models
0.900
GD
Z'+GDCLA
0.875
Attribute
0.850
0.825
0.3
0.4
0.5
0.6
0.7
0.8
Performance
Figure 5: PFC of in-domain and cross-domain debiasing over the Bios-gender dataset.
E.5
HOw DOES BIAS MITIGATION AFFECT FAIRNESS FOR UNOBSERVED GROUPS?
Proxy group labels (z') are dynamically adjusted each minibatch during training, which differs from fixed protected labels in supervised debiasing. As result, supervised debiasing methods based the observed protected attributes Z mitigate biases for particular protected groups.
While the proposed proxy label approaches focus on the group of instances that are underrepre- sented during training, which is expected to be more general than debiasing to a particular protected attribute. Figure [ demonstrates the difference in AUC-PFC scores between in-domain debiasing and cross-domain debiasing. For each debiasing method, we train the debiased model and conduct model selections based on the source protected attribute (Economy). The trained models are then evaluated w.r.t. the target unobserved protected attribute (Gender). The ability to generalize to un- observed protected attributes is measured the difference between in-domain and cross-domain AUC-PFC.
Table 13 summaries AUC-PFC differences across difference datasets w.r.t. subset of debiasing methods. Overall, Vanilla shows the best generalization to unobserved protected attributes, and unsupervised debiasing methods are better than supervised and semi-supervised debiasing methods.
Intuitively, the Vanilla model and unsupervised debiasing methods (ARL and ULPL+GDoLA) are agnostic cross-attribute generalization, and their difference should be close to 0 which is clearly not the case. The AUC-PFC decrease these methods caused by the model selection, i.c., though the training process is identical, different models may be selection based their pro-
22

---

Published as a conference paper at ICLR 2023
tected attributes. As shown in Figure 5 the selected model of ULPL+GDOLA based on economy labels at accuracy around 0.6 is not Pareto point for gender, confirming that decreases in AUC-PFC of Vanilla and unsupervised debiasing methods are caused by cross-domant model selection.
E.6 THE APPLICATION OF ULPL TO DIFFERENT DEBIASING APPROACHES
GDeLA are representatives of debiasing methods that directly optimize loss parity. In particular, the training objective of GD CLA is:
LGDELA = + AGDOLA 1E L%
where |,4:-- - is optimized achieve better fairness. Since - L., and L are average losses, their magnitude are irrelevant to subset sizes (Ne.g. Re. and n, respectively), which in turn applies the same strength of fairness regularization all subset of instances De.g. Ve, g. other words, GDCLA ignores the influence of group size in bias mitigation, resulting in robustness to imbalanced class distributions.
There is perfect alignment between ULPL and GDCLA A in the sense that the proxy group x' = will always be upweighted during the optimization of GDCLA- Lee. ULPL+GD reduces the loss disparities across instances within each target class, which in turn improves the lower bound of group-wise fairness, especially for the EO fairness. As a result the consistency, we observe that ULPL+GDCLA outperforms other unsupervised methods.
GDGLB is a variant of GDCLA, and they can only be differentiated by the way of incorporating faireess regularization:
LGDELa = C AGD
ΣΣLs L),
where the average loss in the regularization term is based on all instances (C), differing from the average loss within each target class (L9) for GLCA As result, GDGLE additionally encourages the performance parity across target class, which is typically known as long-tail learning. However, ULPL+GDGLB could potentially lead to worse results for better-pett cetter-perfomed target classes. For exam- ple, for ith target class, assuming that loss differences have been minimized (i.e., L =0 Liz'=l), it is possible that all instances with target label y = i will be under-fitted if L L.
Adv represents a different family of debiasing methods which aims learning fair hidden represen- tations. The training objective of Adv includes the mutual information (MT) to the training objective addition to standard loss:
Ladv L MI h).
where the h = e(x)) is the hidden representation of imput æ extracted from the encoder e. By minimizing the .MI objective, we expected the learned hidden representations h are orthogonal to protected attributes 7.
In practice, MI(z, h)) expressed by the combination of marginal entropy (z) and conditional entropy (H)z)h)): MI(z, h) Hzz) Hz)h),, where H is a constant number and can be ignored the optimization, and H (z(h) is estimated by an adversary (d) that is trained to identify the protected attributes (2 = d(h)).
The key step of Adv the training of d, i.e., if can effectively recover 2 from h. One problem associated with ULPL is that the mapping from proxy labels to ground truth labels is class-specific, making it harder for the recovering. Therefore, although the adversaries are non-linear classifiers, the effectiveness of ULPL+Adv is not good as other ULPL +* methods as shown in Table I
F FULL RESULTS
In addition to AUC-PFC scores, we present PFC and evaluation results for each dataset. We inves- tigate two different selection criteria and report the evaluation results over the development and test sets. Specifically, conduct model selection over the development set based on: (1) maximum fairness within performance trade-off threshold 5% (F@P- 1P-5% 3;); and (2) maximum fairness within performance trade-off threshold of 10% (F @ P-10%)
23

---

Published as a conference paper at ICLR 2023
Test Set
Development Set
Selection
Method
Performance
Fairness
Performance
Fairness
Vanilla
71.1 +
111
63.6 1
1.3
71.4
1.2
65.9 1
1.2
GDCLA
74.2 =
0.3
92.9
1.4
73.7= 0.3
94.5 1.0
F@P-5%
GD CLB
75.9 0.4
77.6
1.3
75.2 0.2
79.7 1.6
FairBatch
75.4 0.4
90.3
0.8
74.8
0.2
90.9 0.7
Adv
75,3 0.4
89.7
1.6
74.9 0.3
91.0 0.8
SemiAdv
75.7 0.2
90.1 0.5
74.9 0.4
91.1 0.5
ARL
70.7 1.0
78.5 5.5
69.2 1.0
78.1 5.7
ULPL
66.3 2.0
85.7 4.9
66.1 2.1
86.7 4.6
ULPL+GDeIAA
66.7 2.6
88.0 7.0
66.2 2.4
88.8 6.8
ULPL+GDGILR
72.6 0.4
65.1 1.6
72.5 0.6
66.9 1.3
ULPL+Adv
67.8 5.6
69.4 9.2
68.2 5.5
71.2 8.9
Vanilla
71.1 1.1
63.6 1.3
71.4 1.2
65.9 1.2
GDCLA
74.2 0.3
92.9 1.4
73.7 0.3
94.51 1.0
F@P-10%
GD GLB
75.9 0.4
77.6 L.3
75.2 0.2
79.7 1.6
FairBatch
75.4 0.4
90.3 0.8
74.8 0.2
90.9 0.7
Adv
75.3 0.4
89.7 1.6
74.9 0.3
91.0 0.8
SemiAdv
75.7 0.2
90.1 0.5
71.9 0.4
91.1 0.5
ARL
63.7 3.8
84.6 4.2
63.0
3.9
85.9 3.5
ULPL++GDLA
64.6 3.3
92.5
3.0
61.8
2.2
92.6 2.4
ULPL.+DOLL.
72.6 0.4
65.1
1.6
72.5
0.6
66.9 1.3
ULPL+Adv
67.8 5.6
69.4
9.2
68.2
5.5
71.2
8.9
Table 14: Evaluation results + standard deviation (%) of selected models over the Moji dataset.
For the demonstration purpose, here present the results for Moji. The full disaggregated results of 15 settings can also be seen at htt.:///w..............a../////a bocality-based, Method for Bias Ltgee http:p/eeee./..nennnooooooooorrr
wtt.://oooooonnnnnnnneeeeeeeeee Unispe
1,00
0.95
Models
0.90
Vanilla
0.85
GDaa
FarBatch
0.80
SemiAdv
0.75
0.70
Z
2's GDAA
2+Addv
0.50
0.55
0.60
0.65
0.70
0.75
Performance
Figure 6: PFC over the Moji dataset.
Moji Figure shows shows iectormance-firmenn trade-off curves (PFC) of each debiasing method. Ta- ble summarises performance and equal opportunity fairness results w.r.t. two different selection criteria the test and dev set.
The areas under each PFC in Figure 6 correspond to number in Table | Consistent with the Ta- ble [ can been from Figure 6 that GDCLA results in the best PFC. addition, PFCs of Adv and SemiAdv are highly overlapped with each other, and their AUC-PFC scores are also identical in Table Last but not least, ULPL+GD better than ARL most of the time in Figure 6 which is summarized as a 0.016 improvement in terms of AUC-PFC score in Table [
24


Reviews:
[{"review_id": "xbEFQzTSVN", "reviewer": "Reviewer_r7Nv", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new framework to achieve fair outcomes in settings where access to sensitive attributes is not available. The method is called Unsupervised Locality-base Proxy Label assignment (ULPL) and is based on assigning proxy labels according to model predictions for poor vs well-modeled instances. The authors perform various experiments on datasets from ML and NLP domains.", "summary_of_the_review": "Overall, the paper is a well-written and motivated paper; however, some deeper theoretical discussions can improve the paper drastically. The approach has also some flaws in terms of dependability to data, parameters, as well as some performance related concerns which I discussed in detail under weaknesses.", "strength_and_weaknesses": "**Strengths:**\n1. The paper studies an important subject. Achieving fair outcomes in settings where access to protected attributes is limited is important and interesting.\n2. Authors perform various experiments on datasets both from NLP and ML domains to showcase various aspects of the paper.\n3. The method is easy and straightforward.\n4. Paper is well-written and easy to follow.\n\n\n**Weaknesses:**\n1. It seems like the method is data dependent. Maybe results from section 5.1 on TrustPilot dataset is exactly due to this reason? I think further analysis or discussion need around this issue.\n2. Related to my above comment, while the paper does a good job in terms of empirical analysis, theoretical discussion is lacking. This theoretical discussion maybe address my above comment (1). That this approach can be applicable regardless of specific data properties.\n3. The performance of the approach is not impressive considering that comparison with the only unsupervised approach ARL is not done in a fair manner. \n4. How much do you think the base supervised approach that ULPL is implemented over is having effect on the results. Although authors discuss this briefly in \"Different ULPL methods:\" section. More in depth discussion might be useful to improve the paper.\n5. The approach is dependable on some parameters such as k which might make it inefficient. Is there a good way to make this more flexible?\n\n**Additional Questions that needs clarification:**\n\nThe framework based on indicating poorly vs well modeled instances seems more aligned with equalized odds or equality of opportunity notions of fairness. Do authors know why their approach works for statistical parity which is different in nature than what is proposed here? \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "o_NPFiLMkZV", "reviewer": "Reviewer_sY6r", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on the problem of debiasing representations learned by encoder-decoder models when demographic information is missing. The authors propose Unsupervised Locality-based Proxy Label assignment (ULPL), which assigns a binary “proxy” label to each example based on whether the model incurs a larger or smaller than average loss on that example. Once examples have been assigned proxy labels, it is possible to apply existing debiasing methods. In order to be robust to noisy outliers, the paper also proposes smoothing the proxy label for each example via a majority vote among that example's neighbors in the encoded space. In experiments on 5 datasets (with various protected attributes in focus for each dataset) they show that ULPL leads to larger areas under the performance fairness curve than ARL and a baseline that does not employ fairness interventions; ULPL is claimed to lead to similar performance as some supervised methods. Analysis showing that proxy labels generated by ULPL are generally correlated with protected labels is provided, as well as some analysis of kNN smoothing and an experiment where groups are identities by cross-products of protected attributes.\n", "summary_of_the_review": "The paper is written clearly but some important details are missing. One of the methods proposed in this paper (label smoothing) does not consistently provide benefit and seems impossible to tune. The primary method proposed (ULPL) seems reasonable but doesn’t lead to significant gains. \n", "strength_and_weaknesses": "Strengths:\n- The crux of the approach in this paper is to try to equalize training loss among all examples. The proposed method, ULPL, simply groups the examples into examples that experience lower than average loss, and those that experience higher.  Once labeled, other debiasing methods are used to equalize the loss. This seems reasonable as a way to improve fairness. The argument made in the paper is that examples with lower loss are likely to be of the same (under-represented) demographic group. While there is some empirical evidence that supports this claim in previous work, I find it weaker and somewhat inconsequential: if training loss is more evenly divided across examples (and training loss is a good proxy for performance) then fairness should improve across all groups.\n- This paper is clearly written and easy to follow.\n- Experiments are performed with 5 datasets. Significant analysis is also presented.\n\nWeaknesses:\n- The proposed label smoothing is not consistently effective. Table 3 shows that in 50% of experimental conditions, no label smoothing is better than label smoothing.\n- Label smoothing seems to be impossible to tune in realistic settings (in terms of the number of neighbors k). A standard approach of tuning k would be selecting k that yields the best results on the validation set. Since the metric being optimized is AUC-PFC, demographic group information is required to compute the metric. But the motivation of this paper includes the assumption that no demographic information is available.\n- The results are difficult to interpret. Table 1, which includes the main results, reports AUC-PFC, which characterizes the tradeoff between performance and fairness. I assume that the proposed method helps to improve fairness, but does it also improve performance? A more detailed characterization of how ULPL affects performance and fairness would help. Moreover, the difference between the proposed approach and other competing methods is often a few thousandths of a unit of area, which seems small, but as a reader it is difficult to be certain. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "eZmnkBRrrZ", "reviewer": "Reviewer_y5Cm", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an unsupervised method for bias-mitigation. The presented approach leverages proxy labels derived from comparing the instance-wise loss values to a threshold. These proxy labels are consolidated/smoothed through nearest neighbors in latent space (through embedding). The labels indicate poorly and well represented groups, which can be used in combination with arbitrary supervised methods for obtaining fairer classification models. The approach is extensively evaluated and is competitive with group-label supervised methods for bias mitigation.", "summary_of_the_review": "The topic of bias mitigation is relevant and the empirical results in this paper are interesting. While the empirical evidence suggests that the method improves fairness on real datasets, there are no theoretical guarantees or considerations, making it hard to deploy this method in high-stakes applications. However, I found the experiments well-designed and executed and the results very promising, which is why I am currently leaning towards acceptance.", "strength_and_weaknesses": "Strengths:\n\n1. The paper is well written. I enjoyed reading this paper and found it well-structured and good to follow.\n2. The paper addresses an important and timely problem. Bias is common in machine learning and methods to deal with it are relevant in practice. Furthermore, bias is often introduced implicitly, i.e., without group labels known at model training time. Therefore, the proposed method may be a valuable step towards fairer automated decisions.\n3. The experimental evaluation is extensive and rigorous and overall convincing. It includes several data modalities (tabular and text), fairness measures, datasets and baselines. Furthermore, important sanity checks such as the validity of the nearest-neighbor approximation for smoothing are in place. I briefly looked at the experimental code which appears to be in a good state. Intersectional discrimination of multiple attributes is also considered.\n4. The approach is simple and can be implemented in a few lines of code. Also, it only has one hyperparameter (the number of neighbors used for smoothing). This makes the method ready-to-use for practical applications.\n\nWeaknesses:\n\n1. There is no comparison between proxy labels and ground truth groups (e.g., given by race, gender, etc.). However, it is frequently claimed that the proxy labels correlate with demographic groups (for instance, right in the abstract, it says: “[...] that correlate with unknown demographic data”). The claim is repeated in 5.1, however, the performance gap is compared to the proxy labels only and not the ground truth group labels. This could be achieved by providing the statistics, e.g., confusion matrices, with ground truth group labels.\n2. The theoretical grounding of the work is weak, which raises the question whether there exist other simpler and principled solutions. While the paper provides some intuition for why the proposed method works, it lacks theoretical evidence from point to point. For instance, it is not clear why U should be generally replaced with the Loss in Section 3.2 and how the group-based formulation is transformed to an individual one. I see that the experimental results are promising, but the derivation could be backed with more evidence or theoretical results.\n3. There are no theoretical guarantees that the proposed method will lead to fairer models and that the proxy objective corresponds to the actual one (or bounds on the error). This also raises the question if the fairness could possibly be decreased. This could be a problem in high-stakes scenarios. In such settings, a black-box method like the one presented is unlikely to be deployed.\n\nMinor comments:\n1. I would have appreciated further evidence or intuition for some claims made in the paper. Here are some examples: \n(1) Because TrustPilot is already fair, the ground truth labels cannot be aligned with the TPR gaps (Section 5.1) \n(2) Why is GD-CLA more robust to conflated classes (final paragraph of 4.2) \n(3) The discussion of upstream and downstream debiasing (Section 2, “Unsupervised bias mitigation”). As far as I understand, upstream debiasing is also unsupervised? Thus, a comparison with one of these methods could be insightful in terms of future work, since these methods could be applicable to the same set of problems.\n2. Related work: Please use \\citep (end of first paragraph, Point (1) in second paragraph).\n\nQuestions:\n1. How exactly was the ROC-RFC computed? Were the scores normalized? For instance, looking at Figure 5b), the area under the curve is well above 0.8 x 0.8=0.64 for both methods. Even if you normalize the curve (taking the left and right-most point as references), the areas are well above 0.5. However, in Table 1, values reported for Bios-G are all below 0.5. How can this be explained?\n2. ROC-PFC values for Trust Pilot, Adult, Compas are extremely low (~0.1). This indicates that performance is hurt drastically even with slowly introduced fairness. Why is this the case? Is the AUC-PFC a still a representative measure in this case? Maybe this is connected to the first question.\n3. Table 3 shows the best values for the nearest neighbors smoothing which are different across datasets. How would you choose a suitable value for k in practice, without having access to ground truth fairness measures?\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0UI8uUpqpk", "reviewer": "Reviewer_djVz", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Bias creeps into ML models trained from human behavioural data, which can usually be mitigated against but (typically) with labeled data for that purpose. This paper proposes a meta-algorithm for debiasing representations called Unsupervised Locality-based Proxy Label assignment. It is evaluated over five datasets, from NLP to structured data, and this evaluation shows that the provided technique recovers proxy labels that correlate with ‘unknown demographic data’. Performance is competitive with supervised methods. ", "summary_of_the_review": "\n- The positives of moderate novelty, fairly good empirical approaches (sub analyses, datasets, baselines), and the importance of this space outweigh negatives of glossed-over rationales and assumptions, and perhaps some overstatement of performance.", "strength_and_weaknesses": "Stengths\n\n- Bias remains a crucial (and ongoing) challenge across many subdomains of ML. It is commendable that this work generalized beyond one of these subdomains in its analysis. Moreover, the focus on unlabeled demographics is more ecologically valid and therefore useful.\n- Good, relevant, and (mostly) recent literature is effectively compared and contrasted in a well-structured manner.\n- The datasets and baselines for empirical evaluation are well chosen\n- The questions asked in each of the sub analyses of Sec 5 are well-chosen and mostly well executed\n\nWeaknesses\n\n- There are multiple group-wise measures of fairness beyond the one adopted in Sec 3.1. A minor point, but it would be preferable to show some rationale for the method chosen, and some indication of the alternatives.\n- Certain assumptions seem to be glossed over in Sec 3.2, such as how (or whether) over- or under-represented groups can be binarized according to z_i’. This may of course be a useful heuristic empirically, but whether it has anything to do with actual groups or is merely correlated with them for other, more spurious reasons should be better explained (e.g.., in sec 5.2). \n- Empirical results are fairly similar across rows, for multiple datasets and attributes, in Table 1. That this is the case for the unsupervised method is actually a positive outcome, but claims of ‘consistently outperforming’ other approaches seems to require some tempering, and some statistical tests of significance should nevertheless be run.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster