Fairness Under Demographic Scarce Regime
Authors: Patrik Joslin Kenfack
Samira Ebrahimi Kahou
Ulrich Aïvodji
Venue: Accepted by TMLR ()

Abstract: Most existing works on fairness assume the model has full access to demographic information. However, there exist scenarios where demographic information is partially available because a record was not maintained throughout data collection or for privacy reasons. This setting is known as demographic scarce regime. Prior research has shown that training an attribute classifier to replace the missing sensitive attributes (proxy) can still improve fairness. However, using proxy-sensitive attributes worsens fairness-accuracy tradeoffs compared to true sensitive attributes. To address this limitation, we propose a framework to build attribute classifiers that achieve better fairness-accuracy tradeoffs. Our method introduces uncertainty awareness in the attribute classifier and enforces fairness on samples with demographic information inferred with the lowest uncertainty.  We show empirically that enforcing fairness constraints on samples with uncertain sensitive attributes can negatively impact the fairness-accuracy tradeoff. Our experiments on five datasets showed that the proposed framework yields models with significantly better fairness-accuracy tradeoffs than classic attribute classifiers.  Surprisingly, our framework can outperform models trained with fairness constraints on the true sensitive attributes in most benchmarks. We also show that these findings are consistent with other uncertainty measures such as conformal prediction.

Published in Transactions on Machine Learning Research (09/2024)
Fairness Under Demographic Scarce Regime
Patrik Joslin Kenfack ÉTS Montréal, Mila
patrik-jostin kenfack. elsmll...
Samira Ebrahimi Kahou
University of Colgary, Mila Canada CIFAR AI Chair
UIrich Ařvodji ETS Montréal, Mila
Aibich.Aivodiieetst..coom
Reviewed OpenReview: https:
https://openreeveeu. openreview.
net/ forum? id= TB18G0w6Ld
Abstract
Most existing works fairness assume the model has full access to demographic information. However, there exist scenarios where demographic information is partially available because a record was not maintained thronghout data collection or for privacy reasons. This setting known as demographic scarce regime. Prior research has shown that training an attribute classifier to replace the missing sensitive attributes (prozy) can still improve fairness. However, using proxy-sensitive attribules worsens fairness-accuracy tradeofTs compared to true sensitive attributes. To address this limitation, we propose a framework to build attribute classifiers that achieve better fairness-accuracy tradcoffs. Our method introduces uncertainty awareness in the attribute classifier and enforces fairness samples with demographic information inferred with the lowest uncertainty. We show empirically that enforcing fairness constraints on samples with uncertain sensitive attributes can negatively impact the fairness-accuracy tradeoff. Our experiments on five datasets showed that the proposed framework yields models with significantly better lairness-accurag, tradeoffs than classic attribute classifiers. Surprisingly, our framework can outperform models trained with fairness constraints on the true sensitive attributes in most benchmarks. We also show that these findings are consistent with other uncertainty measures such conformal prediction. The source code is available at
1 Introduction
Mitigating machine learning bias against certain demographic groups becomes challenging when demographic information is wholly partially missing. Demographic information can be missing for various reasons, e.g., due to legal restrictions, prohibiting the collection of sensitive information of individuals, or voluntary disclosure of such information. As people are more concerned about privacy, reluctant users will not provide sensitive information. As such, demographic information available only for a few users. A demographic scarce regime was the term used by |Awasthi et al. (2021) describe this particular setting. The data this setting can be divided into two sets: D, and D2. The dataset D, does not contain demographic information, while D2 contains both sensitive and non-sensitive information. The goal is to train fair classifter with respect to different (unobserved) demographic groups in D1. Without demographic information in D1, it is more challenging to enforce group fairness notions such as statistical parity (Dwork et al.) 2012) and equalized odds (Hardt el al. 20166). Algorithms to enforce these notions require access to sensitive attributes to quantify and mitigate the model's disparities across different groups (Hardt et al. 2016 Agarwal et al.) 2018). However, having access to another dataset where sensitive attributes are available gives room train a sensitive attribute classifier that can serve as prozy for the missing ones. We are interested in understanding what level of fairness/aue /accuracy one can achieve if proxy-sensitive attributes used to replace

---

Published in Transactions on Machine Learning Research (09/2024)
the true sensitive attributes and properties of the sensitive attribute classifier and the data distribution that influences the fairness-accuracy tradeoff.
In their study, Awasthi et al. (2021) demonstrated a counter-intuitive finding: when using proxy-sensitive attributes, neither the highest accuracy nor an equal error rate of the sensitive attribute classifier has an impact on the accuracy of the bias estimation. Although Gupta et al. (2018) showed that improving fairness for the prozy demographic group can improve fairness with respect the true demographic group; it remains unclear how existing fairness mechanisms would perform in the presence of proxy-sensitive attributes and how the fairness- accuracy tradeoff is impacted. We show that existing fairness-enhan enhancing methods (Hardt ct al. 2016 Agarwal et al.) 2018) can be robust to noise introduced in the sensitive attribute space by the proxy attribute classifier, i.e., unfairness can be mitigated when proxy attributes are used instead of the sensitive attribute. However, the fairness-accuragy tradeoff worsens when fairness constraints are imposed on these proxy-sensitive attributes. We aim to provide insights into the distribution of sensitive attributes that can yield better fairness and accuracy performances. We hypothesize that the uncertainty of the sensitive altribute classifier plays critical role in improving fairnes--aceuraey tradeoffs downstream tasks. Specifically, we argue that samples with reliable demographic information should be used to fit fairness constraints, backed up by the intuition that these samples are easier to discriminate against, while samples with uncertain demographic information are already hard to discriminate against. Validating this hypothesis requires effective uncertainty estimation in the sensitive attribute space.
In this paper, we propose a framework to handle Fairness under Demographic Scarce Regime (FairDSR). FairDSR. consists of two phases. In the first phase, we construct an uncertainty- -ware Deep Neural Network (DNN) model to predict demographic information. The training is semi-supervised using self-ensembling (Tar vainen & Valpola, 2017), and the uncertainty of the predictions is measured and improved during the training using Monte Carlo dropout (Gal & Ghahramani 2016). The first phase outputs for each data point, the predicted sensitive attribute, and the uncertainty of the prediction. In the second phase, the classifier for the target task is trained with fairness constraints w.r.t to the predicted sensitive attributes. However, fairness constraints are imposed only on samples whose sensilive altribule values are predicted with low uncertainly. Our results rely heavily on dropout regularization and the effectiveness of the uncertainty measure. Existing uncortainty measures compute prediction uncertainty using the model in certain epoch, e.g., in the last training epoch. However, the uncertainty measure can vary for different models in different training epochs due to different perturbations, such as network dropout, input noise, and data ordering. Hence, we adopt self- ensembling to compute more consistent uncertainty by considering the training dynamic. In self-ensembling learning, the model ensemble is obtained using an exponential running average of model snapshots. In the semi-supervised setup, an unsupervised loss term ensures the model's output remains consistent with different. perturbations. For example, given a set of labeled or unlabeled samples, the unsupervised loss enforces the model's 's output at the current epoch to be similar to the output aggregated from previous training epochs under different perturbations, The aggregated outputs can be obtained using the moving average of the model's output Laine & Aila (2177) or the model's weights Tarvainen & Valpola| (2017) throughout the training. In the first phase of FairDSR, we use similar approach to train the attribute classifier with a reliable uncertainty measure in the data without demographic information. In addition lo supervised loss, we add an unsupervised loss term that ensures the model's uncertainty remains consistent within epochs.
On the other hand, considering the other aspect of our hypothesis, the uncortainty estimated by our method is used to derive a subset of the dalaset having higher uncertainty in the demographic information prediction. We demonstrate that using this subset to train a model without fairness constraints can yield fairer outcomes. Furthormore, we show how other uncertainty mcasures such as conformal prediction (Angclopoulos et al.) 2023) can be integrated with our framework while achieving similar performance gains regarding tradeoffs in fairness and accuracy. Our main contributions are summarized as follows:
We show that the fairness-an accneess-ccuracyy tradeoff is suboptimal when using proxy-sensitive attributes instcad of truo-sensitive attributes. Specifically, when the sensitive attribute is partially available, replacing missing values using data imputation techniques based on k-nearest neighbor (KNN) or DNN models can yield reasonably fair model but a worse fairness-accuracy tradeoff.
2

---

Published in Transactions on Machine Learning Research (09/2024)
We propose simple yet effective framework to improve the tradeoff by exploiting the uncertainty of the sensitive attribute predictions, which we show can play an important role in achieving better accuracy-fairnes tradeoffs. We hypothesize that a better fairness-accuracy tradeoff can be achieved when fairness constraints are imposed on samples whose sensitive attribute values can be predicted with low uncertainly. We also show that a model trained without fairness constraints but using data with higher uncertainty in the predictions of sensitive attributes tends to be fairer across different. fairness metrics.
We perform extensive experiments on wide range of real-world datasets to demonstrate the effectiveness of the proposed framework compared to existing methods. Our results also show that the FairDSR can significantly oulperform a model trained with fairness constraints observed sensitive attributes. Applying our method in settings where demographic information is fully available can yield better fairnes--accuracy tradcoffs.
2 Related Work.
Various metries have been proposed the litcrature to mcasure unfairness in classification, as well as numerous methods to enforce fairness as per these metrics. The most popular fairness metrics include demographic parity (Dwork et al.) 2012), equalized odds, and equal opportunity (Hardt et al. 2016). Demographic parity enforces the models" positive outcome to be independent of the sensitive attributes, while equalized odds aim at equalizing models' true positive and false positive rates across different demographic groups. Fairness- enhancing methods are categorized into three groups: pre-processing (Zemel et al., 2013; Kamiran & Calders) 2012), in-processing (Agarwal et al., 2018 Zhang et al. 2018), and post-processing (Hardt et al.) 2016), depending on whether the fairness constraint is enforeed before, during, or after model training respectively. However, enforcing these fairness notions often requires access to demographic information. There are fairness notions that do not require demographic information to be achieved, such as the Rawlsian Mar- Min fairness
notion [Rawls] ) which aims at maximizing the utility of the worst-case (unknown) group (Hashimoto
ct al. 2018 Lahoti et al. 2020 Liu et al. 2021 Levy et al.) 2020). Specifically, these methods focus on maximizing the accuracy of the unknown worst-case group. However, they often fall short in effectively
targeting the specific disadvantaged demographic groups or improving group fairness metrics (Franke 2021 Lahoti et al. 2020). In contrast, we aim to achieve group fairness through proxy-attributes using limited demographic information. Recent efforts have explored bias mitigation when demographic information is noisy (Wagg et al. 2020; Chen et al. 2022a Noise can be introduced in the sensitive feature space due to human annotation, privacy mechanisms, or inference (Chen et al. 2022b). Chen et al. (2022a) aims to correct the noise in the sensitive attribute space before using them in chiress--enannee
algorithms. Another line of work focuses on alleviating privacy issues in collecting and using sensitive attributes. This group of methods aims to train fair models under privacy-prosorvation of the sensitive attributes. They design fair models using privacy-preserving mechanisms such as trusted third party (Veale & Binns] 2017). secure multiparty
computation (Kilbertus et al.) 2018), and differential privacy (Jagielski et al. 2019].
The most related work includes methods relying on proxy-sensitive attributes to enforce fairness when partial demographic information is available. Guppt et al. (2018) used non-protected features to infer proxy demographic information to replace the unobserved real ones. They showed empirically that enforcing fairness with respect to proxy groups generalizes well to the real protected groups and can be effective in practice. While they focus on post proceerocessing techniques, we are interested in in-processing methods. [oston et al. (2019); Liang et al. (2023) assumed sensitive attribute available either in a source domain or the target domain and used domain adaptation-like techniques to enforce fairness in the domain with missing sensitive attributes. Moreover, while these methods can improve fairness regarding true sensitive attributes, they result in a worse trade-off fairness accuracy than the method using the true sensitive attributes. Diana et all (2022) showed that training proxy classifier under multi-accuracy constraints can be good substitute for the ground truth sensitive altributes when the latter is missing; however, the resulting proxy sensitive altributes also yield worse fairness- fairnes--accuracy tradeoff. Liang et al. (2023) also considers the uncertainty in sensitive space and proposes to replace uncertain predictions with random sampling from the empirical conditional distribution of the sensitive attributes given the non-sensitive attributes P(A[X, Y). However, the random sampling of sensitive attributes is sensitive to noise in the empirical distribution, which can negatively impact.
3

---

Published in Transactions on Machine Learning Research (09/2024)
the fairness accuracy tradeoff. Morcover, this work provides comprehensible analysis of the impact of uncertainty in sensitive attribute space over the fairness-acurracy tradeoff. On the other hand, Awasthi et al.] (2021) proposed an active sampling approach to improve bias assessment using predicted sensitive attributes. While their method focuses on bias assessment using proxy attributes, we want to improve the tradcoff between fairness and accuracy, and bias assessment under missing sensitive attributes is out of the scope of the paper. In sum, related work relying on proxy-sensitive attributes mostly focuses on assessing what level of fairness can be achieved when proxy-sensitive attributes are used (Coston et al. 2019), properties of the sensitive attribute classifier (Diana et al., 2022; Coston et al.) 2019), and bias assessment via proxy sensitive features (Awasthi et al. 2021). Our proposed method focuses on reducing the accuracy-fainma fairness tradeoff when proxy attributes are used.
3 Problem Setting and Preliminaries.
Problem formulation. We consider a dataset D1 (x.3) where X = { represents the non- sensitive input feature space and y = {0,1} represents the target variable. The goal to build a classifier, f X:X + V. that can predict y while ensuring fair outcomes for samples from different demographic groups. However, demographic information of samples in D1 is unknown. We assume the existance of another dataset D2 = A} sharing the same input feature space as D2 and for which demographic information is available, i.e., (0, We assume binary demographie groups for simplicity. Therefore, the dalaset D, contains label information and D2 contains demographic information. Our goal is to leverage D2 to train an attribute classifier h X - A that can serve as a proxy to the sensitive attributes for samples in D1, for which a fairness metric can be enforced in way to improve fairness with respect to the true sensitive attributes. Attribute classifiers have been used in health (Brown et al.] 2016 Fremont et al.] 2005 and finance (Zhang) 2018 Silva et al., 2019) to infer missing sensitive attributes, in particular when users or patients self-report their protected information. To be able to estimate the true disparities in the label classifier f, we assume there exists a small set of samples drawn from the joint distribution X' x y x A, i.e., samples that jointly have label and demographic information. If this subset is not available, one can consider using the active sampling technique proposed by Awasthi et al. (2021) to approximate bias with respect to the predicted sensitive attributes. This estimation is beyond the scope of this work. We aim effectively assess fairness without being overly concerned about bias overestimation or underestimation.
Reducing the tradeoff between fairness and accuracy is significant challenge within the fair machine-learning community (Dutta et al. 2020 Kenfack et al.] 2021). Our primary goal is to design a method that effectively leverages proxy features to achieve similar or better fairnes--accuracy tradeoffs compared to settings where the true sensitive attributes are available. To this end, we considered a different range of fairness metrics and lairness-enhancing techniques.
Fairness Metrics. In this work, we consider three popular group fairness metrics: demographic par- ity (Dwork et al. 2012). equalized odds, and equal opportunity (Hardt et al. 2016). These metrics aim to equalize the model's 's performance across different demographic groups; we provide more details in Appendix
Fairness Mechanism. We focus on in-processing techniques to improve the models' fairness. These methods introduce constraints in the classification problem to satisfy a given fairness metric. Our study focuses state-of-the-art techniques in this category, ie., exponentiated gradient (Agarwal et al. 2018 and adversarial debiasing (Zhang et al.) 2018). We considered these methods as they allow better control over lairness and accuracy. In general, the optimization problem contains a parameter that controls the balance between fairness and accuracy, i.e,, a higher value of À would force the model to achieve higher fairness (respectively lower accuracy) while a smaller yiclds higher accuracy (respectively lower fairness). We aim to design a sensitive attribute predictor that achieves better fairnss-aaccuracy tradeoff, i.c., for the same value of X build model that provides higher accuracy and lower unfairness.
4

---

Published in Transactions on Machine Learning Research (09/2024)
190000
Di Do
Student madel
Uncertainty
fibering
D2
EMA
Fairners constraints
D1
ronde!
Step Uncertainty-anee aware sonsitive prodiction
Stop prodiction with fairness constralists
Figure 1: Overview of FairDSR. Our framework consists of two steps. In the first step (left), the dataset D2 is used to train the attribute classifier for the teacher framework. The first step produces proxy-sensitive altribules (h(X) Å) and the uncertainly of their predictions (U). In the second step (right), the fair model is trained using only samples with reliable proxy-sensitive attributes. These samples are selected based a defined threshold of their uncertainties.
4 Method
This section presents the methodology and all components involved in the framework FairDSR. Figure presents an overview of the stages in our framework and the interactions between and within each stage. The first stage consists of training the attribute classifier with uncortainty awareness. This step outputs for each sample with missing sensitive altribule, its predicted sensitive attribute (proxy), and the prediction's uncertainty. In the second stage, the label classifier is trained with fairness constraints enforced using the predicted sensitive aftributes. To validate our hypothesis, fairness is enforced only on samples with the lowest uncertainty in the sensitive attribute prediction, i.e., samples with an uncertainty lower than a predefined uncertainty threshold H.
4.1
Uncertainty-Aaar Attribute Prediction
We build the sensitive attribute classifier using a semi-supervised learning approach that accounts for the uncertainty of the predictions of samples with missing sensitive attributes, similar to Yu et al. (2019); Tarvainen & Valpola (2017); Laine & Aila (2017). Motivated by the uncertainty estimation in Bayesian networks, we estimate the uncertainty with the Monte Carlo Dropout (Gal & Ghahramani 2016). However, measuring the uncertainty of the predictions using the model in given training step (e.g.. the last epoch) might result in an unreliable uncertainty measure due to different perturbations, such as network dropout, input noise, and data ordering (He et al.) (023).. In addition, we are interested in designing an attribute classifier that minimizes the uncertainty of the predictions in the data with missing sensitive attributes.
overcome this challenge, we adopt self-ensemble learning lo ensure consistent uncertainly measurement by maintaining the moving average of the attribute classifier's weights during the training (Tavaaine & Valpola 2017). More specifically, the model maintains moving average of itself during the training to provide stable learning signal. We refer to the model at the current training epoch as the student model and its moving average (self-ensembling) from previous epochs as the teacher model.
Student Model. The student model is implemented as a neural network and is trained on D2 (samples with sensitive attributes) to predict sensitive attributes. Under a semi-supervised setup, the attribute classifier is optimized to minimize a double loss: the classification loss (L,), i.c., the cross-entropy loss, and the consistency loss (Le) YYu al. 2019). The consistency loss (or unsupervised loss) forces the student model to focus on samples with low uncertainty in the sensitive aftributes guided by the uncertainty estimation
5

---

Published in Transactions on Machine Learning Research (09/2024)
from the teacher model. This loss is defined the mean squared difference between the teacher's and student outputs (logits) on samples whose uncertainty does not exceed a threshold R.
Overall, the attribute classifier is trained to minimize the following loss:
min E JEF (r.0)-DxXA
L(((()),)+)
E L.(((z),M()) INDi-D,
(1)
where f() is the student model, h(-) the teacher model, and À a parameter controlling the consistency loss. The empirical loss minimized is defined by the following equations for classification (L,) and consistency loss Le:
C,
Σ a (o)o)((())) + (1 (1-a) a) log(11 f fx))
(2)
D22
a2,E
Le D22 + D1
Σ
fl(()) ((x) 1|
(3)
SR
The consistency loss is applied only on samples, x, whose uncertainty, M2y is lower than the threshold R. Following Srrvastava al. ) Baldi & Sadowski (013)) R and À are updated using Gaussian warmup function to prevent the model from diverging at the beginming of the training. The motivation behind the consistency loss is the focus on our primary goal for the attribute classifier, which is to find the missing sensitive attributes in Dj with low uncertainty. In the experiments (Section 5.2.3) we show that training the model without consistency loss results in worse accuracy tradeoffs.
Teacher Model. The teacher model maintains the moving average of the student's weights and is used for uncertainty estimation. Specifically, the teacher weights are updated within each training epoch, t, using the exponential moving average (EMA) of student weights as follows:
W1 aw:-1 1 + (1-a))
(4)
where 0 and w denote the respective weights of student and teacher and a controls the moving decay. The use of EMA to update the teacher model is motivated by previous studies (Laine & Aila 2017 Yu et al.) 2019) that have shown that averaging model parameters at different training epochs can provide more reliable predictions than using the most recent model weights in the last epoch. The teacher model gets as input both labeled and unlabeled samples (D, and D2) and computes the uncertainly of their predictions using Monte Carlo (MC) dropout (Gal & Ghahramani 2016).
Uncertainty Estimation. MC dropout is an approximation of a Bayesian neural network widely used to interpret the paramcters of noural networks (Abdar et al.) 2021). MC dropout can effectively capture various sources of uncertaintics (Alcatoric and Epistemic uncertainty). It uses dropout at test time to compute prediction uncertainty from different sub-networks that can be derived from the whole original neural network. Dropout is generally used to improve the generalization of DNNs. During training, the dropout layer randomly removes unit with probability p. Thercfore, each forward and backpropagation pass is done on different model (sub-nelwork). forming ensemble of models that are aggregated together to form final model with lower variance (Srivastava et al. 2014 Baldi & Sadowski 2013). The uncertainty of each sample is computed using T stochastic forward passes on the teacher model to output T independent and identically distributed predictions, i.e., (z), ... ,,)(()) The softmax probability of the output set is calculated. The ...
uncortainty of the prediction (u,) quantified using the resulting entropy: u, = Σ, (ar) log(Pa (x)), where (a( the probability that sample x belongs to demographic group estimated over T stochastic forward passes, i.e, (x) = + ΣT, (x).. In the experiments (Section 5.2.3), we compare with another uncertainty measure based on the confidence interval of the prediction probability and show the limitation of this measure in identifying highly uncertain samples. Furthermore, we also show that using reliable uncertainty measures tradeoff.
such conformal prediction
(aneatiionnnnnnalaaa (Angelopoulos et al. 2023) also improve the

---

Published in Transactions on Machine Learning Research (09/2024)
4.2 Enforcing Fairness w.r.t Reliable Proxy Sensitive Attributes
After the first phase, the attribule classifier can produce for every sample in Di, i.e., samples with missing sensitive attributes, their predicted sensitive attribute (proxy) A = {h(zz),,eD, J. and the uncertainty of the prodiction U = )M, To validate our hypothesis, we define a confidence threshold H for samples used to train the label classifier with fairness constraints, i.., the label classifier with fairness constraints is trained on a subset Dj C DD defined as follows:
= [(= S H1
(5)
Note that the threshold R in the previous step is only used to train the attributes classifier while ensuring the student and teacher remain consistent for samples with uncertainly lower than R. The threshold in this step used test time to select samples with low uncertainty, and its value can be tuned over a validation set. The hypothesis of enforcing fairness samples whose sensitive attributes from the fact that the model can confidently distinguish these samples based on their sensitive altributes in the latent space. In contrast, the label classifier is inherently fairer if an attribute classifier cannot reliably predict sensitive attributes from training data (Kenfack et al. 2023). We further support this in section 5.2 by comparing the sensitive attribute uncertainty in the New Adult dataset (Ding et al. 2021) and the old version of the dataset (Asuncion Newman 2007). The fairness constraints on samples with unreliable sensitive attributes could push the model's decision boundary in ways that penalize accuracy and/or fairness. We support these arguments in the experiments. In addition to this method, we considered two other approaches built around our hypothesis: the weighted approach- FairDSR (weighted) and the uncertain approach- FairDSR (uncertain)
reliably prodicted stems
FairDSR (weighted). Given that downsampling can reduce accuracy when uncertainty is low, instead of pruning out samples based on the uncertainty of their sensitive attributes, we considered a training process in the second step that enforces fairness constraints on all samples but weighted proportionally to the uncertainty of their sensitive attributes, i.e., samples with less uncertainty receive higher weights. We show in the experiments that this weighted approach can better preserve accuracy.
FairDSR (uncertain). Considering the ethical risks of inferring missing sensitive information, also considered a variant in the second step where the model is trained without fairness constraints, i.e., without using sensitive information. Instead, we train the model using samples with higher uncertainty in their sensitive attributes for a given uncertainty threshold. We show in the experiments that this variant can significantly improve fairness, although no fairness constraints are applied.
5
Experiments
In this section, we rate the elfectiveness our framework on five datasets and compare it to different. basclines.
5.1
Experimental Setup
Datasets. We validate our method on five real-world benchmarks widely used for bias assessment: Adult Income (Asuncion & Newman 2007 Compas (Jeff et al., 20166), Law school (LSAC), CclebA (Liu al., 2018) Wightman 1998), and the New Adult (Ding et al.] 2021) dataset. We use 20% of each dataset as the group-labeles dataset (D2) and 80% the dataset without sensitive attributes (Dz). Supplementary C.1 shows that our results hold even with smaller group-labeled ratio, e.g,, 5%. More details about the datasets appear in Supplementary B.2
Attribute classifier. The student and teacher models were implemented as feed-forward Multi-layer Perceptrons (MLPs) with Pytorch (Paszke et al. 2019). and the loss function [ is minimized using the Adam optimizer (Kingma & Ba 2014) with learning rate 0.001 and batch size 256. Following Yu et al.] (2019); Laine & Aila 2017), we used 0.99 for the EMA parameter for updating the teacher weights using the student's
hittps///archiee htw.:iiiww...oo....cdd/////////e wtt.:accw./m////aaaaat////////// Adult

---

Published in Transactions on Machine Learning Research (09/2024)
weights across epochs. The uncertainty threshold is finetuned over the interval [0.1, using 10% of the training data. The best-performing threshold is used for the second step's threshold to obtain D'1. The uncertainty threshold that achieved the best results are 0.30, 0.60, 0.66, and 0.45 for the Adult, Compas, LSAC, and CelebA datasets, respectively.
Baselines. For fairness-enhancinn mechanisms, we considered the Fairlean (Bird et al, 2020) implementation of the exponentiated gradient (Agarwal et al. 2018). We considered the three variants of our approach described in section 4.2 FairDSR (weighted), FairDSR (uncertain), and our base method, FairDSR (certain).
For comparison, considered methods aiming to improve fairness without (full) demographic information. We compare with the following methods:
FairRF [haoo et al. y This method assumes that non-sensitive features correlating with sensitive attributes are known. It. leverages these related features to improve fairness w.r.t the unknown sensitive attributes.
FairDA (Liang et al. 2023 Similar to our sctting, this method assumes the sensitive information is available in source domain (dataset D2 in our setting). It uses a domain adaptation-based approach to transfer demographic information from the source domain to improve fairness in the target using an adversarial approach.
CGL (Jung et al. )022;: With partial access to sensitive attributes, this method also uses an attribute classifier and replaces uncertain predictions with random sampling from the empirical conditional distribution of the sensitive attributes given the non-sensitive attributes P))(X,Y).
ARL (Lhhoi et al. 2020): The method uses adversarial approach to upweight samples in regions hard to learn, i.c., regions where the model makes the most mistakes.
Distributionally Robust Optimization (DRO) (Hashimoto al. 2018): It optimizes for the worst-case distribution around the empirical distribution. Similar to ARL, the goal is to improve the accuracy of the worst-case group.
CVarDRO (Levy et al.
)020): It is an improved variant of DRO.
KSMOTE (Yan et al. 2020) performs clustering to obtain pseudo groups and use them as substitutes to oversample the minority groups.
For each method considered for comparison, we used the code provided by the author along with the recommended hyperparameters. We considered two vanilla baselines: baseline where the model is trained without fairness constraints (Vanilla (without fairness)) and a bascline model trained with fairness constraints over the true sensitive attributes (Vanilla (with fairness)). We also considered baselines trained with fairness constraints directly using predicted sensitive attributes obtained by data imputation using student network (Proxy-NNN) and imputation using K-ncarest neighbor (ProXy-KNN).
For comparison, in addition to the accuracy, we consider the three fairness metrics described in the appendix B i.e, equalized odds (ABOD), equal opportunity (ABOP), and demographic parity (App). All the baselines are trained on 70% of D,, and fairness and accuracy are evaluatod on the 30% as the test set. We assume the sensitive attribute is observed the test to report the true fairness violation. We trained each baseline times and averaged the results. We use logistic regression as the base classifier for all the baselines and train each baseline to achieve minimal fairness violation.
5.2 Results and Discussion
5.2.1
Uncertainty of the sensitive attribute and fairness.
We first analyze the relation between the sensitive attribute's uncertainty and the fairness of downstream models. Table showcases the average uncertainty of the ivnsitive attribute prediction estimated by our
We implemented FairDA and reproduced using the instructions the paper [Liang et 2023
3 Appendix C.2 shows comparison with MLP model.
8

---

Published in Transactions on Machine Learning Research (09/2024)
Dataset
Mean uncertainty (4)
Accuracy sensitive attribute (t)
Apr
ÅEoD
Adult
0.15
85%
0.18
0.20
0.13
New Adult
0.42
68%
0.06
0.05
0.04
Compas
0.39
72%
0.28
0.32
0.32
CelebA 0.66
0.21
83%
0.17
0.19
0.19
LSAC
55%
0.014
0.005
0.049
Table 1: Average uncertainty of the attribute classifier and fairness of label classifier on the dataset with missing sensitive attributes.
Method
Accuracy
App
ΔBOD
Vanilla (without fairness)
0.851 + 0.008
0.171 + 0.004
0.088 + 0.033
0.091 + 0.030
Vanilla (with fairness)
0.829 1 0.002
0.005 11 0.004
0.021 1 0.014
0.017 1 0.007
Proxy-DNN
0.829 + 0.002
0.009 + 0.005
0.024 + 0.014
0.019 + 0.017
FairRF
0.838 = 0.002
0.162 = 0.015
0.063 = 0.027
0.072 = 0.019
FairDA
0.809 = 0.009
0.087 + 0.028
0.071 = 0.046
0.078 = 0.039
CGL
0.834 + 0.002
0.009 0.006
0.027 + 0.020
0.026 + 0.016
ARL
0.850 + 0.002
0.173 0.013
0.028 0.090
0.097 + 0.031
CVarDRO
0.820 1 0.012
0.200 0.005
0.160 0.030
0.100 1 0.027
KSMOTE
0.814 + 0.003
0.302 0.007
0.160 + 0.021
0.196 + 0.003
DRO
0.823 1: 0.003
0.184 + 0.042
0.092 1: 0.041
0.105 + 0.011
FairDSR (weighted)
0.846 + 0.003
0.032 0.013
0.050 + 0.033
0.027 1 0.019
FairDSR (uncertain)
0.825 = 0.013
0.106 0.036
0.065 + 0.047
0.068 = 0.032
FairDSR (certain)
0.830 = 0.004
0.007 + 0.005
0.015 + 0.010
0.018 = 0.016
Table 2: Results on the Adult dataset. Bolded values represent the best-performing basclines among the bairness-enhancin methods without (full) demographic information. All the baselines are trained on the dataset without the sensitive attributes (Dz). Each experiment is conducted 7 times, and the fairness and accuracy are averaged.
method and different fairness measures of logistic regression model trained without fairness constraints on the dataset D1. These results show the correlation between the uncertainty of the sensitive attribute prediction and the fairness of the model. For example, we observe that the uncertainty in the Adult dataset is lower than in the New Adult dataset, while the unfairness in the Adult datasct is higher. On the other hand, the LSAC dataset has the highest uncertainly of the sensilive attribute (0.66), and the model without. fairness constraints has the smallest fairness violation across all fairness metrics. In sum, we can observe that unfairness is higher for datasets with lower uncertainty in the sensitive attributes, c.g., the Adult and CelebA datasets. These results support hypothesis that model can hardly discriminate against samples with uncertain demographic information. Furthermore, we investigate the relation between the level of uncertainty of the sensitive attributes in the training data and the fairness of downstream classifiers trained without fairness constraints. Our study reveals that a model without fairness constraints tends to be fairer as the uncertainty of the sensitive attribute the training data increases.
Specifically, we trained different classifiers (Logistic Regression and Random Forest) without fairness con- straints but using training data with higher uncertainty in the sensitive attributes. For different uncertainty thresholds H E {0.0, 0.1, 0.6), prune out samples whose uncertainty is lower than H and train the model without fairness constraints using the remaining training data, i.c., {(z, y) E D2 M2 2 H), where u, is the estimated uncertainly provided by our method. In particular, when H = 0, all the data points are used for the training, and in other cases, we only maintain samples with uncertainty higher than H, ie., the model is trained on samples with more uncertain sensitive attributes. We train the model seven times with different. random seeds for each uncertainty threshold and report fairness and accuracy in the testing set containing sensitive attributes.
9

---

Published in Transactions on Machine Learning Research (09/2024)
Method
Accuracy
App
ΔроP
ABOD
Vanilla (without fairness)
0.681 + 0.011
0.285 + 0.026
0.325 + 0.029
0.325 + 0.029
Vanilla (with fairness)
0.634 = 0.009
0.032 = 0.011
0.089 = 0.024
0.041 = 0.016
Proxy-DNN
0.634 + 0.005
0.049 + 0.008
0.099 + 0.036
0.080 = 0.022
FairRF
0.669 + 0.001
0.289 + 0.003
0.319 + 0.004
0.319 + 0.004
FairDA
0.668 +: 0.019
0.229 + 0.018
0.265 +: 0.024
0.265 + 0.024
CGL
0.612 = 0.019
0.032 - 0.014
0.065 + 0.019
0.065 = 0.027
ARL
0.672 + 0.009
0.290 0.016
0.310 + 0.010
0.320 + 0.010
CVarDRO
0.668 1: 0.008
0.279 0.018
0.300 0.010
0.287 0.015
KSMOTE
0.670 +: 0.012
0.286 0.028
0.321 0.028
0.321 0.028
DRO
0.672 = 0.010
0.282 0.026
0.296 0.017
0,296 0.017
FairDSR (weighted)
0.672 = 0.009
0.027 0.016
0.069 + 0.038
0.083 0.035
FairDSR (uncertain)
0.671 + 0.009
0.272 0.016
0.300 + 0.039
0.300 + 0.034
FairDSR (certain)
0.676 + 0.009
0.085 0.016
0.067 + 0.039
0.074 + 0.034
Table 3: Results on the Compas dataset.
Adult
Compas
Celeba
New Adult
LSAC
0.15
Unfairness
0.30
0.15
0.08
0.10
0.25
0.10
0.04
0.05
0.05
0.05
0.02
0.03
0.00
0.00
0.80
0.69
Accuracy
0.82
0.80
0.88
0.68
0.79
0.88
0.80
0.78
0.67
0.78
0.78
0.76
0.77
0.87
0.00
0.25
0.50
0.00
0.25
0.50
0.00
0.25
0.50
0.00
0.25
0.50
0.00
0.25
0.50
Uncertainty Threshold
Figure 2: Training Random Forest classifiers without fairness constraints using samples with high uncertainty of sensitive attribute predictions. For each uncertainty threshold H, the model trained on samples with uncertainty 2 H. The training is done seven times, and the average fairness (first row) and accuracy (sccond row) reported. Shaded represents the standard deviation.
Figure shows the fairness and accuracy of Random Forest classifier trained without fairness constraints. In the figure, each column represents the results on each dataset, and the first and the second rows provide the plots fairness and accuracy for different uncertainty thresholds, respectively. Across different datasets, the results show that unfairness decreasos as the uncertainty threshold increases. We observe that the improvement in fairness is consistent for different fairness metrics considered, i.e., demographic parity, equal opportunity, and equalized odd. We also observe decrease in the accuracy, which is justified by reduced dataset size and improved fairness (tradcoff). On the LSAC dataset, fairness and accuracy remain almost. constant the average uncertainty in predicting the sensitive attribute on this dataset is 0.66, i.e., most of the samples already have the highest uncertainty. We observed similar results for Logistic Regression classifier presented in the Supplementary (Figure 8 However, this method incurs a higher drop in accuracy and does not necossarily guarantee that an adversary cannot reconstruct the sensitive attributes from the trained model (eerr al. 2023).
5.2.2 Fairness-accuracy tradeoffs.
Table and 3 show the effectiveness of the Fai compared other baselines on the Adult and Compas datasets, respectively. Results for the CelebA and LSAC appear in Appendix (Table and 7 respectively).
10

---

Published in Transactions on Machine Learning Research (09/2024)
(a) Compas dataset
Unfair
Ground truth
Proxy-DNN
Proxy-KNN
-X- CGL
FairDSR (ours)
0.68
0.68
0.68
Accuracy
Accuracy
Accuracy
0.66
0.66
0.64
0.64
0.64
0.62
0.62
0.2
0.05
0.10
0.15
0.20
0.25
0.05
0.10
0.15
0.20
0.25
AGP
Agop
(b) Adult dataset
0.84
Accuracy
Accuracy
Acouracy
0.84
0.62
0.80
0.83
0.83
0.00
0.05
0.10
0,025 0,050 0.075 0.100
0.02
0.04
0.06
0.08
0.10
App
Agco
Figure 3: Accuracy-faire fairness tradeoffs for various fairness metrics (Apr, AEOr, AEOD) and proxy sensitive attributes. The top-left is the best (highest accuracy with the lowest unfairness). Curves are created by sweeping range of fairness coefficients X, taking the median of 7 runs per and computing the Pareto front. The exponentiated gradient. is the fairness mechanism with Random Forests the base classifier. The standard deviations are shaded in the figures.
It is important to note that methods aiming to improve worst-case group accuracy (ARL, DRO, CVarDRO) do not necessarlly improve fairness in terms of demographic party or equalized odds. In particular, results across different datasets show that ARL can improve the Equal Opportunity metric but fails to improve demographic parity. It also yields the most accurate classifier as this method does not have a tradeoff with accuracy. On the other hand, FairDA {Liang et al. 2023) and CGL (Jung et al.) (022), which also exploit limited demographic information, show an improvement in fairness compared to other baselines. However, it incurs a higher drop in accuracy while our method using fairness constraints on samples with reliable sensitive attributes significantly outperforms them across all datascts. In other words, the results show that our method with fairness constraints on samples with reliable sensitive attribules provides Pareto dominant. points in terms of fairness and accuracy.
On the other hand, the variant using model trained without fairness constraints (without using sensitive altributes) provides better fairness-accuracy tradeoffs compared to other baselines on the Adult and the CelebA datasets while providing comparable results on datasets with higher uncertainty (LSAC and Compas). For example, the LSAC dataset has an average uncertainty of 0.66, meaning most samples already have uncertain sensitive information, and the unfairness is already low. As no fairness constraints are enforced in FairDSR (uncertain), it less impacts fairness and accuracy as most data samples are preserved due high overall uncertainty. On the other hand, we observe that FairDSR (weighted) can outperform other baselines and provide better accuracy than FairDSR (certain) at the cost of higher unfairness. The improved accuracy is explained by the use of all data points in the weighted approach. At the same time, FairDSR (certain) can strengthen fairness when only samples with reliable sensitive attributes are used. FairDSR (certain) also provides better Pareto dominant points than CGL and PairDA. While CGL, which also accounts for the uncertainty of the sensitive attributes, can outperform other baselines, especially those directly using predicted proxy-scnsitive attributes. These results support our hypothesis that using data points with certain sensitive attributes is more beneficial to fairness and accuracy. Figure Bhows that imputation methods can yield reasonably good fairness-accuracy tradeoffs on the Adult and Compas datasets but suboptimal
11

---

Published in Transactions on Machine Learning Research (09/2024)
(a) Adult dataset
Clean
Praxy- DNN
+
H=0.1
--
H=0.3
H
-
H=0.6
H=0.7
Demographic Parity
Equal Opportunity
Equalized Odds
0.85
0.85
0.84
Accuracy
Accuracy
Accuracy
0.84
0.84
0.82
0.83
0.80
0.00
0.05
0.10
0.025
0.050
0.075 0.100
0.025
0.050
0.075
0.100
AgoP
Aςоо
(b) Compas dataset
Demographic Parity
Equal Opportunity
Equalized Odds
Accuracy
0.65
Accuracy
Accurapy
0.65
0.65
0.60
0.60
0.60
0.0
0.1
0.2
0.3
0.0
0.1
0.2
0.1
0.2
0.3
App
АçoP
AcoD
Figure 4: The impact of the uncertainty threshold H on the fairness-accuracy tradeoff for (a) Adult and (b) Compas datasets.
compared to the model using the true sensitive attributes. We observed similar results with different base classifiers (logistic regression and cradient-boosted trees) and fairness mechanisms. These results appear Appendix C.3
5.2.3 Ablation experiments
Impact of the uncertainty threshold. Figure showcases the impact of the uncertainty threshold the fairness-accuracy Uhreshold. When the feature space encodes much information about the sensitive attribute as in the Adult dataset (Figure 4a) with 85% accuracy of predicting the sensitive attributes, the results show that the more we enforce fairness w.r.t. samples with the lower uncertainty, the better the fairness-accuracy tradeoffs. In this regime, enforcing unfairness helps the model maintain a better accuracy level (Figure 4p). In contrast, a low bias regime, ie.. when the feature space does not encode enough information about the sensitive attributes, such on the Compas and the New Adult dataset, the model achieves better fairness-accuracy tradeoffs when higher uncertainty threshold is used. In this regime, most of the samples have higher uncertainty in the sensitive attribute prediction, and fairness violation is smaller (see Table as can be observed in Figure 4p, the use of lower uncertainty threshold leads to decrcase in accuracy while fairness is improved. We observe similar results in the New Adult, CelebA, and LSAC datasets (Fig 13 in Supplementary). The accuracy drops since more and more samples were pruned out from the datasets, and this suggests that the feature space is more informative for the target task than the demographic information. In the appendix (D). we show that while under-represented demographic groups can have higher uncerlainty on average than well-represented groups, minority groups are still consistently represented when a lower threshold is used.
Importance of the consistency loss. In the proposed framework, we use trained attribute classifier with a double loss: the cross-entropy loss (supervised loss) and the consistency loss (unsupervised loss). The parameter À controlling the consistency loss updated using a Gaussian ramp-up function starting from zero the beginning of the training. The consistency loss pushes the student to focus on data points with highly certain sensitive attributes. Without this second loss term, the student model minimizes the classification error without explicit consideration of the uncertainty in the dataset. without sensitive attributes.
12

---

Published in Transactions on Machine Learning Research (09/2024)
Clean
Proxy-DNN
FairDSR consistency loss)
FairDSR (wlo consistency loss)
Demographic Parity
Equal Opportunity
Equalized Odds
0.84
Accuracy
Accuracy
0.84
Accuracy
0.84
0.82
0.83
0.83
0.80
0.00
0.05
0.10
0.05
0.10
0.02
0.04
0.06
0.08
0.10
ADP
Figure 5: Consistency loss study the Adult dataset. The predicted sensitive attributes are obtained using our student model with and without consistency loss.
To demonstrate the importance of consistency loss (and thus the importance of self-ensembling with the teacher), experiment with the student model trained without the consistency loss, i.c., set À 0 in Equation! 1 We applied the samne procedure of the second step of FairDSR (certain) to derive data points with sensitive attributes predicted with low uncertainty.
Figure showcases the Pareto front of our FairDSR trained without (w/) and without (w/o) the consistency loss. The base classifier is Random Forest with exponentiated gradient as the fairness mechanism. The Figure also shows the Pareto front of the model trained with the ground truth sensitive attribute and the predicted sensitive attribute (Proxy-DNN). We observe that training the model with the consistency loss provides Pareto dominant points across different fairness metrics. This demonstrates the benefit of the consistency loss in effectively identifying samples with sensitive attributes predicted with low uncertainty in the dataset. without demographics. We also observed that the mean uncertainty for the model without the consistency loss is around 0.19 0.08, while the model that uses the consistency loss is 0.15 = 0.12. This suggests that the consistency loss effectively enforces the student model focus on data points with low uncertainty, Moreover, we also observe that the model without the consistency loss still outperforms the model that directly uses the sensitive attributes (Proxy-DNN) and the model that uses true sensitive attributes. This demonstrates that our hypothesis still holds even if the uncertainty measure is less effective, and a better uncertainty measure can provide even better results.
Experiment with uncertainty measure based confidence intervals. We adopted Monte Carlo dropout because it efficiently captures various sources of uncertainty, both in the data and the model. Another classic approach to measuring uncertainty is the model's prediction confidence. In this experiment, we evaluate the effect using confidence intervals as the uncertainty measure on cairess-accuuaan tradeoffs. Given a trained attribute classifier, recall we obtain the predicted group label by thresholding the prediction probability, i.c., f()) (P = a|X = x) > 0.5).
We construct the confidence interval using threshold 5 [0.5, 1|, such that samples with higher uncertainty have their prediction probability closer to 0.5. A given data point is in the high uncertainty set if 1 < P(Å = a = x) < T, while z is in the low uncertainty set otherwise. More specifically, in this experiment, we derive the subset of data points with low uncertainty of the sensitive attribute prediction as follows
P(Å a|X a|x== x) [0,1 1-7] -],
(x, v. f(x)) E D' if
or P(A== x) [7,
Va E {0.1}
We experiment with different confidence thresholds on the Adult dataset. Specifically, for each confidence threshold (0.6, 0.7, 0.8, 0.9), we trained the Random Forest model using data points with high confidence. Setting the confidence threshold to 0.5 is similar to using all data points to train the model. As shown
13

---

Published in Transactions on Machine Learning Research (09/2024)
Clean
Proxy-DNN
FairDSR
0.9
0=0.8
0.7
r= 0.6
Demographic Parity
Equal Opportunity
Equalized Odds
Accuracy
0,825
Accuracy
0.825
Accuracy
0.825
0.800
0.800
0.800
0.775
0.775
0.775
0.00
0.05
0.10
0.025
0.050
0.075 ДςОР
0.100
0.000
0.025
0.050
0.075
0.100
ApP
Figure 6: Study of uncertainty measure based on confidence interval.
Clean
ProXy-DNN
FairDSR
== 0.05
==0.1
== 0.2
Demographic Parity
Equal Opportunity
Equalized Odds
0.84
0.845
Accuracy
Accuracy
Accuracy
0.84
0.840
0.82
0.835
0.83
0.830
0.00
0.05
0.10
0.025
0.050
0.075
0.100
0.025
0.050
0.075
0.100
App
Figure 7: Study of uncertainty measure based on conformal predictions.
in Figure 6 models trained using confidence intervals as uncertainly measures yield worse Pareto points. Specifically, we observe significant drop in accuracy while fairness performances significantly improve. We also observe that a higher confidence threshold tends to provide better fairness for a similar level of accuracy. While this observation also supports our hypothesis, the drop in accuracy suggests that the confidence interval is less effective in identifying data points with low uncertainty in sensitive attributes. We suspect this results from the unreliability of the model probability due to overconfidence or underconfidence in wrong predictions (Guo et al.) 2017). As the confidence interval is obtained using single model, low-confidence prediction could be solely caused by poor calibration or randomness in the algorithm. We validate this in the next ablation study by calibrating the attribute classifier and using conformal prediction to measure uncertainty.
Ablation on uncertainty mcasure based on conformal predictions. In this experiment, we inves- tigate the validity of our hypothesis under different and reliable uncertainty measures, such as conformal prediction (Shafer & Vovk 2008). Conformal prediction, a machine learning framework, constructs prediction sets containing possible labels. These sets are guaranteed to include the true label with a probability of 1 e, where e [0, 1] user-defined error rate. For instance, sctting c 0.1 ensures the prediction set has least. 90% chance of containing the correct label. The size of the set reflects the model's confidence in its prediction. Ideally, the set would contain only one label, indicating high confidence. Larger sets suggest greater uncertainty about the predicted label. To construct the prediction set of the sensitive attribute, with the 1 - guarantee, we implement split conformal prediction as described by Angelopoulos et al. (2023). which is the widely-used version of conformal prediction. Split conformal prediction takes the pretrained attribute classifier and calibration set to compute the conformal prediction threshold used to include labels in the prediction set of the test set. We employ the Model Agnostic Prediction Interval Estimator (MAPIE) library to train a conformal classifier and measure the uncertainties in the attribute classier (Cordier al. 2023 The classifier wraps the original attribute classifier and produces conformal prediction sets with 4See
Angelopoulos al. 2023 more details on how prediction sets constructed
14

---

Published in Transactions on Machine Learning Research (09/2024)
Sensitive attributes
Coverage e Accuracy
APP
ΔFOP
ΔBOD
Uncertain + Certain
N/A
0.82
0.08
0.07
0.07
0.05
0.84
0.123
0.089
0.118
Certain
0.1
0,83
0.109
0.119
0.100
0.2
0.83
0.110
0.081
0.109
0.05
0.77
0.009
0.012
0.007
Uncertain
0.1
0.77
0.006
0.017
0.009
0.2
0.76
0.003
0.005
0.008
Table 4: Study of models without fairness constraints using different levels of uncertainty measured with conformal predictions.
the guarantced marginal coverage rate c. We used 10% of the dataset with sensitive attributes as the calibration set and to train the calibrated attribute classifier.
In the second step of our framework, generate prediction sets of the sensitive attributes in the dataset where demographic information is missing (Di). Since the sensitive attributes are binary, there are four
possible prediction sets for each data point: {0), {0, 1), {1}, and the emply set {Ø). This means each prediction set for every sample consists of either a single attribute value, both values or no value at all (Gupta et al. 2020). Samples with prodiction sets containing exactly one attribute value are grouped into subsots of samples with low uncertainty (D)). On the other hand, samples with empty prediction sets or sets containing both attribute values indicate higher uncertainty regarding the sensitive attributes. As we used score-based method to compute non-conformity scores, the prediction set for uncertainty predictions might be empty if a set size smaller than 1 is necessary to ensure 1 - coverage (Cordier et al. 2023
In our experiments on the Adult dataset, we varied 6 across values {0.05, 0.1, This means for < 0.05, = 0.1, and = 0.2, the prediction sets are guaranteed to include the true sensitive attribute with probabilities of 95%, 90% and 80%, respectively. Our model training incorporated fairness constraints specifically on data points where the prediction set contained a single attribute value, indicating higher confidence in the prediction for smaller values of c. Following the previous evaluation, we trained the Random Forest classifier for each and sweeping range of fairness coefficients X, taking the median of 7 runs and computing the Pareto front. Figure 7 showcases the Pareto front of different models with fairness constraints on samples with greater certainty of the sensitive attributes (samples with single-valued prediction sets) for different values of c. As can be seen in Fig 7 all models trained using samples with low uncortainty in the sensitive attributes achieve better Pareto front. than the model using all the predicted sensitive values (Proxy-NNN) and even ground truth sensitive attributes (clean). This demonstrates the benefit of applying fairness constraints over data points with low uncortainty in the sensitive on-fairness accuracy tradcoff. Morcover, the figure shows that smaller values of result in better Pareto dominant points, i.e., better fairness-accuracy tradeoff. This is justified by the fact that smaller values of increase the certainty of samples with a single value in their prediction set. Specifically for e = 0.05, the Pareto front is closer to our baseline method using the tropout-based uncertainty measure. We observed a similar trend on other datasets and with different base classifiers. These results also support our hypothesis that imposing fairness constraints on the sample with low uncortainty in the sensitive attribute can better improve fairness and accuracy.
On the other hand, using conformal predictions to measure uncertainty, we also evaluated the variant of our hypothesis where the model is trained without fairness constraints using samples with greater uncertainty in the predicted sensitive attributes. This means we trained the model without fairness constraints but using samples whose prediction sets are emply or contain two values. We also trained models without fairness constraints using samples with greater certainty of the sensitive attributes, ie., samples with single values in their prediction sets. Table shows the fairness and accuracy of Random Forest models trained without. fairness constraints using data with different uncortainty levels on the Adult dataset. More specifically, we considered three cases:
15

---

Published in Transactions on Machine Learning Research (09/2024)
Certain Uncertain: The model is trained using all data points, i.., the data contain samples with certain and uncertain sensitive attributes.
Certain: The model is trained using only samples with greater certainty of the sensitive attribute.
This mcans the training data contains only samples with single the prediction sets.
The model is trained using only samples with greater uncertainty of the sensitive attribute. This means the training data contains only samples whose prediction sets are empty or contain two values.
Uncerlaón:
The results depicted in Table show that, without fairness constraints, the models using samples with greater certainty of sensitive attributes have better accuracy but tend to exacerbate unfairness, while models using data with lower uncertainty significantly improve fairness at the expense of accuracy. More specifically, for = 0.05, the model using only data with uncertain sensitive attributes achieves fairness in terms of demographic parity of 0.009 compared to 0.12 for the model using data with certain sensitive attributes. This corresponds to a 92% improvement in fairness with a smaller drop of 8% for accuracy. This shows models hardly discriminate against samples with uncertain sensitive attributes, and the tradeoffs be controlled by uncertainty in sensitive attribute space. This also supports hypothesis under different and reliable uncertainty measures.
6 Conclusion
In this work, introduced FairDSR, framework to improve the fairness-accuracy tradeoff when only limited demographic information is available. Our method introduces uncertainty awareness in the sensitive attributes classifier. We showed that uncertainty in the attribute classifier plays an important role in the faimess-acccraayy tradcoffs achieved the downstream model with fairness constraints. We showed that enforcing fairness on samples whose sensitive attributes are predicted with low uncertainty can yield models with better fairncss-aceuracy tradcoffs. Subsoquently, we considered different variations around our hypothesis and demonstrated the benefit of uncertainty quantification in the sensitive attribute space for designing fair models under unknown demographic information.
Impact Statement
The proposed framework shows evidence that models hardly discriminate against samples with high uncertainty in sensitive attribute prediction. However, our base model relies on the predicted missing sensitive information. Inferring sensitive information can raise ethical concerns and face legal restrictions, especially when individuals do not choose to disclose their sensitive information. The line of methods relying upon proxy attributes or inferring sensitive attributes faces this limitation (Diana et al. 2022 Awasthi et al., 2021 Coston al.) 2019 Liang al.) 2023). For this reason, we proposed a variant of our model (FairDSR. (uncertain), Sec. 4.2h that derives dataset with higher uncertainty in sensitive attributes without using predicted information.In addition, verifying that group of users is left out when training the model due to the lower uncertainly of their sensitive altributes is important. While we analyzed group representation and showed that groups are still relatively well represented even with a higher uncertainty threshold (Fig. 16 in Supplementary), there are still a risks of disparate impact.
We emphasize that the inference of the sensitive altributes using our proposed method should not be used for any purpose other than bias assessment and mitigation.
Acknowledgments
SEK supported by CIFAR and NSERC DG (2021-4086) and UA by NSERC DG (2022-04006). We thank the anonymous reviewers for the feedback, which has helped improve the paper.
16

---

Published in Transactions on Machine Learning Research (09/2024)
References
Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Ficguth, Xiaochun Cao, Abbas Khosravi, U Rajendra Acharya, et al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion, 76:243-201 297, 2021.
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. A reductions approach to fair classification. In International Conference on Machine Learning, pp. 60-69. PMLR, 2018.
Anastasios N Angelopoulos, Stephen Bates, et al. Conformal prediction: A gentle introduction. Foundations and Trendol in Machine Learning, 16(4):494-55 591, 2023.
Arthur Asuncion and David Newman. Uci machine learning repository, 2007.
Pranjal Awasthi, Alex Beutel, Matthäus Kleindessner, Jamie Morgenstern, and Xuezhi Wang. Evaluating fairness of machine learning models under uncertain and incomplete information. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Trunspareccy, pp. 206-21211 214, 2021.
Pierre Baldi and Peter J Sadowski. Understanding dropout. Advances in neural information processing systems, 26, 2013.
Sarah Bird, Miro Dudik, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. Fairlearn: toolkit for assessing and improving fairness ai. Microsoft, Tech, Rep. MSR- TR-2020-32, 2020.
David P Brown, Caprice Knapp, Kimberly Baker, and Meggen Kaufmann. Using bayesian imputation to assess racial and ethnic disparitics in pediatric performance measures. Health services research, 51(3): 1095- 1108, 2016.
Canyu Chen, Yueqing Liang, Xiongxiao Xu, Shangyu Xie, Yuan Hong, and Kai Shu. On fair classification with mostly private sensitive attributes. arXiv preprint arXin:207. 08336, 2022a.
Canyu Chen, Yueqing Liang, Xiongxiao Xu, Shangyu Xie, Yuan Hong, and Kai Shu. When fairness meets privacy: Fair classification with semi-private sensitive attributes. In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurIPS 2022, 2022b.
Jiahao Chen, Nathan Kallus, Xiaojie Mao, Geoffry Svacha, and Madeleine Udell. Fairness under unawareness: Assessing disparity when protected class unobserved. In Proceedings of the conference on fairness, accountability, and transporency, pp. 339 339-348, 2019.
Thibault Cordier, Vincent Blot, Louis Lacombe, Thomas Morzadec, Arnaud Capitaine, and Nicolas Brunel. Flexible and systematic uncertainty estimation with conformal prediction via the mapie library. In Conformal and Probabilistic Prediction with Applications, pp. 549-581. PMLR, 2023.
Amanda. Coston, Karthikeyan Natesan Ramamurthy, Dennis Wei, Kush R Varshney, Skyler Speakman, Zairah Mustahsan, and Supriyo Chakraborty, Fair transfer learning with missing protected attributes. In Proceedings of the 2019 AAAT/ACM Conference on AI, Ethics, and Society, pp. 91-98, 2019.
Emily Diana, Wesley Gill, Michael Kearns, Krishnaram Kenthapadi, Aaron Roth, and Sacod Sharifi- MalvaiiiMaa Multiaccurate proxies for downstream fairness. In 2022 ACM Conference on Fairness, Accountability, and Transporency, pp. 1207-1201 2022.
Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. Advances in neural information processing systems, 34:6478 6490, 2021.
Sanghamitra Dutta, Dennis Wei, Hazar Yucksel, Pin-Yu Chen, Sijia Liu, and Kush Varshney. there trade- off between fairness and accuracy? perspective using mismatched hypothesis testing. In International PMLR, 2020.
Conference Machine Learning, pp.
17

---

Published in Transactions on Machine Learning Research (09/2024)
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovalions in theoretical computer science conference, pp. 214-226, 2012.
Julien Ferry, Ulrich Aivodji, Sébastien Gambs, Marie-José Huguet, and Mohamed Siala. Exploiting fairness enhance sensitive attributes reconstruction. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 18-41. IEEE, 2023.
Clrik Franke. Rawls's original position and algorithmic fairness. Philosophy 8 Technology, 34(4):1803-18) 2021.
Allen M Fremont, Arlene Bierman, Steve L. Wickstrom, Chloe E, Bird, Mona Shah, José J Escarce, Thomas Horstman, and Thomas Rector. Use of geocoding in managed care settings to identify quality disparities. Health Affairs, 24(2):516-526, 2005.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp.
PMLR, 2016.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp.
PMLR, 2017.
Chirag Gupta, Aleksandr Podkopaev, and Aaditya Ramdas. Distribution-free binary classification: prediction sets, confidence intervals and calibration. Advonces in Neural Information Processing Systems, 33:3711 3723, 2020.
Maya Gupta, Andrew Cotter, Mahdi Milani Fard, and Serena Wang. Proxy fairness. arXiv:1806. 11111180611222. 11212, 2018.
ar preprint
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demo- graphics in repeated loss minimization. International Conference on Machine Learning, pp. 1938. PMLR, 2018.
Muyang He, Shuo Yang, Tiejun Huang, and Bo Zhao. Large-scale dataset pruning with dynamic uncertainty. arXiv preprint arxiv:206.00 051 75, 2023.
Matthew Jagielski, Michael Kearns, Jieming Mao, Alina Oprea, Aaron Roth, Saeed Sharifi-Mard Malvajerdi, and Jonathan Ullman. Differentially private fair learning. In International Conference on Machine Learning, pp. 3000 3008. PMLR, 2019.
Larson Jeff, Mattu Surya, Kirchner Lauren, and Angwin Julia. How we analyzed the compas recidivism algorithm, 2016.
Sangwon Jung, Sanghyuk Chun, and Taesup Moon. Learning fair classifiers with partially annotated group labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 10348 10357, 2022.
Faisal Kamiran and Toon Calders. Data preproccssing techniques for classification without discrimination. Knowledge and information systems, (3:11 33, 2012.
Patrik Joslin Kenfack, Adil Mehmood Khan, SM Ahsan Kazmi, Rasheed Hussain, Alma Oracevic, and Asad Masood Khattak. Impact of model ensemble the fairness of classifiers in machine learning. Tn 2021 International conference on applied artificial intelligence (ICAPAI), pp. 1-6. IEEE, 2021.
Patrik Joslin Kenfack, Adín Ramírez Rivera, Adil Khan, and Manuel Mazzara. Learning fair representations through uniformly distributed sensitive attributes. In First IEEE Conference on Secure and Trustworthy Machine Learning, 2023.
18

---

Published in Transactions on Machine Learning Research (09/2024)
Niki Kilbertus, Adrià Gascón, Matt Kusner, Michael Veale, Krishna Gummadi, and Adrian Weller. Blind justice: Fairness with encrypted sensitive attributes. In International Conference on Machine Learning, pp- 2630 2639. PMLR, 2018.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. Xiv preprint 0rXiu.142.69980 2014.
Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographics through adversarially reweighted learning. Advances in neural information processing systems, 33:728 740, 2020.
Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. In Internalional Conference on Learning Representations, 2017.
Daniel Levy, Yair Carmon, John C Duchi, and Aaron Sidford. Large-scale methods for distributionally robust. optimization. Aduances in Newral Information Processing Systems, 33:8847-8860, 2020.
Yueqing Liang, Canyu Chen, Tian Tian, and Kai Shu. Fair classification via domain adaptation: A dual adversarial learning approach. Frontiers in Big Data, 5:129, 2023.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781-6799 PMLR, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Large-scale celebfaces attributes (celeba) dataset. Retrieved August, 15(0018):11, 2018.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
John Rawls. A theory of justice: Revised edition. Harvard university press, 2020.
Glenn Shafer and Vladimir Vovk. tutorial on conformal prediction. Journal of Machine Learning Research, 9(3), 2008.
Gabriella C Silva, Amal N Trivedi, and Roee Gutman. Developing and evaluating methods to impute race/ethnicity in an incomplete dataset. Health Services and Outcomes Research Methodology, 19(2-3): 175-195, 2019.
Nitish Srivastava, GeofTrey Hinton, Alex Krizhevsky, Πya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1): 1929-1958, 2014.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-avcraged consistency (argets improve semi-supervised deep learning results. Advances in neural informalion processing systems, 30, 2017.
Michael Veale and Reuben Binns. Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data & Society, 422):063355777743333 2017.
Serena Wang, Wenshuo Guo, Harikrishna Narasimban, Andrew Cotter, Maya Gupta, and Michael Jordan. Robust optimization for fairness with noisy protected groups. Advonces in Neural Information Processing Systems, 14:5190-5203, 2020.
Linda F Wightman. Lsac national longitudinal bar passage study. Isac research report series. 1998.
Shen Yan, Hsien-te Kao, and Emilio Ferrara. Fair class balancing: Enhancing model fairness without observing sensitive attributes. In Proceedings of the 29th ACM International Conference on Information Knowledge Management, pp. 1715 1724, 2020.
19

---

Published in Transactions on Machine Learning Research (09/2024)
Lequan Yu, Shujun Wang, Xiaomeng Li, Chi-Wing Fu, and Pheng-Ami Heng. Uncertainty-aware self- ensembling model for semi-supervised 3d left atrium segmentation. In Medical Image Compuling and Computer Assisted Intervention MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part II 22, pp. 605-613 Springer, 2019.
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International conference on machine learning, pp. 325 333. PMLR, 2013.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAT/ACM Conference AT, Ethics, and Sociely, pp- 335-340, 2018.
Yan Zhang. Assessing fair lending risks using pace/ethmicis proxies. Management Science, 64(1):178 197, 2018.
Tianxiang Zhao, Enyan Dai, Kai Shu, and Suhang Wang. Towards fair classifiers without sensitive altributes: Exploring biases in related features. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 1433 1442, 2022.
20

---

Published in Transactions on Machine Learning Research (09/2024)
Supplementary Material
A Limitations
Our method is evaluated mainly on one fairness-enhancing algorithm (i.e., Exponentiated Gradient). It will be interesting to explore if our hypothesis applies to pre-processing and post-processing techniques and with different fairness-enhancing algorithms. Finally, assumption that the true sensitive attributes are available in the test dataset for fairness evaluation might not be true in several practical scenarios. This might require evaluation using proxy-sensitive altributes. These proxies are likely noisy and might require evaluations using bias assessment methods that effectively quantify fairness violation w..tt to true sensitive attribute (Chen et al. 2019: Awasthi al. 2021). On the other hand, will be interesting to back our empirical findings with thcoretical results by characterizing fairness violation bounds based on the uncertainty quantification of the sensitive altribute predictions.
B Fairness Metrics & Datasets
In the main paper, considered the three popular group fairness metrics: demographic parity, equalized odds, and equal opportunity. These metries aim to equalize different models' performances across different demographic groups. Samples belong to the same demographic group if they share the same demographic information, A, e.g., gender and race.
B.1 Fairness Metrics
Demographic Parity Also known statistical parity, this measure requires that the positive prediction
(f(X) 1) the model be the same regardless the demographic group to which the user belongs (Dwrk et al. 2012). More formally the classifier achieves demographic parity if P(f()) = 1|A = a) = P((()) - 1)
In other words, the model's outcome should be independent of sensitive attributes. practice, this metric is
measured follows:
APP (f)
E [[f()))) a 1l
131 x|A=1
E 1lf()) 13]
Where I[(-) is the indicator function.
This metric enforces the True Positive Rate (TPR) and False Positive Rate (FPR) for different demographic groups be the same P(f(X) 1 0, y) P(f(X) 1/A 1,Y v). vy e (0, 1). The metric is measured follows:
Equalized Odds
A og +
(8)
Where,
E [ff(x) aj =
1}}
E [[f()) = 1H =|A=1,Y=j
(9)
Equal Opportunity In certain situations, bias in positive outcomes can be more harmful. Therefore, the Equal Opportunity metric enforces the same TPR across demographic (Hardt et al. 2016) and measured using on (Eq. 9
B.2 Datasets
In the Adult dataset, the task is to predict an individual's annual income greater or less than $50k per year. We also considered the recent version of the Adult dataset (New Adult) for 2018 across different states in US (Ding et al., 2021). For the Compas dataset, the task to predict whether a defendant will recidivate within two years. The LSAC dataset contains admission records to law school. The task to predict whether
21

---

Published in Transactions on Machine Learning Research (09/2024)
Dataset
Size
#features
Domain
Sensitive
attribute
Adult Income
18,812
15
Finance
Gender
Compas
6,000
11
Criminal justice
Race
LSAC
20,798
12
Education
Gender
New Adult
1.6M
10
Finance
Gender
CelebA
202,600
40
Image
Gender
Table 5: Datasets
candidate will pass the bar exam. The Celeb. dataset contains 40 facial attributes of humaine annotated images. We consider the task of predicting allractiveness using gender as sensitive attribute. We randomly sample 20% of the data to train the sensitive attribute classifier (D2). All the input features used to train the attribute classifier except for the target variable. Table 5 provides more details about cach dataset and sensitive attributes used.
Demographic Parity
Equal Opportunity
Equalized Odds
Accuracy
Adult
Compas
Celeba
New Adult
LSAC
Unfairness
0.30
0.35
0.06
0.08
0.05
0.30
0.04
0.10
0.02
0.25
0.00
0.00
Accuracy
0.68
0.78
0.80
0.84
0.67
0.78
0.77
0.80
0.79
0.66
0.76
0.25
0.00
0.25
Uncertainty Threshold
Figure 8: Training Logistic Regression without fairness constraints using samples with high uncertainty of sensitive attribute predictions. For each uncertainty threshold H, the model is trained on samples with uncertainty 2 Н. The training is done seven times, and the average fairness (first row) and accuracy (second row) are reported. Shadod represents the standard deviation.
C Additional Results
C.1 Impact of the group-labeled ratio.
all the previous experiments, we considered the ratio of the group-labeled dataset (D2) as 20% of the original datasct. In many real-world scenarios, data labeling costly, especially given the rising concerns surrounding privacy. As these concerns intensify, fewer users may be inclined to disclose their sensitive attributes. It is, therefore, important to study the impact of the group-labeled ratio the performance of our method. In this ablation study, consider the attribute classifier trained with different ratios of D2, i.e., 3%, 5%, 10%, 15%, and 20% of the original dataset. We experiment on the Adult, wherein 3% represents around 1,465 data points among the 48,842 data points available.
Shttp://mmaabbie.co www.k...kkkee......... www.eecw......cceeeeeeee... hwt..ehhhe////oeeet/e/eeaaaaaaa
22

---

Published in Transactions on Machine Learning Research (09/2024)
Proxy-DNN
[22]
[
5%
[21
10%
722
15%
|Pa 20%
Demographic Parity
Equal Opportunity
Equalized Odds
0.84
0.845
Accuracy
Accuracy
Accurapy
0.84
0.840
0.82
0.835
0.00
0.05
0.10
0.025
0.050
0.075
0.100
0.02
0.04
0.06
0.08
0.10
AD
Figure 9: Study of group-labeled ratio. The sensitive attribute classifier is trained using different ratios of the dataset with sensitive attributes (D2).
Figure 9 shows the Pareto front for different group-labeled ratios. Our findings remain consistent even in the most demographic scare regime. While we observe a slight drop in accuracy for smaller group-labeled ratios, the fairness performances are maintained. The Pareto fronts still dominate the model directly using the predicted sensitive attributes. These results suggest that our hypothesis holds even when very limited demographic information available, long as the dataset is predictive for sensitive attributes. On the other hand, even if the dataset is predictive enough for the sensitive attributes, results show that training model without fairness constraints will exhibit less disparities across the unknown demographic groups.
C.2 Results Nonlinear Classifier.
Table f compares with other baselines on the CelebA dataset with logistic regression the base classifier. In the main paper, compared existing methods using Logistic Regression as the base classifier. We performed experiments with more complex non-linear model to analyze its impact on the performance of different methods. We considered a Multi-Layer Perceptron (MLP) with one hidden layer of 32 units and Relu as the activation function for all the baselines. On the Adult dataset, Table S shows that when using a more complex model, our method (FairDSR (certain)) still provides Pareto dominants points terms of fairness and accuracy compared other baselines. At the same time, we observed an improvement in the accuracy of other methods due to the increased model capacity.
Method
Accuracy
App
ΔgOP
ÁBOD
Vanilla (without fairness)
0.803 + 0.002
0.176 + 0.010
0.183 + 0.015
0.183 + 0.015
Vanilla (with fairness)
0.782 +'t 0.001
0.008 + 0.005
0.018 + 0.014
0.017 + 0.014
Proxy-DNN
0.800 + 0.001
0.008 + 0.036
0.011 + 0.022
0.036 + 0.011
FairDA
0.802 0.002
0.155 + 0.010
0.165 + 0.018
0.165 + 0.018
CGL
0.782 + 0.002
0.008 + 0.005
0.019 + 0.011
0.020 + 0.009
ARL
0.803 + 0.002
0.157 + 0.010
0.157 +: 0.010
0.166 + 0.016
CVarDRO
0.781 0.002
0.155 + 0.010
0.162 + 0.016
0.162 + 0.016
KSMOTE
0.773 0.008
0.020 + 0.067
0.110 + 0.082
0.144 + 0.068
DRO
0.796 + 0.006
0.142 + 0.020
0.152 + 0.020
0.129 + 0.028
FairDSR (weighted)
0.799 + 0.004
0.038 + 0.010
0.020 + 0.012
0.033 + 0.013
FairDSR (uncertain)
0.782 + 0.004
0.071 0.046
0.107 + 0.033
0.107 + 0.033
FairDSR (certain)
0.793 0.000
0.003 0.000
0.001 + 0.001
0.007 0.002
Table 6: Results on the CelebA dataset.
23

---

Published in Transactions on Machine Learning Research (09/2024)
Method
Accuracy
ADP
ÁFOP
ÁFOD
Vanilla (without fairness)
0.793 0.007
0.014 0.005
0.005 + 0.005
0.049 + 0.026
Vanilla (with fairness)
0.796 0.009
0.004 = 0.004
0.002 + 0.001
0.025 = 0.016
Proxy-DNN
0.792 + 0.009
0.018 + 0.009
0.004 + 0.001
0.040 + 0.005
FairRF
0.753 = 0.120
0.021 = 0.013
0.016 = 0.017
0.044 = 0.015
FairDA
0.716 0.210
0.001 0.000
0.000 0.005
0.003 = 0.004
CGL
0.765 = 0.022
0.005 = 0.004
0.003 + 0.001
0.035 = 0.017
ARL
0.807 + 0.024
0.014 + 0.015
0.009 + 0.014
0.037 + 0.022
CVarDRO
0.776 = 0.052
0.024 = 0.010
0.019 = 0.014
0.045 = 0.015
KSMOTE
0.655 0.055
0.022 0.034
0.030 + 0.022
0.060 + 0.018
DRO
0.580 = 0.220
0.023 = 0.014
0.021 + 0.017
0.038 = 0.020
FairDSR (weighted)
0.806 + 0.001
0.008 - 0.005
0.004 + 0.002
0.035 = 0.018
FairDSR (uncertain)
0.794 + 0.001
0.015 = 0.002
0.006 = 0.001
0.055 = 0.000
FairDSR (certain)
0.805 0.001
0.001 0.002
0.000 = 0.001
0.002 0.000
Table 7: Results on the LSAC dataset.
Method
Accuracy
ADP
ΔFOP
ÁFOD
Vanilla (without fairness)
0.853 0.004
0.183
0.019
0.100 + 0.025
0.102 + 0.023
Vanilla (with fairness)
0.801 0.009
0.006 = 0.004
0.049 + 0.011
0.017 = 0.007
Proxy-DNN
0.795 + 0.009
0.029 + 0.004
0.067 + 0.020
0.042 + 0.011
FairRF
0.853 = 0.002
0.164 = 0.009
0.077 = 0.026
0.091 = 0.013
FairDA
0.813 0.014
0.118 0.023
0.091 + 0.050
0.099 + 0.037
CGL
0.800 = 0.014
0.009 = 0.002
0.017 = 0.021
0.032 + 0.010
ARL
0.851 + 0.003
0.166 + 0.015
0.087 + 0.019
0.090 + 0.016
CVarDRO
0.850 + 0.003
0.183 + 0.018
0.095 + 0.027
0.101 + 0.026
KSMOTE
0.814 0.020
0.201 0.055
0.120 + 0.021
0.130 = 0.023
DRO
0.837 + 0.016
0.232 + 0.057
0.110 + 0.057
0.140 + 0.045
FairDSR (weighted)
0.849 + 0.027
0.018 + 0.010
0.061 - 0.034
0.047 = 0.032
FairDSR (uncertain)
0.801 + 0.027
0.110 + 0.022
0.067 + 0.027
0.059 + 0.024
FairDSR (certain)
0.818 0.004
0.009 0.008
0.028 + 0.020
0.027 = 0.017
Table 8: Results on the Adult dataset using an MLP with hidden layer of 34 units as base classifier.
C.3
Comparison with Other Baselines
To assess the effect of the attribute classifier over the performances of downstream classifiers with fairness constraints w.r.t the proxy, we also performed extensive comparisons with different methods of obtaining the missing sensitive attributes:
Ground truth sensitive attribute. We considered the case where the sensitive attribute is fully available and trained the model with fairnoss constraints w.r.t the ground truth. This represents the ideal situation where all the assumptions about the availability of demographic information are satisfied. This baseline expected achieve the best trade-offs.
Proxy-KNN. Here the missing sensitive attributes are handled by data imputation using the k-nearest neighbors (KNN) of samples with missing sensitive attributes.
Proxy-DNN. For this baseline, an MLP is trained on D2 predict the sensitive attributes without uncertainty awareness. The network architecture and the hyperparameter are the same as for the student in our model.
24

---

Published in Transactions on Machine Learning Research (09/2024)
CGL (Jung al. 2022 Similar to setup, this method also uses attribute classifier and replaces uncertain predictions with random sampling from the empirical conditional distribution of the sensitive attributes given the non-sensitive attributes P).
(a) Compas
Unfair
Ground truth
PNoXy-DNN
Proxy-KNN
CGL
FairDSR (ours)
0.68
0.68
0.68
Accuracy
Accuracy
Accuracy
0.66
0.66
0.66
0.64
0.62
0.62
0.15
0.10
0.15
0.20
App
Δςоо
(b) Adult
0.84
Accuracy
Accuracy
0.84
Accuracy
0.84
0.82
0.83
0.83
0.82
0.00
0.05
0.10
0.025
0,050
0.075 0.100
0.02
0.04
0.06
0.08
0.10
ADP
Δςоо
(c) LSAC
0.875
0.875
Accuracy
0.87
Accurapy
0.870
Accuracy
0.870
0.865
0.86
0.860
0.885
0.00
0.01
0.02
0.002
0.004
0.006
0.008
0.02
0.03
0.04
App
АEОP
Δςоо
(d) CelebA
0.810
0.810
0.810
Accuracy
0.805
Accuracy
0.808
Accuracy
0.805
0.800
0.806
0.800
0.795
0.804
0.795
0.00
0.05
0.10
0.15
0.10
App
AgaP
Agap
Figure 10: Accuracy-fairness trade-offs for various fairness metrics (Apr, ABOP, ΔBOD) and proxy sensitive attributes. The top-left is the best (highest accuracy with the lowest unfairness). Curves are created by sweeping range of fairness coefficients X, taking the median of 7 runs per and computing the Pareto front. The exponentiated gradient is the fairness mechanism with Random Forests as the base classifier. The standard deviations are shaded in the figures.
For fairness-enhancing mechanisms, we considered the Fairlean BBird et al. 2020) implementation of the exponentiated gradient [Agawwal et al. 2018) and adversarial debiasing (Zhang et. al. 2018) (Section 3). We
25

---

Published in Transactions on Machine Learning Research (09/2024)
used various base classifiers for the exponentiated gradient, including logistic regression, random forest, and gradient-boosted trees. Random forest was initialized with a maximum depth of 5 and minimum sample leaf of 10, and default parameters were used for logistic regression without hyperparameter tuning. The same models and hyperparamoters were used across all the datascts. Adversarial debiasing works for denographic parity and equalized odds. The architecture of the classifier, the adversary, and other hyperparamettes used is the same as recommended by the original paper (Zhang et al., 2018). We evaluate the aairuesy-acuuracy trade-off of every baseline by analyzing the accuracy achieved in different fairness regimes, i.e,, by varying the parameter é [o, 1]. controlling the balance between fairness and accuracy. For value of close to O, the label classifier is enforced to achieve higher accuracy, while for a value close to 1, it is encouraged to achieve lower unfairness. For each value of e, we trained each baseline 7 times on a random subset of D1 (70%) using their predicted sensitive altributes, and the accuracy and fairness are measured on the remaining subset (30%), where we assumed that the joint distribution XXYY,A) is available for fairness evaluation. The results are avcraged, and the Parcto front is computed.
Figure 10 shows the Pareto front of the exponentiated gradient method on the Adult, Compas, and LSAC datasets using Random Forests the base classifier. The figure shows the trade-off between fairness and accuracy for the different methods of inferring the missing sensitive attributes. From the results, we observe on all datasets and across all fairness metrics that data imputation can be an effective strategy for handling missing sensitive attributes, i.e., this fairness mechanism can efficiently improve the model's fairness with respect to the true sensitive attributes, although fairness constraints were enforced on proxy-sensitive attributes. However, we observe a difference in the bainesss-accurae trade-off for each attribute classifier. The KNN-based attribute classifier has the worst fairness-accurace trade-off on all datasets and fairness metrics. This shows that assigning sensitive attributes based on the nearest neighbors does not produce sensitive attribules useful for achieving trade-off close to the ground truth. While the DNN-based attribute classifier produces a better trade-off, it is still suboptimal compared to the ground truth-sensitive attributes. We observed similar results with different baseline models, such as logistic regression and gradient-booted trees, and for adversarial debiasing as the fairness mechanism. In contrast, our method consistently achieves a better trade-off on all datasets and the fairness met considered. Similar results are obtained on the exponentiated gradient with logistic regression and gradient- -bossedd trees as base classifiers and adversarial debiasing (sce Section G The uncertainty threshold's choice depends on the datasct's bias level, i.c., the level of information about the sensilive altribute encoded in the feature space.
Figures d), Il) and 12 d), depict the Pareto front of various baselines on the CelebA dataset. It shows that models trained with imputed sensitive attributes via KNN consistently achieve comparable tradcoffs to models trained with fairness const raints based on the true sensitive attribute. This could be explained by the fact that gender clusters are perfectly defined in the latent space. We observed that KNN-based imputation achieved 95% accuracy in predicting gender. Conversely, the figures illustrate that our method outperforms baselines using ground truth-sensitive attributes and imputation methods, yielding more Pareto-dominant points. This highlights the advantages of applying fairness constraints to samples with low uncertainty in the sensitive attributes. Furthermore, Figure 14 (c) shows decrcasing the uncortainty threshold further improves fairness while preserving the accuracy. We note the CelebA dataset can raise ethical concerns and is used only for evaluation purposes. For instance, predicting the attractiveness of a photo using other facial attributes as sensitive attributes can still harm individuals even if the model predicting attractiveness is not biased.
Exponentiated gradient with different baseline classifiers Figure 1 and 12 show fairness-accuracy tradc-offs achieved by the exponentiated gradient with logistic regression and gradient-boosted troes, re- spectively. Similar to the results presented in the main paper, our method achieves better fairness-accuracy trade-offs.
Figure 14 shows the accuracy-fairness trade-off Exponentiated gradient using gradient-boostee trees as the base classifier for various uncertainty thresholds, the true sensitive attributes, and the predicted sensitive attributes with DNN. The results obtained are similar to random forests as the base classifier. The smaller uncortainty threshold produced the best trado-off in a high-bias regime like the Adult dataset. On datascts that do not encode much information about the sensitive attributes (most samples have high uncertainty), such as the New Adult and LSAC datasets, the accuracy decreases as the uncertainty threshold reduces
26

---

Published in Transactions on Machine Learning Research (09/2024)
(a) Compas
Unfair
Ground buth
Proxy-DNN
ProXy-KNN
'K- CGL
FairDSR (ours)
0.70
0.70
0.68
0.675
Accuracy
Accuracyy
Accuracy
0.68
0.650
0.64
0.625
0.80
0.62
0.55
0.1
0.1
0.1
App
ASOp
(b) Adult
0.854
0.85
0.85
Accuracy
Accuracy
Accuracy
0.84
0.850
0.84
0.83
0.848
0.82
0.846
0.00
0.05
0.10
0.15
0.10
0.15
0.05
0.10
0.15
ApP
(c) LSAC
0.83
0.9
0.82
0.82
Accuracy
0.8
Accuracy
Accuracy
0.81
0.7
0.80
0.80
0.6
0.5
0.78
0.00
0.01
0.02
0.03
0.000
0.005
0.010
0.015
0.00
0.02
0.04
0.06
APP
АEOP
AgOD
(c) CelebA
0.805
0.804
0.80
Acouracy
Accuracy
Accuracy
0.800
0.802
0.79
0.795
0.800
0.798
0.78
0.00
0.05
0.10
0.15
0.00
0.05
0.10
0.15
0.00
0.05
0.10
0.15
AnP
Figure 11: Socuracy-fainesse trade-offs for various fairness metrics (ApP, ΔOPP, ABOD) and proxy- sensiivee attributes. Top-left is the best (Highest accuracy with the lowest unfairness). The fairness mechanism is the Exponentiated gradient with logistic regression as the base classifier on the Compas (a), Adult (b), LSAC (c), and CelebA (c)aatases. The standard deviation is shaded the figure.
while fairness is improved or maintained. On the LSAC dataset (Figure 14(d)): observe that increasing the uncertainty threshold results in a much higher drop in accuracy. This explained by the high average uncertainty (0.66), and using a smaller threshold removes most of the data.
Experiments with adversarial debaising Figure 15 shows the trade-offs for adversarial debiasing. Our methods achieve a better trade-off on the Adult datasets, while for the Compas dataset, the ground-truth sensitive achieves better trade-off. It. is worth noting that adversarial debiasing is unstable to train.
27

---

Published in Transactions on Machine Learning Research (09/2024)
(a) Compas
Unfair
Ground truth
Proxy-DNN
Proxy-KNN
"M CGL
FaiDDSR (ours)
0.68
0.68
0.675
Accuracy
0.66
Hie Mayt Mepee
Accuracy
0.66
Accuracy
0.650
0.64
0.84
0.62
0.625
0.62
0.0
0.1
0.2
0.05
0.10
0.15
0.20
0.25
0.05
0.10
0.15
0.20
0.25
ApP
Аςоо
(b) Adult
0.83
0.830
0.830
0.82
Accuracy
Accuracy
0.825
Accuracy
0.825
0.81
0.820
0.820
0.815
0.815
0.810
0.810
0.01
0.02
0.03
0.04
0.02
0.03
0.04
0.025
0.050
0.075
0.100
App
AEOD
(c) LSAC
0.878
0,876
0.876
Accuracy
Accuracy
Accuracy
0.876
0.874
0.874
0.874
0.872
0.872
0.872
0.870
0.870
0.870
0.010
0.015
0.020
0.025
0.005
0.006
0.007
0.04
0.05
ADP
Δςоо
(d) CelebA
0.810
0.810
0.810
Accuracy
0.805
Apouracy
0.808
Accuracy
0.805
0.806
0.800
0.800
0.804
0.795
0.795
0.790
0.00
0.05
0.10
0.15
0.05
0.10
0.15
0.05
0.10
0.15
ADP
AsgP
Figure 12: Accuras tradeofls for various fairness metrics (App, ΔBOP, ABOD) and proxy sensitive gradient-boosted
attributes. The fairness mechanism used is the Exponentiated gradient with gradient- boosted trees as the base classifier on the Compas (a) Adult (b), (c) LSAC, and (d) CclobA datascts. The standard deviation is shaded in the figure.
Uncertainty Estimation of Different Demographic Groups
In the main paper, we showed that when the dalaset does not encode enough information about the sensitive attributes, the attribute classifier has, on average, greater uncertainty the predictions sensitive attributes. This encourages a choice of a higher uncertainty threshold keep enough samples maintain the accuracy, i.c., to prune only the most uncertain samples. Figure 16 shows that the gap between demographic groups increase smaller uncertainty threshold is used. This explained by the fact that the model is more
28

---

Published in Transactions on Machine Learning Research (09/2024)
confident about samples from well-represented groups than samples from under-represennted groups. While this gap between demographic groups can increase, our results show there are still enough samples from the disadvantaged group with reliable sensitive attributes. Thus, tuning the uncertainty threshold can result in a model that achieves a better trade-off between accuracy and various fairness metrics. Note that we observed the same trend for the LSAC dataset. The average uncertainty is 0.66, and the minimum uncertainty is 0.62. We also observed that group representation remains consistent (35% difference) when using the average uncertainty value.
29

---

Published in Transactions on Machine Learning Research (09/2024)
(a) Adult
Clean
Proxy-DNN
H=0.1
H=0.3
--
H=0.5
-
H=0.6
H=0.7
Demographic Parity
Equal Opportunity
Equalized Odds
0.85
0.85
0.84
Accuracy
Acouracy
Accuracy
0.84
0.84
0.82
0.83
0.80
0.00
0.025
0,050
0.075 0.100
0.025
0.050
0.075
0.100
App
AgOP
(b) Compas
remographic Parity
Equal Opportunity
Equalized Odds
Accuracy
0.65
Accuracy
Accuracy
0.65
0.65
0.60
0.60
0.60
0.3
App
ASOD
(c) New Adult
Demographic Parity
Equal Opportunity
Equalized Odds
0.80
0.8
0.80
Side de
Accuracy
0.75
Accuracy
Apcuracy
0.75
0.7
0.70
0.65
0.65
0.6
0.00
0.05
0.10
0.00
0.05
0.10
0.15
0.025
0.050
0.075
0.100
AOP
(c) CelebA
Demographic Parity
Equal Opportunity
Equalized Odds
0.808
Accuracy
0.805
Accuracy
Accuracy
0.805
0.806
0.800
0.800
0.804
0.795
0.00
0.05
0.10
0.00
0.02
0.04
0.06
0.08
0.025
0.050
0.075
0.100
0.125
App
Asoo
(d) LSAC
Clean
PNoXy-DNN
- H=0.61
- Hm0.62
- H=0.66
-
H=0.661
-
H=0.662
Demographic Parity
Equal Opportunity
Equa lzzd Odds
Accuracy
0.8
Accuracy
0.8
Accuracy
0.8
0.6
0.6
0.6
0.4
0.4
0.000
0.005
0.010
0.015
0.000
0.005
0.010
0.00
0.01
0.02
0.03
Agp
AEOP
Agoo
Figure 13: The impact of the uncertainty threshold on the fairness-accurant trado-off. For the exponentiatod gradient with Random Forest. as the base classifier for the (a) Adult, (b) Compas, (c) New Adult, (c) CelebA dataset, and (d) LSAC datasets.
30

---

Published in Transactions on Machine Learning Research (09/2024)
(a) Adult
Clean
Proxy-DNN
H=0.1
H=0.3
--
H=0.5
H=0.6
-
H=0.7
Demographic Parity
Equal Opportunity
Equalized Odds
0.84
Accuracy
Accuracy
0.83
Accuracy
0.83
0.82
0.82
0.82
0.81
0.81
0.81
0.00
0.05
0.10
0.10
0.15
App
ΑgOP
(b) Compas
Demographic Parity
Equal Opportunity
Equalized Odds
Accuracy
Accuracy
0.65
Accuracy
0.65
0.60
0.60
0.60
App
Agop
Agod
(c) New Adult
Demographic Parity
Equal Opportunity
Equalized Odds
0.75
0.75
0.75
Accuracy
Accuracy
0.70
Accuracy
0.70
Spacwin
0.65
0.65
1
0.65
0.60
0.00
0.01
0.02
0.03
0.00
0.05
0.10
0.00
0.05
0.10
App
Δgоо
(c) CelebA
Demographic Parity
Equal Opportunity
Equa ized Odds
0.810
0.81
0.808
Accuracy
0.805
Accuracy
Accuracy
0.80
0.800
0.804
0.795
0.802
0.79
0.00
0.05
0.10
0.15
0.00
0.05
0.10
0.15
0.05
0.10
ApP
Agoo
(d) LSAC
Clean
Proxy-DNN
H=0.61
H=0.62
H=0.66
H=0.061
H=0.662
Demographic Parity
Equal Opportunity
Equalized Odds
+
0.8
0.8
0.8
Accuracy
Accuracy
Accuracy
0.6
0.6
0.4
0.4
0.4
0.03
0.06
ApP
Agop
Figure 14: Exponentiated gradient with gradient-boosted trees as the base classifier. The impact of the uncertainty threshold H on the fairness-accuracy trade-off for the (a) Adult, (b) Compas, (c) New Adult, (c) ColobA dataset, and (d) LSAC datasets.
31

---

Published in Transactions on Machine Learning Research (09/2024)
(a)
Adult datasct
Unfair
Ground truth
Proxy-DNN
Proxy-KNN
FairDSR
0.855
Accuracy
0.84
Accuracy
0.850
0.845
0.82
0.840
0.80
0.835
0.00
0.05
0.10
0.15
0.050
0.075
0.100
ADP
(b) Compas dataset
0.70
0.70
0.65
0.65
Accuracy
Accuracy
0.60
0.60
0.55
0.55
0.50
0.50
0.1
0.2
0.3
0.1
0.2
0.3
ADP
ΔEOD
Figure 15: Adversarial debiasing. Accuracy-Gairness trade-ofls for various fairness metrics (ADp, AFOP) and proxy-sensitive attributes.
New Adult
Disparity between demographic groups
80
Adult
Compas
Celeba
60
40
20
0.7
0.5
0.2
Uncertainty threshold
Figure 16: Demographic group representation each dataset for different uncertainty thresholds. The gap between groups increases the threshold becomes smaller. The plot reveals there are samples from the minority group that exhibit lower uncertainty in the prediction of their sensitive attributes.
32


Reviews:
[{"review_id": "FUrYyphTnu", "reviewer": "Reviewer_sdai", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- This paper proposes an intuitive and interesting approach to handle fairness where demographic data is incomplete or unavailable.\n\n- The use of uncertainty-aware models and self-ensembling techniques is well-founded and effectively improves the reliability of demographic attribute predictions. The validation on multiple datasets and ablation studies strengthens the effectiveness of the proposed method.\n \n- The exploration of different variants of the method, such as the weighted approach and training without fairness constraints is interesting.\n\n\nWeaknesses:\n- The effectiveness of the method heavily relies on the accuracy and reliability of the uncertainty measure. The performance of the framework might degrade if these measure are not well-calibrated. \n\n- The paper primarily focuses on empirical results. That would be great if sufficient theoretical support could be provided for the observed improvements.(but I know this may be out of the scope of this paper.)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel framework called FairDSR to enhance fairness in machine learning models when demographic information is incomplete or unavailable. The key contributions include:\n\n-\tUncertainty-Aware Attribute Classifier: An approach using self-ensembling and Monte Carlo dropout to reliably predict demographic attributes with quantified uncertainty.\n-\tFairness Constraints: Enforcing fairness constraints only on samples with low uncertainty in their predicted demographic attributes.\n-\tDemonstrating the framework's effectiveness on five datasets, outperforming traditional methods.\n-\tExploring different methods such as weighted fairness constraints and training on high-uncertainty samples without fairness constraints.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the weaknesses mentioned above.", "broader_impact_concerns": "The authors have already discussed the ethical impact of the paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4HReQN2kml", "reviewer": "Reviewer_JCP5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n-\tThis paper clearly states a main hypothesis and answers this hypothesis with supporting results. In general, this paper is well written and easy to follow. There are some minor typos and suggestions I mention in Requested Changes. The takeaway message of this work is clear.\n-\tAlthough I am not entirely familiar with all baselines presented (nor would I be able to point to any other baselines that should be considered), the empirical results section contains comparisons to many other existing methods. The ablation study suggests that the demonstrated gain in performance is not due to some hyperparameter-hacking and makes the presented results trustworthy. Despite not going through the provided code extensively, I am confident in the reproducibility of the results in this paper.\n-\tI have no concerns regarding correctness, as no novel theory is proposed.\nWeaknesses:\n-\tThe technical novelty is rather limited. The framework combines a semi-supervised learning technique with MC dropout. Technical novelty is limited to applying this combination to a task that previously had not been considered. The uncertainty that the technique captures is geared towards capturing uncertainty in the model fit, rather than of the prediction (i.e. if the sensitive attribute has high levels of variability itself, I’m not convinced that MC Dropout will capture that). I could not identify any theoretical insights to support the main claim and have to rely on the empirical results alone.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The main method presented incorporates uncertainty quantification in a sensitive-attribute classifier to achieve a better fairness-accuracy tradeoff. The key idea behind this method is that samples with lower uncertainty should be used for fairness constraints. Uncertainty quantification is achieved through Monte Carlo dropout. Empirical results support the hypothesis that enforcing fairness constraints with certain data gives more desirable results compared to enforcing these constraints using uncertain or all data. Although the uncertainty quantification mechanism used is somewhat straightforward, the finding that uncertainty quantification is useful in this setting is a valuable contribution.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The confidence interval paragraph (at the end of Section 5) could use additional details. What is the form of the interval/how is it being calculated? This information should be included. I believe that a prediction interval would be more appropriate in this context (using Conformal Prediction, for example). My instinct is that this would quantify uncertainty that is not captured with the current methods. Including such a comparison would strengthen the work, while some discussion on why a prediction interval would not be necessary seems critical.\n\nMinor Typographical Errors and Writing Suggestions:\n-\tSecond paragraph on page 2 mentions “In the first phase” but then does not mention a second phase. I had to reread to make sure I didn’t skip over text about the second phase. This could be combined with the text 2 paragraphs later, giving a full overview of the framework. The paragraph in the middle is heavy on details of the first phase which are not necessary to go into before a full overview.\n-\tThe final sentence of Section 1 seems to be missing an “and”.\n-\tBottom of page 4 the Gal and Ghahramani in-line citation should be in parentheses", "broader_impact_concerns": "The impact statement clearly states that this work should only be used for bias assessment and mitigation. This section also mention that this method may face legal restrictions.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vCCLmbcAQi", "reviewer": "Reviewer_6s5W", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n - The observation is nice.  \n - The writing is clear enough.  \n - The experiments a convincing that the core idea works\n\nWeaknesses:\n - There are a few design choices which could have been done differently (choice of classifier, notion of fairness).  But this is a weak weakness, as all choices are reasonable.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper considers the problem of training a classifier f : R^d -> {0,1} under some fairness constraint from R^d to some other binary demographic attribute (they consider equalized odds, equal opportunity, and demographic parity).  \nThe starting point is the observation that when ability to predict the demographic attribute has a lot of uncertainty, then optimizing for the main classification tasks induces less unfairness.  \nSo in the context of training main classifier f, subject to some fairness constraint, the paper introduces the following idea:\n  1. assess uncertainty on each data points wrt how well it can predict the demographic attribute\n  2. only train f wrt fairness for data x with uncertainty below some threshold.  \n\nThe paper sets of an experimental design with careful test train split, finds several data sets where this task is relevant, provides multiple baseline methods from the past few years, and ultimately shows that their approach often empirically performs the best.  \n\nThe experiments appear well organized and properly done, the writing is (mostly) clear, and results are convincing.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.  Some of the related work discussion could be improved to not just summarize the main idea in papers, but to also explain why these ideas do not need to be directly compared against in more depth.  For instance, in page 3, the Diana etal (2020) paper is described as\n  \"...training a model to predict the sensitive attributes can be a good substitute for the ground truth sensitive attributes when the latter is missing\"\n It would be useful to explain in the context of that paper's discussion why we cannot simply apply their ideas.  And similar for other papers in that paragraph.  \n\n2.  It was not clear in the student model on page 5 if the threshold R was \"predefined\" and fixed (as said above equation 1), or \"updated\" through training or cross-validation (as implied in the last line of the page, after equation 3).  \n  Moreover, explain how R relates to H in equation (5) and discussed later in the paper.", "broader_impact_concerns": "Some data points or attributes that have weak association with demographics may be ignored because of this uncertainty.  This could systematic, and a as a result a group could be left out of classification because of this.  I think this effect is probably neutral in the grand scheme of things, but perhaps should be discussed more.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces FairDSR, a framework to improve the fairness-accuracy tradeoff in machine learning models when demographic information is incomplete or unavailable. The paper proposes to train an attribute classifier to predict missing demographic information and quantify the uncertainty of the classifier. The fairness constraints are only enforced on data with low uncertainty, instead of all data. Experiments demonstrate the advantages of the proposed method over multiple baselines.

The reviewers generally agree that the paper conducts solid experiments to demonstrate the effectiveness of the proposed method (all reviewers). Reviewers 6s5W and JCP5 further agree that the paper is well-written, easy to follow, and results are presented clearly. During the rebuttal, the authors addressed the reviewers' concerns regarding the dependence on reliable uncertainty measures, missing details of the method, and incomplete discussions with baselines. The reviewers acknowledged that their concerns have been addressed and recommended acceptance for the paper.
Decision: Accept (poster)