Evaluating Fairness Without Sensitive Attributes: A Framework Using Only Auxiliary Models
Authors: Zhaowei Zhu
Yuanshun Yao
Jiankai Sun
Yang Liu
Hang Li
Venue: Submitted to ICLR 2023 (2023)
Keywords: Fairness evaluation
noise transition matrix
sensitive attributes

Abstract: Although the volume of literature and public attention on machine learning fairness has been growing significantly in recent years, in practice some tasks as basic as measuring fairness, which is the first step in studying and promoting fairness, can be challenging. This is because the sensitive attributes are often unavailable in a machine learning system due to privacy regulations. The straightforward solution is to use auxiliary models to predict the missing sensitive attributes. However, our theoretical analyses show that the estimation error of the directly measured fairness metrics is proportional to the error rates of auxiliary models' predictions. Existing works that attempt to reduce the estimation error often require strong assumptions, e.g. access to the ground-truth sensitive attributes in a subset of samples, auxiliary models' training data and the target data are i.i.d, or some form of conditional independence. In this paper, we drop those assumptions and propose a framework that uses only off-the-shelf auxiliary models. The main challenge is how to reduce the negative impact of imperfectly predicted sensitive attributes on the fairness metrics without knowing the ground-truth sensitive attribute values. Inspired by the noisy label learning literature, we first derive a closed-form relationship between the directly measured fairness metrics and their corresponding ground-truth metrics. And then we estimate some key statistics (most importantly transition matrix in the noisy label literature), which we use, together with the derived relationship, to calibrate the fairness metrics. Our framework can be applied to all popular group fairness definitions as well as multi-class classifiers and multi-category sensitive attributes. In addition, we theoretically prove the upper bound of the estimation error in our calibrated metrics and show our method can substantially decrease the estimation error especially when auxiliary models are inaccurate or the target model is highly biased. Experiments on COMPAS and CelebA validate our theoretical analyses and show our method can measure fairness significantly more accurately than baselines under favorable circumstances.

Under review as a conference paper at ICLR 2023
EVALUATING FAIRNESS WITHOUT SENSITIVE AT- TRIBUTES: A FRAMEWORK USING ONLY AUXILIARY MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
Although the volume of literature and public attention machine learning fair- ness has been growing significantly in recent years, in practice some tasks as basic measuring fairness, which is the first step in studying and promoting fairesss, can be challenging. This is because the sensitive attributes are often unavailable machine learning system due privacy regulations. The straightforward solu- tion is to use auxiliary models to predict the missing sensitive attributes. However, of the directly measured
our theoretical analyses show that the estimation fairness metries is proportional to the error rates of auxiliary models' predictions. Existing works that attempt reduce the estimation error often require strong as- sumptions, e.g. access to the ground-trun -rruh sensitive attributes in subset of sam- ples, auxiliary models' training data and the target data are i.idd, or some form of conditional independence. In this paper, we drop those assumptions and propose a framework that uses only off-the-shelf auxiliary models. The main challenge is how to reduce the negative impact of imperfectly predicted sensitive attributes on the fairness metrics without knowing the ground-truth sensitive attribute values. Inspired by the noisy label learning literature, first derive closed-fome rela- tionship between the directly measured fairness metrics and their corresponding ground-truth metries. And then estimate some key statistics (most importantly transition matrix in the noisy label literature), which we use, together with the derived relationship, calibrate the faimess metrics. Our framework be ap- plied to all popular group fairness definitions as well as multi-class classifiers and cateery sensitive attributes. In addition, we theoretically prove the upper bound of the estimation error in our calibrated metrics and show our method can substantially decrease the estimation error especially when auxiliary models are inaccurate or the target model is highly biased. Experiments on COMPAS and CelebA validate our theoretical analyses and show our method can measure fair- ness significantly more accurately than basclines under favorable cireumstances.
INTRODUCTION
Despite numerous literature in machine learning fairesss (Corbelt-Davin & Goel 2018), in practice even measuring fairness, which is the first step in studying and mitigating fairness, can be challeng- ing it requires access sensitive attributes of samples, which are often unavailable due to privacy regulations (Andrus et al. [222 Holstein et al. 2019 Veale & Binns 2017). It is a problem that the industry is facing, which signiticantly slows down the progress studying and promoting fairness.
Existing methods to estimate fairesss without access to ground-truth sensitive attributes mostly fall into two categories. First, some methods assume they have access to the ground- sensitive at- tributes subset of samples they can label them if unavailable, e.g. Youtube asks its creators to voluntarily provide their demographic information (Wojcicki) 2021). But it either requires labeling resource depends on the volunteering willingness, and also the resulting measured fairness can be inaccurate due to sampling bias. Second, many works assume there exists an auxiliary dataset that can be used train models to predict the missing sensitive attributes on the target dataset (i.e. the dataset that we want to measure the faimess on), e.g. Meta (Alao et al. (02T)) and others (Elliott et al. 2009 Awasthi et al. 2021 Diana et al. 2022). However, they often need to assume the aux-

---

Under review as a conference paper at ICLR 2023
iliary dataset and the target dataset are ii.dd, and some form of conditional independence, which are not realistic. In addition, since the auxiliary dataset also contains sensitive information (i.e. the sen- sitive labels), it might be more and more difficult to obtain such training data from the open-source projects given the increasingly stringent privacy regulations today. Note that similar to our work, some researchers also draw insight from noisy label literature (Lamy et al. 2019) Celis | al. [02T] Awasthi et al., (2020). But they assume the noise on sensitive attributes follow assumptions such as conditional independence or known transition probabílities. Furthermore, their goal is mitigate bias rather than estimating fairness disparity. We emphasize the value of estimating faimess because the metric is vital in reporting and studying fairness in real- weal-world systems.
In this work, we drop many commonly made assumptions, i.e. 1) access to labeling resource, 2) access to auxiliary model's training data, 3) data i.ddd and 4) conditional independence. Instead we only rely on off-the-shelf auxiliary models, which can be easily obtained via various open-source projects (without their training data). The requirement of the auxiliary model is also flexible. We do not need the auxiliary model's input to share the exactly same feature set as the target data. We only need the auxiliary model's s input features have some overlap with the target dataset's features Our contributions are summarized as follows.
We theoretically show that directly using auxiliary models to estimate fairness (by predicting the missing sensitive attributes) would lead to fairness metric whose estimation error is proportional to the prediction error of auxiliary models and the true fairness disparity (Theorem I Corollary Motivated by the above finding, we propose a general framework (Figure D Algorithm I to calibrate the noisy fairness metrics using auxiliary models only. The framework is based on a derived closed form relationship between the directly estimated noisy faimess metrics and their corresponding ground-truth metrics (Theorem 2) in terms of two key statistics: transition matrix and clean prior probability, which are well-studied in the noisy label literature. To estimate them, our framework can leverage any existing estimator. We show an example by adapting HOC (Zhu et al., 2021b) (Algorithm 2). The estimator only assumes that auxiliary models are informative and different auxiliary models make i.i.d. predictions.
We prove the error upper bound of our estimation (Theorem 3), and show that, in a simplified case, our estimated fairness metrics are guaranteed to be closer to the true metrics than the uncalibrated noisy metrics when auxiliary models are inaccurate or the target model is biased (Corollary 2 Experiments on COMPAS and CelebA consolidate our theoretical findings and show our cali- brated faimess is significantly more accurately than baselines under favorable circumstances.
2 PRELIMINARIES
Consider K-class classification problem with target dataset D == {IIn. Bn) In E [N], where N the number of instances, _,, the feature, and ya is the label. Denote by X' the feature space, y [K] : 11, .... K} the label space, and (X, Y) the random variables of (In, Bn), Vn. The target model f X + [ maps X predicted label class f(X) E [K]. We aim at measuring group fairness conditioned
a sensitive attribute A E [M] := 21, .M] which is unavailable in D Denote the dataset with ground-truth sensitive attributes by =- {(Im. yn. an)ln E [N]], the joint distribution of (X, Y, A) by D. The task is to estimate the fairness metries of f on De without sensitive attributes such that the resulting metrics are as close to the fairness metrics evaluated on D (with ground-truth truth A) as possible. See Appendix [A.] for a summary of notations. We consider three group fairness (Wang et al. 2020) Cotter et al. 2019) definitions and their corresponding measurable metrics: demographic parity (DP) (Calders et al. 22099 Chouldechova 2017), equalized odds (EOd) /Woodworth et al. 2017). and equalized opportunity (EOp) (Hardt et al. 2016).
Fairness Definitions. To save space. all our discussions in the main paper are specific to DP. We include the complete derivations for EOd and EOp in the Appendix. DP metric defined as:
Definition 1 (Demographic Parity). The demographic parity metric of f on D conditioned A is:
A DP (D. M(M - 1)K
Σ
= x P((
For example, if the target dataset contains features about user information (name, location, interests etc.), then our method is applicable as long as the auxiliary model can take any one of those features as input and predict sensitive attributes, e.g. predicting race from name.
2

---

Under review as a conference paper at ICLR 2023
Matrix-form Metrics. To unify different fairness metrics, we define matrix H an intermediate variable. Each column of H denotes the probability needed for evaluating faimesss with respect to f(X) (and Y for EOd and EOp). For DP, H is M x K matrix. The a-th row, k-th column, and element of H are denoted by H
element
[a] H & k), and [a, k), respectively. We have H ki == k| A M)]T. Denote by (H{]
(TP( A = 1). ..... (x(
[a], H
H[a')
== [a
H [a']li/coo H the normalized 4 distance between two rows of H. where (Ol( is the number of columns in H. Denote by V/ (H) == Ca.eeja (H [a], H(
(aa))/(M - 1)). We define the
following disparity as general statistical group faimess metric on distribution D (Chen et al. al.] (022):
Definition 2 (Group Fairness Metric), The group fairness of model f on data distribution
(X.Y, Y. A) ~ DD writes as A(D, f) (H).
We can unify Definitions 4 and 5 (in Appendix A.2 using H. Next, we study how the fairness metrics can be evaluated without A.
Using Auxiliary Models Directly. A direct way measure fairness to approximate with an auxiliary model : X' + [M] (Ghazimatin et al. 2022 Awasthi et al. 2021 Chen et al.) 2019) and get Á : (X). Note the input of g can be any subsets of feature X, and we write the input of as X just for notation simplicity. In practice, there might be C. auxiliary models denoted by the
set 9 == {91. .c............... }. The noisy sensitive attributes can be denoted by A. := 9x(X), Vc E [C] and the corresponding target dataset with À is D { In. (ā)) I [W]] with its distribution denoted as D. Similarly, by replacing with À in H, we can compute H, which is the corresponding matrix-form fairness metric estimated by the auxiliary model (or G if multiple auxiliary models are used). Both notations 9 and 9 are used interchangeably in the remainder. Define the directly measured noisy fairness metric of f on D as follows.
Definition 3 (Noisy Group Fairness Metric). The noisy group fairness of model f on data distribu- tion (x, Y. A) D directly estimated using g writes as () = VD (H).
From the above definitions, if we can calibrate the direct noisy estimate H back to the ground-truth fairness matrix H, the estimation error will be greatly reduced. We defer more details to Theorem 2
Transition Matrix. The relationship between H and H is largely dependent on the relationship between A and Á because it is the single changing variable. Define the matrix T to be the transition probability from A À where (a, a--th element is T(a, a| P(Ā āl A a). Similarly, denote by
Tr the local transition matrix conditioned on f(X) = k, where the (a, a)-th element is Th [a, a] :=
P(Á = āl averaging Tg. Many prior works (Awasthi et al.) 2021 Prost et al. 202T: Fogliato et al. 2020)
A A a). Note T can be seen a global transition matrix by weighted
assume A and f(X) are conditionally independent on A. We drop this assumption in our theoretical framework. We further define clean (i.e. ground-truth) prior probability of A as p := [IP(A
1), ,PA., = .P] and the noisy prior probability of Á as p = [P(Á = 1). ... P(Á = M)]
KEY INSIGHT: WHY WE NEED CALIBRATION?
Now we study the error of direct noisy fairness metrics and motivate the necessity of calibration.
Estimation Error Analysis. Intuitively, the estimation error of directly measured noisy fairness metrics is dependent on the eтrоr of the auxiliary model g. Recall p. p. T and T3 are clean prior, noisy prior, global transition matrix, and local transition matrix defined in Sec. 2 Denote by Ap and Ap the square diagonal matrices constructed from P and p. We formally prove the upper bound of estimation error in the directly measured metrics in Theorem (See Appendix B. for the proof).
Theorem (Error Upper Bound of Noisy Metrics). Denote by Err == PP DP f) the estimation error of the directly measured toisyfairnesse fairness metrics. Its upper bound is the following:
Erraw
2
Σ
The (T 'T. A) 1|1 + +8x AA IlI1
K
RE[A]
cond. indep violation
where hy : 1 Σ M aE[M)]
H(a k), 66
max ae[ad]
H[a, k] hull-
3

---

Under review as a conference paper at ICLR 2023
Auxiliary Model
Estimated Transition Matrix Prior Prob. P
9
Noisy Attribute
Target Dataset
Noisy Fairness Matrix H
Calibrated Fairness Matrix H
D
Target Model
Uncalibrated Metric ((D.
Calibrated Metric
((D.f)
(a) Directly Estimate Fairness
(b) Calibrate
Overview of our framework to estimate fairness without sensitive attribute given auxiliary models only. Step 1 (a): Estimate noisy fairness matrix. Step 2 (b): Calibrate the fairness matrix using the estimated transition matrix and prior probability.
Figure 1:
Theorem D reveals that the estimation error of directly measured metric depends on:
hy: The average confidence of fXX on class i over all sensitive groups. For example, if f is crime prediction model and A is race, biased f (Angwin et al. 2016) may predict that the crime (k = 1) rate for different races are 0.1, 0.2 and 0.6 respectively, then hy 0.1+0.2+0.6 = 0.3, and it is an approximation (unweighted by sample size) of the average crime rate over the entire population. The term is dependent on 'D and f, and independent of any estimation algorithm.
3
ōg: The maximum disparity between confidence of f(X) on class 1 and average confidence h& across all sensitive groups. Using the same example, ō1 = max( 0.1 0.3), J0.2 0.3), |0.6 0.3]) = 0.3. It is an approximation of the underlying fairness disparity, and larger 6µ indicates is more biased on D. The term is also dependent on and ʃ (i.e. the true fairness disparity), and independent of any estimation algorithm.
Conditional Independence Violation: The term is dependent on the auxiliary model g's prediction À in terms of the transition matrix (T and Tx) and noisy prior probability (p). The term goes to 0 when T = Tg. which implies À and f(X) are independent conditioned on A. This is the common assumption made in the prior work (Awasthi et al. 202T Prost et al., 202T; Fogliato al. 2020).
And this term measures how much the conditional independence assumption is violated.
Error of g: Similarly, this term is dependent on the auxiliary model It goes to 0 when Ti which implies the error rates of g's prediction is i.e. g perfectly accurate. measures the impact of g's error on the fairness estimation error.
To help better understand the upper bound, we consider a simplified case when f is a binary model and A is binary variable. We further assume the conditional independence condition remove the third term listed above in Theorem [ See Appendix A.3 for the formal definition of conditional independence P Corollary summarizes the result.
Corollary 1. For binary classifier f and binary sensitive attribute A E {1,2}, when (Ā 1 f(X))A A) holds, Theorem is simplified to Erraw < 25(e1 + 62), where 61 and 62 are transition probabilities from noisy attributes to clean attributes, i.e. 61 = P(A = 1 2), e2 = P(A A 6) IP((()) 1|A= -)((( 1|A 2)2/2.
Why Calibrate? Corollary I cllearly shows the estimation error of the directly measured fairness proportional to the true underlying disparity between sensitive groups (i.e. ô) and the auxiliary model's error rates (i.e. 2₁ and e2). In other words, the uncalibrated metrics can be highly inaccurate when o is highly biased or g has poor performance. Both are practical cases since when we want to measure I's fairness, it has already shown some fairness-related concerns and the fairness disparity not negligible. Moreover, the auxiliary model g is usually not highly accurate due distribution shift. Hence, in those cases we should calibrate the metrics to get more accurate measurements.
We only assume it for the purpose of demonstrating less complicated theoretical result, we do not need this assumption in our proposed algorithm later.
4

---

Under review as a conference paper at ICLR 2023
4 METHODOLOGY
In this section, we introduce our calibration framework and algorithm (Sec. 4.1 prove the error up- per bounds for our calibration methods (Sec. 4.2). and elaborate key steps of algorithm (Sec. 4.3
4.1 PROPOSED FRAMEWORK
With given auxiliary model g that labels sensitive attributes, between the true disparity and the noisy disparity. We have the following theorem for DP. See Appendix B.2 for results with respect to EOd and EOp and their proofs.
anatomize the relationship
Theorem 2. [Closed-form Relationship (DP)) The closed-form relationship between the true fair- ness vector H k] and the corresponding directly measured noisy fairness vector H [. k) is the following:
H (T
[K].
Framework Overview. Theorem reveals that the noisy disparity and the corresponding true disparity are related in terms of three key statistics: noisy prior p. clean prior and local transition matrix Tg. Ideally, we can obtain the ground-truth values of them, then we can calibrate the noisy fairness vectors to their corresponding ground-truth vectors (and therefore the perfectly accurate fairness metrics) using the closed-form Theorem 2 Hence, the most important step to estimate Tk. P. and p without knowing the ground-truth values of A. Once have those estimated key statistics, we can easily plug them into the above equation as the calibration step. Figure shows the overview of our framework.
Algorithm. We summarize our framework in Algorithm | In Line 4 we use sample mean in the uncalibrated form to estimate H as H [ā = P((()) k| a) xe Ex= 1(f(=n = Σ x== l@ā, a). vā [M]. In Line in an existing transition matrix and prior probability estimator to estimate Tù and with only mild adaption that will be introduced in Sec. 4.3 Note that although we choose specific estimator to
klā, a)) and as p|a] PPA = à) =
plug
use, our framework is flexible and compatible with any StatEst mator proposed in the noisy
label literature (Liu Chen 2017 Zhu al. 2021b 2022)
4.2 ESTIMATION ERROR ANALYSIS
calibrated metrics in similar way in Sec. The derivation is based on local estimates Tg, global estimates T would be similar. Denote by DP (D. the calibrated DP disparity evaluated on our calibrated fairness matrix H. We have: Theorem 3 (Error Upper Bound of Calibrated Metrics). Denote the estimation error of the cali- brated fairness metrics by En A DP (D, f) AD, f)}. Its upper bound is the following:
We theoretically analyze estimation
Enl
| || H Alll efTr.p).
kE[K]
where e -= T +
T I is the error induced by calibration. With a perfect estimator; iee. T3 = Tę and pa: = Ps, vk E [K]. we have Enfal 0.
Theorem 3 shows the upper bound of estimation enror mainly depends on the estimates T. and p. i... the following two terms eTT,. k. p): ||A," A, I || II1 and || T&T, '11. When the estimates perfect, i.e. T₃ Tg and p, then both terms go to 0 because A,," A, I and T I. We now compare the above error upper bound with the exact error (not its upper bond) in the case of Corollary I and summarize the result in Corollary 2
Corollary 2. When assumptions in Corollary 1 hold, further assume p = [0.5. 0.5]T, then the proposed calibration method is guaranteed to be accurate than the uncalibrated measurement, i.e. Enfal Err w if (TTT, p) $7:=
max
tsteg
Wk E (1,2).
k'E(1.2)
2PF(D.F)
5

---

Under review as a conference paper at ICLR 2023
Algorithm Fairness calibration framework (DP)
1: Input: A set of auxiliary models G = (gi.... gc). Target dataset D Target model f-
Transition matrix and prior probability estimator St atEstimattor.
2: a 6- [e([n), VC E [C], n [N]
# Predict sensitive attributes using all g E g # Build the dataset with noisy sensitive attributes Directly estimate faimess matrix and prior with sample mean
3: + ((In; Vu: (ā e
4: H DireccEst(D,,
5: # Estimate key statistics: and Ta
6: (Ti,... T P t StatEstimator f)
7: vk E [K] H b (T (T)
Calibrate each faimess vector with Theorem
8: A(D,J) - (H)
# Calculate the final faimess metric as Definition
9: Output: The calibrated fairness metric f)
Corollary shows when the error e(T., p) that is induced by inaccurate T. and P is below the threshold 7, our method is guaranteed to lead to smaller estimation error compared to the uncali- brated mcasurement under the considered setting. The threshold implies that, adopting our method rather than the uncalibrated measurement can be greatly beneficial when e1 and e2 high (i.e. is inaccurate) or when the normalized (true) fairness disparity A (D. is high (i.e. f is highly [[ :
biased).
4.3 ESTIMATING KEY STATISTICS
As mentioned, our framework can plug in any existing estimator transition matrix and prior probability. We choose HOC (Zhu al. 2021b) because is free of training. Some methods (Liu & <To 2015; Scout, 2015: Patrini et al. 2017) require extra training with target data and auxiliary model outputs, which introduces extra cost. Morcover, brings practical challenge hyper- parameter tuming given we have no ground-truth sensitive attributes. HOC decodes T3 by checking both the agreements and disagreements among noisy attributes (auxiliary model predictions). See more details in Appendix C.I For successful decoding, HOC makes the following assumptions:
Assumption 1 (HOC: Informativeness). The noisy attributes given by each classifier g are infor- mative, i.e Vk E [M]. I) Tę is non-singular and 2) either Tæ(a, a) P(A alf(X) k)) or Ta a] > Ta a'), Va' + aa.
Assumption 2 (HOC: Independence). Given three auxiliary models, the noisy attributes predicted by them are independent and identically distributed (i.id.), i.e. (X)),, (2(X), and gx(X) are i.i.d.
Assumption iis the prerequisite of getting a feasible and unique estimate of T (Zhu et al., 2021b). where the non-singular assumption ensures the matrix inverse in Theorem 2 exists and the con- straints on T& [a, a) describes the worst tolerable performance of g. When M 2, the constraints can be simplified as Ti [1, 2] - [2, 1] < 1 (Liu & Chen 2017 Liu & Guo 2020 If this assumption violated, there might exist more than one Teasible estimates of Te. making the problem insoluble. Assumption ensures the additional two auxiliary models provide more information than using only one classifier. Note it has been proved by Liu (2022) that three is the sufficient and necessary num- ber of auxiliary models to provide sufficient information to identify Tg. If Assumption 2 is violated, we would still get an estimate but may be inaccurate.
Adapting HOC. Algorithm shows how we adapt HOC as Stat Est imator (in Algorithm I Line 6). namely HOCFair. The original HOC uses one auxiliary model and simulates the other two based on clusterability assumption (Zhu et al. 2021b), which assumes In, and its 2-nearest- neighbors share the same true sensitive attribute, and therefore their noisy attributes can be used to simulate the output of auxiliary models. If this assumption does not hold (Zhu et al.) 2022). we can directly use more auxiliary models. With a sufficient number of noisy attributes, we can randomly select three of them for every sample as Line 6 and then approximate T₃ with T, in Line In our experiments, test both using one auxiliary model and multiple auxiliary models.

---

Under review as a conference paper at ICLR 2023
Algorithm 2 StatEstimator: HOCFair
1: Input: Noisy dataset D. Target model f.
2: C' < #Attribute())
Get the number of noisy attributes (i.e. number aux. models)
3: if C < 3 then
# Get Ger Neiieseerest of x. and save their attributes as z. 's attribute
4:
D + ((En Vn. (a))"" (" E [N]] & Get
5: end if
6: {(āh, a (m E [N]] f Sample(D)
# Randomly sample 3 noisy attributes for each instance
7: (T, & HOC( (o C an E [N]])
Use HOC to get global estimates T =z T and p P HOC a (n E f(z,) = k}), Wk E [K] # Get local estimates Tę Fs T₃
8:
9: Output: (T, Tk T.P.
# Return the estimated statistics
EXPERIMENTS
5.1 EXPERIMENT SETUP
We test the performance of our method on two real-world datasets: COMPAS (Angwin al. 2016 and CelabA (Liu et al.) 2015). We report results on all three group faireess metrics (DP; EOd, and EOp) whose true disparities (estimated using the ground-truth sensitive attributes) are denoted by A (D, f), A SE0d(D, f). ED, f) respectively. We train the target model f on the dataset without using A, and use the auxiliary models downloaded from open-source projects. The detailed settings are the following:
COMPAS (Angwin et al. 2016): Recidivism prediction data. Feature X: tabular data. Label Y: recidivism within two years (binary). Sensitive attribute A: race (black and non-black). Target models / (trained by us): decision tree, random forest, boosting, SVM, logit model, and neural network (accuracy range 66%-70% for all models). Three auxiliary models (g), 92, 92.93) racial classifiers given name as (Sou & Laohaprapanon (2018) (average accuracy 68.85%).
(Sood
CelabA (Liu et al. 2015): Face dataset. Feature X: facial images. Label Y: smile or not (binary). Sensitive attribute A: gender (male and female). Target models f: ResNet18 (He al, 2016) (accuracy 90.75%, trained by us). We only use one auxiliary model (g1): gender classifier that takes facial images as input (Serengil & Ozpinari 202T) (accuracy 92.55%) We then use the clusterability simulate the other two auxiliary models as Line in Algorithm 2
Practical Estimates of Ty: Local vs. Global. According to Theorem 3 when T.s are accurately estimated, we should always rely on the local estimates Line g of Algorithm 2 to achieve zero calibration error. However, in practice, each time when we estimate a local Tr. the estimator would introduce certain error on the T3 (discussed in Sec. 4.3 and the matrix inversion Theorem might amplify the estimation error on Tr each time, leading to large overall error on the metric. One heuristic is to use a single global transition matrix T estimated once on the full dataset ò as Zine of Algorithm 2 replace all Te's. Intuitively, T can be viewed as the weighted average of all Th's to stabilize estimation error (variance reduction) on Tg. Admittedly, the average will introduce bias since the equation in Theorem 2 would not hold when replacing T. with T. The justification is that the error introduced by violating the equality might be smaller than the error introduced by using severely inaccurately estimates of T 's. Therefore, we offer two options for estimating Ti in practice: locals estimates Tę R T3 and global estimates T.- a T. Although it is hard to guarantee which option must be better in reality, we report the experimental results using both options and provide insights for choosing between both estimates in Sec. 5.2
Method. We test our proposed framework with global estimates T. T (Global) and local esti- mates T3 ~ T3 (Local). We compare with two baselines: the directly estimated metric without any calibration (Base) and Soft (Chen et al./ 2019) which also only uses auxiliary models to calibrate the measured fairness by re-weighting metric with the soft predicted probability from the auxiliary model.
7

---

Under review as a conference paper at ICLR 2023
Table I: Normalized estimation error on COMPAS. Each row represents a different target model f.
COMPAS
DP Normaliged Eroor (%)
EOd Normalized Eror (%k L
EOp
Normaliged Eror (%)
True disparity: ~ 0.2
Base
Solt
Global
Local
Base
Solt
Global
Local
Base
Solt
Global
Local
43.82
61.26
22.29
39.81
45.86
63.96
23.09
42.81
54.36
70.15
13.27
49.49
forest
43.68
60.30
19.65
44.14
45.60
62.85
18.56
44.04
53.83
69.39
17.51
63.62
boosting
43.82
61.26
22.29
44.64
45.86
63.96
23.25
49.08
54.36
70.15
13.11
54.67
SVM
50.61
66.50
30.95
42.00
53.72
69.69
32.46
47.39
59.70
71.12
29.29
51.31
logit
41.54
60.78
31.91
41.69
60.55
19.48
34.22
43.34
62.99
19,30
43.24
54.50
68.50
14.20
59.95
compus pettre
41.28
58.34
59.79
11.80
18.65
48.78
Table 2: Normalized error CelebA. Each row represents differentt pre-trained model generate feature representations that we use to simulate the other two auxiliary models 92. 93 (Line Algo- rithm 2). Base and Soft are computed ÿ1 and not changed since they are independent of feature representations. The ground-truth fairness metrics are DP: 0.13, EOd: 0.03, EOp: 0.05.
DP Normalized Error (%) L
EOd Normalized Error (%)
EOp
Normalized Error (%)
ColebA
Base
Soft
Global
Local
Base
Soft
Global
Local
Base
Soft
Global
Local
Facenet
15.33
12:54
22.17
10.89
4.11
6.46
7.54
0.26
2.82
0.34
12.22
2.93
Facener512
15,33
12.54
21.70
7.26
4.11
6.46
4.85
0.52
2.82
0.34
11.80
3.24
OpentFace
4.11
6.46
AreFace
15,33
12.54
19.59
9.69
4.11
6.46
5.72
0.23
2.82
0.34
11.16
3.85
Dlib
5.30
4.11
6.46
4.87
4.25
9.74
SFace
15.33
12.54
17.00
4.77
4.11
6.46
4.04
3.91
2.82
0.34
9.36
3.28
Evaluation Metric.
Let (D,, f) be the ground-truth fairness metric. For given estimated metric
E, we define three estimation errors: Raw Error := |E A(D, f)), Normalized Error(E) := Error(E)
Raw Emro((E) and Improvement( E) == Raw Error(Base) where Base is the directly measured metric.
ATD.))
5.2 RESULTS AND ANALYSES
COMPAS Results. Table ! reports the normalized error on COMPAS (See Table in Ap- pendix D.1 for the other two evaluation metries). There two main observations. First, calibrated metrics outperform baselines with a big margin on all three fairness definitions. Com- pared to Base, Corollary 2 this is because the target models f are highly biased (Table and the auxiliary models g are inaccurate (accuracy 68.9%). As a result, Base has large normalized error (40-60%). Sec- ond, Global outperforms Local, since with inaccurate auxiliary models, Assumptions D on HOC estimator may not hold in local dataset, inducing large estimation errors in local estimates.
metrics 39.6%-88.22 28% more accurate (Improvement). As pointed out by
CelebA Results. Teble the normalized error on CelebA where each row represents using a different pre-trained model generate feature representations used to simulate the other two aux- iliary models (See Table in Appendix D.2 for the full results). although our method still outperforms baselines most of time, the margin is smaller and derperformed by Soft when estimating EOp. Similarly this is because the conditions in Corollary 2 do not hold, i.e. f is barely biased in EOd and EOp (Table 8 and g is accurate (accuracy 92.6%). As a result, Base only has a moderate normalized emror in DP (15.3%), and small normalized errors in EOd (4.1%) and EOp (2.8%). Given the highly accurate Base, the benefit of adapting our method is outweighed by the estimation error introduced by calibration (mostly the key statistic estimator). Second, contrary to COMPAS, Local outperforms Global. This is because now the auxiliary models are accurate, Assumption always holds and Assumption 2 is also likely to hold when 92 and gs are sellasimulated. Consequently, Local is estimated accurately while Global induces extra error due to violating the equality in Theorem 2 Finally, even though our method is underperformed in EOp, the raw error of our method is acceptable, which is less than 0.01 Table 8 in Appendix D.2
have two observations. First,
Ablation Study. To better understand when our method can give a clear advantage with different quality of g. by to bring down g accuracy and report the results Table (See Appendix [D.2] for the full results). When g becomes less accurate, method can outperform baselines, which validates
an ablation study CelebA. We randomly flip the predicted sensitive attributes
8

---

Under review as a conference paper at ICLR 2023
Table 3: Normalized error on the CelebA when adding noise by randomly flipping the predicted attributes to bring down the performance of auxiliary models. Each row represents the noise magni- tude and accuracy of auxiliary models, e.g. "[0.2, 0.0] (82.44%)" means T 2] 0.2, [ 0.0 and accuracy is 82. 22.44%.
CelebA
DP Normalized Emrr (%
EOd
Normalized Error (%)
EOp
Normalized Error (%%
F1ceNar512
Base
Soft
Global
Local
Base
Soft
Global
Local
Base
Soft
Global
Local
0.2.0.0
(82.445 44%)
7.37
11.65
20.58
5.05
26.99
6.43
0.10
24.69
27.27
11.11
0.2.0.2
(75.54%)
30.21
31.57
24.25
13,10
44.73
46.36
11.26
9.04
37.67
38.77
20.94
27.98
0.4.0.2 (65. 36%)
51.32
54.56
19.42
10.47
62,90
11,09
19.15
56.51
58.73
23.86
23.55
0.4.0.4 (58.45%)
77.76
78.39
9.41
19.80
79.31
80.10
24.49
8.02
78.35
79.62
10.61
5.71
Corollary 2 In addition, Local still outperforms Global. This is because add random noise following Assumption] 2 and therefore the estimation error of Local is not increased significantly.
Takeaways. Our experimental results imply two takeaways: 1) our calibration method can give a clear advantage when the error rates of g are moderate to high (e.g. error 2 15%) or f is highly biased (e.g. fairess disparity 2 0.1) and 2) we can prefer Local when the auxiliary model is accurate and Global otherwise. In practice, given no ground-truth A, we can roughly estimate auxiliary models' accuracy range from the estimated transition matrix
RELATED WORK
Fairness with Imperfect Sensitive Attributes. The closest work to ours is (Chen et al. 2019). which also assumes only auxiliary models. It is only applicable to demographic disparity, and we compare it in the experiments. In addition, other works focus on how to train auxiliary models from a given auxiliary dataset (Awasthi et al., 2021] Diana et al. 2022). For example, Awasthi et al. (202T) propose active learning scheme and assume there exists auxiliary dataset that is ki.d with the target dataset. In our work, we do not need the auxiliary dataset; nor do we need to assume the auxiliary model's training set is i.id with the target dataset. [Lmmy et al. (2019) also draws insights from noisy label literature. However, the attribute noise is assumed to come from the mutually contaminated assumption rather than from an auxiliary model. Furthermore, Prost et al. (202T) and Fogliato et al. (2020) theoretically study the error gap of estimating fairness. but they do not propose any calibration method. There are other parallel works that aim to mitigate bias without estimating it (Hashimoto et al., 2018 Lahoti et al.] 2020: Wang et al. 2020); Yan et al. 2020)) We emphasize the value estimating fairness because the metrics themselves are vital in reporting and studying fairness in real-world systems.
Noisy Label Learning. Label noise may come from various sources, e.g., human annotation error (Xiao et al. 2015) Wei et al. 2013: Berthelot et al. 2019; Zhu et al. (02 which can be characterized by transition
2022 Agarwal et al., (016) and model prediction error (Lee)
matrix on label (Liu] 2022 Bae et al. [2022] Yang et al. 22021) Applying the noise transition matrix
to ensure fairness emerging Wang al.] 2021 Liu & Wang 2021 Lamy al. 2019). There exist
two lines of works for estimating transition matrix. The first line relies on anchor points (samples belonging to a class with high certainty) or their approximations (Liu & Tao 2015 [Soot| 2015)
Patrini et al. 2017 Xia et al. 2019: Northcutt et al. 2021). These works requires training neural
network the (X, A = g( ()). The second line of work, which we leverage, is data-centric (Liu
& Chen 2017 Liu et al. 2020 Zhu et al.] 20216] (022) and training-free. The main idea is to check
the agreements among multiple noisy attributes discussed in Section 4.3
LIMITATION AND FUTURE WORK
We point out two limitations in our work. First, our method shows limited improvement over directly measured metrics when auxiliary models are highly accurate and the true fairness disparity is small. Second, our theoretical guarantee the superiority of our method (Corollary P is only theoretically proven in a simplified case. One future work is to apply the same method to unfairness mitigation algorithm in addition to evaluating faimess.

---

Under review as a conference paper at ICLR 2023
ETHICS STATEMENT
Our goal is to better study and promote fairness. Without a promising estimation method, given the increasingly stringent privacy regulations, would be difficult for academia and industry to measure, detect, and mitigate bias in many real-world scenarios. However, we need to caution readers that, needless say, no estimation algorithm perfect. Theoretically, in our framework, if the transition matrix is perfectly estimated, then our method can measure faimess with 100% accuracy. However, Assumptions 2 required by our estimator in Algorithm 2 do not hold, our calibrated metrics might have non-negligible error, and therefore could be misleading. In addition, the example we use to explain terms in Theorem [ is based on conclusions from (Angwin et al., (0T6)). We do not have any biased opinion the crime rate across different racial groups. Furthermore, aware that many sensitive attributes are not binary, e.g. race and gender. We use the binary sensitive attributes in experiments because 1) existing works have shown that bias exists in COMPAS between race "black" and others and 2) the ground-truth gender attribute in CelebA is binary. Finally, all the data and models use from open-source projects, and the bias measured on them do not reflect our opinions about those projects.
fully
10

---

Under review as a conference paper at ICLR 2023
REFERENCES
Vibhu Agarwal, Tanya Podchiyska, Juan M Banda, Veena Goel, Tiffany Leung, Evan P Minty, Timothy Sweeney, Elsie Gyang, and Nigam Shah. Learning statistical models of phenotypes using noisy labeled training data. Journal of the American Medical Informaties Association, 23 (6):1166-1175 2016.
Rachad Alao, Miranda Bogen, Jingang Miao, llya Mironov, and Jonathan Tannen. How Meta is working to assess fairness relation race the U.S. across its products and sys- wwwpe///wwbiciiiooo/owweteaeaaae
taap------sssss-iiiess--- tems. htpps//ai.facebooee itrrossstt--rroductss produucs--nd--syttems. syd-systems, 2021. [Online; accessed 15-Sep- soo-race-in-the--- -n--th--s--acros--iss 2022].
McKane Andrus, Elena Spitzer, Jeffrey Brown, and Alice Xiang. What can't measure, can't understand: Challenges to demographic data procurement in the pursuit of fairness. In Proceedings of the 2021 ACM Conference Fairness, Accountability, and Transparency, pp- 249-260) 2021.
Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias, In Ethics of Data and Analytics, pp. 254-264. Auerbach Publications, 2016.
Pranjal Awasthi, Matthäus Kleindessner, and Jamie Morgenstern. Equalized odds postprocessing under imperfect group information. In Proc. of AlStat, 2020.
Pranjal Awasthi, Alex Beutel, Matthäus Kleindessner, Jamie Morgenstern, and Xuezhi Wang. Eval- uating fairness of machine learning models under uncertain and incomplete information. In Proc. of FAccT, 2021.
HeeSun Bac, Seungjae Shin, Byeonghu Na, JoonHo Jang, Kyungwoo Song, and Il-Chul Moon. From noisy prediction to true label: Noisy prediction calibration via generative model. In Inter- national Conference Machine Learning, pp. 1277-1297. PMLR, 2022.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimiza- tion and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine learning, 301):1-122, 2011.
Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy. Building classifiers with independency constraints. In 2009 IEEE International Conference on Data Mining Workshops, pp. 13-18. IEEE, 2009.
Elisa Celis, Lingxiao Huang, Vijay Keswani, and Nisheeth Vishnoi. Fair classification with noisy protected attributes: A framework with provable guarantees. In International Conference on Machine Learning, pp. 1349-1361. PMLR, 2021.
Jiahao Chen, Nathan Kallus, Xiaojie Mao, Geoffry Svacha, and Madeleine Udell. Fairness under unawareness: Assessing disparity when protected class is unobserved. In Proc. of FAceT, 2019.
Yatong Chen, Reilly Raab, Jialu Wang, and Yang Liu. Faimess transferability subject to bounded distribution shift. arXiv preprint orXi::2206.00/2 222299 2022.
Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism
prediction instruments. Big data, 5(2):153-163, 2017.
Sam Corbett-Davies and Sharad Goel. The measure and mismeasure fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.0000 2018.
Andrew Cotter, Heinrich Jiang, Maya R Gupta, Serena Wang, Taman Narayan, Seungil You, and Karthik Sridharan. Optimization with non- con-differentiable constraints with applications to faimess, recall, chur, and other goals. J. Mach. Learn. Res., 20(17)):1-59, 2019.
11

---

Under review as a conference paper at ICLR 2023
Emily Diana, Wesley Gill, Michael Kearns, Krishnaram Kenthapadi, Aaron Roth, and Saeed Sharifi- Malvajerdi. Multiaccurate proxies for downstream fairness. In Proc. of FAccT, 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
Mare Elliott, Peter A Morrison, Allen Fremont, Daniel F McCaffrey, Philip Pantoja, and Nicole Lurie. Using the census bureau's s surname list to improve estimates of race/ethnicity and associ- ated disparities. Health Services and Outcomes Research Methodology, 9(2):69 83, 2009.
Riccardo Fogliato, Alexandra Chouldechova, and Max G'Sell. Fairness evaluation in presence of biased noisy labels. In Proc. of AlStat, 2020.
Azin Ghazimatin, Matthaus Kleindessner, Chris Russell, Ziawasch Abedjan, and Jacek Golebiowski. Measuring fairness of rankings under noisy sensitive information. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 2263-2279, 2022.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29:3315-3323, 2016.
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In Proc. of ICML, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miro Dudik, and Hanna Wallach. Improving fairness in machine learning systems: What do industry practitioners need? In Pro- ceedings of the 2019 CHI conference on human factors in computing systems, pp. 1-16, 2019.
Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographics through adversarially reweighted learning. Proc. of NeurlPS, 2020.
Alex Lamy, Ziyuan Zhong, Aditya K Menon, and Nakul Verma. Noise-tolerant fair classification. 2019.
Dong-Hyun Lee al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, pp. 896, 2013.
Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. JEEE Transactions pattern analysis and machine intelligence, 38(3):447-461, 2015.
Yang Liu. Identifiability of label noise transition matrix. arXiv e-prints, pp. arXiv-2202, 2022.
Yang Liu and Yiling Chen. Machine-leaming aided peer prediction. In Proceedings of the 2017 ACM Conference on Economics and Computation, pp. 63-80, 2017.
Yang Liu and Hongyi Guo. Peer loss functions: Learning from noisy labels without knowing noise rates. In Proceedings of the 37th International Conference on Machine Learning. ICML '20, 2020.
Yang Liu and Jialu Wang. Can less be more? when increasing-to--blancee label noise rates con- sidered beneficial. Advances in Neural Information Processing Systems, 34, 2021.
Yang Liu, Juntao Wang, and Yiling Chen. Surrogate scoring rules. In Proceedings of the 21 ACM Conference on Economics and Computation, pp. 653-871, 2020.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV) December 2015.
12

---

Under review as a conference paper at ICLR 2023
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In International Conference on Machine Learning, pp. 3384-3393. PMLR. 2018.
Curtis Northeutt, Lu Jiang, and Isaac Chuang. Confident learning: Estimating uncertainty in dataset labels. Journal of Artificial Intelligence Research, 70: 10:3331-1,1, 2021.
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1944-1952, 2017.
Flavien Prost, Pranjal Awasthi, Nick Blumm, Aditee Kumthekar, Trevor Potter, Li Wei, Xuezhi Wang, Ed H Chi, Jilin Chen, and Alex Beutel. Measuring model fairness under noisy covariates: A theoretical perspective. In Proc. of AIES, 2021.
Clayton Scott. A rate of convergence for mixture proportion estimation, with application to learning from noisy labels. In AISTATS, 2015.
Sefik Ilkin Serengil and Alper Ozpinar. Hyperextended lightface: A facial attribute analysis frame- work. In 2021 International Conference Engineering and Emerging Technologies (ICEET), pp. 1-4. IEEE, 2021. doi: 10.1109/CEETT5442.2200 20211444...221.6656677 URL
10. www.1119ICEETTSS
9659697
Gaurav Sood and Suriyan Laohaprapanon. Predicting race and ethnicity from the sequence of char- acters in a name. arXiv preprint arXiv: 109, 2018.
Michael Veale and Reuben Binns. Fairer machine learning in the real world: Mitigating discrimina- tion without collecting sensitive data. Big Data & Society, 40):20539517177455500 2017.
Jialu Wang, Yang Liu, and Caleb Levy. Fair classification with group-dependent label noise. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp- 526-5336 2021.
Jialu Wang, Xin Eric Wang, and Yang Liu. Understanding instance-level impact of fairness straints. In International Conference on Machine Learning, pp. 231 14-23130, PMLR, 2022.
Serena Wang, Wenshuo Guo, Harikrishna Narasimhan, Andrew Cotter, Maya Gupta, and Michael Jordan. Robust optimization for fairness with noisy protected groups 2020.
Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels revisited: A study using real-world human annotations. In International Confer- ence Learning Representations, 2022. URL http:tttt//ppereeeeeeee wtt.://wwwww.eeftt/.....o fortmmrr
net
TBWA6PLJZQm
Susan Wojeicki. Letter from Susan: Our 2021 Priorities. https://blog http://blooo.ouuueee.e youtube/ fttp://ttttttt----sss--a----- 2011-pri 2021. [Online; accessed 20-22.
inside inside-youtube youtee-youubbeeeeeeeeeeee
Blake Woodworth, Suriya Gunasekar, Mesrob Ohannessian, and Nathan Srebro. Learning non- discriminatory predictors. In Conference on Learning Theory, pp. 1120-1955, PMLR, 2017.
Xiaobo Xia, Tongliang Liu, Nannan Wang, Bo Han, Chen Gong, Gang Niu, and Masashi Sugiyama. Are anchor points really indispensable label-noise learning? In Advances in Neural Information Processing Systems, pp. 6838-6849 2019.
Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. Learning from massive noisy labeled data for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2691-2699, 2015.
Shen Yan, Hsien-te Kao, and Emilio Ferrara. Fair class balancing: Enhancing model fairness without observing sensitive attributes. In Proc. of CIKM, 2020.
Shuo Yang, Erkun Yang, Bo Han, Yang Liu, Min Xu, Gang Niu, and Tongliang Liu. Estimating instance-dependens laiel-noise transition matrix using dnns. arXiv preprint arXiv:2105.1300, 2021.
13

---

Under review as a conference paper at ICLR 2023
Zhaowei Zhu, Tianyi Luo, and Yang Liu. The rich get richer: Disparate impact of semi-superviie learning. arXiv preprint
2021a.
Zhaowei Zhu, Yiwen Song, and Yang Liu. Clusterability as an alternative to anchor points when learning with noisy labels. In International Conference on Machine Learning, pp. 12912-12923. PMLR, 2021b.
Zhaowei Zhu, Jialu Wang, and Yang Liu. Beyond images: Label noise transition matrix estimation for tasks with lower-quality features. arXiv preprint arXiv:220..01273, 2022.
14

---

Under review as a conference paper at ICLR 2023
Appendix
The Appendix is organized follows.
Section A presents summary of notations, more faimesss definitions, and clear statement of the assumption that common the literature. Note our framework does not rely on this assumption. Section B presents the full version of our theorems (for DP, EOd, EOp), corollaries, and the corresponding proofs.
Section shows how HOC works and analyzes why other learning-centin methods in the noisy label literature may work in our setting.
C
D The code and data for reproducing our experiments will be released after acceptance.
Section presents more experimental results and takeaways.
A
MORE DEFINITIONS AND ASSUMPTIONS
A.1 SUMMARY OF NOTATIONS
Table 4: Summary of key notations
Notation
Explanation
G (8) -90
C auxiillaryy models for generating noisy sensitive attrihutes
X, Y, A, and A== g(X)
Random variables of feature, label, ground-truth sensitive attribute, and noisy sensitive attributes feature, label, ground sensitive altribute The number of instances, label classes, categories of sensitive attributes
N.K.M
counting from Space of X. target model Target datasct
N. X. [K]
Dd (( Mn (V
D: ((Fn: Gn [)]
Tt with ground-truth sensitive attributes
= [[2.. No. (G. I E [N])
DP with noisy sensitive attributes
(X.Y, D. D,K.Y.AA
Distribation of and Ž)
(DP, EOd, EOd) A'D (D (D, f. A
unified notation of fairness definitions, e.g. EOd, EOp, EOd
True, (direct) moisy, and calibrated group fairness metries data distributions True, (direct) noisy, and calibrated group fairness mettics datasets Faimess matrix, its a-th row, k-th column,, (a, k)) element
ADD &. DD..D. (...
H. H,H, H H. H
Noisy faimess matris with respect to
T. T.T P(A a
Global noise transition matrix
T. o. P(Á G/A ((X)=
Local noise transition matrix
p P(A= PiA
Clean prior probability
P(-=),
PIA
Clean prior probubility
A.2 MORE FAIRNESS DEFINITIONS
We present the full version of fairness definitions and the corresponding matrix form for DP, EOd, and EOp as follows.
Fairness Definitions. We consider three group faimess (Wang et al. 2020 Cotter et al., 2019)
definitions and their corresponding measurable metrics: demographic parity (DP) (Calders et al,,
2009 Chouldechova|2001 (011)) equalized odds (EOd) (Woodworth et al. 2017), and equalized oppor-
tunity (EOp) (Hardt et al. 2016)).
Definition (Demographic Parity). The demographic parity metric of f on D conditioned on A is:
PP (P(D).) M(M 1)K
Σ
[P(((X) = k|A = a) P(f(() = =|A =|A=a))
a.a'ElM kE(K]
Definition (Equalized Odds). The equalized odds metric of f on 'D conditioned on A is:
A EOd (D)( = M(M - 1)K2
Σ E
IP((() = k/Y y, 4 a) a)-P( = -
kE RE(K])EE(K]
15

---

Under review as a conference paper at ICLR 2023
Definition 5 (Equalized Opportunity). The equalized opportunity metric of f on D conditioned on is:
A EOP (D, M(M - e.a'E|M]
Σ [P(f(X)= = 1| 1, -. - 1|Y = 1, = a')l
Matrix-form Metrics. To unify three fairness metrics general form, we represent them with matrix H. Each column of H denotes the probability needed for evaluating faimesss with respect to classifier prediction f(X). For DP; H ki] denotes the following column vector:
H k] = (x = k) A 1), P((() k|A M))*
Similarly for EOd and EOp, let A @ K (k- + y be the 1-d flattened index that represents the 2-d coordinate in fXX x Y, H k @ y] is defined as the following column vector:
H 2 (X) k|Y y. y. 1), P(f(X) k = y-
The sizes of H for DP, EOd and EOp are M K, M K², and M 1 respectively. The noise transition matrix related to EOd and EOp is Tkwy where the (a, ā)-th element is denoted by Tkey (a, a| P(Á ( Y y, A a).
A.3
COMMON CONDITIONAL INDEPENDENCE ASSUMPTION IN THE LITERATURE
We present below a common conditional independence assumption the literature (Awasthi al. 2021 Prost et al. 2021 Fogliato et al. 2020). Note out framework successfully drops this assump- tion.
Assumption 3 (Conditional Independence). À and f(X) are conditionally independent given A (and Y for EOd, EOp):
DP: P(Ā a\) = - -= -; a|A = a), Va, a [M] [K].
(i.e.A L. [()))A).
EOd/ EOp: xx)=)((x))k;; y, = a) P(Ā a|Y y, a), [M],k e [K].
(i.e.ā L f(X))Y, A).
B PROOFS
B. FULL VERSION OF THEOREM AND ITS PROOF
Denote by Ty the attribute noise transition matrix with respect to label y, whose (a, ā)-th element
Ty (a, a] == P(Ā A 2|A a. Y y). Note different from Tg. Denote by Tx the attribute
noise transition matrix when f(X) k and Y y. where the (a, ä)-th element is T Teesy (a, ā]
P(Ā (X) k, = y, a). Denote by Py = [P(A 1|Y y), ... P(A K|Y
v)] and p. = - 1 = y), P(A K = = the clean prior probabilities and noisy prior probability, respectively.
Theorem (Error Upper Bound of Noisy Metrics), Denote by Err.aw := (6 (PD,
the estimation error of the directly measured noisy fairness metrics. Its upper bound is:
DP:
Σ KE]K]
Tip 1A -
1JA tōp T
K
cond. indep. violation
arror of
where h4 = Σ
H(a, kl, 8g == max
H H[, k] h2l
ME[M]
M
16

---

Under review as a conference paper at ICLR 2023
EOd:
Erd -
hkov |A (Ty TRey I)A p),,,,11 tok@y A Tkey A p. I112
K²
ke[R]]NE[K]
cond. indep. violation
of g
where hxzv == M
Σ H(a, k & y). ōkØy == max [Ha, @ y) - hNSOl a
aE[M
EOp: We obtain the result for EOp by simply letting k = and L, i.e..
Errgop 2
Σ
huooy |A (T TTyTTTess keyy
D)A
111 + tRROOw A T Tww A Pv.
Ill1
Py
k=1.y=1
cond. indep.
emor of
where hEsy"
1 M
E aE|Ms]
H(a, k y). ōkSy
H[a, y] hvsul- aE(M
max
Proof. The following proof builds on the relationship derived the proof for Theorem encourage readers to check Appendix B.2 before the following proof.
We
Recall Ta (P(Á a' A a, Y v)- Note
Ap, T. Ap, + (T Ap, -
Ap,1
Denote by
: k 2 y) hkov1 Vk09'
where haooy :- TV EEEIM (( = A = a, - =y). We have
Ap, H H s0 y) = hksy Ap, Ap, vkay hu )-1)) + Ap, Bkay
We further have
H &
A TT
Ap, 1) H H
p,
k@w
-TT kBy (T A 1 + A koy Ap, Useoy Tuxcoyy
Ukevy + H.;: k @ u]
-hasy b, (Tkey (T,) )
Ap,11 1
A
'T koy BoyyAp Ap, - 1) Ukoy H k y).
Noting |A| B - A + Bl < A +
|B/ we have
IA B - |B $ |A]. Therefore,
(ea - ea. H t: 2 m|
- (ea -wa) H ② w|l
show (ea ex.)
A px
(Ty -xkyy - 1) Ap,
(Term 1)
+ (ea eT))
(A kzy Ap, - 1) 1x@y
(Term 2)
Term-l and Term-2 be upper bounded as follows.
Term 1: With the Hölder's inequality, we have
h
(Ea
ear)
A A Pv
(T. 1)
Ap, 1
< <haoy
lea
earll
| 1 (T
Thee I) Ap,
<2
-1 (T TThey -
A p. 10
<2h
-1 (T- 10-TTss Thsy 1) p,
(T-IT
-2h
Ap, (T, T Tkyy
I) A py
17

---

Under review as a conference paper at ICLR 2023
Term 2: Denote by dkey
max [H I &y aE[M]
hkovl which is the largest absolute offset from
its mean. With the Hölder's inequality, we have
(ea ea.)
A
By
T
kwy
pu
< lea ex || || (A
key Ap,
koy
<2
A vs
'Ti key App - 1) UkSy
00
=26k80
Ap A koy
A Ap,
260
Ap, Tk:yy A Ar,
Wrap-up:
(ea -xx))
H|: @ vl -((ea -x) H H
<2haov
Ap,
( Tkyy 1) A T. + 20k@y
A Ap.Tkov pr
1/15
Denote by A = H
H a', k @ yll the noisy disparity and 4 kgy = | [ k 2 yl
a.a'
H k @ y] the clean disparity between attributes and ä' in the case when f(X) = k: and Y = y- We have
Ä EOd (D,f) A EOd (D.
1
Σ
A kSy
Aa.ā'
a.a' k@y
M(M - K
à, R.E(KK
2
Σ
Tree |Ap, (T (TRoy 1) A + 5Kroy
Ap, ARo
=11))
M(M 1)K2
[.NEE[M].ANV(K]]
Σ
hhasv
Ap,. (Ty TTkey
1) A=1 Px
+ ôK30 Tkey
A= Pv
11 111))
K2
k.yE[K]
The results for DP can be obtained by dropping the dependence on Y y. and the results for EOp can be obtained by letting k = -1 and 1.
B.2 FULL VERSION THEEREM AND ITS PROOF
Recall p. p, T and T3 clean prior, noisy prior, global transition matrix, and local transition matrix defined Sec. 2. Denote by Ap and Ap the square diagonal matrices constructed from P and p.
Theorem 2. [Closed-form relationship ED..EOd. The relationship between the true fairness vector h" and the corresponding noisy fairness vector h. writes as
===
(T TT A
Vu E (DP, EOd, EOp),
where Ag and Ap denote the square diagonal matrix constructed from D" and p", u unifies different fairness metrics. Particularly,
DP (vk E [K]): DP- == -P = =),,... ,P(A= = M)] 8 == (PĀ= P(Ā
M)J.
DP Thi where the (a, a) element of is Ta == P(A= -, -,
h DP = H[::k]] = [P(((X) = k/A = 1)....................... P((xx., = =A|4 -M]
DP - (P((X) = = 1
M)JT.
18

---

Under review as a conference paper at ICLR 2023
EOd and EOp (Vk, y [K], E {EOd, EOp)): yk.y E [K] koy K(k 1) y,
p" Py == [PAA 1|Y = y). ..... P(A M y)] p" p=
== P = II((A 1
y). ... IP(A= = M\Y = w)]T. T := TRey. where the (a, a--th element TfTss is Teosy (a, a ==
P(Á alf(x) = = =;,A= a).
h" = H = [P((() = k-Y v. -((()) - -aY = -vAA=Ma
h = H 2 vl = (P(()) = k = -,, "),"",,PU(X("" = =KYY u -,AA=M)
Proof. We first prove the theorem for DP, then for EOd and EOp.
Proof for DP. In DP, each element of DP satisfies:
P(f(X) k|Ā= a)
EnEIM P([(X) = k A ū. A= aa)
P(A ā) EMEM P(Ā= ālf
k. A
P(F((( k|A a)
P(A =1=0] à)
Recall Tg is the attribute noise transition matrix when f(X) k. where the (a, a)-th element is Tala, a| == P(Á (X) k, A = a). Recall p == (P 1). (PA M)] T and p :=
[P(A = 1), ... , IPAÁ = M)] the clean prior probabilities and noisy prior probability, respectively.
The above equation can be re-written matrix form
H Ap
T
A H k),
which is equivalent to
H k] = ((T App H: kl.
Proof for EOd, EOp.
In EOd or EOp, each element of h satisfies:
P(f(X) = K|Y = y, A -= a)
P((()) = = A= à) IP(Y 1, A= ā)
EnEiM P(((X) k. y, ā, A 0)
P(Y = A =āā à)
EnEtM| a(A=x k,,Y = = a) IP(Y y. y,A= a))a) P(((X) k/Y = y,
P(Y = y. ā)
Denote by Tkwy the attribute noise transition matrix when f(X) k and Y = where the (a, a)-th element is T Tk (a, a] P(Ā a( = k, y, A a). Denote by Py [T A 1 v). P(A = K|Y = T)]TT and Py := [P 1 = y) ... PPA = K|Y = y)] the clean
prior probabilities and noisy prior probability. respectively. The above equation can be re- re-written as a matrix form as
H A Ap
T keby Ap, H|
which is equivalent to
H k]
(T ( Ap,) p.))
Ap, H(.Ak.
Wrap-up.
We can conclude the proof by unifying the above results with
19

---

Under review as a conference paper at ICLR 2023
B.3 PROOF FOR COROLLARY I
Proof. When the conditional independence (Assumption 3
v), Va', a E [M] holds, we have Ty T Toooy and Term-l in Theorem I can be dropped. For Term-2, to get a tight bound in this specific case, we apply the Hölder's inequality by using &e norm on eg- Ea i.e.
P(A = a' =
= v) = = (Ā = d' A a, -(x)
K, Y
(ea ea.)
[A (Ap,
AT
keey Ap, 1) vkey
lea-
ear earllo0
(A,,
Tky Ap, 1) Un-
p,
(A (1-
AByy Ap, 1) Ukey
< K ÖHey
A Aj
TT Tessy Ap, - 1
=
K . dReey
Ap, Thesy
A -1 - -1
P.
Therefore,
- (D0 n)
A (Edd(D, (
K ÓNSy
|Ap TKSy A 12 -
k
Σ
ōkeu
App Ty. -1 - I
By
k.BE(K]
|i - I
K k.yE(K]
where T. PAA A a, Y = y)
Special binary case in DP In addition to the conditional independence, when the sensitive at- tribute is binary and the label class is binary, considering DP, we have
A (D.f)
DP (D. - 20, T 1 |
where āa = PPA a Ā = a). Let T [1,2] = c1, 1,, = 62, we know
T = 1 1- 62 el
1 1-e1
and
O (D. (D. s 28, (e1 + ez).
Note the equality in above incquality always holds. To prove it, firstly we note
P(f(X) k|A a)
EnEiM P( = - A= a)
ā.
P(A =ū) a)
EMEIM MI P(A = al5(x)
hi A 1= = al)=)) o(A= P((X) k| = a)
P(A= =)) ā)
EneiM P(A â|A a) P(A = a) P(f(X) = =)
P(A= a)
=E P(A= PP((() k),=
ME[M]
20

---

Under review as a conference paper at ICLR 2023
L.e. H Hi,kk] TT H[: Denote by : 1] [h, TT We have (a a')
(ea - - H : 1]
= |/ - 111 11 61 - e2l
and
((ea ewi
H [i,
1/1 1//.
Therefore, letting
1, 2, we have
DP (D [) A AD)(DD))
Σ
(ei - 02)
H k. - (e e2) H[:
2
kE(122)
(e) - 02) HH(
11
[ee| e2) H 1]
=|h 1/ M1 le + t
=8 (el + C2),
where 8 [P( 1 1) P( 1/A = 2). Therefore, the equality holds.
B.4 PROOF FOR THEOREM
Theorem 3 (Error upper bound of calibrated metrics), Denote the error of the calibrated fairness metrics by En ("( (D, It can be upper bounded as:
DP:
2
Errop
Σ
||p
|
8 PT. D).
K
kE[K]
where E <TT,. p) = || A
111
|| &
- T 1| is the error induced by calibration.
EOd:
Erreod
cal S
2
Σ
|
H| H|s s villso (TTks..P.)
K2
kE(K),
where (Troyy p.) = || p, Ap, Ill
T T 11 + Trooy To is the error induced
by calibration.
EOp:
Enrey 2
Σ
|
Ap, Hi: 1l e <TTrooy Pw),
k=1yy=1
where E T(TROV. py) = || Ap,
[
T
Tkoy T key 11 is the error induced 11 + + kgy
by calibration.
Proof. We prove with EOd.
Consider the case when f(X) = k and Y = y. For ease of notations, we use T to denote the estimated local transition matrix (should be Tkey). Denote the noisy (clean) fairness vectors with respect to f(X) is and Y = yyby by h (h). The error can be decomposed by
(e_ ear) A (A
A Apph)) Rn) - (ea eT))
Ap,h)
(ea - ear) T (A77) A (TT -A h
(ea ew)) (T (e, ew:) (T
-1
Temm2
21

---

Under review as a conference paper at ICLR 2023
Now upper bound them respectively.
Term-1:
(ea )x)) T (AA7)) - -T,) - 11)
(ea ) T (AA7) - -Ap,) (Tryy
key / /T A Ap H|: k 60 wi)
Dv
(b)
(ea ea') T (A Ap.
1) 1)Ap Ap, H [i @ ml)
p, Ap, - I)
A
-
|Tbll A|,
H |: t @ ulll
Py
-2
A pu A| Hii k @ U11l
20 (1 - 1 1le [T<|)1]
where equality (a) holds due to
P, k1 = Tre Ap, H k: 69 yl
and equality (b) holds because we denote the eπror matrix by T3. i.e.
T= T + T3 = Tk T
Term-2:
Before preceeding, we introduce the Woodbury matrix identity:
A--VVV) = A-'-AA -- + VA VA-
Let A == T KeoN'
C= I, = I. - Theyy By Woodbury matrix identity, we have
(T )
= =(T koy
+ (T TTeen)) 1
I(T -Tiso) (I+ (T+(T))))) (- -Troo)) -1
-(Twoy)
(T
(Trbo))
Term-2 can be upper bounded as:
(en - e (Ap. 1 R - [ee. - -car" (Api (Tbb)) -Ap,
(e)
(en - e (AD) ( (Iribe) -
-((hss)) (' -ess)) (I+ (TbSs) -1 '(T - -Irbu)) (Ihee) Ap,
(e. ea)) T (Tbbo)) App -Ap.Ti)
(ea ea) T
Ap (T -I( - -Taes) (I+ (Thew)
- -Theo)) (Irbe) -1A Fe
(6)
|,a. e |||
A (T -(TT -Troy) (I+ (Téow)
Tkeu)) )-1 (Tran) -1 AA Fr
<2
| 1
(ibb -1 ( -Tlow) (I + (Trbw) (T Iheew)) (TNbo) -1 Ap, Pr
-2 | 1
I + (Tees) ) T(f - -Tiba)) 1) (I (Thsu) -I(TT -Ttww)) (TTby)) A
-2 ||4 |1
I (I + (Thev)
-' -I(TT -Irew))
(Trbw) - Ap,77
20
-2 -2 |1
[ - T-To 11) (T -1 Ap, h
52 2
| - Tolli (Teau) Ap h
1x
|| Toll || Ap, Hi: @ oe
where the key steps are:
22

---

Under review as a conference paper at ICLR 2023
(a): Woodbury identity.
(b): Hölder's inequality.
(c): T T3 TTxy and triangle inequality
(d):
H 2 = Ts Hi,
+ (T Aし, Ab. H k
Ap, H | k
Wrap-up Combining the upper bounds of Term-l and Term-2, we have (recovering full notations)
Caa ex)) (A (TT)(
Ao h1
- (ea - -ea.)
AAp (T 11 h
<2
Ap k A ps Ap, - -1)
|Till + |1 -oll1
10
=2
Ap, H
|
ho vl
(1
177 Ap, - 1)
Tkoy Tody || + I- TeSsy kgy l
Ps
Denote by A á.a'
H t 69 U]l the calibrated disparity and A koy HH k y| H k @ w|l the clean disparity between attributes à and ă' in the case when f(X) = k and = y. We have
H Hak
kgy
Eod (D, A EOd (D.))
1
Σ a. G.R'E[M
Ad.a' KBy
22.0' kow
M(M 1)K2
Σ
A
Ap,
H|:
(/AA,AA,
1)
Troyy
T-1 kSy
+
||
Texy k
l.).
K2
[A
The above inequality can be generalized to DP by dropping dependency on y and EOp by requir- ing k = 1 and y
B.5 PROOF FOR COROLLARY
Proof.
Consider DP. Denote by H = = (T. We know 8 8= 11112 AD,
Suppose p < 1/2, Il 1/p and
A 2
([; Alll = max(ph, (1 (1-p)h').
Recall
(Tr = || Ap
ITI 111 || Te
By requiring the error upper bound in Theorem 3 less than the exact error in Corollary have (when k = 1)
2 A 11.00 ||
H [ Alll <TTr. 2 8 (c: + +2)
8. (e) + E2)
+* (Tr, p)
|A |1 AA H i kill
8 (Ei + +2)
+ (TT < max(h,
p))h/p))
If p 1/2, noting max(h, R1) (1h h' + 11 - h11)/2, we further have (when k 1)
1h - hil (el + +22
61 + e2
+
c eTT k. p) <
h' + 1 hl
K-
+
AUF(D.f)
hth
23

---

Under review as a conference paper at ICLR 2023
To make the above equality holds for all & (1, we have
C1 +
8 (TR.
S
(E((1.) 1 + A (D.))
max
H 1
Wk E {1,2}.
C MORE DISCUSSIONS ON TRANSITION MATRIX ESTIMATORS
C.I HOC
bb2 relies on checking the agreements and disagreements among three noisy at- HOC (Zhu et al.
tributes of one feature. For example, given a three-tuple (ā), a "²,), each noisy attribute may agree or disagree with the others. This consensus pattern encodes the information of noise transition ma- trix T. Suppose (ā 0 a
are drawn from random variables (A), A A$) satisfying Assumption
Denote by
e1 = P(A = 2|A 1) 2/ = 1) 2 P(A = 2 1)
and
2 P - PM 1 2).
Note A1
A2 = A3. We have:
First order equations:
P(A = 1I= (1- P(A = 2) .2
P(A 2) P(A' = =) P = 2)) (1--2))
Second order equations:
1) =) -
=
1, - = 2). -P(A' = -2)
=
-(A'=2)
Similarly,
P(A = 1, = (1-eiee P(A 1) + e2 P(A =2)
P( = 2, = = e1e P(A + (1 02) P(A' =
P(A = 2 = = =2 P(A = +)+ + (1 - c2) P(4 = 2)).
-P
Third order equations:
P(Á' = 1 = P = + 2)
P(A 1, 1,4 2) (1- R) -/A== P(A 2)
P(A" = 1, 2, = 2) = (1 )) ()===))((-<@)) P(A = 2)
P(A' = 2
1)
(1 e1) =1 P(A 2)
P(A' 2, - 1,
1)
(1 e P(A' +1)+(1+))))) P(A' 2)
P(A" 2, 1 = 2) = [1 e)
1)
P(A" = =2)
P( - = (1 e e (1 P(A 2) P(A'
= P(A 1) P(A 2).
With the above equations, we can count the frequency of each pattern (LHS) as (a!!), c!2) cl and solve the equations. See the key steps summarized in Algorithm
24

---

Under review as a conference paper at ICLR 2023
Algorithm 3 Key Steps of HOC
1: Input: A set three-tuples: {(a,,, an. [n la E [M]]
2: (alt), aldl, e <- Count Freq( (ã,, a (n) dn)ln E [M]) // Count 1st, 2nd, and 3rd-order patterns I/ Solve equations
3: Find T such that match the counts (c al2), cl
Table Normalized error (x100) of a learning- estimator.
Method
DP Global
DP Local
BOd Global
EOd Local
EOp Global
EOp Local
Base
15.33
4.11
/
2.82
Northcutt et al. 2021
15.37
15.49
4.07
4.02
2.86
2.95
Table 6: Disparities in the COMPAS dataset
True EOd
Uncalibrated Noisy
COMPAS
DP
EOp
DP
EOd
EOp
tree
0.2424
0.2013
0,2541
0.1362
0.1090
0,1
forest
0.2389
0.1947
0.2425
0.1346
0.1059
0.1120
boosting
0.2424
0.2013
0.2541
0,1362
0.1090
0.1160
SVM
0.2535
0.2135
0.2577
0,1252
0.0988
0.1038
logit
0,2000
0.1675
0.2278
0,1
0.0950
0.1120
nn
0,2318
0.1913
0.2359
0.1352
0.1084
0,1073
compas_score
0.2572
0.2217
0,2586
0.1511
0.1276
0.1324
C.2 OTHER ESTIMATORS THAT REQUIRE TRAINING
The other estimators in the noisy label literature mainly focus on training a new model to fit the noisy data distribution. The intuition is that the new model has the ability to distinguish between true attributes and wrong attributes. In other words, they believe the prediction of new model is close to the true attributes. It is useful when the noise attributes are random. However, this intuitions is hardly true in our setting since we need to train new model to learn the noisy attributes given by an auxiliary model, which are deterministic. One caveat of this approach that the new model is likely to fit the auxiliary model when both the capacity of the new model and the amount of data are sufficient, leading to a trivial transition matrix estimate that is an identity matrix, i.c., T = I. In this case, the performance is close Base. We reproduce Northcutt al. 2021) follow the setting in Table [ and summarize the result in T which verifies that the performance of this kind of approach is close Base.
D MORE EXPERIMENTAL RESULTS
D.1 MORE TABLES FOR THE COMPAS DATASET
We have two tables in this subsection.
Table o shows the raw disparities measured on the COMPAS dataset.
Table 7 is the full version of Table
D.2 MORE TABLES FOR THE CELEBA DATASET
We have three tables in this subsection.
Table g is the full version of Table 2
Table 9 is the full version of Table 3
Table 10 is similar to Table but the error metric is changed to Improvement defined Sec-
tion 5.1
25

---

Under review as a conference paper at ICLR 2023
D.3 MORE DISCUSSIONS
We cross-reference different tables and show several takeaway messages. Note some of them have been introduced in the main paper.
Uncalibrated measurement is sensitive to noise rates and raw disparity The target models trained on the COMPAS dataset are usually biased (details in Table [ and the auxiliary models are inaccurate (accuracy 68.85% in binary classifications), while f is almost not biased in EOd and EOp (details in Table 8) and is accurate (accuracy 92.55%). As a result, all the three types of directly measured fairness metrics (Base) have large normalized errors (~ 40-60%) as in Table moderate normalized error in DP (15.33%), and small normalized errors in EOd (4.1 (%) and EOp which is consistent with our results in Theorem I and Corollary [
(2.82%) as in Table 2
Local vs. Global Table also shows our Global method works consistently better than the Local method, while Table has the reversed result. Intuitively, when the auxiliary models highly inaccurate (accuracy 68.85%), Assumptions 2 implementing HOC may not hold well in every local dataset, inducing large estimation errors local estimates and unstable calibrations. On the contrary, when the auxiliary models are accurate (92.55% accuracy in Table 2). Assumption always hold and most instances will satisfy Assumption auxiliary models 92 and 9a given 91, then Local will outperform since it can achieve O error if both assumptions perfectly hold and Global induces extra error due to approximation. Note Table shows Local is still statistically better than Global when the noise rate is high. This is because the extra random flipping follows Assumption 2 and the estimation error of Local is not improved significantly. Therefore, prefer Local when the original auxiliary model is accurate and Global to stabilize the calibration otherwise.
we carefully choose the other two
When our method is better Both Table DP in Table 2 and Table 3 show our method is
significantly better than both baselines, where the noise rates of are moderate high (e.g.. 2 15%) or f is biased (e.g.. 2 0.1). This observation is also consistent with our result in Corollary 2 In other cases when both the noise rate and original disparity are low, our calibration may not be perfect compared with others without calibration, e.g., EOp in Table 2 However, the raw error of EOp is sufficiently small (< 0.01) for all approaches, indicating the absolute performance of our method not bad although it fails to be better than others.
26

---

Under review as a conference paper at ICLR 2023
Table 7: Performance on the COMPAS dataset. The method with minimal normalized error is bold.
DP Normalized Error (%)
EOd Normalized Error (%)
EOp Normalized Error (%)
COMPAS
Base
Soft
Global
Local
Base
Soft
Global
Local
Base
Soft
Global
Local
tree
43.82
61.26
22.29
39.81
45.86
63.96
23,09
42.81
54,36
70.15
13.27
49,49
forest
43.68
60.30
19.65
44.14
45.60
62.85
18.56
44.04
53.83
69.39
17.51
63.62
bousting
43.82
61.26
22.29
44.64
45.86
63.96
23.25
49.08
54.36
70.15
13,11
54.67
SVM
50.61
66.30
30.95
42.00
51.72
69.69
32.46
47. 47.39
$9.70
71.12
29.29
51.31
logit
41.54
60.78
16.98
35.69
43.26
63.15
21.42
31.91
50.86
65.04
14.90
26.27
nm
41.69
60.55
19.48
34,22
43.34
62.99
19.30
43.24
$4.50
68.50
14.20
59.95
compas_score
41.28
58.34
11.24
14.66
42.43
59.79
11.80
18.65
48.78
62.24
5.78
23.80
DP Raw Disparity J
EOd Raw Dispority
Raw
EOp Row Dispority 1
tree
0.1362
0.0939
0.1884
0.1459
0.1090
0.0726
0.1548
0.1151
0.1160
0,0759
0.2204
0,1283
forest
0,1345
0.0948
0.1919
0.1334
0.1059
0,0723
0.1586
0.1090
0.1120
0.0743
0.2001
0.0882
bonsting
0,1362
0.0939
0.1884
0.1342
0.1090
0.0726
0.1545
0.1025
0.1160
0.0759
0.2208
0,1152
SVM
0.1252
0.0849
0.1750
0.1470
0.0988
0.0647
0.1442
0.1123
0.1038
0.0744
0.1822
0.1255
logit
0.1169
0.0784
0. 1660
0.1286
0.0950
0.0617
0.1316
0.1140
0.1120
0.0797
0.1939
0,1680
0.1352
0.0915
0.1867
0.1525
0.1084
0.0708
0.1544
0.1086
0.1073
0,2024
cumpass.scre
0.1510
0,1072
0.2283
0.2195
0.1276
0.0891
0.1955
0,1803
0.1324
0,0976
0,2436
0,1970
DP Raw Error J
EOd Emoor
EOp Row Error
0.1062
0.1485
0.0540
0.0965
0.0923
0,1288
0.0465
0,0862
0.1381
0.1782
0.0337
0.1257
forest
0.1043
0.144n
0.0469
0.1054
0.0888
0,1224
0.0361
0.0858
0.1306
0.1683
0.0425
0.1543
boosting
0,1062
0,1485
0.0540
0.1082
0.0923
0,1288
0.0468
0.0988
0.1381
0.1782
0.0333
0.1389
SVM
0,1283
0.16
0.0785
0.1064
0.1
0,1488
0.0693
0.1012
0.1538
0.1833
0.0755
0.1322
logit
0.0831
0.1215
0.0340
0,0714
0.0724
0,1057
0.0359
0.0534
0.1159
0.1482
0.0339
0.0598
mn
0,0966
0.1404
0.0452
0.0793
0.0829
0.1205
0.0369
0.0827
0.1286
0.1616
0.0335
0,1414
0.1062
0.1500
0.0377
0.1325
0.0261
0,0413
0.1261
0.0150
DP Improvement (%)
EOd Improvement (%) 1
EOp Improvement (%)
Iree
0.00
.39799
49.15
9.15
0.00
-39.48
49.65
6.64
0.00
-29.05
75.60
8.96
forest
0.00
.38.05
55.01
-1.06
0.00
-37.83
59.30
3.42
0.00
-28.89
67.47
-18.18
boosting
0.00
-39.79
49.15
-1.87
0.00
-39.48
49.30
-7.04
0.00
-29.05
75.89
-0.57
SVM
0.00
-31.40
17.02
-29.72
11.78
0.00
50.93
14:05
logit
0.00
-46.30
59.12
14.08
0.00
-45.98
50.47
26.24
0.00
-27.87
70.70
48.35
0.00
-45.23
17.93
-45.34
55.47
0.23
-25.69
73.94
compas.acuur scure
0.00
-41.33
72.77
64.48
0.00
-40.92
72.20
56.04
0.00
-27.59
88.15
51.21
Table 8: Performance on the CelebA dataset. The method with minimal normalized error is bold.
DP Normalized Erar 4
EOd Normulized (%) J
EOp Normalized
CelebA
Base
Soft
Global
Local
Base
Soft
Global
Local
Base
Soft
Global
Local
15.33
12.54
10.89
6.46
2.82
0.34
12.22
Facenet512
15.33
12.54
21.70
7.26
4.11
6.46
4.85
0.52
2,82
0.34
11.80
3.24
OpenFace
15.33
12.54
10.31
9.39
6.46
10.43
2.82
0.34
0.56
ArcPace
15.33
12.54
19.59
9.69
4.11
6.46
5.72
0.23
2.82
0.34
11.16
3.85
15.33
12.54
15.09
5.30
6.46
4.87
2.82
0.34
9.74
SPace
15.33
12.54
17.00
4.77
4.11
6.46
4.04
3.91
2.82
0.34
9.36
3.28
DP Raw Disparity 4
EOd Raw Disparity
EOp Row Disparity
Facenct
0.1522
0.1485
0.1612
0.1464
0.0316
0.0309
0.0355
0.0331
0.0573
0.0559
0.0625
0.0573
Facenet512
0,1522
0.1485
0.1606
0.1416
0,0316
0.0309
0.0346
0.0328
0.0573
0.0559
0.0623
0.0575
OpenFace
0.1522
0.1485
0.1456
0.1444
0.0309
0.0559
0.0554
ArcFace
0,1522
0.1485
0.1578
0.1448
0.0316
0.0309
0.0349
0.0329
0.0873
0.0559
0.0619
0.0578
Dlib
0.1522
0.1485
0.1519
0.1390
0.0316
0.0309
0.0346
0.0316
0.0573
0.0559
0.0611
0.0544
Stace
0.1522
0.1485
0.1544
0.1383
0.0316
0.0309
0.0343
0.0317
0.0573
0.0559
0.0609
0.0539
DP Row Erorr 4
Raw Error 4
EOp Raw Error J
0.0202
0.0165
0.0293
0.0144
0.0021
0.0025
0.0016
0.0002
0.0068
0.0016
Facenet512
0.0202
0.0165
0.0286
0.0096
0.0014
0.0021
0.0016
0.0002
0.0016
0.0002
0.0066
0.0018
OpenFace
0.0202
0.0165
0.0136
0.0124
0.0014
0.0021
0.0017
0.0016
0.0002
0.0003
ArcFace
0.0202
0.0165
0.0259
0.0128
0.0014
0.0021
0.0019
0.0001
0.0016
0.0002
0.0062
0.0021
0.0202
0.0165
0.0199
0.0070
0.0014
0.0021
0.0016
0.0014
0.0016
0.0002
0.0054
SPace
0.0202
0.0165
0.0224
0.0063
0.0014
0.0021
0.0013
0.0013
0.0016
0.0002
0.0052
0.0018
DP Improvement (%))
EOd Improvemnut (%)
EOp Improvement (9k
Facenct
0.00
18.22
-44.58
28.
0.00
-57.38
-83.62 62.62.2
93.64
0.00
88.05
-333,85
-3.97
Facenet512
0.00
18.22
-41.50
52.65
0.00
-57.38
-18.15
87.29
0.00
88.05
-319.18
-15.09
OpenFace
18.22
-57.38
-154.12
.22.45
80.03
ArcFace
0.00
18.22
-27.78
36.78
0.00
-57.38
-39.45
94.31
0.00
88.05
-296.25
-36.65
Dlib
0.00
18.22
1.56
65.46
0.00
-57.38
-18.55
-3.43
0.00
$8.05
-245.95
17.61
SFace
0.00
18.22
-10.87
68.87
0.00
-57.38
1.61
4.85
0.00
88.05
-232.48
-16.46
27

---

Under review as a conference paper at ICLR 2023
Table 9: Normalized Error on CelebA with different noise rates
Normalizze Eraor 4
EOd Normalized Evnor (%) L
EOp Normalized Eraor (%)
CelehA
Base
Soft
Global
Local
Base
Soft
Global
Local
Base
Soft
Global
Local
Facenet [0.0, 0.0]
15.33
12.54
22.17
6.46
7.54
0.26
2.82
0.34
12.22
2.93
Facenet [0.2] 0.0]
7.39
11.65
20.75
10.82
25.05
26.99
9.87
6.63
24.69
27.27
11.55
2.77
Facenct [0.2, 0.2]
30.24
31.57
24.27
8.45
44.71
46.36
15.10
3.99
37.67
38.77
21.79
16.73
Facenet [0.4,02]
51.37
54.36
20.12
20.66
62.94
65.10
3.45
3.67
56.53
58.73
15.75
2.70
Facenet 0.41
77.82
78.39
8.76
21.94
79.36
80.10
51.32
148.05
78.39
79.62
71.38
146.20
Facenet5 [0.0,
15.33
12.54
21.70
1.26
6.46
4.85
0.52
2.82
0.34
11.30
3.24
Facenet512 12 [0.2. 0.0]
7.37
11.65
20.58
5.05
25.06
26.99
6.43
0.10
24.69
27.27
11.11
1.07
Facenet512 [0.2.
30.21
31.57
24.25
13.10
44.73
46.36
11.26
9,04
37.67
38.77
20.94
27,98
Facenet512 [0.4, 0.2]
51.32
54.56
19.42
10:47
62,90
65.10
11.09
19.15
56.51
58.73
23.86
23.55
Facenet512 [0.4, 0.4]
77.76
78.39
9.41
79.31
80.10
8,02
78.35
79.62
10.61
OpenFace [0.0, U.U]
15.33
12.34
10.31
9.39
4.11
6.46
10:43
5,03
2.82
0.34
0.56
0.93
OpenFace [0.2. 0.0]
7.39
11.65
8.93
6.60
25.05
26.99
9.86
13.01
24.69
27.27
1,08
10.96
OpenFace [0.2. 0.2]
30.24
31.57
13.32
21.46
44.74
46.36
7.56
15.88
37.69
38.77
5.90
7.40
OpenFace [0.4, 0.2]
51.39
54.56
10.66
25.16
62.96
65.10
6.47
24.94
56.55
58.73
6.11
47.12
OpenFace [0.4. 0.4]
77.84
78.39
1,60
11..27
79,38
80.10
34.00
19.47
78.41
79.62
37.42
31,99
ArcFace [0.0, 0.0]
15.33
12.54
19.59
9.69
4,11
6.46
5.72
0.23
2.82
0.344
11.16
3.85
ArcFace [0.2, 0.0]
7.39
7.74
26.99
24.69
27.27
8.81
3.37
ArcFace [0.2. 0.2]
30.19
31.57
21.77
8.97
44,77
46.36
12.12
18.91
37 69.69
38.77
21.19
17.99
ArcFace [0.4, 0,2]
51.32
54.56
17.33
44.52
62.91
65.10
14.60
29.74
56.53
58.73
24.39
4.92
ArcFace [0.4, 0.4]
77.79
78.39
8.38
84.37
79.34
80,10
8.31
165.03
78.39
79.62
16.98
62.34
Diab [D.0, 0X0)
15.33
12.54
13.09
5.30
4.11
6.46
4.87
4,25
2.82
0.34
9.74
2.32
Dlib [0.2, 0.0]
7.35
11.65
14.39
1.06
25.07
26.99
3.78
2.63
24.69
27.27
7.09
2,36
Dlib (0.2, 0.2]
30.23
31.57
16.78
1.95
44,77
46.36
9,50
11.28
37.72
38.77
15.88
22.43
Dlib 10.4.
51.40
54.56
12.83
10.34
56.57
58.73
18.90
11.17
Dlib (0.4, 0.4]
77.84
78.39
0.46
96.58
79.38
80.10
7.99
86.36
78.41
79.62
8.45
14.78
STace [0.0, 0.0]
13.33
6.46
3.91
2.82
0.34
9.36
SFace [0.2, 0.0]
7.41
11.65
15.18
1.94
25.04
26.99
3,31
8.82
24.69
27.27
7.24
13.05
SFace [0.2. 0.2]
30.22
31.57
18.16
20.95
44.72
46.36
4.58
20.93
37.67
38.77
11.55
34.72
SFace 10.4, 0.2]
51.35
54.56
14.72
48.96
62.92
65.10
2.95
68.93
56.51
58.73
15.22
68.85
SFace [0.4, 0.4]
77.78
78.39
3.37
31.25
79.33
80.10
21.56
178.21
78.37
79.62
20.03
86.59
Table 10: Improvement on CelebA with different noise rates
DP Improvement (%)
EOd Improvemnn (%) f
EOp Improvement (%)
CelebA
Base
Solt
Global
Local
Base
Solt
Global
Local
Base
Solt
Global
Local
Facenet [0.0, 0.0]
0.00
18.22
-44.58
-57.38
-83.62
93.64
0.00
88.05
-333.85
-3,97
Facenet [0.2.0] 0.0]
0.00
-57.70
-180,88
-46.45
0.00
-7.75
60.60
73.52
0.00
-10.44
53.24
88.80
Facenct [0.2. 0.2]
0.00
-4.39
19.75
72.05
0.00
-3.69
66.22
91.07
0.00
-2.92
42.17
55.58
Facenet [0.4. 0.2]
0.00
-6.20
60.83
59.79
0.00
-3.44
94.51
94.17
0.00
-3.90
72.13
95.23
Facenct [0.4. 0.4]
0.00
-0.73
88.74
71.81
0.00
-0,94
35.33
-86.56
0.00
-1.57
8.94
-86.50
Facemer5 12 [0.0,
0.00
18.22
-11.00
32.65
0.00
-57.38
-18.15
87.29
0.00
88:05
-319.18
-15.09
Facenet512 12 [0.2. 0.0]
0.00
-58.10
-179.28
31.43
0.00
-7.70
74.32
99.58
0.00
-10.44
54.98
95 05.68
Facemet512 [0.2, 0.2)
0.00
-3.64
0.00
44.40
Facenet512 [0.4, 0.2]
0.00
-6.32
62.17
79.60
0.00
-3.50
82.37
69.55
0.00
.3.44
$7.78
58.33
Facenet512 [0.4, 0.4]
0.00
$7.90
74.54
69.12
0.00
-1.63
86.45
92.71
OpenFace [0.0, 0.0]
0.00
18.22
32.76
38.75
0.00
-57.38
-154,12
-22.45
0.00
88.05
80.03
67.15
OpenFace [0.2, 0.0]
0.00
-57.70
-20,83
10.69
0.00
-7.75
60.65
48.05
0.00
-10.44
95.64
55.62
OpenFace [0.2, 0.2]
0.00
-4.8
55.97
29.06
0.00
-3.62
83.11
64.51
0.00
-2.66
84,35
80.38
OpenFace [0.4, 0.2]
0.00
-6.16
79.25
51 11.05
0.00
-3.41
89.72
60.39
0.00
-3.86
89.10 19
16.67
OpenFace [0.4. 0.4]
0.00
-0.71
97.94
-50.65
-0.92
57.17
75.47
0.00
-1.54
52.28
59.20
ArcFace [0.0, 0.0]
0.00
18.22
-27.78
36.78
0.00
-57.38
-39.45
94,31
0.00
88.05
.296.25
-36.65
ArcFace [0.2, 0.0]
0.00
-57.70
-4.72
-7.75
75.31
92.72
64.32
ArcFace [0.2,
0.00
-.56
27.91
70.28
0.00
-3.55
72.94
$7.76
0.00
-2886
43.79
52.27
ArcFace [0.4. 0.2]
0.00
-6.31
66.22
13.25
-3.49
$2.72
0.00
-3.90
56.
91.29
AscFace [0.4, 0.4]
0.00
-0.78
89.23
-8.47
0.00
-0.97
89.53
-108.01
0.00
-1.57
78.34
20.47
Dlib [0.0, 0.0]
0.00
18.22
1.56
65 46
0.00
-57.38
-18.55
-3.43
0.00
88.05
-245.95
17.61
Dlib [0.2. 0.0]
0.00
-58.50
-95,79
85 62.62
-7.66
89.53
0.00
-10.44
71.30
90.42
Dlib [0.2, 0.2]
0.00
-4.43
44.49
93.54
0.00
-3.53
78.78
74.80
0.00
-2.80
57. 89
40.54
Dlib [0.4, 0.2]
0.00
-6.15
75.03
65.59
-3.39
83.58
81.78
0.00
-3.82
66.59
80.25
Dlib [0.4. 0.4]
0.00
-0.71
99.41
-24.07
0.00
-0.92
89.94
-8,80
0.00
-1.54
89.22
$1.15
SFace [0.0. 0.0]
0.00
18.22
-10.87
08.87
0.00
-57.38
1.61
4,85
0.00
88.05
-232.48
-16.46
SFace 10.2. 0.0
0.00
-57.31
-104.91
73.84
0.00
-7.79
86.78
64.75
0.00
-10.44
70.66
47.12
SFace [0.2. 0.2]
0.00
-4.55
39.93
30.68
0.00
-3.67
89.76
53.18
0.00
-292
69.34
7.82
SPace 10.4, 0.2]
0.00
-6.24
71.34
4.66
0.00
-3.47
95.32
-9.55
0.00
-3.4
73.06
-21.85
SFace [0.4, 0.4]
0.00
-0.78
95.67
59.82
0.00
-0.98
72.82
-124.64
0.00
-1.60
74.44
-10.49
28

---

Under review as a conference paper at ICLR 2023
D.4 EXPERIMENTS COMPAS WITH THREE- THREE-CLASS CLASS SENSITIVE ATTRIBUTES
We experiment with three categories of sensitive attributes: black, white, and others, and show the result in Table 11 Note EOp is not defined in the case with more than two categories. Table shows our proposed framework with global estimates is consistently and significantly better than the baselines, which also consistent with the results from Table
Table 11: Normalized estimation on COMPAS. Each row is different target model f.
COMPAS
DP Normalized Eroo (%) |
EOd Normalized Frmr (%h
True dispority: 0.2
Base
Soft
Global
Local
Base
Solt
Global
Local
Iree
24.87
64.91
13.84
25.16
30.15
67.30
13.11
27.84
forest
23.94
63.61
10.00
26.64
29.19
65.73
11.61
33.85
boosting
24.87
64.91
13.84
25.44
30.15
67.30
15.74
33.20
SVM
40.37
66.97
25.96
34.73
49.57
70.58
29.33
42.91
logit
16.71
64.18
7.39
22.17
17.23
66.47
7.02
25.38
5.38
22.63
compas scooe
29.00
63.33
10.02
31.32
33.43
63.82
12.15
36.03
D.5 PRELIMINARY RESULTS ON DISPARITY MITIGATION WITH OUR CALIBRATION FRAMEWORK
We apply our calibration framework to mitigate disparity during training. Specifically, the local method is applied on the CelebA dataset. The preprocess of the dataset and generation of noisy sensitive attributes are the same as the experiments Table 2. The backbone network VIT-B_8 Dosoviiskiy et al. 2020 The aim is to improve the classification accuracy while ensuring DP, where A(D,)) is the constraint during training. Specifically, the optimization problem is
min Σ (((())) n=1
s.t. AD. =,
where E is the cross-entropy loss. Recall ((D,)) is obtained from our Algorithm (Line 8), and D = (Zn, Vn; ān)(n E [N]]. Noting the constraint is not differentiable since it depends on the sample counts, i.c.
N
1ff(z. klāū = a)).
N 10
To make it diffcrentiable, we use relaxed measure follows:
Madras et al. 2018 Wang et al. 2022 as
-((x)
Σ
[k)
1,
where fz, [k] is the model's prediction probability on class k. and Na is the number of samples that have noisy attribute à. The standard method of multipliers employed to train with constraints (Boyd et al. 1).1). We train the model for 20 epochs with stepsize of 256. Table 12 shows the accuracy and DP disparity on the test data averaged with results from the last epochs training. From the table, conclude that, with any selected pre-trained model, the mitigation based calibration results significantly outperforms the direct mitigation with noisy attributes in terms of both accuracy improvement and disparity mitigation.
29

---

Under review as a conference paper at ICLR 2023
Table 12: Disparity mitigation with our calibration framework. Results are averaged with results from the last epochs. DP the considered fairness metric. Base: Direct mitigation with noisy sensitive attributes. Facenet, Facenet 512, etc.: Pre-trained models to generate feature representa- tions used simulate the other two auxiliary models.
CelehA
A (1700) S
Accuracy
Base
0.0578
0.8422
Facenet
0.0453
0.8466
Facenet512
0.0273
0.8557
OpenFace
0.0153
0.8600
ArcFacee
0.0435
0.8491
0.8522
SFace
0.0315
0.8568
30


Reviews:
[{"review_id": "caiG3J8O9R9", "reviewer": "Reviewer_vWhJ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper focuses on learning the sensitive attributes using auxiliary models for estimating fairness metrics. Theoretical analysis is done to understand the relationship between the directly measured fairness metrics and their corresponding ground-truth metrics.", "summary_of_the_review": "The paper suffers from clarity, and the writing could be improved for better understanding. \nThe problem studied is of importance but would be helpful to add motivation with some problems from real-world datasets or settings.", "strength_and_weaknesses": "An important problem is studied, motivation from real-world settings is lacking.\nClarity in writing can be improved.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "ZO3ARpb7wN", "reviewer": "Reviewer_Za6Z", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper aims to evaluate the fairness of the model without accessing sensitive group attributes. The main idea is to utilize auxiliary models of estimating group attributes by calibrating their outputs. The paper first theoretically explains why we need calibration on the auxiliary models. Then, the paper suggests a calibration-based training framework that uses the existing distribution estimation techniques. The proposed algorithm is evaluated in two datasets, COMPAS and CelebA, and achieves a more precise evaluation when the auxiliary model is accurate and the target model is highly biased.", "summary_of_the_review": "The paper tries to solve the important issue, which evaluates model fairness in a real-world scenario where sensitive attributes are not available. As the paper gives several insights in their theoretical discussion, I vote for marginally accepting this paper if it can clarify the contribution of the proposed framework and enhance the experimental results (or discussion).\n", "strength_and_weaknesses": "[Strength]\n\nS1: The paper tries to evaluate the fairness of the models without accessing sensitive attributes, which is an important and practical issue. \n\nS2: The theoretical analyses in the paper give several nice insights. For example, I enjoy reading Section 3, which well analyzes the factors that affect the estimation error in model fairness.\n\nS3: The proposed algorithm is effective in specific empirical scenarios, as shown in their theoretical results.\n\n[Weakness]\n\nOverall, although the paper has various interesting aspects in its theoretical discussion, the proposed framework and experimental results are relatively weak.\n\nW1: The paper uses HOC as their estimator without comparing other methods. Although the paper explains some reasons for choosing HOC (e.g., free of training), it is not clear that the proposed framework can show similar results when using other estimators. The experimental results seem to be somewhat determined by the performances of HOC, so it may be questionable whether the current experimental results are dependent on HOC or not. It would be great if the paper can clarify the robustness of the proposed framework w.r.t. the estimators. \n\nW2: As shown in Tables 1–3, the performances when using global T and local T_k are very different, but there is no detailed discussion on how to choose either global or local. This result makes the framework less attractive, as the estimation errors are largely dependent on a specific hyperparameter (i.e., choosing global or local).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "NixxIT3oq5", "reviewer": "Reviewer_oWCf", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper the authors study how to compute different group fairness measures when relying on pre-trained models to predict group attributes.  The paper provides a general mathematical formulation that focuses on estimating the transition probabilities (essentially the normalized confusion matrix) between ground truth and predicted group information conditioned on main task class.  The paper then suggests multiple approaches for this estimation, including one from the noisy label literature, and empirically tests which methods work best.  They find that in some cases (some datasets and fairness goals) their method outperforms baselines.\n", "summary_of_the_review": "Overall the paper is a thorough study of an important problem, but the key contributions that move the field forward are hard to extract.\n", "strength_and_weaknesses": "S1. Technically challenging and important problem.\n\nS2. The formulation provided is general and clarifies what information is needed for accurate estimation.\n\nS3. The experiments provide a useful breakdown of when different methods excel.\n\nW1. Clarity: While the formulation is general, it is hard to parse in many cases the intuition behind many terms.  Further, a core contribution of the paper seems to be about leveraging noisy label approaches most significantly the HOC algorithm, but the algorithm is not at all described in the paper.\n\nW2. One core claim of the paper is that they don't need the conditional independence assumption of past literature but the \"Global\" method (which often works best at least for COMPAS) seems to directly be enforcing the conditional independence assumption of past work.  As such, it is not clear what is the core contribution of the paper.\n\nW3. Strongly strongly suggest a discussion of the limitation and ethical fraught-ness of using classifiers for sensitive attributes (eg predicting race based on name or gender based on an image).  This research is still important but there are many caveats with predicting these attributes in practice.\n\n\nDetails while reading:\n\nNot obvious how Thm 1 would be extended to equality of odds/opportunity.\n\nFor clarity purposes overall I think the paper would be easier to read assuming binary labels and groups, with generalizations to many groups being included later.\n\nSec 4.1 - Wouldn't use T to estimate all $T_k$ be exactly the conditional independence assumption from past work as described in Sec 2 (Transition Matrix)?\n\nAlso how would $T_k$ be estimated based on $\\tilde{D}$? I thought $T_k$ needs both $A$ and $\\tilde{A}$ as it is the transition probabilities between the two.\n\nHOC seems critical to the method in this paper but is not described.  For completeness please include\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "7cG9V3AGUGB", "reviewer": "Reviewer_pHkG", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This proposed a framework to evaluate fairness when ground-truth sensitive attributes are not accessible. It calibrates the noisy fairness metrics using auxiliary models only and it drops many assumptions. The auxiliary model is flexible, the auxiliary model can be used if the input features have some overlaps with targets. ", "summary_of_the_review": "This work proposes a new method with fewer assumptions to estimate fairness metrics in the setting of missing sensitive attributes using auxiliary models. The theoretical analytics and the experimental results show the effectiveness. The proposed method is a plug-in method which means it is flexible.\n\nIt would be more complete and convincing if the estimated metrics are used for bias mitigation.", "strength_and_weaknesses": "Strength:\n\n+ It proves the proposed method has less error than the direct estimate model under a threshold.\n+ The proposed method is a plug-in method, it can be used as long as the auxiliary model’s input features have overlap with the target features.\n\nWeakness:\n\n- This paper removes some assumptions but the common assumption on T is still very strong.\n- The proposed research is evaluated on experiments with binary classification and binary sensitive attributes.\n- This work estimates the unfairness metrics in the missing sensitive attribute setting. As a framework, it would be more convincing if the mitigation solution is also provided.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject