FairCal: Fairness Calibration for Face Verification
Authors: Tiago Salvador
Stephanie Cairns
Vikram Voleti
Noah Marshall
Adam M Oberman
Venue: ICLR 2022 Poster (2022)
Keywords: face verification
bias
fairness
clustering
calibration

Abstract: Despite being widely used, face recognition models suffer from bias: the probability of a false positive (incorrect face match) strongly depends on sensitive attributes such as the ethnicity of the face. As a result, these models can disproportionately and negatively impact minority groups, particularly when used by law enforcement. The majority of bias reduction methods have several drawbacks: they use an end-to-end retraining approach, may not be feasible due to privacy issues, and often reduce accuracy. An alternative approach is post-processing methods that build fairer decision classifiers using the features of pre-trained models, thus avoiding the cost of retraining. However, they still have drawbacks: they reduce accuracy (AGENDA, FTC), or require retuning for different false positive rates (FSN). In this work, we introduce the Fairness Calibration (FairCal) method, a post-training approach that simultaneously: (i) increases model accuracy (improving the state-of-the-art), (ii) produces fairly-calibrated probabilities, (iii) significantly reduces the gap in the false positive rates, (iv) does not require knowledge of the sensitive attribute, and (v) does not require retraining, training an additional model or retuning. We apply it to the task of Face Verification, and obtain state-of-the-art results with all the above advantages.

Published as a conference paper at ICLR 2022
FAIRCAL: FAIRNESS CALIBRATION FOR FACE VERIFI- CATION
Tiago , SStepaanie Cairns 133, Vikram Voleti Noah Marshall Adam Oberman'
1 McGill University Université de Montréal 3 *iia
1,3 (tiago.se danhassa vador, stephanie.... rns, noah. maaehhmmaraaalll 112] 8ma www..im.i...... 2,3 vikram. voleti @gmmmmll...... il. com, 1.3 adam. oberman@megi11..c
ABSTRACT
Despite being widely used, face recognition models suffer from bias: the probability of a false positive (incorrect face match) strongly depends on sensitive attributes such as the ethnicity of the face. As a result, these models can disproportionaaely and negatively impact minority groups, particularly when used by law enforcement. The majority of bias reduction methods have several drawbacks: they use an end-to- end retraining approach, may not be feasible due to privacy issues, and often reduce accuracy. An alternative approach is post-processing methods that build fairer decision classifiers using the features of pre-trained models, thus avoiding the cost of retraining. However, they still have drawbacks: they reduce accuracy (AGENDA, PASS, FTC), require retuning for different false positive rates (FSN). In this work, we introduce the Fairness Calibration (FairCal) method, a post-training approach that simultaneously: (i) increases model accuracy (improving the state- of-the-art), (ii) produces cailby-al probabilities, (iii) significantly reduces the gap in the false positive rates, (iv) does not require knowledge of the sensitive attribute, and (v) does not require retraining, training an additional model, or retuning. We apply it to the task of Face Verification, and obtain state-of-the-uu results with all the above ad advantages.
INTRODUCTION
Face recognition (FR) systems are being increasingly deployed worldwide a variety of contexts, from policing and border control to providing security for everyday consumer electronics. According to Garvie et al. (0166), face images of around half of all American adults are searchable in police databases. FR systems have achieved impressive results in maximizing overall accuracy (Jain et al., 2016). However, they have also been shown to exhibit significant bias against certain demographic subgroups (Buolamwini & Gebru 2018 Orcutt 2016 Alvi al., 2018), defined by sensitive attribute such as ethnicity, gender, or age. Many FR systems have much higher false positive rates (FPRs) for non-white faces than white faces (Grother al. 2019). Therefore, when FR is employed by law enforcement, non-white individuals may be more likely to be falsely detained (Allyn 2020). Thus, is of utmost importance devise an casily-implementable solution mitigate FR bias.
Most efforts to mitigate FR bias have been directed towards learning less biased representations
(Liang et al. 2019; Wang et al. 2019b Kortylewski et al. 2019; Yin al. 2019) Gong al. 2020
Wang & Deng 22200] Huang et al.) 2020 Gong et al. 2021). These approaches have enjoyed varying degrees of success: though the bias reduced, so the accuracy (Gong et al.) 2020). They often
require retraining the models from scratch, which is computationally expensive. Moreover, they
often require the sensitive attribute of the face (such as ethnicity) (Gong et al., 2020; Dhar et al.)
2020 Terhörst et al. 2020a Dhar et al.) |021 Robinson et al. (222), which may not be feasible to obtain. Hence, an approach that improves fairness in existing models without reducing accuracy, or knowing the sensitive attribute, require retraining, is missing.
In this work, we focus on the face verification problem, a major subproblem in FR: determine whether two face images depict the same person. Current ofate-of-the-ee art (SOTA) classifiers are based on deep neural networks that embed images into a now-dimensional space (Taigman et al., 2014). The "Baseline" method is that deems pair of images is match if the cosine similarity between

---

Published as a conference paper at ICLR 2022
Baseline
AGENDA
FTC
FairCal (Ours)
False Positive Rate
Aliicnn Asiani
African
Arican
Air can
African
0.15
Asion:
Asian
Asian
Asian
Indian
Indian)
Indian
Indian
0.10
Caucasien
Cauces en
Cautasion
Ceucesian
Caucasian
0.05
0.00
0.00 0.00 0.02 0.44 o.ee 0.06 0.10 0.02 2.00 0.04 0.30 0.05 0:30 Global FPR
Global FPR Figure 1: (Lines closer together is better for fairness, best viewed in colour) Illustration of improved
Global FPR
Global FPR
Global FPR
fairness / reduction in bias, as measured by the FPRs evaluated on intra-ethnicity pairs on the RFW dataset with the FaceNet (Webface) feature model. The significant subgroup bias in the baseline method is reduced with post-processing methods AGENDA (Dhar et al., (2020). FTC (Terhörst et al. 2020ab, FSN (Terhörst al. 2020b), and FairCal (ours). Our FairCal performs best.
their embeddings exceeds a certain threshold. We illustrate and compare the bias of current models in Figure I and compare their characteristics in Table L
The fairesss of face verification classifiers is typically assessed by three quantities, explained below: (i) fairness calibration across subgroups, (ii) equal false positive rates (FPRs) across subgroups i.e. predictive equality, (iii) equal false negative rates (FNRs) across subgroups, i.e. equal opportunity.
classifier is said to be globally calibrated if its predicted probability Pared of face matches in a dataset is equal to the fraction Ptrue of true matches in the dataset (Guo et al.) 2017). This requires the classifier to additionally output an estimate of the true probability of match, i.e. confidence. However, we suggest it is even more crucial for models to be fairly calibrated, i.e., for calibration to hold conditionally on each subgroup (see Definition 2). Otherwise, if the same predicted probability is known to carry different meanings for different demographic groups, users including law enforcement officers and judges may be motivated to take sensitive attributes into account when making critical decisions about arrests sentencing (Pleiss et al. 2017). Moreover, previous works such as Canetril et al. 2019) and Pleiss et al. (2017) presume the use of a fairly-calibrated model for their success.
While global calibration can be achieved with off offlth post-hoc calibration methods (Platt 1999
Zadrozny& & Elkan [2001] Niculescu-Mizil & Caruana, [005) /uo et al.] 2017). they do not achieve fairness-calibration. This illustrated in Figure 2 However, achieving all three fairness definitions
mentioned above: (i) fairness-calibrate
(ii) equal FPRs, (iii) equal FNRs, is impossible in
2016 Chouldechova 2017 Kleinberg et al.. 2017 Pleiss et al. (017). We elaborate further on this in section 3] the key takeaway is that we may aim to satisfy at most two of the three fairness notions. In the particular context of policing, equal FPRs i.e. predictive equality is considered more important than equal FNRs, as false positive errors (false arrests) risk causing significant harm, especially to members of subgroups already at disproportionate risk for police scrutiny or violence. As result, our goals are to achieve lairness-calibratee and predictive equality across subgroups, while maintaining accuracy, without retraining.
practice as several authors have shown (Hardt et al.
We further assume that we do not have access to the sensitive attributes, as this may not be feasible in practice due to (a) privacy concerns, (b) challenges in defining the sensitive attribute (c.g. ethnicity cannot be neatly divided into discrete categories), and (c) laborious and expensive to collect.
In this paper, we succeed in achieving all of the objectives mentioned above. The main contribution is the introduction of a post-training Fairness Calibration (FairCal) method:
1. Accuracy: FairCal achieves state-of-the-art accuracy TTble 2) for face verification on two large datasets.
2. Fairniss-calibration:: Our face verification classifier outputs state-of-the-art fairness- calibrated probabilities without knowledge of the sensitive attribute (Table 3).
3. Predictive equality: Our method reduces the gap in FPRs across sensitive attributes Ta ble 4p. For example, in Figure 2, at a Global FPR of 5% using the baseline method Black people are 15X more likely to false match than white people. Our method reduces this to 1.2X (while SOTA for post-hoc methods is 1.7X).
4. No sensitive attribute required: Our approach does not require the sensitive attribute, neither at training nor at test time. In fact, it outperforms models that use this knowledge.
2

---

Published as a conference paper at ICLR 2022
Baseline
R
Calibration
FairCal (Ours)
Cosine Similadtyy Score
Frebability
Frebability
Frobablitty
Genuire imposter
Gemuline Imposter
Genuie Imposter
Genuine Imposter
Amricn
Asian Caucasien indlan siciiiity
Airican
Acian
Caucasian kidian
Alrican
Asian
Caucasian an Indian
Alricn
Asian
Ca. casion Indian
Ethnicity
Ethnicity
Ethnicity
Figure 2: Illustration of bias reduction. False Positives correspond to the imposter pairs above decision threshold value. Black horizontal line: threshold which achieves global FPR of 5%; Red lines: thresholds for subgroup FPRs of 5%. The deviation of the subgroup FPRs from the global FPR is a measure of bias (red lines closer to black horizontal line is better). The baseline method is biased. Calibration (based cosine similarity alone) does not reduce the bias. The Oracle method reduces bias by using subgroup membership labels for calibration. The FairCal method (ours) reduces bias by using feature vectors for calibration without using subgroup information.
5.
No retraining: Our method has no need for retraining or tuning the feature representation model, or training an additional model, since performs statistical calibration post-hoc. hoc.
RELATED WORK
Fairness, or Bias mitigation: Work on bias mitigation for deep FR models can be divided into two main camps: (i) methods that learn less biased representations, and (ii) cost-processing approaches
that attempt to remove bias from a pre-trained feature representation model by building fairer decision
systems (Srinivas al. 2019 Dhar et al. 2020 2021 Terhörst et al.) 2020a b Robinson al. 2020).
Several approaches have been pursued in (i), including domain adaptation (Wang et al. 2019a).
margin-bass losses (Khan al. 2019 Huang et al. 22020), data augmentation (Kortylewski et
2019), feature augmentation (Yin et al. 2019 Wang et 2019b), reinforcement learning (Wang & 2018 Gong et al.) 2020), and attention mechanisms (Gong et al.) (02T). While these methods mitigate bias, this is often achieved at the expense of recognition accuracy and/or at high computational cost. The work we present in this paper fits into (ii), and thus we focus reviewing those approaches. For an in-depth literature review of bias mitigation for FR models, see Drozdowski et al. (2020). We compare all these methods in Table [
Deng 2020), adversarial learning (Liu et al.
Srinivas et al. (2019) proposed an ensemble approach, exploring different strategies for fusing the scores of multiple models. This work does not directly measure bias reduction, and presents only the results of applying the method to protected subgroup. Terhörst al. (020b) show that while this method is effective in increasing overall accuracy, fails to reliably mitigate bias.
Dhar et al. (020)) proposed the Adversarial Gender De-biasing algorithm (AGENDA) to train shallow network that removes the gender information of the embeddings of a pre-trained network.
Dhar et al. (2021) extended this work with PASS to deal with any sensitive attribute, and proposed novel discriminator training strategy. This leads to reduced gender bias in face verification, but at the cost of accuracy. Also, AGENDA and PASS require the sensitive attributes during training.
Terhörst et al. 2020a) proposed the Fair Template Comparison (FTC) method, which replaces the computation of the cosine similarity score by an additional shallow neural network trained using cross-entropy loss, with fairness penalization term and an l2 penalty term to prevent overfitting. While this method does indeed reduce a model's bias, it comes at the expense of an overall decrease in accuracy. Moreover, it requires training the shallow neural network and tuning the loss weights. Most importantly, it requires the sensitive attributes during training.
Robinson et al. (2020) proposed GST which uses group-specific threshold. However, the calibration sets in GST are defined by the sensitive attributes themselves. Terhörst et al. (2020b) proposed the Fair Score Normalization (FSN) method, which is essentially GST with unsupervised clusters. FSN normalizes the scores by requiring the model's 's FPRs across unsupervised clusters to be the same at predefined global FPR. These methods are not designed to be fairly-calibrated. Moreover, if different global FPR is desired, the method needs to recomputed. In contrast, instead fixing one FPR threshold, our method converts the cosine similarity scores into calibrated probabilities, which
3

---

Published as a conference paper at ICLR 2022
Table 1: Comparison of desirable features of the different fairness methods for face verification.
Improves Fairly accuracy calibrated
Predictive Does require sensitive attribute time during training
Does require re-training
Method (requires re-training)
equality
D' (Liu et al. 2018
FTL Yin et ml. 2019
X
x
X
LMEXFTDN Wang et al
2019h
X
SYN (KortyleWski
K
X
UMML Khan mi 2019
X
X
X
CLMLE (Fiiaan clele 2020
DebFace-ID (Gong
X
RL-RBN (Wang & Deng
X
X
GAC Gong et ar 2021
Improves accuracy calibrated
Fairly
Predictive Does require sensitive attribute
Does require additional training
Method (post-training)
equality
during training
test time
AGENDA Dhar et al.
2020
x
x
PASS
Dhai et al 2021
x
x
FTC
Terhorst
x
x
GST
Robinson
2020
V
X
X
FSN
Terhorst et al.
2020m
Oracle (Ours)
FairCal (Ours)
then be used for any choice of fair FPRs. In addition, approach be extended group fairness for K KCclaasification problems, which is not possible with GST or FSN.
Calibration: Calibration is closely related to uncertainty estimation for deep networks (Guo et al. 2017). Several post-hoc calibration methods have been proposed such as Platt' 's scaling or temperature scaling (Platt 1999 Guo et al.. 2017). histogram binning (Zadrozny & Elkan 200T), isotonic
regression Miizul & Caruana (005), spline calibration (Gupta et al., 2021). and beta calibration (Kull et al. 2017). among others. All of these methods involve computing a calibration function. As such, any calibration method can be readily applied, leading to models that are calibrated but not fairly-calibrated. An algorithm achieve the latter for binary classifier has been proposed in Johheon et al. (2018), but the work remains theoretical and no practical implementation is known. Fairly calibrated models are the missing ingredient in the works of Canetti et al. (2019) and Pleiss et al. (2017), since they work with the presumption that fairly calibrated model exists.
3 FACE VERIFICATION AND FAIRNESS
In order to discuss bias mitigation strategies and their effectiveness, must first agree what constitutes a fair algorithm. We start by rigorously defining the face verification problem as a binary classification problem. Then present the notion of probabilistic classifier, which outputs calibrated probabilities that the classification is correct. Finally, since several different definitions of fairness have been proposed (see Verma & Rubin (0088; Garg al. (2020) for comparative analysis), we review the ones pertinent to our work.
3.1 BINARY CLASSTFIERS AND SCORE OUTPUTS
Let f denote a trained neural network that encodes an image æ into an embedding f(x) = E 2 Rd Pairs of images of faces are drawn from global pair distribution (æ1, x22) ~ P. Given such pair, let ( x²) = 1 if the identities of the two images are the same and = 0 otherwise. The face verification problem consists of learning a binary classifier for Y.
The baseline classifier for the face verification problem is based on the cosine similarity between the /(2,)² f(æ)) ((xz)
feature embeddings of the two images, s(21, 222) = [[ [[](==)]l]](mm)] (m2)]l The cosine similarity score
used to define a binary classifier Y xxxX + {0, 1} by thresholding with a choice of 5thr E {--1,
which is determined by a target FPR: ( x22 = 1 if s(x1. x2) 2 and 0 otherwise.
3.2 CALIBRATION
calibrated probabilistic model provides a meaningful output: its confidence reflects the probability of genuine match, in the sense of the following definition.
4

---

Published as a conference paper at ICLR 2022
Definition 1. The probabilistic model Č X x X - [0, is said to be calibrated if the true probability of match is equal to the model's confidence output e: Pei (Y =1 [
Any score-based classifier can be converted to calibrated probabilistic model using standard post-hoc calibration methods (Zadrozny & Elkan 2001] Niculescu- Mizil & Caruana 2005; Platt 1999 Kull et al. 2017) (see Appendix I for brief description of ones used in this work). This probabilistic
model can be converted to a binary classifier by thresholding the output probability.
Applied to face verification, calibrated probabilistic model outputs the likelihood/confidencc that pair of images (I1, (2) are a match. Calibration is achieved by using a calibration set Stal to learn calibration map j from the cosine similarity scores to probabilities. Since calibration defines monotonically increasing calibration map μ., a binary classifier with a score threshold Schr can be obtained by thresholding the probabilistic classifier ju(Sthr).
3.3 FAIRNESS
In the context of face verification, fairness implies that the classifier has similar performance on different population subgroups, i.e. being calibrated conditional on subgroup membership. Let g(x)) E G } denote the sensitive/protected attribute of such as ethnicity, gender, or age. Each sensitive attribute 9: E induces population subgroup, whose distribution on the intra-subgroup pairs we denote by Gr.
The probabilistic model Č is fairly-calibrated for subgroups 9r and 92 if the classifier is calibrated when conditioned on each subgroup.
Definition 2.
P Paiæzn 22r0; (Y = =1|
Po (Y 1 =1
Intuitively, a model is considered biased if its accuracy alters when tested on different subgroups. However, in the case of face recognition applications, predictive equality (defined below), meaning equal FPRs across subgroups, is often primary importance.
Definition 3. A binary classifier Y exhibits predictive equality for subgroups gi and if the classifier has equal FPRs for each subgroup.
P( ( = = (a ( = = 0).
In certain applications of FR (such as office security, where high FNRs would cause disruption), equal opportunity (defined below), meaning equal FNRs across subgroups, also important.
Definition 4. A binary classifier y exhibits equal opportunity for subgroups gi and g2 if the classifier has equal FNRs for each subgroup,
((.))0))~ ( = = 1) = (( (P = = =).
Prior works (Hardt et al. 2016 Chouldechova 2017: Kleinberg et al. 2017 Pleiss et al. 2017)
have shown that it is impossible practice to satisty all three fairness properties at the same time: (i) fairness calibration (definition D (ii) predictive equality (definition 3, (iii) equal opportunity (definition 4). At most, two of these three desirable properties can be achieved simultaneously in practice. In the context of our application, predictive equality is more critical than equal opportunity. Hence we choose to omit equal opportunity as our goal.
FAIRNESS CALIBRATION (FAIRCAL)
In this section we describe our FairCal method, which constitutes the main contribution of this work. Simply calibrating model based on cosine similarity scores will not improve fairness: if the baseline score-based classifier biased, the resulting probabilistic classifier remains equally biased. This is illustrated in Figure 2) where for both Baseline and Baseline Calibration, any choice of global threshold will lead to different FPRs across sensitive subgroups; consequently, these models will fail to achieve predictive equality.
'This notion is also referred to well-calibration Verma & Rubin 2018) and calibration within subgroups
Kleinherg al. 2017 Berk
202T
5

---

Published as a conference paper at ICLR 2022
Our proposed solution is to introduce conditional calibration methods, which involve partitioning the pairs into sets, and calibrating each set. Given an image pair, we first identify its set membership. and then apply the corresponding calibration map. In our FairCal method, the sets are formed by clustering the images' feature vectors in an unsupervised fashion. In our Oracle method, the sets are defined by the sensitive attributes themselves. Both methods designed to achieve fairness-calibration.
4.1 FAIRNESS CALIBRATION (FAIRCAL, OUR METHOD)
(i) Apply the K-means algorithm to the image features, geal, partitioning the embedding space E into K clusters Z1,... Zz. These form the K calibration sets:
Skat {5 (1 x2) (2x1) E Zn or ((x2) E Zx}; k k-,,...;
(ii) For each calibration set Sg use a post-hoc calibration method to compute the calibration map P4 that maps scores 8 s(x), 22) to cluster-conditional probabilities μr u (8(x1, Ø2)). We use beta
calibration (Kull et al. 2017). the recommended method when dealing with bounded scores.
(iii) For image pair (æ1, 22)) find the clusters they belong to. If both images fall into the same cluster k define the pair's calibrated score as the cluster's calibrated score:
c( c( x2) = Mi (5(x1, 22))
Blse, if the images are in different clusters, 1 and 22, respectively, define the pair's calibrated score as the weighted average of the calibrated scores in each cluster:
1 x22 = μkx (s(x), x2)) + (1 (1- 0) Mrg (s(x)) x2)),
(3)
where the weight is the relative population fraction of the two clusters,
Scal /( Scal
Seal
(4)
Since FairCal uses unsupervised clusters, it does not require knowledge of the sensitive attributes. Moreover, FairCal is post-training statistical method, hence it does not require any retraining.
4.2 COMPARISON WITH FSN (TERHORST ET AL. 2020B
Building the unsupervised clusters allows us to not rely on knowing the sensitive attributes. In contrast, the FSN method normalizes the scores by shifting them such that thresholding at predefined global FPR leads to that same FPR on each calibration set. This is limiting in two crucial ways: I) FSN only applies to the binary classification problem (e.g. face verification): 2) the normalizing shift in FSN depends on the global FPR chosen priori. Our FairCal method has neither of these limitations. By converting the scores to calibrated probabilities, we can extended it to the multi-class setting, and we do not need to choose global FPR priori.
simple analogy to explain the difference is consider the problem of fairly assessing two classrooms of students with different distribution of grades. Fair calibration means the same grade should mean the same for both classrooms. Equal fail rate implies the percentage of students who fail is the same in both classrooms. Global (unfair) methods would choose pass or fail students from both classrooms using the same threshold. Two possible fair approaches are: (A) Estimate a different threshold for each classroom based fixed fail rate for both. (B) Calibrate the grades the distributions of the two classrooms match, i.e. the same grade means the same in both classrooms, and then choose a fair threshold. Method B automatically ensures equal fail rate for both classes. Method A is what FSN does, Method B is what our FairCal method achieves.
4.3 ORACLE CALIBRATION (SUPERVISED FAIRCAL, ALSO OURS)
We include a second calibration method, Oracle, which proceeds like the FairCal defined above, but creates the calibration sets based on the sensitive attribute instead in unsupervised fashion. If the images belong to different subgroups i.e. g(x1) * g(x22, then the classifier correctly outputs zero This method is not feasible in practice, since the sensitive attribute may not be available, or because using the sensitive attribute might not permitted for reasons of discrimination or privacy. However, the Oracle method represents ideal baseline for fairness-calibration

---

Published as a conference paper at ICLR 2022
5
EXPERIMENTAL DETAILS
Models: We used three distinct pretrained models: two Inception Resnet models (Szegedy et al. 2017) obtained from Esler (202)) (MIT License), one trained on the VGGFace2 dataset (Cao et al.
2018) and another on the CASIA- -ebbfcc dataset (Yi et al. 2014), and an ArcFace model obtained from Sharma (202T) (Apache 2.0 license) and trained on the refined version of MS-Celeb- IM (Guo et al. 2016). We will refer to the models as Facenet (VGGFace2), Facenet (Webface), and ArcFace, respectively. is standard for face recognition models, we pre-processed the images by cropping them using Multi-Task Convolution Neural Network (MTCNN) algorithm (Zhang et al. 2016). If the algorithm failed to identify face, the pair was removed from the analysis.
Datasets: We present experiments two different datasets: Racial Faces in the Wild (RFW) (Wang et al.) (019a) and Balanced Faces in the Wild (BFW) (Robinson et al., 2020), both of which are available under licenses for non- non-commercial research purposes only. Both datasets already include predefined pairs separated into five folds. The results we present are the product of leave-one-out cross-validation. The RFW dataset contains 1:1 ratio of genuine/imposter pairs and 23,54 pairs in total (after applying the MTCNN). The dataset's ' images are labeled by ethnicity (African, Asian, Caucasian, or Indian), with all pairs consisting of same-ethnicity images. The BFW dataset, which possesses 1:3 ratio of genuine/imposter pairs, is comprised of 890,347 pairs (after applying the MTCNN). Its images are labeled by ethnicity (African, Asian, Caucasian, or Indian) and gender (Female Male), and it includes mrxeedggende and mixed-ethnicity pairs. The RFW and BFW datasets are made up of images taken from MS-Celeb- M (Guo et al. (016) and VGGFace (Cao et al. 2018), respectively. Thus, to expose the models to new images, only the two FaceNet models can be evaluated on the RFW dataset, while only the FaceNet (Webface) and ArcFace models can be evaluated on the BFW dataset.
Methods:
For both the FSN and FairCal method we used = 100 clusters for the K K-means algorithm, recommended by Terhörst al. (2020b). For FairCal, employed the recently proposed beta calibration method (Kull et al.] 2017). Our FairCal method robust to the choice of number of clusters and post-hoc calibration method (see (see Appendix ]foo more details). We discuss the parameters used in training AGENDA (Dhar et al. 22220) PASS (Dhar et al. 2021) and FTC (Terhörst
al. 2020a)
Appendix B in Appendix Cland Appendix D respectively.
Notice that the prior relevant methods (Baseline, AGENDA, PASS, FTC, GST, and FSN) output scores that, even when rescaled to [0, J. do not result in calibrated probabilities and hence, by design, cannot be fairly-calibrated. Therefore, in order to fully demonstrate that our method is superior to those approaches, when measuring fairness-calibration we apply beta calibration their final score outputs as well, which is the same post-hoc calibration method used in our FairCal method.
6 RESULTS
In this section we report the performance of models with respect the three metries: (i) accuracy, (ii) fairness calibration, and (iii) predictive equality. Our results show that among post hoc calibration methods,
1. FairCal is best global accuracy, Table 2
2. FairCal is best on fairness calibration, Table 3
3. FairCal is best on predictive equality, i.c., equal FPRs, see Table 4
4. FairCal does not require the sensitive attribute, and outperforms methods that use this knowledge, including a variant of FairCal that uses the sensitive attribute (Oracle).
5. FairCal does not require retraining of the classifier.
We discuss these results further detail below. We provide additional detailed discussion and results, including equal opportunity and the standard deviations that result from the 5-fold cross-validation study, in the Appendix. Overall, our method satisfies both faimess definitions without decreasing baseline model accuracy. In contrast, while the FTC method obtains slightly better predictive equality results than our method in one situation, this achieved only at the expense of a significant decrease in accuracy.
7

---

Published as a conference paper at ICLR 2022
Table 2: Global accuracy (higher is better) measured by AUROC, and TPR at two FPR thresholds.
RFW
BFW
FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFace
(1) AUROC 0.1% FPR 1% EPR
TPR
TPR @
AUROC
0.1% FPR 1 FPR
TPR @ TPR @
AUROC
0.1% FPR 1% FPR
TPR @ TPR
AUROC
0.1% FPR 1% FPR
TPR @ TPR @
Baseline
88.26
18.42
34.88
83.95
11.18
26.04
96.06
33.61
58.87
97.41
86.27
90.11
AGENDA
76.83
8,32
18.01
74.51
6.38
14.98
82.42
15.95
32.51
95.09
69.61
79.67
PASS
86.96
13.67
29.30
81.44
7.34
20.93
92.27
17.21
38.32
96.55
77.38
85.26
FTC
86.46
6.86
23.66
81,61
4.65
18.40
93.30
13.60
43.09
96.41
82.09
88.24
GST
89.57
22.61
40.72
84.88
17.34
31.56
96.59
44.49
66.71
96.89
86.13
89.70
FSN
90.05
23.01
40.21
85.84
17.33
32.80
96.77
47.11
68.92
97.35
86.19
90.06
FairCal (Ours)
90.58
23.55
41.88
86.71
20.64
33.13
96,90
46.74
69.21
97.44
86,28
90.14
Onncle (Ours)
89.74
21.40
41.83
85.23
16.71
31.60
67.56
98.91
86.41
Table 3: Fairness calibration measured by the mean KS across the sensitive subgroups. Bias: measured by the deviations of KS across subgroups: Average Absolute Deviation (AAD), Maximum Absolute Deviation (MAD), and Standard Deviation (STD). (Lower is better in all cases.)
RFW
BFW
FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFace AAD
(4)
Mean AAD
MAD STD
Mean
AAD
MAD
STD
Mean
AAD
MAD STD
Mean
MAD
STD
Baseline
6.37
2.89
5.73
3.77
5.55
2.48
4.97
2.91
6.77
3.63
5.96
4.03
2.57
1.39
2.94
1.63
AGENDA
7.71
3.11
6.09
3,86
5.71
2.37
4.28
2.85
13.21
6.37
12.91
7.55
5.14
2.48
5.92
3.04
PASS
8.09
2.40
4.10
2.83
7.65
3.36
5.34
3.85
13.16
5.25
9.58
6.12
3.69
2.01
4.24
2.37
FTC
5.69
2.32
4.51
2.95
4.73
1.93
3.86
2.28
6.64
2,80
5.61
3.27
2.95
1.48
3.03
1.74
GST
0.98
3.24
1.21
1.93
1.34
3.09
2.80
1.65
3.34
2.19
FSN
1.43
0.35
0.57
0.40
2.49
0.84
1.19
0.91
2.76
1.38
2.67
1.60
2.65
1.45
3.23
1.71
Cair (Ours)
1.37
0.28
0.50
0.34
1.75
0.41
0.64
0.45
3.09
1.34
2,48
1.55
2.49
1.30
2.68
1.52
Oracle (Ours)
1.18
0.28
0.53
0.33
1.35
0.38
0.66
0.43
2.23
1.15
2.63
1.40
1.41
0.59
1.30
0.69
Table 4: Predictive equality: For two choices of global FPR (two blocks of rows): 0.1 0.1% and 1%, we compare the deviations in subgroup FPRs in terms of: Average Absolute Deviation (AAD), Maximum Absolute Deviation (MAD), and Standard Deviation (STD). (lower is better in all cases)
RFW
BFW
FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface) AAD
ArcFace MAD
(4)
AAD
MAD
STD
AAD
MAD
STD
MAD
STD
AAD
STD
Baseline
0.10
0.15
0.10
0.14
0.26
0.16
0.29
1.00
0.40
0.12
0.30
0.15
AGENDA
0.11
0.20
0.13
0.12
0.23
0.14
0.14
0.40
0.18
0.09
0.23
0.11
0.1 0.1%% PPR
PASS
0.11
0.18
0.12
0.11
0.18
0.12
0.36
1.21
0.49
0.12
0.29
0.14
FTC
0.10
0.15
0.11
0.12
0.23
0.14
0.24
0.74
0.32
0.09
0.20
0.11
GST
0.13
0.24
0.15
0.09
0.16
0.10
0.13
0.35
0.16
0.10
0.24
0.12
FSN
0.10
0.18
0.11
0.11
0.23
0.13
0.09
0.20
0.11
0.11
0.28
0.14
FairCal (Ours)
0.09
0.14
0.10
0.09
0.16
0.10
0.09
0.20
0.11
0.11
0.31
0.15
Oracle (Ours)
0.11
0.19
0.12
0.11
0.20
0.13
0.12
0.25
0.15
0.12
0.27
0.14
Baseline
0.68
1.02
0.74
0.67
1.23
0.79
2.42
7.48
3.22
0.72
1.51
0.85
AGENDA
0.73
1.14
0.81
0.73
1.08
0.78
1.21
3.09
1.51
0.65
1.78
0.84
1% FPR
PASS
0.89
1.52
1.01
0.68
0.99
0.73
3.30
10.18
4.34
0.72
2.00
0.93
FTC
0.60
0.91
0.66
0.54
1.05
0.66
1.94
5.74
2.57
0.54
1.04
0.61
GST
0.52
0.92
0.60
0.30
0.57
0.37
1.05
3.01
1.38
0.44
1.13
0.56
FSN
0.37
0.68
0.46
0.35
0.61
0.40
0.87
2.19
1.05
0.55
1.27
0.68
FairCal (Ours)
0.28
0.46
0.32
0.29
0.57
0.35
0.80
1.79
0.95
0.63
1.46
0.78
Oracle (Ours)
0.40
0.69
0.45
0.41
0.74
0.48
0.77
1.71
0.97
0.83
2.08
1.07
It is important to emphasize that accuracy and predictive equality metries are determined at specified global FPR. This entails determining for each prior method a threshold that achieves the desired global FPR. However, this itself promotes our method's 's advantages: (a) previous methods such as FSN rely on a predefined FPR, while ours does not; (b) while other methods need recomputation for different thresholds, our method simultaneously removes the bias at multiple global FPRs.
6.1 GL.BBA ACCURACY
The receiver operating characteristic (ROC) curve plots the true positive rate (TPR = 1-FNR) against the FPR, obtained by thresholding the model different values. The area under the ROC curve (AUROC) is thus holistic metric that summarizes the accuracy of the classifiers (Davis & Goadrich 2006). A higher AUROC is better, uninformative classifier has an AUROC of 50%.
8

---

Published as a conference paper at ICLR 2022
Our FairCal method achieves SOTA results, as shown in Table 2 We report the AUROC for the different pre-trained models and datasets, as well as the TPRs at 0.1 % and 1% global FPR thresholds. FairCal achieves the best values of AUROC in all cases, the highest TPR in seven of the eight cases, and surpasses GST and our Oracle method (which use subgroup information) on the RFW dataset.
This overall accuracy improvement of our FairCal method can be explained as follows: the feature vectors contain more information than the score alone, so they can be used to identify pairs where the probability of an error is higher or lower, allowing per-cluster calibration give better accuracy.
In our FairCal method, the subgroups are formed based on the features of the model, and not on manual human-assigned attributes. This allows for potentially more complex and effective subgroups. We visualized the cluster images, and found they indeed have semantic meaning, see Appendix A
6.2 FAIRNESS CALIBRATION
The prior relevant methods (Baseline, AGENDA, PASS, FTC, GST, and FSN) do not output probabil- ities, hence by design they cannot be fairly- calibaed However, in order to fully demonstrate that our method is superior to those approaches, we apply beta calibration (the same post-hoe calibration method used in our FairCal method) to their score outputs.
model's output probabilities of matches from the actual results. When considering subgroups, this is measured by two quantities: (i) the calibration error on each subgroup (lower is better, since it means fewer errors), which can be measured using the Brier score (BS) (DeGroot & Fienberg 1983), Expected Calibration Error (ECE) (Guo et al. (017). or Kolmogorov-Shme Smiinov (KS) (Gupta et al. 2021) (discussed in (Appendix E We present results for KS, which has been established to be the best measure (Gupta al.) 2021). (ii) The deviation in these quantities across subgroups (lower is better, since it is more fair), measured as: Average Absolute Deviation (AAD), Maximum Absolute Deviation (MAD), and Standard Deviation (STD).
Calibration error is measure of the deviation
Our FairCal method achieves the best results, as shown in Table 3) the best mean in three of the four cases (smaller errors) and the best deviation in nine of the nine cases (more fair across subgroups). The improvement is most significant on the less accurate models. We discuss these results in more detail in the Appendix.
6.3 PREDICTIVE EQUALITY
As predictive equality is achieved through equal FPRs between different subgroups, we can measure bias by quantifying the deviation of these FPRs. We report three measures of deviation (AAD, MAD, and STD) at two different global FPRs: 0.1% and at 1.0%.
Our FairCal method achieves the best results, as shown Table 4. the best or tied measure of predictive equality in 18 of the 24 cases. In the cases where FairCal is not best, i.c., when the ArcFace model evaluated on the BFW dataset, FTC and GST provide the best results, but the differences between AGENDA's, PASS's, FTC's, GST's, FSN's, and FairCal's deviation measures are within a fraction of %. Moreover, when applied to ArcFace, the FTC and GST methods reduce the model's accuracy.
7 CONCLUSION
We introduce FairCal, a post-training calibration method that (i) achieves SOTA accuracy, (ii) is fairly-calibrated, (iii) achieves predictive equality (equal FPRs) on challenging face verification datasets, (iv) without the knowledge of any sensitive attribute. It can be readily applied to existing FR systems, as is a statistical method that (v) does not require any retraining. can aid human- users in using FR systems make better informed decisions based on interpretable, unbiased results.
Only under very strict conditions (e.g. a perfect classifier) can all three fairness notions (fairness calibration, predictive equality, equal opportunity) be satisfied. While we have shown which for in practice-ir requires the use of calibration dataset. Deploying method on substantially different dataset would most likely require further calibration. Calibrating models for never-seen data is currently an open problem.
Limitations:
that our FairCal method can achieve two out of the three definitions which is the best we can hope

---

Published as a conference paper at ICLR 2022
ETHICS STATEMENT
Bolstered by dramatically improving accuracy, Facial recognition systems have exploded in popularity in recent years, and have been applied to innumerable settings under various use cases. However, the severe biases within these systems limit their applicability, and open the doors to widespread social harm. Our paper is primarily focused on addressing the lack of fairness as well as potential unethical uses of current face recognition systems, specifically the context of face verification. We address this issue by proposing novel post-prose processiin method that significantly reduces the false positive rates across demographic subgroups without requiring knowledge of any sensitive attributes. At many places in the paper, we have referenced several prior works that mention and analyze the current and potential risks posed by current facial recognition systems. We identified the shortcomings in the previous methods, and methodically established scientific way of identifying and measuring fairness in the current systems. Our experiments involve large public datasets of faces, Our work explicitly addresses problems with facial recognition systems that could misuse sensitive attributes in these datasets, such as ethnicity, race, gender, etc. The motivation of our method is in trying to mitigate bias and improve fairness in the systems using these datasets.
REPRODUCIBILITY STATEMENT
In order to ensure the reproducibility of our work, we described our proposed methods, FairCal and Oracle, in detail in section 4 The details of our experiments, including the datasets and pre- pre-processing steps, can be found in section 5) Additional details on how we implemented AGENDA and FTC, which require training additional models, are provided in Apppendix B and Appendix D respectively. The embeddings from the pretrained models were obtained on a machine with one GeForce GTX 1080 Ti GPU. All methods were implemented in Python, and the code is provided in the supplemental material.
ACKNOWLEDGMENTS
This material is based upon work supported by the Air Force Office Scientific Research under award number FA9550-18 18-50-18---1661 (A.0.) Thanks CIFAR for their support through the CIFAR AI Chairs program. We would also like to thank Muawiz Chaudhary and Vicent Mai for their valuable feedback.
REFERENCES
Bobby Allyn. 'The Computer Got It Wrong': June 2020.
How Facial Recognition Led To False Arrest Of totptro--oooouttttt----t----- www.ttttoo-ooww----
Black Man.
Mohsan Alvi, Andrew Zisserman, and Christoffer Nellaaker. Turning blind eye: Explicit removal of biases and variation from deep neural network embeddings. In Proceedings of the European Conference Computer Vision (ECCV) Workshops, September 2018
Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 50 (1):3 44, 2021. doi: 10.1177/0049124118877
URL lhttt::// o0tp://00000000/00111
104912411878255
Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Sorelle A. Friedler and Christo Wilson (eds.), Proceedings of the Ist Conference on Fairness, Accountability and Transparency, volume 81 of Proceedings of Machine Learning Research, pp. 77-91, New York, NY, USA, 23 24 Feb 2018. PMLR. URL http: html
//proceedings.oo mir pttpa//vl//oooaaa///oaaaaaaaaaaa
Ran Canetti, Aloni Cohen, Nishanth Dikkala, Govind Ramnarayan, Sarah Scheffler, and Adam Smith. From soft classifiers to hard decisions: How fair can we be? In Proceedings of the Conference on Fairness, Accountabiliry and Transparency, FAT* 19, pp. 309-318, New York, NY. USA, 2019. Association for Computing Machinery. ISBN 9781450361255. URL httpt::t////t. //doi httpg/.....1//225511
doi: 00.1144//287550.22875666 htttppps/obllooogg0011111//33333 328766
10

---

Published as a conference paper at ICLR 2022
Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, and Andrew Zisserman. VGGFace2: A dataset for recognising faces across pose and age. In International Conference on Automatic Face and Gesture Recognition, 2018.
Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2):153-163, 2017.
Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In Proceedings of the 23rd International Conference on Machine Learning. ICML '06, pp. 233-240, New York, NY. USA, 2006, Association for Computing Machinery. ISBN 1595933832. doi: 10.1 1,5/1 143844. 143874. URL httppttss://ciii http://:///......1.1.....1...... 11tp:///....11111111.1 1ttp//11134444.11388888 1143874
Morris H DeGroot and Stephen E Fienberg. The comparison and evaluation forecasters. Journal of the Royal Statistical Society: Series D (The Statistician), 32(1-2):12-22, 1983.
Prithviraj Dhar, Joshua Gleason, Hossein Souri, Carlos Domingo Castillo, and Rama Chellappa. An adversarial learning algorithm for mitigating gender bias in face recognition. CoRR, abs/2006.07845, 2020. URL https: http:/xwwwwvrraaa////aaaaaaa
abs
07845
Prithviraj Dhar, Joshua Gleason, Aniket Roy, Carlos D Castillo, and Rama Chellappa. Pass: Pro- tected attribute suppression system for mitigating bias in face recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15087-15096, 2021.
Pawel Drozdowski, Christian Rathgeb, Antitza Dantcheva, Naser Damer, and Christoph Busch. Demographic bias in biometrics: A survey on an emerging challenge. IEEE Transactions on Technology and Society, 1(2):89-103, 2020. doi: 10.1109/TTS.200022299244
Tim Esler. Face recognition using pytorch.
https://github. htt.ggittub..oo/..... mes /er/
Facenet-pytorahe 2021.
Pratyush Garg, John Villasenor, and Virginia Foggo. Fairness metrics: A comparative analysis. In 2020 IEEE International Conference on Big Data (Big Data), pp. 3662-3666, 2020. doi: 10.1 (t00///iDDDa55002.2020....2
C. Garvie, Georgetown University. Center on Privacy, Technology, and Georgetown University. Law Center. Center on Privacy & Technology. The Perpetual Line-up: Unregulated Police Face Recognition in America. Georgetown Law, Center on Privacy & Technology, 2016. URL
https::/booke: google de steebbooksidddee (iobookkrr.duAAAmagAAAAA
Sixue Gong, Xiaoming Liu, and A Jain. Jointly de-biasing face recognition and demographic attribute estimation. ECCV, 2020.
Sixue Gong, Xiaoming Liu, and Anil K. Jain. Mitigating face recognition bias via group adaptive clas- sifier. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 3414-3424, 2021.
Patrick Grother, Mei Ngan, and Kayee Hanaoka. Face recognition vendor test part 3:. Technical report, National Institute of Standards and Technology, December 2019. URL/https:/
org/10. 10/010..0 6028/nistt.... ........ 8280
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On Calibration of Modern Neural Networks. In Proceedings of the 34th International Conference Machine Learning, ICML 2017, Syddne, NSW, Australia, 6-11 August 2017, pp. 1321-1330, 2017.
Yandong Guo, Lei Zhang, Yuxiao Hu, X. He, and Jianfeng Gao. Ms-celeb- 1m: A dataset and benchmark for aarge-scale face recognition. In ECCV, 2016.
Kartik Gupta, Amir Rahimi, Thalaiyasingam Ajanthan, Thomas Mensink, Cristian Sminchisescu, and Richard Hartley. Calibration of neural networks using splines. In International Confer- ence on Learning Representations, 2021. URL 1e@8DD
net/ hwt./oorrmmii.
11

---

Published as a conference paper at ICLR 2022
Moritz Hardt, Eric Price, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In D. Lee, M. Sugiyama, U. Luxburg. I. Guyon, and R. Gamett (eds.), Ad- vances in Neural Information Processing Systems, volume 29, pp. 3315-3323, Curran Asso- neurips cc cc/paper/20111. 6/10016fflee.
ciates, Inc., 2016. URL
Ursula Hebert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Cal- masses. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 1939-1948. PMLR, 10-15 Jul 2018. URL
ibration for the Comppuationally-ddnnfffal
http://proceedingg mir. press hwwpss/vv0//eeeeee attp:///aaeeerttoooooaaaaaaaaaaa jww.sooosnnonnteeoooo
Chen Huang, Yining Li, Chen Change Loy, and Xiaoou Tang. Deep imbalanced learning for face recognition and attribute prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(1 1):211)2122222
2020. doi: 10.1 109/TPAMM.2009..20. 2914600.
Anil K. Jain, Karthik Nandakumar, and Arun Ross, 50 years of biometric research: Accomplishments. challenges, and opportunities. Pattern Recognition Letters, 19:80-105, 2016. ISSN 0167-8655. doi: https://doi.org/100 http://101111111111p
12.011. URL https: www scw.ncediiiiiccccccooooooo rect com/
sci
Salman Khan, Munawar Hayat, Syed Waqas Zamir, Jianbing Shen, and Ling Shao. Striking the right balance with uncertainty. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pultern Recognition (CVPR), June 2019.
Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. Schloss Dagstuhl Leibniz-Zentrum fuer Informatik GmbH, Wadeen/Saarbruecken, Germany, 2017. doi: / //drops. dagstuhl............
URL http
Adam Kortylewski, Bernhard Egger, Andreas Schneider, Thomas Gerig, Andreas Morel-Forster, and Thomas Vetter. Analyzing and reducing the damage of dataset bias face recognition with synthetic data. In 2019 IEFF/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2261-2268, 2019. doi: 00.109//CVRR..2119000077
Meelis Kull, Telmo Silva Filho, and Peter Flach. Beta calibration: well-founded and easily implemented improvement on logistic calibration for binary classifiers. In Aarti Singh and Jerry Zhu (eds.), Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pp. 623-631, Fort Lauderdale, FL, USA. 20-22 Apr 2017. PMLR. URL htRp:tt::/ htllhttppp/prrceeiingg.. ml pre.s html
Meelis Kull, Miquel Perello Nieto, Markus Kängsepp, Telmo Silva Filho, Hao Song, and Peter Flach. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. Advances in Neural Information Processing Systems, 11122211112222 2019.
Jian Liang, Yuren Cao, Chenbin Zhang, Shiyu Chang, Kun Bai, and Zenglin Xu. Additive adversarial learning for unbiased authentication. In 2019 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11420-114223 11429, 2019. doi: 10.1109/CVP..2019.00 169.
Yu Liu, Fangyin Wei, Jing Shao, Lu Sheng, Junjie Yan, and Xiaogang Wang. Exploring disentangled feature representation beyond face identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2080-2089, 2018
Brianna Maze, Jocelyn Adams, James A. Duncan, Nathan Kalka, Tim Miller, Charles Otto, Anil K. Jain, W. Tyler Niggel, Janet Anderson, Jordan Cheney, and Patrick Grother. Iarpa janus benchmark c: Face dataset and protocol. In 2018 International Conference Biometrics (ICB), pp. 158-165 188-165, 2018. doi: 1091/09/C 18.0003...
Alexandru Niculescu-Min Mizil and Rich Caruana. Predicting good probabilities with supervised learning. In Proceedings of the 22nd International Conference Machine Learning, ICML '05, pp. 625- 632, New York, NY, USA, 2005. Association for Computing Machinery. ISBN 1595931805. doi: 10.1 145/1 102351. 102430. URL 111.:/..000/111111111111000100 11..0..111111111 1102430
12

---

Published as a conference paper at ICLR 2022
Jeremy Nixon, Michael W Dusenberry Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring calibration in deep learning. In CVPR Workshops, volume 2, 2019.
M. Orcutt. Are face recognition systems accurate? depends on your race. In MIT Technology Review. 2016.
Kanil Patel, William H. Beluch, Bin Yang, Michael Pfeiffer, and Dan Zhang. Multi-class uncer- tainty calibration via mutual information maximization- binning. International Confer- ence on Learning Representations, 2021. URL http:hts///ppeeeiiiiiee neeteeeeweeeeeeee..e.........eee http://wwwweeeeoeee.eeemmmm forum? did= AICNpd8ke-m
John C. Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In ADVANCES IN LARGE MARGIN CLASSIFIERS, pp. 61-74, MIT Press, 1999.
Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. On fairness and calibration. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances Neural Information Processing Systems, volume 30, pp. 5680- 5689. Curran Associates, Inc., 2017. URL http:tttt///:stceeell nggs neurips neurips.cc/paper pdf
Joseph P. Robinson, Gennady Livitz, Yann Henon, Can Qin, Yun Fu, and Samson Timoner. Face recognition: Too bias, or not too bias? In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 6 2020.
Abhinav Sharma. ONNX Model Zoo. https: htt:://githu.... com/ ownx/modelxeeee 2021.
N. Srinivas, K. Ricanek, D. Michalski, D. S. Bolme, and M. King. Face recognition algorithm bias: Performance differences on images of children and adults. In 2019 JEEE/CVE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2269-2277, 2019. doi: 10. 101100/PR..209..000 9.00280.
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander Alemi. Inception-v4, inception- resnet and the impact of residual connections on learning. Proceedings of the AAAI Conference on Artificial Intelligence, (1)),, Feb. 2017. URL htttt::t///ort aaal. org/ index.nix.. php/ (4AA/artiicle/vee//111
Yaniv Taigman, Ming Yang, Marc" Aurelio Ranzato, and Lior Wolf. Deepface: Closing the gap to human-level performance face verification. 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701-1708, 2014. doi: 10. 109/CVPR.2014.220
P. Terhörst, M. L. Tran, N. Damer, F. Kirchbuchner, and A. Kuijper. Comparison-leve mitigation of ethnic bias in face recognition. In 2020 8th International Workshop on Biometrics and Forensics (IWBF), pp. 1-6, 2020a. doi: 10. 109/0/ 200./4997722020990900 202099907966.
Philipp Terhörst, Jan Niklas Kolf, Naser Damer, Florian Kirchbuchner, and Arjan Kuijper. Post- comparison mitigation of demographic bias in face recognition using fair score normalization. Pattern Recognition Letters, 140:332 338, 2020b. ISSN 0167-8655. doi: https://doi.or//10.1001 2020. pii/ p11/50167865520300
URL http:// wwtp///wwww... www.::/eiineeeiieeeooooco. http://re.....ecceeeeee////atttt
Sahil Verma and Julia Rubin. Fairness definitions explained. In Proceedings of the International Workshop on Software Fairness, FairWare '18 pp. 1-7, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781450357463.
doi: 10.1145/1944700.1144766 URL
3194776
Mei Wang and Weihong Deng. Mitigating bias in face recognition using skewness-aware reinforce- ment learning. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Mei Wang. Weihong Deng, Jiani Hu, Xunqiang Tao, and Yaohai Huang. Racial faces in- wild: Reducing racial bias by information maximization adaptation network, 2019a.
13

---

Published as a conference paper at ICLR 2022
Pingyu Wang. Fei Su, Zhicheng Zhao, Yandong Guo, Yanyun Zhao, and Bojin Zhuang. Deep class-skewed learning for face recognition. Neurocomputing. 363:35 45, 2019b. ISSN 0925-2312. doi: https://bbboooooll.ooollloooomuu com/sci http:///e/eneerrrdcoee////////22 930832X
URL https: www www.nsecirecceetttttcccc........
Dong Yi, Zhen Lei, Shengcai Liao, and Stan Z. Li. Learning face representation from scratch, 2014.
Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, and Manmohan Chandraker. Feature transfer learning for face recognition with under-represented data. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Proceedings of the Eighteenth International Conference on Machine Learning. ICML '01, pp. 609-616, San Francisco, CA, USA, 2001. Morgan Kaufmann Publishers Inc. ISBN 1558607781.
Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10);1499-1500 10 2016. ISSN 1558 2361. doi: 10.1 20/1000222221122203333322 URL http:///ooo doi 1109/ LTw./LS..... 2016 2603442
14

---

Published as a conference paper at ICLR 2022
Supplemental Material
A EXAMPLES OF THE UNSUPERVISED CLUSTERS
In order to not rely on the sensitive attribute like the Oracle method, our FairCal method uses unsupervised clusters computed with the K-means algorithm based the feature embeddings of the images. We found them to have semantic meaning. Some examples are included in Figure 3
(a)
Figure 3: Examples of clusters obtained with the K-means algorithm (k: 100) the RFW dataset based on the feature embeddings computed with the FaceNet model: (a) Indian men with no facial hair, (b) Indian men with moustaches, (c) Caucasian women with blond hair, (d) young Asian women with dark hair.
B AGENDA METHOD
The Adversarial Gender De-biasing algorithm (AGENDA) learns shallow network that removes the gender information of the embeddings from a pre-trained network producing new embeddings. The algorithm entails training generator model M, classifier C and discriminator E. As proposed, the algorithm only removes the gender information using an adversarial loss that encourages the discriminator produce equal probabilities for both male and female gender. Since the RFW dataset
15

---

Published as a conference paper at ICLR 2022
contains the ethnicity attribute (as opposed to gender) and the BFW dataset contains both gender and ethnicity, we modify the loss to encourage the discriminator to produce equal probabilities amongst all possible sensitive attributes.
The shallow networks used for M, C' and E are the same as the ones specified in Dhar et al. 2020). with the exception that E has many outputs possible values of sensitive attributes (4 for RFW and 8 for BFW).
AGENDA requires the use of validation set to determine if the discriminator should continue be updated or not. Hence the embeddings used for training into 80/20 split, with the 20 split used for the validation.
In Stage 1, the generator M and classifier C are trained for 50 epochs. Then the bulk of training consists of 100 episodes: (i) Stage 2 is repeated every 10 episodes and consists in training the discriminator E' for 25 epochs; (ii) in Stage both M and C' are trained with the adversarial loss for 5 epochs with À 10; (iii) in Stage 4 the discriminator is updated for 5 epochs, unless its accuracy on the validations set is higher than 90%. All training is done with batch size of 400 and an ADAM optimizer with a learning rate 10-3 For more details, we refer to the code provided the supplemental material.
C PASS METHOD
Similar to AGENDA, PASS (Dhar et al.) (021)) learns a shallow network to removes the sensitive information of the embeddings from a pre-trained network producing new embeddings. The algorithm entails training generator model M, classifier C' and ensemble of discriminator E.
The shallow networks used for M, C' and E are the same as the ones specified in Dhar et al. (021)) Like AGENDA, it requires the use of a validation set. The same partition as in AGENDA was used.
In Stage 1, the generator M and classifier C are trained for 100 epochs. Then the bulk of training consists of 100 episodes: (i) Stage 2 is repeated every 10 episodes and consists in training the discriminator E for 100 epochs; (ii) in Stage both M and C are trained with the adversarial loss for 5 epochs with À 10; (iii) in Stage 4 use the One-At-a-time strategy update of the discriminators in the ensemble E. is updated for 5 epochs, unless its accuracy on the validations set is higher than 95%. All training done with batch size of 400 and ADAM optimizer with learning rate of 10-3.
The availability of sensitive attributes determined the choice of number of discriminators used. For the RFW dataset, since only the race is know, we use an ensemble with K = 2 discriminators. For the BFW, since both and gender race sensitive attribute and K = 3 discriminators for the gender sensitive attribute. The latter in fact constitutes the MultiPASS variant of PASS. We choose to name here PASS for simplicity. For more details, we refer to the code provided in the supplemental material.
known, we use an ensemble of K
discriminators for the
D FAIR TEMPLATE COMPARISON (FTC) METHOD
The Fair Template Comparison (FTC) method Terhörst et al. (2020a) learns a shallow network with the goal of outputting fairer decisions. We implemented the FTC method as follow. In order keep the ratios between the dimensions of layers the same as in the original paper Terhörst et al. (2020a). we used 512 512-dimensional input layer, followed by two 2048-dimensional intermediate layers. The final layer is a fully connected linear layer with 2-dimensional output with softmax activation. All intermediate layers are followed by a ReLU activation function and dropout (with = 0.3). The network was trained with batchsize of b 200 over 50 epochs, using an Adam optimizer with learning rate of 10-3 and weight decay of 10-3 Two losses, one based on subgroup fairness and the other both subgroup and individual fairness, were proposed in Terhörst et al. (2020a). Based on the paper's 's recommendations, we used the individual fairness loss with a trade-off parameter of A 0.5.
16

---

Published as a conference paper at ICLR 2022
E MEASURING CALIBRATION ERROR
There are different metries available measure if probabilistic classifier is calibrated fairly- calibrated. Calibration error is the error between the true and estimated confidences and is typically measured by the Expected Calibration Error (ECE) (Guo al. 2017):
Despite being the most popular calibration error metric, the ECF. has several weaknesses, chief among which is its dependence the binning scheme Nixon al. (219). Recently, Gupta al.] (2021) introduced simple, bin-free calibration measure. For calibrated scores P(Y c)==c)=c we have, by Bayes' rule:
=
Inspired by the Kolmogorov-Sminn Smirnov (KSL) statistic test, Gupta et (2021 proposed measure
the calibration error by comparing the cumulative distributions of P(Y 1,C c) and cP(C c). which empirically correspond to computing the sequences
h4-1 1 and h4 h C//N
with ho ho O, and N the total number of samples. Then the KS calibration error metric given by
KS = max h; h4
Another measure is the Brier score (BSL) (DeGroot & Fienberg 1983), which estimates the mean squared error between the correctness of prediction and the confidence score:
BS(D)
1
Σ
(1 c)2
(5)
D
(C...JED
For all the above metrics (ECE, KS, BS), lower better.
F FAIRNESS CALIBRATION
Since the calibration map produced by beta calibration is monotone, the ordering of the images provided by the scores is the same the ordering provided by the probabilities; therefore, the accuracy of the methods wheh thresholding remains unchanged. The calibration error (CE) measured (KS) test (described in Appenndix Ep is computed
with adaptation of the Kolmoggoov-Smimna for each subgroup of interest. Notice that for the BFW dataset we consider the eight subgroups that result from the intersection of the ethnicity and gender subgroups.
We first observe that all methods are equally globally calibrated (i.e., the calibration error is low) after the post-hoc calibration method is applied, except for the FTC the RFW dataset (see the Global column in Table 5 and Table 6).
notice that, after calibration, the Baseline method results in models that are not fairly-call callbecalibrated, though perhaps not in the way one would expect. Typically, bias directed against minority groups, but in this case, is the Caucasian subgroups that have the higher CEs. This is consequence of the models' above average accuracy on this subgroup, which is underestimated and therefore not captured by the calibration procedure. is important to point out that this is not failure of the calibration procedure, since the global CE (i.e., the CE measured on all pairs) low, as discussed above.
By inspecting Table 5 and Table
G EQUAL OPPORTUNITY (EQUAL FNR)
While equal opportunity (equal FNRs between subgroups) is not prioritized for the FR systems when used by law enforcement, it may be prioritized in different contexts such office building security. Empirically, our method also mitigates the equal opportunity bias at low global FNRs, as can seen in Table 7
17

---

Published as a conference paper at ICLR 2022
Table KS on all the pairs (Global (GI)) and on each ethnicity subgroup (African (Af), Asian (As),
Caucasian (Ca), Indian (In) using beta calibration the RFW dataset.
FaceNet (VGGFace22
FaceNet (Webface) Ca
(4)
GI
Af
As
Ca
In
GI
Af
As
In
Baseline
0.78 6.16 5.74 12.06 1.53
0.69
3.89 4.34
10.52
3.46
AGENDA
1.02 3.66 6.97 13.76
6.46
1.21 1.75
4.94
9.39
6.77
PASS
0.81 7.08 8.97
12.01
4.32
0.81
8.06
6.11
12.99
3,43
FTC
1.12 5.13 5.41
10.19
2.02
1.25 3.19 3.81
8.59
3.35
GST
0.77 1.99 3.66
2.30
1.40
0.71
2.01
4.97
3.83
2.16
FSN
0.77 1.27
1.62
1.49
1.35
0.85
1.94
3.06
1.70
3.27
FairCal (Ours)
0.81 1.11
1.41
1.29
1.70
0.70
1.48
1.62
1.68
2.21
Oracle (Ours)
0.76 0.99
1.28
1.2
1.25
0.62
1.54
1.46
1.13
1.25
Table 6: KS all the pairs (Global (Gl)) and each ethnicity and gender subgroup (African Females (AfF), African Males (AfM), Asian Females (AsF), Asian Males (AsM), Caucasian Females (CF), Caucasian Males (CM), Indian Females (IF), Indian Males (IM)) using beta calibration on the BFW dataset.
FaceNet (Webface) AsF
ArcFace AsF
GI
AIF
AfM
AsM
CF
CM
IF
IM
GI
AfF
AfM
AsM
CF
CM
IF
IM
Buseline
0.48
5.00
2.17
11.19
2.93
12.06
10.4
5,58
4,80
0.37
1.52
3.17
5,30
4.28
1.31
1.10
2.09
1.81
AGENDA
23.43
791
PASS
0.95
11.02
13.80
12.93
11.58
21.21
16.50
12.74
5.49
0.72
3.66
4.47
7.72
6.25
1.56
0.94
2.76
2.16
FTC
0.56
7.33
4.06
5.71
3.68
12.25
10.47
4.13
5.51
0.49
2.02
3.56
5.77
4.62
1.80
1.03
2.65
2.15
GST
1.62
4.52
4.61
2.34
2.72
0.48
3.50
7.46
5.03
1.59
FSN
0.39
2.35
3.12
4.16
4.40
1.50
0.99
3.54
2.02
0.38
1.74
3.01
5.70
4.30
1.02
1.15
2.45
1.81
FairCal (Ours)
0,59
3.83
2.55
2.92
3.79
3.70
2.43
3,21
2.32
0.49
L.73
3.12
4.79
3.81
1.05
1.66
2.28
1.97
Oracle (Ours)
0.43
1.67
2.3
2.83
2.49
0.67
1.24
4.6
2.02
0.32
1,26
0.99
1.93
1.72
0.86
1.15
1.64
1.74
Table 7: Equal opportunity: Each block of rows represents a choice of global FNR: 0.1% and %. For fixed global FNR, compare the deviations in subgroup FNRs in terms of three deviation measures: Average Absolute Deviation (AAD), Maximum Absolute Deviation (MAD), and Standard Deviation (STD) (lower is better).
RFW
BFW
FaceNet (VGGFace2)
FaceNet (Webfacc)
FaceNet (Wehface)
AreFacc MAD
(1)
AAD
MAD
STD
AAD
MAD
STD
AAD
MAD
STD
AAD
Baselinc
0.09
0.13
0.10
0.10
0.16
0.11
0.09
0.23
0.11
0.11
0.31
AGENDA
0.11
0.22
0.13
0.11
0.14
0.34
0.16
0.09
0.24
PASS
0.09
0.14
0.10
0.11
0.20
0.12
0.10
0.33
0.14
0.10
0.31
0.14
FTC
0.09
0.11
0.09
0.08
0.14
0.1
0.04
0.09
0.05
0.06
0.14
0.07
0.1%
GST
0.09
0.13
0.10
0.12
0.21
0.13
0.10
0.26
0.12
0.12
0.37
0.15
FSN
0.09
0.13
0.09
0.09
0.14
0.10
0.07
0.22
0.10
0.12
0.33
0.15
FairCal (Ours)
0.10
0.14
0.10
0.11
0.17
0.12
0.10
0.27
0.13
0.09
0.17
0.10
Oracle (Ours)
0.11
0.12
0.13
0.24
0.11
0.32
Baseline
0.60
0.96
0.67
0.45
0.81
0.53
0.39
0.84
0.47
0.75
1.85
0.93
AGENDA
0.99
1.97
2.39
1.15
0.72
1.54
1% FNR
PASS
0.77
1.06
0.81
1.64
0.99
0.72
1.79
0.89
0.71
1.76
FTC
0.48
0.83
0.56
0.32
0.58
0.38
0.3
0.62
0.34
0.49
1.12
0.6
GST
0.39
0.60
0.44
0.54
0.96
0.62
0.49
1.02
0.57
0.83
2.27
1.07
FSN
0.28
0.47
0.32
0.40
0.78
0.48
0.41
0.92
0.49
0.77
1.91
0.96
FairCal (Ours)
0.30
0.51
0.34
0.39
0.72
0.48
0.32
0.74
0.40
0.65
1.48
0.80
Oracle (Ours)
0.38
0.67
0.42
0.67
0.37
0.77
0.5
1.11
18

---

Published as a conference paper at ICLR 2022
Table 8: Fairness calibration: measured by the mean KS across the sensitive subgroups on the UB-C dataset. Bias: measured by the deviations of KS across subgroups in terms of three deviation measures: Average Absolute Deviation (AAD), Maximum Absolute Deviation (MAD), and Standard Deviation (STD). (Lower is better in all cases.)
FaceNet (VGGFace2)
FaceNet (Webface) AAD
(4)
Mean
AAD
MAD
STD
Mean
MAD
STD
Baseline
3.54
1.90
5.46
2.38
3.57
2.01
4.60
2.37
AGENDA
2.61
1.31
3.82
1.65
3.41
1.28
4.33
1.76
PASS
7.22
3.24
8.40
3.98
6.57
3.50
9.12
4.25
FTC
2.92
1.44
3.98
1.80
2.98
1.41
3,43
1.69
GST
2.38
1.36
5.09
1.88
3.52
1.59
6.88
2.42
FSN
2.12
1.18
3.74
1.53
2.45
1.05
3.49
1.42
FairCal (Ours)
1.8
0.95
2.81
1.2
2.12
0.99
2.51
1.18
Oracle (Ours)
1.84
1.05
4.32
1.53
1.87
0.91
3.23
1.27
Table Global accuracy (higher is better) measured by AUROC, and TPR at two FPR thresholds on the IJBC dataset.
FaceNet (VGGFace2) TPR @
FaceNet (Webface) TPR @
TPR @
TPR @
(t)
AUROC
0.1% FPR
1% FPR
AUROC
0.1% FPR
1% FPR
Baseline AGENDA
92.72
43.53
62.80
89.23
19.50
50.10
89.86
44.78
61.65
81.33
32.90
46.71
PASS
86.68
24.06
39.46
81.54
9.89
28.66
FTC
91.60
39.41
60.96
87.64
17.70
47.24
GST
92.78
47.65
66.34
89.13
25.06
50.57
FSN
92.48
53.4
68.29
88.00
44.74
58.52
FairCal (Ours)
94.74
52.54
68.45
92.04
44.28
58.98
Oracle (Ours)
93.18
48.4
66.96
89.8
24.13
53.62
H IJB-C RESULTS
We report additional results in Table 9 Table 8 and Table 10 on the IJB-C dataset (Maze al. 2018 following the same guidelines as the RFW and BFW datasets. We include results for the FaceNet (VGGFace2) and FaceNet (Webface) models and they are in agreement with the results presented in the main paper RFW and BFW. (We do not include results for the ArcFace model were unable to overcome software issues to compute its embeddings).
I STANDARD POST-HOC CALIBRATION METHODS
For completeness, we provide brief description of the post-hoc calibration methods used in this work. Beta calibration Kull et al. (017)) was used to obtain our main results, but we show below that choosing others methods (Histogram Binning (Zadrozny & Elkan 2001) or Isotonic Regression (Zadrozny & Elkan 2001; Miculescu-Mizz our FairCal method.
Caruana 2005)) does not impact the performance of
Throughout this section, we denote by peal the pairs of images the calibration set and geal their respective cosine similarity scores such that
Scal {s(()) (0)) ) (0i, .22) E peal].
19

---

Published as a conference paper at ICLR 2022
Table 10: Predictive equality: For two choices of global FPR (two blocks of rows): 0.1% and 1%, we compare, on the UB-C dataset, the deviations in subgroup FPRs in terms of: Average Absolute Deviation (AAD), Maximum Absolute Deviation (MAD), and Standard Deviation (STD). (lower better in all cases)
FaceNet (VGGFace2)
FaceNet (Webface)
AAD
MAD
STD
AAD
MAD
STD
Baseline
0.10
0.33
0.13
0.11
0.51
0.17
AGENDA
0.09
0.28
0.12
0.09
0.37
0.13
0.1% FPR
PASS
0.12
0.48
0.17
0.13
0.65
0.21
FTC
0.10
0.33
0.13
0.10
0.47
0.16
GST
0.08
0.33
0.12
0.07
0.24
0.10
FSN
0.06
0.24
0.09
0.06
0.20
0.08
FairCal (Ours)
0.06
0.25
0.09
0.05
0.13
0.06
Oracle (Ours)
0.06
0.26
0./
0.08
0.27
0.11
Baseline
0.93
3.15
1.23
0.85
3.41
1.21
AGENDA
0.65
2.04
0.85
0.63
2.60
0.94
1% FPR
PASS
1.10
3.69
1.45
1.16
4.94
1.72
FTC
0.90
2.88
1.15
0.81
3.13
1.14
GST
0.60
2.64
0.92
0.68
2.51
0.95
FSN
0.52
1.85
0.70
0.49
1.65
0.65
FairCal (Ours)
0.47
1.72
0.66
0.44
1.33
0,56
Oracle (Ours)
0.46
1.85
0.69
0.47
1.52
0.64
1.1 HISTOGRAM BINNING
In Histogram Binning Zadrozny Elkan 2001 we partition geal into m bins B3 with equal-mass, where 1, ..... m. Then, given pair of images (æ1, (2) with score s(x1, x22) Bi, we define
c( .2)
1
Σ
[(tb))=(((b))
B&
EE (ar (m).#g))Peal)
In other words, simply count the number of scores in each bin that correspond to genuine pairs of images, i.e., images that belong to the same person. By construction, a confidence score (Equation 6) satisfies the binned version of the standard calibration (Definition T]. As for the bins, they can be chosen so as to have equal mass or to be equally spaced, or else by maximizing mutual information, as recently proposed in Patel et al. (20TT) In this work, we created bins with equal mass.
Despite being an extremely computationally efficient method and providing good calibration, his- togram binning not guaranteed to preserve the monotonicity hetween scores and confidences, which is typically desired property. Monotonicity ensures that the accuracy of the classifier is the same when thresholding either the scores or the calibrated confidences.
1.2 ISOTONIC REGRESSION
Isotonic Regression (Zadrozny & Elkan 2001 function μ IR + R by solving
Niculescu-Mizil & Caruanal 2005) learns monotonic
arg min #
Σ
(n(s(01, - 1 11()))=
Peal
(a EPeall
The confidence score is then given by c(æ1, x22 (((((111111 x2))
20

---

Published as a conference paper at ICLR 2022
1.3 BETA CALIBRATION
Beta calibration PPttel al. 2021 is parametric calibration method, which learns calibration map a IR- + R of the form
co p1s; 01,
1 1+1/ ( 20
(1-a)²2
where the parameters (0,,02,00 0,) 0z) E RR& are chosen by minimizing the log-loss function
LL(e, v) y log(c)) (1 y)( [og(1 - c))
where c = ((s((11 By restricting, and to be positive, the calibration map is monotone.
J ROBUSTNESS OF FAIRCAL RESULTS TO HYPERPARAMETERS
In this section we show that the results presented in the main paper still hold we vary model hyperparameters, such as the number K of clusters used in FairCal, and the calibration method.
Choice of post-hoc calibration: The implementation of the FairCal method requires choosing post-hoc calibration method and the number of clusters K' in the K K-means algorithm. Our method is (the metric interest when comes
robust to the choice of both with respect to fairness-calibrateo to calibration) and its bias depicted in Figure Figure 5 Figure 6 and Figure 7 We compare with binning and isotonic regression. For the former, we chose and 25 bins for the RFW and BFW datasets, respectively, given the different number of pairs in each dataset.
Comparison FSN (Terhörst et al. 2020b): The improved performance of FairCal over FSN is consistent across different choices of K. Fixing the choice of the post-hoc calibration method as beta calibration as in the results in the paper, we compare the two, together with Baseline and Oracle for
additional baselines. Results are displayed in Figure 8, Figure 9. Figure 10, Figure 11
Figure 12
Figure 13 and Figure 14
K EXTENDING FAIRCAL TO MULTI-CLASS CLASSIFICATION
Consider a probabilistic classifier p : X' + AІ that outputs that outputs class probabilities for classes 1,...,k, k. Here X' denotes the input space and A& = {(g...... 9k) Σ Qk 1}. Suppose is computed with deep neural network where f denotes its feature encoder and g the final fully connected layer. Then we write p = softmax(g( (x))). Post-hoc calibration methods (like Dirichlet calibration (Kull et al., 2019). the generalization of beta calibration) compute calibration map : A + A using a calibration set cal C X' such that fa p is calibrated.
FairCal can be extended multi-class classification follows:
Compute geal {f(x)) X
(ii) Apply the K-means algorithm to the image features, Zral partitioning the embedding space Z into K clusters Z,.... Zz. These form the K calibration sets:
xcall {a f(x) Z+}.
k=L...,K
(iii) For each calibration set x use a post-hoc calibration method to compute the calibration map [Ak-
(iv) For new input æ find the clusters it belongs to and output (4u (p(x))
Notice that in the multi-class classification setting, each input (now single image) belongs to one and only one cluster, thus simplifying the final setup. We leave for future work its implementation and study through thorough empirical evaluation.
longer dealing with pairs and therefore
21

---

Published as a conference paper at ICLR 2022
L RESULTS PRESENTED WITH STANDARD DEVIATIONS
Recall that the results presented in the main text were computed by taking the mean of 5-fold leave-one-out vasssavaii,..n.. Below, we report the corresponding standard deviations of the five
folds. The standard deviations for the results on accuracy reported in Table 2 can be found in
Table 11. Table 12 Table 13 For calbbraton in Table they can be found in Table
Table 15 Table 16 Table 17
Finally, for predictive equality and equal opportunity in Table 4 and
Table 7 they can found Table 18 Table 19
Table 20 and Table 21
Table 22
Table 23
NPW Facellc (VGGFace2)
AFW - Facellet (Webface)
BPW BPW-Faceller FacelNet (Wubficc)
BFW) ArcFace
Bata Calorsson
Bdd callroion botont Regrosiee
Dala
Iuderic
Istiana Ragroailn
Figure 4: Comparison of chirness-calibration as measured by the subgroup mean of the KS across the sensitive subgroups for different values of and different choices of post-hoc calibration methods. Shaded regions refer to the standard error across the 5 different folds in the datasets.
RPW -wccNatt (VGGFace2)
FFW FaceNett (Weoface)
BPW FaceNet (Wiebface)
BFW ArcFace
teation
Dota
Callerti
fegrevite
boder:: Nog moae
betbn Regrossaa
betans Rogrossan
6
Figure 5: Bias in fairnass as measured by the AAD (Average Absolute Deviation) in the KS across the sensitive subgroups for different values of K and different choices of post-hoc calibration methods. Shaded regions refer to the standard error across the 5 different folds in the datasets.
Fets Calbottonn
Bots Cilanriinn
BHO Callrriin
I-D Callowwn
20 10 250560
75 197190255515
10
100 150 25530
Figure 6: Bias in fairness-calibration as measured by the MAD (Maximum Absolute Deviation) in the KS across the sensitive subgroups for different values of K and different choices of post- hoc calibration methods. Shaded regions refer to the standard error across the different folds in the datasets.
M ADDITIONAL FIGURES
In Figure I provided qualitative analysis of the improved faimess/reduction in bias in terms of predictive equality on the RFW dataset with the FaceNet (Webface) model. We replicate this figure for the FaceNet (VGGFace2) model on the RFW dataset in Figure 15, and the BFW dataset for the FaceNet (Webface) model in Figure 16 and ArcFace model in Figure 17
22

---

Published as a conference paper at ICLR 2022
Table 11: Global accuracy measured by the AUROC.
RFW
BFW
(t)
FaceNet (VGGFace22
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baseline
88.26:1: 0.19
83.95 0.22
96.061 0.16
97.41 0.34
AGENDA
76.83: 0,57
74,51 0.94
82.42: 0.45
95.09= 0.55
PASS
86.961 0.46
81.44 0.91
92.271 0.72
96.551 0.28
FTC
86.46 0.17
81.61 0.57
93.30 0.70
96.41
GST
89.571 0.25
84.8811 0.18
96.591 0.20
96.891 0.38
FSN
90.05 0.26
85.84 0.34
96.77 0.20
97.35= 0.33
FairCal (Ours)
90.581 0.29
86.711 0.25
96.90.1 0.17
97.441 0.34
Oracle (Ours)
89.741 0.31
85,23 0.18
97.281 0.13
98.91 0.12
Table 12: Global accuracy measured by the TPR at 0.1% FPR threshold.
RFW
BFW
FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baseline
18.42:1 1.28
11.18 3.45
33.61 2.10
86.27 1.09
AGENDA
8.32. 1.86
6.38 0.78
15.95= 1.53
69.61 2.40
PASS
13.674 2.39
7.34= 1.74
17.21 3.04
77.38+ 1.51
FTC
6.86 5.24
4.65= 2.10
13.60 4.92
82.09 1.11
GST
22.61 1.52
17.34-t 2.10
44.49- 1.13
86.13+
FSN
23.01 2,00
17.334 3.01
47.11 1,23
86.19+ 1.13
FairCal (Ours)
23,55 1.82
20.64 3.09
46.74 1.49
86.28= 1.24
Oracle (Ours)
21,40 3.54
16.71 1.98
45.13 1.45
86.41 1.19
Table 13: Global accuracy measured by the TPR at 1% FPR threshold.
RFW
BFW
(t) FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baseline
34.88 3.27
26.04+ 2.11
58.87 0.92
90.11 0.87
AGENDA
18.01 1.44
14.98 1.11
32.511 1.24
79.67=2.00
PASS
29.30+t 2.24
20.93 1.12
38.324 5.48
85.26 0.79
FTC
23.661 6.58
18.40 4.02
43.09 5.70
88.241 0.63
GST
40.724 2.73
31.56 1.18
66.71+
89.70= 0.85
FSN
40,21 2.09
32.80 1.03
68.9211 1.01
90.06: 0.84
FairCal (Ours)
4 1.99
33.13+ 1.67
69.21 1.19
90.14+ 0.86
Oracle (Ours)
47.83+ 2.98
31.60+ 1.08
67.564 1.05
90.40+ 0.97
Table 14: Fairness-calibration as measured by the mean KS across sensitive subgroups.
RFW
BFW
(4) FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baseline
6.37+ 0.35
5.55# 0.14
6.77+ 0.57
2.574 0.43
AGENDA
7.71 0.27
5.71 0.28
13.2+ 1.04
5.144 0.40
PASS
8.09: 1.16
7.65+t 1.08
13.1 2,90
3.694 0.23
FTC
5,69 0.14
4.73+ 0.53
6.64#0.41
2.95 0.45
GST
2.34# 0.05
3.24+ 0.40
3.09+ 0.22
3.34 0.28
FSN
1.43 0.28
2.49 0.46
2.76 0.21
2.654 0.43
FairCal (Ours)
1.37+ 0.17
1.75 0.26
3.09+ 0.37
2.49 0.43
Oracle (Ours)
1.18 0.05
1.35+ 0.09
2.23# 0.14
1.41+ 0.33
23

---

Published as a conference paper at ICLR 2022
Table 15: Bias in fairness-calibratio measured by the deviations of KS across subgroups in terms of AAD (Average Absolute Deviation).
RFW
BFW
(4)
FaceNel (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baseline AGENDA
2.89-t 0.29
2.48+ 0.36
3.63:t 0.63
1.394 0.28
3.11# 0.25
2.37 0.33
6.37 0.62
2.48 0.50
PASS
2,40 0.76
3.36 0.87
5.25+ 1.39
2.01+ 0.20
FTC
2.32+ 0.28
1.93+ 0.35
2.80+ 0.55
1,48+ 0.31
GST
0,82 0.22
1.21 0.41
1.45= 0.27
1.81 0.43
FSN
0.35+ 0.15
0.84+ 0.38
1.38+ 0.27
1.45+ 0.31
FairCal (Ours)
0.28 0.12
0.41 0.19
1.34 0.24
1.30 0.26
Oracle (Ours)
0.28 0.08
0.38 0.20
1.15 0.24
0.59 0.18
Table 16: Bias in fairness-calibration as measured by the deviations of KS across subgroups in terms of MAD (Maximum Absolute Deviation).
RFW
BFW
(4)
FaceNet (VGGFace22
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baseline AGENDA
5.73 0.63
4.97 0.72
5,96 1.05
2.941 0.99
6.09+ 0.65
4.28+ 0.38
12.9 0).77
5.92+ 1.86
PASS
4.10= 1.11
5.34 1.43
9.58 1.81
4.24+
FTC
4.51 0.64
3,86 0.70
5,61 0.66
3.03 0.88
GST
1.58+ 0.39
1.93+ 0.56
2.80 0.64
4.21 1.38
FSN
0.57 0.21
1.19
2.67 0.32
3.234 0.99
FairCal (Ours)
0.50 0.15
0.64 0.28
2.48 0.41
2.684 1.07
Oracle (Ours)
0.53 0.18
0.66+ 0.28
2.63+ 0.60
1.30+ 0.29
Table 17: Bias in failmass-calibrattonn measured by the deviations of KS across subgroups in terms of STD (Standard Deviation).
RFW
BFW
(4)
FaceNet (VGGFace22
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baseline AGENDA
3.77# 0.33
2.91 0.41
4.03:t 0.70
1.63+ 0.40
3,86: 0.24
2,85 0.33
7.55 0.60
3.04 0.65
PASS
2,83# 0.83
3.85 0.91
6.12+ 1.54
2.37+ 0.30
FTC
2.95 0.32
2.28 0.43
3.27 0.46
1.741 0.42
GST
0 0,26
1.34 0.40
1.65 0.26
2.19 0.55
FSN
0.40 0.15
0.91 0.36
1.60 0.23
1.71 0.41
FairCal (Ours)
0.34 0.12
0.45 0.20
1.55 0.24
1.524 0.37
Oracle (Ours)
0.33+ 0.10
0..3+ 0.20
1.40+ 0.27
0.69+ 0.18
24

---

Published as a conference paper at ICLR 2022
Predictive equality: Each block of rows represents a choice of global FPR: 0.1% and 1%. For fixed a global FPR, compare the deviations subgroup FPRs terms of AAD (Average Absolute Deviation). We report the average and standard deviation error across the folds.
Table 18:
RFW
BFW
(4)
FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baseline AGENDA
0.10 0.02
0.144
0.29 0.04
0.12+
0.11+ 0.04
0.12+ 0.03
0.14+ 0.04
0.09- 0.03
0.1% FPR
PASS
0.11+ 0.02
0.114 0.02
0.36+ 0.03
0.12+ 0.03
FTC
0.10+ 0.02
0.12+ 0.04
0.24+ 0.02
0.09+ 0.02
GST
0.13+ 0.02
0.09+
0.13+ 0.03
0.10+
FSN
0.10+ 0.05
0.11+ 0.04
0.09+ 0.03
0.11+ 0.02
FairCal (Ours)
0.09 0.03
0,09 0.03
0.09 0.02
0.111 0.03
Oracle (Ours)
0.11+ 0.05
0,114 0.03
0.12+ 0.03
0,12+ 0.04
Baseline
0.68+ 0.06
0.67+
2.42+ 0.14
0.72+ 0.19
AGENDA
0.73+ 0.11
0,73+ 0.08
1.21 0.27
0.65+ 0.13
1% FPR
PASS
0.89+ 0.10
0.68+ 0.19
3.30+ 0.25
0.72+ 0.13
FTC
0.60 0.11
0.54+ 0.12
1.94+ 0.22
0.54+ 0.09
GST
0.52+ 0.12
0,30+t 0.03
1.05+ 0.15
0,44+
FSN
0.37 0.12
0.35+ 0.16
0.87 0.11
0.55 0.11
FairCal (Ours)
0.28 0.11
0.291 0.10
0.80 0.10
0.63 0.15
Oracle (Ours)
0.40+ 0.09
0.41+ 0.10
0.77+ 0.17
0,83+ 0.15
Table 19: Predictive equality: Each block of rows represents a choice of global FPR: 0.1% and 1%. For fixed global FPR, compare the deviations in subgroup FPRs in terms of MAD (Maximum Absolute Deviation). We report the average and standard deviation eтror across the folds.
RFW
(L)
FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baseline AGENDA
0.15. 0.05
0.26.L
1.00.L 0.28
0.30.L 0.08
0,20.L 0.10
0,23.L 0.07
0.40.1 0.16
0,231 0.10
0.1% FPR
PASS
0.181L 0.04
0.18.L 0.03
1.21 0.26
0.29 0.11
FTC
0.15.L 0.03
0.23.L 0.08
0.74-1 0.22
0.20 0.03
GST
0,244 0.06
0.16.L
0.354L 0.16
0.241 0.06
FSN
0.18 0.10
0,23-t 0.07
0,20 0.06
0.28 0.08
FairCal (Ours)
0.14: 0.04
0.164 0.06
0.20 0.04
0.31+ 0.10
Oracle (Ours)
0.191 0.10
0.20.L 0.07
0.25 0.06
0,27 0.09
Baseline
1.02.L
1.23-L
7.48 1.75
1.51 0.44
AGENDA
1.144 0.22
1,08 0.10
3.09+ 1.06
1.78 0.76
1% FPR
PASS
1.524 0.37
0,994 0.08
10.14 2.31
2.00+: 0.64
FTC
0.91+ 0.08
1.054 0.17
5.74+ 1.73
1.044 0.15
GST
0.92+ 0.25
0,574 0.06
3.01+ 1.04
1.13+ 0.39
FSN
0.68+ 0.23
0.61+ 0.25
2.19+ 0.58
1.27+ 0.35
FairCal (Ours)
0.46+ 0.16
0.57+ 0.23
1.79 0.54
1.46+ 0.29
Oracle (Ours)
0.69- 0.19
0.744 0.23
1.71 0.59
2.08 0.57
25

---

Published as a conference paper at ICLR 2022
Table 20: Predictive equality: Each block rows represents a choice of global FPR: 0.1% and 1%. For fixed global FPR, compare the deviations subgroup FPRs in terms of STD (Standard Deviation). We report the average and standard deviation error across the 5 folds.
RFW
BFW
(4)
FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baseline AGENDA
0.10 0.03
0.164
0.40 0.09
0.15+ 0.04
0.134 0.05
0,141 0.04
0,182 0.05
0.11 0.04
0.1% FPR
PASS
0.124 0.03
0,12: 0.02
0,49: 0.08
0.14# 0.04
FTC
0.11# 0,02
0.141 0.05
0.32+ 0.05
0.11# 0.02
GST
0.154 0.03
0.101 0.03
0.16: 0.05
0.121 0.03
FSN
0,11 0.06
0.13: 0.04
0.11 0.03
0.14 0.03
FairCal (Ours)
0.101 0.03
0.101 0.03
0.11 0.03
0.151 0.03
Oracle (Ours)
0.12+ 0.05
0.134 0.03
0.15+ 0.03
0.144 0.04
Baseline
0.74+ 0.04
0.79+
3,22+ 0.44
0.85+ 0.20
AGENDA
0.81 0.11
0,7811 0.06
1,51 0.33
0.841 0,23
1% FPR
PASS
1.01 0.16
0.73 0.15
4.34 0.53
0.931 0.19
FTC
0,661 0.09
0.66.1 0.12
2.57. 0.45
0.61 0.08
GST
0.60 0.14
0.37
1.38 0.29
0.56 0.18
FSN
0,46 0.14
0,40.L 0.17
1.05 0.18
0.68 0.14
FairCal (Ours)
0.32.1 0.12
0.35 0.13
0.95.L 0.16
0.78. 0.15
Oracle (Ours)
0.45+ 0.11
0.48+ 0.12
0.91+ 0.22
1.07 0.18
Table 21: Equal opportunity: Each block of rows represents choice of global FNR: 0. 1% and I 1%. For a fixed global FNR, compare the deviations in subgroup FNRs in terms of AAD (Average Absolute Deviation). We report the average and standard deviation error across the folds.
RFW
BFW
(4) FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baselinc AGENDA
0.09 0.01
0.10# 0.02
0.09 0.03
0.11 0.02
0.11 0.04
0.10# 0.02
0.14 0.01
0.09: 0,02
0.1% FPR
PASS
0.09: 0.03
0.11 0.03
0.10# 0.02
0.10= 0.02
FTC
0.09= 0.01
0.08=: 0.03
0.04 0.02
0.06= 0.01
GST
0.09 0.02
0.12# 0.02
0.10: 0.03
0.12# 0.01
FSN
0.09: 0.02
0.09: 0.02
0.07 0.02
0.12# 0.01
FairCal (Ours)
0.10: 0.02
0.11# 0.02
0.10# 0.02
0.09 0.02
Oracle (Ours)
0.11.L 0.02
0.12.1 0.02
0.09.1 0.02
0.11 0.02
Baseline
0.60-L 0.17
0.45-L: 0.09
0.39.L 0.05
0.75.1 0.16
AGENDA
0.99.1 0.24
0.6741 0.17
0 0.09
0.72. 0.19
1% FPR
PASS
0.77+ 0.10
0.834: 0.14
0.72- 0.13
0.71 0.16
FTC
0,48+ 0.06
0.32-1 0.12
0.30 0.07
0.49 0.14
GST
0.39+ 0.05
0.54+ 0.16
0.49-t
0.83+
FSN
0.28 0.06
0.40+ 0.19
0,41+ 0.10
0.77+ 0.17
FairCal (Ours)
0.30+ 0.14
0.39+ 0.12
0.32+ 0.10
0.65+ 0.11
Oracle (Ours)
0,38 0.15
0.56# 0.11
0.37+ 0.09
0.50# 0.10
26

---

Published as a conference paper at ICLR 2022
Table 22: Equal opportunity: Each block of rows represents choice of global FNR: 0.1% and 1%. For fixed global FNR, compare the deviations in subgroup FNRs in terms of MAD (Maximum Absolute Deviation). We report the average and standard deviation error across the folds.
RFW
BFW
(4) FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFace
Baseline AGENDA
0.131 0.02
0.161 0.08
0.23# 0.11
0.31 0.07
0,221 0.08
0.144 0.08
0.34 0.05
0.24# 0.09
0.1% FPR
PASS
0,141 0.06
0.20= 0.08
0.33 0.12
0.31 0.14
FTC
0,111 0.02
0.14: 0.07
0.09 0.03
0.14= 0.04
GST
0,131 0.06
0.21 0.07
0.26 0.12
0.37= 0.05
FSN
0,131 0.06
0.141 0.06
0.22 0.11
0.332 0.06
FairCal (Ours)
0,141 0.06
0.171 0.09
0.27 0.09
0.17 0.05
Oracle (Ours)
0,18+ 0.07
0,21+ 0.08
0,24+ 0.08
0.32+ 0.15
Baseline
0 0.21
0.81 0.14
0.84 0.14
1,85 0.66
AGENDA
1,97 0.48
1.33 0.35
2.39 0.74
1.54= 0.42
1% FPR
PASS
1,061 0.20
1.641 0.28
1.79 0.71
1.76 0.74
FTC
0,831 0.21
0,58 0.24
0.62.1 0.11
1.12 0.29
GST
0.60:1 0.07
0.961 0.35
1.02. 0.39
2.27 0.76
FSN
0.47 0.15
0.781 0.38
0.92 0.28
1.91 0.67
FairCal (Ours)
0.511 0.25
0.721 0.20
0.74. 0.18
1.48 0.36
Oracle (Ours)
0.61+ 0.15
1.06+ 0.18
0.77+ 0.19
1.114 0.27
Table 23: Equal opportunity: Each block of rows represents a choice of global FNR: 0.1 and 1%. For fixed global FNR, compare the deviations in subgroup FNRs in terms of STD (Standard Deviation). We report the average and standard deviation error across the 5 folds.
RFW
BFW
(4) FaceNet (VGGFace2)
FaceNet (Webface)
FaceNet (Webface)
ArcFacc
Baselinc AGENDA
0.10# 0.01
0.11 0.03
0.11 0.04
0.14 0.02
0.13 0.04
0,11# 0.03
0.16 0.02
0.12# 0.03
0.1% FPR
PASS
0.10=t 0.03
0.12+ 0.04
0.14 0.04
0.14= 0.04
FTC
0.09 0.01
0.10= 0.03
0,05 0.02
0.07= 0.02
GST
0.10# 0.03
0.13# 0.03
0.12# 0.04
0.15# 0.02
FSN
0.09: 0.03
0,10= 0.03
0.10 0.04
0.15 0.02
FairCal (Ours)
0.10# 0.02
0.12# 0.03
0.13# 0.03
0.10= 0.02
Oracle (Ours)
0.12.1 0.03
0.131 0.03
0.111 0.03
0.141 0.04
Baselinc
0.67 0.15
0.53# 0.09
0,47 0.06
0.93# 0.23
AGENDA
1.16 0.27
0.81 0.19
1.15# 0.17
0.84 0.22
1% FPR
PASS
0,81 0.10
0.99 0.15
0.89 0.16
0.90= 0.24
FTC
0,56 0.10
0.38+ 0.15
0.34 0.06
0.60 0.16
GST
0,44 0.04
0.62+ 0.18
0.57::..4
1.07
FSN
0.32+ 0.09
0,48 0.23
0.49#
0.96 0.23
FairCal (Ours)
0.34 0.17
0.48# 0.14
0.40 0.10
0.80 0.13
Oracle (Ours)
0.42. 0.14
0.671 0.11
0.441 0.10
0.60 0.12
27

---

Published as a conference paper at ICLR 2022
RFW- (VGGfacee)
Faceller (Webface)
Facelde (Vetffsce)
BFW Areface
Colltation lrving Aegrestion
Deta Caloritio
Data Calloraien
Calloraion
ariing
Bniting
luaeli: log saionn
lotbne Regraaden
vadda
20 25 10
100:50 256560
35 15 233399353366
10
20
75
10
201530
Figure Bias in caibrasn as measured by the STD (Standard Deviation) in the KS across the sensitive subgroups for different values of K and different choices of post-hoc calibration methods. Shaded regions refer to the standard error across the different folds the datasets.
0.007
Hi
0.01
6907
FG)
0.83
Eeaiin
Dulte (hurk
Figure 8: Global accuracy measured by the AUROC for different values of K for Baseline, FSN Terhörst al. (2020b), FairCal, and Oracle methods. Shaded regions refer to the standard error across the 5 different folds in the datasets.
HPW: Fosshet (VOOP2
Paschec
BMW
6275
Basch
8,115
EMPO 00.71
0.20
8,193
0329
3.06
0.50
(7a4)
3.00
0.5
63300
Figure 9: Global accuracy measure by the TPR at different a global 0.1% FPR for different values of K for Baseline, FSN Terhörst et (2020b), FairCal, and Oracle methods. Shaded regions refer to the standard error across the 5 different folds in the datasets.
Turnhlet (667am)3)
(wethees)
(Wlechow)
3TW wwW-Aeefaae
6.79
Dexeine
0.2
(accal Dom
0.48
Sser
3:30
0.3
Bainee
0:42
0.000
8.8%
4.0
500
15
121 350140
1003 50
Figure 10: Global accuracy measure by the TPR at different a global 1% FPR for different values of K for Baseline, FSN Terhörst et al. (2020b), FairCal, and Oracle methods. Shaded regions refer to the standard error across the 5 different folds in the datasets.
RPW Fazelet (VGGFace2)
RFW Facelet (Webface)
BPW Facellet (Wiebface)
BFV ArcFace
PM
Faarall un oraite shur
Farca (Curo Crade (cur
Faxcall (Duro Cradi (Durt
Faircall tound Cradie 1Ount
Figure 11: Comparison of fairness-calibration as measured by the subgroup mean of the KS across the sensitive subgroups for different values of K for Baseline, FSN Terhörst al. (2020b). FairCal, and Oracle methods. Shaded regions refer to the standard error across the 5 different folds in the datasets.
28

---

Published as a conference paper at ICLR 2022
RPW FaceMet (VGGFacc2)
Facellet (Webhace)
BPW FaceMet (Viefface)
BFW: Arcface
Bastin
Zaminne
furica (Cura) Druck (Curs)
Fer (Dus))
Fance (Puss
Cradie iDunt
Drocie TOurs
FeaCill 10 Oradie 10urS
Figure 12: Bias in fairness-calibration as measured by the AAD (Average Absolute Deviation) in the KS across the sensitive subgroups for different values of K for Baseline, FSN [Terhrrst et al. (2020b), FairCal, and Oracle methods. Shaded regions refer to the standard error across the 5 different folds in the datasets.
RFW Farelet (VGGFace2)
BFV Fww.-Faaeeetteeeee (Webtbce)
BFW FoceNet (Webfacc)
BFW ArcFace
Easelne
FSN
Fancill 1ount
(0uns)
Dasiin
Dascin
F5A
7SM
TerCe (Cora)
TeaCal
310:5223056
100
10
500150
300 2033-0
Figure 13: Bias in fairness-calib as measured by the MAD (Maximum Absolute Deviation) in the KS across the sensitive subgroups for different values of K for Bascline, FSN Terhörst et al. (2020)), FairCal, and Oracle methods. Shaded regions refer to the standard error across the 5 different tolds in the datasets.
(VGGFace2)
ArcFace
Bocine
Raskine
Faical (dun)) doicle Jouen
oedte tDurt
Ceaite ung
FarCal (Ceno
Cradi (Cur
250300
50150220052
Bias in fairness-calibrate measured by the STD (Standard Deviation) in the KS across the sensitive subgroups for different values of K for Baseline, FSN Terhörst et al. (2020b), FairCal, and Oracle methods. Shaded regions refer the standard error across the 5 different folds in the datasets.
Figure 14:
Baseline
AGENDA
FTC
FSN
FairCal (Ours)
False Positive Rate
Mittnn
Nricen
Nritat
Ni can
Aricen
Adian
Asian
Asian
Asiani
Asian
Caucationy
Caucds an
Cauracian
Caucacian
Caucasian
0.1.0
00 0.02 on 0.08
0.00 0.02 0.44 0.08
Global FPR
Global FPR
Global FPR
Global FPR
Global FPR
Figure 15: (Lines closer together is better for fairness, best viewed in colour) Illustration of improved fairness reduction in bias, as measured by the FPRs evaluated on intra-ethnicity pairs on the RFW dataset with the FaceNet (VGGFace2) feature model.
29

---

Published as a conference paper at ICLR 2022
Baseline
AGENDA
FTC
FairCal (Ours)
False Positive Rate
Alitatt
Ari
Arisatt
Alr can
Arisan
Asian
Asian
Asian
Caucasian
Indlan
Indian
Indian
Indian
Indian
0.15
0.10
0.05
0.00
0.08 Global FPR
0.02
0.02 0:04 Global FPR
Global FPR
Global FPR
Global FPR
Figure 16: (Lines closer together is better for fairness, best viewed in colour) Illustration of improved fairness reduction in bias, as measured by the FPRs evaluated on intra-cthnicity pairs on the BFW dataset with the FaceNet (Webface) feature model.
Baseline
AGENDA
FairCal (Ours)
False Positive Rate
Alican
African
Arican
Air can
African
Asian
Asiani
Asian
Asian
Caucesian
Indiani
Indian
Indian
Indian
Indian
0.15
0.10
0.05
0.02 0.65 Global FPR
0.08 0.0z 2.00
Global FPR
Global FPR Global FPR
Global FPR
Figure 17: (Lines closer together is better for fairness, best viewed in colour) Illustration of improved fairness reduction in bias, as measured by the FPRs evaluated on intra-cthnicity pairs on the BFW dataset with the ArcFace feature model.
30


Reviews:
[{"review_id": "rUEB1q2ZERP", "reviewer": "Reviewer_xwE8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nPros: \n\n1. The paper addresses an important problem in face verification: unfairness of classifiers with respect to demographic groups (different FPRs and FNRs). It was shown in previous and this work that it is impossible to choose a single threshold that would yield similar false positive and negative rates for different sensitive groups. Therefore, post-processing approaches focus on calibrating the model scores of threshold to get fair face recognition models. \n\n2. In most scenarios practitioners do not have access to or can not use sensitive attributes for calibrating the model and this work proposes an approach for calibrating model scores without access to sensitive attributes. \n\n3. The authors conduct experiments to demonstrate the effectiveness of FairCal and compare it to previous works. They show that FairCal achieves better overall accuracy, false positive rate and fairness calibration compared to other calibration methods, including ones that do have access to sensitive attributes.\n\n\nI have a few questions and comments for the authors: \n\n1. I do not exactly understand why fairness calibration is a desired property for face verification systems. Why should we expect to have the same proportion of positive matches across sensitive groups in the data? \n2. How was the number of clusters K chosen? \n3. In section 4.2 it is said that by converting scores to calibrated probabilities, one can extend FairCal to multi-class setting. Could the authors please elaborate on that? Was it meant in the context of face recognition? \n4. What loss was used for training models on VGGFace2 and CASIA datasets? SOTA face recognition models are usually trained with angular margin losses (ArcFace, CosFace, SphereFace etc.) to ensure better feature separability in the angular space. It looks that the TPR for ArcFace model is significantly higher than it is for other models (90% vs 69% on BFW dataset). \n5.  In the section 6.2 on fairness calibration it is said that prior methods can not be fairly-calibrated, therefore the beta-calibration is applied to their score outputs to compare with FairCal. Is that the case only for the results in Table 3 or was beta-calibration applied to these methods for all experiments (including ones in Tables 2 and 4)?\n6.  I think it would be better to explain the beta-calibration method in the main body of the paper as it is a \"kernel\" of FairCal method.  \n", "summary_of_the_paper": "The paper proposes an approach, called FairCal, for fairly calibrating face verification models. The method does not require access to sensitive attributes for calibrating, instead the paper uses k-means algorithm to find clusters of data and applies beta-calibrating algorithm on each cluster. The authors show that after calibrating with FairCal the model still achieves good accuracy and outputs more fairness-calibrated probabilities. Also, the method reduces the FPR gap across sensitive attributes. ", "summary_of_the_review": "The main novelty of the paper is applying beta-calibration on data clusters found in an unsupervised manner for face verification models. One of the selling points of FairCal is that the method allows to fairly-calibrate the classifier, i.e. the calibrated classifier will produce equal probabilities of positive matches across sensitive groups. I do not exactly understand why that is a desired property (in contrast to having equal FPRs and FNRs across subgroups) for face verification system. It would be very helpful to hear from the authors on that! Because of that (and other questions I have) I give the score of 5 for this paper, but I am willing to increase it after the rebuttal. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "RXAej-drEqT", "reviewer": "Reviewer_aX6s", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main strengths of this paper is that the propsoed method is a unsupervised method, and does not require retraining. However, there are some weaknesses of this paper. For example, the global accuracy improvements compared with the state-of-the-art methods are very limited  as shown in Table 2. Moreover, the presentations and layout of the whole paper is not clear to the reader. ", "summary_of_the_paper": "This paper introduces a Fairness Calibration method for the task of face verification problem. The authors claimed that the proposed method can produce fairly-calibrated probabilities, reduce the gap in the false positive rates, does not require knowledge of the sensitive attribute, and does not require retraining, training an additional model, or retuning. Experimental results demonstrate that the proposed\nmethod can achieve state-of-the-art performance over the RFW and BFW databases.", "summary_of_the_review": "Considering that the proposed bias reduction method is simialr to the idea of Fair Score Normalization, and the performance improvements are very limited, and moreover the writing of this paper is not good. Thus, I suggest to give the decision of rejection to this paper. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8DTXU6PsaW", "reviewer": "Reviewer_JNHP", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength\n+ Simple approach for post-hoc bias mitigation.\n+ Improvement over SOTA Arcface\n+ Extensive evaluation using different fairness metrics.\n\nWeaknesses\n- (Major) No comparison with group-specific thresholding proposed by Robinson et al (CVPRW 2020). \nRobinson et al  (CVPRW 2020) propose a similar method and report results on BFW dataset. Additionally, they also report improvement in face verification accuracy. This makes me question the novelty of this work. The authors are advised to mention the key differences and add comparisons with this work. \n\n- Missing analysis of 'Baseline + Calibration (2nd plot in Fig. 2)' in Table 2,3,4.\n\n- Results (Table 7). \nFrom this table, can we infer that FairCal does not achieve Equal Opportunity (compared to FTC, FSN)?\n\n- Missing details about building calibration set.  How are samples in the calibration set chosen?\n- Missing notational details (For ex: in Section F1 and F2, what is P^{cal} )\n\n\n- (Minor) Missing citations and comparison\n'Group Adaptive Classifier' by Gong et al (CVPR 2021)  and 'Protected Attribute Suppression System'  by Dhar et al (ICCV 2021) recently proposed debiasing methods. While Gong et al (CVPR 2021) present an end-to-end method, Dhar et al (ICCV 2021) present a post-hoc de-biasing approach, that is comparable to FairCal.  \n\n- (Minor) IJB-C is one of the most widely used datasets for evaluating face recognition algorithms, which contains gender and skin tone labels. So, it is recommended that the authors add results related to skin tone  and gender bias on IJBC.\n\n-  (Suggestion, not weakness) The authors are suggested to add plots similar to Fig 1 for Arcface on BFW. Also, it may help to add some more intuition behind Eq 3.\n", "summary_of_the_paper": "The authors propose a simple cluster-conditional post-hoc strategy to calibrate the similarity scores obtained using a pre-trained network. In essence, this work proposes to use a cluster-specific (or group-specific in case of Oracle) calibration function (\\mu), which results in improving the scores pertaining to verification of a x1-x2 pair (where both x1 and x2 belong to the same demographic group) mapping function. This is similar to choosing group-specific threshold demonstrated in the BFW paper (Robinson et al, CVPRW 2020). Unlike existing methods such as AGENDA, FTC etc. FairCal does not require the protected attribute labels during training and testing.", "summary_of_the_review": "The topic of this paper is currently very important in the CV/ML community. Although the novelty of this work is incremental,  the results justify the superiority of FairCal (and oracle). While the readability and thoroughness of this work is quite high, there are still some issues in this work. So, I vote for weak accept. I hope the authors address my concerns (especially about similarities with the BFW paper and add comparison to this paper). ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers agree that the presented approach to fair calibration of face verification models is interesting and needed in the field. The method does not require access to sensitive attributes for calibrating, which makes it sustainable. The reviewers are satisfied with the presented experimental studies in most cases. The rebuttal addressed a large majority of additionally raised questions. I believe that the paper will be of interest to the audience attending ICLR and would recommend a presentation of the work as a poster.
Decision: Accept (Poster)