Original: F AIR C AL : F AIRNESS C ALIBRATION FOR F ACE V ERIFICATION
Revised: F AIR C AL : F AIRNESS C ALIBRATION FOR F ACE V ERIFICATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Despite being widely used, face recognition models suffer from bias: the probability of a false positive (incorrect face match) strongly depends on sensitive attributes such as the ethnicity of the face.
Revised: Despite being widely used, face recognition models suffer from bias: the probability of a false positive (incorrect face match) strongly depends on sensitive attributes such as the ethnicity of the face.

Original: As a result, these models can disproportionately and negatively impact minority groups, particularly when used by law enforcement.
Revised: As a result, these models can disproportionately and negatively impact minority groups, particularly when used by law enforcement.

Original: The majority of bias reduction methods have several drawbacks: they use an end-toend retraining approach, may not be feasible due to privacy issues, and often reduce accuracy.
Revised: The majority of bias reduction methods have several drawbacks: they use an end-toend retraining approach, may not be feasible due to privacy issues, and often reduce accuracy.

Original: An alternative approach is post-processing methods that build fairer decision classiﬁers using the features of pre-trained models, thus avoiding the cost of retraining.
Revised: An alternative approach is post-processing methods that build fairer decision classiﬁers using the features of pre-trained models, thus avoiding the cost of retraining.

Original: However, they still have drawbacks: they reduce accuracy (AGENDA,
Revised: However, they still have drawbacks: they reduce accuracy (AGENDA,

Original:  FTC), or require retuning for different false positive rates (FSN).
Revised: PASS, FTC), or require retuning for different false positive rates (FSN).

Original: In this work, we introduce the Fairness Calibration (FairCal) method, a post-training approach that simultaneously: (i) increases model accuracy (improving the state-of-the-art), (ii) produces fairly-calibrated probabilities, (iii) signiﬁcantly reduces the gap in the false positive rates , (iv) does not require knowledge of the sensitive attribute , and (v) does not require retraining , training an additional model, or retuning.
Revised: In this work, we introduce the Fairness Calibration (FairCal) method, a post-training approach that simultaneously: (i) increases model accuracy (improving the stateof-the-art), (ii) produces fairly-calibrated probabilities, (iii) signiﬁcantly reduces the gap in the false positive rates , (iv) does not require knowledge of the sensitive attribute , and (v) does not require retraining , training an additional model, or retuning.

Original: We apply it to the task of Face Veriﬁcation, and obtain state-of-the-art results with all the above advantages.
Revised: We apply it to the task of Face Veriﬁcation, and obtain state-of-the-art results with all the above advantages.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Face recognition (FR) systems are being increasingly deployed worldwide in a variety of contexts, from policing and border control to providing security for everyday consumer electronics.
Revised: Face recognition (FR) systems are being increasingly deployed worldwide in a variety of contexts, from policing and border control to providing security for everyday consumer electronics.

Original: According to Garvie et al.
Revised: According to Garvie et al.

Original: (2016), face images of around half of all American adults are searchable in police databases.
Revised: (2016), face images of around half of all American adults are searchable in police databases.

Original: FR systems have achieved impressive results in maximizing overall accuracy (Jain et al., 2016).
Revised: FR systems have achieved impressive results in maximizing overall accuracy (Jain et al., 2016).

Original: However, they have also been shown to exhibit signiﬁcant bias against certain demographic subgroups (Buolamwini & Gebru, 2018; Orcutt, 2016; Alvi et al., 2018), deﬁned by a sensitive attribute such as ethnicity, gender, or age.
Revised: However, they have also been shown to exhibit signiﬁcant bias against certain demographic subgroups (Buolamwini & Gebru, 2018; Orcutt, 2016; Alvi et al., 2018), deﬁned by a sensitive attribute such as ethnicity, gender, or age.

Original: Many FR systems have much higher false positive rates (FPRs) for non-white faces than white faces (Grother et al., 2019).
Revised: Many FR systems have much higher false positive rates (FPRs) for non-white faces than white faces (Grother et al., 2019).

Original: Therefore, when FR is employed by law enforcement, non-white individuals may be more likely to be falsely detained (Allyn, 2020).
Revised: Therefore, when FR is employed by law enforcement, non-white individuals may be more likely to be falsely detained (Allyn, 2020).

Original: Thus, it is of utmost importance to devise an easily-implementable solution to mitigate FR bias.
Revised: Thus, it is of utmost importance to devise an easily-implementable solution to mitigate FR bias.

Original: Most efforts to mitigate FR bias have been directed towards learning less biased representations (Liang et al., 2019;
Revised: Most efforts to mitigate FR bias have been directed towards learning less biased representations (Liang et al., 2019;

Original: Wang et al., 2019b; Kortylewski et al., 2019;
Revised: Wang et al., 2019b; Kortylewski et al., 2019;

Original: Yin et al., 2019;
Revised: Yin et al., 2019;

Original: Gong et al., 2020; Wang & Deng, 2020; Huang et al., 2020).
Revised: Gong et al., 2020; Wang & Deng, 2020; Huang et al., 2020; Gong et al., 2021).

Original: These approaches have enjoyed varying degrees of success: though the bias is reduced, it may still endure (Wang et al., 2019a), and this ends up reducing the accuracy of the system (Gong et al., 2020).
Revised: These approaches have enjoyed varying degrees of success: though the bias is reduced, is reduced (Gong et al., 2020).

Original: Moreover, they often require retraining the models from scratch, which is computationally expensive.
Revised: They often require retraining the models from scratch, which is computationally expensive.

Original: They also often require the sensitive attribute of the face (such as ethnicity)
Revised: Moreover, they often require the sensitive attribute of the face (such as ethnicity)

Original: (Gong et al., 2020; Dhar et al., 2020;
Revised: (Gong et al., 2020; Dhar et al., 2020;

Original: Terhörst et al., 2020a), which may not be feasible to obtain.
Revised: Terhörst et al., 2020a; Dhar et al., 2021; Robinson et al., 2020), which may not be feasible to obtain.

Original: Hence, an approach that improves fairness in existing models without reducing accuracy , or knowing the sensitive attribute , or require retraining , is missing.
Revised: Hence, an approach that improves fairness in existing models without reducing accuracy , or knowing the sensitive attribute , or require retraining , is missing.

Original: In this work, we focus on the face veriﬁcation problem, a major subproblem in FR: determine whether two face images depict the same person.
Revised: In this work, we focus on the face veriﬁcation problem, a major subproblem in FR: determine whether two face images depict the same person.

Original: Current state-of-the-art (SOTA) classiﬁers are based on deep neural networks that embed images into a low-dimensional space (Taigman et al., 2014).
Revised: Current state-of-the-art (SOTA) classiﬁers are based on deep neural networks that embed images into a low-dimensional space (Taigman et al., 2014).

Original: The “Baseline” method is one that deems a pair of images is a match if the cosine similarity between their embeddings exceeds a certain threshold.
Revised: The “Baseline” method is one that deems a pair of images is a match if the cosine similarity between their embeddings exceeds a certain threshold.

Original: We illustrate and compare the bias of current models in Figure 1, and compare their characteristics in Table 1.
Revised: We illustrate and compare the bias of current models in Figure 1, and compare their characteristics in Table 1.

Original: The fairness of face veriﬁcation classiﬁers is typically assessed by three quantities, explained below: (i) fairness calibration across subgroups, (ii) equal false positive rates (FPRs) across subgroups i.e. predictive equality , (iii) equal false negative rates (FNRs) across subgroups, i.e. equal opportunity.
Revised: The fairness of face veriﬁcation classiﬁers is typically assessed by three quantities, explained below: (i) fairness calibration across subgroups, (ii) equal false positive rates (FPRs) across subgroups i.e. predictive equality , (iii) equal false negative rates (FNRs) across subgroups, i.e. equal opportunity.

Original: A classiﬁer is said to be globally calibrated if its predicted probability p pred of face matches in a dataset is equal to the fraction p true of true matches in the dataset (Guo et al., 2017).
Revised: A classiﬁer is said to be globally calibrated if its predicted probability p pred of face matches in a dataset is equal to the fraction p true of true matches in the dataset (Guo et al., 2017).

Original: This requires the classiﬁer to additionally output an estimate of the true probability of a match, i.e. conﬁdence.
Revised: This requires the classiﬁer to additionally output an estimate of the true probability of a match, i.e. conﬁdence.

Original: However, we suggest it is even more crucial for models to be fairly calibrated , i.e., for calibration to hold conditionally on each subgroup (see Deﬁnition 2).
Revised: However, we suggest it is even more crucial for models to be fairly calibrated , i.e., for calibration to hold conditionally on each subgroup (see Deﬁnition 2).

Original: Otherwise, if the same predicted probability is known to carry different meanings for different demographic groups, users including law enforcement ofﬁcers and judges may be motivated to take sensitive attributes into account when making critical decisions about arrests or sentencing (Pleiss et al., 2017).
Revised: Otherwise, if the same predicted probability is known to carry different meanings for different demographic groups, users including law enforcement ofﬁcers and judges may be motivated to take sensitive attributes into account when making critical decisions about arrests or sentencing (Pleiss et al., 2017).

Original: Moreover, previous works such as Canetti et al.
Revised: Moreover, previous works such as Canetti et al.

Original: and Pleiss et al.
Revised: and Pleiss et al.

Original: (2017) presume the use of a fairly-calibrated model for their success.
Revised: (2017) presume the use of a fairly-calibrated model for their success.

Original: While global calibration can be achieved with off-the-shelf post-hoc calibration methods (Platt, 1999;
Revised: While global calibration can be achieved with off-the-shelf post-hoc calibration methods (Platt, 1999;

Original: Zadrozny & Elkan, 2001; Niculescu-Mizil & Caruana, 2005; Guo et al., 2017), they do not achieve fairness-calibration .
Revised: Zadrozny & Elkan, 2001; Niculescu-Mizil & Caruana, 2005; Guo et al., 2017), they do not achieve fairness-calibration .

Original: This is illustrated in Figure 2.
Revised: This is illustrated in Figure 2.

Original: However, achieving all three fairness deﬁnitions mentioned above: (i) fairness-calibration , (ii) equal FPRs , (iii) equal FNRs, is impossible in practice as several authors have shown (Hardt et al., 2016; Chouldechova, 2017; Kleinberg et al., 2017;
Revised: However, achieving all three fairness deﬁnitions mentioned above: (i) fairness-calibration , (ii) equal FPRs , (iii) equal FNRs, is impossible in practice as several authors have shown (Hardt et al., 2016; Chouldechova, 2017; Kleinberg et al., 2017;

Original: Pleiss et al., 2017).
Revised: Pleiss et al., 2017).

Original: We elaborate further on this in section 3, the key takeaway is that we may aim to satisfy at most two of the three fairness notions.
Revised: We elaborate further on this in section 3, the key takeaway is that we may aim to satisfy at most two of the three fairness notions.

Original: In the particular context of policing, equal FPRs i.e. predictive equality is considered more important than equal FNRs, as false positive errors (false arrests) risk causing signiﬁcant harm, especially to members of subgroups already at disproportionate risk for police scrutiny or violence.
Revised: In the particular context of policing, equal FPRs i.e. predictive equality is considered more important than equal FNRs, as false positive errors (false arrests) risk causing signiﬁcant harm, especially to members of subgroups already at disproportionate risk for police scrutiny or violence.

Original: As a result, our goals are to achieve fairness-calibration and predictive equality across subgroups, while maintaining accuracy , without retraining .
Revised: As a result, our goals are to achieve fairness-calibration and predictive equality across subgroups, while maintaining accuracy , without retraining .

Original: We further assume that we do not have access to the sensitive attributes , as this may not be feasible in practice due to (a) privacy concerns, (b) challenges in deﬁning the sensitive attribute (e.g. ethnicity cannot be neatly divided into discrete categories), and (c) laborious and expensive to collect.
Revised: We further assume that we do not have access to the sensitive attributes , as this may not be feasible in practice due to (a) privacy concerns, (b) challenges in deﬁning the sensitive attribute (e.g. ethnicity cannot be neatly divided into discrete categories), and (c) laborious and expensive to collect.

Original: In this paper, we succeed in achieving all of the objectives mentioned above.
Revised: In this paper, we succeed in achieving all of the objectives mentioned above.

Original: The main contribution is the introduction of a post-training Fairness Calibration (FairCal) method:
Revised: The main contribution is the introduction of a post-training Fairness Calibration (FairCal) method:

Original: Accuracy : FairCal achieves state-of-the-art accuracy (Table 2) for face veriﬁcation on two large datasets.
Revised: Accuracy : FairCal achieves state-of-the-art accuracy (Table 2) for face veriﬁcation on two large datasets.

Original: Fairness-calibration : Our face veriﬁcation classiﬁer outputs state-of-the-art fairnesscalibrated probabilities without knowledge of the sensitive attribute (Table 3).
Revised: Fairness-calibration : Our face veriﬁcation classiﬁer outputs state-of-the-art fairnesscalibrated probabilities without knowledge of the sensitive attribute (Table 3).

Original: Predictive equality: Our method reduces the gap in FPRs across sensitive attributes (Table 4).
Revised: Predictive equality: Our method reduces the gap in FPRs across sensitive attributes (Table 4).

Original: For example, in Figure 2, at a Global FPR of 5% using the baseline method Black people are 15X more likely to false match than white people.
Revised: For example, in Figure 2, at a Global FPR of 5% using the baseline method Black people are 15X more likely to false match than white people.

Original: Our method reduces this to 1.2X (while SOTA for post-hoc methods is 1.7X).
Revised: Our method reduces this to 1.2X (while SOTA for post-hoc methods is 1.7X).

Original: No sensitive attribute required : Our approach does not require the sensitive attribute, neither at training nor at test time.
Revised: No sensitive attribute required : Our approach does not require the sensitive attribute, neither at training nor at test time.

Original: In fact, it outperforms models that use this knowledge.
Revised: In fact, it outperforms models that use this knowledge.

Original: No retraining : Our method has no need for retraining or tuning the feature representation model, or training an additional model, since it performs statistical calibration post-hoc .
Revised: No retraining : Our method has no need for retraining or tuning the feature representation model, or training an additional model, since it performs statistical calibration post-hoc .

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Fairness, or Bias mitigation: Work on bias mitigation for deep FR models can be divided into two main camps: (i) methods that learn less biased representations, and (ii) post-processing approaches that attempt to remove bias from a pre-trained feature representation model by building fairer decision systems (Srinivas et al., 2019; Dhar et al., 2020; 
Revised: Fairness, or Bias mitigation: Work on bias mitigation for deep FR models can be divided into two main camps: (i) methods that learn less biased representations, and (ii) post-processing approaches that attempt to remove bias from a pre-trained feature representation model by building fairer decision systems (Srinivas et al., 2019; Dhar et al., 2020; 2021;

Original: Terhörst et al., 2020a;b).
Revised: Terhörst et al., 2020a;b; Robinson et al., 2020).

Original: Several approaches have been pursued in (i), including domain adaptation (Wang et al., 2019a), margin-based losses (Khan et al., 2019; Huang et al., 2020), data augmentation (Kortylewski et al., 2019), feature augmentation (Yin et al., 2019; Wang et al., 2019b), reinforcement learning (Wang & Deng, 2020),and adversarial learning (Liu et al., 2018; Gong et al., 2020).
Revised: Several approaches have been pursued in (i), including domain adaptation (Wang et al., 2019a), margin-based losses (Khan et al., 2019; Huang et al., 2020), data augmentation (Kortylewski et al., 2019), feature augmentation (Yin et al., 2019; Wang et al., 2019b), reinforcement learning (Wang & Deng, 2020), adversarial learning (Liu et al., 2018; Gong et al., 2020), and attention mechanisms (Gong et al., 2021).

Original: While these methods mitigate bias, this is often achieved at the expense of recognition accuracy and/or at high computational cost .
Revised: While these methods mitigate bias, this is often achieved at the expense of recognition accuracy and/or at high computational cost .

Original: The work we present in this paper ﬁts into (ii), and thus we focus on reviewing those approaches.
Revised: The work we present in this paper ﬁts into (ii), and thus we focus on reviewing those approaches.

Original: For an in-depth literature review of bias mitigation for FR models, see Drozdowski et al.
Revised: For an in-depth literature review of bias mitigation for FR models, see Drozdowski et al.

Original: We compare all these methods in Table 1.
Revised: We compare all these methods in Table 1.

Original: Srinivas et al.
Revised: Srinivas et al.

Original: (2019) proposed an ensemble approach, exploring different strategies for fusing the scores of multiple models.
Revised: (2019) proposed an ensemble approach, exploring different strategies for fusing the scores of multiple models.

Original: This work does not directly measure bias reduction, and presents only the results of applying the method to a protected subgroup.
Revised: This work does not directly measure bias reduction, and presents only the results of applying the method to a protected subgroup.

Original: Terhörst et al.
Revised: Terhörst et al.

Original: (2020b) show that while this method is effective in increasing overall accuracy , it fails to reliably mitigate bias.
Revised: (2020b) show that while this method is effective in increasing overall accuracy , it fails to reliably mitigate bias.

Original: Dhar et al.
Revised: Dhar et al.

Original: (2020) proposed the Adversarial Gender De-biasing algorithm (AGENDA) to train a shallow network that removes the gender information of the embeddings of a pre-trained network.
Revised: (2020) proposed the Adversarial Gender De-biasing algorithm (AGENDA) to train a shallow network that removes the gender information of the embeddings of a pre-trained network.

Revised: Dhar et al.

Revised: (2021) extended this work with PASS to deal with any sensitive attribute, and proposed a novel discriminator training strategy.

Original: Consequently, this also leads to reduced gender bias in face veriﬁcation, but at the cost of accuracy .
Revised: This leads to reduced gender bias in face veriﬁcation, but at the cost of accuracy .

Original: Also, it requires the sensitive attributes during training.
Revised: Also, AGENDA and PASS require the sensitive attributes during training.

Original: Terhörst et al.
Revised: Terhörst et al.

Original: (2020a) proposed the Fair Template Comparison (FTC) method, which replaces the computation of the cosine similarity score by an additional shallow neural network trained using cross-entropy loss, with a fairness penalization term and an l 2 penalty term to prevent overﬁtting.
Revised: (2020a) proposed the Fair Template Comparison (FTC) method, which replaces the computation of the cosine similarity score by an additional shallow neural network trained using cross-entropy loss, with a fairness penalization term and an l 2 penalty term to prevent overﬁtting.

Original: While this method does indeed reduce a model’s bias, it comes at the expense of an overall decrease in accuracy .
Revised: While this method does indeed reduce a model’s bias, it comes at the expense of an overall decrease in accuracy .

Original: Moreover, it requires training the shallow neural network and tuning the loss weights.
Revised: Moreover, it requires training the shallow neural network and tuning the loss weights.

Original: Most importantly, it requires the sensitive attributes during training.
Revised: Most importantly, it requires the sensitive attributes during training.

Revised: Robinson et al.

Revised: (2020) proposed GST which uses a group-speciﬁc threshold.

Revised: However, the calibration sets in GST are deﬁned by the sensitive attributes themselves.

Original: Finally, Terhörst et al.
Revised: Terhörst et al.

Original: (2020b) proposed the Fair Score Normalization (FSN) method, which  normalizes the scores by requiring the model’s FPRs across unsupervised clusters to be the same at a predeﬁned global FPR.
Revised: (2020b) proposed the Fair Score Normalization (FSN) method, which is essentially GST with unsupervised clusters. FSN normalizes the scores by requiring the model’s FPRs across unsupervised clusters to be the same at a predeﬁned global FPR.

Original: This method is not designed to be fairly-calibrated .
Revised: These methods are not designed to be fairly-calibrated .

Original: Moreover, if a different global FPR is desired, the method needs to be recomputed.
Revised: Moreover, if a different global FPR is desired, the method needs to be recomputed.

Original: In contrast, instead of ﬁxing one FPR threshold, our method converts the cosine similarity scores into calibrated probabilities, which can then be used for any choice of fair FPRs.
Revised: In contrast, instead of ﬁxing one FPR threshold, our method converts the cosine similarity scores into calibrated probabilities, which can then be used for any choice of fair FPRs.

Original: In addition, our approach can be extended to group fairness for K -classiﬁcation problems, which is not possible with  FSN. 
Revised: In addition, our approach can be extended to group fairness for K -classiﬁcation problems, which is not possible with GST or FSN. [Table]

Original: Calibration: Calibration is closely related to uncertainty estimation for deep networks (Guo et al., 2017).
Revised: Calibration: Calibration is closely related to uncertainty estimation for deep networks (Guo et al., 2017).

Original: Several post-hoc calibration methods have been proposed such as Platt’s scaling or temperature scaling (Platt, 1999; Guo et al., 2017), histogram binning (Zadrozny & Elkan, 2001), isotonic 
Revised: Several post-hoc calibration methods have been proposed such as Platt’s scaling or temperature scaling (Platt, 1999; Guo et al., 2017), histogram binning (Zadrozny & Elkan, 2001), isotonic regression (Niculescu-Mizil & Caruana, 2005), spline calibration (Gupta et al., 2021)

Original: [Table]
Revised: [Table]

Original: regression (Niculescu-Mizil & Caruana, 2005), spline calibration (Gupta et al., 2021), and beta calibration (Kull et al., 2017), among others.
Revised: and beta calibration (Kull et al., 2017), among others.

Original: All of these methods involve computing a calibration function.
Revised: All of these methods involve computing a calibration function.

Original: As such, any calibration method can be readily applied, leading to models that are calibrated but not fairly-calibrated .
Revised: As such, any calibration method can be readily applied, leading to models that are calibrated but not fairly-calibrated .

Original: An algorithm to achieve the latter for a binary classiﬁer has been proposed in Hebert-Johnson et al.
Revised: An algorithm to achieve the latter for a binary classiﬁer has been proposed in Hebert-Johnson et al.

Original: , but the work remains theoretical and no practical implementation is known.
Revised: , but the work remains theoretical and no practical implementation is known.

Original: Fairly calibrated models are the missing ingredient in the works of Canetti et al.
Revised: Fairly calibrated models are the missing ingredient in the works of Canetti et al.

Original: and
Revised: and

Original: Pleiss et al.
Revised: Pleiss et al.

Original: (2017), since they work with the presumption that a fairly calibrated model exists.
Revised: (2017), since they work with the presumption that a fairly calibrated model exists.

Original: Closely related to face veriﬁcation is the problem of face identiﬁcation: to determine the identity of a person from a set of known people.

Original: However, previous works (Eickeler et al., 2000; Kral & Lenc, 2015; Eliades et al., 2019; Xie et al., 2020) did not address fairness and bias.

Original:  F ACE V ERIFICATION AND F AIRNESS
Revised:  F ACE V ERIFICATION AND F AIRNESS

Original: In order to discuss bias mitigation strategies and their effectiveness, one must ﬁrst agree on what constitutes a fair algorithm.
Revised: In order to discuss bias mitigation strategies and their effectiveness, one must ﬁrst agree on what constitutes a fair algorithm.

Original: We start by rigorously deﬁning the face veriﬁcation problem as a binary classiﬁcation problem.
Revised: We start by rigorously deﬁning the face veriﬁcation problem as a binary classiﬁcation problem.

Original: Then we present the notion of a probabilistic classiﬁer, which outputs calibrated probabilities that the classiﬁcation is correct.
Revised: Then we present the notion of a probabilistic classiﬁer, which outputs calibrated probabilities that the classiﬁcation is correct.

Original: Finally, since several different deﬁnitions of fairness have been proposed (see Verma & Rubin (2018); Garg et al.
Revised: Finally, since several different deﬁnitions of fairness have been proposed (see Verma & Rubin (2018); Garg et al.

Original: (2020) for a comparative analysis), we review the ones pertinent to our work.
Revised: (2020) for a comparative analysis), we review the ones pertinent to our work.

Original: 3. B INARY CLASSIFIERS AND SCORE OUTPUTS
Revised: 3. B INARY CLASSIFIERS AND SCORE OUTPUTS

Original: Let f denote a trained neural network that encodes an image x into an embedding f ( x ) =
Revised: Let f denote a trained neural network that encodes an image x into an embedding f ( x ) =

Original: z ∈ Z = R d .
Revised: z ∈ Z = R d .

Original: Pairs of images of faces are drawn from a global pair distribution ( x 1 , x 2 ) ∼ P .
Revised: Pairs of images of faces are drawn from a global pair distribution ( x 1 , x 2 ) ∼ P .

Original: Given such a pair, let Y ( x 1 , x 2 )
Revised: Given such a pair, let Y ( x 1 , x 2 )

Original: =
Revised: =

Original: if the identities of the two images are the same and Y = 0 otherwise.
Revised: if the identities of the two images are the same and Y = 0 otherwise.

Original: The face veriﬁcation problem consists of learning a binary classiﬁer for Y .
Revised: The face veriﬁcation problem consists of learning a binary classiﬁer for Y .

Original: The baseline classiﬁer for the face veriﬁcation problem is based on the cosine similarity between the feature embeddings of the two images, s [Equation]
Revised: The baseline classiﬁer for the face veriﬁcation problem is based on the cosine similarity between the feature embeddings of the two images, s ( x 1 , x 2 ) =

Original:  x [Equation] x [Equation]
Revised: f ( x 1 ) T f ( x 2 ) (cid:107) f ( x 1 ) (cid:107)(cid:107) f ( x 2 ) (cid:107) .

Original: The cosine similarity score is used to deﬁne a binary classiﬁer (cid:98) Y :
Revised: The cosine similarity score is used to deﬁne a binary classiﬁer (cid:98) Y :

Original: X ×X → { 0 , 1 } by thresholding with a choice of s thr ∈ [ − 1 , 1] , which is determined by a target FPR.
Revised: X ×X → { 0 , 1 } by thresholding with a choice of s thr ∈ [ − 1 , 1] , which is determined by a target FPR.

Original: [Equation]
Revised: (cid:98) Y ( x 1 , x 2 ) = 1 if s ( x 1 , x 2 ) ≥ s thr ; 0 otherwise

Original: 3. C ALIBRATION
Revised: 3. C ALIBRATION

Original: A calibrated probabilistic model provides a meaningful output: its conﬁdence reﬂects the probability of a genuine match, in the sense of the following deﬁnition.
Revised: A calibrated probabilistic model provides a meaningful output: its conﬁdence reﬂects the probability of a genuine match, in the sense of the following deﬁnition.

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: The probabilistic model (cid:98) C :
Revised: The probabilistic model (cid:98) C :

Original: X × X → [0 , 1] is said to be calibrated if the true probability of a match is equal to the model’s conﬁdence output c , P [Equation] 1 , x 2 ∼P ( Y = 1 | (cid:98) C = c ) = c.
Revised: X × X → [0 , 1] is said to be calibrated if the true probability of a match is equal to the model’s conﬁdence output c : P x 1 , x 2 ∼P ( Y = 1 | (cid:98) C = c ) = c.

Original: Any score-based classiﬁer can be converted to a calibrated probabilistic model using standard post-hoc calibration methods (Zadrozny & Elkan, 2001; Niculescu-Mizil & Caruana, 2005; Platt, 1999; Kull et al., 2017) (see Appendix F for a brief description of ones used in this work).
Revised: Any score-based classiﬁer can be converted to a calibrated probabilistic model using standard post-hoc calibration methods (Zadrozny & Elkan, 2001; Niculescu-Mizil & Caruana, 2005; Platt, 1999; Kull et al., 2017) (see Appendix G for a brief description of ones used in this work).

Original: This probabilistic model can be converted to a binary classiﬁer by thresholding the output probability.
Revised: This probabilistic model can be converted to a binary classiﬁer by thresholding the output probability.

Original: Applied to face veriﬁcation, a calibrated probabilistic model outputs the likelihood/conﬁdence that a pair of images ( x 1 , x 2 ) are a match.
Revised: Applied to face veriﬁcation, a calibrated probabilistic model outputs the likelihood/conﬁdence that a pair of images ( x 1 , x 2 ) are a match.

Original: Calibration is achieved by using a calibration set S cal to learn a calibration map µ from the cosine similarity scores to probabilities.
Revised: Calibration is achieved by using a calibration set S cal to learn a calibration map µ from the cosine similarity scores to probabilities.

Original: Since calibration deﬁnes a monotonically increasing calibration map µ , a binary classiﬁer with a score threshold s thr can be obtained by thresholding the probabilistic classiﬁer at µ ( s thr ) .
Revised: Since calibration deﬁnes a monotonically increasing calibration map µ , a binary classiﬁer with a score threshold s thr can be obtained by thresholding the probabilistic classiﬁer at µ ( s thr ) .

Original: 3. F AIRNESS
Revised: 3. F AIRNESS

Original: In the context of face veriﬁcation, fairness implies that the classiﬁer has similar performance on different population subgroups, i.e. being calibrated conditional on subgroup membership.
Revised: In the context of face veriﬁcation, fairness implies that the classiﬁer has similar performance on different population subgroups, i.e. being calibrated conditional on subgroup membership.

Original: Let g ( x ) ∈ G = { g i } denote the sensitive/protected attribute of x , such as ethnicity, gender, or age.
Revised: Let g ( x ) ∈ G = { g i } denote the sensitive/protected attribute of x , such as ethnicity, gender, or age.

Original: Each sensitive attribute g i ∈ G induces a population subgroup, whose distribution on the intra-subgroup pairs we denote by G i .
Revised: Each sensitive attribute g i ∈ G induces a population subgroup, whose distribution on the intra-subgroup pairs we denote by G i .

Original: Deﬁnition 2.
Revised: Deﬁnition 2.

Original: The probabilistic model (cid:98) C is fairly-calibrated 1 for subgroups g 1 and g 2 if the classiﬁer is calibrated when conditioned on each subgroup.
Revised: The probabilistic model (cid:98) C is fairly-calibrated 1 for subgroups g 1 and g 2 if the classiﬁer is calibrated when conditioned on each subgroup.

Original: [Equation]
Revised: [Equation]

Original: Intuitively, a model is considered biased if its accuracy alters when tested on different subgroups.
Revised: Intuitively, a model is considered biased if its accuracy alters when tested on different subgroups.

Original: In the case of face recognition applications, predictive equality , meaning equal FPRs across subgroups, is often of primary importance.
Revised: However, in the case of face recognition applications, predictive equality (deﬁned below), meaning equal FPRs across subgroups, is often of primary importance.

Original: Deﬁnition 3.
Revised: Deﬁnition 3.

Original: A binary classiﬁer (cid:98) Y exhibits predictive equality for subgroups g 1 and g 2 if the classiﬁer has equal FPRs for each subgroup,
Revised: A binary classiﬁer (cid:98) Y exhibits predictive equality for subgroups g 1 and g 2 if the classiﬁer has equal FPRs for each subgroup,

Original: [Equation]
Revised: [Equation]

Original: In certain applications of FR (such as ofﬁce security, where high FNRs would cause disruption), equal opportunity, meaning equal FNRs across subgroups, is also important.
Revised: In certain applications of FR (such as ofﬁce security, where high FNRs would cause disruption), equal opportunity (deﬁned below), meaning equal FNRs across subgroups, is also important.

Original: Deﬁnition 4.
Revised: Deﬁnition 4.

Original: A binary classiﬁer (cid:98) Y exhibits equal opportunity for subgroups g 1 and g 2 if the classiﬁer has equal FNRs for each subgroup,
Revised: A binary classiﬁer (cid:98) Y exhibits equal opportunity for subgroups g 1 and g 2 if the classiﬁer has equal FNRs for each subgroup,

Original: [Equation]
Revised: [Equation]

Original: Prior works (Hardt et al., 2016; Chouldechova, 2017; Kleinberg et al., 2017; Pleiss et al., 2017) have shown that it is impossible in practice to satisfy all three fairness properties at the same time: (i) fairness calibration (deﬁnition 2), (ii) predictive equality (deﬁnition 3), (iii) equal opportunity (deﬁnition 4).
Revised: Prior works (Hardt et al., 2016; Chouldechova, 2017; Kleinberg et al., 2017; Pleiss et al., 2017) have shown that it is impossible in practice to satisfy all three fairness properties at the same time: (i) fairness calibration (deﬁnition 2), (ii) predictive equality (deﬁnition 3), (iii) equal opportunity (deﬁnition 4).

Original: At most, two of these three desirable properties can be achieved simultaneously in practice.
Revised: At most, two of these three desirable properties can be achieved simultaneously in practice.

Original: In the context of our application, predictive equality is more critical than equal opportunity.
Revised: In the context of our application, predictive equality is more critical than equal opportunity.

Original: Hence we choose to omit equal opportunity as our goal.
Revised: Hence we choose to omit equal opportunity as our goal.

Original:  F AIRNESS C ALIBRATION (F AIR C AL )
Revised:  F AIRNESS C ALIBRATION (F AIR C AL )

Original: In this section we describe our FairCal method, which constitutes the main contribution of this work.
Revised: In this section we describe our FairCal method, which constitutes the main contribution of this work.

Original: Simply calibrating a model based on cosine similarity scores will not improve fairness: if the baseline score-based classiﬁer is biased, the resulting probabilistic classiﬁer remains equally biased.
Revised: Simply calibrating a model based on cosine similarity scores will not improve fairness: if the baseline score-based classiﬁer is biased, the resulting probabilistic classiﬁer remains equally biased.

Original: This is illustrated in Figure 2, where for both Baseline and Baseline Calibration, any choice of global threshold will lead to different FPRs across sensitive subgroups; consequently, these models will fail to achieve predictive equality .
Revised: This is illustrated in Figure 2, where for both Baseline and Baseline Calibration, any choice of global threshold will lead to different FPRs across sensitive subgroups; consequently, these models will fail to achieve predictive equality .

Original: Our proposed solution is to introduce conditional calibration methods, which involve partitioning the pairs into sets, and calibrating each set.
Revised: Our proposed solution is to introduce conditional calibration methods, which involve partitioning the pairs into sets, and calibrating each set.

Original: Given an image pair, we ﬁrst identify its set membership, and then apply the corresponding calibration map.
Revised: Given an image pair, we ﬁrst identify its set membership, and then apply the corresponding calibration map.

Original: In our FairCal method, the sets are formed by clustering the images’ feature vectors in an unsupervised fashion.
Revised: In our FairCal method, the sets are formed by clustering the images’ feature vectors in an unsupervised fashion.

Original: In our Oracle method, the sets are deﬁned by the sensitive attributes themselves.
Revised: In our Oracle method, the sets are deﬁned by the sensitive attributes themselves.

Original: Both methods are designed to achieve fairness-calibration .
Revised: Both methods are designed to achieve fairness-calibration .

Original: 4. F AIRNESS C ALIBRATION (F AIR C AL , OUR METHOD )
Revised: 4. F AIRNESS C ALIBRATION (F AIR C AL , OUR METHOD )

Original: (i) Apply the K -means algorithm to the image features, Z cal , partitioning the embedding space Z into K clusters Z 1 , . .
Revised: (i) Apply the K -means algorithm to the image features, Z cal , partitioning the embedding space Z into K clusters Z 1 , . .

Original: , Z k .
Revised: , Z k .

Original: These form the K calibration sets:
Revised: These form the K calibration sets:

Original: [Equation]
Revised: [Equation]

Original: (ii)
Revised: (ii)

Original: For each calibration set S cal k , use a post-hoc calibration method to compute the calibration map µ k that maps scores s ( x 1 , x 2 ) to cluster-conditional probabilities µ k ( s ( x 1 , x 2 )) .
Revised: For each calibration set S cal k , use a post-hoc calibration method to compute the calibration map µ k that maps scores s ( x 1 , x 2 ) to cluster-conditional probabilities µ k ( s ( x 1 , x 2 )) .

Original: We use beta calibration (Kull et al., 2017), the
Revised: We use beta calibration (Kull et al.

Original:  recommended method when dealing with bounded scores.
Revised: 2017), the recommended method when dealing with bounded scores.

Original: iii)
Revised: (iii)

Original: For an image pair ( x 
Revised: For an image pair ( x 1 , x 2 )

Original: 1 ,x 2 ), ﬁnd the clusters they belong to.
Revised: , ﬁnd the clusters they belong to.

Original: If both images fall into the same cluster k , deﬁne the pair’s calibrated score as the cluster’s calibrated score:
Revised: If both images fall into the same cluster k , deﬁne the pair’s calibrated score as the cluster’s calibrated score:

Original: [Equation]
Revised: [Equation]

Original: Else, if the images are in different clusters, k 1 and k 2 , respectively, deﬁne the pair’s calibrated score as the weighted average of the calibrated scores in each cluster:
Revised: Else, if the images are in different clusters, k 1 and k 2 , respectively, deﬁne the pair’s calibrated score as the weighted average of the calibrated scores in each cluster:

Original: [Equation]
Revised: [Equation]

Original: where the weight θ is the relative population fraction of the two clusters,
Revised: where the weight θ is the relative population fraction of the two clusters,

Original: [Equation]
Revised: [Equation]

Original: Since FairCal uses unsupervised clusters, it does not require knowledge of the sensitive attributes .
Revised: Since FairCal uses unsupervised clusters, it does not require knowledge of the sensitive attributes .

Original: Moreover, FairCal is a post-training statistical method, hence it does not require any retraining .
Revised: Moreover, FairCal is a post-training statistical method, hence it does not require any retraining .

Original: 4. C OMPARISON WITH FSN (T ERHÖRST ET AL ., 2020 B )
Revised: 4. C OMPARISON WITH FSN (T ERHÖRST ET AL ., 2020 B )

Original: Building the unsupervised clusters allows us to not rely on knowing the sensitive attributes .
Revised: Building the unsupervised clusters allows us to not rely on knowing the sensitive attributes .

Original: In contrast, the FSN method normalizes the scores by shifting them such that thresholding at a predeﬁned global FPR leads to that same FPR on each calibration set.
Revised: In contrast, the FSN method normalizes the scores by shifting them such that thresholding at a predeﬁned global FPR leads to that same FPR on each calibration set.

Original: This is limiting in two crucial ways: 1)
Revised: This is limiting in two crucial ways: 1)

Original: FSN only applies to the binary classiﬁcation problem (e.g. face veriﬁcation); 2) the normalizing shift in FSN depends on the global FPR chosen a priori.
Revised: FSN only applies to the binary classiﬁcation problem (e.g. face veriﬁcation); 2) the normalizing shift in FSN depends on the global FPR chosen a priori.

Original: Our FairCal method has neither of these limitations.
Revised: Our FairCal method has neither of these limitations.

Original: By converting the scores to calibrated probabilities, we can extended it to the multi-class setting, and we do not need to choose a global FPR a priori.
Revised: By converting the scores to calibrated probabilities, we can extended it to the multi-class setting, and we do not need to choose a global FPR a priori.

Original: A simple analogy to explain the difference is : consider the problem of fairly assessing two classrooms of students with different distribution of grades.
Revised: A simple analogy to explain the difference is : consider the problem of fairly assessing two classrooms of students with different distribution of grades.

Original: Fair calibration means the same grade should mean the same for both classrooms.
Revised: Fair calibration means the same grade should mean the same for both classrooms.

Original: Equal fail rate implies the percentage of students who fail is the same in both classrooms.
Revised: Equal fail rate implies the percentage of students who fail is the same in both classrooms.

Original: Global (unfair) methods would choose to pass or fail students from both classrooms using the same threshold.
Revised: Global (unfair) methods would choose to pass or fail students from both classrooms using the same threshold.

Original: Two possible fair approaches are: (A) Estimate a different threshold for each classroom based on a ﬁxed fail rate for both.
Revised: Two possible fair approaches are: (A) Estimate a different threshold for each classroom based on a ﬁxed fail rate for both.

Original: (B) Calibrate the grades so the distributions of the two classrooms match, i.e. the same grade means the same in both classrooms, and then choose a fair threshold.
Revised: (B) Calibrate the grades so the distributions of the two classrooms match, i.e. the same grade means the same in both classrooms, and then choose a fair threshold.

Original: Method B automatically ensures equal fail rate for both classes.
Revised: Method B automatically ensures equal fail rate for both classes.

Original: Method A is what FSN does, Method B is what our FairCal method achieves.
Revised: Method A is what FSN does, Method B is what our FairCal method achieves.

Original: 4. O RACLE C ALIBRATION ( SUPERVISED F AIR C AL , ALSO OURS )
Revised: 4. O RACLE C ALIBRATION ( SUPERVISED F AIR C AL , ALSO OURS )

Original: We include a second calibration method, Oracle, which proceeds like the FairCal deﬁned above, but creates the calibration sets based on the sensitive attribute instead in an unsupervised fashion.
Revised: We include a second calibration method, Oracle, which proceeds like the FairCal deﬁned above, but creates the calibration sets based on the sensitive attribute instead in an unsupervised fashion.

Original: If the images belong to different subgroups i.e. g ( x 1 ) (cid:54) =
Revised: If the images belong to different subgroups i.e. g ( x 1 ) (cid:54) =

Original: g ( x 2 ) , then the classiﬁer correctly outputs zero.
Revised: g ( x 2 ) , then the classiﬁer correctly outputs zero.

Original: This method is not feasible in practice, since the sensitive attribute may not be available, or because using the sensitive attribute may be not permitted for reasons of discrimination or privacy.
Revised: This method is not feasible in practice, since the sensitive attribute may not be available, or because using the sensitive attribute may be not permitted for reasons of discrimination or privacy.

Original: However, the Oracle method represents an ideal baseline for fairness-calibration .
Revised: However, the Oracle method represents an ideal baseline for fairness-calibration .

Original:  E XPERIMENTAL D ETAILS
Revised:  E XPERIMENTAL D ETAILS

Original: Models : We used three distinct pretrained models: two Inception Resnet models (Szegedy et al., 2017) obtained from Esler (2021) (MIT License), one trained on the VGGFace2 dataset (Cao et al., 2018) and another on the CASIA-Webface dataset (Yi et al., 2014), and an ArcFace model obtained from Sharma (2021)
Revised: Models : We used three distinct pretrained models: two Inception Resnet models (Szegedy et al., 2017) obtained from Esler (2021) (MIT License), one trained on the VGGFace2 dataset (Cao et al., 2018) and another on the CASIA-Webface dataset (Yi et al., 2014), and an ArcFace model obtained from Sharma (2021)

Original: (Apache 2.0 license) and trained on the reﬁned version of MS-Celeb-1M (Guo et al., 2016).
Revised: (Apache 2.0 license) and trained on the reﬁned version of MS-Celeb-1M (Guo et al., 2016).

Original: We will refer to the models as Facenet (VGGFace2), Facenet (Webface), and ArcFace, respectively.
Revised: We will refer to the models as Facenet (VGGFace2), Facenet (Webface), and ArcFace, respectively.

Original: As is standard for face recognition models, we pre-processed the images by cropping them using a Multi-Task Convolution Neural Network (MTCNN) algorithm (Zhang et al., 2016).
Revised: As is standard for face recognition models, we pre-processed the images by cropping them using a Multi-Task Convolution Neural Network (MTCNN) algorithm (Zhang et al., 2016).

Original: If the algorithm failed to identify a face, the pair was removed from the analysis.
Revised: If the algorithm failed to identify a face, the pair was removed from the analysis.

Original: Datasets : We present experiments on two different datasets: Racial Faces in the Wild (RFW) (Wang et al., 2019a) and Balanced Faces in the Wild (BFW) (Robinson et al., 2020), both of which are available under licenses for non-commercial research purposes only.
Revised: Datasets : We present experiments on two different datasets: Racial Faces in the Wild (RFW) (Wang et al., 2019a) and Balanced Faces in the Wild (BFW) (Robinson et al., 2020), both of which are available under licenses for non-commercial research purposes only.

Original: Both datasets already include predeﬁned pairs separated into ﬁve folds.
Revised: Both datasets already include predeﬁned pairs separated into ﬁve folds.

Original: The results we present are the product of leave-one-out cross-validation.
Revised: The results we present are the product of leave-one-out cross-validation.

Original: The RFW dataset contains a 1:1 ratio of genuine/imposter pairs and 23,541 pairs in total (after applying the MTCNN).
Revised: The RFW dataset contains a 1:1 ratio of genuine/imposter pairs and 23,541 pairs in total (after applying the MTCNN).

Original: The dataset’s images are labeled by ethnicity (African, Asian,
Revised: The dataset’s images are labeled by ethnicity (African, Asian,

Original: Caucasian, or Indian), with all pairs consisting of same-ethnicity images.
Revised: Caucasian, or Indian), with all pairs consisting of same-ethnicity images.

Original: The BFW dataset, which possesses a 1:3 ratio of genuine/imposter pairs, is comprised of 890,347 pairs (after applying the
Revised: The BFW dataset, which possesses a 1:3 ratio of genuine/imposter pairs, is comprised of 890,347 pairs (after applying the

Original: MTCNN).
Revised: MTCNN).

Original: Its images are labeled by ethnicity (African, Asian, Caucasian, or Indian) and gender (Female or Male), and it includes mixed-gender and mixed-ethnicity pairs.
Revised: Its images are labeled by ethnicity (African, Asian, Caucasian, or Indian) and gender (Female or Male), and it includes mixed-gender and mixed-ethnicity pairs.

Original: The RFW and BFW datasets are made up of images taken from MS-Celeb-1M (Guo et al., 2016) and VGGFace (Cao et al., 2018), respectively.
Revised: The RFW and BFW datasets are made up of images taken from MS-Celeb-1M (Guo et al., 2016) and VGGFace (Cao et al., 2018), respectively.

Original: Thus, to expose the models to new images, only the two FaceNet models can be evaluated on the RFW dataset, while only the FaceNet (Webface) and ArcFace models can be evaluated on the BFW dataset.
Revised: Thus, to expose the models to new images, only the two FaceNet models can be evaluated on the RFW dataset, while only the FaceNet (Webface) and ArcFace models can be evaluated on the BFW dataset.

Original: Methods : For both the FSN and FairCal method we used K = 100 clusters for the K -means algorithm, as recommended by Terhörst et al. (2020b).
Revised: Methods : For both the FSN and FairCal method we used K = 100 clusters for the K -means algorithm, as recommended by Terhörst et al. (2020b).

Original: For FairCal, we employed the recently proposed beta calibration method (Kull et al., 2017).
Revised: For FairCal, we employed the recently proposed beta calibration method (Kull et al., 2017).

Original: Our FairCal method is robust to the choice of number of clusters and post-hoc calibration method (see Appendix G for more details).
Revised: Our FairCal method is robust to the choice of number of clusters and post-hoc calibration method (see Appendix H for more details).

Original: We discuss the parameters used in training AGENDA (Dhar et al., 2020) andFTC
Revised: We discuss the parameters used in training AGENDA (Dhar et al., 2020), PASS (Dhar et al., 2021) and

Original:  (Terhörst et al., 2020a) in Appendix B and Appendix C, respectively.
Revised: FTC (Terhörst et al., 2020a) in Appendix B, in Appendix C and Appendix D, respectively.

Original: Notice that the prior relevant methods (Baseline, AGENDA,  FTC,  and FSN) output scores that, even when rescaled to [0,1], do not result in calibrated probabilities and hence, by design, cannot be fairlycalibrated .
Revised: Notice that the prior relevant methods (Baseline, AGENDA, PASS, FTC, GST, and FSN) output scores that, even when rescaled to [0,1], do not result in calibrated probabilities and hence, by design, cannot be fairly-calibrated .

Original: Therefore, in order to fully demonstrate that our method is superior to those approaches, when measuring fairness-calibration we apply beta calibration to their ﬁnal score outputs as well, which is the same post-hoc calibration method used in our FairCal method.
Revised: Therefore, in order to fully demonstrate that our method is superior to those approaches, when measuring fairness-calibration we apply beta calibration to their ﬁnal score outputs as well, which is the same post-hoc calibration method used in our FairCal method.

Original:  R ESULTS
Revised:  R ESULTS

Original: In this section we report the performance of our models with respect to the three metrics: (i) accuracy , (ii) fairness calibration , and (iii) predictive equality .
Revised: In this section we report the performance of our models with respect to the three metrics: (i) accuracy , (ii) fairness calibration , and (iii) predictive equality .

Original: Our results show that among post hoc calibration methods,
Revised: Our results show that among post hoc calibration methods,

Original: FairCal is best at global accuracy , see Table2.
Revised: FairCal is best at global accuracy , see Table2.

Original: FairCal is best on fairness calibration , see Table 3. 3.
Revised: FairCal is best on fairness calibration , see Table 3. 3.

Original: FairCal is best on predictive equality ,
Revised: FairCal is best on predictive equality ,

Original: i
Revised: i

Original: e., equal FPRs, see Table 4. 4.
Revised: e., equal FPRs, see Table 4. 4.

Original: FairCal does not require the sensitive attribute , and outperforms methods that use this knowledge, including a variant of FairCal that uses the sensitive attribute (Oracle).
Revised: FairCal does not require the sensitive attribute , and outperforms methods that use this knowledge, including a variant of FairCal that uses the sensitive attribute (Oracle).

Original: FairCal does not require retraining of the classiﬁer.
Revised: FairCal does not require retraining of the classiﬁer.

Original: We discuss these results in further detail below.
Revised: We discuss these results in further detail below.

Original: We provide additional detailed discussion and results, including on equal opportunity and the standard deviations that result from the 5-fold cross-validation study, in the Appendix.
Revised: We provide additional detailed discussion and results, including on equal opportunity and the standard deviations that result from the 5-fold cross-validation study, in the Appendix.

Original: Overall, our method that satisﬁes both fairness deﬁnitions without decreasing baseline model accuracy .
Revised: Overall, our method that satisﬁes both fairness deﬁnitions without decreasing baseline model accuracy .

Original: In contrast, while the FTC method obtains slightly better predictive equality results than our method in one situation, this is achieved only at the expense of a signiﬁcant decrease in accuracy .
Revised: In contrast, while the FTC method obtains slightly better predictive equality results than our method in one situation, this is achieved only at the expense of a signiﬁcant decrease in accuracy .

Original: It is important to emphasize that accuracy and predictive equality metrics are determined at a speciﬁed global FPR.
Revised: It is important to emphasize that accuracy and predictive equality metrics are determined at a speciﬁed global FPR.

Original: This entails determining for each prior method a threshold that achieves the desired global FPR.
Revised: This entails determining for each prior method a threshold that achieves the desired global FPR.

Original: However, this itself promotes our method’s advantages: (a) previous methods such as FSN rely on a predeﬁned FPR, while ours does not; (b) while other methods need recomputation for different thresholds, our method simultaneously removes the bias at different global FPRs.
Revised: However, this itself promotes our method’s advantages: (a) previous methods such as FSN rely on a predeﬁned FPR, while ours does not; (b) while other methods need recomputation for different thresholds, our method simultaneously removes the bias at multiple global FPRs.

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: is better in all
Revised: Oracle (Ours) 0. FP R Baseline 0.

Original: [Table]
Revised: 0.85 AGENDA 0. 0.84 PASS 0. 0.93 FTC 0.

Original: [Table]
Revised: 0.61 GST 0. 0.56 FSN 0. 0.68 FairCal (Ours) 0. Oracle (Ours) 0.

Original: 6. G LOBAL ACCURACY
Revised: 6. G LOBAL ACCURACY

Original: The receiver operating characteristic (ROC) curve plots the true positive rate (TPR, equal to 1 − FNR) against the FPR, obtained by thresholding the model at different values.
Revised: The receiver operating characteristic (ROC) curve plots the true positive rate (TPR = 1 − FNR) against the FPR, obtained by thresholding the model at different values.

Original: The area under the ROC curve (AUROC) is thus a holistic metric that summarizes the accuracy of the classiﬁers (Davis & Goadrich, 2006).
Revised: The area under the ROC curve (AUROC) is thus a holistic metric that summarizes the accuracy of the classiﬁers (Davis & Goadrich, 2006).

Original: A higher AUROC is better, an uninformative classiﬁer has an AUROC of 50%.
Revised: A higher AUROC is better, an uninformative classiﬁer has an AUROC of 50%.

Original: Our FairCal method achieves SOTA results, as shown in Table 2.
Revised: Our FairCal method achieves SOTA results, as shown in Table 2.

Original: We report the AUROC for the different pre-trained models and datasets, as well as the TPRs at 0.1% and 1% global FPR thresholds.
Revised: We report the AUROC for the different pre-trained models and datasets, as well as the TPRs at 0.1% and 1% global FPR thresholds.

Original: FairCal achieves the best values of AUROC in all cases,and the highest TPR in seven of the eight cases. In addition, our FairCal method surpasses  our Oracle method (which uses subgroup information ) on the RFW dataset.
Revised: FairCal achieves the best values of AUROC in all cases, the highest TPR in seven of the eight cases, and surpasses GST and our Oracle method (which use subgroup information ) on the RFW dataset.

Original: This overall accuracy improvement of our FairCal method can be explained as follows: the feature vectors contain more information than the score alone, so they can be used to identify pairs where the probability of an error(at a given similarity score) is higher or lower, allowing per-cluster calibration to give better accuracy .
Revised: This overall accuracy improvement of our FairCal method can be explained as follows: the feature vectors contain more information than the score alone, so they can be used to identify pairs where the probability of an error is higher or lower, allowing per-cluster calibration to give better accuracy .

Original: In our FairCal method, the subgroups are formed based on the features of the model, and not on manual human-assigned attributes.
Revised: In our FairCal method, the subgroups are formed based on the features of the model, and not on manual human-assigned attributes.

Original: This allows for potentially more complex and effective subgroups.
Revised: This allows for potentially more complex and effective subgroups.

Original: We conﬁrmed this by plotting the images of each cluster. We found the clusters to indeed have semantic meaning, see Appendix A.
Revised: We visualized the cluster images, and found they indeed have semantic meaning, see Appendix A.

Original: 6. F AIRNESS CALIBRATION
Revised: 6. F AIRNESS CALIBRATION

Original: The prior relevant methods (Baseline, AGENDA,  FTC, and FSN) do not output probabilities, hence by design they cannot be fairly-calibrated .
Revised: The prior relevant methods (Baseline, AGENDA, PASS, FTC, and FSN) do not output probabilities, hence by design they cannot be fairly-calibrated .

Original: However, in order to fully demonstrate that our method is superior to those approaches, we apply beta calibration (the same post-hoc calibration method used in our FairCal method) to their score outputs.
Revised: However, in order to fully demonstrate that our method is superior to those approaches, we apply beta calibration (the same post-hoc calibration method used in our FairCal method) to their score outputs.

Original: Calibration error is a measure of the deviation in a model’s output probabilities of matches from the actual results.
Revised: Calibration error is a measure of the deviation in a model’s output probabilities of matches from the actual results.

Original: When considering subgroups, this is measured by two quantities: (i) the calibration error on each subgroup (lower is better, since it means fewer errors), which can be measured using the
Revised: When considering subgroups, this is measured by two quantities: (i) the calibration error on each subgroup (lower is better, since it means fewer errors), which can be measured using the

Original: Brier score (BS) (DeGroot & Fienberg, 1983), Expected Calibration Error (ECE) (Guo et al., 2017), or Kolmogorov-Smirnov (KS) (Gupta et al., 2021) (discussed in Appendix D).
Revised: Brier score (BS) (DeGroot & Fienberg, 1983), Expected Calibration Error (ECE) (Guo et al., 2017), or Kolmogorov-Smirnov (KS) (Gupta et al., 2021) (discussed in Appendix E).

Original: We present results for
Revised: We present results for

Original: KS, which has been established to be the best measure (Gupta et al., 2021).
Revised: KS, which has been established to be the best measure (Gupta et al., 2021).

Original: (ii)
Revised: (ii)

Original: The deviation in these quantities across subgroups (lower is better, since it is more fair), measured as: Average Absolute
Revised: The deviation in these quantities across subgroups (lower is better, since it is more fair), measured as: Average Absolute

Original: Deviation (AAD), Maximum Absolute Deviation (MAD), and Standard Deviation (STD).
Revised: Deviation (AAD), Maximum Absolute Deviation (MAD), and Standard Deviation (STD).

Original: Our FairCal method achieves the best results, as shown in Table 3: the best mean in three of the four cases (smaller errors) and the best deviation in nine of the nine cases (more fair across subgroups).
Revised: Our FairCal method achieves the best results, as shown in Table 3: the best mean in three of the four cases (smaller errors) and the best deviation in nine of the nine cases (more fair across subgroups).

Original: The improvement is most signiﬁcant on the less accurate models.
Revised: The improvement is most signiﬁcant on the less accurate models.

Original: We discuss these results in more detail in the Appendix.
Revised: We discuss these results in more detail in the Appendix.

Original: 6. P REDICTIVE EQUALITY
Revised: 6. P REDICTIVE EQUALITY

Original: As predictive equality is achieved through equal FPRs between different subgroups, we can measure bias by quantifying the deviation of these FPRs.
Revised: As predictive equality is achieved through equal FPRs between different subgroups, we can measure bias by quantifying the deviation of these FPRs.

Original: We report three measures of deviation (AAD, MAD, and STD) at two different global FPRs: 0.1% and at 1.0%.
Revised: We report three measures of deviation (AAD, MAD, and STD) at two different global FPRs: 0.1% and at 1.0%.

Original: Our FairCal method achieves the best results, as shown in Table 4: the best or tied measure of predictive equality in 18 of the 24 cases.
Revised: Our FairCal method achieves the best results, as shown in Table 4: the best or tied measure of predictive equality in 18 of the 24 cases.

Original: In the cases where FairCal is not best, i.e., when the
Revised: In the cases where FairCal is not best, i.e., when the

Original: ArcFace model evaluated on the BFW dataset, FTC provides the best results, but the differences between AGENDA’s,  FTC’s,  FSN’s, and FairCal’s deviation measures are within a fraction of 1%.
Revised: ArcFace model evaluated on the BFW dataset, FTC and GST provide the best results, but the differences between AGENDA’s, PASS’s, FTC’s, GST’s, FSN’s, and FairCal’s deviation measures are within a fraction of 1%.

Original: Moreover, when applied to ArcFace, the FTC method reduces the model’s accuracy : at FPRs of 0.1% and 1%, the TPRs are, respectively, 4% and 2% lower.
Revised: Moreover, when applied to ArcFace, the FTC and GST methods reduce the model’s accuracy .

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We introduce FairCal, a post-training calibration method that (i) achieves SOTA accuracy , (ii) is fairly-calibrated , (iii) achieves predictive equality (equal FPRs) on challenging face veriﬁcation datasets, (iv) without the knowledge of any sensitive attribute .
Revised: We introduce FairCal, a post-training calibration method that (i) achieves SOTA accuracy , (ii) is fairly-calibrated , (iii) achieves predictive equality (equal FPRs) on challenging face veriﬁcation datasets, (iv) without the knowledge of any sensitive attribute .

Original: It can be readily applied to existing
Revised: It can be readily applied to existing

Original: FR systems, as it is a statistical method that (v) does not require any retraining .
Revised: FR systems, as it is a statistical method that (v) does not require any retraining .

Original: It can aid humanusers in using FR systems to make better informed decisions based on interpretable, unbiased results.
Revised: It can aid humanusers in using FR systems to make better informed decisions based on interpretable, unbiased results.

Original: Societal Impact : Bolstered by dramatically improving accuracy , Facial Recognition (FR) systems have exploded in popularity in recent years and have been applied to innumerable settings under various use cases.
Revised: Societal Impact : Bolstered by dramatically improving accuracy , Facial Recognition (FR) systems have exploded in popularity in recent years and have been applied to innumerable settings under various use cases.

Original: However, the severe biases within these systems limit their applicability and open the door to widespread social harm.
Revised: However, the severe biases within these systems limit their applicability and open the door to widespread social harm.

Original: In this work, we address this issue by proposing a novel postprocessing method that signiﬁcantly reduces the false positive rates across demographic subgroups without requiring knowledge of any sensitive attributes.
Revised: In this work, we address this issue by proposing a novel postprocessing method that signiﬁcantly reduces the false positive rates across demographic subgroups without requiring knowledge of any sensitive attributes.

Original: Limitations : Only under very strict conditions (e.g. a perfect classiﬁer) can all three fairness notions ( fairness calibration , predictive equality , equal opportunity) be satisﬁed.
Revised: Limitations : Only under very strict conditions (e.g. a perfect classiﬁer) can all three fairness notions ( fairness calibration , predictive equality , equal opportunity) be satisﬁed.

Original: While we have shown that our FairCal method can achieve two out of the three deﬁnitions—which is the best we can hope for in practice—it requires the use of a calibration dataset.
Revised: While we have shown that our FairCal method can achieve two out of the three deﬁnitions—which is the best we can hope for in practice—it requires the use of a calibration dataset.

Original: Deploying our method on a substantially different dataset would most likely require further calibration.
Revised: Deploying our method on a substantially different dataset would most likely require further calibration.

Original: Calibrating models for never-seen data is currently an open problem.
Revised: Calibrating models for never-seen data is currently an open problem.

Original: E THICS S TATEMENT
Revised: E THICS S TATEMENT

Original: Our paper is primarily focused on addressing the lack of fairness as well as potential unethical uses of current face recognition systems, speciﬁcally in the context of face veriﬁcation.
Revised: Our paper is primarily focused on addressing the lack of fairness as well as potential unethical uses of current face recognition systems, speciﬁcally in the context of face veriﬁcation.

Original: At many places in the paper, we have referenced several prior works that mention and analyze the current and potential risks posed by such systems.
Revised: At many places in the paper, we have referenced several prior works that mention and analyze the current and potential risks posed by such systems.

Original: We identiﬁed the shortcomings in the previous methods, and methodically established a scientiﬁc way of identifying and measuring fairness in the current systems.
Revised: We identiﬁed the shortcomings in the previous methods, and methodically established a scientiﬁc way of identifying and measuring fairness in the current systems.

Original: Our experiments involve large public datasets of faces.
Revised: Our experiments involve large public datasets of faces.

Original: Our work explicitly addresses problems with face recognition systems that could misuse the sensitive attributes in these datasets, such as ethnicity, race, gender, etc.
Revised: Our work explicitly addresses problems with face recognition systems that could misuse the sensitive attributes in these datasets, such as ethnicity, race, gender, etc.

Original: The motivation of our method is in trying to mitigate bias and improve fairness in the systems using these datasets.
Revised: The motivation of our method is in trying to mitigate bias and improve fairness in the systems using these datasets.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: In order to ensure the reproducibility of our work, we described our proposed methods, FairCal and
Revised: In order to ensure the reproducibility of our work, we described our proposed methods, FairCal and

Original: Oracle, in detail in section 4.
Revised: Oracle, in detail in section 4.

Original: The details of our experiments, including the datasets and pre-processing steps, can be found in section 5.
Revised: The details of our experiments, including the datasets and pre-processing steps, can be found in section 5.

Original: Additional details on how we implemented AGENDA and FTC, which require training additional models, are provided in Appendix B and Appendix C, respectively.
Revised: Additional details on how we implemented AGENDA and FTC, which require training additional models, are provided in Appendix B and Appendix D, respectively.

Original: The embeddings from the pretrained models were obtained on a machine with one GeForce GTX 1080 Ti GPU.
Revised: The embeddings from the pretrained models were obtained on a machine with one GeForce GTX 1080 Ti GPU.

Original: All methods were implemented in Python, and the code is provided in the supplemental material.
Revised: All methods were implemented in Python, and the code is provided in the supplemental material.

Original: Supplemental Material A E XAMPLES OF THE UNSUPERVISED CLUSTERS
Revised: Supplemental Material A E XAMPLES OF THE UNSUPERVISED CLUSTERS

Original: In order to not rely on the sensitive attribute like the Oracle method, our FairCal method uses unsupervised clusters computed with the K -means algorithm based on the feature embeddings of the images.
Revised: In order to not rely on the sensitive attribute like the Oracle method, our FairCal method uses unsupervised clusters computed with the K -means algorithm based on the feature embeddings of the images.

Original: We found them to have semantic meaning.
Revised: We found them to have semantic meaning.

Original: Some examples are included in Figure 3.
Revised: Some examples are included in Figure 3.

Original: [Figure]
Revised: [Figure]

Original: B AGENDA METHOD
Revised: B AGENDA METHOD

Original: The Adversarial Gender De-biasing algorithm (AGENDA) learns a shallow network that removes the gender information of the embeddings from a pre-trained network producing new embeddings.
Revised: The Adversarial Gender De-biasing algorithm (AGENDA) learns a shallow network that removes the gender information of the embeddings from a pre-trained network producing new embeddings.

Original: The algorithm entails training a generator model M , a classiﬁer C and a discriminator E .
Revised: The algorithm entails training a generator model M , a classiﬁer C and a discriminator E .

Original: As proposed, the algorithm only removes the gender information using an adversarial loss that encourages the discriminator to produce equal probabilities for both male and female gender.
Revised: As proposed, the algorithm only removes the gender information using an adversarial loss that encourages the discriminator to produce equal probabilities for both male and female gender.

Original: Since the RFW dataset contains the ethnicity attribute (as opposed to gender) and the BFW dataset contains both gender and ethnicity, we modify the loss to encourage the discriminator to produce equal probabilities amongst all possible sensitive attributes.
Revised: Since the RFW dataset contains the ethnicity attribute (as opposed to gender) and the BFW dataset contains both gender and ethnicity, we modify the loss to encourage the discriminator to produce equal probabilities amongst all possible sensitive attributes.

Original: The shallow networks used for M , C and E are the same as the ones speciﬁed in Dhar et al.
Revised: The shallow networks used for M , C and E are the same as the ones speciﬁed in Dhar et al.

Original: (2020), with the exception that E has as many outputs as possible values of sensitive attributes (4 for RFW and 8 for BFW).
Revised: (2020), with the exception that E has as many outputs as possible values of sensitive attributes (4 for RFW and 8 for BFW).

Original: AGENDA requires the use of a validation set to determine if the discriminator should continue to be updated or not.
Revised: AGENDA requires the use of a validation set to determine if the discriminator should continue to be updated or not.

Original: Hence the embeddings used for training into a 80/20 split, with the 20 split used for the validation.
Revised: Hence the embeddings used for training into a 80/20 split, with the 20 split used for the validation.

Original: In Stage 1, the generator M and classiﬁer C are trained for 50 epochs.
Revised: In Stage 1, the generator M and classiﬁer C are trained for 50 epochs.

Original: Then the bulk of training consists of 100 episodes: (i) Stage 2 is repeated every 10 episodes and consists in training the discriminator E for 25 epochs; (ii) in Stage 3 both M and C are trained with the adversarial loss for 5 epochs with λ = 10 ; (iii) in Stage 4 the discriminator is updated for 5 epochs, unless its accuracy on the validations set is higher than 90%.
Revised: Then the bulk of training consists of 100 episodes: (i) Stage 2 is repeated every 10 episodes and consists in training the discriminator E for 25 epochs; (ii) in Stage 3 both M and C are trained with the adversarial loss for 5 epochs with λ = 10 ; (iii) in Stage 4 the discriminator is updated for 5 epochs, unless its accuracy on the validations set is higher than 90%.

Original: All training is done with a batch size of 400 and an
Revised: All training is done with a batch size of 400 and an

Original: ADAM optimizer with a learning rate of 10 − 3 .
Revised: ADAM optimizer with a learning rate of 10 − 3 .

Original: For more details, we refer to the code provided in the supplemental material.
Revised: For more details, we refer to the code provided in the supplemental material.

Original: C F AIR T EMPLATE C OMPARISON (FTC) METHOD
Revised: C PASS METHOD

Revised: Similar to AGENDA, PASS (Dhar et al., 2021) learns a shallow network to removes the sensitive information of the embeddings from a pre-trained network producing new embeddings. The algorithm entails training a generator model M , a classiﬁer C and ensemble of discriminator E . The shallow networks used for M , C and E are the same as the ones speciﬁed in Dhar et al.

Revised: Like AGENDA, it requires the use of a validation set.

Revised: The same partition as in AGENDA was used.

Revised: In Stage 1, the generator M and classiﬁer C are trained for 100 epochs.

Revised: Then the bulk of training consists of 100 episodes: (i) Stage 2 is repeated every 10 episodes and consists in training the discriminator E for 100 epochs; (ii) in Stage 3 both M and C are trained with the adversarial loss for 5 epochs with λ = 10 ; (iii) in Stage 4 we use the One-At-a-time strategy to update one of the discriminators in the ensemble E .

Revised: It is updated for 5 epochs, unless its accuracy on the validations set is higher than 95%.

Revised: All training is done with a batch size of 400 and an ADAM optimizer with a learning rate of 10 − 3 .

Revised: The availability of sensitive attributes determined the choice of number of discriminators used.

Revised: For the RFW dataset, since only the race is know, we use an ensemble with K = 2 discriminators.

Revised: For the BFW, since both race and gender are known, we use an ensemble of K = 2 discriminators for the race sensitive attribute and K = 3 discriminators for the gender sensitive attribute.

Revised: The latter in fact constitutes the MultiPASS variant of PASS.

Revised: We choose to name it here PASS for simplicity.

Revised: For more details, we refer to the code provided in the supplemental material.

Revised: D F AIR T EMPLATE C OMPARISON (FTC) METHOD

Original: The Fair Template Comparison (FTC) method Terhörst et al.
Revised: The Fair Template Comparison (FTC) method Terhörst et al.

Original: (2020a) learns a shallow network with the goal of outputting fairer decisions.
Revised: (2020a) learns a shallow network with the goal of outputting fairer decisions.

Original: We implemented the FTC method as follow.
Revised: We implemented the FTC method as follow.

Original: In order to keep the ratios between the dimensions of layers the same as in the original paper Terhörst et al.
Revised: In order to keep the ratios between the dimensions of layers the same as in the original paper Terhörst et al.

Original: (2020a), we used a 512-dimensional input layer, followed by two 2048-dimensional intermediate layers.
Revised: (2020a), we used a 512-dimensional input layer, followed by two 2048-dimensional intermediate layers.

Original: The ﬁnal layer is a fully connected linear layer with 2-dimensional output with a softmax activation.
Revised: The ﬁnal layer is a fully connected linear layer with 2-dimensional output with a softmax activation.

Original: All intermediate layers are followed by a ReLU activation function and dropout (with p = 0 . 3 ).
Revised: All intermediate layers are followed by a ReLU activation function and dropout (with p = 0 . 3 ).

Original: The network was trained with a batchsize of b = 200 over 50 epochs, using an Adam optimizer with a learning rate of 10 − 3 and weight decay of 10 − 4 .
Revised: The network was trained with a batchsize of b = 200 over 50 epochs, using an Adam optimizer with a learning rate of 10 − 3 and weight decay of 10 − 4 .

Original: Two losses, one based on subgroup fairness and the other on both subgroup and individual fairness, were proposed in Terhörst et al.
Revised: Two losses, one based on subgroup fairness and the other on both subgroup and individual fairness, were proposed in Terhörst et al.

Original: (2020a).
Revised: (2020a).

Original: Based on the paper’s recommendations, we used the individual fairness loss with a trade-off parameter of λ = 0 . 5 .
Revised: Based on the paper’s recommendations, we used the individual fairness loss with a trade-off parameter of λ = 0 . 5 .

Original: D M EASURING C ALIBRATION E RROR
Revised: E M EASURING C ALIBRATION E RROR

Original: There are different metrics available to measure if a probabilistic classiﬁer is calibrated or fairlycalibrated.
Revised: There are different metrics available to measure if a probabilistic classiﬁer is calibrated or fairlycalibrated.

Original: Calibration error is the error between the true and estimated conﬁdences and is typically measured by the Expected Calibration Error (ECE)
Revised: Calibration error is the error between the true and estimated conﬁdences and is typically measured by the Expected Calibration Error (ECE)

Original: Guo et al.
Revised: Guo et al.

Original: Despite being the most popular calibration error metric, the ECE has several weaknesses, chief among which is its dependence on the binning scheme Nixon et al.
Revised: Despite being the most popular calibration error metric, the ECE has several weaknesses, chief among which is its dependence on the binning scheme Nixon et al.

Original: Recently, Gupta et al.
Revised: Recently, Gupta et al.

Original: (2021) introduced a simple, bin-free calibration measure.
Revised: (2021) introduced a simple, bin-free calibration measure.

Original: For calibrated scores P ( Y = 1 | C = c ) =
Revised: For calibrated scores P ( Y = 1 | C = c ) =

Original: c we have, by Bayes’ rule:
Revised: c we have, by Bayes’ rule:

Original: [Equation]
Revised: [Equation]

Original: Inspired by the Kolmogorov-Smirnov (KS ↓ ) statistic test, Gupta et al.
Revised: Inspired by the Kolmogorov-Smirnov (KS ↓ ) statistic test, Gupta et al.

Original: (2021) proposed to measure the calibration error by comparing the cumulative distributions of P ( Y = 1 , C = c ) and cP ( C = c ) , which empirically correspond to computing the sequences
Revised: (2021) proposed to measure the calibration error by comparing the cumulative distributions of P ( Y = 1 , C = c ) and cP ( C = c ) , which empirically correspond to computing the sequences

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: with h 0 = ˜ h 0 = 0 , and N is the total number of samples.
Revised: with h 0 = ˜ h 0 = 0 , and N is the total number of samples.

Original: Then the KS calibration error metric is given by
Revised: Then the KS calibration error metric is given by

Original: [Equation]
Revised: [Equation]

Original: Another measure is the Brier score (BS ↓ )
Revised: Another measure is the Brier score (BS ↓ )

Original: (DeGroot & Fienberg, 1983), which estimates the mean squared error between the correctness of prediction and the conﬁdence score:
Revised: (DeGroot & Fienberg, 1983), which estimates the mean squared error between the correctness of prediction and the conﬁdence score:

Original: [Equation] [Table] [Table]
Revised: [Equation] [Equation]

Original: For all the above metrics (ECE, KS, BS), lower is better.
Revised: For all the above metrics (ECE, KS, BS), lower is better.

Original: E F AIRNESS CALIBRATION AND E QUAL O PPORTUNITY ( EQUAL FNR) E. F AIRNESS CALIBRATION
Revised: F F AIRNESS CALIBRATION AND E QUAL O PPORTUNITY ( EQUAL FNR) F. F AIRNESS CALIBRATION

Original: Since the calibration map produced by beta calibration is monotone, the ordering of the images provided by the scores is the same as the ordering provided by the probabilities; therefore, the accuracy of the methods wheh thresholding remains unchanged.
Revised: Since the calibration map produced by beta calibration is monotone, the ordering of the images provided by the scores is the same as the ordering provided by the probabilities; therefore, the accuracy of the methods wheh thresholding remains unchanged.

Original: The calibration error (CE) measured with an adaptation of the Kolmogorov-Smirnov (KS) test (described in the Appendix) is computed for each subgroup of interest.
Revised: The calibration error (CE) measured with an adaptation of the Kolmogorov-Smirnov (KS) test (described in the Appendix) is computed for each subgroup of interest.

Original: Notice that for the BFW dataset we consider the eight subgroups that result from the intersection of the ethnicity and gender subgroups.
Revised: Notice that for the BFW dataset we consider the eight subgroups that result from the intersection of the ethnicity and gender subgroups.

Original: We ﬁrst observe that all methods are equally globally calibrated (i [Table] e., the calibration error is low) after the post-hoc calibration method is applied, except for the FTC on the RFW dataset (see the Global column in Table 5 and Table 6).
Revised: We ﬁrst observe that all methods are equally globally calibrated (i.e., the calibration error is low) after the post-hoc calibration method is applied, except for the FTC on the RFW dataset (see the Global column in Table 5 and Table 6).

Original: By inspecting Table 5 and Table 6, we notice that, after calibration, the Baseline method results in models that are not fairly-calibrated, though perhaps not in the way one would expect.
Revised: By inspecting Table 5 and Table 6, we notice that, after calibration, the Baseline method results in models that are not fairly-calibrated, though perhaps not in the way one would expect.

Original: Typically, bias is directed against minority groups, but in this case, it is the Caucasian subgroups that have the higher CEs.
Revised: Typically, bias is directed against minority groups, but in this case, it is the Caucasian subgroups that have the higher CEs.

Original: This is a consequence of the models’ above average accuracy on this subgroup, which is underestimated and therefore not captured by the calibration procedure.
Revised: This is a consequence of the models’ above average accuracy on this subgroup, which is underestimated and therefore not captured by the calibration procedure.

Original: It is important to point out that this is not a failure of the calibration procedure, since the global CE (i.e., the CE measured on all pairs) is low, as discussed above.
Revised: It is important to point out that this is not a failure of the calibration procedure, since the global CE (i.e., the CE measured on all pairs) is low, as discussed above.

Original: E. E QUAL O PPORTUNITY
Revised: F. E QUAL O PPORTUNITY

Original: While equal opportunity (equal FNRs between subgroups) is not prioritized for the FR systems when used by law enforcement, it may be prioritized in different contexts such as ofﬁce building security.
Revised: While equal opportunity (equal FNRs between subgroups) is not prioritized for the FR systems when used by law enforcement, it may be prioritized in different contexts such as ofﬁce building security.

Original: Empirically, our method also mitigates the equal opportunity bias at low global FNRs, as can be seen in Table 7. 
Revised: Empirically, our method also mitigates the equal opportunity bias at low global FNRs, as can be seen in Table 7. [Table] [Table]

Revised: subgroup FNRs in terms of three deviation measures: Average Absolute Deviation (AAD), Maximum Absolute Deviation (MAD), and Standard

Revised: Deviation (STD) (lower is better).

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: F S TANDARD P OST - HOC C ALIBRATION M ETHODS
Revised: Baseline 0. 0.93 AGENDA 0. 0.84 PASS 0. 0.90 FTC 0.

Revised: 0.6 GST 0.

Revised: 1.07 FSN 0.

Revised: [Table]

Revised: [Table]

Revised: Ours

Revised: [Table]

Revised: G S TANDARD P OST - HOC C ALIBRATION M ETHODS

Original: For completeness, we provide a brief description of the post-hoc calibration methods used in this work.
Revised: For completeness, we provide a brief description of the post-hoc calibration methods used in this work.

Original: Beta calibration Kull et al.
Revised: Beta calibration Kull et al.

Original: (2017) was used to obtain our main results, but we show below that choosing another method (histogram binning Zadrozny & Elkan (2001), isotonic regression (Zadrozny & Elkan, 2001; Niculescu-Mizil & Caruana, 2005)) does not impact the performance of our FairCal method. 
Revised: (2017) was used to obtain our main results, but we show below that choosing another method (histogram binning Zadrozny & Elkan (2001), isotonic regression (Zadrozny & Elkan, 2001; Niculescu-Mizil & Caruana, 2005)) does not impact the performance of our FairCal method. Throughout this section, we denote by P cal the pairs of images in the calibration set and S cal their respective cosine similarity scores such that [Equation]

Original: F. H ISTOGRAM B INNING
Revised: G. H ISTOGRAM B INNING

Original: In histogram binning Zadrozny & Elkan (2001), we partition S cal into m bins B i , where i = 1 , . . .
Revised: In histogram binning Zadrozny & Elkan (2001), we partition S cal into m bins B i , where i = 1 , . . .

Original: , m .
Revised: , m .

Original: Then, given a pair of images ( x 1 , x 2 ) with score s ( x 1 , x 2 ) ∈
Revised: Then, given a pair of images ( x 1 , x 2 ) with score s ( x 1 , x 2 ) ∈

Original: B i , we deﬁne
Revised: B i , we deﬁne

Original: [Equation]
Revised: [Equation]

Original: In other words, we simply count the number of scores in each bin that correspond to genuine pairs of images, i.e., images that belong to the same person.
Revised: In other words, we simply count the number of scores in each bin that correspond to genuine pairs of images, i.e., images that belong to the same person.

Original: By construction, a conﬁdence score c (Equation 6) satisﬁes the binned version of the standard calibration (Deﬁnition 1).
Revised: By construction, a conﬁdence score c (Equation 6) satisﬁes the binned version of the standard calibration (Deﬁnition 1).

Original: As for the bins, they can be chosen so as to have equal mass or to be equally spaced, or else by maximizing mutual information, as recently proposed in Patel et al.
Revised: As for the bins, they can be chosen so as to have equal mass or to be equally spaced, or else by maximizing mutual information, as recently proposed in Patel et al.

Original: In this work, we created bins with equal mass.
Revised: In this work, we created bins with equal mass.

Original: Despite being an extremely computationally efﬁcient method and providing good calibration, histogram binning is not guaranteed to preserve the monotonicity between scores and conﬁdences, which is typically a desired property.
Revised: Despite being an extremely computationally efﬁcient method and providing good calibration, histogram binning is not guaranteed to preserve the monotonicity between scores and conﬁdences, which is typically a desired property.

Original: Monotonicity ensures that the accuracy of the classiﬁer is the same when thresholding either the scores or the calibrated conﬁdences.
Revised: Monotonicity ensures that the accuracy of the classiﬁer is the same when thresholding either the scores or the calibrated conﬁdences.

Original: F. I SOTONIC R EGRESSION
Revised: G. I SOTONIC R EGRESSION

Original: Isotonic Regression (Zadrozny & Elkan, 2001; Niculescu-Mizil & Caruana, 2005) learns a monotonic function µ : R → R by solving
Revised: Isotonic Regression (Zadrozny & Elkan, 2001; Niculescu-Mizil & Caruana, 2005) learns a monotonic function µ : R → R by solving

Original: [Equation]
Revised: [Equation]

Original: The conﬁdence score is then given by c ( x 1 , x 2 ) =
Revised: The conﬁdence score is then given by c ( x 1 , x 2 ) =

Original: µ ( s ( x 1 , x 2 )) .
Revised: µ ( s ( x 1 , x 2 )) .

Original: F. B ETA C ALIBRATION
Revised: G. B ETA C ALIBRATION

Original: Beta calibration Patel et al.
Revised: Beta calibration Patel et al.

Original: (2021) is a parametric calibration method, which learns a calibration map µ : R → R of the form
Revised: (2021) is a parametric calibration method, which learns a calibration map µ : R → R of the form

Original: [Equation]
Revised: [Equation]

Original: where the parameters θ  1 , θ 2 , θ 3  ∈ R  are chosen by minimizing the log-loss function
Revised: where the parameters θ = ( θ 1 , θ 2 , θ 3 ) ∈ R 3 are chosen by minimizing the log-loss function

Original: [Equation]
Revised: [Equation]

Original: where c = µ ( s ( x 1 , x 2 )) .
Revised: where c = µ ( s ( x 1 , x 2 )) .

Original: By restricting, a and b to be positive, the calibration map is monotone.
Revised: By restricting, a and b to be positive, the calibration map is monotone.

Original: G R OBUSTNESS OF F AIR C AL RESULTS TO PARAMETERS
Revised: H R OBUSTNESS OF F AIR C AL RESULTS TO PARAMETERS

Original: In this section we show that the results presented in the main paper still hold if we vary model hyperparameters, such as the number K of clusters used in FairCal, and the calibration method.
Revised: In this section we show that the results presented in the main paper still hold if we vary model hyperparameters, such as the number K of clusters used in FairCal, and the calibration method.

Original: Choice of post-hoc calibration : The implementation of the FairCal method requires choosing a post-hoc calibration method and the number of clusters K in the K -means algorithm.
Revised: Choice of post-hoc calibration : The implementation of the FairCal method requires choosing a post-hoc calibration method and the number of clusters K in the K -means algorithm.

Original: Our method is robust to the choice of both with respect to fairness-calibration (the metric of interest when it comes to calibration) and its bias as depicted in Figure 4, Figure 5, Figure 6 and Figure 7.
Revised: Our method is robust to the choice of both with respect to fairness-calibration (the metric of interest when it comes to calibration) and its bias as depicted in Figure 4, Figure 5, Figure 6 and Figure 7.

Original: We compare with binning and isotonic regression.
Revised: We compare with binning and isotonic regression.

Original: For the former, we chose 10 and 25 bins for the RFW and BFW datasets, respectively, given the different number of pairs in each dataset.
Revised: For the former, we chose 10 and 25 bins for the RFW and BFW datasets, respectively, given the different number of pairs in each dataset.

Original: Comparison to FSN (Terhörst et al., 2020b) :
Revised: Comparison to FSN (Terhörst et al., 2020b) :

Original: The improved performance of FairCal over FSN is consistent across different choices of K .
Revised: The improved performance of FairCal over FSN is consistent across different choices of K .

Original: Fixing the choice of the post-hoc calibration method as beta calibration as in the results in the paper, we compare the two, together with Baseline and Oracle for additional baselines.
Revised: Fixing the choice of the post-hoc calibration method as beta calibration as in the results in the paper, we compare the two, together with Baseline and Oracle for additional baselines.

Original: Results are displayed in Figure 8, Figure 9, Figure 10, Figure 11, Figure 12,
Revised: Results are displayed in Figure 8, Figure 9, Figure 10, Figure 11, Figure 12,

Original: Figure 13 and Figure
Revised: Figure 13 and Figure

Original: H
Revised: Consider a probabilistic classiﬁer p : X → ∆ k that outputs that outputs class probabilities for k classes 1 , . . k . Here X denotes the input space and ∆ k

Original: R ESULTS
Revised: = { ( q 1 , . . . , q k ) : (cid:80) ki = 1 q k = 1 } . Suppose p is computed with a deep neural network where f denotes its feature encoder and g the ﬁnal fully connected layer. Then we write p = softmax( g ( f ( x ))) .

Revised: Post-hoc calibration methods (like Dirichlet calibration Kull et al.

Revised: (2019), the generalization of Beta calibration) compute a calibration map µ : ∆ k → ∆ k using a calibration set X cal ⊆

Revised: X such that µ ◦ p is calibrated.

Revised: FairCal can be extended to multi-class classiﬁcation as follows:

Revised: (i) Compute Z

Revised: cal

Revised: =

Revised: { f ( x ) : x ∈ X cal } .

Revised: (ii) Apply the K -means algorithm to the image features, Z cal , partitioning the embedding space Z into K clusters Z 1 , . .

Revised: , Z k .

Revised: These form the K calibration sets:

Revised: [Equation]

Revised: (iii)

Revised: For each calibration set X cal k , use a post-hoc calibration method to compute the calibration map µ k .

Revised: (iv) For a new input

Revised: x ﬁnd the clusters it belongs to and output µ k ( p ( x )) .

Revised: Notice that in the multi-class classiﬁcation setting, we are no longer dealing with pairs and therefore each input (now a single image) belongs to one and only one cluster, thus simplifying the ﬁnal setup.

Revised: We leave for future work its implementation and study through a thorough empirical evaluation.

Original:  PRESENTED WITH S TANDARD D EVIATIONS
Revised: J R ESULTS PRESENTED WITH S TANDARD D EVIATIONS

Original: Recall that the results presented in the main text were computed by taking the mean of a 5-fold leave-one-out cross-validation.
Revised: Recall that the results presented in the main text were computed by taking the mean of a 5-fold leave-one-out cross-validation.

Original: Below, we report the corresponding standard deviations of the ﬁve folds.
Revised: Below, we report the corresponding standard deviations of the ﬁve folds.

Original: The standard deviations for the results on accuracy reported in Table 2 can be found in Table 8,
Revised: The standard deviations for the results on accuracy reported in Table 2 can be found in Table 8,

Original: Table 9, Table 10.
Revised: Table 9, Table 10.

Original: For fairness-calibration in Table 3, they can be found in Table 11, Table 12,Table 13,
Revised: For fairness-calibration in Table 3, they can be found in Table 11, Table 12,Table 13,

Original: Table 14.
Revised: Table 14.

Original: Finally, for predictive equality and equal opportunity in Table 4 and Table 7, they can be found in Table 15, Table 16, Table 17 and Table 18, Table 19, Table 20. 
Revised: Finally, for predictive equality and equal opportunity in Table 4 and Table 7, they can be found in Table 15, Table 16, Table 17 and Table 18, Table 19, Table 20. K A DDITIONAL F IGURES

Revised: In Figure 1, we provided a qualitative analysis of the improved fairness/reduction in bias in terms of predictive equality on the RFW dataset with the FaceNet (Webface) model.

Revised: We replicate this ﬁgure for the FaceNet (VGGFace2) model on the RFW dataset in ??

Revised: and on the BFW dataset for the

Revised: FaceNet (Webface) model in Figure 16 and ArcFace model in Figure 17.

Original: [Figure]
Revised: [Table] [Table] [Table] [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: 08 FairCal (Ours) 0.14 ± 0.

Revised: 0.16 ± 0.

Revised: 0.20 ± 0.

Revised: 0.31 ± 0.

Revised: Oracle (Ours) 0.19 ± 0.

Revised: 0.20 ± 0.

Revised: 0.25 ± 0.

Revised: 0.27 ± 0.

Revised: FP R

Revised: Baseline 1.02 ± 0.

Revised: 1.23 ± 0.

Revised: 7.48 ± 1.

Revised: 1.51 ± 0.

Revised: AGENDA 1.14 ± 0.

Revised: 1.08 ± 0.

Revised: 3.09 ± 1.

Revised: 1.78 ± 0.

Revised: PASS 1.52 ± 0.

Revised: 0.99 ± 0.

Revised: 10.1 ± 2.

Revised: 2.00 ± 0.

Revised: FTC 0.91 ± 0.

Revised: 1.05 ± 0.

Revised: 5.74 ± 1.

Revised: 1.04 ± 0.

Revised: GST 0.92 ± 0.

Revised: 0.57 ± 0.

Revised: 3.01 ± 1.

Revised: 1.13 ± 0.

Revised: FSN 0.68 ± 0.

Revised: 0.61 ± 0.

Revised: 2.19 ± 0.

Revised: 1.27 ± 0.35 FairCal (Ours) 0.46 ± 0.

Revised: 0.57 ± 0.

Revised: 1.79 ± 0.

Revised: 1.46 ± 0.

Revised: Oracle (Ours) 0.69 ± 0.

Revised: 0.74 ± 0.

Revised: 1.71 ± 0.

Revised: 2.08 ± 0.

Revised: [Table]

Revised: [Table]

Revised: FairCal (Ours) 0.10 ± 0.

Revised: 0.11 ± 0.

Revised: 0.10 ± 0.

Revised: 0.09 ± 0.

Revised: Oracle (Ours) 0.11 ± 0.

Revised: 0.12 ± 0.

Revised: 0.09 ± 0.

Revised: 0.11 ± 0.

Revised: FP R

Revised: Baseline 0.60 ± 0.

Revised: 0.45 ± 0.

Revised: 0.39 ± 0.

Revised: 0.75 ± 0.

Revised: AGENDA 0.99 ± 0.

Revised: 0.67 ± 0.

Revised: 0.90 ± 0.

Revised: 0.72 ± 0.

Revised: PASS 0.77 ± 0.

Revised: 0.83 ± 0.

Revised: 0.72 ± 0.

Revised: 0.71 ± 0.

Revised: FTC 0.48 ± 0.

Revised: 0.32 ± 0.

Revised: 0.30 ± 0.

Revised: 0.49 ± 0.

Revised: GST 0.39 ± 0.

Revised: 0.54 ± 0.

Revised: 0.49 ± 0.

Revised: 0.83 ± 0.

Revised: FSN 0.28 ± 0.

Revised: 0.40 ± 0.

Revised: 0.41 ± 0.

Revised: 0.77 ± 0.

Revised: FairCal (Ours) 0.30 ± 0.

Revised: 0.39 ± 0.

Revised: 0.32 ± 0.

Revised: 0.65 ± 0.

Revised: Oracle (Ours) 0.38 ± 0.

Revised: 0.56 ± 0.

Revised: 0.37 ± 0.

Revised: 0.50 ± 0.

Revised: [Table]

Revised: [Table]

Revised: FairCal (Ours) 0.10 ± 0.

Revised: 0.12 ± 0.

Revised: 0.13 ± 0.

Revised: 0.10 ± 0.

Revised: Oracle (Ours) 0.12 ± 0.

Revised: 0.13 ± 0.

Revised: 0.11 ± 0.

Revised: 0.14 ± 0.

Revised: FP R

Revised: Baseline 0.67 ± 0.

Revised: 0.53 ± 0.

Revised: 0.47 ± 0.

Revised: 0.93 ± 0.

Revised: AGENDA 1.16 ± 0.

Revised: 0.81 ± 0.

Revised: 1.15 ± 0.

Revised: 0.84 ± 0.

Revised: PASS 0.81 ± 0.

Revised: 0.99 ± 0.

Revised: 0.89 ± 0.

Revised: 0.90 ± 0.

Revised: FTC 0.56 ± 0.

Revised: 0.38 ± 0.

Revised: 0.34 ± 0.

Revised: 0.60 ± 0.

Revised: GST 0.44 ± 0.

Revised: 0.62 ± 0.

Revised: 0.57 ± 0.

Revised: 1.07 ± 0.

Revised: FSN 0.32 ± 0.

Revised: 0.48 ± 0.

Revised: 0.49 ± 0.

Revised: 0.96 ± 0.

Revised: FairCal (Ours) 0.34 ± 0.

Revised: 0.48 ± 0.

Revised: 0.40 ± 0.

Revised: 0.80 ± 0.

Original: [Table]
Revised: Oracle (Ours) 0.42 ± 0. 0.67 ± 0. 0.44 ± 0. 0.60 ± 0.

Original: [Table] [Table]
Revised: [Figure] [Figure]

Original: [Table] [Table] [Table]
Revised: [Figure] [Figure]

Original: [Table] [Table] [Table]
Revised: [Figure] [Figure]

Original: [Table] [Table] [Table] [Table]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure][Figure] [Figure] [Figure]
Revised: [Figure]

Original: [Figure][Figure] [Figure] [Figure]
Revised: [Figure]