Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition
Authors: Samuel Dooley
Rhea Sanjay Sukthanker
John P Dickerson
Colin White
Frank Hutter
Micah Goldblum
Venue: NeurIPS 2023 oral (2023)
Keywords: Bias Mitigation
Fairness
Facial Recognition
TL;DR: We find that bias is inherent to neural network architectures and hyperparameters, yet we can mitigate it by searching for fair ones

Abstract: Face recognition systems are widely deployed in safety-critical applications, including law enforcement, yet they exhibit bias across a range of socio-demographic dimensions, such as gender and race.  Conventional wisdom dictates that model biases arise from biased training data.  As a consequence, previous works on bias mitigation largely focused on pre-processing the training data, adding penalties to prevent bias from effecting the model during training, or post-processing predictions to debias them, yet these approaches have shown limited success on hard problems such as face recognition.  In our work, we discover that biases are actually inherent to neural network architectures themselves.  Following this reframing, we conduct the first neural architecture search for fairness, jointly with a search for hyperparameters. Our search outputs a suite of models which Pareto-dominate all other high-performance architectures and existing bias mitigation methods in terms of accuracy and fairness, often by large margins, on the two most widely used datasets for face identification, CelebA and VGGFace2. Furthermore, these models generalize to other datasets and sensitive attributes. We release our code, models and raw data files at https://github.com/dooleys/FR-NAS.

Rethinking Bias Mitigation: Fairer Architectures Make for Fairer Face Recognition
Samuel Dooley* University of Maryland, Abacus.AI samuel Cabauusaaauu.aaa.aa ai
Rhea Sanjay Sukthanker' University of Freiburg sukthank@cs. umi-freiburg... de
John P. Dickerson
Colin White Caltech, Abacus.AI
University Maryland, Arthur Al j johnduumd edu
crwhit@@caltech.cm edu
Frank Hutter
Micah Goldblum New York University goldblum@nyu. edu
University of Freiburg fhêcs. uni mmi-freiburg.de
Abstract
Face recognition systems widely deployed in safety-critical applications, in- cluding law enforcement, yet they exhibit bias across a range of socio-demographic dimensions, such as gender and race. Conventional wisdom dictates that model biases arise from biased training data. As a consequence, previous works on bias mitigation largely focused on pre-procee pre-processing the training data, adding penalties to prevent bias from effecting the model during training, dictions debias them, yet these approaches have shown limited success on hard problems such as face recognition. In our work, we discover that biases are actually inherent neural network architectures themselves. Following this reframing, we conduct the first neural architecture search for faimess, jointly with search for hyperparameters. Our search outputs a suite models which Pareto-dominate all other nigh-performance architectures and existing bias mitigation methods in terms of accuracy and fairness, often by large margins, on the two most widely used datasets for face identification, CelebA and VGGFace2. Furthermore, these models generalize to other datasets and sensitive attributes. We release our code, thub. com hwt.//oolllly.oooo woom//ooley///RMMMM. //gt :/tts::/get
post-processing pre-
models and raw data files at https:
Introduction
Machine learning is applied to a wide variety of socially-consequenntall domains, e-g., credit scoring, fraud detection, hiring decisions, criminal recidivism, loan repayment, and face recognition [78, 81, 61, 3J. with many of these applications significantly impacting people's lives, often in discriminatory ways [5, 55, 114]. Dozens of formal definitions of fairness have been proposed [80], and many algorithmic techniques have been developed for debiasing according to these definitions [106]. Existing debiasing algorithms broadly fit into three (or arguably four [96]) categories: pre- pre-processing [e.g., 32, 93, 89, 10]. processsn [e.g., 123, 124, 25, 35, 83, 110, 73, 79, 24, 59], or post-processsing [e.g., 44, 114].
indicates equal contribution
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Architectures
Hyperparams
Design search space using hest model. Run NAS+HPO
Examples:
Examples:
Inception
MagFace (head)
Vilransformer
ArcFace (head)
VowNet
Case (head)
fiat
MobileNetV3 Dual Path Net
Optimizer
Discover new architecture
Learning Rate
Error
Ertor
355 models trained
Pareto frontier
Novel Search Space
Pareto dominant
Figure I: Overview of our methodology.
Conventional wisdom is that in order to effectively mitigate bias, we should start by selecting a model architecture and set of hyperparameters which are optimal in terms of accuracy and then apply mitigation strategy to reduce bias. This strategy has yielded little success in hard problems such as face recognition [14]. Moreover, even randomly initialized face recognition models exhibit bias and in the same ways and extents as trained models, indicating that these biases are baked in to the architectures already [13]. While existing methods for debiasing machine learning systems use fixed neural architecture and hyperparameter setting, we instead, ask fundamental question which has received little attention: Does model bias arise from the architecture and hyperparameters? Following an affirmative answer this question, exploit advances in neural architecture search (NAS) [30] and hyperparameter optimization (HPO) [33] to search for inherently fair models.
We demonstrate our results on face identification systems where pre-, post-, and in nn-processing techniques have fallen short of debiasing face recognition systems. Training fair models in this setting demands addressing several technical challenges [14]. Face identification is a type of face recognition deployed worldwide by government agencies for tasks including surveillance, employment, and housing decisions. Face recognition systems exhibit disparity in accuracy based on race and gender [37, 92, 91, 61]. For example, some face recognition models are to 100 times more likely to give false positives for Black Asian people, compared to white people [2]. This bias has already led to multiple false arrests and jail time for innocent Black men in the USA [48].
In this work, we begin by conducting the first large-scale analysis of the impact of architectures and hyperparameters on bias. We train a diverse set of 29 architectures. ranging from ResNets [47] to vision transformers [28, 68) to Gluon Inception V3 [103] to MobileNetV3 [50] on the two most widely used datasets in face identification that have socio-demographic labels: CelebA [69] and VGGFace2 [8]. In doing so, we discover that architectures and hyperparameters have significant impact on fairness, across fairness definitions.
Motivated by this discovery, we design architectures that are simultancously fair and accurate. To this end, we initiate the study of NAS for fairness by conducting the first use of NAS+HPO to jointly optimize fairness and accuracy. We construct a search space informed by the highest-performin architecture from our large-scale analysis, and we adapt the existing Sequential Model-based Algo- rithm Configuration method (SMAC) [66] for multi-objective architecture and hyperparameter search. We discover a Pareto frontier of face recognition models that outperform existing of-the-----rrr models on both test accuracy and multiple faireess metrics, often by large margins. An outline of our methodology can be found in Figure
We summarize our primary contributions below:
By conducting exhaustive evaluation of architectures and hyperparameters, uncover their strong influence on fairness. Bias is inherent to a model's 's inductive bias, leading to a substantial difference in fairness across different architectures. We conclude that the implicit convention of choosing standard architectures designed for high accuracy is a losing strategy for faimesss.
Inspired by these findings, we propose a new way to mitigate biases. We build an architecture and hyperparameter search space, and we apply existing tools from NAS and HPO to automatically design a fair face recognition system.
Our approach finds architectures which are Pareto-optimal on a variety of fairness metrics on both CelebA and VGGFace2. Moreover, our approach is Pareto-optimal compared to other previous bias mitigation techniques, finding the fairest model.

---

The architectures we synthesize via NAS and HPO generalize to other datasets and sensitive attributes. Notably, these architectures also reduce the linear separability of protected attributes, indicating their effectiveness in mitigating bias across different contexts.
We release our code and raw results at https: / ////ttttiie github. wwt../o///oooooooooooooo so that users can easily adapt our approach to any bias metric or dataset.
2 Background and Related Work
Face Identification. Face recognition tasks can be broadly categorized into two distinct categories: verification and identification. Our specific focus lies in face identification tasks which ask whether a given person in source image appears within a gallery composed of many target identities and their associated images; this is one-to-many comparison. Novel techniques in face recognition tasks, such as ArcFace [108], CosFace [23], and MagFace [75], use deep networks (often called the buckbone) to extract feature representations of faces and then compare those to match individuals (with mechanisms called the head). Generally, backbones take the form of image feature extractors and heads resemble MLPs with specialized loss functions. Often, the term "head" refers to both the last layer of the network and the loss function. Our analysis primarily centers around the face identification task, and we focus our evaluation examining how close images of similar identities are in the feature space of trained models, since the technology relies on this feature representation to differentiate individuals. An overview of these topics can be found in Wang and Deng [109].
Bias Mitigation in Face Recognition. The existence of differential performance of face recognition on population groups and subgroups has been explored in variety of settings. Earlier work [e.g.. 57, 82] focuses on singleddemographic effects (specifically, race and gender) in pre-deep-learning face detection and recognition. Buolamwini and Gebru [5] uncover unequal performance at the phenotypic subgroup level in, specifically, a gender classification task powered by commercial systems. Raji and Buolamwini [90] provide follow-up analysis exploring the impact of the public disclosures of Buolamwini and Gebru [5] - where they discovered that named companies (IBM, Microsoft, and Megvii) updated their APIs within year to address some concerns that had surfaced. Further research continues to show that commercial face recognition systems still have socio vocio-demographic disparities in many complex and permicious ways [29, 27, 54, 54, 26].
Facial recognition is a large and complex space with many different individual technologies, some with bias mitigation strategies designed just for them [63, 118). The main bias mitigation strategies for facial identification are described in Section 4.2.
Neural Archttetuure Search (NAS) and Hyperparameter Optimization (HPO). Deep learning derives its success from the manually designed feature extractors which automate the feature engi- neering process. Neural Architecture Search (NAS) [30, 116]. on the other hand, aims at automating the very design of network architectures for a task at hand. NAS can be seen as subset of HPO [33], which refers the automated search for optimal hyperparameters, such learning rate, batch size, dropout, loss function, optimizer, and architectural choices. Rapid and extensive research on NAS for image classification and object detection has been witnessed as of late [67, 125, 121, 88, 6]. Deploying NAS techniques in face recognition systems has also seen growing interest [129, 1 113]. For example, reinforcement learning- bbaee NAS strategies [121] and one-shot NAS methods [113] have been deployed to search for an efficient architecture for face recognition with low error. How- ever, majority of these methods, the training hyperparameters for the architectures are fixed. We observe that this practice should be reconsidered in order to obtain the fairest, possible face recognition systems. Moreover, one-shot NAS methods have also been applied for multi objett-oeecctvve optimization [39, 71. e.g., optimizing accuracy and parameter size. However, none of these methods can be applied for a joint architecture and hyperparameter search, and none of them have been used to optimize fairness.
For the case of tabular datasets, a few works have applied hyperparameter optimization to mitigate bias in models. Perrone et al. [87] introduced a Bayesian optimization framework to optimize accuracy of models while satisfying a bias constraint. Schmucker et al. [97] and Cruz et al. [17] extended Hyperband [64] to the multi-objective setting and showed its applications to fairness. Lin et al. [65] proposed di-binging face recognition models through model pruning. However, they only considered two architectures and just one set of fixed hyperparameters. To the best of our knowledge.
3

---

no prior work uses any AutoML technique (NAS, HPO, or joint NAS and HPO) to design fair face recognition models, and no prior work uses NAS design fair models for any application.
3 Are Architectures and Hyperparameters Important for Fairness?
In this section, we study the question "Are architectures and hyperparameters important for faimess?" and report an extensive exploration of the effect of model architectures and hyperparameters.
Experimental Setup. We train and evaluate each model configuration
gender-balancee subset of the two most popular face identification datasets: CelebA and VGGFace2. CelebA [69] is a large-scale face attributes dataset with more than 200K celebrity images and total of 10 177 gender- labeled identities. VGGFace2 [8] is a much larger dataset designed specifically for face identification and comprises over 3.1 million images and total 131 gender-labeled identities. While this work analyzes phenotypic metadata (perceived gender), the reader should not interpret our findings absent social lens of what these demographic groups mean inside society. We guide the reader to Hamidi et al. [40] and Keyes [56] for a look at these concepts for gender.
To study the importance of architectures and hyperparameters for fairness, use the following train- ing pipeline - ultimately conducting 355 training runs with different combinations of 29 architectures from the Pytorch Image Model (timm) database [117] and hyperparameters. For each model, use the default learning rate and optimizer that was published with that model. We then train the model with these hyperparameters for each of three heads, ArcFace [108], CosFace [23], and MagFace [75]. Next, we use the model's default learning rate with both AdamW [70] and SGD optimizers (again with each head choice). Finally, we also train with AdamW and SGD with unified learning rates (SGD with learning_rate=0.1 and Adam W with evaluate single architecture between 9 and 13 times (9 times if the default optimizer and learning rates are the same as the standardized, and 13 times otherwise). All other hyperparameters are held constant fortraining of the model.
In total, we thus
Evaluation procedure. As is commonplace in face identification tasks [12, 13], we evaluate the performance of the learned representations. Recall that face recognition models usually learn representations with an image backbone and then learn mapping from those representations identities of individuals with the head of the model. We pass each test image through trained model and save the learned representation. To compute the representation error (which we will henceforth simply refer to as Error), we merely ask, for given probe image/identity, whether the closest image in feature space not of the same person based on l2 distance. We split each dataset into train, validation, and test sets. We conduct our search for novel architectures using the train and validation splits, and then show the improvement of our model on the test set.
The most widely used fairness metric in face identification is rank disparity, which is explored in the NIST FRVT [38]. To compute the rank of a given image/identity, we ask how many images of different identity are closer to the image in feature space. We define this index as the rank of a given image under consideration. Thus, Rank(image) > 0 if and only if Error(image) = 1. We examine the rank disparity: the absolute difference of the average ranks for each perceived gender in a dataset D:
if and only if Error(image) = O: (image)
Rank (x)
Σ DE
Rank (x)
(Dmatel
|Diematel
DE
We focus on rank disparity throughout the main body of this paper is the most widely used in face identification, but we explore other forms of fairness metrics in face recognition in Appendix C.4.
Results and Discussion. By plotting the performance of each training run on the validation set with the error on the r-axis and rank disparity on the y-axis in Figure 2, we can easily conclude two main points. First, optimizing for enror does not always optimize for fairness, and second, different architectures have different fairness properties. We also find the DPN architecture has the lowest error and Pareto-optimal on both datasets; hence, we use that architecture to design our search space in Section 4.

---

CelebA Validation Set
VGGFace2 Validation Set
Rank Disparity
2. on
Rank Disparity
04-
0.2
0.05
Error
Error
RoXNer
Iteption
DPN
DVN e Bac
Figure 2: (Left) CelebA (Right) VGGFace2. Error-Rank Disparity Pareto front of the architectures with lowest error (< 0.3). Models in the lower left corner are better. The Pareto front is denoted with a dashed line. Other points are architecture and hyperparameter combinations which are not Pareto-optimal.
We note that in general there a low correlation between error and rank disparity (e-g., for models with error < 0.3, p .113 for CelebA and p = .291 for VGGFace2). However, there are differences between the two datasets the most extreme low errors. First, for VGGFace2, the baseline models already have very low error, with there being 10 models with error < 0.05; CelebA only has three such models. Additionally, models with low error also have low rank disparity on VGGFace2 but this is not the case for CelebA. This can be seen by looking at the Pareto curves in Figure 2.
The Pareto-optimal models also differ across datasets: on CelebA, they are versions of DPN, TNT, ReXNet, VovNet, and ResNets, whereas on VGGFace2 they are DPN and ReXNet. Finally, we note that different architectures exhibit different optimal hyperparameters. For example, on CelebA, for the Xception65 architecture finds the combinations of (SGD, ArcFace) and (Adam ArcFace) as Pareto-optines whereas the Inception -eettion-ResNet architecture finds the combinations (SGD, MagFace) and (SGD, CosFace) Pareto-optimal.
4 Neural Architecture Search for Bias Mitigation
Inspired by our findings on the importance of architecture and hyperparameters for faimess in Sec- tion 3, we now initiate the first joint study of NAS for fairness in face recognition, also simultaneously optimizing hyperparameters. We start by describing our search space and search strategy. We then compare the results of our NAS+HPO-based bias mitigation strategy against other popular face recognition bias mitigation strategies. We conclude that our strategy indeed discovers simultancously accurate and fair architectures.
4.1 Search Space Design and Search Strategy
We design our search space based on our analysis in Section 3, specifically around the Dual Path Networks[10] architecture which has the lowest error and is Pareto-optimal on both datasets, yielding the best trade-off between rank disparity and accuracy seen in Figure 2.
Hyperparameter Search Space Design. We optimize two categorical hyperparameters (the archi- tecture head/loss and the optimizer) and one continuous one (the learning rate). The learning rate"s 's range is conditional on the choice optimizer; the exact ranges are listed in Table 6 in the appendix.
Architecture Search Space Design. Dual Path Networks [10] for image classification share common features (like ResNets [46]) while possessing the flexibility explore new features [52] through a dual path architecture. We replace the repeating with simple recurring searchable block. Furthermore, we stack multiple such searched blocks to closely follow the architecture of Dual Path Networks. We have nine possible choices for each of the three operations the DPN block, each of which we give number 0 through 8. The choices include vanilla convolution, a convolution with normalization and convolution with nosssoormaiizaion, each them paired with kernel sizes Ix 3x3, or 5x5 (see Appendix C.2 for full details). We thus have 729 possible architectures (in addition to an infinite number of hyperparameter configurations). We denote each of these architectures by XYZ where X, Y, Z E (O,..., 8); e.g., architecture 180 represents the architecture which has operation 1, followed by operation 8, followed by operation 0.
--t-3333_11111x---- conv-ixi_ooo conv block
5

---

Validation Set
Test
Rank Disparity
Rank Disparity
0.05
0.1
0.15
0.2
0.25
0.35
0.05
0.25
0.3
0.35
0.4
Error
Mobileeier
TNT
ReXNer
EeebbVNe
Deneelll
DPN Megface
DIN_ Cosface
SMAC
Figure 3: Pareto front of the models discovered by SMAC and the rank-1 models from timm for the (a) validation and (b) test sets on CelebA. Each point corresponds to the mean and standard error of an architecture after training for 3 seeds. The SMAC models Pareto-dominate the top performing timm models (Error < 0.1).
Search strategy. To navigate this search space have the following desiderata:
Joint NAS+HPO. Since there are interaction effects between architectures and hyperparam- eters, require an approach that can jointly optimize both of these.
Multi-objective optimization. We want to explore the trade-off between the accuracy of the face recognition system and the fairness objective of choice, so our joint NAS+HPO algorithm needs to supports multi-objective optimization [84, 21, 71].
Efficiency. A single function evaluation for our problem corresponds to training deep neural network on given dataset. As this can be quite expensive on large datasets, we would like to use cheaper approximations with multi-fidelity optimization techniques [98, 64, 31].
To satisfy these desiderata, we employ the multi fultii-iieeity Bayesian optimization method SMAC3 [66] (using the SMAC4MF facade), casting architectural choices as additional hyperparameters We choose Hyperband [64] for cheaper approximations with the initial and maximum fidelities set to 25 and 100 epochs, respectively, and 2. Every archieccure-hyperprmmmn trained using the same training pipeline in Section 3. For multi-ohjective optimization, we use the ParEGO [21] algorithm with P set to 0.05.
configuration evaluation
4.2 Empirical Evaluation
We now report the results of our NAS+HPO-base bias mitigation strategy. First, we discuss the models found with our approach, and then we compare their performance to other mitigation baselines.
Setup. We conducted one NAS+HPO search for each dataset by searching on the train and validation sets. After running these searches, we identified three new candidate architectures for CelebA (SMAC_o) CMMAC_00,, SMAC 010, and SMAC 680), and one candidate for VGGFace2 (SMAC330) where the naming convention follows that described in Section 4.1. We then retrained each of these models and those high performing models from Section 3 for three seeds to study the robustness of error and disparity for these models; we evaluated their performance on the validation and test sets for each dataset, where we follow the evaluation scheme of Section 3.
Comparison against timm models. CelebA (Figure 3), our models Pareto-dominate all of the timm models with nontrivial accuracy on the validation set. On the test set, our models still Pareto-dominate highly competitive models (with Error<0.1) I), but one of the original configurations (DPN with Magface) also becomes Pareto-optimal. However, the error of this architecture is 0.13, which significantly higher than our models (0.03-0.04). Also, some models (e.g.. Vo and DenseNet) show very large standard errors across seeds. Hence, it becomes important to also study

---

Validation Set
Test Set
0.8
0.7
0.6
Rank Disparity
0.5
Rank Dispurity
0.4
0.3
0.3
0.2
0.1
0
0.01
0.014
0.015
0.14
0.02
0.06
0.08
0.12
Error
Ervor
Dandomere
ReXNee
CadFare
Other
EseVoVNert
DPN MugFace
TNT
HRRee
SMAC
Figure 4: Pareto front of the models discovered by SMAC and the rank-1 models from timm for the (a) validation and (b) test sets on VGGFace2. Each point corresponds to the mean and standard error of an architecture after training for seeds. The SMAC models are Pareto-optimal the top performing timm models (Error<0.1).
the robustness of models across seeds along with the accuracy and disparity Pareto front. Finally, on VGGFace2 (Figure 4), our models are also Pareto-optimal for both the validation and test sets.
Novel Architectures Outperform the State of the Art. Comparing the results of our automatically- found models to the current state of the art baseline AreFace [23] terms of error demonstrates that our strategy clearly establishes a new state of the art. While ArcFace [23] achieves an error of 4.35% with our training pipeline on CelebA, our best- pess-performig novel architecture achieves a much lower еттоr of 3. 10%. Similarly, the current VGGFace2 state of the art baseline [1 12] achieves an error of 4.5%, whereas our best performing novel architecture achieves a much lower error of 3.66%.
Novel Architectures Pareto-Domineee other Bias Mitigation Strategies. There three common pre-, post-, and in- in-processing bias mitigation strategies in face identification. First, Chang et al. [9] demonstrated that randomly flipping labels in the training data of the subgroup with superior accuracy can yield fairer systems; we call this technique Flipped. Next, Wang and Deng [110] use different angular margins during training and therefore promote better feature discrimination for the minority class; we call this technique Angular. Finally, Morales et al. [76] introduced Sensitiveliet tivelets which sensitive information removal network trained on top of a pre-trained feature extractor with an adversarial sensitive regularizer. While other bias mitigation techniques exist in face recognition, these three are the most used and pertinent to face identification. See Cherepanova et al. [14] for an overview of the technical challenges of bias mitigation in face recognition. We take the top performing, Pareto-optimal timm models from the previous section and apply the three bias mitigation techniques (Flipped, Angular, and SensitiveNetar We also apply these same techniques to the novel architectures that we found. The results Table show that the novel architectures from our NAS+ mitigation strategy Pareto-dominate the bias-mitigated models. In VGGFace2, the SMAC_30 model achieves the best performance, both in terms of error and fairness, compared to the bias-mitigated models. On CelebA, the same is true for the SMAC 680 model.
VAS+HPO-Based Bias Mitigation can be Combined with other Bias Mitigation Strategies. Additionally, we combined the three other bias mitigation methods with the SMAC models that resulted from our NAS+HPO-basee bias mitigation strategy. More precisely, we first conducted our NAS+HPO approach and then applied the Flipped, Angular, and SensitiveNets approach afterwards. On both datasets, the resulting models continue to Parmia-dominate the other bias mitigation strategies used by themselves and ultimately yield the model with the lowest rank disparity of all the models (0.18 on VGGFace2 and 0.03 on CelebA). In particular, the bias improvement of CAAC_000+Fiipp
model is notable, achieving a score of 0.03 whereas the lowest rank disparity of any model from Figure 3 is 2.63, 98.9% improvement. In Appendix C.6, we demonstrate that this result robust to the fairness metric - specifically our bias mitigation strategy Pareto-dominates the other approaches all five fairness metrics.
7

---

Table 1: Comparison of bias mitigation techniques where the SMAC models were found with NAS+HPO bias mitigation technique and the other three techniques are standard in facial recognition: Flipped [9], Angular [76], and SensitiveNets [110]. Items in bold are Pareto Pareto-optimal. The values show (Error;Rank Disparity). Other metrics are reported in Appendix C.6 and Table 8.
Trained on VGGFace2
Trained on CelehA Flipped
Model
Baseline
Flipped
Angular
SensitiveNets
Model
Baseline
Angular
SensitiveNets
SMAC (3.66;0.23)
(4.95;0.18)
(4.14:0.25)
(6.20:0.41)
(3:25:2. 18)
(5,20:0.03)
(344.2.28)
(3.45:2.15
DPN
(3.56:0.27)
(5.87:0.32)
(6.06;0336
(4.76:0.44
SMAC_DI0
(4.14:2.27)
(12.27: 5.
(4.50:2.50)
(3.99:2.12)
RcXNet
(4.09:0.2))
(5.73.045)
(5.47.0.2))
(4.75:0.25)
SMAC_680
(3.22;1.96)
(12.42:4550)
(3.80/4.16)
(3.29:2.09)
Swin
(5.47:0.38)
(5.75:0.44)
(5.23;0.25)
(5,03:0.30)
ArcFace
(11.30:4.6)
(13.56:2.70)
(9.90:5.60)
(9:10:3.00)
Table 2: We transfer the evaluation of top performing models on VGGFace2 and CelebA onto six
other common face recognition datasets: LFW [53], CFP_ [100], CFP FP [100], AgeDB [77],
CALFW [128], CPLPW [127]. The novel architectures found with our bias mitigation strategy
significantly outperform other models in terms of accuracy. Refer Table 9 for the complete results.
Architecture (trained on VGGFace2)
LFW
CFP_FF
CFP_FP
AgeDB
CALFW
CPLFW
Rexnet 200
82.60
80.91
65.51
59.18
68.23
62.15
DPN_ RNN_
93.0
91.81
78.96
71.87
78.27
72.97
DPN_ AdamW
78.66
77.17
64.35
61.32
64.78
60.30
SMAC_301
96.63
95.10
86.63
79.97
86.07
81.43
Architecture (trained CclebA)
LFW
CFP FF
CFP FP
AgeDB
CALFW
CPLFW
DPN CosFace
87.78
90.73
69.97
65.55
75.50
62.77
DPN MagFace
91.13
92.16
70.58
68.17
76.98
60.80
SMAC_ 000
94.98
95.60
74.24
80.23
84.73
64.22
SMAC 010
94.30
94.63
73.83
80.37
84.73
65.48
SMAC_680
94.16
95.68
72.67
79.88
84.78
63.96
Novel Architectures Generalize to Other Datasets. We observed that when transferring novel architectures to other facial recognition datasets that focus on fairness-related aspects, our architectures consistently outperform other existing architectures by a significant margin. We take the state-of-the-arr models from our experiments and test the weights from training on CelebA and VGGFace2 different datasets which the models did not during training. Specifically, transfer the evaluation of the trained model weights from CelebA and VGGFace2 onto the following datasets: LFW [53], CFP_ [100], CFP [100], AgeDB [77], CALFW [128], CPLFW [127]. Table 2 demonstrates that our approach consistently achieves the highest performance among various architectures when transferred other datasets. This finding indicates that our approach exhibits exceptional generalizability compared to state-of-the of-the-art face recognition models in terms of transfer learning to diverse datasets.
Novel Architectures Generalize to Other Sensitive Attributes. The superiority of our novel architectures even goes beyond rccuracy-relatee metrics when transferring other datasets novel architectures have superior fairness propertics compared to the existing architectures even on datasets which have completely different protected attributes than were used in the architecture search. Specifically, to inspect the generalizability of our approach to other protected attributes, we transferred our models pre-trained on CelebA and VGGFace2 (which have gender presentation category) ) to the RFW dataset [111] which includes a protected attribute for race and the AgeDB dataset [77] which includes protected attribute for age. The results detailed in Appendix C.7 show that our novel architectures always outperforms the existing architectures, across all five fairness metrics studied in this work both datasets.
Novel Architectures Have Less Senaaa-Separability Protected Attributes. Our comprehensive evaluation of multiple face recognition benchmarks establishes the importance of architectures for fairness in face-recognition. However, it is natural to wonder: "Whal makes the discovered architectures fair in the first place? To answer this question, linear probing to dissect the

---

Table 3: Linear Probes on architectures. Lower gender classification accuracy is better
Architecture (trained on VGGFace2)
Accuracy on Layer N +
Accuracy on Layer N-1 +
DPN_ Ma__MMaaaae_SSS S
86.042%
95,461% %
DPN_ CosFace_SGiD
90.719%
93.787%
DPN CosFace A W
87.385%
94.444%
SMAC 301
69.980%
68.240%
intermediate features of our searched architectures and DPNs, which our search space is based upon. Intuitively, given that our networks are trained only on the task of face recognititon, we do not want the intermediate feature representations to implicitly exploit knowledge about protected attributes (e.g., gender). To this end we insert linear probes[ 1] at the last two layers of different Pareto-optimal DPNs and the model obtained by our NAS+HPO-bass MLP on the feature representations extracted from the pre-trained models and the protected attributes as labels and compute the gender-classificaten accuracy on a held-out set. We consider only the last two layers, so assumes the values of N and N 1 with N being the number of layers in DPNs (and the searched models). We represent the classification probabilities for the genders by IPk = softmar(W, + b), where Wa is the weight matrix of the k-th layer and b is bias. We provide the classification accuracies for the different pre-trained models on VGGFace2 in Table 3. This demonstrates that, as desired, our searched architectures maintain a lower classification accuracy for the protected attribute. In line with this observation, in the t-SNE plots in Figure 18 in the appendix, the DPN displays higher degree of separability of features.
bias mitigation. Specifically, we train an
Comparison between different NAS+HPO techniques We also perform an ablation across different multi-objective NAS+HPO techniques. Specifically we compare the architecture derived by SMAC with architectures derived by the evolutionary multi-objective optimization algorithm NSGA-II [22] and oujti-obeective asynchronous successive halving (MO-ASHA) [98]. We obesrve that the architecture derived by SMAC -orrit-dominttss the other NAS methods in terms of accuracy and diverse fairness metrics Table 4. We use the implementation of NSGA-II and MO-ASHA from the syne-tune library [95] to perform an ablation across different baselines.
Table 4: Comparison between architectures derived by SMAC and other NAS baselines
NAS Method
Accuracy
Rank Disparity..
Disparity.
Ratio
Rank Ratio J.
Error Ratio.,
MO-ASHA_I0S
95.212
0.408
0.038
0.041
0.470
0.572
NSGA-IL_72
86.811
0.599
0.086
0.104
0.490
0.491
SMAC 301
96.337
0.230
0.030
0.032
0.367
0.582
Conclusion, Future Work and Limitations
Conclusion. Our approach studies novel direction for bias mitigation by altering network topology instead of loss functions or model parameters. We conduct the first large-scale analysis of the relationship among hyperparameters and architectural properties, and accuracy, bias, and disparity in predictions across large-scale datasets like CelebA and VGGFace2. Our bias mitigation technique centering around Neural Architecture Search and Hyperparameter Optimization is very competitive compared to other common bias mitigation techniques facial recognition.
Our findings present paradigm shift by challenging conventional practices and suggesting that seeking a fairer architecture through search is more advantageous than attempting to rectify an unfair one through adjustments. The architectures obtained by our joint NAS and HPO generalize across different face recognition benchmarks, different protected attributes, and exhibit lower linear- separability of protected attributes.
Future Work. Since our work lays the foundation for studying NAS+HPO for fairness, opens up plethora of opportunities for future work. We expect the future work in this direction to focus

---

studying different multi multi-objective algorithms [34, 60] and NAS techniques [67. 125, 115] to search for inherently fairer models. Further, would be interesting to study how the properties of the architectures discovered translate across different demographics and populations. Another potential avenue for future work is incorporating priors and beliefs about fairness in the society from experts to further improve and aid NAS+HPO methods for fairness. Given the societal importance, it would be interesting to study how our findings translate to real-life face recognition systems under deployment. Finally, it would also be interesting to study the degree to which NAS+HPO can serve general bias mitigation strategy beyond the case of facial recognition.
Limitations. While our work a step forward in both studying the relationship among architectures, hyperparameters, and bias, and in using NAS techniques to mitigate bias in face recognition models, there are important limitations to keep in mind. Since we only studied few datasets, our results may not generalize to other datasets and fairesss metrics. Second, since face recognition applications span government surveillance [49], target identification from drones [72], and identification in personal photo repositories [36], our findings need to be studied thoroughly across different demographics important before they could be deployed in real-life face recognition systems. Furthermore,
to consider how the mathematical notions of fairness used in research translate to those actually impacted [94], which is a broad concept without a concise defimition. Before deploying a particular system that is meant to improve fairesss in real-life application, we should always critically ask ourselves whether doing so would indeed prove beneficial to those impacted by the given sociotechnical system under consideration or whether it falls into one of the traps described by Selbst et al. [99]. Additionally, work in bias mitigation, writ-large and including our work, can be certainly encourage techno-solutionissm which views the reduction of statistical bias from algorithms justification for their deployment, use, and proliferation. This of course can have benefits, but being able to reduce the bias in a technical system is a different question from whether technical solution should be used on a given problem. We caution that our work should not be interpreted through a normative lens on the appropriateness of using facial recognition technology.
In contrast to some other works, we do, however, feel, that our work helps to overcome the portability trap [99] since it empowers domain experts to optimize for the right faimess metric, in connection with public policy experts, for the problem at hand rather than only narrowly optimizing one specific metric. Additionally, the bias mitigation strategy which we propose here can be used in other domains and applied to applications which have more widespread and socially acceptable algorithmic applications [19].
Acknowledgments
This research was partially supported by the following sources: NSF CAREER Award IIS- 1846237,, NSF D-ISN Award #2039862, NSF Award CCF-1852352, NIH ROI Award NLM-013039-0 NIST GARD #HRO01 10000011200000 12020007. DoD WHS Award #HQ003420F0035, ARPA-E Award #4334192; TAILOR, a project funded by EU Horizon 2020 research and innovation programme under GA No 952215; the German Federal Ministry of Education and Research (BMBF, grant RenormalizedFlows 011S 111719077C): the Deutsche Forschungsgemeinschatt (DFG, German Research Foundation) under grant number 417962828; the European Research Council (ERC) Consolidator Grant "Deep Learning 2.0" (grant no. 101045765). Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the ERC. Neither the European Union nor the ERC can be held responsible for them.
MSE Award #20126334, DARPA
Funded by the European Union
References
[1] Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv: 1610.0164, 2016.
[2] Bobby Allyn. "The computer got it wrong': How facial recognition led false arrest of black man. NPR, June, 24, 2020.
10

---

[3] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Faimess in machine learning, NIPS Tutorial, 2017.
[4] Irwan Bello, William Fedus, Xianzhi Du, Ekin Dogus Cubuk, Aravind Srinivas, Tsung Yi Lin, Jonathon Shlens, and Barret Zoph. Revisiting resnets: Improved training and scaling strategies. Advances in Neural Information Processing Systems, 34:22614 2222611--2667, 2021.
[5] Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the 1st Conference on Faimess, Account- ability and Transparency, volume 81, pages 77-91, 2018. URL. httphttt://prrcceedngg mllr press/ press/v81/ //////ulll///a///// buolamiini html.
[6] Han Cai, Ligeng Zhu, and Song Han. Proxylessnas: Direct neural architecture search on target task and hardware. arXiv preprint arXiv: /2/.v0////200032 2018.
[7] Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. Once-for-all: Train one network and specialize it for efficient deployment. arXiv preprint arXiv: 008809991, 2019.
[8] Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and Andrew Zisserman. Vggface2: A dataset for recognising faces across pose and age. In 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018), pages 67-74. IEEE, 2018
[9] Hongyan Chang, Ta Duy Nguyen, Sasi Kumar Murakonda, Ehsan Kazemi, and Reza Shokri. On adversarial bias and the robustness of fair machine learning. arXiv preprint arXiv:2006.0888 08669, 2020.
[10] Yunpeng Chen, Jianan Li, Huaxin Xiao, Xiaojie Jin, Shuicheng Yan, and Jiashi Feng. Dual path networks. Advances in neural information processing systems, 30, 2017.
[11] Zhengsu Chen, Lingxi Xie, Jianwei Niu, Xuefeng Liu, Longhui Wei, and Qi Tian. Visformer: The vision-friendly transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 589-598, 2021.
[12] Valeriaa Cherepanova, Micah Goldblum, Harrison Foley, Shiyuan Duan, John P. Dickerson, Gavin Taylor, and Tom Goldstein. Lowkey: leveraging adversarial attacks to protect social media users from facial recognition. In International Conference on Learning Representations (ICLR), 2021.
[13] Valeriia Cherepanova, Steven Reich, Samuel Dooley, Hossein Souri, Micah Goldblum, and Tom Goldstein. deep dive into dataset imbalance and bias in face identification. arXiv preprint arXiv:2203.00 08235, 2022.
[14] Valeriia Cherepanova, Vedant Nanda, Micah Goldblum, John P Dickerson, and Tom Goldstein. Technical challenges for training fair neural networks. 6th AAAI/ACM Conference on AI, Ethics, and Society, 2023.
[15] François Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1251- 1258, 2017.
[16] Xiangxiang Chu, Zhi Tian, Yuqing Wang, Bo Zhang. Haibing Ren, Xiaolin Wei, Huaxia Xia, and Chunhua Shen. Twins: Revisiting the design of spatial attention in vision transformers. Advances in Neural Information Processing Systems, 34:9355-9366, 2021.
[17] André F Cruz, Pedro Saleiro, Catarina Belém, Carlos Soares, and Pedro Bizarro. A bandit-based algorithm for faimess-awaae hyperparameter optimization. arXiv preprint urXiv:2010.0366) 2020.
[18] Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Bichen Wu, Zijian He, Zhen Wei, Kan Chen, Yuandong Tian, Matthew Yu, Peter Vajda, et al. Fbnetv3: Joint architecture search using predictor pretraining. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 16276-16285, 2021.

---

[19] Richeek Das and Samuel Dooley, Fairer and more accurate tabular models through nas. arXiv preprint arXiv:2310.12145. 2023.
[20] Stéphane d' Ascoli, Hugo Touvron, Matthew Leavitt, Ari Morcos, Giulio Biroli, and Levent Sagun. Convit: Improving vision transformers with soft convolutional inductive biases. arXiv preprint arXiv:2103. 10697, 2021.
[21] Joan Davins- Valldaraa, Saïd Moussaoui, Guillermo Pita-Gil, and Franck Plestan. Parego extensions for multi-objective optimization of expensive evaluation functions. Journal of Global Optimization. 67(11:79-99) 2017.
[22] Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE transactions on evolutionary computation, 6 (2):182-197, 2002.
[23] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference Computer Vision and Paltern Recognition, pages 4690-4699, 2019.
[24] Emily Diana, Wesley Gill, Michael Kearns, Krishnaram Kenthapadi, and Aaron Roth. Conver- gent algorithms for (relaxed) minimax fairness. arXiv preprint 2rXiv:2011.03/11 2020.
[25] Michele Donini, Luca Oneto, Shai Ben-David, John S Shawe-Taylor, and Massimiliano Pontil. Empirical risk minimization under fairness constraints. In Advances in Neural Information Processing Systems, pages 2791-2800 2018
[26] Samuel Dooley, Ryan Downing, George Wei, Nathan Shankar, Bradon Thymes, Gu- drun Thorkelsdottir, Tiye Kurtz-Miott, Rachel Mattson, Olufemi Obiwumi, Valeriia Cherepanova, et al. Comparing human and machine bias in face recognition. arXiv preprint arXiv:2110.08336, 2021.
[27] Samuel Dooley, George Z Wei, Tom Goldstein, and John Dickerson. Robustness disparities in face detection. Advances in Neural Information Processing Systems, 35:38245-38259, 2022.
[28] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 2020.
[29] Pawel Drozdowski, Christian Rathgeb, Antitza Dantcheva, Naser Damer, and Christoph Busch. Demographic bias in biometrics: A survey emerging challenge. IEEE Transactions Technology and Society, 1(2):89-103. 10399-003222000 2020.
[30] Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey. The Journal of Machine Learning Research, 20(1):1997-2017 2019.
[31] Stefan Falkner, Aaron Klein, and Frank Hutter. Bohb: Robust and efficient hyperparameter optimization scale. In International Conference on Machine Learning, pages 1437-1446. PMLR, 2018.
[32] Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkata- subramanian. Certifying and removing disparate impact. In Proceedings of the Annual Conference on Knowledge Discovery and Data Mining (KDD), pages 259-268, 2015.
[33] Matthias Feurer and Frank Hutter. Hyperparameter optimization. In Automated machine learning, pages 3-33. Springer, Cham, 2019.
[34] HC Fu and P Liu. A multi-objective optimization model based non-dominated sorting genetic algorithm. International Journal of Simulation Modelling, 18(3)-510-520, 2019.
[35] Naman Goel, Mohammad Yaghini, and Boi Faltings. Non-discriminatory machine learning through convex fairness criteria. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), 2018. URL https::/ //ojs. aaai org/ index. phpp AAAI/ arrt wwwccleeeeeeeeeeeeeeeeeeeeeeeeee 11662.
12

---

[36] Google, How google uses pattern recognition to make sense of images, 2021. URL https:
//policies. gowgle.......o....ooooooooo..... cow/technologiaeeccccccc wttpriisepatttteeeeeeeee www.errreoooooionnnooeeeeieiinea tioniiltonnhheeee
[37] Patrick Grother, Mei Ngan, and Kayee Hanaoka. Face Recognition Vendor Test (FVRT): Part
3, Demographic Effects. National Institute of Standards and Technology, 2019.
[38] Patrick J. Grother, George W. Quinn, and P J. Phillips. Report on the evaluation of 2d still- image face recognition algorithms. NIST Interagency/nnternal Report (NISTIR), 2010. URL https://dol:
org/ 10/.//0.//0 6028/NISSI IR. 7709.
[39] Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, and Jian Sun. Single path one-shot neural architecture search with uniform sampling. In European conference on computer vision, pages 544-560. Springer, 2020.
[40] Foad Hamidi, Morgan Klaus Scheuerman, and Stacy M Branham. Gender recognition or gender reductionism? the social implications of embedded gender recognition systems. In Proceedings of the 2018 chi conference human factors in computing systems, pages 1-13, 2018.
[41] Dongyoon Han, Sangdoo Yun, Byeongho Heo, and YoungJoon Yoo. Rexnet: Diminishing representational bottleneck on convolutional neural network. arXiv preprint arXiv:2007.0099) 6, 2020.
[42] Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. Ghostnet: More features from cheap operations. In Proceedings the IEEE/CVF conference computer vision and pattern recognition, pages 1580-1589, 2020.
[43] Kai Han, An Xiao, Enhua Wu, Jianyuan Guo, Chunjing Xu, and Yunhe Wang. Transformer in transformer. Advances in Neural Information Processing Systems, 14:15908-15913 2021.
[44] Moritz Hardt, Eric Price, Nati Srebro, et al. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pages 3315-33222 2016.
[45] Caner Hazirbas, Joanna Bitton, Brian Dolhansky, Jacqueline Pan, Albert Gordo, and Cris- tian Canton Ferrer. Towards measuring fairness in ai: the casual conversations dataset. arXiv preprint wrXiv:2104.0282) 2021.
[46] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[47] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings the IEEE conference computer vision and pattern recognition, pages 770-778, 2016.
[48] Kashmir Hill. Another arrest, and jail time, due to a bad facial recognition match. The New York Times, 29, 2020.
[49] Kashmir Hill. The secretive company that might end privacy as we know it. In Ethics of Data and Analyties, pages 170-177. Auerbach Publications. 2020.
[50] Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang. Vijay Vasudevan, et al. Searching for mobilenetv3. In Proceedings of the IEEF/CVE International Conference on Computer Vision, pages 1314 1324, 2019.
[51] Jie Hu, Li Shen, and Gang Sun. Squeere-and-exciie networks. In Proceedings of the IEEE conference on computer vision and puttern recognition, pages 7132-7141, 2018.
[52] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q Weinberger. Densely connected convolutional networks. 20/7 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2261-2269, 2017. doi: 10.1109//CVPR.2017.24...
13

---

[53] Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned- Miller.. Labeled faces in the wild: database forstudying face recognition in unconstrained environments. In Workshop on faces in Real-Life 'Images: detection, alignment, and recognition, 2008.
[54] Siddharth Jaiswal, Karthikeya Duggirala, Abhisek Dash, and Animesh Mukherjee. Two-face: Adversarial audit of commercial face recognition systems. In Proceedings of the International AAAI Conference on Web and Social Media, volume 16, pages 381-392, 2022.
[55] Jungseock Joo and Kimmo Kärkkäinen. Gender slopes: Counterfactual faimess for computer vision models by attribute manipulation. arXiv preprint htth::0005.04330 2020.
[56] Os Keyes. The misgendering machines: Trans/hci implications of automatic gender recognition. Proceedings of the ACM on human-computer interaction, 2(CSCW))1-22, 2018.
[57] Brendan F Klare, Mark Burge, Joshua Klontz, Richard W Vorder Bruegge, and Anil K Jain. Face recognition performance: Role of demographic information. IEEE Transactions Information Forensics and Security, 16)):188--180, 2012.
[58] Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres, Quantifying the carbon emissions of machine learning. arXiv preprint (rXivv1910009700 2019.
[59] Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed H. Chi. Fairness without demographics through adversarially reweighted learning. 2020.
arXiv preprint 2rXiv:2006.131
[60] Marco Laumanns and Jiri Ocenasek. Bayesian optimization algorithms for multi-objective optimization. In International Conference on Parallel Problem Solving from Nature, pages 298-307. Springer, 2002.
[61] Erik Learned-Miller Vicente Ordóñez, Jamie Morgenstern, and Joy Buolamwini. Facial recognition technologies in the wild, 2020.
[62] Youngwan Lee and Jongyoul Park. Centermask: Real-time anchor-free instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13906 13915, 2020.
[63] David Leslie. Understanding bias in facial recognition technologies. arXiv preprint UrXiv:2010.07023, 2020.
[64] Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. Hyperband: novel bandit-based approach to hyperparameter optimization. The Journal of Machine Learning Research, 18(1):6765-6816 2017.
[65] Xiaofeng Lin, Seungbae Kim, and Jungseock Joo. Fairgrape: Fairness-aware gradient pruning method for face attribute classification. arXiv preprint arXiv:2207. 200.88807..8888 2022.
[66] Marius Lindauer, Katharina Eggensperger, Matthias Feurer, André Biedenkapp, Difan Deng, Carolin Benjamins, Tim Ruhkopf, René Sass, and Frank Hutter. Smac3: A versatile bayesian optimization package for hyperparameter optimization. Journal of Machine Learning Research, 23(54):1-9, 2022. URL http: /mlr.. .org/ wwtp/oaerr////eeeeeeoeee.. papprss/233211-6666
html.
[67] Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.0900 2018.
[68] Ze Liu, Yutong Lin, Yue Cao. Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 10012-10022, 2021.
[69] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pages 3730-3738, 2015.
[70] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL httpps://openreeeeee net ffrum? id-Bkg6RiCqY7.
14

---

[71] Gong Mao-Guo, Jiao Li-Cheng, Yang Dong- Dong, and Ma Wen-Ping. Evolutionary multi- objective optimization algorithms. Journal of Software, 20(2), 2009.
[72] James Marson and Brett Forrest. Armed low-cost drones, made by turkey, reshape battlefields and geopolitics. The Wall Street Journal, Jun, 2021.
[73] Natalia Martinez, Martin Bertran, and Guillermo Sapiro. Minimax pareto fairness: A multi objective perspective. In Proceedings of the 37th International Conference on Machine Learning, volume 119, pages 6755-6764, 2020. URL http::ttttp:ceeeeeeeeeee v119/martinez20a html.
mlr alr.press/
[74] Dushyant Mehta, Oleksandr Sotnychenko, Franziska Mueller, Weipeng Xu, Mohamed El- gharib, Pascal Fua, Hans-Peter Seidel, Helge Rhodin, Gerard Pons-Moll, and Christian Theobalt. Xnect: Real-time multi-person 3d human pose estimation with a single rgb camera. arXiv preprint arXiv:1907.000
2019.
[75] Qiang Meng, Shichao Zhao, Zhida Huang, and Feng Zhou. Magface: A universal representa- tion for face recognition and quality assessment. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 14225-14234, 2021.
[76] Aythami Morales, Julian Fierrez, Ruben Vera-Rodriguez, and Ruben Tolosana. Sensitivenets: Learning agnostic representations with application to face images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
[77] Stylianos Moschoglou, Athanasios Papaioannou, Christos Sagonas, Jiankang Deng, Irene Kotsia, and Stefanos Zafeiriou. Agedb: the first manually collected, in-the-wild age database. In proceedings of the IEEE conference computer vision and pattern recognition workshops, pages 51-59, 2017.
[78] Amitabha Mukerjee, Rita Biswas, Kalyanmoy Deb, and Amrit P Mathur. Multi-objectivy evolutionary algorithms for the risk-return trade off in bank loan management. International Transactions in operational research, 2002.
[79] Vedant Nanda, Samuel Dooley, Sahil Singla, Soheil Feizi, and John P Dickerson. Fairness through robustness: Investigating robustness disparity in deep learning. In Proceedings the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 466-477, 2021.
[80] Arvind Narayanan. Translation tutorial: 21 fairness definitions and their politics. In Proc. Conf. Fairness Accountability Transp., New York, USA, 2018.
[81] Eric WT Ngai, Yong Hu, Yiu Hing Wong, Yijun Chen, and Xin Sun. The application of data mining techniques in financial fraud detection: A classification framework and an academic review of literature. Decision support systems, 50(3):559-56) 2011.
[82] Alice OoTole, Jonathon Phillips, Xiaobo An, and Joseph Dunlop. Demographic effects on estimates of automatic face recognition performance. Image and Vision Computing, 30(3):
169-175 199,176,270 2012.
[83] Manisha Padala and Sujit Gujar. Fnne: Achieving fairness through neural networks. In Proceedings of the TwentyNinth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 2277-2283 International Joint Conferences on Artificial Intelligence Organi- zation, 2020. doi: 10.2 0..2466//jcaii222200 htthtss: //doi. org/10. 24963/ / jcai. 2020/315.
URL https:
[84] Biswajit Paria, Kirthevasan Kandasamy, and Barnabás Póezos. A flexible framework for multi-objective bayesian optimization using random scalarizations. In Uncertainty in Artificial Intelligence, pages 766-776. PMLR, 2020.
[85] Amandalynne Paullada, Inioluwa Deborah Raji, Emily M Bender, Emily Denton, and Alex Hanna. Data and its (dis) contents: A survey of dataset development and use in machine learning research. Patterns, 2001) 2021.
[86] Kenny Peng, Arunesh Mathur, and Arvind Narayanan. Mitigating dataset harms requires stewardship: Lessons from 1000 papers. arXiv preprint arXiv:2108.0290 2021.
15

---

[87] Valerio Perrone, Michele Donini, Muhammad Bilal Zafar, Robin Schmucker, Krishnaram Kenthapadi, and Cédric Archambeau. Fair bayesian optimization. Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 854 2021.
[88] Hieu Pham, Melody Guan, Barret Zoph, Quoe Le, and Jeff Dean. Efficient neural architecture search via parameters sharing. In International conference on machine learning, pages 4095-. 4104. PMLR, 2018.
[89] Novi Quadrianto, Viktoriia Sharmanska, and Oliver Thomas. Discovering fair repre- sentations in the data domain. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA. June 16-20, 2019, pages 8227-8236. Computer Vision Foundation / IEEE, 2019. doi: 10.1109/CVPP.201 2084000842.. URL http: Discovering.
//openaccess thecvf. com/ content_ CVPR_ Btt___VVRRR_________ 2019/htmm/Quadrianntn Fair Risi__Repreentationn in_ the_Data__DDeeeeeeeeeeeeeee Daee_Data_DDminn____aeee CVPR______________ 2019 ______0011________ _apprr .tml.
[90] Inioluwa Deborah Raji and Joy Buolamwini. Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial ai products. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 435, 2019.
[91] Inioluwa Deborah Raji and Genevieve Fried. About face: survey of facial recognition evaluation. arXiv preprint arXivv2102.008133 2021.
[92] Inioluwa Deborah Raji, Timnit Gebru, Margaret Mitchell, Joy Buolamwini, Joonseok Lee, and Emily Denton. Saving face: Investigating the ethical concerns of facial recognition auditing. In Proceedings of the AAAI/ACM Conference AI, Ethics, and Society, pages 145-151, 2020.
[93] Hee Jung Ryu, Hartwig Adam, and Margaret Mitchell. Inclusivefacenet: Improving face attribute detection with race and gender diversity, arXiv preprint arXiv: 2018.
[94] Debjani Saha, Candice Schumann, Duncan C. McElfresh, John P. Dickerson, Michelle L. Mazurek, and Michael Carl Tschantz. Measuring non-expert comprehension of machine learning fairness metrics. In Proceedings of the International Conference on Machine Learning (ICML), 2020.
[95] David Salinas, Matthias Seeger, Aaron Klein, Valerio Perrone, Martin Wistuba, and Cedric Archambeau. Syne tune: A library for large scale hyperparameter tuning and reproducible research. In International Conference on Automated Machine Learning, pages 16-1. PMLR, 2022.
[96] Yash Savani, Colin White, and Naveen Sundar Govindarajulu. Intra-processing methods for debiasing neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[97] Robin Schmucker, Michele Donini, Valerio Perrone, Muhammad Bilal Zafar, and Cédric Archambeau. Multi-objective multi-fidelit hyperparameter optimization with application to fairness. In NeurlPS Workshop on Meta Meta-Learning, volume 2, 2020.
[98] Robin Schmucker, Michele Donini, Muhammad Bilal Zafar, David Salinas, and Cédric Ar- chambeau. Multi-objective asynchronous successive halving. arXiv preprint (tt:22006112334 2021.
[99] Andrew D. Selbst, Danah Boyd, Sorelle A. Friedler, Suresh Venkatasubramaniin, and Janet Vertesi. Fairness and abstraction in sociotcchnical systems. Proceedings of the Conference on Fairness, Accountability, and Transparency, F 11,, page 59-68, New York, NY, USA, 2019. Association for Computing Machinery, ISBN 978 4503611555.. doi: 10. (0.1445/287560032335588 URL https://dooi org/ 10. 1144/112776660 3287598.
[100] Soumyadip Sengupta, Jun-Cheng Chen, Carlos Castillo, Vishal M Patel, Rama Chellappa, and David W Jacobs. Frontal to profile face verification in the wild. In 2016 IEEE winter conference on applications of computer vision (WACV), pages 1-9, IEEE, 2016.
[101] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 1xXv:1140911556, 2014.
16

---

[102] Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose estimation. In CVPR, 2019.
[103] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 2818-2822 2016.
[104] Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence, 2017.
[105] Mingxing Tan and Quoe Le. Efficcinttet:: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105-61 14. PMLR, 2019.
[106] Sahil Verma and Julia Rubin. Fairness definitions explained. In 2018 IEEE/ACM International Workshop on Software Fairness (FairWare), pages 1-7. IEEE, 2018.
[107] Chien- Yao Wang. Hong- Hong-Yuan Mark Liao, Yuch-Hua Wu, Ping- Yang Chen, Jun-Wei Hsich, and I-Hau Ych. Cspnet: A new backbone that can enhance learning capability of cnn. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 390-391, 2020.
[108] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition, In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5265-5274, 2018.
[109] Mei Wang and Weihong Deng. Deep face recognition:
survey. arXiv preprint
arXiv:1804.06633 2018.
[110) Mei Wang and Weihong Deng. Mitigating bias in face recognition using skewness-aware reinforcement learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9322-9331, 2020.
[111] Mei Wang, Weihong Deng, Jiani Hu, Xunqiang Tao, and Yaohai Huang. Racial faces in the wild: Reducing racial bias by information maximization adaptation network. In Proceedings of the IEEE International Conference on Computer Vision, pages 692-702, 2019.
[112] Qingzhong Wang. Pengfei Zhang, Haoyi Xiong, and Jian Zhao Face. evolve: A high- performance face recognition library. arXiv preprint arXiv:2107.0866
2021.
[113] Xiaobo Wang. Teacher guided neural architecture search for face recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 2817-2825, 2021.
[114] Zeyu Wang, Klint Qinami, loannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. Towards fairness in visual recognition: Effective strategies for bias mitigation, 2020.
[115] Colin White, Willie Neiswanger, and Yash Savani. Bananas: Bayesian optimization with neural architectures for neural architecture search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10293-10301, 2021.
[116] Colin White, Mahmoud Safari, Rhea Sukthanker, Binxin Ru, Thomas Elsken, Arber Zela, Debadeepta Dey, and Frank Hutter. Neural architecture search: Insights from 1000 papers. arXiv preprint arXiv:2301.088 08727, 2023.
[117] Ross Wightman. Pytorch image models.
pytorch- pytorch-imagg-.. imaai--mmeelll models, 2019.
[118] Wenying Wu, Pavlos Protopapas. Zheng Yang, and Panagiotis Michalatos. Gender classifi- cation and bias mitigation in facial images. In 12th acm conference on web science, pages 106-114, 2020.
[119] Saining Xie, Ross B. Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. CoRR, 0bs/1611105433 2016. URL. attp://arais org/ ab//a/as/11..../////// 05431.
17

---

[120] Weijian Xu, Yifan Xu, Tyler Chang, and Zhuowen Tu. Co-scale conv-attentional image transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9981-9990, 2021.
[121] Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Guo-Jun Qi. Qi Tian, and Hongkai Xiong. Pe-darts: Partial channel connections for memory-efficiet architecture search. arXiv preprint arXiv: 0et1::907.000 077.05737, 2019.
[122] Fisher Yu, Dequan Wang, Evan Shelhamer, and Trevor Darrell. Deep layer aggregation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2403-2412, 2018.
[123] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P. Gummadi. Fairness constraints: Mechanisms for fair classification. In Proceedings of the 20th Interna- tional Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA, volume 54 of Proceedings of Machine Learning Research, pages 962-970. PMLR, 2017. URL http::ttppppettt pprtte///prooeeiingss mlr, prres//54/zafaee
html.
[124] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P. Gummadi. Faimess constraints: A flexible approach for fair classification. Journal of Machine Learning Research, 20(75): 2019. URL http: /mrrr./mrrr/////// org/ httg/aapers/v20/18 262. html.
[125] Arber Zela, Thomas Elsken, Tonmoy Saikia, Yassine Marrakchi, Thomas Brox, and Frank Hutter. Understanding and robustifying differentiable architecture search. arXiv preprint Seti::1909.096355 2019.
[126] Zizhao Zhang, Han Zhang, Long Zhao, Ting Chen, and Tomas Pfister. Aggregating nested transformers. arXiv preprint arXiv:2105.12723, 2021.
[127] Tianyue Zheng and Weihong Deng. Cross-pose Ifw: database for studying cross-pose face recognition in unconstrained environments. Beijing University of Posts and Telecommunica- tions, Tech. Rep, 5(7), 2018.
[128] Tianyue Zheng, Weihong Deng, and Jiani Hu. Cross-age Ifw: A database for studying cross-age face recognition in unconstrained environments. arXiv preprint arXi::1708.08197 2017.
[129] Ning Zhu. Neural architecture search for deep face recognition. arXiv:1904.088 09523, 2019.
arXiv preprint
18

---

A Ethics Statement
Face recognition systems are being used for more and more parts of daily lives, from government surveillance [49], to target identification from drones [72], to identification in personal photo reposito- ries [36]. It also increasingly evident that many of these models are biased based on race and gender [37, 92, 91]. If left unchecked, these technologies, which make biased decision for life-changing events, will only deepen existing societal harms. Our work seeks to better understand and mitigate the negative effects that biased face recognition models have on society. By conducting the first large-scale study of the effect of architectures and hyperparameters on bias, and by developing and open-sourcing face recognition models that are more fair than all other competitive models, we provide a resource for practitioners to understand inequalities inherent in face recognition systems and ultimately advance fundamental understandings of the harms and technological ills of these systems.
That said, we would like to address potential ethical challenges of our work. We believe that the main ethical challenge of this work centers on our use of certain datasets. We acknowledge that the common academic datasets which we used to evaluate our research questions - CelebA [69] and VGGFace2 [8] - are all datasets of images scraped from the web without the informed consent of those whom are depicted. This also includes the datasets we transfer to, including LFW, CFP_ FF, CFP_FP, Ag CALFW, CPLFW, and RFW. This ethical challenge is one that has plagued the research and computer vision community for the last decade [86, 85] and we are excited to see datasets being released which have fully informed consent of the subjects, such as the Casual Conversations Dataset [45]. Unfortunately, this dataset in particular has a rather restrictive license, much more restrictive than similar datasets, which prohibited its use in our study. Additionally, these datasets all exhibit representational bias where the categories of people that are included in the datasets not equally represented. This can cause many problems; see [14] for a detailed look at reprsentational bias's impact in facial recognition specifically. At least during training, we did address representaional bias by balancing the training data between gender presentations appropriately.
We also acknowledge that while our study is intended to be constructive in performing the first neural architecture search experiments with fairness considerations, the specific ethical challenge we highlight is that of unequal unfair treatment by the technologies. We note that our work could be taken as a litmus test which could lead to the further proliferation of facial recognition technology which could cause other harms. If system demonstrates that it is less biased than other systems, this could be used as a reason for the further deployment of facial technologies and could further impinge upon unwitting individual's freedoms and perpetuate other technological harms. We explicitly caution against this form of techno-solutionisn and do not want our work to contribute to a conversation about whether facial recognition technologies should be used by individuals.
Experiments were conducted using a private infrastructure, which has a carbon efficiency of 0.373 kgCO>eq/k Wh. A cumulative of 88 493 hours of computation was performed on hardware of type RTX 2080 Ti (TDP of 250W). Total emissions are estimated to be 8,251.97 kgCOzeq of which 0% was directly offset. Estimations were conducted using the MachineLearning Impact calculator presented in [58]. By releasing all of our raw results, code, and models, we hope that our results will be widely beneficial to researchers and practitioners with respect designing fair face recognition systems.
B Reproducibility Statement
We ensure that all of our experiments are reproducible by releasing our code and raw data files at https://githu: com/ dooleys/RR----eooo NAS. We also release the instructions to reproduce our results with the code. Furthermore, we release all of the configuration files for all of the models trained. Our experimental setup described in Section 3 and Appendix C. We provide clear documentation on the installation and system requirements in order to reproduce our work. This includes information about the computing environment, package requirements, dataset download procedures, and license information. We have independently verified that the experimental framework is reproducible which should make our work and results and experiments easily accessible to future researchers and the community.
19

---

Table 5: The fairness metries explored in this paper. Rank Disparity is explored in the main paper and the other metrics are reported in Appendix C.4
Fairness Metric
Equation
Rank Disparity
Rank(male) - Rank( [
Disparity
aacuracy(male Accuracy (fenuaaacyymmle))
Ratio
1 Accuracy(maaee)
Rank(male)
Rank Ratio
Rank( f femate)
Error Ratio
Erroe (troe(male)
Emort J Jemale)
C Further Details Experimental Design and Results
C.1 Experimental Setup
The list of the models we study from timm are: peat_litessmall [120], convit_base [20].
cspdarkmet53 [107], dla102x2 [122], dpn107 [10], ese_ ves_vvoee [62], fbnetv3_g [18].
[102), jg_resnext101__244 _2x0_x__xddd_ [119], [103], nluon_xceptionee [15], hrnet_w64
ghostnet_100 [42], clluoo_inceetionn..
inception_resnet...
[104], inception_
[104], jx_nest.base [126]. Legacy_senet.ll
[51], aobilenetv3_large_err
[50].
resnetrs101 [4], rexnet_200 [41], selecs1s60b [74], swin Srsi__base__aee http__paach__iiiioepp _224
[68], tf afficientnet_b7_nn [105], 'tnt_s_ s_tt___pe_t __t______ptchh______ [24[43),_3), wwins_svt_las [16]
vgg19 [101], vgg19_bn [101]. visformer_smail [11], xception and xception65 [15].
We study at most configurations per model default configuration corresponding to the original model hyperparameters with CosFace as head. Further, we have at most 12 configs consisting of the heads (CosFace, ArcFace, MagFace) 2 learning aates(0.1,,.000) All the other hyperparameters are held constant for training all the models. All model configurations are trained with a total batch size of 64 on 8 RTX2080 GPUS for 100 epochs each.
optimizers (SGD, AdamW),
We study these models across five important fairness metrics in face identification: Rank Disparity, Disparity, Ratio, Rank Ratio, and Error Ratio. Each of these metrics is defined in Table 5.
C.2 Additional details on NAS+HPO Search Space
We replace the repeating 1x1_coov-_00 conv-3x3 2atrv-333_ooo_____1 conv-1xl 1xnnyiiooovvvvv simple recurring searchable block. depicted in Figure Furthermore, we stack multiple such searched blocks to closely follow the architecture of Dual Path Networks. We have nine possible choices for each of the three operations the DPN block, each of which we give a number through 8, depicted in Figure 6. The choices include vanilla convolution, convolution with pre- normalization and a convolution with post-normalization Table ensure that all the architectures are tractable in terms of memory consumption during search we keep the final projection layer (to 1000 dimensionality) in timm.
conv block in Dual Path Networks with
C.3 Obtained architectures and hyperparameter configurations from Black-Box-Optimizztin
In Figure 5 we present the architectures and hyperparameters discovered by SMAC. Particularly we observe that conv 3x3 followed batch norm is preferred operation and CosPace is the preferred head/loss choice.
C.4 Analysis of the Pareto front of different Fairness Metrics
In this section, we include additional plots that support and expand on the main paper. Primarily, we provide further context of the Figures in the main body in two ways. First, we provide replication plots of the figures in the main body but for all models. Recall, the plots in the main body only show models with Error<0.3, since high performing models are the most of interest to the community.
20

---

Gardells
Corrodila
BaCondad
DiConita
Figure 5: SMAC discovers the above building blocks with (a) corresponding to architecture with CosFace, with SGD optimizer and learning rate of 0.2813 3 as hyperparamters (b) corresponding to CosFace, with SGD optimizer and learning rate of 0.32348 and (c) corresponding CosFace, with AdamW as optimizer and learning rate of 0.0006
Table 6: Searchable hyperparameter choices.
Hyperparameter
Choices
Architecture Head/Los
MagFace, ArcFace, CosFace Adam, AdamW, SGD
Optimizer Type
Learning rate (conditional)
Adam/Adam's - [le 4, le 2], SGD [0.09, 0.8]
Table 7: Operation choices and definitions.
Operation Index
Operation
Definition
BnConvixI
Batch Normalization Convolution with 1xl kernel Convolution with 1x kernel + Batch Normalization Convolution with 1x1 kernel
Conv Ix1 Bn
2
Convlx1
BnConv3x3
Batch Normalization Convolution with 3x3 kernel Convolution with 3x3 kernel + Batch Normalization Convolution with 3x3 kernel
Conv3x3Bn
Conv3x3
BnConv5x5
Batch Normalization + Convolution with 5x5 kernel Convolution with 5x5 kernel Batch Normalization Convolution with 5x5 kernel
Conv5x5Bn
Conv5x5
Figure 6: DPN block (left) vs. our searchable block (right).
21

---

35
30-
Rank Disparity
25
20
15-
10
0-
0.2
0.4
0.6
0.8
Error
Eeee
ReXNet
Inception
DPN
TNT
Figure 7: Replication of CelebA Figure 2 with all data points. Error-Rank Disparity Pareto front of the architectures with any non-trivial error. Models in the lower left corner are better. The Pareto front notated with a dashed line. Other points are architecture and hyperparameter combinations which are not Pareto-dominant.
VGGFace2 Validation Set
12
Rank Disparity
8-
6-
4-
2-
0-
Error
ReXNet
DPN
Figure 8: Replication of VGGFace2 Figure 2 with all data points. Error-Rank Disparity Pareto front of the architectures with any non-trivial error. Models the lower left corner are better. The Pareto front is notated with dashed line. Other points are architecture and hyperparameter combinations which are not Pareto-dominant.
Second, we also show figures which depict other fairness metrics used in facial identification. The formulas for these additional fairness metries be found in Table 5.
We replicate Figure Figure and Figure We add additional metrics for CelebA Figure Figure 11 and for VGGFace2 in Figure 12-Figure 15.
C.5 Correlation between fairness and model statisties
We examine the correlation of each fairness-metrie with model statistics. We compute statistics like number of parameters, model latency, number of convolutions, number of linear layers, and number of batch-norms in model's definition. Interestingly, observe very low and non-significant
22

---

Ratio of Ranks
Ratio Errors
0.6
0.5
Faimess Value
0.4
0.3
0.2
0.1
0-
0
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
Error
Error
HRRNe
Inception
ReXNet
DPN
EseVoVNet
MobileNet
ResNet-RS
VGG19
Figure 9: Replication of Figure on the CelebA validation dataset with Ratio of Ranks (left) and Ratio of Errors (right) metrics.
Disparity
[Acc(male)-Acee Acc(femall))
0.15
0.1
0.05
0.2
0.4
0.6
0.8
Error
swin base
gluon_xceppe xccption65
dpn107
efficieenter
cspdarknet53
Figure 10: Replication of Figure on the CelebA validation dataset with the Disparity in accuracy metric.
correlations between parameter sizes and different fairness metrics Figure This observation supports the claim that increases in accuracy and decreases in disparity are very closely tied to the architectures and feature representations the model, irrespective the parameter size the model. Hence, we do not constraint model parameter sizes to help our NAS+HPO approach search in much richer search space.
C.6 Comparison to other Bias Mitigation Techniques on all Fairness Metrics
We have shown that our bias mitigation approach Pareto-dominates the existing bias mitigation techniques in face identification the Rank Disparity metric. Here, we perform the same experiments evaluate on the four other metrics discussed in the face identification literature: Disparity, Rank Ratio, Ratio, and Error Ratio.
23

---

Ratio
1
|1~ (cc(male)Acc(femel)
0.8
0.6
0.2
0-
0.2
0.4
0.6
0.8
Error
swin base
xceptions
dpa107
II efficientnet
cspdarkne153
Figure 11: Replication of Figure on the CelebA validation dataset with the Ratio accuracy metric.
VGGFace2 Validation Set
0.6
Rank Ratio
0.4
0.3
0.2
0.1
0-
0
0.2
0.4
0.6
0.8
Error
ReXNet
DPN
dla162x2
Figure 12: Replication of Figure 8 on the VGGFace2 validation dataset with Ratio of Ranks metric.
Recall, we take top performing, Pareto-optimal models from Section 4 and apply the three bias miti- gation techniques: Flipped, Angular, and SensitiveNeet We also apply these same techniques to the novel architectures that we found. We report results in Table
In Table 8, we see that in every metric, the SMAC_ 301 architecture is Pareto-dominant and that the RMAC_301, demonstrating the robustness of our approach.
C.7 Transferability to other Sensitive Attributes
The superiority of our novel architectures even goes beyond accuracy when transfering other datasets our novel architectures have superior fairness property compared to the existing architec- tures even on datasets which have completely different protected attributes than were used in the architecture search. To inspect the generalizability of our approach to other protected attributes, we transferred our models pre-trained on CelebA and VGGFace2 (which have gender presentation category) to the RFW dataset [111] which includes protected attribute for race. We see that our novel architectures always outperforms the existing architectures across all five fairness metrics studied in this work. See Table 10 for more details each metric. We example Pareto fronts for

---

VGGFace2 Validation Set
0.5
0.4
Error Ratio
0.3
0.2
0.1
Emrr
Inception
VinFormer
XCeption
TNT
convit_aaee
HRNet
XCeption
twiss_sv__ee sa_geee
TT_ EfficientNet
DPN
plupo_xception55
Figure 13: Replication of Figure 8 on the VGGFace2 validation dataset with Ratio Errors metric.
VGGFace2 Validation Set
0.15
Disparity
0.1
0
0.2
0.4
0.6
0.8
Error
VisPormer
TNT
XCeption
EfficienNNet
DPN
glupo_xceptio665
Figure 14: Replication of Figure 8 on the VGGFace2 validation dataset with the Disparity in accuracy metric.
these transfers in Figure 17. They are always on the Pareto front for all fairness metrics considered, and mostly Pareto-dominate all other architectures on this task. In this setting, since the race label in RFW is not binary, the Rank Disparity metric considered in Table 10 and Figure 17 computed as the maximum rank disparity between pairs of race labels.
Furthermore, we also evaluate the transfer of fair properties our models across different age groups on the AgeDB dataset [77]. In this case we use age as protected attribute. We group the faces into 4 age groups 1-25, 26-50, 51-75 and 10- Then we compute the max disparity across age groups. As observed in Table 11 the models discovered by NAS and HPO, Pareto-dominate other competitive hand-crafted models, This further emphasise the generalizabitlity of the fair features learnt by these models.
25

---

VGGFace2 Validation Set
0.5-
0.4
Ratio
0.3
0.2
0.1
0-
0
0.4
0.8
Error
VisFormer
DPN
Figure 15: Replication of Figure on the VGGFace2 validation dataset with the Ratio in accuracy metric.
objectives
dispaiity
0:12
0.11
0.29
0.15
or
0.19
dispartyy
-0.07
0.05
mat
0.05
0.01
0.27
0.00
rMo 0.20
0.01
0.09
0.01
0.02
-0.05
Bneer
bmd
Figure 16: Correlation map between different fairness metrics and architecture statistics. We find no significant correlation between these objectives, egg. between fairness metrics and parameter count.
Table 11: Taking the highest performing models from the Pareto front of both VGGFace2 and CelebA, we transfer their evaluation onto dataset with different protected attribute age on the AgeDB dataset [77]. The novel architectures which we found with our bias mitigation strategy are Pareto-dominant with respect the Accuracy and Disparity metries
Architecture (trained on VGGFace2) Overall Accuracy 1 Disparity
foxnet__000
59.18
28.915
DPN_SGD LGD
71.87
22.4633
DPN_Adam
61.32
21.1437
SMAC_30
79.97
18.827
Architecture (trained on CelebA)
Accuracy
Disparity
MPN_CosFace
65.55
27. 27.2434
DPN MagFace
68.17
31.2903
SMAC_ _000
80.23
19,6481
SMAC_010 010
80.37
26.5103
SMAC_680
79.88
20.0586
26

---

Table 8: Comparison bias mitigation techniques where the SMAC models were found on VGGFace2 with NAS bias mitigation technique and the other three techniques are standard in facial recognition: Flipped 19], Angular [76], and Discriminator [110]. Items in bold are Pareto-optimal. The values show (Errormetricc)
Rank Disparity Flipped
Disparity
Model
Baseline
Angular
SensitiveNets
Baseline
Flipped
Angular
SensitiveNets
SMAC_301
(3.66:0.23)
(4.95;0.18)
(4.14:0.25)
(6.20;0.41)
(3.66;0.03)
(4.95;0.02)
(4.14:0.04)
(6.14:0.04)
DPN
(3.56:0.27)
(5.87:0.32)
(6.06:0.36)
(4.76:0.34)
(3.98:0.04)
(5.87:0.05)
(6.06:0.05)
(4.78:0.05)
ReXNet
(4.09:0.27)
(5.73:0.45)
(5.47:0.26)
(4.75:0.25)
(4.09:0.03)
(5.73:0.05)
(5.47:0.05)
(4.75:0.04)
Swin
(5.47:0.38)
(5.75:0.44)
(5.23:0.25)
(5.03:0.30)
(5.47:0.05)
(5.75:0.04)
(5.23:0.04)
(5.03:0.04)
Rank Ratio
Ratio
Model
Baseline
Flipped
Angular
SensitiveNets
Baseline
Plipped
Angular
SensitiveNets
SMAC_301
(3.66;0.37)
(4,95;0.21)
(4.14:0.39)
(6.14:0.41)
(3.66;0.03)
(4.95:0.02)
(4.14:0.04)
(6.14:0.05)
DPN
(3,98:0.49)
(5.87:0.49
(6)06:0.54)
(4,78:0.49 (.4)
(3,98:0.04)
(5.87:0.06)
(6.06:0.06)
(4.78:0.05)
ReXNet
(4.09:0.41)
(5.73:0.53)
(5.47:0.38)
(4.75:0.34)
(4.09:0.04)
(5.73:0.05)
(5.47:0.05)
(4.75:0.04)
Swin
(5.47:0.47)
(5.75;0.47)
(5.23;0.42)
(5.03;0.43)
(5.47:0.05)
(5.75;0.05)
(5.23;0.05)
(5.03;0.05)
Error Ratio
Model
Baseline
Flipped
Angular
SensitiveNets
SMAC_301
(3.66;0.58)
(4.95:0.29)
(4.14:0.60)
(6.14:0.52)
DPN
(3.98:0665)
(5.87:0.62)
(6.06:0.62)
(4.78:0.69)
ReXNet
(4.09:0.60)
(5.73:0.57)
(5.47,0.59)
(4.75:0.58)
Swin
(5.47:0.60)
(5.75:0.56)
(5.23:0.60)
(5.03:0.60)
Table 9: Taking the highest performing models from the Pareto front of both VGGFace2 and CelebA, transfer their evaluation onto six other common face recognition datasets: LFW [53], CFP_FF [100], CFP_FT FP [100], AgeDB [77]. CALFW [128), CPLPW [127]. The novel architectures which we found with our bias mitigation strategy significantly out perform all other models.
Architecture (trained on VGGFace2)
LFW
CFP_FF
CFP FP
AgeDB
CALFW
CPLPW
Rexnet_ 100
82,60
80.91
65.51
59.18
68.23
62.15
DPN_SGD _SGD
93,00
91.81
78.96
71.87
78.27
72.97
DPN W
78.66
77.17
64.35
61.32
64.78
60.30
SMAC_301
96.63
95.10
86.63
79.97
86.07
81.43
Architecture (trained CelebA)
LFW
CFP_FF
CFP FP
AgeDB
CALFW
CPLFW
Rexnet_200
71.533
73.528
54.30
55,933
60.517
52.25
DPN RR_CCsFace
87.78
90.73
69.97
65.55
75.50
62.77
DPN MagFace
91.13
92.16
70.58
68.17
76.98
60.80
DenseNet161
82,60
80.16
64.07
56.93
66.36
59.35
Ese Vovnet39
77.00
76.50
64.68
49.38
61.283
61.016
SMAC _00
94.98
95.60
74.24
80.23
84.73
64.22
SMAC_010
94.30
94.63
73.83
80.37
84.73
65,48
SMAC_680
96.16
95.68
72.67
79.88
84.78
63.96
Table 10: Taking the highest performing models from the Pareto front of both VGGFace2 and CelebA, we transfer their evaluation onto dataset with different protected attribue race on the RFW dataset [111]. The novel architectures which we found with our bias mitigation strategy are always on the Pareto front, and mostly Pareto-dominant of the traditional architectures.
Fairness Metric
Transfer from CelebA
Transfer from VGGFace2
Rank Disparity
Pareto-dominat Pareto-dominant
Pareto-optimal
Disparity
tereio-dommnant
Rank Ratio
Pareto-optimal
Pareto-optimal
Ratio
Pareto-dominant
Pareto-dominant
Error Ratio
Pareto-optimal
Pareto-optimal
27

---

Trained on Celebo;;
VGGFace2; Tested on
Models SMAC
180
Ofte:
Refbfee
DoeaNNe DPN,
TIN CosPact
160
EscVoiieer
142
CorFacc
Ronk Disperite
SMAC
123
81
+
Emur
Bimor
Figure 17: Models trained on CelebA (left) and VGGFace2 (right) evaluated on dataset with different protected attribute, specifically on RFW with the racial attribute, and with the Rank Disparity metric. The novel architectures out perform the existing architectures in both settings.
Gender data projection
Gender data projection
100
100
30
comp 2
come
0-
-25
-39
-75
-100
-100
(a)
(b)
Gender projaction
Gender data
malc
300
comp
-20
-100
-100
100
-00
comp-1
comp-1
(c)
(d)
Figure 18: TSNE plots for models pretrained on VGGFace2 on the test-set (a) SMAC model last layer (b) DPN MagFace on the last layer (b) SMAC model second last layer (b) DPN MagFace on the second last layer. Note the better linear separability for DPN MagFace comparison with the SMAC model
28


Reviews:
[{"review_id": "JULuwLG0pZ", "reviewer": "Reviewer_jfj2", "summary": "The authors offer a fresh view on mitigating fairness bias in ML by leveraging neural architecture (NAS) search and hyperparameter optimization (HPO).\nThe authors demonstrate their idea on the exemplary problem of face identification, where fairness biases have tangible consequences on society. They utilize a wide range of model architectures, and define a NAS+HPO search strategy where multi-objective optimization helps balance the tradeoff between accuracy and fairness (as quantified e.g. by the rank disparity metric) .\n", "questions": "Have you considered other alternatives to SMAC3 or ParEGO? How generalizable are the insights in section 4.2. to possible alternatives? ", "limitations": "Sufficiently discussed", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "+ The authors formally describe a new paradigm compared with classical bias mitigation techniques that have traditionally focused on postprocessing/rectifying ML predictions, preprocessing/balancing the datasets, or extending the loss.\n+ The solution is systematically devised and the experiments are straightforward to follow. \n+ The experimental results are insightful and helpful in practice. I find it inspiring that the NAS models generalize to new protected attributes in new datasets.\n+ The authors provide their source code besides a variety of analysis scenarios available to run via notebooks.", "weaknesses": "- There was no discussion on the impact of pretraining. With the availability of a large number of foundational models, it would be very relevant to shed light into which ones generalize better and why.\n- The theoretical analysis is a. bit lacking. I was expecting more explanation of why the NAS models outperform other bias mitigation strategies and better generalize to other sensitive attributes. Is it because you are forcing the model to work harder and to avoid misusing these sensitive attributes as shortcuts when making predictions? (this would explain the reduced linear separability of protected attributes).\n- The visualization of the results could be more insightful. For example a confusion matrix / similarity matrices might be useful to conduct error analysis and shed light into the improvements facilitated by the NAS paradigm. [Such analysis](https://arxiv.org/abs/2007.06068) has revealed many characteristics of VGGFace, e.g., oftentimes, gender misclassification is due to labeling issue (e.g. the image scrapped is not of the actor, but of their opposite-gender spouse).\n\nMinor and language issues:\n- The figures could be better annotated (e.g. to explain that the two red dots in Figure 2 are two variants of DPN)\n- when comparing to other bias mitigation techniques => compared?\n- at the most extreme low errors => unclear\n- oepration\n- to supports\n- are Pareto-optimal the top performing … => are the Pareto-optimal top-performing …\n- recognititon", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your time and thoughtful feedback on our manuscript. We appreciate that you see our view on mitigating fairness bias in ML as fresh and interesting, our solution systematically devised, and our experiments straightforward to follow. Further, we are glad that you find our results insightful, inspiring and useful in practice. We address each of your questions below:\n\n**W1: The Effect of Pre-training**\nThank you for raising this important point. Prompted by your feedback, we now fine-tuned a pre-trained DPN model obtained from timm on vggface-2, as DPN is the most representative of our search space. We compare the error trajectory of this model with and without pre-training. Interestingly, we observe that while the pre-trained model starts strong in terms of accuracy, the model trained from scratch catches up quickly. And more importantly, the disparity of the pre-trained model is **much** higher compared to the model trained from scratch. You can find the plot for the same in the attached PDF (Figure 2 (a) and (b)). We have updated our working draft accordingly, and we will perform more experiments to include in the updated manuscript. \n\n**W2: Theoretical Analysis**\nThe proposed multi-objective neural architecture search and HPO simultaneously optimizes two objectives, firstly the accuracy and secondly the fairness metric (e.g. rank disparity). Hence, we bias the search toward models which do not exploit the protected attribute (e.g. gender) to make classifications. This is also reflected in the reduced linear separability of the features of the models discovered by SMAC. We hypothesize that the SMAC models learn to use more fine grained facial features to distinguish faces instead of exploiting obvious coarse features like protected attributes (gender, race, age). We leave a more detailed analysis on the properties of the features learned for future work.\n\n**W3: Visualizations**\nWe appreciate this feedback. We have now conducted a new analysis in accordance with your suggestion in Figure 1 (a) of the attached PDF. Specifically, we visualize a dendrogram of the SMAC\\_301 model as well as three highly performing DPNs. The visualization shows the correlation of the logits for each image in each identity. From this analysis, we observe that SMAC\\_301 and DPN\\_CosFace\\_SGD have smaller cross-logit similarities, pointing to the fact that they do not try to exploit easier image properties like protected attributes to cluster images. The average similarities are lower for these models compared to others and the max similarities are much higher. We hypothesize that this property aids fair classifications. \n\n**Q1 SMAC3 and ParEGO Pareto-dominate other methods**\nWe agree that given the plethora of methods for multi-objective NAS+HPO, there are multiple algorithms one could choose from. Given that SMAC3 supports parallelization and multi-fidelity search, we primarily restricted ourselves to it for compute optimal search. However, following your advice, we now studied two other multi-objective methods: MOASHA [1] and NSGA-II [2] from the syne-tune [3] library, using our search space design. Note that we run the search for a limited time budget of 48 hrs, so the models discovered may improve with a longer search budget. We will include an extended search comparison in our updated manuscript.  The results are found below and indicate that our chosen method Pareto-dominates the other methods for all metrics except for Error Ratio where it is Pareto-optimal.\n\n|                  |   Accuracy |   Rank Disparity |   Disparity |     Ratio |   Rank Ratio |   Error Ratio |\n|:-----------------|-----------:|-----------------:|------------:|----------:|-------------:|--------------:|\n| MO-ASHA_032 |   0.934739 |         0.390588 |   0.0485621 | 0.0533381 |     0.448144 |     0.542336 |\n| NSGA-II_728 |   0.868105 |         0.599085 |   0.0857516 | 0.103913  |     0.490213 |      **0.490651** |\n| SMAC_301    |  **0.963366** |         **0.230327** |   **0.0300871** | **0.0317269** |     **0.367554** |      0.582215 |\n\n\n[1] Schmucker, R., Donini, M., Zafar, M.B., Salinas, D. and Archambeau, C., 2021. Multi-objective asynchronous successive halving. arXiv preprint arXiv:2106.12639\n\n[2] Deb, K., Pratap, A., Agarwal, S. and Meyarivan, T.A.M.T., 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation, 6(2), pp.182-197\n\n[3] Salinas, D., Seeger, M., Klein, A., Perrone, V., Wistuba, M. and Archambeau, C., 2022, September. Syne tune: A library for large scale hyperparameter tuning and reproducible research. In International Conference on Automated Machine Learning (pp. 16-1). PMLR\n"}, {"review_id": "myf8hnxbsN", "reviewer": "Reviewer_Kze1", "summary": "This paper presents a new perspective on bias mitigation in machine learning models, challenging the conventional belief that one should first find the highest-performing model and then apply a bias mitigation strategy. The authors propose that finding a fairer architecture offers significant gains compared to conventional bias mitigation strategies. To test this hypothesis, they conduct the first neural architecture search for fairness and a search for hyperparameters in face recognition. ", "questions": "Please refer to the WeakNess", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper proposes a new way to mitigate biases in face recognition systems from the perspective of fairer model architectures.\nThis paper conducts the first large-scale analysis of the impact of architectures and hyperparameters on bias in face recognition, demonstrating that the implicit convention of choosing the highest-accuracy architectures is a sub-optimal strategy for fairness.\nThis paper may be the first to apply existing tools from NAS and HPO to design a fair face recognition model automatically.", "weaknesses": "（1）According to the definition of rank difference, the smaller the model's error, the fairer the model. Therefore, the difference in model parameters will lead to a difference in model performance. There is a lack of consistent constraints on the magnitude of the model parameters when searching for a fairer structure in this paper.\n（2）Why the results in Table 2 are far worse than those reported in other papers.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We first thank you for your time and thoughtful feedback on our manuscript. We are glad that you find our approach novel and interesting. We address each of your questions below:\n\n**W1: Rank Disparity Definition**\nThank you for raising this point. Precisely as per the definition of rank disparity, Rank(image) = 0 if and only if Error(image) = 0. This, however, **doesn’t** necessarily imply that decreasing error would correspond to decreasing rank disparity. Unlike the ratio of errors metric, rank disparity is a much richer metric which **doesn’t**  have a strong correlation with error rate. \n\nTo probe this question, we conducted a new analysis (Figure 1 (a) in the PDF) which examines the correlation of each fairness-metric with model statistics. We compute statistics like number of parameters, model latency, number of convolutions, number of linear layers, and number of batch-norms in a model’s definition. Interestingly, we observe very low and non-significant correlations between parameter sizes and different fairness metrics. This observation supports the claim that increases in accuracy and decreases in disparity are very closely tied to the architectures and feature representations of the model, irrespective of the parameter size of the model. Hence, not constraining the parameter size helps our NAS+HPO approach search in a richer search space.\n\n**W2: Table 2 Results**\nTable 2 reports results of transfer learning from the given datasets (VGGFace2 on top and CelebA below) to the given datasets. Thus, the performance will be lower than if each model were fine-tuned or hyperparameters were optimized for each model on each dataset, or if a different pre-training dataset were used. We highlight here that the transfer learning result is strong and indicative that the representations that are learned by our novel architectures are indeed generalizable in a way that the other models are not. We have clarified this point in our updated manuscript.\n"}, {"review_id": "sLRMlJKLUi", "reviewer": "Reviewer_Nw7v", "summary": "The paper focuses on Bias Mitigation for face identification, i.e., ensuring that face identification works “well” for different identities: gender, race, etc. Unlike prior work, which focuses on model backbone agnostic methods to mitigate bias, this work explores the relevance of the inductive bias encoded in different Deep Learning model's backbones to the bias issue. In other words, are there model backbones better at learning robust features and ensuring good performance on samples from different identities? The authors uncover model configurations that significantly improve performance over standard model backbones through an extensive empirical analysis based on Neural Architecture Search on two face identification datasets. More surprisingly, the authors prove that these backbones' performance is better or more competitive than when paired with standard bias mitigation methods. Moreover, the authors confirm the generalizability of the configurations uncovered using CelebA and VGGFace2 by testing them on other datasets, further confirming their competitive performance. Finally, the authors analyze the new model configurations and ensure that the features learned by the models are less likely to be discriminative between the biased groups confirming that they learn more diverse and robust features. \n", "questions": "I am overall positive about this work. However, I need further clarifications as outlined in the Weakness sections, particularly questions: (2,3,4). I am happy to increase my score upon adequate further clarification. \n", "limitations": "I commend the authors for explicitly discussing their work's technical limitations and that while it improves the notion of technical fairness, the advancement could still be harmful in downstream applications. \n", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1- The work is well motivated; prior work needs to include an analysis of model architecture relevance to the bias issue. \n\n2-The results are interesting and relevant to fairness/bias community practitioners.  \n\n3-The advantages of newly discovered model configurations are explored by a well-designed empirical analysis that confirms the configurations' more robust learned features.  \n", "weaknesses": "1- Some choices in the experimental design would benefit from further motivation. For example: \nWhy was the “multi-fidelity Bayesian optimization method SMAC3” chosen in particular? Are there other methods that could also work? \n\n\n2- SMAC_301 was the architecture that works well across datasets. I understand that 301 denotes the operations that constitute the novel architecture. However,  the authors do not discuss the details of these operations or why they think they are meaningful choices compared to other choices ruled out by the NPS. Some discussion here would be helpful. Why do the authors think this configuration is better able to learn non-linearly separable bias features? \n\n3- In Section 3, why are the models trained on a gender-balanced subset of the dataset? Wouldn’t one want to train on a gender-imbalanced split to see which architectures are less likely to be influenced by the imbalance? Is this the same in Section 4? Please clarify. \n\n4- In the analysis in Section 4.2, were the hyper-parameters (learning rate, optimizer) of the timm models tuned too? Or were they the ones used by the original papers? If it is the latter, then it is an unfair comparison to the SMAC models since those hyper-parameters were tuned as outlined in Section 4.1.\n\n5- From reading the paper in detail, I understand now that one of the motivations of Section 3 analysis was to limit the number of architectures explored in Section 4 (only DPN was considered since it achieved Pareto optimal performance on both datasets). However, I did not get that from the first read, so further clarification in the text about that would be helpful. \n\n6- In Section 3, the hyper-parameters of the different models were not optimized. It is likely very computationally expensive. Nevertheless, it does undermine some elements of the analysis, so I would make that clear as limitations.  \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and thoughtful feedback on our manuscript. We appreciate that you see the novelty in our work being the first to systematically conduct a large-scale analysis on the problem of fairness face recognition with different architectures and hyperparameters. We address each of your points below.\n\n**W1: The choice of SMAC3**\nThank you for your suggestion. We agree that given the plethora of methods for multi-objective NAS+HPO, there are multiple algorithms one could choose from. Given that SMAC3 supports parallelization across GPUs and multi-fidelity search, we initially restrict ourselves to SMAC3. Following your advice, we now studied two other multi-objective methods, MOASHA [1] and NSGA-II [2] from the syne-tune[3] library, using our search space design. Note that we run the search for a limited time budget of 48 hrs, so the models discovered would likely improve with a longer search budget. We will include an extended experiment in our updated manuscript.  We present the results below:\n\n|                  |   Accuracy |   Rank Disparity |   Disparity |     Ratio |   Rank Ratio |   Error Ratio |\n|:-----------------|-----------:|-----------------:|------------:|----------:|-------------:|--------------:|\n| MO-ASHA_032 |   0.934739 |         0.390588 |   0.0485621 | 0.0533381 |     0.448144 |     0.542336 |\n| NSGA-II_728 |   0.868105 |         0.599085 |   0.0857516 | 0.103913  |     0.490213 |      **0.490651** |\n| SMAC_301    |  **0.963366** |         **0.230327** |   **0.0300871** | **0.0317269** |     **0.367554** |      0.582215 |\n\n\n[1] Schmucker, R., Donini, M., Zafar, M.B., Salinas, D. and Archambeau, C., 2021. Multi-objective asynchronous successive halving. arXiv preprint arXiv:2106.12639\n\n[2] Deb, K., Pratap, A., Agarwal, S. and Meyarivan, T.A.M.T., 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation, 6(2), pp.182-197\n\n[3] Salinas, D., Seeger, M., Klein, A., Perrone, V., Wistuba, M. and Archambeau, C., 2022, September. Syne tune: A library for large scale hyperparameter tuning and reproducible research. In International Conference on Automated Machine Learning (pp. 16-1). PMLR\n\n\n\n**W2: Why is SMAC_301 the best model?**\nIn most of the fair models discovered by NAS+HPO, we see a prevalence of BnConv3x3 operation (every architecture containing at least one or more of such operations). Furthermore, in terms of the optimal face recognition head, we surprisingly find a strong preference for “CosFace” instead of “MagFace” and “ArcFace”. We find that “ArcFace” has the least preference during search. Moreover, we also discover that the SGD optimizer with high learning rates > 0.1 is often preferred in comparison to AdamW and Adam optimizers from our search space. We believe that these are just some of the important characteristics (architectural and hyper-parameter pipeline) for making models more fair.  We will include an extended discussion of these components in our updated manuscript. \n\n**W3: Gender-balanced training**\nWe have employed the training regime for fair face identification as described by [4], which shows the importance of training models with fully balanced datasets (both balanced in identities and number of images per identity). They point out how these two types of imbalances (both at training and testing time) can cause researchers to draw misleading or incorrect conclusions. Thus, balancing the training and testing data as we did in our experiments is an important step to disaggregate the disparity introduced by the model architecture and hyperparameters, from the disparity introduced by the data imbalance.\n\n[4] Cherepanova, V., Reich, S., Dooley, S., Souri, H., Goldblum, M., & Goldstein, T. (2022). A deep dive into dataset imbalance and bias in face identification. Sixth AAAI/ACM Conference on Artifical Intelligence, Ethis, and Soceity, 2023.\n\n\n**W4: Hyperparameter tuning**\nWe conduct our large scale analysis with handcrafted architectures and the hyperparameters as reported in their respective papers. In addition to this, we also study every model with 9-13 different hyperparameter combinations for each model, to allow for more flexibility in terms of optimizers, face-recognition heads, and learning rates (Section 3.2 Experimental Setup). Our goal is to compare these already strong pipelines with ones that can be discovered automatically using joint NAS+HPO. \n\n**W5&6: Clarity of writing**\nWe greatly appreciate your careful read of our paper. We have updated the manuscript to incorporate this feedback, and we will include these edits in our updated manuscript.\n"}, {"review_id": "XWWmTqt8hh", "reviewer": "Reviewer_mXr9", "summary": "This paper propose a brand new framework (NAS+HPO) to mitigate biases in FR. The discussion is extensive and interesting. But experiments on authoritative face recognition dataset are required, e.g., Ms1m, Glint360 and webface260m.", "questions": "null", "limitations": "null", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 4, "strengths": "a. The presentation is easy to follow.\nb. The discussion is extensive and interesting.\nc. The paper propose a new framework (NAS+HPO jointly) to mitigate biases in FR.", "weaknesses": "The reported FR performance of this method should be also verified in large scale FR datasets, since lots of methods just work in small datasets but always fail in large ones. Some authoritative face recognition datasets are suggested, e.g., Ms1m v3[1], Glint360K[2] and webface260m[3].\n\n[1] Lightweight face recognition challenge.\n[2] Killing Two Birds with One Stone:Efficient and Robust Training of Face Recognition CNNs by Partial FC\n[3] WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We’d like to first thank you for your time and thoughtful feedback on our manuscript. We appreciate that you find our presentation easy to follow, our discussion extensive and interesting. We have conducted new analysis and answer your question below:\n\n**New Results**\n\n**The Effect of Pretraining**\nWe now study the effect of pre-training vs. training from scratch (Figure 2 (a) and (b) in rebuttal PDF) for face-recognition using the Dual Path Network architecture which is the basis for our search space definition. Interestingly, we find that the disparity of the pre-trained model is **much** higher compared to the model trained from scratch.  Moreover, we observe that while the pre-trained model starts strong in terms of accuracy, the model trained from scratch eventually catches up. This opens up an interesting direction of future work on how to effectively exploit pre-trained models for face-recognition systems without increasing bias.\n\n**SMAC Pareto-dominates other NAS Methods**\nWe now study models discovered by other NAS methods (using a limited time-budget for search), and we observe that SMAC (multi-fidelity+Bayesian Optimization) optimizes compute-efficiency and performance.\n\n|                  |   Accuracy |   Rank Disparity |   Disparity |     Ratio |   Rank Ratio |   Error Ratio |\n|:-----------------|-----------:|-----------------:|------------:|----------:|-------------:|--------------:|\n| MO-ASHA_032 |   0.934739 |         0.390588 |   0.0485621 | 0.0533381 |     0.448144 |     0.542336 |\n| NSGA-II_728 |   0.868105 |         0.599085 |   0.0857516 | 0.103913  |     0.490213 |      **0.490651** |\n| SMAC_301    |  **0.963366** |         **0.230327** |   **0.0300871** | **0.0317269** |     **0.367554** |      0.582215 |\n\n\n**Fairness w.r.t. Age Groups on AgeDB**\nWe conducted an analysis on fairness across age groups on AgeDB and find that our models are **pareto-dominant**. \n\n| Dataset  | Model       | Accuracy   | Disparity  |\n|----------|-------------|------------|------------|\n| CelebA   | DPN_CosFace | 64.84      | 0.2824     |\n|          | DPN_MagFace | 60.00      | 0.3129     |\n|          | SMAC_000    | 80.23      | 0.2188     |\n|          | SMAC_010    | **82.35**  | **0.1229** |\n| VGGFace2 | DPN_SGD     | 71.866     | 0.2247     |\n|          | DPN_AdamW   | 61.316     | 0.2114     |\n|          | Rexnet_100  | 59.1833    | 0.2892     |\n|          | SMAC_301    | **81.533** | **0.1883** |\n\n\n**Training on Larger Datasets**\nWe appreciate your point that our learned architectures were not evaluated on additional very large-scale FR datasets. We did not conduct these experiments and leave them for future work since we specifically focused on datasets which have protected attribute labels, unlike Glint360K. During the rebuttal period, we were unable to obtain the WebFace360M dataset given the process and license agreement protocol. Finally, as the ethics reviewer has stated, the use of some of these datasets is controversial, MS-Celeb-1M for example is listed as a [deprecated dataset](https://neurips.cc/public/deprecated-datasets) by NeurIPS itself. \n"}, {"review_id": "IaIcLF0QIX", "reviewer": "Reviewer_3GhH", "summary": "The paper aims at a fairer face recognition model.\n\nFirst, the authors conduct large-scale experiments to show that architectures and hyperparameters matter for fairness (Section 3). Concretely, a wide range of models in different architectures and hyperparameters are evaluated in terms of performance (metric: “Error”) and fairness (metric: Rank Disparity), showing that some models (e.g., DPN) are indeed Pareto-optimal compared to others.\n\nMotivated by this finding, unlike previous bias mitigation strategies based on a fixed neural architecture and a set of hyperparameters, the paper provides a new angle on bias mitigation by searching for fairer neural architectures and hyperparameters.\n\nThe paper designs a search strategy to satisfy three desiderata: (1) both architectures and hyperparameters are optimized, (2) both accuracy and fairness are used as the objective, and (3) the searching process should be efficient. To this end, the paper uses some existing approaches, such as SMAC, Hyperband, and ParEGO.\n\nThe results show that the proposed method is Pareto-optimal compared to existing methods on two face datasets. Furthermore, the experiments show that the proposed method can also generalize to other datasets and protected attributes.", "questions": "1. Can the authors add the fairness-performance results of cross-dataset generalization instead of only showing the performance results?\n2. I appreciate the authors’ efforts in explaining why the proposed method work (L293). However, in terms of neural architecture, is there any pattern that makes some architectures more Pareto-optimal than others? This would be interesting because future works may use such a pattern to manually design a fairer architecture.\n", "limitations": " The authors have adequately addressed the limitations (L333-348). From my perspective, the paper does not have a potential negative societal impact.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The proposed method is well-motivated by the experiments in Section 3 to show that architectures and hyperparameters matter for fairness.\n2. The paper gives a novel angle from architectures and hyperparameters toward bias mitigation.\n3. The paper provides insights into why this method works from the perspective of linear separability of protected attributes (L293).\n4. The experiments are extensive.\n5. In terms of the results, the proposed method is Pareto-optimal, beating existing bias mitigation methods.\n6. The code is provided for better reproducibility.\n7. The paper is well-written and easy to follow.\n", "weaknesses": "## Generalization of Pareto-optimal results to different datasets\nI appreciate the results of cross-dataset generalization. However, Table 2 only shows the performance results. Therefore, whether or not the proposed method is still Pareto-optimal remains unknown.\n\n\n### Minor Comments:\n\nThe plots in Figure 2-4 are in low resolution. I suggest the authors export the plots in PDF, SVG, or EPS formats instead of image formats (e.g., jpeg).\n\nFonts in Table 1 are in a strange aspect ratio. I suggest the authors use the “adjustbox” package to adjust the table size.\n\nL300: probes[1] -> probes [1]\n\nAppendix, L806, L808: broken \\ref link to the figure. “Figure ??” -> Figure 16\n\nAppendix, Caption of Figure 17: “(b) SMAC model second last layer (b) DPN MagFace on the second last layer” -> “(c) SMAC model second last layer (d) DPN MagFace on the second last layer.”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and thoughtful feedback on our manuscript. We appreciate that you find our approach well-motivated, our angle of architectures and hyperparameters novel, our experiments extensive, our method reproducible and the paper well-written and easy to follow. We address each of your points below:\n\n**Weakness & Q1: Fairness performance on cross-dataset generalization** \nThank you for raising this point, and we have updated our manuscript to include Rank Disparity in Table 2. We replicate that table here below. We note that the only dataset with usable protected attribute labels is AgeDB so thus, we present that result here. We divide the ages into groups of 0-25 yrs, 25-50yrs, 50-75yrs and 75-100yrs. Further, we report the maximum disparity amongst these groups. We note that the SMAC models are Pareto-*dominant* here showing the lowest error and lowest rank disparity. \n| Dataset  | Model       | Accuracy   | Disparity  |\n|----------|-------------|------------|------------|\n| CelebA   | DPN_CosFace | 64.84      | 0.2824     |\n|          | DPN_MagFace | 60.00      | 0.3129     |\n|          | SMAC_000    | 80.23      | 0.2188     |\n|          | SMAC_010    | **82.35**  | **0.1229** |\n| VGGFace2 | DPN_SGD     | 71.866     | 0.2247     |\n|          | DPN_AdamW   | 61.316     | 0.2114     |\n|          | Rexnet_100  | 59.1833    | 0.2892     |\n|          | SMAC_301    | **81.533** | **0.1883** |\n\n**Q2: Why are these architectures more fair?**\nThank you for your question. We precisely search for a recurring Dual Path Network block in-terms of architecture. The handcrafted DPN block contains a Conv3x3Bn (Conv 3x3 followed by batchnorm), BnConv5x5 (batch norm followed by 5x5 convolution) and BnConv3x3 ( batch norm followed by 5x5 convolution). Amongst the architectures, we find a strong preference for the BnConv3x3 operation (every architecture containing at least one or more of such operations). Furthermore, in terms of the optimal face recognition head, we surprisingly find a strong preference for “CosFace” instead of “MagFace” and “ArcFace”. We find that “ArcFace” has the least preference during search. Moreover, we also discover that the SGD optimizer often with high learning rates > 0.1 is often preferred in comparison to AdamW and Adam optimizers from our search space. \n\nThe proposed multi-objective neural architecture search and HPO simultaneously optimize two objectives, firstly the accuracy and secondly the fairness metric (e.g. rank disparity). Hence, we bias the search toward models which do not exploit the protected attribute (e.g. gender) to make classifications. We hypothesize that the SMAC models learn to use more fine grained facial features to distinguish faces instead of exploiting obvious coarse features like protected attributes (gender, race, age). We leave a more detailed analysis of the properties of learned features for future work.\n"}]

Author rebuttal:
We first thank all the reviewers for their insightful feedback and suggestions. Our work shows that bias in face recognition systems is actually inherent to their architectures and hyperparameters, and we can design fairer systems by searching for fair architectures, in fact significantly surpassing previous approaches. We appreciate that the reviewers find our perspective on bias mitigation interesting and fresh (**jfj2**, **kze1**, **Nw7v**, **3GhH**), our presentation clear and well-motivated (**jfj2**, **Nw7v**, **mXr9**, **3GhH**) and our experiments and analysis thorough and extensive ( **mXr9**, **3GhH**, **Nw7v**). Following suggestions made by the reviewers, we conducted further analyses and evaluations, some of which we highlight below:

**New Results**

1. We now conduct an analysis on fairness across age groups on AgeDB and find that our models are **Pareto-dominant**. 

| Dataset  | Model       | Accuracy   | Disparity  |
|----------|-------------|------------|------------|
| CelebA   | DPN_CosFace | 64.84      | 0.2824     |
|          | DPN_MagFace | 60.00      | 0.3129     |
|          | SMAC_000    | 80.23      | 0.2188     |
|          | SMAC_010    | **82.35**  | **0.1229** |
| VGGFace2 | DPN_SGD     | 71.866     | 0.2247     |
|          | DPN_AdamW   | 61.316     | 0.2114     |
|          | Rexnet_100  | 59.1833    | 0.2892     |
|          | SMAC_301    | **81.533** | **0.1883** |

2. We now study models discovered by other NAS methods (using a limited time-budget for search), and we observe that SMAC (multi-fidelity+Bayesian Optimization) optimizes compute-efficiency and performance.

|                  |   Accuracy |   Rank Disparity |   Disparity |     Ratio |   Rank Ratio |   Error Ratio |
|:-----------------|-----------:|-----------------:|------------:|----------:|-------------:|--------------:|
| MO-ASHA_032 |   0.934739 |         0.390588 |   0.0485621 | 0.0533381 |     0.448144 |     0.542336 |
| NSGA-II_728 |   0.868105 |         0.599085 |   0.0857516 | 0.103913  |     0.490213 |      **0.490651** |
| SMAC_301    |  **0.963366** |         **0.230327** |   **0.0300871** | **0.0317269** |     **0.367554** |      0.582215 |


3. We now study the effect of pre-training vs. training from scratch (Figure 2 (a) and (b) in rebuttal PDF) for face-recognition using the Dual Path Network architecture which is the basis for our search space definition. Interestingly, we find that the disparity of the pre-trained model is **much** higher compared to the model trained from scratch. Moreover, we observe that while the pre-trained model starts strong in terms of accuracy, the model trained from scratch eventually catches up. This opens up an interesting direction of future work on how to effectively exploit pre-trained models for face-recognition systems without increasing bias.

**Ethical Concerns**
We strongly believe that our findings need to be placed into the larger sociotechnical context of facial recognition. The impacts of facial recognition technologies on individuals are well-documented, and our work considers a new way to reduce harms caused by disparities in these systems. We adhered to the [NeurIPS deprecated dataset guidelines](https://neurips.cc/public/deprecated-datasets) for our choice of datasets. MS-Celeb-1M and MegaFace are two datasets widely used by the face recognition community, even today, which we omitted from our experiments due to ethical issues. We have updated our manuscript to reflect these points and to further highlight the representational issues with CelebA as pointed out by reviewers.


Meta-review: This paper addresses bias mitigation with Neural Architecture Search (NAS) and Hyperparameter Optimization (i.e. finding fairer architectures in contrast to finding a good performing architecture first and then applying bias mitigation strategies). The experiments focus on bias mitigation in face recognition and they are conducted on two popular benchmarks: CelebA and VGGFace2. 

All the reviewers agreed on the novelty of the approach with respect to previous work on bias mitigation, and valued the extended experiments and the relevance of the results. The reviewers also highlighted some weaknesses and made some questions, which were satisfactorily addressed during the authors feedback (the authors provided new results, clarifications, and further discussions). 

One reviewer flagged the paper for ethical revision, and the paper was reviewed by two ethics reviewers. One of the ethics reviewers found no ethical issues, but recommended extending the discussion on potential social and ethical impacts. The other ethics reviewer considered the paper to have ethical issues because of the use of the CelebA dataset. This dataset has representation bias and was collected without informed consent. The ethics reviewer recommends adding a discussion on the representation bias issue to the paper (the informed consent issue was already discussed in the ethics statement). The authors have already committed to follow these recommendations for the final version of the paper.

The paper was further discussed with the NeurIPS Ethics Chairs during the AC-SAC discussion period. During this discussion the Ethics Chairs request that the authors address these specific points for follow-up in the camera-ready manuscript:
1) To discuss if their test datasets (in Table 2, etc.) have similar ethical issues brought up
2) To discuss the limitations of their work, particularly as it might encourage techno-solutionism in the form of removing the bias and proceeding as usual without deeper considerations.
Decision: Accept (oral)