SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Samuel Dooley*  
University of Maryland, Abacus.AI  
samuel@abacus.ai
John P. Dickerson  
University of Maryland, Arthur AI  
johnd@umd.edu
Frank Hutter  
University of Freiburg  
fh@cs.uni-freiburg.de
Rhea Sanjay Sukthanker*  
University of Freiburg  
sukthank@cs.uni-freiburg.de
Colin White  
Caltech, Abacus.AI  
crwhite@caltech.edu
Micah Goldblum  
New York University  
goldblum@nyu.edu
Face recognition systems are widely deployed in safety-critical applications, including law enforcement, yet they exhibit bias across a range of socio-demographic dimensions, such as gender and race. Conventional wisdom dictates that model biases arise from biased training data. As a consequence, previous works on bias mitigation largely focused on pre-processing the training data, adding penalties to prevent bias from effecting the model during training, or post-processing predictions to debias them, yet these approaches have shown limited success on hard problems such as face recognition. In our work, we discover that biases are actually inherent to neural network architectures themselves. Following this reframing, we conduct the first neural architecture search for fairness, jointly with a search for hyperparameters. Our search outputs a suite of models which Pareto-dominate all other high-performance architectures and existing bias mitigation methods in terms of accuracy and fairness, often by large margins, on the two most widely used datasets for face identification, CelebA and VGGFace2. Furthermore, these models generalize to other datasets and sensitive attributes. We release our code, models and raw data files at https://github.com/dooleys/FR-NAS.
Machine learning is applied to a wide variety of socially-consequential domains, e.g., credit scoring, fraud detection, hiring decisions, criminal recidivism, loan repayment, and face recognition [78, 81, 61, 3], with many of these applications significantly impacting people's lives, often in discriminatory ways [5, 55, 114]. Dozens of formal definitions of fairness have been proposed [80], and many algorithmic techniques have been developed for debiasing according to these definitions [106]. Existing debiasing algorithms broadly fit into three (or arguably four [96]) categories: pre-processing [e.g., 32, 93, 89, 110], in-processing [e.g., 123, 124, 25, 35, 83, 110, 73, 79, 24, 59], or post-processing [e.g., 44, 114].
* indicates equal contribution
Figure 1: Overview of our methodology.
Conventional wisdom is that in order to effectively mitigate bias, we should start by selecting a model architecture and set of hyperparameters which are optimal in terms of accuracy and then apply a mitigation strategy to reduce bias. This strategy has yielded little success in hard problems such as face recognition [14]. Moreover, even randomly initialized face recognition models exhibit bias and in the same ways and extents as trained models, indicating that these biases are baked in to the architectures already [13]. While existing methods for debiasing machine learning systems use a fixed neural architecture and hyperparameter setting, we instead, ask a fundamental question which has received little attention: Does model bias arise from the architecture and hyperparameters? Following an affirmative answer to this question, we exploit advances in neural architecture search (NAS) [30] and hyperparameter optimization (HPO) [33] to search for inherently fair models.
We demonstrate our results on face identification systems where pre-, post-, and in-processing techniques have fallen short of debiasing face recognition systems. Training fair models in this setting demands addressing several technical challenges [14]. Face identification is a type of face recognition deployed worldwide by government agencies for tasks including surveillance, employment, and housing decisions. Face recognition systems exhibit disparity in accuracy based on race and gender [37, 92, 91, 61]. For example, some face recognition models are 10 to 100 times more likely to give false positives for Black or Asian people, compared to white people [2]. This bias has already led to multiple false arrests and jail time for innocent Black men in the USA [48].
In this work, we begin by conducting the first large-scale analysis of the impact of architectures and hyperparameters on bias. We train a diverse set of 29 architectures, ranging from ResNets [47] to vision transformers [28, 68] to Gluon Inception V3 [103] to MobileNetV3 [50] on the two most widely used datasets in face identification that have socio-demographic labels: CelebA [69] and VGGFace2 [8]. In doing so, we discover that architectures and hyperparameters have a significant impact on fairness, across fairness definitions.
Motivated by this discovery, we design architectures that are simultaneously fair and accurate. To this end, we initiate the study of NAS for fairness by conducting the first use of NAS+HPO to jointly optimize fairness and accuracy. We construct a search space informed by the highest-performing architecture from our large-scale analysis, and we adapt the existing Sequential Model-based Algorithm Configuration method (SMAC) [66] for multi-objective architecture and hyperparameter search. We discover a Pareto frontier of face recognition models that outperform existing state-of-the-art models on both test accuracy and multiple fairness metrics, often by large margins. An outline of our methodology can be found in Figure 1.
We summarize our primary contributions below:
- By conducting an exhaustive evaluation of architectures and hyperparameters, we uncover their strong influence on fairness. Bias is inherent to a model's inductive bias, leading to a substantial difference in fairness across different architectures. We conclude that the implicit convention of choosing standard architectures designed for high accuracy is a losing strategy for fairness.
- Inspired by these findings, we propose a new way to mitigate biases. We build an architecture and hyperparameter search space, and we apply existing tools from NAS and HPO to automatically design a fair face recognition system.
- Our approach finds architectures which are Pareto-optimal on a variety of fairness metrics on both CelebA and VGGFace2. Moreover, our approach is Pareto-optimal compared to other previous bias mitigation techniques, finding the fairest model.
- The architectures we synthesize via NAS and HPO generalize to other datasets and sensitive attributes. Notably, these architectures also reduce the linear separability of protected attributes, indicating their effectiveness in mitigating bias across different contexts.
We release our code and raw results at https://github.com/dooleys/FR-NAS, so that users can easily adapt our approach to any bias metric or dataset.
Face Identification. Face recognition tasks can be broadly categorized into two distinct categories: verification and identification. Our specific focus lies in face identification tasks which ask whether a given person in a source image appears within a gallery composed of many target identities and their associated images; this is a one-to-many comparison. Novel techniques in face recognition tasks, such as ArcFace [108], CosFace [23], and MagFace [75], use deep networks (often called the backbone) to extract feature representations of faces and then compare those to match individuals (with mechanisms called the head). Generally, backbones take the form of image feature extractors and heads resemble MLPs with specialized loss functions. Often, the term "head" refers to both the last layer of the network and the loss function. Our analysis primarily centers around the face identification task, and we focus our evaluation on examining how close images of similar identities are in the feature space of trained models, since the technology relies on this feature representation to differentiate individuals. An overview of these topics can be found in Wang and Deng [109].
Bias Mitigation in Face Recognition. The existence of differential performance of face recognition on population groups and subgroups has been explored in a variety of settings. Earlier work [e.g., 57, 82] focuses on single-demographic effects (specifically, race and gender) in pre-deep-learning face detection and recognition. Buolamwini and Gebru [5] uncover unequal performance at the phenotypic subgroup level in, specifically, a gender classification task powered by commercial systems. Raji and Buolamwini [90] provide a follow-up analysis - exploring the impact of the public disclosures of Buolamwini and Gebru [5] - where they discovered that named companies (IBM, Microsoft, and Megvii) updated their APIs within a year to address some concerns that had surfaced. Further research continues to show that commercial face recognition systems still have socio-demographic disparities in many complex and pernicious ways [29, 27, 54, 54, 26].
Facial recognition is a large and complex space with many different individual technologies, some with bias mitigation strategies designed just for them [63, 118]. The main bias mitigation strategies for facial identification are described in Section 4.2.
Neural Architecture Search (NAS) and Hyperparameter Optimization (HPO). Deep learning derives its success from the manually designed feature extractors which automate the feature engineering process. Neural Architecture Search (NAS) [30, 116], on the other hand, aims at automating the very design of network architectures for a task at hand. NAS can be seen as a subset of HPO [33], which refers to the automated search for optimal hyperparameters, such as learning rate, batch size, dropout, loss function, optimizer, and architectural choices. Rapid and extensive research on NAS for image classification and object detection has been witnessed as of late [67, 125, 121, 88, 6]. Deploying NAS techniques in face recognition systems has also seen a growing interest [129, 113]. For example, reinforcement learning-based NAS strategies [121] and one-shot NAS methods [113] have been deployed to search for an efficient architecture for face recognition with low error. However, in a majority of these methods, the training hyperparameters for the architectures are fixed. We observe that this practice should be reconsidered in order to obtain the fairest possible face recognition systems. Moreover, one-shot NAS methods have also been applied for multi-objective optimization [39, 7], e.g., optimizing accuracy and parameter size. However, none of these methods can be applied for a joint architecture and hyperparameter search, and none of them have been used to optimize fairness.
For the case of tabular datasets, a few works have applied hyperparameter optimization to mitigate bias in models. Perrone et al. [87] introduced a Bayesian optimization framework to optimize accuracy of models while satisfying a bias constraint. Schmucker et al. [97] and Cruz et al. [17] extended Hyperband [64] to the multi-objective setting and showed its applications to fairness. Lin et al. [65] proposed de-biasing face recognition models through model pruning. However, they only considered two architectures and just one set of fixed hyperparameters. To the best of our knowledge,
no prior work uses any AutoML technique (NAS, HPO, or joint NAS and HPO) to design fair face recognition models, and no prior work uses NAS to design fair models for any application.
3 Are Architectures and Hyperparameters Important for Fairness?
In this section, we study the question "Are architectures and hyperparameters important for fairness?" and report an extensive exploration of the effect of model architectures and hyperparameters.
Experimental Setup. We train and evaluate each model configuration on a gender-balanced subset of the two most popular face identification datasets: CelebA and VGGFace2. CelebA [69] is a large-scale face attributes dataset with more than 200K celebrity images and a total of 10177 gender-labeled identities. VGGFace2 [8] is a much larger dataset designed specifically for face identification and comprises over 3.1 million images and a total of 9131 gender-labeled identities. While this work analyzes phenotypic metadata (perceived gender), the reader should not interpret our findings absent a social lens of what these demographic groups mean inside society. We guide the reader to Hamidi et al. [40] and Keyes [56] for a look at these concepts for gender.
To study the importance of architectures and hyperparameters for fairness, we use the following training pipeline - ultimately conducting 355 training runs with different combinations of 29 architectures from the Pytorch Image Model (timm) database [117] and hyperparameters. For each model, we use the default learning rate and optimizer that was published with that model. We then train the model with these hyperparameters for each of three heads, ArcFace [108], CosFace [23], and MagFace [75]. Next, we use the model's default learning rate with both AdamW [70] and SGD optimizers (again with each head choice). Finally, we also train with AdamW and SGD with unified learning rates (SGD with learning_rate=0.1 and AdamW with learning_rate=0.001). In total, we thus evaluate a single architecture between 9 and 13 times (9 times if the default optimizer and learning rates are the same as the standardized, and 13 times otherwise). All other hyperparameters are held constant for training of the model.
Evaluation procedure. As is commonplace in face identification tasks [12, 13], we evaluate the performance of the learned representations. Recall that face recognition models usually learn representations with an image backbone and then learn a mapping from those representations onto identities of individuals with the head of the model. We pass each test image through a trained model and save the learned representation. To compute the representation error (which we will henceforth simply refer to as Error), we merely ask, for a given probe image/identity, whether the closest image in feature space is not of the same person based on  $l_{2}$  distance. We split each dataset into train, validation, and test sets. We conduct our search for novel architectures using the train and validation splits, and then show the improvement of our model on the test set.
The most widely used fairness metric in face identification is rank disparity, which is explored in the NIST FRVT [38]. To compute the rank of a given image/identity, we ask how many images of a different identity are closer to the image in feature space. We define this index as the rank of a given image under consideration. Thus,  $\operatorname{Rank}(\mathrm{image}) = 0$  if and only if  $\operatorname{Error}(\mathrm{image}) = 0$ ;  $\operatorname{Rank}(\mathrm{image}) > 0$  if and only if  $\operatorname{Error}(\mathrm{image}) = 1$ . We examine the rank disparity: the absolute difference of the average ranks for each perceived gender in a dataset  $\mathcal{D}$ :
$$
\left| \frac {1}{\left| \mathcal {D} _ {\text {m a l e}} \right|} \sum_ {x \in \mathcal {D} _ {\text {m a l e}}} \operatorname {R a n k} (x) - \frac {1}{\left| \mathcal {D} _ {\text {f e m a l e}} \right|} \sum_ {x \in \mathcal {D} _ {\text {f e m a l e}}} \operatorname {R a n k} (x) \right|. \tag {1}
$$
We focus on rank disparity throughout the main body of this paper as it is the most widely used in face identification, but we explore other forms of fairness metrics in face recognition in Appendix C.4.
Results and Discussion. By plotting the performance of each training run on the validation set with the error on the  $x$ -axis and rank disparity on the  $y$ -axis in Figure 2, we can easily conclude two main points. First, optimizing for error does not always optimize for fairness, and second, different architectures have different fairness properties. We also find the DPN architecture has the lowest error and is Pareto-optimal on both datasets; hence, we use that architecture to design our search space in Section 4.
Figure 2: (Left) CelebA (Right) VGGFace2. Error-Rank Disparity Pareto front of the architectures with lowest error  $(< 0.3)$ . Models in the lower left corner are better. The Pareto front is denoted with a dashed line. Other points are architecture and hyperparameter combinations which are not Pareto-optimal.
We note that in general there is a low correlation between error and rank disparity (e.g., for models with error  $< 0.3$ ,  $\rho = .113$  for CelebA and  $\rho = .291$  for VGGFace2). However, there are differences between the two datasets at the most extreme low errors. First, for VGGFace2, the baseline models already have very low error, with there being 10 models with error  $< 0.05$ ; CelebA only has three such models. Additionally, models with low error also have low rank disparity on VGGFace2 but this is not the case for CelebA. This can be seen by looking at the Pareto curves in Figure 2.
The Pareto-optimal models also differ across datasets: on CelebA, they are versions of DPN, TNT, ReXNet, VovNet, and ResNets, whereas on VGGFace2 they are DPN and ReXNet. Finally, we note that different architectures exhibit different optimal hyperparameters. For example, on CelebA, for the Xception65 architecture finds the combinations of (SGD, ArcFace) and (AdamW, ArcFace) as Pareto-optimal, whereas the Inception-ResNet architecture finds the combinations (SGD, MagFace) and (SGD, CosFace) Pareto-optimal.
4 Neural Architecture Search for Bias Mitigation
Inspired by our findings on the importance of architecture and hyperparameters for fairness in Section 3, we now initiate the first joint study of NAS for fairness in face recognition, also simultaneously optimizing hyperparameters. We start by describing our search space and search strategy. We then compare the results of our NAS+HPO-based bias mitigation strategy against other popular face recognition bias mitigation strategies. We conclude that our strategy indeed discovers simultaneously accurate and fair architectures.
4.1 Search Space Design and Search Strategy
We design our search space based on our analysis in Section 3, specifically around the Dual Path Networks[10] architecture which has the lowest error and is Pareto-optimal on both datasets, yielding the best trade-off between rank disparity and accuracy as seen in Figure 2.
Hyperparameter Search Space Design. We optimize two categorical hyperparameters (the architecture head/loss and the optimizer) and one continuous one (the learning rate). The learning rate's range is conditional on the choice of optimizer; the exact ranges are listed in Table 6 in the appendix.
Architecture Search Space Design. Dual Path Networks [10] for image classification share common features (like ResNets [46]) while possessing the flexibility to explore new features [52] through a dual path architecture. We replace the repeating 1x1_conv-3x3_conv-1x1_conv block with a simple recurring searchable block. Furthermore, we stack multiple such searched blocks to closely follow the architecture of Dual Path Networks. We have nine possible choices for each of the three operations in the DPN block, each of which we give a number 0 through 8. The choices include a vanilla convolution, a convolution with pre-normalization and a convolution with post-normalization, each of them paired with kernel sizes  $1 \times 1$ ,  $3 \times 3$ , or  $5 \times 5$  (see Appendix C.2 for full details). We thus have 729 possible architectures (in addition to an infinite number of hyperparameter configurations). We denote each of these architectures by XYZ where  $X, Y, Z \in \{0, \dots, 8\}$ ; e.g., architecture 180 represents the architecture which has operation 1, followed by operation 8, followed by operation 0.
Figure 3: Pareto front of the models discovered by SMAC and the rank-1 models from timm for the  $(a)$  validation and  $(b)$  test sets on CelebA. Each point corresponds to the mean and standard error of an architecture after training for 3 seeds. The SMAC models Pareto-dominate the top performing timm models ( $Error < 0.1$ ).
Search strategy. To navigate this search space we have the following desiderata:
- Joint NAS+HPO. Since there are interaction effects between architectures and hyperparameters, we require an approach that can jointly optimize both of these.
- Multi-objective optimization. We want to explore the trade-off between the accuracy of the face recognition system and the fairness objective of choice, so our joint NAS+HPO algorithm needs to supports multi-objective optimization [84, 21, 71].
- Efficiency. A single function evaluation for our problem corresponds to training a deep neural network on a given dataset. As this can be quite expensive on large datasets, we would like to use cheaper approximations with multi-fidelity optimization techniques [98, 64, 31].
To satisfy these desiderata, we employ the multi-fidelity Bayesian optimization method SMAC3 [66] (using the SMAC4MF facade), casting architectural choices as additional hyperparameters. We choose Hyperband [64] for cheaper approximations with the initial and maximum fidelities set to 25 and 100 epochs, respectively, and  $\eta = 2$ . Every architecture-hyperparameter configuration evaluation is trained using the same training pipeline as in Section 3. For multi-objective optimization, we use the ParEGO [21] algorithm with  $\rho$  set to 0.05.
4.2 Empirical Evaluation
We now report the results of our NAS+HPO-based bias mitigation strategy. First, we discuss the models found with our approach, and then we compare their performance to other mitigation baselines.
Setup. We conducted one NAS+HPO search for each dataset by searching on the train and validation sets. After running these searches, we identified three new candidate architectures for CelebA (SMAC_000, SMAC_010, and SMAC_680), and one candidate for VGGFace2 (SMAC_301) where the naming convention follows that described in Section 4.1. We then retrained each of these models and those high performing models from Section 3 for three seeds to study the robustness of error and disparity for these models; we evaluated their performance on the validation and test sets for each dataset, where we follow the evaluation scheme of Section 3.
Comparison against timm models. On CelebA (Figure 3), our models Pareto-dominate all of the timm models with nontrivial accuracy on the validation set. On the test set, our models still Pareto-dominate all highly competitive models (with Error  $< 0.1$ ), but one of the original configurations (DPN with Magface) also becomes Pareto-optimal. However, the error of this architecture is 0.13, which is significantly higher than our models (0.03-0.04). Also, some models (e.g., VoVNet and DenseNet) show very large standard errors across seeds. Hence, it becomes important to also study
Figure 4: Pareto front of the models discovered by SMAC and the rank-1 models from timm for the  $(a)$  validation and  $(b)$  test sets on VGGFace2. Each point corresponds to the mean and standard error of an architecture after training for 3 seeds. The SMAC models are Pareto-optimal the top performing timm models (Error  $< 0.1$ ).
the robustness of models across seeds along with the accuracy and disparity Pareto front. Finally, on VGGFace2 (Figure 4), our models are also Pareto-optimal for both the validation and test sets.
Novel Architectures Outperform the State of the Art. Comparing the results of our automatically-found models to the current state of the art baseline ArcFace [23] in terms of error demonstrates that our strategy clearly establishes a new state of the art. While ArcFace [23] achieves an error of  $4.35\%$  with our training pipeline on CelebA, our best-performing novel architecture achieves a much lower error of  $3.10\%$ . Similarly, the current VGGFace2 state of the art baseline [112] achieves an error of  $4.5\%$ , whereas our best performing novel architecture achieves a much lower error of  $3.66\%$ .