SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Novel Architectures Pareto-Dominate other Bias Mitigation Strategies. There are three common pre-, post-, and in-processing bias mitigation strategies in face identification. First, Chang et al. [9] demonstrated that randomly flipping labels in the training data of the subgroup with superior accuracy can yield fairer systems; we call this technique Flipped. Next, Wang and Deng [110] use different angular margins during training and therefore promote better feature discrimination for the minority class; we call this technique Angular. Finally, Morales et al. [76] introduced SensitiveNets which is a sensitive information removal network trained on top of a pre-trained feature extractor with an adversarial sensitive regularizer. While other bias mitigation techniques exist in face recognition, these three are the most used and pertinent to face identification. See Cherepanova et al. [14] for an overview of the technical challenges of bias mitigation in face recognition. We take the top performing, Pareto-optimal timm models from the previous section and apply the three bias mitigation techniques (Flipped, Angular, and SensitiveNets). We also apply these same techniques to the novel architectures that we found. The results in Table 1 show that the novel architectures from our NAS+HPO-based mitigation strategy Pareto-dominate the bias-mitigated models. In VGGFace2, the SMAC_301 model achieves the best performance, both in terms of error and fairness, compared to the bias-mitigated models. On CelebA, the same is true for the SMAC_680 model.
NAS+HPO-Based Bias Mitigation can be Combined with other Bias Mitigation Strategies. Additionally, we combined the three other bias mitigation methods with the SMAC models that resulted from our NAS+HPO-based bias mitigation strategy. More precisely, we first conducted our NAS+HPO approach and then applied the Flipped, Angular, and SensitiveNets approach afterwards. On both datasets, the resulting models continue to Pareto-dominate the other bias mitigation strategies used by themselves and ultimately yield the model with the lowest rank disparity of all the models (0.18 on VGGFace2 and 0.03 on CelebA). In particular, the bias improvement of SMAC_000+Flipped model is notable, achieving a score of 0.03 whereas the lowest rank disparity of any model from Figure 3 is 2.63, a  $98.9\%$  improvement. In Appendix C.6, we demonstrate that this result is robust to the fairness metric — specifically our bias mitigation strategy Pareto-dominates the other approaches on all five fairness metrics.
Table 1: Comparison of bias mitigation techniques where the SMAC models were found with our NAS+HPO bias mitigation technique and the other three techniques are standard in facial recognition: Flipped [9], Angular [76], and SensitiveNets [110]. Items in bold are Pareto-optimal. The values show (Error;Rank Disparity). Other metrics are reported in Appendix C.6 and Table 8.
Model	Trained on VGGFace2	SensitiveNets	Model	Trained on CelebA
Baseline	Flipped	Angular	SensitiveNets	Baseline	Flipped	Angular	SensitiveNets
SMAC_301	(3.66;0.23)	(4.95;0.18)	(4.14;0.25)	(6.20;0.41)	SMAC_000	(3.25;2.18)	(5.20;0.03)	(3.45;2.28)	(3.45;2.18)
DPN	(3.56;0.27)	(5.87;0.32)	(6.06;0.36)	(4.76;0.34)	SMAC_010	(4.14;2.27)	(12.27;5.46)	(4.50;2.50)	(3.99;2.12)
ReXNet	(4.09;0.27)	(5.73;0.45)	(5.47;0.26)	(4.75;0.25)	SMAC_680	(3.22;1.96)	(12.42;4.50)	(3.80;4.16)	(3.29;2.09)
Swin	(5.47;0.38)	(5.75;0.44)	(5.23;0.25)	(5.03;0.30)	ArcFace	(11.30;4.6)	(13.56;2.70)	(9.90;5.60)	(9.10;3.00)
Table 2: We transfer the evaluation of top performing models on VGGFace2 and CelebA onto six other common face recognition datasets: LFW [53], CFP_FF [100], CFP_FP [100], AgeDB [77], CALFW [128], CPLPW [127]. The novel architectures found with our bias mitigation strategy significantly outperform other models in terms of accuracy. Refer Table 9 for the complete results.
Architecture (trained on VGGFace2)	LFW	CFP_FF	CFP_FP	AgeDB	CALFW	CPLFW
Rexnet_200	82.60	80.91	65.51	59.18	68.23	62.15
DPN_SGD	93.0	91.81	78.96	71.87	78.27	72.97
DPN_AdamW	78.66	77.17	64.35	61.32	64.78	60.30
SMAC_301	96.63	95.10	86.63	79.97	86.07	81.43
Architecture (trained on CelebA)	LFW	CFP_FF	CFP_FP	AgeDB	CALFW	CPLFW
DPN_CosFace	87.78	90.73	69.97	65.55	75.50	62.77
DPN_MagFace	91.13	92.16	70.58	68.17	76.98	60.80
SMAC_000	94.98	95.60	74.24	80.23	84.73	64.22
SMAC_010	94.30	94.63	73.83	80.37	84.73	65.48
SMAC_680	94.16	95.68	72.67	79.88	84.78	63.96
Novel Architectures Generalize to Other Datasets. We observed that when transferring our novel architectures to other facial recognition datasets that focus on fairness-related aspects, our architectures consistently outperform other existing architectures by a significant margin. We take the state-of-the-art models from our experiments and test the weights from training on CelebA and VGGFace2 on different datasets which the models did not see during training. Specifically, we transfer the evaluation of the trained model weights from CelebA and VGGFace2 onto the following datasets: LFW [53], CFP_FF [100], CFP_FP [100], AgeDB [77], CALFW [128], CPLFW [127]. Table 2 demonstrates that our approach consistently achieves the highest performance among various architectures when transferred to other datasets. This finding indicates that our approach exhibits exceptional generalizability compared to state-of-the-art face recognition models in terms of transfer learning to diverse datasets.
Novel Architectures Generalize to Other Sensitive Attributes. The superiority of our novel architectures even goes beyond accuracy-related metrics when transferring to other datasets — our novel architectures have superior fairness properties compared to the existing architectures even on datasets which have completely different protected attributes than were used in the architecture search. Specifically, to inspect the generalizability of our approach to other protected attributes, we transferred our models pre-trained on CelebA and VGGFace2 (which have a gender presentation category) to the RFW dataset [111] which includes a protected attribute for race and the AgeDB dataset [77] which includes a protected attribute for age. The results detailed in Appendix C.7 show that our novel architectures always outperforms the existing architectures, across all five fairness metrics studied in this work on both datasets.
Novel Architectures Have Less Linear-Separability of Protected Attributes. Our comprehensive evaluation of multiple face recognition benchmarks establishes the importance of architectures for fairness in face-recognition. However, it is natural to wonder: "What makes the discovered architectures fair in the first place? To answer this question, we use linear probing to dissect the
Table 3: Linear Probes on architectures. Lower gender classification accuracy is better
Architecture (trained on VGGFace2)	Accuracy on Layer N ↓	Accuracy on Layer N-1 ↓
DPN_MagFace_SGD	86.042%	95.461%
DPN_CosFace_SGD	90.719%	93.787%
DPN_CosFace_AdamW	87.385%	94.444%
SMAC_301	69.980%	68.240%
intermediate features of our searched architectures and DPNs, which our search space is based upon. Intuitively, given that our networks are trained only on the task of face recognition, we do not want the intermediate feature representations to implicitly exploit knowledge about protected attributes (e.g., gender). To this end we insert linear probes[1] at the last two layers of different Pareto-optimal DPNs and the model obtained by our NAS+HPO-based bias mitigation. Specifically, we train an MLP on the feature representations extracted from the pre-trained models and the protected attributes as labels and compute the gender-classification accuracy on a held-out set. We consider only the last two layers, so  $k$  assumes the values of  $N$  and  $N - 1$  with  $N$  being the number of layers in DPNs (and the searched models). We represent the classification probabilities for the genders by  $g p_{k} = \text{softmax}(W_{k} + b)$ , where  $W_{k}$  is the weight matrix of the  $k$ -th layer and  $b$  is a bias. We provide the classification accuracies for the different pre-trained models on VGGFace2 in Table 3. This demonstrates that, as desired, our searched architectures maintain a lower classification accuracy for the protected attribute. In line with this observation, in the t-SNE plots in Figure 18 in the appendix, the DPN displays a higher degree of separability of features.
Comparison between different NAS+HPO techniques We also perform an ablation across different multi-objective NAS+HPO techniques. Specifically we compare the architecture derived by SMAC with architectures derived by the evolutionary multi-objective optimization algorithm NSGA-II [22] and multi-objective asynchronous successive halving (MO-ASHA) [98]. We observe that the architecture derived by SMAC Pareto-dominates the other NAS methods in terms of accuracy and diverse fairness metrics Table 4. We use the implementation of NSGA-II and MO-ASHA from the syne-tune library [95] to perform an ablation across different baselines.
Table 4: Comparison between architectures derived by SMAC and other NAS baselines
NAS Method	Accuracy ↑	Rank Disparity↓	Disparity↓	Ratio↓	Rank Ratio ↓	Error Ratio↓
MO-ASHA_108	95.212	0.408	0.038	0.041	0.470	0.572
NSGA-II_728	86.811	0.599	0.086	0.104	0.490	0.491
SMAC_301	96.337	0.230	0.030	0.032	0.367	0.582
5 Conclusion, Future Work and Limitations
Conclusion. Our approach studies a novel direction for bias mitigation by altering network topology instead of loss functions or model parameters. We conduct the first large-scale analysis of the relationship among hyperparameters and architectural properties, and accuracy, bias, and disparity in predictions across large-scale datasets like CelebA and VGGFace2. Our bias mitigation technique centering around Neural Architecture Search and Hyperparameter Optimization is very competitive compared to other common bias mitigation techniques in facial recognition.
Our findings present a paradigm shift by challenging conventional practices and suggesting that seeking a fairer architecture through search is more advantageous than attempting to rectify an unfair one through adjustments. The architectures obtained by our joint NAS and HPO generalize across different face recognition benchmarks, different protected attributes, and exhibit lower linear-separability of protected attributes.
Future Work. Since our work lays the foundation for studying NAS+HPO for fairness, it opens up a plethora of opportunities for future work. We expect the future work in this direction to focus on
studying different multi-objective algorithms [34, 60] and NAS techniques [67, 125, 115] to search for inherently fairer models. Further, it would be interesting to study how the properties of the architectures discovered translate across different demographics and populations. Another potential avenue for future work is incorporating priors and beliefs about fairness in the society from experts to further improve and aid NAS+HPO methods for fairness. Given the societal importance, it would be interesting to study how our findings translate to real-life face recognition systems under deployment. Finally, it would also be interesting to study the degree to which NAS+HPO can serve as a general bias mitigation strategy beyond the case of facial recognition.
Limitations. While our work is a step forward in both studying the relationship among architectures, hyperparameters, and bias, and in using NAS techniques to mitigate bias in face recognition models, there are important limitations to keep in mind. Since we only studied a few datasets, our results may not generalize to other datasets and fairness metrics. Second, since face recognition applications span government surveillance [49], target identification from drones [72], and identification in personal photo repositories [36], our findings need to be studied thoroughly across different demographics before they could be deployed in real-life face recognition systems. Furthermore, it is important to consider how the mathematical notions of fairness used in research translate to those actually impacted [94], which is a broad concept without a concise definition. Before deploying a particular system that is meant to improve fairness in a real-life application, we should always critically ask ourselves whether doing so would indeed prove beneficial to those impacted by the given sociotechnical system under consideration or whether it falls into one of the traps described by Selbst et al. [99]. Additionally, work in bias mitigation, writ-large and including our work, can be certainly encourage techno-solutionism which views the reduction of statistical bias from algorithms as a justification for their deployment, use, and proliferation. This of course can have benefits, but being able to reduce the bias in a technical system is a different question from whether a technical solution should be used on a given problem. We caution that our work should not be interpreted through a normative lens on the appropriateness of using facial recognition technology.
In contrast to some other works, we do, however, feel, that our work helps to overcome the portability trap [99] since it empowers domain experts to optimize for the right fairness metric, in connection with public policy experts, for the problem at hand rather than only narrowly optimizing one specific metric. Additionally, the bias mitigation strategy which we propose here can be used in other domains and applied to applications which have more widespread and socially acceptable algorithmic applications [19].
This research was partially supported by the following sources: NSF CAREER Award IIS-1846237, NSF D-ISN Award #2039862, NSF Award CCF-1852352, NIH R01 Award NLM-013039-01, NIST MSE Award #20126334, DARPA GARD #HR00112020007, DoD WHS Award #HQ003420F0035, ARPA-E Award #4334192; TAILOR, a project funded by EU Horizon 2020 research and innovation programme under GA No 952215; the German Federal Ministry of Education and Research (BMBF, grant RenormalizedFlows 01IS19077C); the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under grant number 417962828; the European Research Council (ERC) Consolidator Grant "Deep Learning 2.0" (grant no. 101045765). Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the ERC. Neither the European Union nor the ERC can be held responsible for them.
[1] Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016.
[2] Bobby Allyn. 'The computer got it wrong': How facial recognition led to false arrest of black man. NPR, June, 24, 2020.
[120] Weijian Xu, Yifan Xu, Tyler Chang, and Zhuowen Tu. Co-scale conv-attentional image transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9981–9990, 2021.
[121] Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Guo-Jun Qi, Qi Tian, and Hongkai Xiong. Pc-darts: Partial channel connections for memory-efficient architecture search. arXiv preprint arXiv:1907.05737, 2019.
[122] Fisher Yu, Dequan Wang, Evan Shelhamer, and Trevor Darrell. Deep layer aggregation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2403-2412, 2018.
[123] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P. Gummadi. Fairness constraints: Mechanisms for fair classification. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA, volume 54 of Proceedings of Machine Learning Research, pages 962-970. PMLR, 2017. URL http://proceedings.mlr.press/v54/zafar17a.html.
[124] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P. Gummadi. Fairness constraints: A flexible approach for fair classification. Journal of Machine Learning Research, 20(75):1-42, 2019. URL http://jmlr.org/papers/v20/18-262.html.
[125] Arber Zela, Thomas Elsken, Tonmoy Saikia, Yassine Marrakchi, Thomas Brox, and Frank Hutter. Understanding and robustifying differentiable architecture search. arXiv preprint arXiv:1909.09656, 2019.
[126] Zizhao Zhang, Han Zhang, Long Zhao, Ting Chen, and Tomas Pfister. Aggregating nested transformers. arXiv preprint arXiv:2105.12723, 2021.
[127] Tianyue Zheng and Weihong Deng. Cross-posed for studying cross-posed face recognition in unconstrained environments. Beijing University of Posts and Telecommunications, Tech. Rep, 5(7), 2018.
[128] Tianyue Zheng, Weihong Deng, and Jiani Hu. Cross-age lfw: A database for studying cross-age face recognition in unconstrained environments. arXiv preprint arXiv:1708.08197, 2017.
[129] Ning Zhu. Neural architecture search for deep face recognition. arXiv preprint arXiv:1904.09523, 2019.
Face recognition systems are being used for more and more parts of daily lives, from government surveillance [49], to target identification from drones [72], to identification in personal photo repositories [36]. It is also increasingly evident that many of these models are biased based on race and gender [37, 92, 91]. If left unchecked, these technologies, which make biased decision for life-changing events, will only deepen existing societal harms. Our work seeks to better understand and mitigate the negative effects that biased face recognition models have on society. By conducting the first large-scale study of the effect of architectures and hyperparameters on bias, and by developing and open-sourcing face recognition models that are more fair than all other competitive models, we provide a resource for practitioners to understand inequalities inherent in face recognition systems and ultimately advance fundamental understandings of the harms and technological ills of these systems.
That said, we would like to address potential ethical challenges of our work. We believe that the main ethical challenge of this work centers on our use of certain datasets. We acknowledge that the common academic datasets which we used to evaluate our research questions — CelebA [69] and VGGFace2 [8] — are all datasets of images scraped from the web without the informed consent of those whom are depicted. This also includes the datasets we transfer to, including LFW, CFP_FF, CFP_FF, AgeDB, CALFW, CPLFW, and RFW. This ethical challenge is one that has plagued the research and computer vision community for the last decade [86, 85] and we are excited to see datasets being released which have fully informed consent of the subjects, such as the Casual Conversations Dataset [45]. Unfortunately, this dataset in particular has a rather restrictive license, much more restrictive than similar datasets, which prohibited its use in our study. Additionally, these datasets all exhibit representational bias where the categories of people that are included in the datasets are not equally represented. This can cause many problems; see [14] for a detailed look at representational bias's impact in facial recognition specifically. At least during training, we did address representational bias by balancing the training data between gender presentations appropriately.
We also acknowledge that while our study is intended to be constructive in performing the first neural architecture search experiments with fairness considerations, the specific ethical challenge we highlight is that of unequal or unfair treatment by the technologies. We note that our work could be taken as a litmus test which could lead to the further proliferation of facial recognition technology which could cause other harms. If a system demonstrates that it is less biased than other systems, this could be used as a reason for the further deployment of facial technologies and could further impinge upon unwitting individual's freedoms and perpetuate other technological harms. We explicitly caution against this form of techno-solutionism and do not want our work to contribute to a conversation about whether facial recognition technologies should be used by individuals.
Experiments were conducted using a private infrastructure, which has a carbon efficiency of 0.373  $\mathrm{kgCO_2eq / kWh}$ . A cumulative of 88493 hours of computation was performed on hardware of type RTX 2080 Ti (TDP of 250W). Total emissions are estimated to be 8,251.97  $\mathrm{kgCO_2eq}$  of which  $0\%$  was directly offset. Estimations were conducted using the MachineLearning Impact calculator presented in [58]. By releasing all of our raw results, code, and models, we hope that our results will be widely beneficial to researchers and practitioners with respect to designing fair face recognition systems.
We ensure that all of our experiments are reproducible by releasing our code and raw data files at https://github.com/dooleys/FR-NAS. We also release the instructions to reproduce our results with the code. Furthermore, we release all of the configuration files for all of the models trained. Our experimental setup is described in Section 3 and Appendix C.1. We provide clear documentation on the installation and system requirements in order to reproduce our work. This includes information about the computing environment, package requirements, dataset download procedures, and license information. We have independently verified that the experimental framework is reproducible which should make our work and results and experiments easily accessible to future researchers and the community.
Table 5: The fairness metrics explored in this paper. Rank Disparity is explored in the main paper and the other metrics are reported in Appendix C.4
Fairness Metric	Equation
Rank Disparity	|Rank(male) - Rank(female)|
Disparity	|Accuracy(male) - Accuracy(female)|
Ratio	|1 - Accuracy(male)/Accuracy(female)|
Rank Ratio	|1 - Rank(male)/Rank(female)|
Error Ratio	|1 - Error(male)/Error(female)|
C Further Details on Experimental Design and Results
The list of the models we study from timm are: coatlite_small [120], convit_base [20], cspdarknet53 [107], dla102x2 [122], dpn107 [10], ese_vovnet39b [62], fbnetv3_g [18], ghostnet_100 [42], gluon_inception_v3 [103], gluon_xception65 [15], hrnet_w64 [102], ig_resnext101_32x8d [119], inception_resnet_v2 [104], inception_v4 [104], jx_nest_base [126], legacy_senet154 [51], mobilenetv3_large_100 [50], resnetrs101 [4], rexnet_200 [41], selecsls60b [74], swin_base PATCH4_window7_224 [68], tf_efficientnet_b7_ns' [105], 'tnt_spatch16_224[43], twins_svt_large [16] , vgg19 [101], vgg19_bn [101], visformer_small [11], xception and xception65 [15].
We study at most 13 configurations per model ie 1 default configuration corresponding to the original model hyperparameters with CosFace as head. Further, we have at most 12 config consisting of the 3 heads (CosFace, ArcFace, MagFace)  $\times 2$  learning rates(0.1,0.001)  $\times 2$  optimizers (SGD, AdamW). All the other hyperparameters are held constant for training all the models. All model configurations are trained with a total batch size of 64 on 8 RTX2080 GPU for 100 epochs each.
We study these models across five important fairness metrics in face identification: Rank Disparity, Disparity, Ratio, Rank Ratio, and Error Ratio. Each of these metrics is defined in Table 5.
C.2 Additional details on NAS+HPO Search Space
We replace the repeating 1x1_conv-3x3_conv-1x1_conv block in Dual Path Networks with a simple recurring searchable block. depicted in Figure 6. Furthermore, we stack multiple such searched blocks to closely follow the architecture of Dual Path Networks. We have nine possible choices for each of the three operations in the DPN block, each of which we give a number 0 through 8, depicted in Figure 6. The choices include a vanilla convolution, a convolution with pre-normalization and a convolution with post-normalization Table 7. To ensure that all the architectures are tractable in terms of memory consumption during search we keep the final projection layer (to 1000 dimensionality) in timm.
C.3 Obtained architectures and hyperparameter configurations from Black-Box-Optimization