SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We should also note that, unlike most works in the literature, we propose our method for both whitebox and blackbox scenarios and evaluate the transferability of our attack. Similar to [Cole et al., 2017, Duong et al., 2020, Truong et al., 2022], our method is based on the whitebox knowledge of FR model, however our approach for extending our method to the blackbox attack using another FR model has not been used in TI attacks. Last but not least, we define five different attacks against FR systems and evaluate the vulnerability of SOTA FR models to our attacks. To our knowledge, this is the first paper in which the transferability of reconstructed face images in TI attacks has been investigated.
In this section, we present our experiments and discuss our results. First, in Section 4.1 we describe our experimental setup. Then, we present our experimental results in Section 4.2, and discuss our findings.
<sup>2</sup>Several papers in the literature used the intermediate latent space  $\mathcal{W}$  of StyleGAN for image editing [Roich et al., 2022, Alaluf et al., 2022, Hu et al., 2022].
To evaluate the performance of our method, we consider two SOTA FR models $^3$ , including ArcFace [Deng et al., 2019], ElasticFace [Boutros et al., 2022], as the models from which templates are leaked (i.e.,  $F_{\mathrm{database}}$ ). For transferability evaluation, we also use three different FR models with SOTA backbones from FaceX-Zoo [Wang et al., 2021], including HRNet [Wang et al., 2020], AttentionNet [Wang et al., 2017], and Swin [Liu et al., 2021], for the target FR system (i.e.,  $F_{\mathrm{target}}$ ). The recognition performance of these models are reported in Table 3. All these models are trained on
Table 3: Recognition performance of face recognition models used in our experiments in terms of true match rate (TMR) at the thresholds correspond to false match rates (FMRs) of  $10^{-2}$  and  $10^{-3}$  evaluated on the MO-BIO and LFW datasets. The values are in percentage.
model	MOBIO	LFW
FMR=10-2	FMR=10-3	FMR=10-2	FMR=10-3
ArcFace	100.00	99.98	97.60	96.40
ElasticFace	100.00	100.00	96.87	94.70
HRNet	98.98	98.23	89.30	78.43
AttentionNet	99.71	97.73	84.27	72.77
Swin	99.75	98.98	91.70	87.83
MS-Celeb1M dataset [Guo et al., 2016]. We assume that the adversary does not have access to the FR training dataset, and therefore we use another dataset for training our face reconstruction models. To this end, we use the Flickr-Faces-HQ (FFHQ) dataset [Karras et al., 2019], which consists of 70,000 high-resolution (i.e.,  $1024 \times 1024$ ) face images (without identity labels) crawled from the internet. We use  $90\%$  random portion of this dataset for training, and the remaining  $10\%$  for validation.
To evaluate different attacks against FR systems, we consider two other face image datasets with identity labels, including the MOBIO [McCool et al., 2013] and Labeled Faces in the Wild (LFW) [Huang et al., 2007] datasets. The MOBIO dataset consists of bi-modal (face and voice) data captured using mobile devices from 150 people in 12 sessions (6-11 samples in each session). The LFW dataset includes 13,233 face images of 5,749 people collected from the internet, where 1,680 people have two or more images.
For each of the attacks described in Section 2, we build one or two separate FR systems with one or two SOTA FR models based on the attack type. If the target system is the same as the system from which the template is leaked, we have only one FR system. Otherwise, if the target system is different the system from which the template is leaked, we have two FR systems with two different feature extractors. In each case, we use one of our evaluation datasets (i.e., MOBIO and LFW) to build both FR systems (so that the subject with the leaked template be enrolled in the target system too). In each evaluation, we assume that the target FR system is configured at the threshold corresponding to a false match rate (FMR) of  $10^{-3}$ , and we evaluate the adversary's success attack rate (SAR) in entering that system.
We should note that the templates extracted by the aforementioned FR models have 512 dimensions. The input noise  $\mathbf{z} \in \mathcal{Z}$  to the mapping network of StyleGAN's pre-trained network is from the standard normal distribution and has 512 dimensions. The input noise  $\mathbf{n} \in \mathcal{N}$  to our mapping network  $M_{\mathrm{rec}}(.)$  is with dimension of 8 and also from the standard normal distribution. We also use Adam [Kingma and Ba, 2015] optimizer to train our mapping network<sup>4</sup>.
4.2 Analysis
In this section, we consider SOTA FR models and evaluate the performance of our face reconstruction method in five different attacks described in Section 2. We also explore the effect of our WGAN training as well as effect of loss terms as our ablation study. In addition, explore important areas in the reconstructed face images that lead to success TI attack. Finally, we discuss limitations of our face reconstruction model.
Whitebox Knowledge of  $F_{\text{database}}$  For attacks 1-2, the adversary is assumed to have whitebox knowledge of the system from which the template is leaked (i.e.,  $F_{\text{database}}$ ) and use the same feature extraction model for training (i.e.,  $F_{\text{loss}}$ ), thus in such cases  $F_{\text{loss}} = F_{\text{database}}$ . We considered ArcFace
3While we use three different face recognition models in our problem formulation, since these models are applied in separate stages, as our experiments in Section A.4 of appendix shows, there is no issue if the inputs and outputs (e.g., pre-processing steps or dimensions) be different in each of these face recognition models.
Source code of our experiment is available at: https://gitlab.idiap.ch/bob/bob.paper.neurips2023_face(ti
Table 4: Evaluation of attacks with whitebox knowledge of the system from which the template is leaked (i.e.,  $F_{\text{loss}} = F_{\text{database}}$ ) against SOTA FR models in terms of adversary's success attack rate (SAR) using our proposed method on the MOBIO and LFW datasets. The values are in percentage and correspond to the threshold where the target system has  $\text{FMR} = 10^{-3}$ . Cells are color coded according the type of attack as defined in Section 2 for attack 1 (light gray) and attack 2 (dark gray).
Fdatabase	MOBIO	LFW
ArcFace	ElasticFace	HRNet	AttentionNet	Swin	ArcFace	ElasticFace	HRNet	AttentionNet	Swin
ArcFace	92.38	81.90	71.43	70.48	74.29	86.82	74.20	36.57	36.40	58.86
ElasticFace	78.10	87.62	64.29	64.76	69.05	78.25	82.52	41.80	40.25	61.09
Table 5: Evaluation of attacks (with blackbox knowledge of the system from which the template is leaked i.e.,  $F_{\text{database}}$ ) against SOTA FR models in terms of adversary's success attack rate (SAR) using different methods on the MOBIO and LFW datasets. The values are in percentage and correspond to the threshold where the target system has  $\text{FMR} = 10^{-3}$ . M1: NbNetB-M [Mai et al., 2018], M2: NbNetB-P [Mai et al., 2018], M3: [Dong et al., 2021], M4: [Vendrow and Vendrow, 2021], and M5: [Dong et al., 2023]. Cells are color coded according the type of attack as defined in Section 2 for attack 3 (lightest gray), attack 4 (middle dark gray), and attack 5 (darkest gray).
Fdatabase	Floss	Ftarget	MOBIO	LFW
M1	M2	M3	M4	M5	Ours	M1	M2	M3	M4	M5	Ours
ArcFace	ElasticFace	ArcFace	1.90	15.24	2.38	28.10	58.57	81.90	10.68	40.25	12.91	58.88	75.31	77.16
ElasticFace	1.43	11.43	4.29	15.24	37.61	73.81	8.36	34.39	6.35	29.10	50.17	68.06
HRNet	0.95	6.19	2.86	10.00	30.48	57.14	1.30	7.78	1.75	9.20	24.72	28.45
AttentionNet	0	6.67	3.33	4.29	26.67	54.29	1.33	7.17	2.29	9.17	24.16	28.87
Swin	1.43	13.33	3.81	10.95	40.00	67.14	4.27	23.85	5.97	21.75	41.27	48.28
ElasticFace	ArcFace	ArcFace	2.38	18.57	2.86	16.19	48.09	87.14	15.33	48.67	11.81	37.45	65.40	83.20
ElasticFace	3.81	43.81	4.76	43.33	72.38	89.05	21.44	58.16	11.59	52.88	74.08	83.43
HRNet	0.48	20.00	1.43	10.48	42.86	73.81	3.46	18.36	2.74	11.82	32.99	49.02
AttentionNet	1.90	18.10	3.33	9.05	40.00	71.90	2.89	16.31	2.91	10.95	31.15	46.63
Swin	0.95	26.19	2.86	15.24	46.67	75.24	9.22	38.79	8.26	24.62	51.20	66.89
and ElasticFace models and reconstructed face images from the templates extracted by these models in attacks against different FR systems. Table 4 reports the vulnerability of different target systems to our attacks<sup>5</sup> 1-2 in terms of adversary's SAR at the system's FMR of  $10^{-3}$ . Similar results for the system's FMR of  $10^{-2}$  are reported in Table 7 of the appendix. According to these tables, our method achieves considerable SAR against ArcFace and ElasticFace target systems in attack 1. In attack 2, we observe that there is a degradation in SAR with respect to attack 1. However, the reconstructed face images can still be used to enter another target system. Meanwhile, the FR model with a higher recognition accuracy is generally more vulnerable to attack 2. For instance, when ArcFace is considered as  $F_{\mathrm{database}}$ , we observe that ElasticFace and Swin have the highest SAR as target systems, while there is the same order for their recognition performance in Table 3.
Blackbox Knowledge of  $F_{\text{database}}$  For attacks 3-5, the adversary is assumed to have blackbox knowledge of the system from which the template is leaked (i.e.,  $F_{\text{database}}$ ) and use another feature extraction model for training (i.e.,  $F_{\text{loss}}$ ), therefore in such cases  $F_{\text{loss}} \neq F_{\text{database}}$ . Table 5 compares the performance of our method with blackbox methods in the literature [Mai et al., 2018, Dong et al., 2021, Vendrow and Vendrow, 2021, Dong et al., 2023] for attacks 3-5 in terms of adversary's SAR at system's FMR of  $10^{-3}$ . Similar results for the FMR of  $10^{-2}$  are available in Table 8 of the appendix.
As these tables show, our proposed method achieves the highest SAR compared to [Mai et al., 2018, Dong et al., 2021, Vendrow and Vendrow, 2021, Dong et al., 2023] against FR systems on the MOBIO and LFW datasets. In particular, in attack 5 which is the hardest attack, where  $F_{\mathrm{database}}$ ,  $F_{\mathrm{loss}}$ , and  $F_{\mathrm{target}}$  are different, the results show that the target FR system is still vulnerable to our attack. The results of our method for attack 5 also show transferability of our attack to different FR systems. Similar to attack 2, we can also observe that in attack 5, the FR model with a higher recognition accuracy is generally more vulnerable to our attack.
<sup>5</sup>We should highlight that the whitebox methods reported in Table 2 do not have available source code, and we could not compare our method with whitebox methods in Table 2.
The other blackbox methods in the literature do not have available source code and we could not reproduce their results.
Ablation Study To evaluate the effect of WGAN in training our mapping network and the effect of each term in our loss function (i.e., Eq. 1), we consider the ArcFace model in the white-box scenario and train different face reconstruction networks with different loss functions. Then, we attack a system with the ArcFace model as a feature extractor (i.e., attack 1) and compare the SARs as reported in Table 6. According to these results, the proposed adversarial training has a significant effect on our face reconstruction method. Because we fix the synthesis network of StyleGAN, the mapped latent codes need to be of the same distribution as  $\mathcal{W}$ . Otherwise, the generated image is not face-like and training fails to converge. The WGAN training in our method helps our mapping network to learn the distribution of StyleGAN's intermediate latent space, and thus the synthesis network generates
face-like images. When we use the WGAN training and based on the results in Table 6, the ID loss has a high impact on the performance of the template inversion model. While the pixel loss by itself does not achieve a good performance, it improves the performance of ID loss in our reconstruction loss function in Eq. 1. This table confirms that the proposed WGAN training and our reconstruction loss function lead to a more successful attack. More experiments for ablation study on the effect of different elements in our proposed method are presented in the appendix.
Table 6: Evaluating the effect of each loss term in our loss function in attack 1 against ArcFace in terms of SAR in the system with FMRs of  $10^{-2}$  and  $10^{-3}$  evaluated on the MOBIO and LFW datasets. The values are in percentage.
WGAN training (Eqs. 4 and 5)	Reconstruction Loss Function	MOBIO	LFW
FMR=10-2	FMR=10-3	FMR=10-2	FMR=10-3
✓	\( \mathcal{L}_{\text{rec}} = \mathcal{L}_{\text{pixel}} + \mathcal{L}_{\text{ID}} \)	100.00	92.38	93.64	86.82
\( \mathcal{L}_{\text{rec}} = \mathcal{L}_{\text{ID}} \)	98.10	82.38	90.56	80.74
\( \mathcal{L}_{\text{rec}} = \mathcal{L}_{\text{pixel}} \)	0	0	0.65	0.07
×	\( \mathcal{L}_{\text{rec}} = \mathcal{L}_{\text{pixel}} + \mathcal{L}_{\text{ID}} \)	0	0	0.32	0.02
\( \mathcal{L}_{\text{rec}} = \mathcal{L}_{\text{ID}} \)	0	0	0.14	0.02
\( \mathcal{L}_{\text{rec}} = \mathcal{L}_{\text{pixel}} \)	0	0	0.44	0.09
Important Areas in the Reconstructed Face Images As another experiment, we explore important areas in the reconstructed face images. Finding these areas help us to investigate what features between the original template and synthetic images fool the face recognition system, and therefore we can understand what information is encoded in the facial templates. To this end, we apply the Grad-Cam [Selvaraju et al., 2017] algorithm using the face recognition model on the reconstructed face images to see which areas of the reconstructed face images are important and cause the facial templates of our reconstructed face images to be close to the original facial templates. Figure 5 shows results of applying the Grad-Cam algorithm on sample reconstructed face images using our proposed method. As the results in this figure show, important areas that cause the reconstructed face images to have similar templates to the original ones correspond to areas such as eyes, nose, lips, etc. In
Figure 4 also shows sample face images from the LFW dataset and the reconstructed images using our proposed method from ArcFace templates in different attacks. We should highlight that as show in Figure 4, the reconstructed face images in attack 1 and attack 2 are the same, but they are used to enter different target FR system. The same holds for the reconstructed face images in attacks 3-5.
Figure 4: Sample face images from the LFW dataset (first raw) and their corresponding reconstructed images using our template inversion method from ArcFace templates in different attacks, attacks 1-2 (second raw) and attacks 3-5 (second raw, using ElasticFace for  $F_{\mathrm{loss}}$ ). The values below each image show the cosine similarity between the corresponding ArcFace templates of original and reconstructed face images.
0.773
Figure 5: Sample face images from the FFHQ dataset and the corresponding important areas in the reconstructed face images using the Grad-Cam algorithm. The reconstructed face images are generated from ArcFace templates. In each example, the top image is the output of the Grid-Cam algorithm, and the bottom images are real (right) and reconstructed (left) images. The value below each sample is the cosine similarity between the templates of original and reconstructed face images.
0.738
0.768
particular, the area around the eyes seems to be the most important part in most of the reconstructed face images. These results also show that the general shape of the face (e.g., thin or chubby face), hairs, textures, etc., are not often necessarily important in the reconstructed face images, and thus, we can also conclude that these attributes are not well-encoded in the templates extracted by face recognition models.
Limitations Despite the significant performance of our method in terms of success attack rate in all types of attacks reported in Table 4 and Table 5, the reconstructed face images fail to enter the system in some cases. Figure 6 illustrates sample failure cases in the attack 3 against ArcFace (using ElasticFace for  $F_{\mathrm{loss}}$ ) on the LFW dataset. From the failure cases, we can conclude that there is a bias in the face reconstruction for specific demographics, like elderly or dark skin people. Indeed, such kind of bias in the reconstructed face images is caused by inherent biases in datasets used to train FR model, the StyleGAN model, and our mapping network in our face reconstruction model<sup>7</sup>.
In this paper, we proposed a new method to reconstruct high-resolution realistic face images
from facial templates in a FR system. We used a pre-trained StyleGAN3 network and learned a mapping from facial templates to intermediate latent space of StyleGAN within a GAN-based framework. We proposed our method for whitebox and blackbox scenarios. In the whitebox scenario, the adversary can use the feature extraction model for training the face reconstruction network; however, in the blackbox scenario, we assume that the adversary has access to another feature extraction model. In addition, we consider the threat model where the adversary might impersonate in the same or another (i.e., transferable attack) FR system. Based on the adversary's knowledge of the feature extraction model and the target FR system, we defined five different attacks and evaluated the vulnerability of SOTA FR systems to our proposed method. Our experiments showed that the reconstructed face images by our proposed method not only can achieve a high SAR in whitebox and blackbox scenarios, but also are transferable and can be used to enter target FR systems with a different FR model.
rnnnne
Reonrre
0.098
Figure 6: Sample failure cases images from the LFW dataset and their corresponding reconstructed images using our template inversion method from ArcFace templates in the attack 3 (using ElasticFace for  $F_{\mathrm{loss}}$ ). The values below each image show the cosine similarity between the corresponding templates of original and reconstructed face images.
0.163
<sup>7</sup>The biases for different demographics in verification task for ArcFace model are studied in [de Freitas Pereira and Marcel, 2021]. Similarly, biases in StyleGAN generated images and also the FFHQ dataset (i.e., our training dataset) are investigated in [Karakas et al., 2022, Tan et al., 2020, Balakrishnan et al., 2020].
This research is based upon work supported by the H2020 TReSPAsS-ETN Marie Skłodowska-Curie early training network (grant agreement 860813).
Javier Galbally, Sébastien Marcel, and Julian Fierrez. Biometric antispoofing methods: A survey in face recognition. IEEE Access, 2:1530-1552, 2014.
Battista Biggio, Paolo Russu, Luca Didaci, Fabio Roli, et al. Adversarial biometric recognition: A review on biometric system security from the adversarial machine-learning perspective. IEEE Signal Processing Magazine, 32(5):31-41, 2015.
Abdenour Hadid, Nicholas Evans, Sebastien Marcel, and Julian Fierrez. Biometrics systems under spoofing attack: an evaluation methodology and lessons learned. IEEE Signal Processing Magazine, 32(5):20-30, 2015.
Guangcan Mai, Kai Cao, Pong C Yuen, and Anil K Jain. On the reconstruction of face images from deep face templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(5): 1188-1202, 2018.
Sebastien Marcel, Julian Fierrez, and Nicholas Evans. Handbook of Biometric Anti-Spoofing: Presentation Attack Detection and Vulnerability Assessment. Springer, 2023.
Tero Karras, Miika Aittala, Samuli Laine, Erik Harkonen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. Advances in Neural Information Processing Systems, 34, 2021.
Bowen Zhang, Shuyang Gu, Bo Zhang, Jianmin Bao, Dong Chen, Fang Wen, Yong Wang, and Baining Guo. Styleswin: Transformer-based gan for high-resolution image generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11304-11314, 2022.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214-223. PMLR, 2017.
Andrey Zhmoginov and Mark Sandler. Inverting face embeddings with convolutional neural networks. arXiv preprint arXiv:1606.04189, 2016.
Forrester Cole, David Belanger, Dilip Krishnan, Aaron Sarna, Inbar Mosseri, and William T Freeman. Synthesizing normalized faces from facial identity features. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3703-3712, 2017.
Chi Nhan Duong, Thanh-Dat Truong, Khoa Luu, Kha Gia Quach, Hung Bui, and Kaushik Roy. Vec2face: Unveil human faces from their blackbox features in face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6132-6141, 2020.
Thanh-Dat Truong, Chi Nhan Duong, Ngan Le, Marios Savvides, and Khoa Luu. Vec2face-v2: Unveil human faces from their blackbox features via attention-based network in face recognition, 2022. URL https://arxiv.org/abs/2209.04920.
Sohaib Ahmad, Kaleel Mahmood, and Benjamin Fuller. Inverting biometric models with fewer samples: Incorporating the output of multiple models. In 2022 IEEE International Joint Conference on Biometrics (IJCB), pages 1-11. IEEE, 2022.
Xingbo Dong, Zhe Jin, Zhenhua Guo, and Andrew Beng Jin Teoh. Towards generating high definition face images from deep templates. In 2021 International Conference of the Biometrics Special Interest Group (BIOSIG), pages 1-11. IEEE, 2021.
Edward Vendrow and Joshua Vendrow. Realistic face reconstruction from deep embeddings. In NeurIPS 2021 Workshop Privacy in Machine Learning, 2021.
A More Analyses
A.1 Evaluation of Whitebox and Blackbox Attacks at FMR=  $10^{-2}$
Table 7 and Table 8 of this appendix report the evaluation of attacks with whitebox and blackbox knowledge, respectively, of the system from which the template is leaked (i.e.,  $F_{\mathrm{loss}} = F_{\mathrm{database}}$ ) against SOTA FR models at  $\mathbf{FMR} = 10^{-2}$  in terms of adversary's success attack rate (SAR) using our proposed method on the MOBIO and LFW datasets. As the results in these tables show, our method outperforms previous methods in the literature.
Table 7: Evaluation of attacks with whitebox knowledge of the system from which the template is leaked (i.e.,  $F_{\text{loss}} = F_{\text{database}}$ ) against SOTA FR models in terms of adversary's success attack rate (SAR) using our proposed method on the MOBIO and LFW datasets. The values are in percentage and correspond to the threshold where the target system has  $\mathbf{FMR} = 10^{-2}$ . Cells are color coded according the type of attack as defined in Section 2 of the paper for attack 1 (light gray) and attack 2 (dark gray).
Fdatabase	MOBIO	LFW
ArcFace	ElasticFace	HRNet	AttentionNet	Swin	ArcFace	ElasticFace	HRNet	AttentionNet	Swin
ArcFace	100.00	93.81	80.00	81.90	85.24	93.64	90.89	68.08	62.75	76.24
ElasticFace	90.95	93.33	78.57	83.81	84.29	87.88	92.80	71.82	64.24	75.70