Face reconstruction from facial templates by learning latent space of a generator network
Authors: Hatef Otroshi Shahreza
Sébastien Marcel
Venue: Submitted to ICLR 2023 (2023)

Abstract: Face recognition systems are increasingly deployed in different applications. In these systems, a feature vector (also called facial embeddings or templates) is typically extracted from each face image and is stored in the system's database during the enrollment stage, which is later used for comparison during the recognition stage. In this paper, we focus on the template inversion attack against face recognition systems and propose a new method to reconstruct face images from facial templates. Within a generative adversarial network (GAN)-based framework, we learn a mapping from facial templates to the intermediate latent space of a pre-trained face generation network, from which we can generate high-resolution realistic reconstructed face images. We show that our proposed method can be applied in whitebox and blackbox attacks against face recognition systems. Furthermore, we evaluate the transferability of our attack when the adversary uses the reconstructed face image to impersonate the underlying subject in an attack against another face recognition system. Considering the adversary's knowledge and the target face recognition system, we define five different attacks and evaluate the vulnerability of state-of-the-art face recognition systems. Our experiments show that our proposed method achieves high success attack rates in whitebox and blackbox scenarios. Furthermore, the reconstructed face images are transferable and can be used to enter target face recognition systems with a different feature extractor model. 

Under review as a conference paper at ICLR 2023
FACE RECONSTRUCTION FROM FACIAL TEMPLATES BY LEARNING LATENT SPACE OF A GENERATOR NETWORK
Anonymous authors
Paper under double-blind review
ABSTRACT
Face recognition systems are increasingly deployed in different applications. In these systems, a feature vector (also called facial embeddings templates) is typically extracted from each face image and is stored in the system's 's database during the enrollment stage. which is later used for comparison during the recog- nition stage. In this paper, we focus on the template inversion attack against face recognition systems and propose a new method to reconstruct face images from fa- cial templates. Within a generative adversarial network (GAN)-based framework, we learn a mapping from facial templates to the intermediate latent space of a pre-trained face generation network, from which realistic reconstructed face images. We show that our proposed method can be applied in whitebox and blackbox attacks against face recognition systems. Fur- thermore, we evaluate the transferability of our attack when the adversary uses the reconstructed face image to impersonate the underlying subject in attack against another face recognition system. Considering the adversary's 's knowledge and the target face recognition system, we define five different attacks and evalu- ate the vulnerability of state-of-the-are face recognition systems. Our experiments show that our proposed method achieves high success attack rates whitebox and blackbox scenarios. Furthermore, the reconstructed face images are transferable and can used to enter target face recognition systems with a different feature extractor model.
generate high-resolution
Original
Reconstructed
0.815
0.629
0.618
0.728
0.602
0.742
Figure 1: Sample face images from the FFHQ dataset and their corresponding reconstructed images using our template inversion method from ArcFace templates. The values below each image show the cosine similarity between the corresponding templates of original and reconstructed face images.
INTRODUCTION
Face recognition (FR) systems tend toward ubiquity, and their applications, which range from cell phone unlock to national identity system, border control, etc., growing rapidly. Typically, in such systems, a feature vector (called embedding or template) is extracted from each face image using a deep neural network, and is stored in the system's 's database during the enrollment stage. During the

---

Under review as a conference paper at ICLR 2023
FR system from which the lemplate is (Paneeo)
Face Reconstraction Network (traiee using Fiowl
Target FR system (Fage)
Nodse Templaae x0
Dalabese
Feature Extraciion Firaee
Mapping Network (Ouns)
StyleGAA's 's Syathesi Networks
Feature Extraction (Flage)
Databese
Compare
Labent Space (a0-Af_ijn..
Reconstracted Faze Image (F)
Recognized ?
Looked Template
Injecting the Reconstructed Face Imago
Figure 2: Block diagram of our proposed template inversion attack
recognition stage, either verification or identification, the extracted feature vector is compared with the ones in the system's database to measure the similarity of identities. Among potential attacks
against FR systems (Galbally et al. 20144] Marcel et al. [2014] Biggio et al. 2015 Hadid et al. 2015 Mai et al. 2018). the template inversion (TI) attack significantly jeopardizes the users' privacy. In
TI attack, the adversary gains access to templates stored in the FR system's database and aims to reconstruct the underlying face image. Then, the adversary not only achieves privacy-sensitive information (such as gender, ethnicity, etc.) of enrolled users, but also can use reconstructed face images to impersonate.
In this paper, focus the TI attack against FR systems and propose novel method to recon- struct face images from facial templates (Fig. shows sample reconstructed face images using our proposed method). Within a generative adversarial network (GAN)-basel framework, we learn mapping from face templates to the intermediate latent space of StyleGAN3 (Karras et al., 2021), as a pre-trained face generation network. Then, using the synthesis part of StyleGAN3, we can generate high- nigh-resolution realistic face image. Our proposed method can be applied for whitebox and blackbox attacks against FR systems. the whitebox scenario, adversary knows the internal functioning of the feature extraction model and its parameters. However, in the blackbox scenario, the adversary does not know the internal functioning of the feature extraction model and can only use it to extract features from any arbitrary image. Instead, we assume that the adversary has whitebox of another FR model, which can be used for training the face reconstruction network. We also evaluate the transferability of our attack by considering the case where the adversary uses the reconstructed face image to impersonate the underlying subject in an attack against another FR sys- tem (which has different feature extraction model). Considering the adversary's 's knowledge and the target FR system, we define five different attacks, and evaluate the vulnerability of state-of-the-art (SOTA) FR systems. Fig. illustrates the general black diagram of our proposed template inversion attack.
To elaborate on the contributions of our paper, we list them hereunder:
We propose a novel method to generate high-resolution realistic face images from facial templates. Within GAN- framework, learn the mapping from facial templates to the latent space of a pre- pre-trained face generation network.
We propose method for whitebox and blackbox scenarios. While our method is based on the whitebox knowledge of the FR model, we extend our attack blackbox scenario, using another FR model that the adversary has access to.
We define five different attacks against FR systems (based on the adversary's knowledge and the target system), and evaluate the vulnerability of SOTA FR models.
The remainder of the paper is organized as follows: Section 2 introduces the problem formulation and our proposed face reconstruction method, Section 3 covers the related works the literature and compares them with our proposed method. Section 4 presents our experiential results. Finally, the paper is concluded in Section
PROBLEM DEFINITION AND PROPOSED METHOD
In this paper, we consider Tl attack against FR system based on the following threat model:
2

---

Under review as a conference paper at ICLR 2023
Adversary's goal: The adversary aims to reconstruct face image from template, and use the reconstructed face image to enter the same or a different face recognition system, which call the target FR system.
Adversary's 's knowledge:
The adversary knows face template of a user enrolled in the FR system's 's database. The adversary also has either whitebox or blackbox knowledge of the feature extractor model in the same FR system.
Adversary's capability: The adversary can present the reconstructed face image to the tar- get FR system (c.g., using a printed photograph). However, for simplicity, consider that adversary can inject the reconstructed face image as query to the target FR system.
Adversary's strategy: The adversary can train face reconstruction model to invert facial templates and reconstruct underlying face images. Then, the adversary can use the recon- structed face images to inject as query the target FR system, to enter that system.
Let F()) denotes facial feature extraction model, which gets the face image I I and extracts facial template x F(T)= X. According to threat model, the adversary has access the target facial template Tdatalase Fdatabuse (I) and aims to generate a reconstructed face image Í. Then, the adversary can use the reconstructed face image I to impersonate the corresponding subject and attack target FR system with (.). which might be different from Fiataause (-).
To train face reconstruction model, we can use dataset of face images { with N face images (no label is required), and generate training dataset {(a:; [.) where æ;; Fa [.). Then, a face reconstruction model G(.) can be trained to reconstruct face image i G(z) given each facial template E X. To train such face reconstruction model, we consider multi-term face reconstruction loss function as follows:
Lve = Cpixet + LID:
(1)
where ppixel and LiD indicate pixel loss and ID loss, respectively, and are defined as:
pisel Ex G(
(2)
Lw Ex~x Floss (I) Flees (G()))
(3)
The pixel loss is used minimize the pixel-level reconstruction error of the generated face image. The ID loss is also used to minimize the distance between facial templates extracted by F. from original and reconstructed face images. In Eq. 3 Fiess(, (.) denotes feature extraction model that the adversary is assumed to have complete knowledge of its parameters and internal functioning, Based on the adversary's 's knowledge of Fantabuse (-) (i.e., whitebox or blackbox scenarios), Floss (-) might be
the same or different from Fdatabuse (.).
For the face reconstruction model, we consider StyleGAN3 (Karras jet al. [02T), as a pre-trained face generation network. The Style- GAN3 model trained on a dataset of face images using GAN-based framework that can generate high- resolution and realistic face im- ages. The structure of StyleGAN3 composed of two networks, map- ping and synthesis networks. The mapping network MsSyLGAAN.) gets random noise z E Z and gen- erates an intermediate latent code w = MMsylean(()) E W. Then,
Latent Space (w>W)
Noise E
StyleGAN's Mapping Network
reo!
CC WGAN
Critic Network
OGAAN C Mac
Noise (n X/)
Mapping Network (Ours)
StyleGAN's Synthesis Network
Lece
Template (z e X))
Latent Space
Reconstructed Face Image (f)
(s- ([n.ao
the latent code w is given to the synthesis network SSyyLoGAN(.) to generate a face image. In our train- ing process, fix the synthetic network (Sy)eGAN and train a new mapping M. (.) to generate v corresponding to the given fa- cial template X'. Then, the generated latent code v is given the synthesis network SSyyleGAN
Figure 3: Block diagram of our face reconstruction network.
3

---

Under review as a conference paper at ICLR 2023
to generate the reconstructed face image I SSYRKAN (w)). We can train our new mapping Mie using our reconstruction loss function as in Eq. [ However, to obtain a realistic face image from the generated tù through the pre-trained synthetic network SSyyeGAN()) the generated i needs to be in the distribution W; otherwise, the output may not look like a real human face. Hence, to generate vo vectors such that they have the same distribution as StyhleGAN's intermediate latent, use a GAN-based framework to learn the distribution W. To this end, we use the Wasserstein GAN (WGAN) Arjovsky et al. (2017) algorithm to train a critic network C(.) which critics the generated ib vectors compared to the real StyleGAN's w E W vectors, and simultaneously we optimize our mapping network to generate ió vectors with the same distribution as W. Hence, we can consider our mapping network Mrea..) (.) as conditional generator in our WGAN framework, which generates in Mms((n, (n, æ²) given a facial template x E X' and a random noise vector n E N. Then, we can train our mapping network and critic network using the following loss functions:
W,
CUGAN Fue-w(C(()) E Bw- Min (() (C(i))
(C(())
(4)
WWAAN MranMMae ((n, [C(i))]
(5)
In a nutshell, we train new mapping network (.) using our reconstruction loss function in Eq. [ and also optimize M,e(.) within our WGAN framework using Eq.5) Simultaneously, we also train the critic network C(.) within our WGAN using Eq. to learn the distribution of StyleGAN's intermediate latent space W and help our mapping network M.ec(.) to generate vectors with the same distribution as W. Fig. depicts the block diagram of the proposed method. We should note that our mapping network Mree (-) has 2 fully connected layers with Leaky ReLU activation function.
In our problem formulation, we consider three different feature extraction models, including F Fdatabase (-), Frsss (.), and (.) Hence, based on the adversary's 's knowledge and the target system, we can consider five different attacks:
The adversary has whitebox knowledge of the system from which the template is leaked and want to attack the same system (i.c., Fdaraase = Fosss Farget)
Attack I:
The adversary has whitebox knowledge of the feature extractor of the sys- tem from which the template leaked, but aims attack to a different FR system (i.e.. Fittabuse = Floss F Furge)).
Attack 2:
Attack 3: The adversary wants to attack the same system from which the template leaked, but has only blackbex access to the feature extractor of the system. Instead, we assume that the adversary has the whitebox knowledge of another FR model to use for training (i.e., Famatuse = Furge * Fiss).
Attack 4: The adversary aims to attack a different FR system than the one from which the template is leaked. In addition, the adversary has whitebox knowledge of the feature extractor of the target system (i.e., Fdatabase # Fioss = Firget).
Attack 5: The adversary aims to attack a different FR system from which the template is leaked and has only blackbox knowledge of both the target system and the one from which the template leaked. However, the adversary instead has the whitebox knowledge of another FR model to use for training (i.e., Faatabuse F Fluss F Funget).
In the attack 1 and attack 2, the adversary has the whitebox knowledge of the system from which the template is leaked (i.e., Fantaaaee (.)) and uses the same model as From(() for training the reconstruction network. However, in attacks 3-5, the adversary has the blackbox knowledge of the system from which the template is leaked, and therefore uses another FR model as (.)ss(.). Comparing the knowledge of the adversary in these attacks, we expect that attack I be the easiest attack for the adversary and attack be the most difficult one.
3 RELATED WORKS
Table I compares our proposed method with related works in the literature. Generally, the methods for TI attack against FR systems, be categorized based on different aspects. including the reso- lution of generated face images (high/low resolution), the type of attack (whitebox/blackbox attack), and the basis of the method (optimizatioon/eeriing-)
4

---

Under review as a conference paper at ICLR 2023
Table 1: Comparison with related works.
Reference
Resolution
White/Black-box
Basis
Available code
1) optimization
Zhmoginov & Sandler 2016
low
whitebox
2) learning
Cole et al.
2017
low
both
learning
Mai al. (0088)
low
blackbox
learning
Duong ct al.
20200
low both"*
learning
Truong [002
low both" "
learning
Dong et al. [021]]
high
blackbox
learning
Vendrow Vendrow (2021)
high
blackbox
optimization
Dong al. 2022
high
blackbox
optimization
Ours
high
both
learning
method hased on the whitebor attack, and extended blackbox by removing loss that required FR model. The method is based the whitebor attack, and the blackbox allack is performed knowledge distillation of the FR model. The method is hased on the whitebox attack, and is extended to blakhoo using a different FR model.
Zhmoginov & Sandler (016) proposed an optimization-basel method and learning-basee method to generate low-resolution face images in the whitebox attack against FR systems. In their optimization-basee attack, they used gradient-descent-beee imizes the distance of the face template as well as some regularization terms to generate a smooth image, including the total variation and Laplacian pyramid gradient normalization (Burt & Adelson 1987) of the reconstructed face image. In their learning-basee attack, they trained a convolutional neural network (CNN) with the same loss terms to generate face images from given facial templates.
approach to find an image that min-
Cole et al. ) proposed a learning-based attack to generate low-resolution images using a multi- layer perceptron (MLP) to estimate landmark coordinates and CNN to generate face textures, and then reconstructed face images using a differentiable warping based on estimated landmarks and face texture. They trained their networks in an end-to-end fashion, and minimized the errors for landmark estimation and texture generation as well as the distance of face template as their loss function. To extend their method from the whitebox attack to the blackbox attack, they proposed not to minimize the distance of face templates in their loss function.
Mai al. (2018) proposed a learning- based attack to generate low-resolution images in the blackbox attack against FR systems. They proposed new convolutional blocks, called neighborly deconvolu- tion blocks A/B (shortly, NbBlock-A and NbBlock-B), and used these blocks to reconstruct face images. They trained their proposed networks using two loss functions, including pixel loss (i.c., l2 norm of reconstruction pixel error) and perceptual loss (i.c., la norm of distance for intermediate features of VGG-19 (Simonyan & Zisserman 2014) given original and reconstructed face images).
Duong et al. 2020) and Truong et al. (2022) used a same bijection learning framework and trained GAN with a generator with structure of PO-GAN (Karras et al.] (017) and TransGAN (Jiang et al. 2021). respectively. While their method is based on the whitebox attack, they proposed to use knowledge distillation extend to the blackbox attack. To this end, they trained student network that mimics the target FR model. However, they did not provide any details (nor source code) about student network training, such the structure of the student network, etc.
Dong et al. 2222 used pre-trained StyleGAN to generate high-resolution face images in the black- box attack against FR systems. They generated synthetic face images using pre-trained StyleGAN and extracted their embedding. Then, they trained a fully connected network using mean squared error to map extracted embeddings to the corresponding noise in the input of StyleGAN. Instead of a learning-based approach, Vendrow & Vendrow (2021) used a grid search optimization using the simulated annealing (Van Laarhoven & Aarts] 1987) approach to find the noise in the input of StyleGAN, which generates an image that has the same embedding. As their iterative method has a large computation cost, they evaluated their method on 20 images only. Along the same lines, Dong et al. (2022) also tried to solve similar optimization to (Vendrow & Vendrow 2021) with different approach. They used the genetic algorithm to find the noise in the input of StyleGAN that can generate an image with the same embedding.
Compared to most works in the literature that generate low-resolution face images, our proposed method generates high-resolution realistic face images. While low-resolution reconstructed images
5

---

Under review as a conference paper at ICLR 2023
can be used for evaluating the vulnerability of FR systems under some assumptions, high-resolution images can lead to different types of presentation attacks against FR systems. We also propose method for both whitebox and blackbox scenarios and evaluate the transferability of our attack. Similar to (Cole et al. 2017: Duong et al., 2020) [Truong et al. 2022), our method is based on the whitebox knowledge of FR model, however attack using another FR model is novel. Last but not least, we define five different attacks against FR systems and evaluate the vulnerability of SOTA FR models our attacks.
approach for extending our method to the blackbox
4 EXPERIMENTS
In this section, we present our experiments and discuss our results. First, in Section 4.1 we describe our experimental setup. Then, we present experimental results in Section 4.2 and discuss our findings.
4.1 EXPERIMENTAL SETUP
Table 2: Recognition performance of face recognition
To evaluate the performance of our method, we consider two SOTA FR mod- els, including AreFace (Deng et al. 2019). ElasticFace (Boutros et al., 2022), as the models from which templates are leaked (i.e., l'darabss)). For transferabil- ity evaluation, we also use three different FR models with SOTA backbones from FaceX- (Wang et al.] 202T). includ-
models used in our experiments in terms of true match rate (TMR) the thresholds correspond false match rates (FMRs) of 10 and 10 evaluated on the MO- BIO and LFW datasets. The values are in percentage.
MOBIO
LFW
model
FMR=10
FMR=100 FMR=10
FMR=10-
ArcFace
100.00
99.98
97.60
96.40
ElasticFace
100.00
100.00
96.87
94.70
ing HRNet (Wang et al. 2020), Attention-
HRNet
98.98
98.23
89.30
78.43
Net (Wang et al. 2017). and Swin (Liu)
AttentionNet
99.71
97.73
84,27
72.77
al. 2021). for the target FR system (i.e..
Swin
98.98
91.70
Finrger). The recognition performance of
these models are reported in Table All
these models are trained on MS-CelebIM dataset (Guo et al. 2016). We assume that the adversary does not have access to the FR training dataset, and therefore we use another dataset for training our face reconstruction models. To this end, we use the Flickr-Faces-HO (FFHQ) dataset (Karras al.) 2019) which consists of 70,000 high-resolution (i.e., 1024 x 1024) face images (without identity labels) crawled from the internet. We use 90% random portion of this dataset for training, and the remaining 10% for validation.
To evaluate different attacks against FR systems, consider two other face image datasets with identity labels, including the MOBIO (McCool et al. 2013) and Labeled Faces the Wild (LFW) (Huang et al. (007)) datasets. The MOBIO dataset consists of bi-modal (face and voice) data captured using mobile devices from 150 people in 12 sessions (6-11 samples in each session). The IFF dataset includes 13,233 face images of 5,749 people collected from the internet, where 1,680 people have two or more images.
For each of the attacks described in Section 2 we build one or two separate FR systems with one or two SOTA FR models based on the attack type. If the target system is the same as the system from which the template is leaked, we have only one FR system. Otherwise, if the target system is different the system from which the template is leaked, we have two FR systems with two different feature extractors. In each case, we use one of our evaluation datasets (i.e., MOBIO and LFW) to build both FR systems (so that the subject with the leaked template be enrolled in the target system too). In each evaluation, we assume that the target FR system is configured at the threshold corresponding to false match rate (FMR) of 10-3, and we evaluate the adversary's 's success attack rate (SAR) in entering that system.
We should note that the templates extracted by the aforementioned FR models have 512 dimensions. The input noise z E Z to the mapping network of StyleGAN's pre-trained network is from the standard normal distribution and has 512 dimensions. The input noise n E N to our mapping network (.) is with dimension of 8 and also from the standard normal distribution. We also use Adam (Kingma & Ba 20155) optimizer train mapping network.

---

Under review as a conference paper at ICLR 2023
Table 3: Evaluation of attacks with whitebox knowledge of the system from which the template is leaked (i.e., Fluss Fdatauuse against SOTA FR models in terms of adversary's ss success attack rate (SAR) using our proposed method on the MOBIO and LFW datasets. The values are in percentage and correspond to the threshold where the target system has FMR 10 Cells are color coded ac- cording the type of attack as defined in Section 2 for attack 1 light gray ) and attack 2 ( dark gray
MOBIO
LFW
F Fisttalsss
ArcFace ElasticFace
HRNet AttentionNet Swin
ArcFace ElasticFace HRNet AttentionNet
Swin
AreFace 92.38
81.90
71.43
70.48
74.29
86.82
74.20
36.57
36.40
58,86
ElasticFace 78.10
87.62
64.29
64.76
69.05
78.25
82.52
41.80
40.25
61.09
Table 4: Evaluation of attacks (with blackbox knowledge of the system from which the template is leaked i.c., )databaso) against SOTA FR models in terms of adversary's success attack rate (SAR)
using different methods on the MOBIO and LFW datasets. The values are in percentage and corre-
spond to the threshold where the target system has FMR 10-3, M1: NbNetB-N M (Mai et al.
2018 8)
M2: NbNetB-P (Mai et al. (018)) M3: (Dong et al. 202T > and M4: (Vendrow & Vendrow 2021
Cells are color coded according the type of attack as defined in Section for attack 3 lightest gray ),
attack ( middle dark gray ), and attack 5 ( darkest gray
MOBIO
LFW
ddntahose
lass
Fiaget
M1
M2
M3
M4
Ours
M1
M2
M3
M4
Ours
ArcFace
1.90
15.24
2.38
28.10
81.90
10.68
40.25
12.91
58.88
77.16
ElasticFace
1.43
11.43
4.29
15.24
73.81
8.36
34.39
6.35
29.10
68.06
ArcFace
ElasticFace
HRNet
0,95
6.19
2.86
10.00
57.14
1.30
7.78
1.75
9,20
28.45
AttentionNet
0
6.67
3.33
4.29
54.29
1.33
7.17
2.29
9.17
28.87
Swin
1.43
13.33
3.81
10.95
67.14
4.27
23.85
5.97
21.75
48.28
ArcFace
2.38
18.57
2.86
16.19
87.14
15.33
48.65
67
11.81
37.45
83.20
ElasticFace
ArcFace
ElasticFace
3,81
43.81
4.76
43.33
89.05
21.44
58.16
11.59
52.88
83.43
HRNet
0.48
20.00
1.43
10.48
73.81
3.46
18.36
2.74
11.82
49.02
AttentionNet
1.90
18.10
3.33
9.05
71.90
2.89
16.31
2.91
10.95
46.63
Swin
0.95
26.19
2.86
15.24
75.24
9.22
38. 79
8.26
24.62
66.89
4.2 ANALYZE
In this section, we consider SOTA FR models and evaluate the performance of our face reconstruc- tion method in five different attacks described in Section 2 We also explore the effect of our WGAN traning as well as effect of loss terms as our ablation study.
Whitebox knowledge of Faatabuse For attacks 1-2, the adversary is assumed to have whitebox knowledge of the system from which the template is leaked (i.e., Fdarahuse) and use the same feature extraction model for training (i.c., Fioas), thus in such cases Floss = Fdatabase- We considered ArcFace and ElasticFace models and reconstructed face images from the templates extracted by these models in attacks against different FR systems. Table [3 reports the vulnerability of different target systems to our attacks 1-2 in terms of adversary's s SAR at the system's FMR of 10-3, Similar results for the system's 's FMR of 10-2 are reported in Table 6 of Appendix. According these tables, method achieves considerable SAR against ArcFace and ElasticFace target systems in attack 1. In attack 2, we observe that there is a degradation in SAR with respect to attack 1. However, the reconstructed face images can still be used to enter another target system. Meanwhile, the FR model with a higher recognition accuracy is generally more vulnerable to attack 2. For instance, when ArcFace is considered as Fautatuse we observe that FlasticFace and Swin have the highest SAR as target systems, while there is the same order for their recognition performance in Table
Blackbox knowledge of Faatabase For attacks 3-5, the adversary is assumed to have blackbox knowledge of the system from which the template is leaked (i.e., Fdataause) and use another feature extraction model for training (i.c., Flss) ) therefore in such cases Fioss * Fastabosc. Table compares
We should highlight that since there is no whitehox method in the literature with available source code (as mentioned in Table we could compare our proposed method with other whitebox methods.
7

---

Under review as a conference paper at ICLR 2023
the performance of our method with blackbox methods in the literature Mai al. 2018 Dong
et al.
2021 Vendrow & Vendrow! 2021) for attacks 3-5 terms of adversary's s SAR at system FMR of 10 Similar results for the system's FMR of 10 are available in Table of Appendix.
As these tables show, our proposed method achieves the highest SAR compared to (Mai eet al. 2018 Dong et al.] 2021 Vendrow & Vendrow (021) against FR systems the MO- BIO and LFW datasets. In particular, in attack 5 which is the hardest attack, where Fdusabase- Floss and Firget are different, the results show that the target FR system is still vulnerable our attack. The results of our method for attack 5 also show transferability of our attack different FR sys- tems. Similar to attack 2, we also observe that in attack 5, the FR model with a higher recognition accuracy is generally more vulner- able to our attack. Fig. [ also shows sample face images from the LFW dataset and the constructed images using our proposed method from ArcFace templates in different attacks. We should highlight that as show in Fig. 4 the re- constructed face images in attack and attack 2 are the same, but they are used to enter different target FR system. The same holds for the recon- structed face images in attacks 3-5.
Original
Attacks 1-2
0.723
0.682
0.704
Attacks 3-5
0.702
0.650
0.662
Figure 4: Sample face images from the LFW dataset (first raw) and their corresponding re- constructed images using template inversion method from ArcFace templates different at- tacks, attacks 1-2 (second raw) and attacks 3-5 (second raw, using ElasticFace for Fos).. The values below each image show the cosine simi- larity between the corresponding ArcFace tem- plates of original and reconstructed face images.
Ablation Study To evaluate the effect of WGAN in training our mapping network and the effect of each term in our loss function (i.e., Eq. D we consider the AreFace model in the whitehox scenario and train different face re- construction networks with different loss func- tions. Then, we attack a system with the ArcFace model as feature extractor (i.e., attack 1) and compare the SARs as reported in Table [ Ac- cording these results, the proposed adversarial training has a significant effect on our face recon- struction method. In other words, the WGAN framework helps our mapping network to learn the distribution of StyleGAN's intermediate la- tent space to generate face-like images. When
Original
Reconstructed
we use the WGAN training and based on the results in Table [5] the ID loss has high im-
pact on the performance of the template inver-
sion model. While the pixel loss by itself does not achieve a good performance. it improves the performance of ID loss in our reconstruction loss function in Eq. proposed WGAN training and our reconstruction loss function lead to a more successful attack.
0.098
0.126
0.163
Figure 5: Sample failure cases images from the LFW dataset and their corresponding re- constructed images using template inversion method from ArcFace templates the attack 3 (using ElasticFace for Flos). The values below each image show the cosine similarity between the corresponding templates of original and re- constructed face images.
This table confirms that the
Limitations Despite the significant perfor- mance of our method terms success attack rate in all types of attacks reported in Table Jnd Table 4 the reconstructed face images fail to enter the system in some cases. Fig. Fillusttrate sam-
The other blackbox methods in the literature do not have available source code and we could not reproduce their results.
8

---

Under review as a conference paper at ICLR 2023
Table 5: Evaluating the effect of each loss term in our loss function in attack against ArcFace in terms of SAR in the system with FMRs of 10-2 and 10-3 evaluated on the MOBIO and LFW datasets. The values are in percentage.
WGAN training (Eqs. 4 and
Reconstruction
MOBIO
LFW
Loss Function
FMR=10
FMR=10
FMR=10 2
FMR=10
Lee Epinal LiD
100.00
92.38
93.64
86.82
Lie LuD
98.10
82.38
90.56
80.74
Lie ppicel
0
0
0.65
0.07
Gee Epixal LiD
0
0
0.32
0.02
x
Lee LID
0
0
0.14
0.02
fye Ppieet
0.44
0.09
ple failure cases in the attack against ArcFace (using ElasticFace for Floss) on the LFW dataset. From the failure cases, demographies, like elderly or dark skin people. Indeed, such kind of bias in the reconstructed face images caused by inherent biases in datasets used train FR model, the StyleGAN model, and our mapping network in our face reconstruction mode
conclude that there is bias in the face reconstruction for specific
CONCLUSION
In this paper, we proposed a new method reconstruct high-resolution realistic face images from facial templates in FR system. We used pre-trained StyleGiAN3 network and learned mapping from facial templates to intermediate latent space of StyleGAN within a GAN-based framework. We proposed our method for whitebox and blackbox scenarios. In the whitebox scenario, the adversary can use the feature extraction model for training the face reconstruction network; however, in the blackbox scenario, assume that the adversary has access another feature extraction model. In addition, we consider the threat model where the adversary might impersonate in the same or another (i.e., transferable attack) FR system. Based the adversary's knowledge of the feature extraction model and the target FR system, we defined five different attacks and evaluated the vulnerability our proposed method. Our experiments showed that the reconstructed
of SOTA FR systems face images by our proposed method not only can achieve a high SAR in whitebox and blackbox scenarios, but also are transferable and can be used enter target FR systems with a different FR model.
ETHICS STATEMENT
The proposed face reconstruction method is presented with the motivation show- ing vulnerability of face recognition systems to template inversion attacks. We hope this work encourages researcher the community to investigate the next generation of safe and robust face recognition systems and develop new algorithms protect existing systems.
Motivations
Considerations While the proposed method might pose a social threat against unprotected sys- tems, we do not condone using our work with the intent of attacking real face recognition system or other malicious purposes. The authors also acknowledge a potential lack of diversity in the re- constructed face images, stemming from inherent biases of datasets used in our experiments.
Mitigation of such attacks This paper demonstrates an important privacy and security threat to the state-of-the-art unprotected face recognition systems. Along the same lines, data protection frameworks, such as the European Union General Data Protection Regulation (EU-GDPR) (Euro- pean Council 2016). 2016). put legal obligations to protect biometric data as sensitive information. To this end and to prevent such attacks to face recognition systems, several biometric template protection algorithms are proposed the literature (Nandakumar & Jain 2015 Sandhya & Prasad 2017 Kaur
eet al.
2022 Kumar et al. 2020
'The biases for different demographies in verification task for ArcFace model are studied in de Fre-
itas Pereira & Marcel 2021). Similarly, biases in StyleGAN generated images and also the FFHQ dataset
(t.c., training dataset) investigated in (Karakas al. 2022 Tan al. 2020 Balakrishnan 2020

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
In our experiments, we use PyTorcl package and trained our models on system equipped with an NVIDIA GeForce RTX 3090. We use the pre-trained model of StyleGAN to generate 1024 x 1024 high-resolution images. The source code of our experiments is publicly available to help reproduce our results
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pp. 213-223. PMLR, 2017.
Guha Balakrishnan, Yuanjun Xiong, Wei Xia, and Pietro Perona. Towards causal benchmarking of bias in face analysis algorithms. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 547-563. Springer, 2020.
Battista Biggio, Paolo Russu, Luca Didaci, Fabio Roli, et al. Adversarial biometric recognition: A review on biometric system security from the adversarial machine-learnin perspective. JEEE Signal Processing Magazine, 32(5):31-41, 2015.
Fadi Boutros, Naser Damer, Florian Kirchbuchner, and Arjan Kuijper. Elasticface: Elastic margin loss for deep face recognition. Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 1578-1587, 2022.
Peter Burt and Edward H Adelson. The laplacian pyramid as a compact image code. In Readings in computer vision, pp. 671-679 Elsevier, 1987.
Forrester Cole, David Belanger, Dilip Krishnan, Aaron Sarna, Inbar Mosseri, and William T Free- man. Synthesizing normalized faces from facial identity features. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3703-3712, 2017.
Tiago de Freitas Pereira and Sébastien Marcel. Fairness biometrics: a figure of merit to assess biometric verification systems. IEEE Transactions on Biometrics, Behavior, and Identity Science, 4(1):19-29, 2021.
Jiankang Deng, Jia Guo, Xue Niannan, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2019.
Xingbo Dong, Zhe Jin, Zhenhua Guo, and Andrew Beng Jin Teoh. Towards generating high defini- tion face images from deep templates. In 2021 International Conference of the Biometrics Special Interest Group (BIOSIG), pp. 1-11. JEEE, 2021.
Xingbo Dong, Zhihui Miao, Lan Ma, Jiajun Shen, Zhe Jin, Zhenhua Guo, and Andrew Beng Jin Teoh. Reconstruct face from features using gan generator as a distribution constraint. arXiv preprint arXiv:2206.04200 04295, 2022.
Chi Nhan Duong, Thanh-Dat Truong, Khoa Luu, Kha Gia Quach, Hung Bui, and Kaushik Roy. Vec2face: Unveil human faces from their blackbox features in face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1132-1141, 2020.
European Council. Regulation of the european parliament and of the council on the protection of individuals with regard the processing of personal data and the free movement of such data (general data protection regulation), April 2016.
Javier Galbally, Sébastien Marcel, and Julian Fierrez. Biometric antispoofing methods: survey in face recognition. IEEE Access, 1115300-555,,, 2014.
AAaailable at https: / / thub. www/not..... www.bs//atte aaas//sssllllll egan3 'The source code will be available upon acceptance of the paper.
10

---

Under review as a conference paper at ICLR 2023
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb- Im: A dataset and benchmark for large-scale face recognition. In European conference on computer vision, pp. 87-102. Springer, 2016.
Abdenour Hadid, Nicholas Evans, Sebastien Marcel, and Julian Fierrez. Biometrics systems un- der spoofing attack: an evaluation methodology and lessons learned. IEEE Signal Processing Magazine, 32(5):20-30, 2015.
Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. Labeled faces in the wild: A database for studying face recognition unconstrained environments. Technical Report 07-49, University of Massachusetss, Amherst, October 2007.
Yifan Jiang, Shiyu Chang, and Zhangyang Wang. Transgan: Two pure transformers can make one strong gan, and that scale up. Advances in Neural Information Processing Systems, 34: 14745-14758, 2021.
Cemre Efee Karakas, Alara Dirik, Eylül Yalşınkaya, and Pinar Yanardag. Fairstyle: Debiasing style- gan2 with style channel manipulations. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 570-586. Springer, 2022.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for im- proved quality, stability, and variation. arXiv preprint arXiv:1710.10196 2017.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings the the IEEE/CVF conference on computer vision and pattern recognition, pp.
2019.
Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. Advances in Neural Information Process- ing Systems, 34, 2021.
Prabhjot Kaur, Nitin Kumar, and Maheep Singh. Biometric cryptosystems: a comprehensive survey. Multimedia Tools and Applications, pp. 1-56, 2022.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR), San Diego, California., USA, May 2015.
Nitin Kumar et Cancelable biometries: comprehensive survey. Artificial Intelligence Review, 53(5):3403-3446 2020.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. arXiv preprint arXiv:2103.1 /4030, 2021.
Guangcan Mai, Kai Cao, Pong C Yuen, and Anil K Jain. On the reconstruction of face images from deep face templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41()):
1188-1202, 2018.
Sébastien Marcel, Mark Nixon, and Stan Li. Handbook of biometric anti-spoofing, volume Springer, 2014.
Chris McCool, Roy Wallace, Mitchell McLaren, Laurent El Shafey, and Sébastien Marcel. Session variability modelling for face authentication. IET Biometrics, 2(3):117-129, September 2013. ISSN 2047-4938. doi: 10. 1049/itt bmt.2002..059.
Karthik Nandakumar and Anil K Jain. Biometric template protection: Bridging the performance gap between theory and practice. IEEE Signal Processing Magazine, 32(5):88-100, 2015.
Mulagala Sandhya and Munaga VNK Prasad. Biometric template protection: A systematic literature review approaches and modalities. In Biometric Security and Privacy, pp. 323-370. Springer, 2017.
11

---

Under review as a conference paper at ICLR 2023
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 2014.
Shuhan Tan, Yujun Shen, and Bolei Zhou. Improving the fairness of deep generative models without retraining. arXiv preprint arXiv:2012.04842, 2020.
Thanh-Dat Truong, Chi Nhan Duong, Ngan Le, Marios Savvides, and Khoa Luu. Vec2face-v2. Unveil human faces from their blackbox features via attention-based network in face recognition, 2022.
http:aawwi..orggbb.a 220. 04920
Peter JM Van Laarhoven and Emile HL Aarts. Simulated annealing. In Simulated annealing: Theory and applications, pp. 7-15. Springer, 1987.
Edward Vendrow and Joshua Vendrow. Realistic face reconstruction from deep embeddings. In NeurlPS 2021 Workshop Privacy in Machine Learning, 2021.
Fei Wang, Mengqing Jiang. Chen Qian, Shuo Yang. Cheng Li, Honggang Zhang, Xiaogang Wang, and Xiaoou Tang. Residual attention network for image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3156-3164, 2017.
Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao. Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, et al. Deep high-resolution representation learning for visual recognition. IEEE transactions pattern analysis and machine intelligence, 2020.
Jun Wang, Yinglu Liu, Yibo Hu, Hailin Shi, and Tao Mei. Facex-zoo: A pytorch toolbox for face
recognition. In Proceedings of the 29th ACM international conference on Multimedia, 2021.
Andrey Zhmoginov and Mark Sandler. Inverting face embeddings with convolutional neural net- works. arXiv preprint arxiv; 1606.04904 2016.
12

---

Under review as a conference paper at ICLR 2023
A APPENDIX
Table 6: Evaluation attacks with whitebox knowledge of the system from which the template is
leaked (i.c., Fless datahuss) against SOTA FR models in terms of adversary's success attack rate (SAR) using our proposed method on the MOBIO and LFW datasets. The values are in percentage and correspond to the threshold where the target system has FMR= 10-2. Cells are color coded
cording the type of attack as defined in Section for attack light gray ) and attack ( dark gray
MOBIO
LFW
F ddttaabu
AreFace
ElasticFace
HRNet AttentionNet Swin
AreFace
ElasticFace HRNet Attention.et Swin
AreFace
100.00
93.81
80.00
81.90
85.24
93.64
90.89
62.75
76.24
ElasticFace
90.95
93.33
78.57
83.81
84.29
87.88
92.80
71.82
64.24
75.70
Table 7: Evaluation of attacks (with blackbox knowledge of the system from which the template leaked i.e., Fdmaausee against SOTA FR models in terms of adversary's success attack rate (SAR) using different methods the MOBIO and LFW datasets. The values in percentage and
respond to the threshold where the target system has FMR- 10 2 M1: NbNetB-M BM (Mai et al,
2018 M2: NbNetB-P (Mai al. 2018). M3: (Dong al. 202I), and M4: (Vendrow & Vendrow
2021
Cells are color coded according the type of attack as defined in Section for attack
lightest gray attack middle dark gray ), and attack darkest gray )
MOBIO
LFW
F Faatabase
Finss
Furzet
M1
M2
M3
M4
Ours
M1
M2
M4
Ours
ArcFace
26.67
49.05
20.48
67.14
89.52
26.66
61.66
28.31
76.98
87.85
AreFace
ElasticFace
ElasticFace
11.90
49.52
16.19
34.29
86.67
32.42
66.61
23.05
57.84
87.43
HRNet
10.48
24.76
10.00
26.19
79.05
18.69
43.21
17.37
33.55
60.93
AttentionNet
11.43
38.10
24.29
31.88
13.31
53.86
10.48
45.24
10.95
29.52
16.98
38.03
67.80
ArcFace
17.14
49.05
20.95
47.14
95.24
33.08
67.89
26.35
57.48
91.23
ElasticFace
ArcFace
ElasticFace
30.00
70.95
25.7
75.24
94.76
$2.99
81.74
33.53
79.62
93.34
HRNet
8.10
47.14
15.24
31.43
83.81
29.27
60.34
23.22
39.06
76.68
AttentionNet
12.86
47.14
23.43
40.95
87.14
18.53
46.36
17.78
31.53
69.45
Swin
10.00
54.76
13.81
37.14
89.05
24.50
60.19
21.44 40
41.13
80.15
13


Reviews:
[{"review_id": "kDp5ZKHsJ0", "reviewer": "Reviewer_3DHW", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies the template inversion attack against face recognition (FR) systems. It uses results in the literature to train an encoder that generates the latent representation of StyleGAN and, using StyleGAN decoder, produces a realistic face, that is subsequently used to attack a FR system.\n", "summary_of_the_review": "The paper proves how vulnerable are present FR systems to the template inversion attack even if the intruder does not know the model used to produce the template nor the FR algorithm.", "strength_and_weaknesses": "Strength.\n\nThe paper addresses a relevant problem and proves an important vulnerability of present FR systems. It is well written and the experimentation seems correct.\n\nWeaknesses.\n\nMy main complain is the title of the paper, that, in my opinion, is misleading. The main contribution in the paper is using results in the literature to engineer of a set of attacks of an inrceasing degree of difficulty proving in all of them the vulnerability of present FR systems. \n\nThe face generation part is also relevant, but I would attribute the excellent performance of the attacks to StyleGAN's merits. From the experimentation in the paper it is not clear what is the contribution of the adversarial training. In your ablation you only analyze the contribution of (2) and (3), what about (4) and (5)?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "fuCgqZ4l5RH", "reviewer": "Reviewer_bw96", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on template inversion attack against face recognition systems and proposes a method to reconstruct face images from facial templates. In a GAN-based framework, a mapping from facial templates to the intermediate latent space of a pre-trained face generation network is learned, from which high-resolution reconstructed face images can be generated. The proposed method can be\napplied in whitebox and blackbox attacks against face recognition systems. \n", "summary_of_the_review": "The paper is clearly written and easy to read and understand. \n\nIn my evalaution I weighted more the weaknesses and the fact the actual novelty contribution is limited.", "strength_and_weaknesses": "The main contributions of the paper are:\n-\tA method to generate high-resolution realistic face images from facial templates. Within a GAN-based framework, the mapping from facial templates to the latent space of a pre-trained face generation network is learned.\n-\tThe method is proposed for whitebox and blackbox attack scenarios. The method is based on the whitebox knowledge of the face recognition model, and it then extended to an attack blackbox scenario, using another face recognition model that the adversary has access to.\n\nThere are some aspects in the paper that need clarification:\n-\tIn my opinion the novelty of the paper is limited. The proposed method mainly relies on StyleGAN3 for face reconstruction. \n-\tDue to the social security and ethical implications of this works, authors could have described possible solutions to mitigate the effect of the proposed attacks.  \n-\tAuthors did not discuss the results in a good way. It is not clear which are the limitations of the method, and which are the case some reconstructions are not well recognized. No examples are given of such cases. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "fsEM4D6q2nz", "reviewer": "Reviewer_e7hV", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors of this paper propose  new method to reconstruct high-resolution realistic face images from\nfacial templates in a FR system. They focus on the template inversion attack against face recognition systems.", "summary_of_the_review": "The paper is clearly written and easy to read and understand. But the novelty and scope appear limited.  ", "strength_and_weaknesses": "\nStrength\n- The motivation is very clear.\n-  It is well written and the experimentation seems correct.\n- Five different attacks and evaluated the vulnerability of SOTA FR systems to the proposed method are defined.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-f0HJ_3chlJ", "reviewer": "Reviewer_dNW2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, they focus on the template inversion attack against face recognition systems. Within a generative adversarial network (GAN)-based framework,  learn a mapping from facial templates to the intermediate latent space of a pre-trained face generation network. their proposed method achieves high success attack rates in white box and black box scenarios.", "summary_of_the_review": "The paper presented a method for better resolution using Style GAN.\nIts structure and methods looks good and novel, however, more details in mythology and presentation is needed.", "strength_and_weaknesses": "Strength\n1.using a pre-trained StyleGAN3 network and learned a mapping from facial templates to intermediate latent space of StyleGAN within a GAN-based framework\n2. proposed method for whitebox and blackbox scenarios\n3. defined five different attacks and evaluated the vulnerability of SOTA FR systems to our proposed method.\n\nWeaknesses\n1. More details about the method\n2. More details about the attacks\n3. More analysis on white and black box\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject