Make Privacy Renewable! Generating Privacy-Preserving Faces  Supporting  Cancelable Biometric Recognition

Authors: tao  wang
yushu  zhang
xiangli  xiao
lin  yuan
zhihua  xia
jian 2024 weng
  make
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-01 16:14:20

Abstract: The significant advancement in face recognition drives face privacy protection into a prominent research direction. Unlike de-identification, a recent class of face privacy protection schemes preserves identifiable formation for face recognition. However, these schemes fail to support the revocation of the leaked identity, causing attackers to potentially identify individuals and then pose security threats. In this paper, we explore the possibility of generating privacy-preserving faces (not features) supporting cancelable biometric recognition. Specifically, we propose a cancelable face generator (CanFG), which removes the physical identity for privacy protection and embeds the virtual identity for face recognition. Particularly, when leaked, the virtual identity can be revoked and  renew as another one,  preventing re-identification from attackers.  Benefiting from the designed distance-preserving identity transformation, CanFG can guarantee separability and preserve recognizability of virtual identities. Moreover,  to make CanFG lightweight, we design a simple but effective training strategy, which allows CanFG to require only one (rather than two) network for achieving stable multi-objective learning. Extensive experimental results and sufficient security analyses demonstrate the ability of  CanFG to effectively protect physical identity and support cancelable biometric recognition. Our code is available at https://github.com/daizigege/CanFG.

Main paper: 
 introduction: 
 Face recognition has made significant progress in recent years, and is widely applied in various fields, e.g., intelligent surveillance Table 1 : Main advantages of CanFG over mainstream schemes, where PID means physical identity.Li et al. [13] PRO-Face [36] IVFG [37] CanFG and electronic payment. Nevertheless, the large-scale and nonconsensual collection of face images also raises concerns about individual privacy. In 2020, Clearview AI experienced an unauthorized intrusion involving 3 billion face images. Because of the immutable nature of faces, malicious disclosure or misuse can result in irreversible harm to individuals.To protect face privacy, many countries and international organizations issued relevant laws and regulations, e.g., the general data protection regulation (GDPR). Researchers also devotes considerable effort to designing schemes for enhancing face privacy, primarily focusing on generative model-based anonymization [2-4, 10, 14, 21, 33, 35] and adversarial perturbation-based obfuscation [1, 7, 25, 26, 34] . However, almost all schemes aim to remove identifiable information from faces (known as de-identification), thus preventing recognition via machine vision. As a result, face recognition fail to operate properly, greatly diminishing the accuracy and convenience of identity verification.To address the above issue, other schemes preserve identity features for face recognition while anonymizing the facial visual appearance. Li et al. [13] designed a face anonymization model, which can adaptively modify identity-independent attributes to obfuscate visual appearance. Unfortunately, as the number of altered attributes increases, the loss of identity information becomes severe, leading to a significant decrease in recognizability. PRO-Face [36] leverages the idea of data hiding to hide identity information into the obfuscated face. In this way, the protected face is visually similar to the obfuscated one, but the machine vision can distinguish its original identity. The aforementioned schemes rely on physical identity (PID) for recognition, which can be extracted from faces in the real physical world. Nevertheless, PID is unique and immutable (unrevocable), and would be exposed forever once leaked. Attackers can pose permanent security threats to individuals based on the leaked PID, e.g., identity impersonations and reputation violations.Compared to PID, virtual identity (VID) provides stronger security for face recognition. Representatively, IVFG [37] generates faces with different VIDs via key control, where the same PID corresponds to the same VID. IVFG provides satisfactory identifiability and privacy protection, but cannot retain attributes unrelated to identity, thus disabling many vision tasks, e.g., pose detection and background recognition. Additionally, it brings the additional problem of key management and lacks the theoretical guarantee for separability. More importantly, IVFG still fails to support cancelability. Because the VID remains available with the corresponding key, when it should be revoked due to leakages. However, since remembering the leaked VIDs accurately is difficult, the user may mistakenly utilize the leaked VIDs. In such a case, the attacker still can identify the user for potential threatening behaviors such as malicious tracking and privacy invasion.In this paper, we present CanFG, which can generate privacypreserving faces supporting cancelable biometric recognition. Unlike the works of cancelable biometrics [9, 12] which output protected identity features, CanFG focuses on generating protected faces. Fig. 1 illustrates the usage of CanFG. After Alice's face image is protected by CanFG, the PID recognizer cannot determine it as Alice, but the VID recognizer can. In this way, Alice's privacy is protected while face recognition utility is preserved. Furthermore, when the used VID is leaked, it can be revoked and renewed as another VID, avoiding the problem of identity leakage for life. Since the identifiable information (i.e., VID) has been renewed, the attacker can no longer identify users based on this leaked VID, avoiding further security threats. Table 1 shows the main advantages of CanFG over the above schemes.We summarize the main contributions as follows:• We propose a privacy-preserving face generator, i.e., CanFG, which can support cancelable biometric recognition while protecting physical identity. • We construct a distance-preserving identity transformation based on orthogonal matrices, which can ensure separability and maintain recognition performance of virtual identities. • We design a simple but effective training strategy inspired by data hiding, which allows CanFG to require only one (rather than two) network for achieving physical identity removal and virtual identity embedding simultaneously. 
related work 2.1 identifiable face privacy protection: 
 Identifiable face privacy protection means protecting face privacy while allowing face recognition. Privacy-preserving face recognition schemes remove all available content except for identity. Wang et al. [29] discovered that human vision primarily relies on lowfrequency information for image recognition, whereas machine vision focuses on both low and high-frequency information. Building upon this, many schemes [16, 17, 32] remove identity-irrelevant low-frequency features in the frequency domain, ensuring that protected images are used only for face recognition. However, the protected results generated by these schemes lose a significant amount of available information, and thus cannot support simple computer vision tasks, such as face detection.To preserve more utility, recent schemes only change private content while preserving identity features in faces. For private attributes, some schemes [18, 38] are built on attribute manipulation frameworks to invert privacy attributes while maintaining as much identity performance as possible. For private visual identity, some schemes [13, 30, 36] anonymize the face appearance to prevent identity perception by human vision, but retain identity features to allow recognition by machine vision.All of the above works adopt the physical identity (PID) for recognition. However, the PID is unique for anyone and cannot be revoked once compromised. Therefore, recent schemes [31] consider using the virtual identity (VID) to facilitate face recognition. IVFG [37] generates faces with the same VID for different faces with the same PID, where the VID is controlled by the user key. However, this scheme fails to preserve attributes except identity. Furthermore, its simple implementation of separability through inter-group classification loss lacks theoretical guarantees. While Anonym-Recognizer [20] solves the limitations of IVFG, the used binary relationship cyphertext lacks robustness in recognition. Because once one bit goes wrong, the recognition result goes wrong. Moreover, it utilizes two networks for anonymization and recognition, which is not friendly to applications with constrained resources.In biometrics, the unique biometric template can accurately identify individuals but pose significant security risks if compromised. Cancelable biometrics [15, 19, 28] transform the unique biometric template into different templates for identification, enhancing security for biometrics. Specifically, cancellable biometrics must satisfy four characteristics: 1) Non-invertibility: If the protected template is compromised, it is difficult to recover the original biometric sample.2) Comparable performance: Recognition performances before and after protection are comparable. 3) Diversity: The same biometric sample can be protected into multiple templates. 4) Revocability: If the protected template is compromised, it can be revocable and replaced with a new template.Several cancelable biometrics schemes have been proposed and follow the following process: 1) During enrollment, a cancelable template is transformed by some designed transformations for the features extracted from the biometric sample, and then registered in the database. 2) During authentication, the cancelable template is created for the input biometric sample in the same way, and identification is performed by comparing it with the cancelable templates stored in the database. Therefore, the main point is to how to construct the transformation, including filtration [24] , deformation [22] , and encryption [9] . Interestingly, some recent schemes [5, 12] employ data hiding to achieve cancelable biometrics. They guarantee the non-invertibility of the biometric sample by hiding the transformed template into the cover image, because only the specific recognizer can access the transformed template. 
the proposed canfg 3.1 overview: 
 We present a cancelable face generator (CanFG), which can support cancelable biometric recognition while protecting face privacy. As shown in Fig. 1 , given an original face image, the proposed CanFG can generate a protected face image. Firstly, the protected face belongs to an anonymized physical identity (PID), which prevents unauthorized extraction of the real PID, thus protecting face privacy. Secondly, the protected face also is embedded with a virtual identity (VID), which can be extracted by a specific VID extractor and can be revoked when leaked, thus supporting cancelable recognition.As shown in Fig. 2 , the process of utilizing CanFG follows the standard cancelable biometric recognition. In the enrollment, the registered face is protected by CanFG and then stored in the database. In the authentication, the input face is also protected by CanFG, and then the VID extractor obtains the VIDs of the input face and the registered face for verification.Before describing how to train the CanFG, we introduce a new identity transformation and a new physical identity remover. In this subsection, we aim to construct a distance-preserving transformation to transform PIDs into VIDs, i.e., neighboring points in the physical space remain at the same distance in the virtual space after the transformation. There are two main advantages for demanding distance preservation:• Ensuring separability: Separability can prevent different PIDs from being mapped to the same VID, which can avoid recognition errors. IVFG [37] adopts loss optimization to try to achieve separability, but fails to guarantee it theoretically. • Maintaining performance: Since the identity distance can be preserved, the recognition performance in the VID space is fully equivalent to that in the PID space.We select the orthogonal transformation as the distance-preserving transformation. The orthogonal transformation are equidistant and equiangular, thus conforming to the invariance of the Euclidean and cosine distances of the identity features. For any pair of face images, the extracted identity features (e 1 , e 2 ) from them remain the same Euclidean and cosine distances after the orthogonal transformation:EQUATIONwhere Q is an orthogonal matrix, 𝐷𝑖𝑠 (•, •) can be either Euclidean distance or cosine distance. In addition, orthogonal matrices are not unitary. Security for identity transformations can be enhanced by generating random orthogonal matrices. Specifically, different face recognition systems use different orthogonal matrices {Q 1 , Q 2 , ...} so that obtaining different VIDs, thus preventing data-linking attacks between them. That can be formulated as:Q 1 • e 1 ≠ Q 2 • e 1 .(2)In this subsection, we construct a PID remover for assisting privacy protection. Its backbone is set as U-Net [23] . In order to train the PID remover, a pre-trained PID extractor 𝐸 𝑝𝑖𝑑 (ArcFace [6] ) is used to facilitate the adversarial training, which guides the variation of PID through the identity deviation loss:EQUATIONwhere 𝑋 𝑖 is the original face, 𝑋 𝑎 is the anonymized face, and 𝑐𝑜𝑠 (•, •) represents the cosine similarity. However, this identity deviation does not specify a specific optimization direction and thus can easily lead to optimization difficulties. In addition, the uncontrolled protected face may map to other person's identity, causing disturbance to him or her. Therefore, we consider adopting the mean PID features 𝑝𝑖𝑑 of all faces as the direction of optimization, and generate anonymized faces by moving away from the original PID and approaching the mean PID:L 𝑝𝑖𝑑 = 𝑐𝑜𝑠 (𝐸 𝑝𝑖𝑑 (𝑋 𝑖 ), 𝐸 𝑝𝑖𝑑 (𝑋 𝑎 )) -𝑐𝑜𝑠 (𝐸 𝑝𝑖𝑑 (𝑋 𝑎 ), 𝑝𝑖𝑑). (4)In order to reduce the alteration of identity-unrelated attributes, we use the reconstruction loss to retain more usable information:EQUATIONMoreover, adversarial learning is utilized to enhance the clear visual quality of the anonymized results. In particular, we employ WGAN-GP [39] to steady the training process. The adversarial loss is expressed as L 𝑎𝑑𝑣 = L 𝑃 𝑎𝑑𝑣 + L 𝐷 𝑎𝑑𝑣 , where the loss of the remover and the discriminator 𝐷 can be expressed as:EQUATIONandL 𝐷 𝑎𝑑𝑣 = -𝐷 (𝑋 𝑖 ) + 𝐷 (𝑋 𝑎 ) + 𝜆𝑔𝑝 (∥ ∇ 𝑋 ′ 𝐷 (𝑋 𝑎 ) ∥ 2 -1) 2 (7)Finally, the objective for the PID remover is formulated as:L 𝑡𝑜𝑡𝑎𝑙 = L 𝑎𝑑𝑣 + 𝜆 1 L 𝑝𝑖𝑑 + 𝜆 2 L 𝑟𝑒𝑐 (8)where 𝜆 𝑖 is the hyperparameter for balancing losses.Training difficulty. The CanFG can not only protect the physical identity via PID removal, but also support cancelable recognition via VID embedding. An idea to train CanFG is to first PID removal followed by VID embedding. Such idea has been validated in [36] and Anonym-Recognizer [20] , requires two networks to form the CanFG. To minimize resource consumption, a network can be used as the backbone of CanFG for training, but it is difficult for the CanFG to converge well with the multi-task objective, leading to a not satisfactory performance.To solve the above difficulty, we design a simple but effective training strategy inspired by data hiding. Data hiding is a technique for embedding information in digital media without affecting obvious changes. In the case of image media, data hiding can embed information (like text) as a secret into an image (called the cover image), without affecting the appearance of the cover image. Fig. 3 illustrates the training of CanFG. With reference to data hiding, we consider the anonymized face 𝑋 𝑎 obtained by PID remover as the cover image, and the transformed virtual identity as the embedded information. In this way, the protected image 𝑋 𝑝 will have the same appearance as the anonymized face 𝑋 𝑎 while being embedded with the virtual identity. Specifically, the training strategy contains the learning objective and a training trick.3.4.1 Data hiding-based learning objective. We state the learning objective in terms of PID removal and VID embedding.Physical identity removal. Firstly, the protected face should have a different physical identity from the original face, thus preventing recognition by unauthorized face recognizers. For this purpose, we set up a pixel-level loss to reduce the difference between the protected face and the face anonymized by the PID remover:EQUATIONFurthermore, perceptual loss is employed to draw close to the protected face and anonymized face features to enhance the perceptual similarity between 𝑋 𝑝 and 𝑋 𝑎 :EQUATIONwhere 𝑉 𝐺𝐺 (•) is the pretrained VGG-based deep feature extractor.Virtual identity embedding. Secondly, we require a VID extractor 𝐸 𝑣𝑖𝑑 to recognize the virtual identity in the protected face. Specifically, we employ the Euclidean distance to reduce the distances between features:EQUATIONwhere the orthogonal matrix Q is random but fixed in training, and should be discarded after training. For different application systems, the orthogonal matrices they adopted are random and therefore difficult to duplicate. If the VID leaks, an arbitrary system can replace with another random matrix to renew the VID. Similar to PID protector, adversarial learning is used to enhance quality. Finally, the objective for CanFG and VID extractor is formulated as follows:EQUATION)where 𝜆 𝑖 is the hyperparameter for balancing losses. To this end, we introduce a fine tuning-based training trick. Specifically, before training CanFG, the parameters of the PID remover need to be initialized to the CanFG so that it has the ability to generate anonymized faces. In this way, the network structures of CanFG and PID remover need to be consistent, both being U-Net.In this section, we will analyze the reason that why CanFG supports the cancelable biometric recognition.When the protected face is compromised, the attacker can only obtain an anonymized (not real) PID. Such false PID have no obvious correlation with the real PID, so it is difficult for an attacker to recover the real one. More irreversible analysis can be found in the security analysis 4.5. Therefore, CanFG satisfies the requirement of non-invertibility.Since the orthogonal transformation is equidistant and equiangular, the VID space maintains the same distance as the PID space via Euclidean or cosine distances. Therefore, the recognition performance is theoretically completely unchanged before and after the transformation. Since the embedding and extraction of VIDs loses some feature information, making the recognition performance biased but still comparable to the original performance. Therefore, CanFG satisfies the requirement of comparable performance.3.5.3 Diversity. Since orthogonal matrices are not unique and are numerous in quantity, the same PID can be transformed into different VIDs for identification. Therefore, CanFG satisfies the requirement of diversity.3.5.4 Revocability. When VIDs are leaked mainly because of the unsafe protection of VID extractor, the attacker is able to identify individuals based on the leaked VID. To prevent it, we can update the parameters of CanFG and VID extractor by changing the orthogonal matrix, and then renew VIDs while disabling the leaked VIDs, preventing re-identification from attackers. Therefore, CanFG satisfies the requirement of revocability. [37] ) and a representative PID-based scheme (i.e., PRO-Face [36] ) as references to show that the performance of CanFG is comparable. For IVFG, since the training details are not published, we just used the results from the original paper. For PRO-Face, we choose FaceShifter as the obfuscator and select a random face as the target. 
evaluation on privacy protection: 
 The privacy protection goal of CanFG is to prevent the machine recognition of the original physical identity. Qualitatively, Fig. 4 illustrates the protected results generated by CanFG. It can be observed roughly that the protected face has a high visual similarity with the original one, which is contributed by the fact that CanFG modifies mainly the identity-related regions and preserves more irrelevant attributes, e.g., the background and hair. Therefore, compared to IVFG [37] , which changes all image regions, CanFG is more applicable to practical application scenarios such as surveillance video. Upon further scrutiny, we can find some obvious appearance changes in facial areas of the protected face, including the eyes, nose, and mouth. In this way, it is highly probable to visually consider that the protected face and the original face do not belong to the same physical identity. Therefore, the face privacy can be protected well in qualitative analysis.Quantitatively, we utilize an advanced face recognition API (Face++) to evaluate the protection performance. We perform protection on one of the paired faces and later evaluate the identity It is clear that the CanFG obtains the average scores of less than 50% in CelebA and VGGFaces, which is significantly lower than the matching threshold (𝜃 =74.0%). Further, we calculated the protection success rate (PSR) at different matching thresholds, including 62.3%, 69.1%, and 74.0%. As shown in Table 2 , CanFG achieves over 90% protection success rate on the CelebA, as well as a high protection rate on the VGGFace2, which is not seen in training. It should be note that adjusting 𝜆 1 yields a higher PSR but also reduces the degree of attribute retention. Therefore, the face privacy can be protected well in quantitative analysis.In this part, we evaluate the ability of CanFG to preserve recognizability. The orthogonal matrix in CanFG is similar to the key in IVFG [37] , and they are both able to control the change of virtual CelebA VGGFace2 EER↓ AUC↑ EER↓ AUC↑ Original 0.027 0.990 0.074 0.964 IVFG [37] 0.196 0.887 --PRO-Face [36] 0.112 0.954 0.191 0.888 CanFG 0.045 0.988 0.101 0.951 identity. To be comparable, this experiment also requires IVFG to use the same key. Both IVFG and PRO-Face used pretrained Arcface [6] (SEResNet50) for identity extraction, and only CanFG employed a specialized VID extractor (SEResNet50), which guarantees that virtual identity will not be compromised easily. Table 3 shows the recognition performance on protected faces by IVFG, PRO-Face, and CanFG, including equal error rate (EER) and area under curve (AUC). It can be seen that, CanFG achieves the best recognition performance, which only slightly reduces the original performance. This is mainly attributed to the distance-preserving identity transformation.Different PID extractors. Different PID extractors can obtain different identity features, thus affecting the recognition performance. Therefore, we set the PID extractor as various models, including InceptionResNet [27] , IResNet50 [8] , SEResNet50 [11] , and IResNet100 [8] . PRO-Face also supports different models, and thus we compare CanFG with it. Fig. 6 shows the ROC curves on CelebA and VGGFace2. As it can be seen, CanFG reaches AUC values very close to that of the original one in CelebA. Due to the domain discrepancy of the datasets, CanFG does not achieve the same excellent performance on VGGFace as it does on CelebA, but it still maintains high AUC values on all models. Compared to PRO-Face, CanFG also keeps higher AUC values on all models and datasets. Additionally, Different orthogonal matrices. By changing the orthogonal matrix used for identity transformation, protected faces are embedded with different VIDs. We randomly used three orthogonal matrices to train CanFG and test the recognition performance. As shown in Table 4 , CanFG acquires an AUC value of more than 0.986 with different matrices in training, retaining the recognition performance to a higher degree. Moreover, for CanFGs obtained from different orthogonal matrices, they have extremely poor recognition performance in utilizing virtual identities among each other, which prevents data interoperability between different service providers.To demonstrate the effectiveness of CanFG in an understandable way, we visualize the physical and virtual identity features. UMAP is a nonlinear dimensionality reduction technique capable of preserving the local and global Euclidean distance relationship, which can help us to display identity features in 2D space.In Fig. 7 (a), we show the relevant features of 28 face images of the same identity, including the original PID features, the anonymized PID features, the expected transformed VID features, and the actual extracted VID features. It is clear from the observation that the anonymized PID are farther away from the original PID, and hence the PID can be protected well. In addition, the extracted VID nearly overlaps with the region where the transformed VID are located, so that satisfactory recognition performance can be retained.In Fig. 7 (b), we visualize extracted VID features for three different PIDs, each of which has 28 face images. We observe that each PID is assigned to a different VID and that there is significant spacing between VIDs. In addition, we can clearly notice that the distance between VIDs approximates the distance between PIDs. This is due to the fact that UMAP is maintaining the Euclidean distances and is able to reflect the equidistant transformations performed in CanFG. In this way, the separability in virtual identities and the preservation of recognition performance can be theoretically guaranteed.In Fig. 7 (c), we show the feature visualization for the same physical identity protected by different CanFGs. It can be noticed that the VIDs extracted by the corresponding VID extractors are separated from each other. This demonstrates that CanFG supports the availability of non-crossing VIDs for different service providers, avoiding data interoperability between service providers.The goal of the attacker is to obtain the real PID of the protected face. When the protected faces are leaked, because of the satisfactory visual naturalness, the attacker is tricked into thinking that the anonymized PID represented by the protected face is real. Therefore, the attacker has no will to try to recover the real PID. Even if the attacker has the will, it is difficult for him to deduce the true result since the anonymized PID does not have a direct correlation with the real PID. On the basis of the face leakage, we further assume that the attacker has access to additional information.Additional access to CanFG. Since CanFG is irreversible, the attacker cannot recover the original face by CanFG directly. Of course, the attacker can train a reversible CanFG model by obtaining a large number of original and protected face pairs. To enhance security, we suggest adding conditional information to CanFG to generate diverse protected results.Additional access to VID extractor. In this case, the attacker can extract the VIDs via the extractor but still obtain the anonymized PIDs. The failure to access CanFG makes it impossible to capture the correlation between PIDs and VIDs, and thus the real PIDs cannot be deduced. Considering that the VID is leaked, we can revoke it and replace with a new VID by updating the parameters of VID extractor, blocking attackers from using the leaked VID .Additional access to both CanFG and VID extractor. In this case, the attacker can obtain the real PIDs and VIDs pairs. The matrix Q used in orthogonal transformation in CanFG is of 512 dimensions. Firstly, the attacker extracts 512 original PID features Table 5 shows the performance of the ablation experiments, which include recognition performance (EER and AUC), protection performance (PSR), and visual quality (FID and SSIM). Without the data hiding-based learning objective (w/o DH), although the protection performance of PIDs is not affected, it makes the recognition performance and visual quality degrade. Without the fine tuningbased training trick (w/o FT), while the recognition performance and visual quality is improved, the protection performance of PIDs has declined sharply. Therefore, CanFG struggles to converge and fails to achieve superior performance on PID removal and VID embedding when lacking either of the above training strategies.4.7.1 Robustness test. We used Gaussian noise, JPEG compression, and median filtering to evaluate the robustness of CanFG. According to Table 1 , CanFG is least affected by Gaussian noise and most affected by median filtering. It is noteworthy that the AUC values are always above 90%, preserving an available recognition performance. Therefore, the robustness of CanFG is acceptable and also can be enhanced in the future. 4.7.2 Distance preserving test. Although the orthogonal transformation guarantees error-free distance preservation, the results extracted by the VID extractor are still in error from the expected VID. Therefore, we test the absolute errors between the PID distance of the paired faces before protection and the VID distance of the paired faces after protection. As shown in Fig. 8 , both cosine and Euclidean distances of the PID space are well preserved in the VID space with low errors. We randomly select face photos taken with cell phones in real world. As shown in Fig. 9 , the cosine and Euclidean distances of the original PID still can be preserved to a high degree in the VID space. Since the real-world domain is different from the training data domain, the visual naturalness of the protected result is not perfect. In the future, we would like to abbreviate the intra-domain differences so that CanFG can be better applied to real scenarios.  
conclusion: 
 In this paper, we propose CanFG, which can generate privacypreserving faces supporting cancelable biometric recognition. Particularly, when leaked, the used virtual identity can be revoked and renewed as another one. We also introduced a distance-preserving identity transformation, which guarantees the separability and high recognizability of virtual identities. To stably train CanFG with a multi-objective loss, a new training strategy based on data hiding is designed. Sufficient experimental analyses and security analyses demonstrate the effectiveness of CanFG in protecting face privacy and maintaining high recognizability. 
: 
 recognizer cannot determine it as Alice, but the virtual identity (VID) recognizer can. Furthermore, when compromised, the used VID can be revoked and renewed as another VID. 
 
 Appendix:  


Decision: This paper proposes an interesting face generation scheme, which produces faces with arbitrary, yet unique, identities which can be revoked and re-instatiated at the user's choice.
All reviewers agree on the strenghts and merits of the research work and outcomes. Many issues raised by the reviewers have been well addressed and fixed by the authors. At least one of the reviewers upgraded her/his score.
Apart form some minor concerns, most of the issues have been clarified and the paper can provide a valuable addition to the conference program.

Reviews: review: paper_summary: paper_summary: The paper introduces a cancelable face generator (CanFG) that aims to achieve cancelable recognition, providing non-invertibility, comparable performance, diversity, and revocability. CanFG adopts a distance-preserving transformation to ensure separability and maintain performance, and an auxiliary physical identity remover for identity anonymization. The authors also design a training scheme to reduce resource requirements with a data hiding-based training strategy. The paper provides ample experiments and detailed analysis on the performance of CanFG, proving its effectiveness.
strength_weakness: strength_weakness: 1. The proposed system has quite a solid theoretical ground. Analysis of each module, especially the distance-preserving transformation and the auxiliary physical identity remover provides an explicit and clear description of the idea. 
2. The objective of minimizing resource consumption is commendable. By trying to avoid excess and heavy networks, the authors manage to control the size of the system with some tricks on the training strategy, which highly improves the practicability.
3. Abundant experiments on diverse setups prove the effectiveness of CanFG. They are well designed, covering many different aspects and therefore showing that CanFG empirically delivers on the promise of cancelable biometric recognition.
limitations: limitations: 1. Writing can be further improved. Some sentences in the introduction, for example, could be reorganized and rearranged so that the key contributions of CanFG are more emphasized. For instance, it might be better if the narrative of the four requirements of a cancelable biometric scheme is more explicitly presented when the authors introduce existing studies, including PID ones and VID ones, since this seems to be the important concept throughout the whole work.
2. There should be some descriptions of the actual resources spent on training CanFG in Section 4 since the training strategy is intentionally designed to reduce resource consumption. It would be better if the authors compared the resource usage between CanFG and the baselines.
3. In equation (10), there seems to be a function VGG(·) without explanation.
4. The sentences in Lines 404 – 436 are a little confusing. 
5. Some graphs are a bit difficult to read. The font size of Figs 5 – 8 is too small. Words in Fig 7 are extremely hard to read. Figs 1 – 4 are much better.
paper_summary: paper_summary: The paper presents a Cancelable Face Generator (CanFG) that generates privacy-preserving faces while supporting cancelable biometric recognition. CanFG removes physical identity for privacy protection and embeds virtual identity for recognition. The embedded virtual identity can be revoked and renewed if leaked, thus maintaining privacy and utility. The paper discusses the design and training of CanFG, as well as extensive experimental results and security analyses demonstrating its effectiveness.
strength_weakness: strength_weakness: 1. **Robust Experimental Results**: The extensive experimental evaluations cover various aspects such as privacy protection, recognizability preservation, robustness, and real-world applicability. The results demonstrate that CanFG achieves high privacy protection and recognizability preservation while being robust against various attacks.
2. **Security Analysis**: The paper includes a detailed security analysis, demonstrating that CanFG meets the requirements of non-invertibility, comparable performance, diversity, and revocability. This comprehensive analysis strengthens the validity and reliability of the proposed method.
limitations: limitations: 1. **Limited Dataset Diversity**: The experimental evaluations primarily focus on CelebA and VGGFace2 datasets. Evaluating CanFG on a broader range of datasets, e.g. FFHQ or CASIA-Webface, including those with more diverse demographics and environmental conditions, would enhance the generalizability of the results.
2. **Real-World Application Scenarios**: The authors could provide more detailed insights into how CanFG can be integrated into existing facial recognition systems and the potential challenges associated with its deployment in real-world scenarios.
3. **Implementation Complexity**: While the methodology is comprehensive, the implementation of CanFG may be complex and resource-intensive, especially for deployment in real-world scenarios with limited computational resources. Please list the comparison between CanFG and other methods in terms of the number of parameters and running time.
4. **Limited Experiments on Privacy Protection**: In Section 4.2, authors should give comparison of the state-of-art methods and their method on PID protection success rates. Moreover, the authors choose 3 threshold in the quantitative evaluation of PID protection success rates, please explain the reasons for choosing these thresholds.
paper_summary: paper_summary: This paper proposes CanFG. Its goal is to generate protected faces, carrying virtual identities, that can be recognized by a VID extractor and cannot be by any PID extractors. The authors employ a two-stage framework, first to train a PID remover that anonymizes the face, then to end-to-end train a pair of CanFG and VID extractor models to craft protected faces by embedding obfuscated IDs into the anonymized faces. The ID is revocable by opting for new CanFG and VID extractor models.
strength_weakness: strength_weakness: - Interesting research topic: Face protection is an important topic and is worth further exploration. Though cancelable biometric templates have been common in prior literature, the authors extend the goal to the creation of identity-cancelable faces, which is insightful.
- The proposed method seems to achieve its protection goal (with a premise of no further attacks) together with good performances.
- The experiments are extensive.
limitations: limitations: - I have concerns about some deliberate attacks. Though $X_i$ and $X_p$ are recognized by the PID extractor as different persons, (1) their visual appearances are still quite close as in Fig. 4, and (2) their relations are somehow "deterministic" (say, $X_p = E_{vid}^{-1}(Q\cdot E_{pid}(X_i))$, where $E_{vid}^{-1}$ is a fictional mapping and doesn't have to exist). My concern is, that a white-box attacker having access to the CanFG U-Net model can generate many $(X_i, X_p)$ pairs with a shared label $y$ (e.g., Alice). It is also within the attacker's capability to retrain a new (let's call it "PVID") extractor using both $\{X_i, y\}$ and $\{X_p, y\}$ as training data. The PVID extractor may incorporate both $(X_i, X_p)$ and recognize them as the same person. In short, retraining a new arbitrary extractor on both real and protected faces may nullify the virtual identity. This is my main concern. I invite the authors to consider this "retraining extractor" setting in further experiments.
- Similarly, the authors could also consider a generative attacker, that trains a U-Net to reversely fit $X_p$ to $X_i$.
- The effect of some loss terms (e.g., $\mathcal{L}_{adv}$) is unclear in the manuscript. The authors should consider further explaining them.
- The meaning of similarity in Fig. 5 is also unclear. Does it mean cosine similarity, the angle between two templates, or the percentage of angles? That can make a huge difference.
- The paper's writing can be improved.

I invite the authors to further elaborate on these issues. I am open to improving my score should my concerns be clarified.
paper_summary: paper_summary: This paper introduces a cancelable face generator (CanFG), which removes the physical identity (PID) for privacy protection and embeds the virtual identity (VID) for face recognition. CanFG can revoke and renew the VID when it is leaked. Compared with existing schemes that rely on PID for recognition, CanFG uses VID to address the problem caused by the uniqueness of PID. More importantly, CanFG can update parameters to renew VID for cancelable recognition.

The method of CanFG consists of distance-preserving transformation (DPT) and an auxiliary physical identity remover (PIDR). In DPT, orthogonal transformation is adopted to map PID to VID, ensuring separability and maintaining the recognition performance of the PID space in the VID space. In PIDR, the anonymized face generated by PIDR is considered the cover image, and CanFG is initialized with the parameters of PIDR. The author considers that CanFG and PIDR have the same network structure in a fine-tuning-based training trick. The author also analyzes why CanFG supports cancelable biometric recognition in four aspects.  CanFG is evaluated on privacy protection and recognizability
strength_weakness: strength_weakness: This paper innovatively addresses the problem of achieving cancelable biometrics recognition while preserving privacy and provides a suitable solution. This work fills the gap in the joint analysis of revocability and privacy protection in face recognition.

The authors designed CanFG to generate protected faces, considering that current works on cancelable biometrics only output protected identity features while ignoring identity-independent attributes (e.g., hair, background), which can be used for other vision tasks. This approach fully considers the potential for practical application.

The authors use a fine-tuning trick to initialize the parameters of CanFG with those of the PID remover, which is beneficial for model training speed. CanFG can produce anonymous faces, making this training trick valuable when working with large amounts of image data.

In the experimental section, the authors conducted qualitative and quantitative analyses of privacy protection. They also replaced different modules (i.e., PID extractors, and orthogonal matrices) in CanFG to perform comparative experiments on recognizability preservation. The method of verifying the performance of the proposed technique is highly useful for reference.
limitations: limitations: 1. In the orthogonal transformation of PID to VID, the Euclidean distance and cosine distance can be well kept unchanged. However, a simple linear transformation will make it easier for the attacker to crack.(See specific points in the justification section)
2. In the ablation experiment, the influence of different sub-training strategies on privacy
protection, recognition effect, and visual quality is analyzed. However, the lack of
experimental results based on the original model without any sub-strategies may lead to a less rigorous analysis.
3. In section 3.3, there is a lack of definitions of symbols and functions related to adversarial learning.
4. In section 3.4.1, there is no description of the definition of the VGG mapping when computing the perceptual loss.
5. The expression of the learning objective loss of Auxiliary Physical Identity(PID) Remover is rough, and the module structure of PID remover is not clear.
6. In this paper, some unique names (i.g., data hiding and cover image) lack the necessary explanation, which can easily cause misunderstanding and confusion.


Justification For Recommendation And Suggestions For Rebuttal:
1. In contrast to cacelable biometrics works which only output identity features, CanFG can generate protected faces. Although readers can get the why need to generate the full face instead of the identity features from the full-text understanding, I believe that pointing out the advantages of doing so when expressing this method is helpful for readers to understand this paper.
2. In section 3.3 ,the symbols and functions related to adversarial learning, i.g., $D(\cdot), \lambda, g, p$, need to be defined. Then,in section 3.4.1, the definition of $V G G(\cdot)$ should be given. It can reduce the confusion of readers.
3. In the PID remover, the learning objective loss is expressed as a combination of identity deviation loss, reconstruction loss and adversarial loss, but I'm a little confused about expressing it this way. If the author can add the structure diagram of PID remover in the paper, it will be more helpful to understand the definition of the learning objective loss and the workflow of PID remover.
4. The authors propose an effective training strategy for GANFGs inspired by data hiding. However, there is no brief explanation of the concept of data hiding in the full text, which makes me confused. in addition, there is no necessary explanation for the 'cover image' in line 441, which is easy to cause confusion. I hope the author can supplement the explanation.

5. In the transformation of PID to VID, orthogonal transformation has good invariance performance. However, since the linear transformation is easy to crack, a small bias can be added to enhance the security of the transformation with as little impact on the orthogonal characteristics as possible. The authors can consider this situation and implement comparative experiments.
6. In ablation experiments, authors could consider adding an experimental result based on the original model without adopting any training strategy. Based on this, the paper can better analyze the improvement ability of using different sub-training strategies for the original model.

Minor:
The Eq.(1) in line 324: $\left.\operatorname{Dis}\left(Q \cdot e_1, Q \cdot e_2\right)\right)=\operatorname{Dis}\left(e_1, e_2\right)$. There is an extra right parenthesis on the left side of the equation.