FracFace: Breaking the Visual Clues—Fractal-Based Privacy-Preserving Face Recognition
Authors: Wanying Dai
Beibei Li
Naipeng Dong
Guangdong Bai
Jin Song Dong
Venue: NeurIPS 2025 poster ()
Keywords: Privacy Preserving
Facial Recognition
Fractal
Visual Privacy
Defending Reconstruction Attack
Primary area: applications
TL;DR: This work uses fractal features to reduce visual clues that are critical for privacy leakage while preserving high face recognition accuracy.

Abstract: Face recognition is essential for identity authentication, but the rich visual clues in facial images pose significant privacy risks, highlighting the critical importance of privacy-preserving solutions. For instance, numerous studies have shown that generative models are capable of effectively performing reconstruction attacks that result in the restoration of original visual clues. To mitigate this threat, we introduce FracFace, a fractal-based privacy-preserving face recognition framework. This approach effectively weakens the visual clues that can be exploited by reconstruction attacks by disrupting the spatial structure in frequency domain features, while retaining the vital visual clues required for identity recognition. To achieve this, we craft a Frequency Channels Refining module that reduces sparsity in the frequency domain. It suppresses visual clues that could be exploited by reconstruction attacks, while preserving features indispensable for recognition, thus making these attacks more challenging. More significantly, we design a Frequency Fractal Mapping module that obfuscates deep representations by remapping refined frequency channels into a fractal-based privacy structure. By leveraging the self-similarity of fractals, this module preserves identity relevant features while enhancing defense capabilities, thereby improving the overall robustness of the protection scheme. Experiments conducted on multiple public face recognition benchmarks demonstrate that the proposed FracFace significantly reduces the visual recoverability of facial features, while maintaining high recognition accuracy, as well as the superiorities over state-of-the-art privacy protection approaches.

FracFace: Breaking the Visual Clues-Fractal-Basee Privacy-Preserving Face Recognition
Wanying Dail Beibei Li"
Naipeng Dong² Guangdong Bai², Mair,Jn Jinn Song Dong*
'Sichuan University The University of Queensland City University of Hong Kong
*National University of Singapore libeibei @ scn.edu.en, n.dong euq.eduuaa
daiwanying Stt.yyiin@@ssss.stee.o scu.uu.u. bai @ wiw...ccctuuuthteeeeo desdjs @ nus.edu.sg
Abstract
Face recognition is essential for identity authentication, but the rich visual clues in facial images pose significant privacy risks, highlighting the critical importance of privacy preeaccipreeiiinngg.... solutions. For instance, numerous studies have shown that generative models are capable of effectively performing reconstruction attacks that result in the restoration of original visual clues. To mitigate this threat, we introduce FracFace, a fractal-based work. This approach effectively weakens the visual clues that be exploited by reconstruction attacks by disrupting the spatial structure in frequency domain features, while retaining the vital visual clues required for identity recognition. To achieve this, we craft Frequency Channels Refining module that reduces sparsity in the frequency domain. It suppresses visual clues that could be exploited by reconstruction attacks, while preserving features indispensable for recogmition, thus making these attacks more challenging. More significantly, we design Fre- quency Fractal Mapping module that obfuscates deep representations by remapping refined frequency channels into a fractal-based privacy structure. By leveraging the self-similarity of fractals, this module enhances both recognition performance and defense strength, thereby significantly improving the overall robustness of the protection scheme. Experiments conducted on multiple public face recognition benchmarks demonstrate that the proposed FracFace significantly reduces the sual recoverability of facial features, while maintaining high recognition accuracy, as well as the superiorities over state-of-the- privacy protection approaches.
face recognition frame-
Introduction
Face recognition (FR) leverages distinct facial features for biometric identification and is increasingly integrated into security applications such as mobile unlocking, access control, and border security. With the growing deployment of face recognition systems, privacy concerns have intensified, as intricate visual details in facial images may serve rich clues for potential attackers. To tackle these issues, crivacy-preservin
face recognition (PPFR) has emerged as a solution. PPFR protects personal data while retaining the essential functionality of recognition systems by modifying facial data to prevent the reconstruction of the original image, thus balancing privacy with practical utility.
Existing PPFR schemes are generally classified into two categories: cryptographic approaches [13, 19, 23, 25, 8, 52], and non-cryptographic approaches [43, 27, 3. 14, 42, 21J. Cryptographic methods aim to secure facial data by encrypting features or performing recognition within the encrypted data. While these methods have solid theoretical foundation, they are often hindered
"Corresponding Author. 'The School of Cyber Science and Engineering, Sichuan University.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

by high computational costs, limited data usability, and challenges in scaling effectively for real- world applications [44, 57]. Recent cansformation-baseed, con-cryptoga PPFR methods achieve low latency and computational efficiency by suppressing facial visual details [29]. However, the inherent connection between identity and appearance features makes it challenging to balance recognition accuracy with privacy protection, often leaving residual visual clues that can be exploited in reconstruction attacks, with privacy risks remaining [15, 50]. To mitigate these issues, the precise selection of frequency domain channels has become essential. Existing approaches typically retain only those most strongly correlated with identity recognition, aiming to mitigate the suppression of high-frequency features by low-frequency channels and to emphasize the contribution of high- frequency components [28, 15]. However, we observe that the root of this issue lies in the inherent sparsity of the frequency domain, where ceentity-relatee information appears only in few dominant components, sparsely scattered across both low and high frequency channels [20]. If this sparsity is not adequately addressed, there remains risk of inadvertently leaving behind exploitable visual clues for potential attackers.
To overcome these challenges, propose FracFace, a novel privacy- Srivacy-rreservngg face recognition framework built on two core components designed to refine frequency domain processing and disrupt potential visual cues. To address privacy issues arising from frequency domain sparsity and residual visual cue leakage, we present the Frequency Channel Refining (FCR) module, which selectively attenuates frequency bands to significantly diminish visual cues unrelated to identity while preserving key features essential for recognition. Furthermore, to enhance defense against reconstruction attacks, propose the Frequency Fractal Mapping (FFM) module. which remaps the refined frequency representation to a fractal structure space, disrupting the continuity between spatial and frequency channels. By introducing structured perturbations rather than random noise, FFM fundamentally obfuscates the visual cues. The combined effect of these two modules reduces visual sparsity and enhances resilience against reconstruction attacks by generative networks, strengthening privacy protection. This paper makes the following contributions:
First, we propose a novel fractal based framework for privacy preserving face recognition that disrupts spatial regularities in the frequency domain, thereby suppressing visual clues that can be exploited by reconstruction attacks.
Second, we present a systematic frequency channel refining method that reduces sparsity and suppresses non-identity features, and introduce an innovative use of fractal structures to disrupt frequency continuity and enhance reconstruction resistance.
Third, FracFace improves attack resistance by 15% to 60% under both white box and black box reconstruction scenarios compared existing privacy preserving methods.
2
Related Works
Face Reconstruction Attacks. Face reconstruction attacks, especially those using deep learning, present a significant privacy challenge. These attacks are typically classified into optimization-base based [36, 37], and deep bearning-based methods[35, 34]. While optimization-ban based attacks iteratively refine inputs using feedback from face recognition systems, deep baaming-based attacks improve efficiency by learning inverse mappings from facial features to images. U-Net [38] has become a widely used model for image reconstruction, attributed its effective encoder-decoder structure and high reconstruction quality. Early studies by Zhmoginov et al. [56] and others utilized deep neural networks convert facial embeddings into high-quality images, while more recent approaches like those by Cole et al. [4] and Dosovitskiy et al. [7) focused on generating images directly from these embeddings. In deep learning- based attacks, StyleGAN [17, 18] with its stylized generator architecture, enables precise control over facial feature reconstruction. StyleGAN's ability to generate face images nearly indistinguishable from the original poses significant privacy risks. Applications like TediGAN [47] and LAFITE [58] further enhance face generation and editing by manipulating the latent space, demonstrating high fidelity in reconstruction. Beyond GAN-based inversion, diffusion models [49, 16) provide powerful priors that enable adversaries to plausibly recover dlentity-bearing details. Specially, PGDiff [49] enforces high-quality structural and color priors during reverse diffusion, enabling robust reconstructions under complex or composite corruptions and thereby amplifying privacy risks. These advances highlight the growing efficiency of deep learning attacks and the pressing need for stronger privacy protection solutions.

---

Hall Fre
Hog Are
0.110
139
05675
0.1000
0.5425
100
180
0.5000
6.1885
0.330
0.5225
(a) Unrefined Frequency Spectrum
(b) Frequency Channel Interference
(c) Refined Frequency Channel
(d) Refined Frequency Spectrum
Figure 1: Refining targeted frequency channels: An illustration
Face Recognition. FR has become a predominant biometric modality for identity authentication. While state-of-the-an methods [48, 6, 40] achieve remarkable accuracy, the increasing risk of privacy leakage has brought growing attention to privacy-preserving face recognition [28]. The development of deep neural networks [30] and generative adversarial models [7] has greatly exacerbated the risk of reconstruction attacks. Recent advances in PPFR have shown promising progress, with existing methods broadly categorized into cryptographic and non-cryptographic approaches. 1) Cryptographic techniques, such as homomorphic encryption (HE) [12], differential privacy (DP) [33], secure multi-party computation (MPC) [1], and cancelable biometrics (CB) [5] offer robust theoretical privacy guarantees. However, their real-world applicability is hindered by substantial computational overheads, particularly when deployed
cnourrce-constrinee edge devices [27]. The substantial computational demands of these methods introduce significant delays, rendering them impractical for real-time face recognition systems where both speed and efficiency are critical [15]. Consequently, while these cryptographic solutions promise strong privacy protection, their practicality and scalability in real-world applications remain significant challenge. 2) Non- cryptographic approaches, especially cransformation-basse techniques, have been actively explored as lightweight alternatives to cryptographic schemes. These methods apply deliberate transformations in the image or feature domain to obscure sensitive attributes while preserving cues that are essential for distinguishing identities. To achieve privacy protection, Wang al. [43] employed channel-wise shuffling and mixing strategy in the frequency domain. Mi et al. [27] extended this view, emphasizing the complementary role of diverse frequency bands and advocating their joint use. Building on this, Mi et al. [28] randomly selected and filtered a subset of trequency-domain channels to reduce visual face images by reconstructing frequency features from the residual between the original and generated frequency domains. et al. [14] introduced a learnable privacy budget for adaptive trade-offs between privacy and utility, while Yuan et al. [50] devised obfuscation method operating in the frequency domain that balances attribute suppression and visual interpretability. Most recently, Jin et al. [15] retained two key frequency channels and employed gradient-based reconstruction to synthesize structurally complex features resistant to inversion.
cues. Later, they proposed subtraction-baan
method [29], which generates
Nevertheless, despite their practical effectiveness, these methods remain vulnerable to advanced reconstruction attacks. Zhang al. [51] suggested that many approaches rely on unrealistic privacy assumptions, which compromise their reliability. Shahreza et al. [32] further demonstrated that generative models can learn mappings from facial templates to latent spaces of pretrained generators, enabling high-resolution face reconstruction. In addition to this, our evaluations show that models like U-Net [38] and StyleGAN [18] can still recover realistic images even with partial frequency corruption (see Sec. 4.3 for more details). This stems from core issue: although visual cues can be explicitly suppressed, the inherent sparsity of frequency-domain features often preserves global structures and local consistencies, allowing generative models to reconstruct the obscured content by exploiting residual information and frequency patterns (see Fig. 1 and Sec. 4.3 for more details).
3 Methodology
This section presents the technical design of FracFace, that fundamentally reduce privacy leakage in frequency-based face representations by targeting not only visual cues but also the structured patterns embedded in frequency embeddings. The overall pipeline is illustrated in Fig. 2.
3

---

Prequency Fractal Mapping
Imagen
Prectal Lattiee
Ideatity
Truasfarmm
Practal Structursi
Peivacy Map
Databure
Refinel Froquency Congenest
Fracial Geid
Prequency Refining
Prusing Space
Medel Training
Reprosentaioon
Ideatity Privacy Preservation
Frequency Projection
Prequmes Band Atsomaation
Hotain Dieriminative Ideatity Peaturna
The pipeline of proposed FracFace. FracFace first projects input face images [3,H, W] into frequency domain representation [192,H,W] via BDCT. Subsequently, band attenuation is applied to selectively suppress redundant frequency channels, obtaining the shape of [162, H, W]. These features are then refined through FCR to extract privacy-protecter subset of shape [81, H, WJ. FFM is subsequently applied disrupt spatial regularities, generating fractal identity tokens that stored for recognition while enhancing resistance to reconstruction attacks.
Figure 2:
3.1 Overview
Unlike previous work that mainly suppresses facial visual cues, FracFace introduces novel transformation perspective that disrupts spatial regularities while preserving identity discriminative features. Such structural-level protection maintains reliable recognition performance and substantially mitigates the risk of visual reconstruction, thus achieving strong balance between privacy preser- modules. FCR employs a refined selection
vation and recognition accuracy. We present of frequency domain channels to reduce feature sparsity and mitigate visual cue leakage, while preserving coentity-disrimminate
information. FFM transforms frequency domain features into fractal space characterized by self-similarity and local chaos, enhancing irreversibility without compromising recognizability. These modules operate two levels, channel selection and structural mapping, embedding frequency-domate features into a fractal- based framework to strike an optimal balance between privacy preservation and recognition accuracy under constrained channel conditions.
3.2 Frequency Projection
Given an input facial image 1 E R3xH>W we first normalize it into the [0, range and apply an upsampling operation with scaling factor obtain I', The upsampled image I' then transformed into the YCbCr color space, yielding Y E 333rHxxwW which better separates luminance and chrominance information. Next, Y partitioned into non-overlapping Block-based Discrete Cosine Transform (BDCT) is independently applied to each block, resulting in localized frequency representation S e xxxx s x " This transformation decomposes local spatial structures into 64 orthogonal frequency components, ordered from low to high frequency. For each channel, we denote the set of low-frequency components as Fi S and the high-frequency components as Fh C S, where Fj F₁ S and F n Fr = 0. This decomposition into refined frequency bands lays the groundwork for subsequent privacy- shreacc-preserving mechanisms.
blocks, and
3.3 Frequency Channels Refining
Frequency Band Attenuation After mapping the image to the frequency domain, the image features exhibit significant sparsity. As illustrated in Fig. la. the low- cow-frequency channels contain the majority

---

of the energy and visual information. However, this also results in components associated with low frequencies retaining substantial characteristics of the lower frequency bands, thereby suppressing the contribution of channels in the higher frequency range, as shown in Fig. 1b. To address this issue, we propose the Frequency Band Attenuation (FBA) mechanism, which selectively weakens the contribution of channels corresponding to low frequencies and reshapes the frequency features to better balance privacy protection and identity representation. Specifically, we adopt a three level pruning strategy, where frequency channels are grouped into sets Pt. P2, P3. each corresponding to distinct frequency range. The final pruning space is defined as P = Pi U P/ U P3. Next, we perform the channel removal operation the frequency domain feature S, resulting in a refined frequency channel set S (S, P), where S E TR all x = represents the feature tensor after frequency band attenuation, with redundant frequency components removed, effectively reducing the risk of privacy leakage. The channel selection removes those associated with sensitive visual information, such as skin color, lighting, and expressions, while retaining those crucial for identity recognition with low visual sensitivity ensure accuracy. After applying the FCR mechanism, as shown in Fig. Ic, the refined low-frequency channels no longer suppress high-frequency contributions, and instead, as seen in Fig. 1d, each channel independently highlights its unique contribution, leading to a more balanced and robust feature representation. We also utilize t-SNE [24] to evaluate the sparsity of frequency domain channels (see detailed analysis refer to Appendix A. Unlike Mi et al.'s 's PartialFace [28], our frequency band attenuation strategy balances identity recognizability and privacy protection, addressing frequency sparsity and weakening visual cues exploitable by
reconstruction attacks.
Retain Discriminative Identity Features To further refine the frequency band structure, the at-
tenuated frequency domain channels partitioned into two groups: (S(1, (( index(S"). where (()), (()) 3x3881x s/ zw This strategy allows the assignment of different priorities the frequency bands, enabling more granular privacy control (see Algorithm 1 for details).
3.4 Frequency Fractal Mapping
In our proposed FFM mechanism, the fractal kernel Mo and the fractal lattice Lo are randomly initialized, serving as the foundational components for subsequent fractal transformations. The fractal kernel Mo and the fractal lattice Lo. both of size m x n, are initialized such that Mo is populated with random integers drawn from the range [1, x ri]. introducing structural diversity, while Lo defines the relative indexing order of the elements within the fractal kernel My. The initial fractal grid Fu is directly set as Mo, thus initializing the fractal mapping. The fractal transformation proceeds iteratively, where at each step, a new fractal mapping FR is constructed by combining the previous mapping FR-1 with the foundational mapping Mo, scaled by factor 3g = 32* This scaling factor controls the expansion of sub-blocks within the fractal structure, and each element of the new mapping is updated as:
Fali (Mol..)) - 1) B& FR-1
(1)
where i and index the mapping elements. This iterative process progressively refines the fractal structure, with each layer increasing in both size and complexity. To maintain structural coherence across scales, correction F& Fx 1 is applied at certain layers to mitigate misalignments introduced by scaling. The process continues for a predefined number of iterations, ultimately yielding set of fractal mappings F FI,, Fn-1)- The FFM constructs multi-layered and complex fractal structure that effectively mitigates sparsity while preserving essential identity information. Building upon the recursive fractal construction, the proposed FFM forms a non- invertible, nonlinear index transformation through iterative integer-based perturbations, as formally defined below. Let Mo zexn denote the initial index matrix. We define the k-th layer fractal mapping recursively as:
Fl Mo + Σ (b, 1).
B,
II
by
B1 1
where by is the expansion factor at the i-th fractal layer, and F(h] denotes the positional encoding mapping for channel reordering. This recursive structure highlights the layered composition of FFM, in which discrete integer perturbations are progressively accumulated in a nonlinear fashion
5

---

Table 1: The performance of privacy protection methods in terms of face recognition accuracy. The space-time domain is denoted by S. the frequency domain by Fi, and the fractal domain by F2. respectively. Green indicates the proportion at the given rrvacc-protection level (see Appendix A.3)
Mathed
LaW
ColobA
AgeDU
C3P-PP
CALEWW
CPLPW
U1B-6
UB-C
Domain
Protectoo
Vanc
(56)
1c.4
1e.4
Archoee (6]
59:75
95.35
97.99
19h85
95.89
90.59
94.81
91.95
95.09
92.41
CNPR-2019
Auctuce [48]
99.81
95.45
97.38
95.03
93.65
94.92
CVPR
WYR--F
59.39
93.69
07.99
95.55
95.69
90.02
95.67
91.13
94.73
92.40
AAAH-M22
Doetiace [27]
59.81
92.13
96.17
91.24
95.18
92.19
92.63
90.52
95.25
94.16
P1
ACMMM 2022
[16]
1OUV-2075
Minustace (29)
99.79
95.89
90.05
96.94
95.83
92.89
93.89
93.51
95.91
94.9%
P1
CNPR
PRCI-Fare (50)
09.20
91.60
03.79
95.63
90.65
83.27
90.89
IFFR TIFS-3024
PaceUMfaccaoo (15)
59.70
14.50
96.82
90.84
94.43
USINIX 2004
FraePace (ouna)
59.69
95.91
00.76
96.14
93.92
99.15
92.42
90.73
94.09
92.36
F1 F2
NErulP8--105
through scaling and base multiplication. To establish the nonlinearity of the FFM transformation, we demonstrate in Appendix A.2 that it does not satisfy the fundamental properties of linearity, namely homogeneity and additivity, and the irreversibility. (see the Algorithm 2 for implementation details).
3.5 Identity Privacy Preservation
We present privacy-preserving identity recognition framework where facial images first pro- cessed through FFM and uploaded as irreversible obfuscated features. These features are structured via candidate feature sets ensure uniqueness and non-replicability. While retaining sufficient identity-related cues for recognition, they are mathematically and structurally resistant to inversion, thus preventing facial image reconstruction even under full adversarial access. This design balances high recognition accuracy with strong privacy guarantees.
4 Experiments
4.1
Experimental Setup
Models and Datasets To evaluate the FracFace' framework, we utilize various models and datasets. For recognition, we adopt the IR-50 [10] backbone, which offers a favorable trade-off between compactness and accuracy. The model is trained on the MSIMv2 [9] dataset, which is widespread adoption as a standard benchmark in face recognition ensures fair and consistent comparisons with prior work [27]. [29], [28]. To assess privacy robustness, we employ three of the deep learning-based adversaries: a lightweight U-Net [38] autoencoder for reconstruction-bant
attacks, a StyleGAN [18) generator for generative attacks, and PGDiff[49] generator based on reverse diffusion process. These attackers rigorously test the irreversibility and security of FracFace transformations. We evaluate on standard benchmarks, including LFW [11]. CelebA [22], AgeDB [31], CFP-FP [39], CALFW [55], CPLFW [54], IJB-B [46], and JJBC [26].
Evaluation Metries To evaluate the privacy-utility trade-oft off of FracFace, we employ five metrics. Pixel-level differences between reconstructed and original images are quantified by SSIM [45], MSE, PSNR. LPIPS [53], and IDS [41] where lower SSIM, higher MSE, higher LPIPS. and lower IDS indicate stronger privacy protection. These metrics comprehensively assess the effectiveness of FracFace in safeguarding privacy while maintaining utility. In addition, refer to Appendix A.7 for more details on the experiment.
4.2 Recognition Accuracy
We evaluate FracFace in comparison with two widely adopted baselines, ArcFace[6] and ArcFace- FD[48] (both without privacy protection), well six representative crivacyprreeeving methods. Despite introducing privacy protection. FracFace achieves recognition accuracy comparable to the baselines while significantly enhancing identity security. Tab. reveal that while frequency domain methods like PPFR-FD and DuetFace incur 3% 5% accuracy loss, approaches such as MinusFace, FaceObfuscator, and FartialFace still retain 13%, 15%, and 32% identity information, indicating notable privacy risks (refer to Sec. 4.3 for more details). In contrast, FracFace employs FCR and FFM
'Code is available at htth:://anooonooo...goguu///ee//

---

Poediaer
Partalter
Minnafer
ChoMaaChh Frother
Dedion
Partaler
Minaater
Chankfonaeeoo Fratter
(4) U-Ne--bood Attuks
do SylrGAN-teae Artdess
Figure 3: Evaluation of facial reconstruction vulnerabilities under U-Net and StyleGAN attacks.
(a) Target (b) 1
(c) g 2
(d) 3
(c) 94
(f) 5
(g) 9 6
(h) 7
()
Figure Comparative attack results on CelebA targets with StyleGAN. (a) shows the target image of the adversary attack; (b)-(i) shows the attack outputs.
to restructure the feature space, effectively suppressing low-frequency dominance while promoting the contribution of high-frequency components to identity representation. This design delivers strong privacy protection with only marginal drop in accuracy compared to ArcFace-FD. achieving state-of-the-ard privacy performance and an optimal trade-off between utility and security.
4.3 Empirical Evaluation of Privacy Protection
4.3.1
Effectiveness
Reconstruction Vulnerability Assessment In practical attacks, facial data leakage typically enables two reconstruction paths: (1) mapping protected features to original images via models like U-Net when identity embeddings are exposed; (2) leveraging generative models (c.g.. StyleGAN and PGDiff) to reconstruct identities from intercepted features without requiring an explicit degradation model. We evaluate the robustness of FracFace against reconstruction attacks on six benchmark datasets and compare its performance with several representative privacy-preservee methods. As illustrated in Fig. 3, most existing approaches exhibit limited resistance to reconstruction attacks. For instance, PPFR-FD, Pro-Face, and DuetFace lack dedicated defense mechanisms, allowing adversaries to recover significant facial details from identity tokens. We further employ StyleGAN to reconstruct faces from these identity features and observe that PPFR-FD and Pro-Face C tend to preserve visually identifiable traits like hairstyle, while DuetFace enables more accurate reconstruction of facial geometry. Although FartialFace, MinusFace, and FaceObfuscator demonstrate stronger privacy protection, StyleGAN is still able to extract structural cues, resulting in reconstructions that retain notable similarity to the original identities in terms of facial proportions. In contrast, reconstructions from FracFace- Fraepeace-reeeeeeee representations exhibit no clear identity similarity. Building on FCR, this outcome benefits from the local obfuscation introduced by FFM. It is further supported by quantitative results from our subsequent privacy reconstruction evaluation (see Tab. for more details).
Robustness to StyleGAN We evaluate the reconstruction robustness of FracFace under adversarial attacks driven by StyleGAN, as illustrated Fig. 4 and Fig. Given limited identity tokens (Fig. 4a), the attacker employs StyleGAN to synthesize candidate faces (Fig. 4b-Fig. 4i), selecting the most similar ones as style vectors to enhance further attempts. Fig. 5 details the reconstruction process. While StyleGAN can reconstruct global attributes such as pose, hairstyle, and beard, it fails to
7

---

Table 2: Benchmarking the privacy utility tradeoff under U-Net and StyleGAN based reconstruction attacks, evaluated by SSIM, LPIPS, MSE, PSNR, and IDS across two leakage scenarios.
U-Net-losse Reconstruction Ataak
StyleGAN-based Face Reconstraction Attack
Metric
Method
LPW
Celch/
Ag:DB
CFP-IP
CALFW
CPLFW
LPW
Celcb/
AgeDB
CFP FP
CALFW
CPLPW
Accface
0.9642
0,9583
0.5435
0.9351
0.5555
0.1188
0.9900
0.9527
0.5315
0.9520
0.5408
0.5307
Artface ED
0.9623
0,9067
0,9172
0.9021
0.9761
0.9438
09342
0,9411
09616
PPER-FD
04251
0.1488
0.3079
0.3730
0.3557
0.5204
0.3116
0.2681
0.2414
0.3523
0.3123
0.16%
Ductface
0.4%6
0,2570
0.4280
0,2965
0.5158
0.4249
0.3567
0.2303
0.2873
0.2997
0.3245
0.1663
SSIM
PartialFace
0.4953
0.2927
0.2404
0.3892
0.3715
0.4423
0.2645
0.2683
0.2564
0.3314
0.2487
Minusface
0.3864
0.1461
0.2319
0.3407
0.2878
0.3263
0.3421
0.2266
0.2409
0.2962
0.3587
0.1699
PRO-Face C
0.5517
0.3684
0.5135
0,4737
0.4665
0.1685
0.3535
0,2676
0.2339
0.3217
0.2966
0.1656
FueeOhfracttrr
0.377
0.1984
0.3477
0.3438
0.3468
0.3189
0.3654
0.3585
0.7883
0.2960
0.2595
0.1395
Przoface
0.2395
0.2045
0,2749
0.283
0.3378
0.2364
0.3311
0.1045
Artface
0.0141
0,0788
0.0436
0.0330
0.0301
0.0224
0.0163
0,0192
0.0139
0.0117
0.0131
0.0157
Arcface FD
0.0175
0,0676
0.0418
0.0487
00012
0.0831
0.0190
0.0127
0.0133
0,0128
0.0133
0.0172
PDFR-FD
0.433
0:4522
0.5198
0,5430
0.6683
0,5099
0,7206
0.5443
0.6596
0.6022
0.5910
1
Duciface
0.5264
0.4500
0.3328
0.3442
0.3458
0:4249
0.7878
0,5461
0.6842
0.6007
0.412
0,6208
LPMPS
PartialFace
0.5197
0,5056
0.6536
0.5392
1L0552
0.0512
0.7558
0,5052
0.6733
0,5715
0.0604
0
Minusface
0.6809
0,6790
0,6720
0,6305
0.6675
0,7313
0.5894
0.6732
0.6322
0.7253
0.6768
PRO-Faace C
0.5018
0.4341
0.4133
0.4812
0.4645
0.5403
0,7091
0.6004
0.6822
0.5749
0.6335
0.6719
FoceOotussator
06512
0.6289
0.5790
0.6332
1.6364
0.6112
0,7419
0.5520
0.6891
0.6218
0.6614
0.6535
PracFace (ours)
0.907
0,6834
0.7389
0,7796
0.6958
0.0990
0.8307
0,6354
0.6935
0,6412
0.6655
0.6935
Aicface
0,0012
0.0058
000001
0,0015
00002
0.0031
0.0018
00021
0.0029
00016
0.0014
Areface FD
0.0002
0,0054
0.001
0,0000
0.0012
0,0024
0.0014
0,0024
0.0025
0,0023
0.0019
0.0021
PPER.FD
0.170
0.0453
0.0900
0.0475
0.0372
00164
0,0514
0.0466
0,0340
0.0613
0.0666
0.0462
Ducttace
0.2249
0.0424
000253
000244
0L0263
0.0621
0.0452
0.0645
0L063]
0.0728
MSE1
PartialFace
0.0251
0,0415
0.0872
0,0389
0.0832
0.042
0.0156
0,0613
0.0695
0,0637
0.0793
0,0605
MinusSace
0.0619
0.0425
0.0754
0.0549
0.0537
0.0188
0,0729
0.0675
0.0711
0.0646
0.0993
0.0637
PRO)-Face
0.0018
0.0256
0.0127
0,0200
001499
0.1711
0,0567
0.0633
0,0583
0.0635
0.0495
PaceOOlussatoo
0.0418
0:0466
0.0578
0.0512
0.0409
0.0845
0.0769
0,0795
0.0633
0,0698
0.0646
0.0794
FracFace (ours)
0.0921
0,0839
0.1694
0.0855
0.0591
0.0643
0.0869
0.0993
0.0909
0,0750
0.0753
0.0831
Areface
28.3762
26.339
25.0351
28.1658
28,9046
26.0627
27.3627
23,9351
29:92337
26,5816
20,3843
25.7361
Areface-FD
26 1864
26 5831
28.5249
37.9832
1352
26 1258
28 2943
25.1971
20.5851
27 27,6278
23.4935
27,1559
PPER-FD
16.6922
15.2175
14.953
13,2350
14.3056
17.8451
10.6539
13.3151
10.7322
11,4767
11.6374
10.3582
Ductface
16.0352
14.2463
13.1962
16.2930
16.1359
15.7961
10,3639
11,0596
10.0926
10,7314
1
11.6206
PSNR J
PurtialFaae
13,0542
10:3401
10.918
12,9005
12.7318
9.9318
12.1278
11,9616
11.3875
10,5448
Minusfaze
11,2158
9.3362
11.2309
12,0672
11.9623
13,7816
11.3744
11.7088
11.4857
10.7285
12:2753
10.1361
PRO-Faaee C
17.8751
15.933
18,9571
16.7911
1633451
16.5500
10.1446
11,6173
1144588
10,9503
11.9773
7007
FoceOnfussatox
10:3351
10:9748
12.3641
10.6841
10.7845
12.6315
10,6150
10.7325
10.1433
11.9029
10,9551
PracFare (ours)
8.0099
9,9682
9.5171
10.0953
11.7843
10.0827
9.7742
10.0369
16.0421
10,0562
10.9270
10.1239
Arcface
0.9932
0,9966
0,9989
0.9904
0.9928
0.9991
0.9965
0.9534
0.9910
0,8973
019627
0.9624
Arrface FD
0.9927
0,9919
0.9918
0.9919
0.9915
0.9620
0.9837
0.8993
0.6639
PPFR-FD
0.5699
0.6549
0.802
0.8879
0.7968
0.692
0.7587
0.8319
0.6915
0.6512
0.3250
0.698
Ductiace
0,5830
0.6172
0.7921
0.8766
116217
0.6822
0.7385
0.8239
0.6077
0.6288
0:116
0.6799
PartialFace
0.4670
0:4572
0.7308
0.6353
0.5384
0.3204
0.7317
0,8013
0.6391
0,5962
0.7296
0.6194
Minusface
0.3946
0:4007
0.4147
0,5478
0:4218
0.7600
0.7267
0,8062
0.5990
0.5649
0.7329
0.548I
PRID-Face C
0.4124
0.502%
0.7978
0,7548
018325
0.7271
0.8299
0,8501
0.7064
0.6294
0.8562
0.7158
PaceOblussator
0.3830
0.4094
0.4972
0,4565
0,46247
0,3412
0.7187
0.7374
0.3785
0.6218
0.7253
0.6101
FracFace (ours)
0.0057
0,0081
0.0003
0
0.0018
0.0024
0.6705
0.7334
0.5242
0.5239
0.6150
0.5458
faithfully recover fine grained structures like the eyes, nose, and mouth, leading to perceptible distortions in identity. This confirms that FracFace effectively disrupts the coherence of identity- critical features through FFM, offering strong resistance to clue and style-based based inversion attacks. A more detailed analysis of how FCR and FFM influence the visualization of frequency domain channels is provided in Appendix A.4.
Robustness to Diffusion Model We evaluate FracFace under an adaptive white-box threat model against diffusion- reconstruction (Fig. 6). Using a PGDiff model pretrained on the public CelebA dataset, the reconstructions in Figs. 6b to 6i recover only coarse attributes (e.g., pose, hairstyle) while failing to reproduce identity-critical details (eyes, nose, mouth) and they exhibit noticeable texture artifacts. These results indicate that the FCR module disperses energy and reduces sparsity in low-frequency channels, whereas FFM introduces localized perturbations that confound hid/high-frequent cues. Consequently, FracFace impedes faithful inversion, retaining only coarse attributes and thereby enhancing privacy.
Quantitative Comparison To comprehensively evaluate the privacy-utility tradeoff of our method under strong reconstruction threats, we conducted experiments on six public face recognition bench- marks, as summarized in Tab. 2. FracFace was compared with six state-of-the-art face privacy protection methods under two representative reconstruction attacks: a discriminative model based on U-Net and generative model based on StyleGAN. Evaluation was conducted using SSIM, LPIPS,
8

---

Ideenity Original
Figure 5: Visual analysis of StyleGAN vulnerabilities.
Target
(c) D2 (d) Ds
(c) D4
Ds
(g) De
(h) D7
(i) Ds
Figure 6: Comparative reconstruction results on CelebA targets with PGDiff. (a) shows the target
image the adversary attack; (b)-(i) shows the reconstruction outputs.
MSE, PSNR, and identity similarity. FracFace consistently demonstrates stronger robustness across both attack types. achieves significantly lower SSIM values, indicating reduced structural similarity between protected and reconstructed faces. Under StyleGAN attacks, FracFace attains SSIM scores as low as 0.2264 on CFP-FP and 0. 1045 on CPLFW, suggesting the original structure is effectively obfuscated. In terms of perceptual dissimilarity, FracFace achieves the highest I.PIS scores, reaching 0.8307 on LFW and 0.7389 on AgeDB. Compared to structurally-awan baselines such as Partial- Face, FaceObfuscator, and MinusFace, FracFace achieves superior performance, particularly under challenging generative attacks that often exploit residual structural cues. This strength stems from the synergy between our FCR and FFM modules, which jointly mitigate frequency domain sparsity and limit the reconstructive capacity of generative models.
4.4
Ablation Study
4.4.1
Effectiveness of FCR and FFM
This section conducts ablation studies the FCR and FFM modules in FracFace evaluate their respective and combined impacts on recognition accuracy and privacy preservation. As shown in Fig. when the modules exist independently (as shown in Fig. 7b, Fig.7c, Fig. 7f and Fig. 7g), the obtained frequency histogram shows obvious sparsity, which exposes the system to potential statistical attacks. Introducing either FCR or FFM alone fails sufficiently mitigate this sparsity, as the histograms still reveal distinct, reconstructable patterns. By contrast, integrating both modules leads a significantly more compact and uniform frequency distribution, effectively reducing statistical leakage and enhancing privacy robustness, see Fig. 7d and Fig. 7h. As shown in Tab. 3, while applying no module or only FFM preserves high recognition accuracy, it offers little privacy protection due to insufficient disruption of frequency channels. FCR alone enhances privacy but degrades accuracy. It is the joint application of FCR and FFM that ensures both improved recognition performance and robust privacy protection, achieving a balanced compromise between accuracy and security. This indicates that FCR and FFM are capable jointly resisting reconstruction attacks, while leveraging fractal self similarity and local confusion preserve features relevant to identity.

---

4.4.2 Effectiveness of Fractal Depth l
To examine how the fractal depth & in FFM af- fects privacy and recognition accuracy, we per- formed a systematic ablation across varying k val- ues. As shown in Tab. (in Appendix A.9), larger k values lead to sustained and empirically consis- tent privacy gains (LPIPS rising from 0.5291 to 0.8357 and SSIM falling from 0.5227 0.2580), consistent with stronger obfuscation; however, recognition accuracy progressively deteriorates beyond k = 2. In particular, k emerges as the optimal operating point, achieving 20% im- provement in LPIPS relative to k = 1 with only a 0.02% loss in accuracy, thereby preserving near- perfect recognition while materially strengthen- ing privacy. Accordingly, we employ default and view it as the sweet spot, beyond which the incremental privacy improvement is minor relative to the escalating drop in utility.
w/o R&F
(b) w/ R
(c) w/F
(d) w/ R&F
w/o R&F
(f) w/R
(g) W/F
w/ R&F
Figure 7: Visual comparisons on the impact of modules FCR (R) and FFM (F).
Table 3: Ablation study on the joint effect of FCR and FFM on recognition accuracy
Method
2 by
LFW
AgeDB
FCR
FFM
Protection
x
99.71
97.72
X
/
99.30
97.28
x
84.53
76.84
4.4.3 The Sensitivity of FBA Strength
/
99.59
96.35
We quantified how gqquency-domain attenuation shapes the privacy-utility trade-off by performing sensitivity analysis on the FBA attenuation strength (see Tab. 5 in Appendix A.9). We observe that as refinement strength increases, with more channels removed, privacy consistently improves (e.g.. LPIPS: 0.32 + 0.86), whereas recognition utility declines, with pronounced drop beyond 50%. Notably, 50% pruning yields the best balance, increasing LPIPS by 35% over the 20% while maintaining near peur-ptrfect recognition accuracy. Beyond that point, privacy remains approximately constant, whereas accuracy continues to decrease. These results indicate that moderate refining (40-50%) constitutes an effective operating range, improving privacy with minimal impact on utility.
Conclusion
This work introduces FracFace. a novel privacy preserving face recognition framework designed to combat the critical threat of reconstruction attacks by targeting the vulnerability of visual cues in the frequency domain. Specifically, FCR disrupts spatial continuity (sparsity) and selectively refines identity relevant frequency channels, allowing FracFace to effectively reduce visual cues susceptible reconstruction. Additionally, the FFM remaps optimized frequency features into a complex fractal structure space, substantially complicating reverse recovery. By decoupling identity from easily exploitable structural cues, FracFace weakens the implicit mapping between visual information and identity that many attacks rely on. Extensive evaluations on public benchmarks demonstrate that FracFace not only disrupts visual recoverability but also maintains high recognition accuracy, establishing it as an effective and secure solution for privacy sensitive face recognition applications. These findings suggest that fractal-guided frequency transformation may offer a viable path toward reconciling security with interpretability in future face recognition systems.
6 Broader Impacts
Notably, none of the datasets used this work involve private or non-consented data collection. Licensing terms and access conditions for each dataset are documented in [2] (please refer this Supplementary Material for details). Our work centers on privacy-preserving face recognition: we transform face images into frequency-domain representations that are not trivially invertible and empirically reduce identity leakage. Therefore, the proposed method does not target or infer sensitive attributes, and we are aware of no negative societal impacts within the scope of this research.
10

---

Acknowledgements
This research is supported by the National Natural Science Foundation of China (Grant No. 62372313); the Institutional Research Fund of Sichuan University (No. 2024SCUOTTX054); the China Scholarship Council (Award No. 202406240205); and the National Research Foundation, Singapore, under the programme "Enhancing Large Language Models with Rigorous Reasoning" (Award No. A-80031 16-066000 00-000 NAII). We express our deep gratitude to the program chairs and reviewers for their recognition of, and dedication our work. We also thank the NeurlPS committee for granting us the NeurlPS 2025 Scholar Award, which supported our participation in the conference.
References
[1] Renwan Bi, Jinbo Xiong, Changqing Luo, Jianting Ning, Ximeng Liu, Youliang Tian, and Yan Zhang. Communication-effiieenn secret sharing. IEEE Transactions Information Forensics and Security, 19:6722-677 2024.
neural network inference via arithmetic
[2] Fadi Boutros, Meiling Fang, Marcel Klemt, Biying Fu, and Naser Damer. Cr-fiqa: face image quality assessment by learning sample relative classifiability, Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5836-5845, 2023.
[3] M.A.P. Chamikara, P. Bertok, Khalil, D. Liu, and S. Camtepe. Privacy preserving face recognition utilizing differential privacy. Computers & Security, 97:101951, 2020.
[4] Forrester Cole, David Belanger, Dilip Krishnan, Aaron Sarna, Inbar Mosseri, and William T. Freeman. Synthesizing normalized faces from facial identity features. In Proc. IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 1386-3395, 2017.
[5] Wanying Dai, Beibei Li, Qingyun Du, Ziqing Zhu, and Ao Liu. Chaos-based index index-of-men -fi-min hashing scheme for cancellable biometries security. IEEE Transactions on Information Forensics and Security, 19:8982-8997, 2024.
[6] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4685-4694, 2019.
[7] Alexey Dosovitskiy and Thomas Brox. Inverting visual representations with convolutional networks. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4829 4837, 2016.
[8] Ovgu Ozturk Ergun. Privacy preserving face recognition in encrypted domain. In Proc. IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), pages 643-646, 2014.
[9] Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb- Im: A dataset and benchmark for large-scale face recognition. In Proc. European Conference on Computer Vision (ECCV), pages 87-102. Springer, 2016
[10] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Proc. JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.
[11] Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. Proc. Workshop on Faces in Real-Life Images: Delection, Aligument, and Recognition, 2008
[12] Hai Huang and Luyao Wang. Efficient privacy-preserving face identification protocol. IEEE Transactions Services Computing, (6(4):2632-2641, 2023.
[13] Yangsibo Huang, Zhao Song, Kai Li, and Sanjeev Arora. Instahide: Instance-hiding schemes for private distributed learning. Proc. International Conference on Machine Learning (ICMI., pages 4507-4518, 2020.

---

[14] Jiazhen Ji, Huan Wang, Yuge Huang. Jiaxiang Wu, Xingkun Xu, Shouhong Ding, ShengChuan Zhang, Liujuan Cao, and Rongrong Ji. Privacy-reeeevingg face recognition with learnable privacy budgets in frequency domain. In Proc. European Conference on Computer Vision (ECCV), pages 475 491. Springer, 2022.
[15] Shuaifan Jin, He Wang. Zhibo Wang, Feng Xiao, Jiahui Hu, Yuan He, Wenwen Zhang, Zhongjic Ba, Weijie Fang, Shuhong Yuan, and Kui Ren. FaceObfuscator: Defending deep learning-based privacy attacks with gradient descent-resistant features in face recognition. In Proc. 33rd USENIX Security Symposium (USENIX Security), pages 6849-6866, 2024.
[16] Tero Karras, Miik Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding diffusion model with a bad version of itself. Advances in Neural Information Processing Systems (Neur[PS), 37:52996-5300 2024.
[17] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for genera- tive adversarial networks. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4401-4410 2019.
[18] Tero Karras, Samuli Laine, Mika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8107-8116, 2020.
[19] Lamyanba Laishram, Muhammad Shaheryar, Jong Taek Lee, and Soon Jung. Toward privacy-presering face recognition system: A survey of leakages and solutions. ACM Computing Surveys, 57(6):1-38, 2025.
[20] Florian Lieb and Hans- Stark. Audio inpainting: Evaluation of time-freyuency represen- tations and structured sparsity approaches. Signal Processing, 153, 2018.
[21] Shuofeng Liu, Mengyao Ma, Minhui Xue, and Guangdong Bai. Modifier unlocked: Jailbreaking text-to-image models through prompts. In Proc. IEEE Symposium on Security and Privacy (SP), pages 355-372, 2025.
[22] Ziwei Liu, Ping Luo, Xiaogang Wang. and Xiaoou Tang. Deep learning face attributes in the wild. In Proc. of the IEEE International Conference on Computer Vision (ICCV), pages 3730-3738, 2015.
[23] Zhuo Ma, Yang Liu, Ximeng Liu, Jianfeng Ma, and Kui Ren. Lightweight ensemble classification for face recognition. IEEE Internet of Things Journal, 63)):577--5790, 2019.
[24] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9:2579-266 2009922052200 2008
[25] Guangcan Mai, Kai Cao,, Xiangyuan Lan, and Pong C. Yuen. Secureface: Face template protection. IEEE Transactions on Information Forensics and Security. 16:262-277, 2021.
[26] Brianna Maze, Jocelyn Adams, James A Duncan, Nathan Kalka, Tim Miller, Charles Otto, Anil K Jain, Tyler Niggel, Janet Anderson, Jordan Cheney, al. larpa janus benchmark-c: Face dataset and protocol. In Proc. International Conference on Biometrics (ICB), pages 158-165, 2018
[27] Yuxi Mi, Yuge Huang, Jiazhen Ji, Hongquan Liu, Xingkun Xu, Shouhong Ding, and Shuigeng Zhou. Duetface: Collaborative privacy- preeecypreserving face recognition via channel splitting in the frequency domain. In Proc. ACM International Conference on Multimedia, page 6755-6764, 2022.
[28] Yuxi Mi, Yuge Huang, Jiazhen Ji, Minyi Zhao, Jiaxiang Wu, Xingkun Xu, Shouhong Ding, and Shuigeng Zhou. Proc. IEEF/CVE International Conference on Computer Vision (ICCV), pages 19616-19627, 2023.
face recognition using random frequency components. In
12

---

[29] Yuxi Mi, Zhizhou Zhong, Yuge Huang, Jiazhen Ji, Jianqing Xu, Jun Wang, Shaoming Wang, Shouhong Ding, and Shuigeng Zhou. subtraction. In Proc. IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 297-307, 2024.
face recognition using trainable feature
[30] Pranab Mohanty, Sudeep Sarkar, and Rangachar Kasturi. From scores to face templates: A model-based approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12):2065-2077 2007.
[31] Stylianos Moschoglou, Athanasios Papaioannou, Christos Sagonas, Jiankang Deng, Irene Kotsia, and Stefanos Zafeiriou. Agedb: The first manually collected, in-the-wild age database. In Proc. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1997-2005, 2017.
[32] Hatef Otroshi Shahreza and Sébastien Marcel. Face reconstruction from facial templates by learning latent space of a generator network. In Proc. Advances in Neural Information Processing Systems (Neur/PS), volume 36, pages 12703-12720, 2023.
[33] Lu Ou, Shaolin Liao, Shihui Gao, Guandong Huang, and Zheng Qin. Rdp: Ranked differential privacy for facial feature protection in multi-scale sparsified subspaces. IEEE Internet of Things Journal, 2025.
[34] Binhang Qi, Hailong Sun, Xiang Gao, Hongyu Zhang, Zhaotian Li, and Xudong Liu. Reusing deep neural network models through model re-engineering, In Proc. IEEE/ACM International Conference on Software Engineering (ICSE), pages 983-994, 2023.
[35] Binhang Qi, Hailong Sun, Hongyu Zhang, and Xiang Gao. Reusing convolutional neural network models through modularization and composition. ACM Trans. Softw. Eng. Methodol., 33(3), 2024.
[36] Anton Razzhigaev, Klim Kireev, Edgar Kaziakhmedov, Nurislam Tursynbek, and Aleksandr Petiushko. Black-box face recovery from identity features. In Proc. European Conference on Computer Vision (ECCV), pages 462-475. Springer, 2020.
[37] Anton Razzhigaev, Klim Kireev, Igor Udovichenko, and Aleksandr Petiushko, Darker than black-box: Face reconstruction from similarity queries. arXiv preprint arXiv:2106.14220 2021.
[38] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Proc. International Conference on Medical Image Comput- ing and Computer-Assisted Intervention (MICCAI), pages 234-241, 2015.
[39] Soumyadip Sengupta, Jun-Cheng Chen, Carlos Castillo, Vishal M. Patel, Rama Chellappa, and David W. Jacobs. Frontal to profile face verification in the wild. In Proc. IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1-9, 2016.
[40] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji. Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5265-5274, 2018.
[41] Jiayu Wang, Kang Zhao, Yifeng Ma, Shiwei Zhang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jingren Zhou. Facecomposer: A unified model for versatile facial content creation. In Proc. Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 13467-13479, 2023.
[42] Tao Wang, Wenying Wen, Xiangli Xiao, Zhongyun Hua, Yushu Zhang, and Yuming Fang. Beyond privacy: Generating prrecvipres faces supporting robust image authentication. IEEE Transactions on Information Forensics and Security. 20:2564-2577 2025.
[43] Yinggui Wang, Jian Liu, Man Luo, Le Yang, and Li Wang. Privacy-preserving face recognition in the frequency domain. In Proc. AAAI Conference on Artificial Intelligence (AAAI), pages 2566, 2022.
13

---

[44] Zhibo Wang, He Wang, Shuaifan Jin, Wenwen Zhang, Jiahui Hu, Yan Wang, Peng Sun, Wei Yuan. Kaixin Liu, and Kui Ren. Savvacy-presevingg adversarial facial features. In Proc. IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 8212-8221, 2023.
[45] Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions Image Processing, 13(4):600-61, 2004.
[46] Cameron Whitelam, Emma Taborsky, Austin Blanton, Brianna Maze, Jocelyn Adams, Tim Miller, Nathan Kalka, Anil K Jain, James A Duncan, Kristen Allen, al. larpa janus benchmark- b face dataset. In Proc. IEEE/CVF Conference on Computer Vision and Patlern Recognition (CVPR), pages 90-98, 2017.
[47] Weihao Xia, Yujiu Yang, Jing-Har Xue, and Baoyuan Wu. Tedigan: Text-guided diverse face image generation and manipulation. In Proc. IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 2256-2265, 2021.
[48] Kai Xu, Minghai Qin, Fei Sun, Yuhao Wang. Yen-Kuang Chen, and Fengbo Ren. Learning in the frequency domain. In Proc. IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 1737-1746, 2020.
2020.
[49] Peiqing Yang, Shangchen Zhou, Qingyi Tao, and Chen Change Loy. Pgdiff: Guiding diffusion models for versatile face restoration via partial guidance. Advances in Neural Information 2023.
Processing Systems (NeurIPS), 36:32194-32214
[50] Lin Yuan, Wu Chen, Xiao Pu, Yan Zhang, Hongbo Li, Yushu Zhang, Xinbo Gao, and Touradj Ebrahimi. Pro-face c: Privacy-preserving recognition of obfuscated face via feature compensa- 2024.
tion. IEEE Transactions on Information Forensics and Security,
[51] Hui Zhang, Xingbo Dong, Yenl Lai, Ying Zhou, Xiaoyan Zhang, Xingguo Lx, Zhe Jin, and Xuejun Li. Validating crivacy-preserving face recognition under a minimum assumption. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12205-12214, 2024.
[52] Peng-Fei Zhang, Guangdong Bai, Hongzhi Yin, and Zi Huang. Proactive learning for cross-modal retrieval. ACM Transactions Information Systems, (1(:11123 2023.
[53] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreason- able effectiveness of deep features as a perceptual metrie. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 586-595, 2018.
[54] Tianyue Zheng and Weihong Deng. Cross-pose Ifw: A database for studying cross-pose face recognition unconstrained environments. Beijing University of Posts and Telecommanications, Tech. Rep, 5:7, 2018
[55] Tianyue Zheng, Weihong Deng, and Jiani Hu. Cross-age Ifw: A database for studying cross-age face recognition in unconstrained environments. arXiv preprint arXiv:1708.08197 2017.
[56] Andrey Zhmoginov and Mark Sandler. Inverting face embeddings with convolutional neural networks, 2016. https://arxiv.org/bbbsss 1606.0418)
[57] Zhizhou Zhong, Yuxi Mi. Yuge Huang, Jianqing Xu. Guodong Mu, Shouhong Ding. Jingyun Zhang, Rizen Guo, Yunsheng Wu, and Shuigeng Zhou. Slerpface: Face template protection via spherical linear interpolation. In Proc. the Association for the Advancement of Artificial Intelligence (AAAI), pages 10698-107066 2025.
[58] Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, and Tong Sun. Towards language-free free training for text-to-image generation. In Proc. JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17886-17896, 2022.
14

---

A Appendix
A.1 Sparsity Analysis in Frequency Domain
40
200
200
-20
-2
100
100
-60
-40
-30
20
40
60
-60
-60
-20
20
40
60
(a) Frequency Channel Interference
(b) Refined Frequency Channel
Figure 8: Analyzing the sparsity of frequency domain channels through t-SNE.
As shown in Fig, 8a, low frequency channels still exert strong influence their high frequency counterparts. When low frequency components are fully expressed, the interclass distances increase, resulting in sparser overall distribution. In contrast, Fig. 8b illustrates the refined frequency channels, where the contribution of high frequency components becomes more prominent. This leads to more uniform distribution and noticeable reduction in sparsity. These observations further highlight the necessity of regulating sparsity in the frequency domain by refining frequency channels to encourage more compact and balanced distributions, ultimately improving representation quality.
A.2 Nonlinearity Proof of FFM
We aim to demonstrate that the Fractal Feature Mapping (FFM) is nonlinear, by showing that violates both homogeneity and additivity, the two essential conditions for linearity.
Proof (Homogeneity). Let E be a scalar and let the FFM mapping defined as:
f(Mo) = Mo + C,
(3)
where the constant offset C' e p is given by:
C bb 1) Bi.
(4)
Then applying f to scaled input yields:
f(MM) = AMO + C.
(5)
However, if were linear, we would expect:
f(AMo) \f(M)) A(Mo +C) AMo + AC.
(6)
Clearly, unless À 1 or C 0. have:
f(AM)) + Af(M)).
(7)
Since in practice C' * is intentionally introduced to enhance privacy by perturbing identity-irreleant components, and X of 1 holds for general scaling operations, the homogeneity condition is violated.
Thus, FFM is not a homogeneous transformation.
We further justify the validity of the conditions À * and C' *f 0 as follows:
15

---

The constant offset C' is constructed to be non-zero in order to perturb identity- irrelvat features and preserve privacy. If C 0, the transformation degenerates the identity map I M(M)) = Mo. which provides no privacy protection.
The scalar and is commonly used in evaluating homogeneity.
represents any general scaling factor different from the identity scaling,
Hence, under typical and intended design conditions, FFM violates the homogeneity property, confirming its nonlinear behavior.
Proof (Additivity).
Let Ma. M, E Rax be two arbitrary inputs. The FFM mapping is defined as:
f(M) M Σ (Bx(M) -Kk-
where (x))) denotes a nonlinear feature mapping and "yk E Roxm are fixed perturbation matrices.
Then,
K > (M,) - -1) Yks f(M) = M, +
K
f(Ma) M, +
Σ (3x(M) -1) -k-
(9)
1=
1=
were additive, we would have:
f(Ma + Mb) f(Ma) + f(M)).
(10)
However,
K Σ 1=1
f(M, + MD) = M, + Mb
(Be + M,) - 1) .kk
(11)
while
K
f(M.) + f(M)) = M. + Mr +
(B&((M)) - 1) (3x(M)) - -1)] 7k-
(12)
k=
These are not equal unless
(MG + M,) -Be(MMa) + By (Mb),
(13)
which is generally false due the nonlinearity of (3x()) Therefore,
f(Ma + Mb) * f(Ma) + f(Mb),
(14)
proving that FFM does not satisfy additivity either.
In summary, the FFM transformation violates both homogeneity and additivity, it is not linear transformation. In fact, is an affine mapping (due to the additive constant C), and its nonlinearity intentionally designed to obfuscate deeniityirrelevant features and ensure privacy preservation.
Non-invertibility. We also consider a threat model where an adversary aims to reconstruct and identify faces from obfuscated images, assuming knowledge of FracFace parameters (e.g., fractal depth and expansion schedule). While visual cues cannot be fully removed due to recognition needs, FracFace substantially reduces them compared to prior work. In detail, two components the initial fractal kernel Mo n and the index lattice Lo zuxa_are secret and randomized per deployment, ensuring unique, secure instances. In addition, the fractal index mapping is defined recursively as
8-1
[i = Moli.j] +
Σ (bp- Be,
B
3e =
b
(15)
1=1
1=1
This is then projected modulo the channel dimension C:
*[i.o]
F(k] [ mod C.
(16)
16

---

Vanilla Frequeney
(b) Refined Frequency Domain Channel Visualization (Part 1)
Filtered Frequency Domain Channel
(d) Fractal Frequency Domain Channel Visualization
Domain Channel Visualization
Visualization (Part 2)
Figure Visual comparison of refined frequency domain channels.
Because F(l grows exponentially with k many different positions collide to the same index under s/.
(11:51) * (22. s.t. P1i1.11 4122. f2].
but Fih Ail + Fik) (12 ful.
(17)
This shows that v is non- non-injective by design, and no inverse mapping exists. Even if the adversary learns the full mapping rule tb, they cannot resolve the original positions or channel correspondences without knowing the randomized My and Lo:
1 ZC - z
(18)
These design choices ensure that even subtle visual cues are unrecoverable under strong white-box attacks. The non-invertibility of and the randomness from Mo and Lo jointly establish robust privacy boundary, making FracFace highly resistant to reconstruction.
A.3 The Definition and Computation of Protection( %
Definition Protection (%) represents the share of frequency-domain channels that are either (i) filtered out by FCR or (ii) structurally disrupted by FFM, expressed as a percentage of the total number of channels.
P Paask + |remapp
Protection (%)
(19)
Piotal
where, Pioaai the total number of DCT frequency channels (e.g.. 192 for 12 x 16 DCT), P Pmask is the number of low-energy channels pruned FCR, and Premap is the number of remaining channels remapped by FFM.
A.4 Visual Privacy Protection in Face Features
We begin by visualizing the frequency domain distribution extracted from raw facial images, as illustrated in Fig.9a. The distribution exhibits a clear dominance of low frequency components and an overall sparse structure. These characteristies raise potential privacy risks: sparsity may allow adversaries to reconstruct identity revealing features; meanwhile, the dominance of low frequency components suppresses high frequency details, limiting the representational capacity of the frequency space. To mitigate these risks, we apply FCR to perform band attenuation, as shown in Fig.9b. This process selectively preserves features that are identity relevant yet privacy preserving, while filtering out noisy frequency components (see Fig.9c). Then, fractal frequency domain channel is generated via FFM (Fig. 9d), mitigating sparsity and preserving self similarity for recognition, while its inherent randomness obfuscates identity cues to reduce reconstruction risk and enhance privacy protection.
17

---

A.5
The Algorithm of FCR
Algorithm 1 Frequency Channel Refining (FCR)
Require: Grid size M, N
Ensure: Two groups S1, and S2
1: F &- Create matrix (M,N)
freq list 6
3: for i < 0 to M do
row + 1
5: if i mod 0 then
6: Extend (freq_ list, row)
7: else
8: Extend (freq_iist, Reverse (row))
9: end if
10: end for
11: S1 freq_liit [0:00]
12: S2 + freq_list [80:16]
13: return S2 =0
A.6 The Algorithm of FFM
Algorithm 2 Frequency Fractal Mapping (FFM)
Require: Number iterations K, fractal kernel size m n Ensure: Fractal mappings J =
Fx-1]
I: Initialize Mo mn with random integers in [1, m n]
2: Initialize Lo E Zmxn as indexing order
Set Fo Mo
4: for k = 1 to K 1 do
5: Compute scaling factor: 3r = 32t
6: for i to do
7: for = to n do
8: Fxli,j] = (Moli.j) - 1) + Fx-1
9: end for
10: end for
11: if correction required at layer i then
12: =
13: end
14: end for
15: return F=0
A.7 More Implementation Details
Data Preprocessing of Input Images All face images were first resized to 1 112x1 12 and normalized as RGB tensors, then transformed into the frequency domain via BDCT. retain identity-relevant information while reducing redundancy, we applied Frequency Channels Refining (FCR) to extract 81 informative channels. These were further refined using Frequency Fractal Mapping (FFM), which enhances feature consistency across both spatial and scale dimensions. The final 8 81-channel representations were saved as .npy files. Throughout preprocessing, we ensured numerical stability by checking for NaNs and infinities, and accelerated the pipeline using 8- 8-worker parallel loading, This procedure was uniformly applied across training and evaluation datasets, including MSMM-Arceace, LFW, and AgeDB.
Training Details We trained the FracFace model on the MS1Mv2 dataset using PyTorch with two RTX 6000 GPUs (49 GB VRAM each). Training the FracFace model on MS1Mv2 took about 8 days for a total of epochs. During training, the peak memory usage per GPU was about 45GB. The input comprised 81 31-channel feature maps produced by the FracFace pipeline, incorporating DCT
18

---

Figure 10: Resilient identity learning despite frequency degradation
transformation, Frequency Channel Refinement (FCR), and Frequency Fractal Mapping (FFM). An IR-50 backbone with ArcMargin loss employed to enhance identity discrimination. Optimization was performed using Adam (Ir=0.001, weight decay=le-4) with a cosine annealing scheduler. Training leveraged automatic mixed precision (AMP) and gradient clipping (max norm 5.0) for stability. We monitored performance via TensorBoard and validated on a held-out set after each epoch. Data loading parallelized with 8 workers and prefetching, and all experiments used sudeendsccddn..... torch. backends. cudnn. benchmark-Trrue for optimal GPU performance.
A.8 Limitations
As described in Sec. 3, FracFace performs frequency channel refinement via the Frequency Channels Refining (FCR) module after applying the BDCT transform. The retained reeltaty frequency components then mapped into fractal structural space through the Frequency Fractal Mapping (FFM) process. As illustrated Fig. 10, when the input images are of low resolution, the high- frequency bands may carry limited unstable identity cues, making it challenging for the model to extract robust representations. This degradation is likely caused by the loss of discriminative patterns the high-frequency spectrum under adverse imaging conditions. Nonetheless, as shown in Fig. 10, when the training data preserve sufficient frequency fidelity, FracFace can still effectively learn idwntity-aware representations while maintaining strong privacy protection guarantees.
A.9 Ablation Study
Table 4:
Fractal depth k
Accuracy t
SSIM
LPIPS
99.71
0.5227
0.5291
99.69
0.4015
0.6353
96.46
0.3729
0.7925
92.13
0.2580
0.8357
Table 5: FBA pruning strength
Ratio
Accuracy t
SSIM L
LPIPS
99.83
0.7857
0.3184
40%
99.71
0.6291
0.4833
50%
99.69
0,3012
0.6839
60%
89.26
0.3109
0.7294
80%
87.24
0.2793
0.8605
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims presented in the abstract and introduction accurately reflect the paper's s core contributions and scope. The abstract and introduction provide a clear overview of the proposed approach and its significance. The method is elaborated in detail in Sec. 3, while Sec. 4.2, Sec. 4.3, and Sec. 4.4 present extensive experimental results and ablation studies that substantiate the claims made earlier. In addition, we also mentioned the novelty our proposed solution in the key contribution section. Compared with other work in related work, we proposed use fractal to solve the current PPFR problem.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitation in Appendix A.8. addition, we tested our proposed FracFace on 6 different datasets, which contain facial images of different people (based on public datasets). Due to page limitations, we have placed the relevant content in the Appendix. However, if necessary, we will also accept placing it in the main text to explain the issue.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they seale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
20

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
The theoretical foundations of our approach are thoroughly established. All relevant assumptions are explicitly presented, and Appendix A.2 contains the full and correct proofs, ensuring the soundness and transparency of the theoretical claims. In addition, our Sec. 3 also describes in great detail the calculation of each data and the flow of the algorithm, and all steps are explained.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
The paper provides sufficient information to enable reproduction of the main experimental results that support the central claims. We include the core implementation of the FracFace framework, along with access to the training dataset and code necessary to replicate the training procedure. These materials are made available through the link referenced in Sec. 4.1. In addition, presents the essential pseudocode that demonstrates how FracFace can be implemented (refer more for Algorithm. 2).
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
21

---

While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c)
If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper specifies all the necessary training and testing details. These details are thoroughly outlined in Sec. 4.1 of the paper. Additionally, we provide the core pseudocode of FracFace, offering a detailed implementation process. We provide open access both the data and the source code (Code is available wow.mmuuuuue.ssennnnaee..
along with comprehensive instructions to ensure the faithful reproduction of the main experimental results. Detailed implementation guidelines can be found in Appendix. A.7 of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
Poyic)) for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /nips. wttpcc/ppppppppteeeeeeceeeececce sesicc/ggiess.e /eee/eeeeeeeeiiieeeeeeeeeeeeeeee www.blmmisiiioocccccio.c. for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
22

---

The paper specifies all the necessary training and testing details. These details are thoroughly outlined in Sec. 4.1 and Appendix A.7 of the paper. (Code available at https://anonymous..o howp:/yymmouusoops.ceeeeeeeeeeee sopennsciencc/////raaaae) Guidelines:
Justification:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We adhere to the established experimental protocols from prior PPFR works and present the best achieved results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper offers comprehensive details regarding the computational resources necessary for both training and inference. These specifications are meticulously outlined in Appendix A.7, ensuring the full reproducibility of the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make into the paper).
23

---

9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics attpps:/neurips.. httppp.ccccpcccccciiciccccccppee wtw.:/hhiicciiieeeeeeeeeee.
Answer: [Yes]
Justification: The research reported in this paper complies with the NeurIPS Code of Ethics and adheres to all relevant guidelines and standards. All datasets used are publicly available; their licensing terms and access conditions are summarized in Sec. 6.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss the broader impacts in Sec. 6. FracFace aims to enhance facial privacy protection while preserving identity information for secure face recognition, which could have societal impacts in contexts such as privacy-preservie lance minimization, and responsible biometric data usage (see Sec. 4.3 for more details). The method may help mitigate risks of facial data misuse, especially in publicly deployed face recognition systems. All experiments were conducted on publicly available datasets strictly for academic research, and no personally identifiable data was used beyond the scope these datasets.
authentication, surveil-
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]

---

This paper does not have such risks. In addition, all the data sets used in our experiments are public data sets. For an introduction the data sets, please refer Sec. 4.1 Guidelines:
Reason:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
Licenses existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external assets used in this work, including code, data, and models, are properly credited, with their respective licenses and terms of use clearly acknowledged and fully respected. All data and code employed in this paper were acquired with the required legal permissions.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcom thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
25

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsoureing or research involving human subjects, therefore, such details are not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The paper does not involve the usage of Large Language Models (LLMs) as core component of the methodology. LLMs were not integral to the core scientific processes, which did not affect the originality, rigor, or methodology of the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, ntn-standard components.
Please refer to our LLM policy (https: httpe:////euisppeeeee hatp:///eeeeeeeeeee////50//55 for what should or should not be described.
26


Reviews:
[{"review_id": "xZRBgV5q0B", "reviewer": "Reviewer_sFx1", "summary": "The paper introduces FracFace, a privacy-preserving face recognition framework that aims to reduce visual clues in facial features, making it more resistant to reconstruction attacks by generative models. The framework leverages frequency domain processing and fractal-based mapping to obfuscate identity features while maintaining recognition accuracy.", "questions": "1. The results show a slight drop in recognition accuracy due to privacy protection mechanisms. How does FracFace perform in real-world systems where even a small drop in accuracy could significantly impact its usability in critical applications?\n2. Could FracFace be combined with other privacy-preserving face recognition techniques, such as homomorphic encryption or federated learning?\n3. Even with FCR and FFM, are there any residual visual clues that could still be exploited by an adversary? For instance, could subtle attributes like age, ethnicity, or gender still be inferred from the transformed facial features?", "limitations": "1. The study focuses primarily on attacks from U-Net and StyleGAN, but there may be other types of adversarial or black-box attacks that could exploit new vulnerabilities. The model might not be as robust against emerging attack strategies.\n2. FracFace’s performance was evaluated on standard face recognition datasets (LFW, CelebA), which consist of relatively high-quality images. The approach may not perform as well with low-resolution or noisy images.\n3. The privacy evaluation is primarily based on metrics such as SSIM, MSE, and LPIPS, which focus on visual quality and similarity. However, these metrics do not capture all dimensions of privacy, such as identity inference risk or psychological privacy risks.", "rating": 3, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 1, "strengths_and_weaknesses": "Strength:\nFracFace introduces a fractal-based transformation to disrupt the spatial structure in facial data, combining frequency domain refinement and fractal mapping. \n\nWeaknesses:\n1. The idea of using the frequency domain to do privacy-preserving face recognition is very common in this field. The novelty of this paper is limited.\n2. The fractal mapping (FFM) approach might not be the most efficient way to obfuscate features. The introduction of randomized structural perturbations could lead to potential issues in preserving important features needed for accurate recognition, especially in more complex face images or in challenging lighting and pose variations.\n3. There may still be residual privacy risks in certain cases where low-frequency features or mild visual cues remain detectable by adversarial models. Could you make some illustrations?\n4. The proposed method primarily evaluates the framework against synthetic attacks (U-Net and StyleGAN). While these models represent typical reconstruction threats, the paper does not explore how FracFace performs against more realistic adversarial attacks in a dynamic, uncontrolled real-world setting.", "was_revised": true, "final_justification": "Overall, the authors have adequately addressed most of my concerns, particularly regarding the privacy-utility trade-off, compatibility with other techniques (HE and FL), and residual visual cues. However, their response on novelty could be more robust, and their defense of FFM’s efficiency could benefit from more detailed experiments under extreme or edge-case conditions. While their approach to adversarial threat models and privacy risks is strong, there is room for improvement in discussing the robustness against black-box and emerging attacks, as well as performance under noisy or low-resolution images. In summary, the authors have made a good effort to address your feedback, but there are still areas that could benefit from more detailed explanation or additional experiments.", "rebuttal": "Dear **Reviewer sFx1**,\n\nThank you for your valuable comments and suggestions. Our detailed point-by-point responses are provided below.\n\n### **Response to Q1. On the Privacy–Utility Trade-off**\n\nWe would like to clarify the concern as follows. In practice, minor reductions in recognition accuracy are acceptable, since modern face recognition systems adjust thresholds based on deployment context, with stricter settings for border control and more lenient ones for surveillance.\n\nThis trade-off is a well-known challenge in the literature, where enhanced protection often comes at the cost of utility. For example, FaceObfuscator [S1] explicitly discusses the usability–privacy trade-off in real-world deployments, while PPFR-FD [S2] demonstrates that only when removing over 94% of frequency channels would significantly reduce recognition accuracy. Our method retains 50% of these channels, offering both structural protection and utility, as shown in **Table R1**.\n\n**Table R1: Minimal Utility Drop of FracFace Compared to ArcFace**\n\n| Dataset        | ArcFace (%) | FracFace (%) | Results |\n|----------------|-------------|---------------|------------|\n| LFW            | 99.73       | 99.69         | –0.04%     |\n| CelebA         | 95.35       | 95.91         | +0.56%     |\n| AgeDB          | 97.19       | 96.96         | –0.23%     |\n| CFP-FP         | 96.83       | 96.14         | –0.69%     |\n| IJB-C (1e-4)   | 92.26       | 92.66         | +0.40%     |\n| IJB-C (1e-5)   | 93.58       | 93.43         | –0.15%     |\n\n**References**\n\n[S1] FaceObfuscator. USENIX Security 2024.\n\n[S2] Privacy-Preserving Face Recognition in the Frequency Domain. AAAI 2022.\n\n### **Response to Q2. Compatibility with HE and FL**\n\nThe suggestion is interesting! As a modular model-agnostic preprocessing layer operating at the feature level, *FracFace* is inherently well-suited for combination with HE and FL, offering complementary protection across different stages of the pipeline.\nSpecifically, *FracFace* enhances input-level visual privacy by obfuscating frequency and spatial cues *prior* to downstream processing. \n\n| Method                          | Role of FracFace                                               | Benefits                                                                                                                                       |\n| ------------------------------- | -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |\n| **HE** | Acts as a lightweight preprocessing module before encryption   | Suppresses redundant visual features → lowers embedding entropy → mitigates HE’s high inference cost while enhancing visual obfuscation        |\n| **FL**     | Obfuscates facial structures locally before feature extraction | Reduces gradient inversion risk → improves privacy under shared model updates; Frequency pruning reduces redundancy → speeds up local training |\n\n### **Response to Q3. Can residual visual cues reveal age, gender, or ethnicity?**\n\nThank you for this thoughtful question. FracFace is specifically designed to suppress visual cues that contribute to face reconstruction. While it is possible that certain soft-biometric attributes may partially persist after transformation, these signals alone are generally insufficient for reliable identity inference by an adversary.\n\nTo examine this concern, we conducted a user study on 20 FracFace-processed celebrity images. Participants (54 responses) were asked if they recognized the person and which cues they used. As shown in **Table R1**, most relied on features like face shape and hairstyle, not soft-biometrics. Specifically:\n\n**Table R1: User Reliance on Soft-Biometric Attributes for Identity Inference**\n\n| Attribute        | Reported Usage (%) |\n|------------------|--------------------|\n| Ethnicity        | 0.0%               |\n| Age              | 3.7%               |\n| Gender           | 12.3%              |\n\nOver **76%** of participants reported relying on guesswork or vague impressions, citing a lack of clear visual cues for recognition.\n\n### **Response to W1. Novelty**\n\nThank you for the comment. \n\nWhile frequency-domain processing is common in PPFR, we address a novel challenge: the sparsity-driven reconstructibility of DCT features. Our FCR module prunes channels based on semantic relevance rather than fixed frequency bands, reducing redundancy. However, as shown in Fig. 6 and Table 3, this alone is insufficient. To go beyond frequency space, we propose a novel fractal-based mechanism that remaps DCT features into a recursively constructed, non-invertible fractal space. To our knowledge, this is the first use of fractal geometry to disrupt structural coherence in the frequency domain (see Appendix A.2). \n\n### **Response to W2. On the Efficiency of FFM, Challenging Scenarios**\n\nWe thank the reviewer for raising this valuable point. \n\nWe emphasie that this work aims to address the core challenge of balancing privacy and utility in PPFR, which requires to disrupt spatial regularities (resisting reconstruction), while preserving identity-related features to support recognition. Therefore, we evaluated the effeciency with regard to face recoginition accuracy and privacy preservation (measured by SSIM, IDS, MSE, and LPIPS). Compared with 6 SOTA ways to obfuscate feature (Table 1 and 2), our expeirements show that FFM is the most effecient for this specific purpose. Some of existing works e.g., FaceObfuscator and Minusface, can obfuscate features, but the results can be easily reverted, and thus unsuitble for this work.\n\nRegarding computational effeciency, we have provided the FFM complexity calculation in Appendix A.5 Training Details.\n\nRecognition accuracy drop remains negligible (0.04%-2% in our experiments) under practical deployment thresholds, and FracFace performs robustly across challenging conditions. Datasets like IJB-B, IJB-C, and CelebA feature real-world variations in pose, lighting, and resolution, yet FracFace consistently preserves high recognition accuracy while enhancing privacy protection.\n\n### **Response to W3. Residual Privacy Risks and Visual Clues**\n\nThank you for the insightful question. We consider a threat model where an adversary aims to reconstruct and identify faces from obfuscated images, assuming knowledge of FracFace parameters (e.g., fractal depth, expansion schedule). While visual cues cannot be fully removed due to recognition needs, FracFace greatly reduces them compared to prior work. Soft-biometric traits like face shape may remain but are beyond this work’s primary scope. Crucially, our transformation is non-invertible and does not introduce new privacy risks. \n\nIn detail, two components, the initial fractal kernel $M_0 \\in \\mathbb{Z}^{n \\times n}$, and the index lattice $L_0 \\in \\mathbb{Z}^{n \\times n}$, are secret and randomized per deployment, ensuring unique, secure instances. \nIn addition, the fractal index mapping is defined recursively as:\n\n$$\n\\mathcal{F}^{[k]}[i,j] = M_0[i,j] + \\sum_{\\ell=1}^{k} (b_\\ell - 1) \\cdot \\beta_\\ell, \\quad \\beta_1 = 1,\\ \\beta_\\ell = \\prod_{s=1}^{\\ell-1} b_s\n$$\n\nThis is then projected modulo the channel dimension $C$:\n\n$$\n\\psi[i,j] = \\mathcal{F}^{[k]}[i,j] \\bmod C\n$$\n\nBecause $\\mathcal{F}^{[k]}$ grows exponentially with $k$, many different positions collide to the same index under $\\psi$. That is,\n\n$$\n\\exists (i_1, j_1) \\ne (i_2, j_2) \\text{ such that } \\psi[i_1, j_1] = \\psi[i_2, j_2], \\text{ but } \\mathcal{F}^{[k]}[i_1, j_1] \\ne \\mathcal{F}^{[k]}[i_2, j_2]\n$$\n\nThis shows that $\\psi$ is non-injective by design, and no inverse mapping $\\psi^{-1}: \\mathbb{Z}_C \\rightarrow \\mathbb{Z}^{n \\times n}$ exists. Even if the adversary learns the full mapping rule $\\psi$, they cannot resolve the original positions or channel correspondences without knowing the randomized $M_0$ and $L_0$.\n\nThese design choices ensure even subtle visual cues are unrecoverable under strong white-box attacks. The non-invertibility of $\\psi$ and randomness from $M_0$ and $L_0$ jointly establish a robust privacy boundary, making FracFace highly resistant to reconstruction.\n\n### **Response to W4. Adversarial Threat Model**\n\nWe thank the reviewer for the concern. While our experiments use generative reconstruction attacks (e.g., StyleGAN inversion, U-Net), these simulate worst-case white-box threats (see **W3**), not just synthetic scenarios. Our assumed attacker has full access to model details and data priors—far beyond typical real-world adversaries. Yet, FracFace remains highly resistant to reconstruction, achieving:\n\n* Over 60% drop in SSIM and 80% increase in LPIPS compared to unprotected images;\n* Effective removal of visual cues even when all system components (excluding secret seeds) are known.\n\nThis threat model subsumes real-world conditions like lighting, pose, and domain shifts. Since FracFace withstands such strong attacks, its robustness extends to more typical, weaker adversaries. \n\n### **Response to L1. Attack Diversity**\n\nWe additional evaluated against a recent Diffusion-based method.\n\n**Table R1: Comparison of Reconstruction Attacks Against FracFace**\n\n| Attack Model  | SSIM ↓ | MSE ↓ | IDS ↓ | Inference Time |\n| ------------- | ------ | ----- | ----- | -------------- |\n| **U-Net**     | 0.3613  | 0.0853 | 0.3028  | \\~1.2 sec      |\n| **StyleGAN**  | 0.3151  | 0.0962 | 0.4936  | \\~5.3 sec      |\n| **Diffusion** | 0.3809  | 0.0623 | 0.4325  | \\~300 sec      |\n\n### **Response to L2. Low-Resolution/ Noisy Inputs**\n\nSee **W2**.\n\n### **Response to L3. Identity Inference/Psychological Privacy Risks**\n\nTo assess broader privacy, we conducted a user study (see **Q3**).  76% wrongly guessed, indicating weak visual cues. This suggests FracFace effectively limits identity inference and psychological privacy risks. \n\n---\nLooking forward to further discussion and feedback."}, {"review_id": "rxKgjJ7S5q", "reviewer": "Reviewer_NRe7", "summary": "This paper proposes FracFace, a novel PPFR framework that conceals visual clues of face images by operating in the frequency domain. The method introduces two main components: a frequency channel refining module that reduces frequency domain sparsity by suppressing non-identity-related components, and a frequency fractal mapping module that remaps refined features into a fractal structure to hinder reconstruction. Extensive experimental results suggest that FracFace achieves strong resistance to both U-Net and StyleGAN-based attacks while preserving recognition accuracy.", "questions": "-\tThe paper presents incremental novelty: frequency-based PPFR approaches have long been explored in prior work. However, the authors do introduce new techniques to effectively reduce feature sparsity in the frequency domain, which is believed to be critical for defending against reconstruction attacks. The proposed modules are thoughtfully integrated into the frequency-based PPFR framework and demonstrate strong empirical performance.\n-\tAmong different options that could obfuscate reconstruction, what is the motivation for using fractal structures specifically? Could similar gains be achieved via other non-linear mappings or learned index transformations?\n-\tHow sensitive is FracFace to the choice of pruning strategy in FCR? Have the authors considered adaptively learning the channel selection or making it adaptive to image content?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n-\tPrivacy-preserving face recognition is an important topic with societal impact. It is worthy of further investigation.\n-\tThough frequency-based PPFR is not new, the method targets frequency sparsity, a clear and under-addressed vulnerability in prior PPFR designs.\n-\tThe method outperforms prior works in resiliency to both AE- and GAN-based reconstruction attacks, and achieves satisfactory recognition accuracy.\n\n**Weaknesses**\n\n-\tThe motivation behind fractal kernel construction could be more intuitive; some components are overly complex without clear ablation.\n-\tThe reliance on heuristic pruning rules in frequency channel refining seems to lack adaptivity.", "was_revised": true, "final_justification": "I appreciate the authors' response. After considering their rebuttal and the feedback from my fellow reviewers, I have decided to maintain my original score.", "rebuttal": "Dear **Reviewer NRe7**,\n\nThank you for your positive evaluation and encouraging remarks! We carefully investigated each question and weakness trying to address them to the best of our capability within the time frame. Our response are as following:\n\n### **Response to Q1. Novelty**\n> *The paper presents incremental novelty: frequency-based PPFR approaches have long been explored in prior work. However, the authors do introduce new techniques to effectively reduce feature sparsity in the frequency domain, which is believed to be critical for defending against reconstruction attacks. The proposed modules are thoughtfully integrated into the frequency-based PPFR framework and demonstrate strong empirical performance.*\n\nWe greatly appreciate your recognition of the technical contributions in our work, especially regarding the challenge of frequency domain sparsity. As you rightly pointed out, frequency-based PPFR pipelines have been widely explored. *However, most existing approaches focus on filtering or masking coefficients, without explicitly tackling the structural sparsity in the frequency domain*. This sparsity inherently permits reconfigurability and reversibility, posing a serious vulnerability under reconstruction attacks. *In this work, we identify frequency domain sparsity as a key limitation that compromises the balance between privacy and usability*. As shown in our ablation studies (Fig. 6 and Table 3 in our main paper), improvements in the frequency domain, for example through the FCR module, alone are limited in addressing the core challenge. This motivates the need for a novel approach. To this end, we innovatively leverage the inherent self-similarity and recursive locality of fractal structures, and *design the FFM module to effectively mitigate frequency-domain sparsity*.  FFM enables FracFace to systematically overcome frequency sparsity and achieve robust privacy protection without sacrificing usability.\n\n### **Response to Q2. Motivation & Performance**\n> *Among different options that could obfuscate reconstruction, what is the motivation for using fractal structures specifically? Could similar gains be achieved via other non-linear mappings or learned index transformations?*\n\nWe thank the reviewer for this insightful question.  The motivation for designing fractal structures is to defend against reconstruction attacks by disrupting the structure of frequency representations exploited by generative models. In developing this mechanism, we explored various alternatives, such as randomized nonlinear mappings and learned index transformations. We ultimately chose fractal structures because they offer a rare combination of effective reconstruction resistance and preservation of recognition performance.\n\nSpecifically, our choice of fractal-based frequency remapping is motivated by two intrinsic properties of fractals:\n\n**1. Local Recursion:** Fractal mappings expand spatially through recursive integer rules, generating complex and multiscale index patterns that are not easily invertible. This recursive structure increases entropy in the transformed space and helps prevent reconstruction, even when the adversary has full white box access to the approach.\n\n**2. Self-Similarity:** Despite structural complexity, fractals preserve local correlations across scales, which helps maintain the semantic consistency of identity-related signals. This allows recognition models to extract usable embeddings, preserving utility even under aggressive obfuscation.\n\nThese two properties complement each other and together enable us to tackle a core problem in PPFR: balancing privacy and usability. As shown in our results (Table 1 and 2), FracFace significantly boosts resistance to reconstruction attacks (e.g., 60% LPIPS increase), while keeping recognition accuracy almost unchanged (e.g., only 0.04% drop on LFW), highlighting its effectiveness in achieving this balance.\n\nWe explored several alternative transformations, including:\n\n* **Random or pseudo-random channel permutations**, which are easy to reverse if known and often degrade utility due to lack of structural regularity;\n* **Learned index transformations**, which introduce additional parameters and may suffer from overfitting or instability under unseen conditions;\n* **Frequency-space scrambling based on fixed priors**, which often fail to generalize across identities and domains.\n\n*While these alternatives had some potential, they failed to meet two key requirements of our approach: structural non-invertibility and semantic preservation*. In contrast, fractal mappings are deterministic yet chaotic, simple yet expressive. This unique combination allows us to build a defense that is both mathematically analyzable and practically effective. *We will add this analysis*.\n\n### **Response to Q3. On the Sensitivity to FCR Pruning Strategy**\n> *How sensitive is FracFace to the choice of pruning strategy in FCR? Have the authors considered adaptively learning the channel selection or making it adaptive to image content?*\n\nThank you for your thoughtful question. In our work, we carefully analyzed the sensitivity of pruning strategies. Our choice was guided by both empirical results and prior studies, which show that overly aggressive or overly conservative pruning often leads to unstable or suboptimal performance. To understand this behavior in greater detail, we focused our analysis on the 40%–60% pruning range, where the transition became particularly evident. As shown in **Table R1**, we found that retaining approximately 50% of the DCT channels (i.e., 81 out of 192) provided the best trade-off: it significantly improved visual obfuscation (LPIPS increased from 0.3184 to 0.6839) while maintaining high recognition accuracy (99.69%). \n\n**Table R1: Effect of Refining Ratio on Privacy–Utility Trade-off**\n| Pruning | Accuracy (%) | SSIM↓| LPIPS↑|\n| -------| ----------- | ------ | ----- |\n| 20%    | 99.83       | 0.7857 | 0.3184 |\n| 40%    | 99.71       | 0.6291 | 0.4833 |\n| 50%    | 99.69       | 0.3012 | 0.6839 |\n| 60%    | 89.26       | 0.3109 | 0.7294 |\n| 80%    | 87.24       | 0.2793 | 0.8605 |\n\nIn fact, the process of selecting a pruning strategy can be seen as a form of adaptive learning process that evaluates how channel selection affects privacy and utility. \n\nIn practical applications, adapting the pruning strategy to each image introduces non-negligible computational overhead and deployment cost—particularly since FracFace operates on high-resolution images (112×112). For inputs with consistent pixel dimensions, our experiments show that a fixed pruning strategy offers greater efficiency and stability. Therefore, we adopt a fixed pruning approach and recommend cropping images to a consistent resolution when using FracFace.\n\nWe appreciate your suggestion and will add the above in the final verison.\n\n### **Response to W1. Motivation & Ablation**\n> *The motivation behind fractal kernel construction could be more intuitive; some components are overly complex without clear ablation.*\n\nThank you for the valuable comment. We would like to clarity the intuition behind the fractal kernel and will include it in the final version. As detailed in our response to **Q2**, the design is motivated by optimizing the trade-off of between privacy and utility, leveraging the unique properties of fractals. We have also examined ablation effects related to FCR pruning sensitivity in **Q3**. Moreover,  we conducted an additional ablation study, varying the number of fractal iterations $k$, to assess both recognition accuracy and privacy robustness under a StyleGAN-based reconstruction attack. As summarized in **Table R2**, increasing $k$ consistently enhances privacy (LPIPS ↑ from 0.53 to 0.84; SSIM ↓ from 0.52 to 0.26), indicating stronger visual obfuscation. However, recognition accuracy starts to decline beyond $k=2$. We find $k=2$ strikes the best balance, achieving a 20% LPIPS improvement over $k=1$ with only a negligible 0.02% accuracy drop. We will add these findings to the final version to support the discussion.\n\n**Table R2: Impact of Fractal Depth $k$ on Privacy–Utility Trade-off**\n| $k$ | Accuracy (%) | SSIM ↓ | LPIPS ↑ |\n| --- | ------------ | ------ | ------- |\n| 1   | 99.71        | 0.5227 | 0.5291  |\n| 2   | 99.69        | 0.4015 | 0.6353  |\n| 3   | 96.46        | 0.3729 | 0.7925  |\n| 4   | 92.13        | 0.2580 | 0.8357  |\n\n### **Response to W2. The Reliance on Heuristic Pruning Rules in Frequency Channel Refining Seems to Lack Adaptivity**\n> *The reliance on heuristic pruning rules in frequency channel refining seems to lack adaptivity.*\n\nThank you for the insightful comment. We have discussed this issue in **Q3**, where we evaluated the trade-off between pruning strength and performance, and highlighted that a fixed strategy achieves stable results across typical input resolutions (e.g., 112×112). While content-aware refining could offer finer control, it may also introduce additional computational cost. We will clarify this rationale in the final version and include adaptive refinement as a potential direction for future work.\n\n---\n\nWe hope we have addressed your questions. Please let us know your further concerns during discussion."}, {"review_id": "PlkA7TaRmz", "reviewer": "Reviewer_qzRH", "summary": "This study proposes FracFace, a fractal-based privacy-preserving face recognition framework that disrupts the spatial structure of frequency-domain features to weaken reconstruction attacks while retaining identity-critical information. It consists of two modules: Frequency Channels Refining (FCR): attenuates non-identity-related frequency bands (e.g., skin color, lighting, expression) to reduce feature sparsity and frequency interference; and Frequency Fractal Mapping (FFM): leverages fractal self-similarity and local chaos to remap the refined frequency channels into an irreversible fractal structure, imposing structured perturbations on deep representations. Evaluated on common benchmarks, the method delivers lower SSIM and higher LPIPS under U-Net and StyleGAN attacks while maintaining high recognition accuracy, outperforming six state-of-the-art privacy protection methods.", "questions": "1. Ablation on fractal depth k\n  - Please include a systematic study of how varying the number of FFM iterations (k) affects both privacy (e.g. SSIM/LPIPS under your reconstruction attacks) and recognition accuracy.\n  - Actionable Guidance: Report curves for k=1 through k=N (or the relevant range), and identify the “sweet spot” where additional iterations no longer yield meaningful privacy gains or begin to hurt accuracy. And show the results on StyleGAN reconstruction.\n\n2. Sensitivity to FBA pruning strength\n  - Can you quantify the impact of different low-frequency attenuation levels on privacy vs. utility? For instance, evaluate the performance under different levels.\n  - Actionable Guidance: Provide metrics on feature sparsity, SSIM/LPIPS, and recognition accuracy for each level, and recommend an optimal range.\n  - Evaluation Criterion: Demonstrating a clear privacy–utility curve with an identified optimal level band would strengthen the paper’s practical relevance and raise the score.\n\n3. Exact Definition & Computation of Protection (%)\n  - Please provide the precise formula or algorithm you used to compute the “Protection (%)” entries in Table 1. Specifically, state what quantities appear in the numerator and denominator (e.g. number of masked frequency channels over total channels, or number of pixels over total pixels), and include a short numerical example.\n\n4. Explain Separation of S(1) vs. S(2) in “Refined Frequency Channel\n  - The box labeled “Refined Frequency Channel” shows outputs S(1) and S(2) but gives no indication how they are computed or split. Please add a concise algorithmic description (pseudocode or bullet steps) or a small sub‐diagram indicating which operations produce S(1) vs. S(2).\n\n5. Per-channel frequency sensitivity analysis\n  - Which specific frequency bands contribute most to privacy leakage or recognition performance? A “mask-only” vs. “retain-only” experiment per band would clarify this.\n  - Actionable Guidance: For each grouped band (e.g. low/mid/high), measure the change in SSIM/LPIPS and accuracy when that band alone is kept or pruned. Present results as a heatmap.\n  - Evaluation Criterion: If you identify and justify a minimal subset of channels that capture identity while minimizing leak risk, the methodological clarity and impact would improve.\n\nWe suggest consulting Wang et al. [1*]—in particular Sections 4.1 (“White-box Attacking Experiments”) and 4.2 (“Black-box Attacking Experiments”)—for both qualitative and quantitative reconstruction results across diverse attack scenarios. Additionally, see the ablation study in Wang et al. [2*], which examines the relationship between mask ratio and recognition performance.\n\n[1*] Yinggui Wang, Jian Liu, Man Luo, Le Yang, and Li Wang. Privacy-preserving face recognition in the frequency domain. AAAI 2022.\n[2*] K. Wang et al., “FaceMAE: Privacy-preserving face recognition via masked autoencoders,” 2022, arXiv:2205.11090", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: \n\n1. Originality: Introduced a fractal-based paradigm (FFM) to handle PPFR, bringing fresh insights into how self-similar, multi-scale structures can obscure frequency-domain cues.\n2. Comprehensive experiments: Evaluated privacy and recognition trade-offs against two different reconstruction models (U-Net and StyleGAN), demonstrating consistent robustness improvements across both white-box and black-box scenarios.\n3. Quantitative privacy gains: Reported clear improvements—15% to 60% better attack resistance—in SSIM and LPIPS metrics, clearly showing reduced visual recoverability of facial features.\n4. High recognition accuracy: Despite aggressive obfuscation, the method maintained top-tier face-verification performance on multiple public benchmarks (LFW, CelebA, AgeDB, CFP-FP, CALFW, CPLFW), underlining balanced privacy-utility performance.\n5. Thorough comparative study: Benchmarked against six state-of-the-art PPFR methods, providing a clear picture of where FracFace stands relative to existing cryptographic and non-cryptographic techniques.\n\n\nWeaknesses:\n\n1. The impact of different iteration depths k in the FFM module on both privacy protection and recognition accuracy is not fully explored—no systematic study shows how varying k levels shift the trade-off.\n2. The paper does not analyze how varying the attenuation intensity in the Frequency Band Attenuation step affects the balance between residual visual leakage and identity retention.\n3. Although the FCR module is designed to suppress non-identity frequency bands, the manuscript lacks a detailed channel-level study identifying which exact bands most strongly impact privacy leakage versus recognition performance. In particular, the brief mention in [line 158] of “predefining three frequency ranges” is too cursory—additional experiments should systematically evaluate each individual band’s contribution to both reconstruction vulnerability (e.g., SSIM/LPIPS changes when that band alone is masked or retained) and recognition utility, thereby providing a clear guidance on optimal frequency-wise pruning.\n4. The paper does not specify how the \"Protection (%)\" values in Table 1 are computed, making the metric unreproducible and somewhat subjective. [lines 206–207] state: \"Green denotes the proportion associated with the privacy protection level.\" However, it remains unclear whether this refers to the proportion of original frequency channels (or visual cues) that are effectively concealed by each method.\n5. In Fig. 1’s pipeline, the depiction of the “Frequency Band Attenuation” shows a pruning space P that does not appear to match the formula [line 159] P = P₁ ∪ P₂ ∪ P₃ given in the text. Furthermore, in the “Refined Frequency Channel” box there is no indication of how S(1) and S(2) are separated. The paper needs to include a more detailed explanation of these points.", "was_revised": true, "final_justification": "The authors have addressed all of my concerns and questions. After considering the comments from other reviewers and their communications with the authors, I have finally decided to raise my rating.", "rebuttal": "Dear **Reviewer qzRH**,\n\nThank you for your positive evaluation and encouraging remarks! We carefully investigated each question and weakness trying to address them to the best of our capability within the time frame. Our response are as following:\n\n### **Response to Q1. Ablation on Fractal Depth $k$**\n> *Please include a systematic study of how varying the number of FFM iterations (k) affects both privacy and recognition accuracy.*\n\nThank you for this constructive suggestion. We conducted the requested ablation study by varying the number of fractal iterations $k$, evaluating both recognition accuracy and privacy robustness under the StyleGAN-based reconstruction attack.\n\nThe results are summarized in **Table R1** below. As $k$ increases, privacy improves consistently—LPIPS increases from 0.53 to 0.84, and SSIM drops from 0.52 to 0.26—indicating stronger obfuscation. However, recognition accuracy begins to degrade beyond $k=2$. Notably, $k=2$ emerges as the optimal trade-off, offering a 20% LPIPS gain over $k=1$ with a minimal 0.02% drop in accuracy. *The results on StyleGAN reconstruction images cannot be upload in the response. We will include them in the final version.*\n\n**Table R1: Impact of Fractal Depth $k$ on Privacy–Utility Trade-off**\n\n| $k$ | Accuracy (%) | SSIM ↓ | LPIPS ↑ |\n| --- | ------------ | ------ | ------- |\n| 1   | 99.71        | 0.5227 | 0.5291  |\n| 2   | 99.69        | 0.4015 | 0.6353  |\n| 3   | 96.46        | 0.3729 | 0.7925  |\n| 4   | 92.13        | 0.2580 | 0.8357  |\n\n### **Response to Q2. Sensitivity to FBA Pruning Strength**\n\n> *“Can you quantify the impact of different low-frequency attenuation levels on privacy vs. utility?”*\n\nThank you for the insightful suggestion. To assess the sensitivity of our system to FBA pruning strength, we conducted additional experiments varying the refining ratio in the FCR module using an energy-guided channel pruning strategy. Specifically, we ranked DCT channels by global energy and retained the top-N channels corresponding to five pruning levels: 20%, 40%, 50%, 60%, and 80% (inspired by FaceMAE). We evaluated the effect of each level on feature sparsity, privacy robustness (SSIM / LPIPS), and recognition accuracy. The results are presented in **Table R2** below.\n\n**Table R2: Effect of FBA Pruning Strength on Privacy–Utility Trade-off**\n\n| Refining Ratio | Accuracy (%) | SSIM ↓ | LPIPS ↑ |\n| ------------- | ------------ | ------ | ------- |\n| 20%           | 99.83        | 0.7857 | 0.3184  |\n| 40%           | 99.71        | 0.6291 | 0.4833  |\n| 50%           | 99.69        | 0.3012 | 0.6839  |\n| 60%           | 89.26        | 0.3109 | 0.7294  |\n| 80%           | 87.24        | 0.2793 | 0.8605  |\n\nWe observe a clear privacy–utility trade-off curve: as refining becomes stronger (i.e., more channels are removed), privacy consistently improves (e.g., LPIPS: 0.32 → 0.86), but recognition utility drops, especially beyond 50%. Notably, 50% pruning yields the best balance, increasing LPIPS by 35% over the 20% case while maintaining near-perfect recognition accuracy (99.69%). Beyond that point, privacy stays about the same, while accuracy keeps going down (e.g., 60% → 89.26%, 80% → 87.24%). These results highlight that moderate refining (40–50%) forms an optimal operating range, effectively improving privacy with minimal impact on utility. This justifies our default choice of 50% refinement in the main paper. We will include this ablation and its discussion in the final version. The figure on privacy-utility curve cannot be upload in the response. We will include them in the final version. We thank the reviewer for encouraging this more comprehensive analysis. \n\n### **Response to Q3. Definition and Computation of Protection (%)**\n\n> *“Specifically, state what quantities appear in the numerator and denominator, and include a short numerical example.”*\n\nThank you for your insightful question. We agree that the computation of **Protection (%)** warrants a precise definition and will add it to the final version along with a numerical example.\n\n*Definition:*  Protection (%) represents the share of frequency-domain channels that are either (i) filtered out by FCR or (ii) structurally disrupted by FFM, as a percentage of the total number of channels.\n\nProtection (%) = $\\frac{|P_{mask}|+|P_{rempa}|}{P_{total}} $\n\nWhere:\n\n* $P_{\\text{total}}$: total number of DCT frequency channels (e.g., 192 for 12×16 DCT).\n* $P_{\\text{mask}}$: number of low-energy channels pruned by FCR.\n* $P_{\\text{remap}}$: number of remaining channels remapped by FFM.\n\n*Here is a numerical example illustrating the calculation of Protection (%):*\n\n**Table R3: Example calculation**\n| Method             | $P_{total}$ | $P_{mask}$ | $P_{remap}$ | Protection (%) |\n|--------------------|---------|--------|---------|----------------|\n| PartialFace        | 192     | 130    | 0       | 67.7           |\n| FaceObfuscator     | 128     | 112    | 0       | 87.5           |\n| FracFace (Ours)    | 192     | 111    | 81      | 100.0          |\n\nThis unified formulation allows for consistent, interpretable comparison of visual disruption levels across methods, regardless of the specific masking or remapping mechanism. \n\n### **Response to Q4. Pseudocode**\n> *The box labeled “Refined Frequency Channel” shows outputs S(1) and S(2) but gives no indication how they are computed or split.*\n\n*Thank you for the helpful suggestion. We acknowledge the specific calculation process of S(1) and S(2) lacked clarity and appreciate the opportunity to improve our explanation*. To construct the two refined frequency channel groups S(1) and S(2), we apply a index-like traversal over a frequency index grid. The **pseudocode** is as follows:\n```python\n# Step 1: Create frequency index grid\nF = create_matrix(M=12, N=12)\n\n# Step 2: S-shaped traversal\nfreq_list = []\nfor i in range(M):\n    row = F[i]\n    if i % 2 == 0:\n        freq_list.extend(row)         # left-to-right\n    else:\n        freq_list.extend(row[::-1])   # right-to-left\n\n# Step 3: Split into two groups\nS1 = freq_list[0:80]\nS2 = freq_list[80:161]\n```\nThe indexing method is illustrated in **Fig. 8**. We will include the pseudocode and a sub-diagram showing how S(1) and S(2) are generated to improve clarity in the final version.\n\n### **Response to Q5. The Sensitivity Analysis of Different Frequency Bands**\n> *Which specific frequency bands contribute most to privacy leakage or recognition performance?*\n\nThank you for the insightful suggestion. Following your advice, we conducted a per-band frequency sensitivity analysis to investigate which frequency ranges contribute most to privacy leakage and recognition performance. As shown in **Table R4**, we conducted both “retain-only” and “mask-only” experiments on three grouped frequency bands (Low, Mid, High): \n* Low-frequency channels are strongly correlated with visual appearance and structure. Masking them significantly improves privacy (↓SSIM, ↑LPIPS) without hurting recognition (99.41% accuracy), confirming their dominant role in visual leakage.\n* Mid-frequency channels are most critical for identity representation. Retaining only mid-bands achieves 97.69% accuracy, while masking them causes a notable drop to 92.15%, indicating their contribution to discriminative identity cues.\n* High-frequency channels provide limited standalone utility but help encode textures. Masking them has little impact on accuracy but slightly reduces privacy performance. (Clarity: We just provide the result data, as shown in Table R3. And we are unable to provide the visualization result of heatmap per the rebuttal policy, but we will put the result in the final version)\n\n**Table R4 The Sensitivity Analysis of Different Frequency Bands**\n| Group | Mode       | Accuracy (%) ↑ | SSIM ↓   | LPIPS ↑  |\n|-------|------------|----------------|----------|----------|\n| Low   | Retain     | 93.58          | 0.7526   | 0.3583     |\n| Low   | Mask       | 99.41          | 0.4253   | 0.5840    |\n| Mid   | Retain     | 97.69          | 0.6291   | 0.4269     |\n| Mid   | Mask       | 92.15          | 0.5832   | 0.4503    |\n\n### **Response to W1. Ablation on Fractal Depth k**\nPlease see our response to Q1 for detailed analysis.\n\n### **Response to W2. Sensitivity to FBA Refining Strength**\nPlease see our response to Q2 for detailed analysis.\n\n### **Response to W3. The Sensitivity Analysis of Different Frequency Bands**\nPlease see our response to Q5 for detailed analysis.\n\n### **Response to W4. Definition and Computation of Protection (%)**\nPlease see our response to Q3 for detailed analysis.\n\n### **Response to W5. Clarifying “P = P₁ ∪ P₂ ∪ P₃” and the generation of S(1)/S(2)**\n\nThank you for pointing out this important issue. We confirm that our implementation follows the formula $P = P₁ ∪ P₂ ∪ P₃$ as stated in line 159. After converting the input image to the frequency domain, we retain a three-layer structure, where each $P_{i}$ corresponds to a group of channels ordered by energy. Specifically, $P$₁ captures dominant identity-relevant signals, while $P₂$ and $P₃$ reflect more fine-grained appearance traits such as illumination and texture. Our FCR module applies layer-wise frequency refining to remove appearance-related components while preserving discriminative features for recognition.\nRegarding the separation of the refined channels into $S(1)$ and $S(2)$, we apply an S-shaped scanning pattern across the frequency grid and split the resulting index sequence evenly into two subsets. This design ensures spatial and frequency diversity across both groups (see our response to **Q4** for more detail).\nWe appreciate this helpful suggestion and will revise **Fig. 1** in the final version to clearly reflect both the layered pruning structure and the generation of $S(1)$ and $S(2)$.\n\n---\nWe hope we have addressed your questions. Please let us know your further concerns during discussion."}, {"review_id": "jgug2L7G5U", "reviewer": "Reviewer_qSTt", "summary": "The paper proposes FracFace, a privacy-preserving face-recognition (PPFR) framework that (i) first prunes low-utility frequency channels via a Frequency-Channel-Refining (FCR) block and then (ii) remaps the retained spectrum into a recursive fractal index space through a Frequency-Fractal-Mapping (FFM) module. The authors argue that FCR alleviates energy sparsity that leaves visual cues in prior PPFR methods, while FFM introduces a non-linear, non-invertible permutation that further obfuscates spatial regularities. Experiments on six benchmarks (LFW, CelebA, AgeDB, CFP-FP, CALFW, CPLFW) and two attack models (U-Net, StyleGAN) show that FracFace retains ≥ 95 % verification accuracy on LFW while lowering SSIM of reconstructed faces by 25 –60 % relative to MinusFace, PartialFace and FaceObfuscator. The authors claim (a) state-of-the-art privacy–utility trade-off and (b) 15 – 60 % higher resistance to white- and black-box reconstruction.", "questions": "1. Can you provide a quantitative argument (e.g., entropy loss, one-way permutation hardness) that FFM is non-invertible even when its parameters are known?\n\n2. How does FracFace fare against a reconstruction adversary that knows the fractal mapping and trains an end-to-end diffusion or GAN inversion accordingly?\n\n3. What is the privacy–utility trade-off when varying fractal layers and pruning ratios?", "limitations": "No.\nThe manuscript lacks an explicit “Limitations & Societal Impact” section and therefore omits several critical issues. I recommend that the authors:\n\n1. discuss how privacy degrades if the fractal mapping becomes public, and outline mitigation (e.g., key rotation, hardware obfuscation).\n2. reflect on how the technique might be deployed for mass-surveillance systems that further erode civil liberties, and suggest policy or technical safeguards (opt-in consent, on-device storage only, revocation).\n3. include template-linkability, gradient leakage in federated settings, and adaptive diffusion-based reconstruction as open challenges.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 1, "strengths_and_weaknesses": "The empirical section is competently executed, featuring six public benchmarks, two reconstruction attackers (U-Net and StyleGAN2), and comparisons with eight recent PPFR baselines, which provide the reader with a reasonably broad picture of utility–privacy trade-offs. Implementation details are explicit, and reproduced numbers for ArcFace and PartialFace align with those reported in prior work, suggesting sound experimental hygiene. \n\nThe paper follows a logical flow (threat model → method → experiments → analysis); equations are typeset cleanly, and most figures (e.g., the t-SNE visual in Fig. 2 and qualitative reconstructions in Fig. 3) are easy to digest.  Template security in face recognition remains a pressing problem; a lightweight transform that can be embedded at the sensor level would be valuable for edge deployments, making the topic well-motivated.  \n\nDespite the fractal rhetoric, the core technical novelty reduces to a fixed permutation of the retained DCT channels layered on top of a pruning heuristic already explored by MinusFace and PartialFace; no new learning objective, loss function, or cryptographically grounded mechanism is introduced. Because the permutation is deterministic and global, an attacker who uncovers or guesses it can revert the mapping, collapsing FracFace to a vanilla sparsity mask. The paper offers neither an information-theoretic bound nor a key-management strategy, leaving its “non-invertible” claim largely rhetorical. \n\nThe reconstruction gap over prior art (≈ 0.03–0.06 LPIPS on LFW) is marginal and reported only for two dated attackers; stronger diffusion-based or adaptive white-box attacks are absent, so it is unclear whether the method would hold up in practice. There is no ablation on fractal depth or pruning ratio, making it hard to judge robustness. Dataset coverage is narrow (frontal, high-resolution faces); low-resolution surveillance and varying pose scenarios, where frequency scrambling could behave differently, are untested.", "was_revised": true, "final_justification": "The authors’ rebuttal offers several clarifications and additions that partially address the raised concerns, but key issues remain unresolved. Regarding the non-invertibility of the proposed Fractal Feature Mapping (FFM), the authors argue that exponential index growth leads to aliasing via modulo projection, rendering the mapping non-injective. While this supports collision-based non-invertibility, it falls short of establishing cryptographic hardness or one-wayness, as no complexity analysis or entropy bounds are provided. \n\nOn robustness against adaptive attacks, the authors incorporate PGDiff and StyleGAN-based attacks, assuming the transformation is known, which is a step forward. However, these attacks are not optimized for the transformed domain, and the rebuttal concedes that adversarial performance could improve with targeted retraining, leaving concerns about inversion robustness unresolved. The inclusion of ablation studies on fractal depth and pruning is sufficient and directly responds to prior requests, offering interpretable privacy–utility trade-offs. In terms of novelty, the authors reiterate FFM's structural aliasing mechanism and random kernel instantiation but introduce no new learning paradigms or theoretical constructs beyond existing permutation-based privacy-preserving methods. \n\nOverall, although the rebuttal improves the paper by providing ablations and a stronger, albeit still generic, attack baseline, the fundamental concerns about deterministic mapping, robustness under adaptive attacks, and limited methodological novelty are only partially addressed. Therefore, I stand by my original recommendation.", "rebuttal": "Dear **Reviewer qSTt**,\n\nWe sincerely thank you for your valuable comments and suggestions.  We respond to your questions and comments below:\n\n### **Response to Q1. On the Non-Invertibility of FFM**\n\nThank you for this insightful question. The non-invertibility of the FFM is proved in [S1]. We conducted a quantitative analysis to support our claim, drawing from both entropy-based evidence and one-way permutation hardness. \n\nThe mapping defined by FFM is:\n\n\\begin{equation}\n\\mathcal{F}^{[k]}[i,j] = M_0[i,j] + \\sum_{\\ell=1}^{k} (b_\\ell - 1) \\cdot \\beta_\\ell, \\quad \\text{where } \\beta_\\ell = \\prod_{s=1}^{\\ell - 1} b_s,\\ \\beta_1 = 1.\n\\end{equation}\n\nThis mapping grows rapidly with depth \\$k\\$, and the output is projected into a fixed index space of size denoted as \\$C\\$. Notably, each element of $\\mathcal{F}^{[k]}$ increases proportionally with $\\beta_k \\in \\mathcal{O}(3^{2k})$, implying that the range of $\\mathcal{F}^{[k]}$ expands exponentially with $k$. To project the resulting indices into a fixed channel space, we define a modulo-based index mapping $\n\\psi[i,j] = \\mathcal{F}^{[k]}[i,j] \\bmod C.\n$\n\nHowever, due to the rapid growth of $\\mathcal{F}^{[k]}[i,j]$, the modulo projection inevitably causes aliasing:\n\n\\begin{equation}\n\\exists\\ (i_1,j_1) \\neq (i_2,j_2) \\quad \\text{s.t.} \\quad \\psi[i_1,j_1] = \\psi[i_2,j_2].\n\\end{equation}\n\nThis proves that $\\psi$ is not injective, and thus, the inverse $\\psi^{-1}$ is not uniquely defined $\n\\not\\exists\\ \\psi^{-1} : \\mathbb{Z}_C \\to \\mathbb{Z}^{n \\times n}.\n$\n\nThis is not incidental, but structurally guaranteed due to the following:\n\n1）The maximum index in $\\mathcal{F}^{[k]}$ grows as $\\max(\\mathcal{F}^{[k]}) \\in \\mathcal{O}(3^{2k})$;\n\n2）The output space of $\\psi$ is fixed at size $C$;\n\n3）Collisions are guaranteed. For example, when \\$C = 81\\$, \\$k = 2\\$, and \\$b\\_1 = b\\_2 = 3\\$, \\$\\mathcal{F}^{\\[2]}\\$ yields values in $\\[1,162]\\$, projected into $\\[1,81]\\$ by **mod**. \n\nThe FFM transformation is non-invertible even under known parameters, due to its exponential growth in index space and subsequent modulo projection. This structural non-injectivity ensures that channel remapping cannot be deterministically reversed by an adversary, which underpins FFM's privacy-preserving design.\n\n**References**\n\n[S1] Double parameters fractal sorting matrix and its application in image encryption.\n\n### **Response to Q2. Robustness Against Diffusion/GAN-based Reconstruction**\n\nThank you for raising this important question. We extended our evaluation to stronger adversaries by including U-Net, StyleGAN, and Diffusion-based inversion models. As shown in **Table R1**, we benchmarked these attacks on SSIM, MSE, and IDS.\n\n**Table R1: Comparison of Reconstruction Attacks Against FracFace**\n\n| Attack Model  | SSIM ↓ | MSE ↓ | IDS ↓ | Inference Time |\n| ------------- | ------ | ----- | ----- | -------------- |\n| **U-Net**     | 0.3613  | 0.0853 | 0.3028  | \\~1.2 sec      |\n| **StyleGAN**  | 0.3151  | 0.0962 | 0.4936  | \\~5.3 sec      |\n| **Diffusion (PGDiff in NeurIPS2023)** | 0.3809  | 0.0623 | 0.4325  | \\~300 sec      |\n\nWhile diffusion models can generate finer textures, they do not significantly outperform U-Net or StyleGAN in terms of privacy leakage under our setting. In fact, they slightly underperform in IDS and SSIM, and incur substantially higher computational cost (e.g., \\~300s per image even with only 20 denoising steps).\n\nImportantly, this experiment demonstrates FracFace’s resilience even against strong reconstruction adversaries with full knowledge of the fractal mapping. Despite that, FracFace still achieves low identity leakage and reconstruction quality, validating its effectiveness across diverse threat models.\n\nDue to time limit, we used the SOTA pre-trained diffusion model []. It may be possible to train an end-to-end diffisuion or GAN inversion model that perfoms better than the pre-trained model. But we believe the improvement would not be significant, because FracFace fundementally removed detailed visual clues (see Fig. 3) that the reconstruction model relied on.\n\n### **Response to Q3. Privacy–Utility Trade-off under Varying Fractal Layers and Pruning Ratios**\n\nWe performed the ablation studies to investigate how fractal layers \\$k\\$ and pruning ratio affect the privacy–utility trade-off.\n\n#### **(1) Impact of Fractal Layers \\$k\\$**\n\nWe varied the number of FFM iterations \\$k\\$ and evaluated both recognition accuracy and privacy robustness under StyleGAN-based reconstruction. As shown in **Table R2**, increasing \\$k\\$ leads to stronger privacy protection (LPIPS ↑, SSIM ↓), but accuracy begins to decline beyond \\$k=2\\$. We identify \\$k=2\\$ as the optimal point, offering a 20% LPIPS gain over \\$k=1\\$ with only a 0.02% drop in accuracy. Visualization results will be included in the final version, per policy.\n\n**Table R2: Effect of Fractal DepthLayers \\$k\\$ on Privacy–Utility Trade-off**\n\n| \\$k\\$ | Accuracy (%) | SSIM ↓ | LPIPS ↑ |\n| ----- | ------------ | ------ | ------- |\n| 1     | 99.71        | 0.5227 | 0.5291  |\n| 2     | 99.69        | 0.4015 | 0.6353  |\n| 3     | 96.46        | 0.3729 | 0.7925  |\n| 4     | 92.13        | 0.2580 | 0.8357  |\n\n#### **(2) Impact of FBA Pruning Ratios**\n\nTo evaluate sensitivity to pruning, we ranked DCT channels by global energy and retained the top-N at five pruning levels: 20%, 40%, 50%, 60%, and 80%. As shown in **Table R3**, stronger pruning improves privacy but degrades accuracy—especially beyond 50%. The 50% level provides the best trade-off: LPIPS improves by 35% over the 20% case, while accuracy remains high (99.69%).\n\n**Table R3: Effect of Pruning Ratios on Privacy–Utility Trade-off**\n\n| Pruning Ratios | Accuracy (%) | SSIM ↓ | LPIPS ↑ |\n| ------------- | ------------ | ------ | ------- |\n| 20%           | 99.83        | 0.7857 | 0.3184  |\n| 40%           | 99.71        | 0.6291 | 0.4833  |\n| 50%           | 99.69        | 0.3012 | 0.6839  |\n| 60%           | 89.26        | 0.3109 | 0.7294  |\n| 80%           | 87.24        | 0.2793 | 0.8605  |\n\nThese results confirm a clear privacy–utility trade-off. The setting with \n\\$k=2\\$, and 50% pruning achieves strong privacy protection while keeping recognition accuracy high. We will include these findings and full visualizations in the final version.\n\n### **Response to Weakness 1** \nWe thank the reviewer for raising this critical question regarding the novelty and security of our approach.\n\nFirstly, prior works such as MinusFace and PartialFace have explored coefficient pruning in the frequency domain through static filtering. However, they do not address the deeper issue of frequency-domain sparsity, a structural vulnerability that makes frequency maps prone to reconfiguration and inversion. In contrast, FracFace explicitly targets this limitation through a novel fractal-based remapping mechanism, called FFM. As shown in our ablation results (Fig. 6 and Table 3), pruning alone is insufficient to disrupt reconstructive cues, and it is the FFM module that fundamentally improves privacy while preserving recognition performance.\n\nFurthermore, FFM operates through recursive fractal expansion followed by a modulo projection, which inherently leads to non-injective index mappings—that is, multiple inputs may map to the same output. This structural aliasing ensures that even when the mapping function is fully known, it cannot be uniquely inverted. We provide a quantitative argument of this non-invertibility in **Q1**.\n\nFinally, we operate under a strong white-box threat model, where the adversary is assumed to know the FFM mechanism. However, they do not know the specific instance of the random base Fractal Kernal $M_0$, and Fractal Lattice $L_0$, which varies across deployments and is not recoverable from outputs alone (see **Q2**).\n\nWe will include these analyses in the final version to clarify that FracFace innovatively applies fractal remapping to address the above challenge, moves beyond simple pruning via fractal remapping, and achieves provable non-invertibility even under strong threat models.\n\n### **Response to Weakness 2** \n\nWe address each concern as follows:\n\n**First**, while LPIPS improvements may appear numerically small, we emphasize that privacy leakage is assessed through multiple complementary metrics. In addition to LPIPS, we report IDS, SSIM, and MSE, which together offer a fuller picture of reconstruction quality and identity preservation (see Table 2 in our paper). **Next**, we further evaluated FracFace under stronger adversaries, including U-Net, StyleGAN, and a diffusion-based inversion model. Detailed results and discussions are provided in **Q2**. **Moreover**, we have conducted ablation study on fractal depth and pruning ratio (see **Q3**). **In terms of dataset coverage**, FracFace have evaluated on 8 public datasets, including AgeDB, CFP-FP, IJB-B, and IJB-C, which introduce challenges such as pose variation, lighting, partial occlusion. In particular, CelebA, IJB-B,and IJB-C reflect unconstrained and non-frontal conditions. FracFace consistently outperforms baselines under these scenarios (Table 2), validating its robustness beyond frontal high-quality settings. **Finally**, on low-resolution scenarios, we note that degraded images often provide inherent privacy benefits by reducing recoverable visual details. This aligns with our core goal of obscuring identity while preserving utility. \n\n### **Response to Limitations** \n\nThank you for raising this important point. While a brief discussion of limitations is included in **Appendix A.6** of the main paper, we agree that a more explicit and thorough treatment of the suggested issues would strengthen the work. We will incorporate these discussions in the final version. Due to the word limit of this response, we are unable to elaborate in detail here but would be happy to provide further clarification during the next review phase.\n\n---\n\nLooking forward to further discussion and feedback."}]

Author rebuttal:

Meta-review: The paper proposes a new method for privacy-preserving face recognition based on the principle of obfuscating soft biometrics features. While frequency components are widely-used in privacy-preserving approaches, the authors propose a non-invertible remapping strategy making use of fractal kernels. This provides robustness to reconstruction attacks while maintaining high recognition performance, which is a challenging task. 

The paper has a focus on the obfuscation-based perspective for privacy-preserving face recognition, and is more limited on the crypto-based perspective of the problem. While this is not a negative point, the authors might clarify this aspect in order to avoid missed expectations on deterministic mapping. The rebuttal improves the paper by providing ablations and a stronger (generic) attack baseline.

Finally, the paper contributes techniques with potential practical impact in the privacy-preserving ML community, with comprehensive and solid validation that can be published.
Decision: Accept (poster)