UIFace: Unleashing Inherent Model Capabilities to Enhance Intra-Class Diversity in Synthetic Face Recognition
Authors: Xiao Lin
Yuge Huang
Jianqing Xu
Yuxi Mi
Shuigeng Zhou
Shouhong Ding
Venue: ICLR 2025 Poster (2025)
Keywords: face recognition
face image synthesis
diffusion model
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Face recognition (FR) stands as one of the most crucial applications in computer vision. The accuracy of FR models has significantly improved in recent years due to the availability of large-scale human face datasets. However, directly using these datasets can inevitably lead to privacy and legal problems. Generating synthetic data to train FR models is a feasible solution to circumvent these issues. While existing synthetic-based face recognition methods have made significant progress in generating identity-preserving images, they are severely plagued by context overfitting, resulting in a lack of intra-class diversity of generated images and poor face recognition performance. In this paper, we propose a framework to $\textbf{U}$nleash model $\textbf{I}$nherent capabilities to enhance intra-class diversity for synthetic face recognition, shorted as $\textbf{UIFace}$. Our framework first train a diffusion model that can perform denoising conditioned on either identity contexts or a learnable empty context. The former generates identity-preserving images but lacks variations, while the latter exploits the model's intrinsic ability to synthesize intra-class-diversified images but with random identities. Then we adopt a novel two-stage denoising strategy to fully leverage the strengths of both type of contexts, resulting in images that are diverse as well as identity-preserving. Moreover, an attention injection module is introduced to further augment the intra-class variations by utilizing attention maps from the empty context to guide the denoising process in ID-conditioned generation. Experiments show that our method significantly surpasses previous approaches with even less training data and half the size of synthetic dataset. More surprisingly, the proposed $\textbf{UIFace}$ even achieves comparable performance of FR models trained on real datasets when we increase the number of synthetic identities.

Published as a conference paper at ICLR 2025
UIFACE: UNLEASHING INHERENT MODEL CAPABIL- ITY TO ENHANCE INTRA-CLASS DIVERSITY IN SYNTHETIC FACE RECOGNITION
Xiao Lint.,, Yuge Huang-", Jianqing Xu', Yuxi Shuigeng Shouhong Ding
Tencent Youtu Lab 2uudan University
(xiaolin, yugehuang, joe jexu, ericshdi ng) @tencent.com
{yxmi20, sgzhou fudan. edu. on
ABSTRACT
Face recognition (FR) stands as one of the most crucial applications computer vision. The accuracy of FR models has significantly improved in recent years due to the availability of large-scale human face datasets. However, directly us- ing these datasets can inevitably lead to privacy and legal problems. Generating synthetic data to train FR models a feasible solution to circumvent these issues. While existing synthetic-based face recognition methods have made significant progress in generating context overfitting, resulting lack of intra-class diversity of generated images and poor face recognition performance. In this paper, we propose a framework Unleash Inherent capability of the model enhance intra-class diversity for synthetic face recognition, shortened as UlFace. Our framework first trains dif- fusion model that can perform sampling conditioned on either identity contexts or learnable empty context. The former generates identity-preet preserving images but lacks variations, while the latter exploits the model's intrinsic ability to synthe- size satra-claasddivense novel two-stage sampling strategy during inference to fully leverage the strengths of both types of contexts, resulting in images that diverse well identity- preserving. Moreover, an attention injection module introduced to further aug- the intra-class variations by utilizing attention maps from the empty con- text to guide the sampling process in ID-conditioned generation. Experiments show that our method significantly surpasses previous approaches with even less training data and half the size of synthetic dataset. The proposed UIFace even achieves comparable performance with FR models trained on real datasets when we further increase the number of synthetic identities. Code will be released at ttwpps/s//iiituuucooooooo http:///.cccmmeeeennn//eeaaa
images, they are severely plagued by
images but with random identities. Then adopt
INTRODUCTION
Recent years have witnessed incredible improvements in the accuracy of FR models. This can be attributed to the advancements in model architectures (Boutros et al. 2022b Huang et al. 2017). margin-based loss functions (Deng et al. 2019 Huang et al. 20200). and more importantly. the avail- ability of large-scale face datasets (Huang et al.) 2008 Kemelmacher Haaeemmmcce-SSSzzemman et al. [016 Zhu al. 202T), which contain millions of identities with rich variations in age, pose and expression.
However, these large-scale datasets are often collected directly from the Internet, without the ex- plicit consent of individuals, which inevitably leads to privacy and legal issues (Regulation) 2016). Moreover, these datasets suffer from challenges about noisy labels (Wang et al. 201 8a (019) and imbalanced class distribution (Liu et al. 2019 Yi al., 2014). In other words, images with the same label may belong different individuals, and there a significant disparity in the number of images among different identities. These drawbacks limit the further application of real face datasets.
"Equal contribution Corresponding author

---

Published as a conference paper at ICLR 2025
ImprovedRecall
FR accuracy
76.0
96
0.55
95.05
CASIA-Rea Reea
94
93.27
0.45
92
31.5
IDiff-face
90
0.35
88.2
88
Ours samplss from different datasets
IDiff-ace
CASIA-Real
IDiffface
Ours
CASIA Real
b) quantittive comparison dataset diversity
c accuracy of the FR models
Figure I: Visualization of samples from different datasets. Samples from the real face dataset CASIA exhibit variations in attributes such as pose, expression and illumination. However, when we synthesize face data using previous method IDiff-Facce (Boutros et al. 2023), the generated images show poor diversity, more specifically, similar expressions and poses, which is caused by identity overfitting. In contrast, our method can generate wider variety of images, thereby enhancing the accuracy of the trained FR model. b) Quantitative comparison of dataset diversity. We apply LPIPS and Improved Recall to measure the diversity of different datasets. A higher value indicates better diversity. c) Quantitative comparison of final accuracy of the FR models.
Generating synthetic data mon-existent identities to train FR model, also known as synthetic face recognition, is feasible solution to the above issues suggested by DeAndres- -amee et al. (2024).
There have already been some explorations in this field. The initial methods (Shen et al., 2018
Deng al. 2020) employ Generative Adversarial Networks (Goodfellow al.) 2020) to generate synthetic face data and utilize the disentangled latent space to enhance the controllability. However, GAN-based methods have been shown generate only limited number of unique identities (Kim) 2023). leading to poor generalization of the trained FR models. Recently, diffusion models have made sigmificant advancements in image generation (Song al. 2020aa] Dhariwal Nichol 2021 Song et al.] 2020b; Rombach et al.) 2022). Some diffusion-basse methods (Wang et al, Li al. 2024b) Boutros et al.) 2023 Papantoniou et al., 2024 Zhang al. 2024) have been proposed for ddentity-preservingg face generation, which is achieved by conditioning the generation process with identity contexts, i.e., identity features extracted from the pretrained FR models (Deng et al. 2019 [Boutros al.,, (022a). It has been demonstrated that diffusion-basee methods are capa- ble generating greater variety unique identities compared to GAN-based methods (Kim al.) 2023 showing highly promising potential for real applications. Yet, high-quality human face dataset does not just imply a large number of identities but also requires good intra-class diversity. Specifically, the synthetic face images need exhibit variations in attributes such expression, illumination and pose.
ct al.
2024
While existing methods have made significant progress, they still suffer from context overfitting Boutros et al. 2023). More specifically, a fixed identity context not only determines the identity of generated images but also limits their the variations of identity-irelleant attributes, resulting in in- sufficient intra-class diversity. As illustrated in Figure D.. previous methods often exhibit a reduced diversity of synthetic images, i.e., lack of variations in expression and pose, compared to real-world dataset. To further demonstrate this issue, following Kim et al. (2023) and Papantoniou et al.) (2024), we utilize L.IPS (Zhang et al. 2018) and Improved Recall (K (Kynkäänniemi et al.[ to quantita- tively measure the intra-class diversity of these datasets. As shown in Figure 1pp. the diversity of the
synthetic dataset generated by previous approach significantly lags behind that of the real dataset CASIA-Webface (Yi et al. 2014), resulting in poor performance of the trained FR model (Figure Do.. Although methods such as Contextual Partial Dropout (Boutros et al. 2023). style condition extractor (Kim al.] (202)) and paired data generation (He et al.) 2024) have been proposed alle- viate this issue, they either rely on complex network designs or require the introduction additional training data, and still have significant performance gap compared to real data-based methods.
2

---

Published as a conference paper at ICLR 2025
Intuitively, the model inherently possesses the ability to generate diverse images because the real data used to train the generative model contain rich intra-class variations. However, when comes to specific identity context, such inherent capability is limited because of context overfitting. There- fore, in this paper, we propose a novel framework to unleash model inherent capability to enhance intra-class diversity for synthetic face recognition, shortened as UlFace. Specifically, we first train a diffusion model that can perform sampling conditioned on either specific identity contexts or learnable empty context. When conditioned on a specific identity context, the model generates identity- preserving images but with poor diversity due to context overfitting. On the other hand, when conducting generation conditioned on the empty context, the model can fully leverage its in- herent capability to synthesize various images but with random identities. This is because during training, we allow the empty context to generate all the images in the dataset. To To exploit the strength of both types of contexts to synthesize images that are diverse as well as adopt a novel two-stage generation strategy during inference. Our key observation that in the early stages of sampling, the model restores dientity-irrelevann contents, while in the later stage the model recovers the identity-relevant details (Section (.2). Thus, in the first stage, the diffusion model performs sampling conditioned the empty context for unleashing its intrinsic ability to generate large intra-class variations. In the second stage, the model generates identity-preservie details based on given identity conditions. Such two-stage strategy takes advantage of model's 's in- herent ability to enhance the diversity of conditional generation. Moreover, we propose an adaptive partitioning strategy to adaptively determine the boundary of these two stages for each sample based on the difference between adjacent cross- cross-attention maps. To fully harness the diversity of empty context-conditioned generation, we propose an attention injection module to use the attention maps from unconditional generation to guide the conditional generation process, which further leverages the model's inherent ability while maintaining ID-oonsistency.
we
In summary, our contributions are as follows.
We propose novel two-stage synthetic face recognition framework. By allowing the learn- able empty context and identity contexts to dominate different stages of the face generation, our method can fully leverage model's inherent capability to achieve satra-clas-diiersffie image generation while maintaining ddentity-preserving
We propose an adaptive partitioning strategy to adaptively determine the boundaries of two stages for different samples and attention injection module to utilize attention maps from unconditional generation to guide conditional generation. These designs can further unleash model's inherent ability enhance the intra-class variations of synthesized images.
Experimental results show that our method outperforms existing state-of-the-art methods by a significant margin by using fewer training data. We surpass current state-of-the-art methods even when synthesizing fewer than half the number of synthetic identities. When further increasing the number of identities, the proposed UIFace can surprisingly achieve comparable performance with those FR models trained on real datasets.
RELATED WORK
2.1 FACE RECOGNITION
Face recognition involves identifying or verifying person from an enrolled dataset. With the con- tinuous improvement of network architectures (He et al. 20166] Boutros et al., 2022b Huang et al.) 2017) and the introduction of novel loss functions (Boutros et al. 2022a| Deng et al., 2019 Huang et al. 2020 Waag et al.) 20188; Kim et 2022). the accuracy of face recogmition models has made remarkable advancements recent years. More importantly, the improvement performance is also attributed to large-scale face datasets (Huang et al.] 2008 Cao et al.) 2018 Zhu et al.) [022]] Guo et al. 2016, Kemelmacher-Shiieee Shiieerrr-Siiiaaanna et 2016 as well as datasets tailored to address spe- cific challenges (Zheng & Deng 2018 Zheng et al. 2017 Sengupta et al. 2016 Moschoglou al.) 2017]. Nevertheless, these datasets are collected directly from the Internet without explicit indi- vidual consent, leading to inevitable privacy and legal concerns. Moreover, they are suffered from noisy labels and long-tail tail problem YYi al. 2014). The above issues need to be carefully considered before using these datasets.
3

---

Published as a conference paper at ICLR 2025
minimal impact intra-class similarity
0.3
6.2
0.3
0.
0.
0.
500-
3050
1980
1000
1000
1000
1000
1200
1200
3950
different intervals using identity contexts
Figure 2: Effects of different sampling timesteps on identity. The x-axis represents timestep intervals where identity contexts are used as conditions. The empty context is used as substitute in timesteps that not covered in intervals. The y-axis represents the intra-class similarity of the generated face images. The maximum sampling timestep I is set to 1000.
2.2 FACE IMAGE SYNTHESIS
Image generation models have made remarkable progress, allowing the synthesis of high-quality
human face images. GAN-based methods (Brock 2018 Choi et al. 2018: Karras 2017
Karras
ett al. 2019; 2020; Yin et al., 2017) are successful pioneers among them.
They achieve identity-
preserving face generation by decoupling identity and attributes (Bao al.
2018). or introducing
additional classifiers and discriminators (Shen et al. 2018 DiscoFaceGAN (Deng et al. 2020)
introduces more fine-grained decoupled latent space enable precise control of synthesized faces.
Recently, diffusion models (Song et al. 2020a| Ho et al., 2020 Dhariwal & Nichol [021] Song al. 2020b; Rombach | [2222] have made significant advances in the field of image synthesis.
Some methods (Zhang et al. 2024 Wang et al.] [0244] Li et al. 2024b) have achieved high-fidelity
image generation with text and [ID-oonitiooeed diffusion model.
2.3 FACE RECOGNITION WITH SYNTHETIC DATASET
Replacing real datasets with synthetic face datasets can address the legal and class imbalance is- sues. SynFace QQuu al. 202T) applies DiscoFaceGAN (Deng al. 2020) synthesis identity- consistent data for training FR models. DCFace (Kim et al. 2023) proposes an diffusion- based method that employs decoupled style and identity encoders to generate dual conditions and demon- strates that DDPM (Ho et al.) (2200) can generate more unique identities than GAN-based methods, which is crucial for improving the accuracy of FR models. Some works (Boutros et al. [2023] Papan- toniou al. 2024) continue to enhance the quality of synthetic datasets, narrowing the performance gap between FR models trained on synthetic and real data. Nevertheless, as depicted in Figure [ existing methods still suffer from context overfitting, resulting in insufficient diversity in synthetic datasets. While some approaches (Boutros et al. 2023 Kim et al. [023] He et al., 2024) have been suggested to address this challenge, they often depend intricate network architectures or neces- sitate extra training data. In contrast, this paper explores to unleash model's inherent capability to intra-class-diversffee image generation for synthetic-based face recognition.
3 METHOD
The aim of synthetic face recognition is to generate synthetic face data for training FR models, thereby addressing the inherent issues of real datasets collected directly from the Internet, as discussed in Section I Formally, given the training dataset {r. we follow previous works
Overview.
Paaantonnouu et al., 2024; Boutros et al. (023) to utilize pre-trained FR model F to extract the
identity context c; F(x,, for each image, resulting in extended dataset {z,, e, N. Subsequently, we train a conditional diffusion model & that can perform face generation conditioned on either
specific identity contexts e; or en empty context G. (Section 3.1). Then we elucidate the crucial
4

---

Published as a conference paper at ICLR 2025
stagel: unconditional denoising
stage2: conditional denoising
4t-1
Attention Injection
M UNet
Adaptive stage Partition
conditional
contect
denaising
conteat
denolding
synthetic dataset
Figure? 3: Overview proposed UlFace. We propose a two-stage sampling strategy to unleash the intrinsic capability of the model generate diverse images. In the first stage, the model performs unconditional generation based on the empty context Ce- In the second stage, the model restores Adenttty-relevann details conditioned on specific identity contexts We further propose an adaptive stage partition strategy to determine the boundary of these two stages to and an attention injection module enhance diversity of synthetic dataset while maintaining identities.
observation that in the early sampling stage, the model restores dlentity-irrelevarl contents, whereas in the later stage it recovers details that determine the identity (3.2) Next, we introduce the two-stage sampling strategy to generate synthetic face images that simultaneously preserve identity and exhibit diversity (Section (.3). Then, we propose attention injection mechanism to further enhance the quality of the synthetic images (Section 3.4). Lastly, we present details about synthetic dataset The framework overview shown in Figure
generation (Section 3.5
3.1 PRELIMINARY
Latent Diffusion Model (Rombach al. (022)) distribution of latent representations z of real images æ. Variational Autoencoder is first applied to map the images to corrupted by adding Gaussian noise according to a predefined schedule during training. Formally,
employs denoising process to approximate the
latent space
E(x). The images are progressively
2t
2 VI Vā,ā, ā,e,
(1)
where t and 2g stand for the diffusion timestep and clean latent representation respectively. The reverse process (also known as the sampling process) is defined by the following equation
2t-1 = ue (2e; c)
a4 VI-
Eg(2e, t.c)
Vo
where the textual condition. The optimization of the UNet (Ronneherger et al. 2015) backbone eg is achieved by recovering the random noise and minimizing the following loss
L = E E. [||c c0 f, c)1[]]
In synthetic-baat face recognition, e the identity context extracted from a pre-trained FR model. Our method introduce an additional learnable empty context ce. During training iterations, the iden- tity context is probabilistically replaced with Ce. Thus, C. can be used to generate images of any identities present in the training data. When training finished, the model can generate identity- preserving images conditioned on any given c, or random face images conditioned on ce- Our train- ing process is consistent with previous ID-conditioned diffusion methods except for the introduction of empty context. Next, we will describe the inference process of our method.
3.2 EFFECTS OF DIFFERENT SAMPLING TIMESTEPS ON IDENTITY
In this subsection, we present our observations about effects of different sampling timesteps on the identity of generated images. For this purpose, inject identity contexts generation conditions
5

---

Published as a conference paper at ICLR 2025
in different timestep intervals and caleulate the intra-class similarity of generated images, which is based on the distance of features from a pre-trained FR model just as the standard metric in this field. As depicted in Figure [ during the initial generation phase (for larger t values), employing identity contexts as conditions exhibits minimal enhancement in the intra-class similarity of synthesized im- ages, while condition the generation process in the later timesteps (for smaller values) can rapidly improve the intra-class similarity. It implies that the model recovers identity-rrelevant contents at early sampling stage and restores identity-rlevant details in later stage. Based on these observa- tions, we propose two-stage sampling strategy to unleash the inherent diversity of the model while maintaining the identity consistency of generated images.
3.3 TWO-STAGE SAMPLING STRATEGY
As mentioned in the Section | previous synthetic- based methods suffer from the overfitting of iden- tity context c, leading to poor diversity of generated face images and degenerated face recognition performance. To address this issue, we propose two-stage sampling strategy during inference to unleash the model's inherent capability to enhance intra-class diversity. According to the observa- tions from Section 3.2] we instruct the model to perform sampling conditioned on the empty identity ce in the early stage, which helps to introduce greater intra-class variations. In the later stage, we condition the generation process with specific identity context to generate ddentity-relevann details. This strategy leads to both diverse and
images. The overall formula is as follows,
2--1
voi
2
vI-ai
1-a, colze, ce))
if [to. 7], if [0, 4o).
(4)
1-a0 vI-ai
((w + (,, (co(zz,(,e)))
2t
Vai
where to serves as the boundary between the two stages during inference and w is the scale of classifier-free guidance (Ho Salimans, 2022). Next we further introduce an adaptive partitioning strategy to assign unique boundary to for each sample.
Adaptive partitioning. We observed that the -rtss-att maps between UNet features and iden- tity context exhibit significant variations during the early sampling timesteps when the model has not yet started focusing on details that determine the identity. And decrease of temporal difference in cross-attention maps signifies the focus of sampling process shifting towards dlentity-relevan details in images (See Appendix for further details). Thus we propose an adaptive partition strategy based on the temporal differences of cross-altention maps. Specifically, let he denote the normal- ized cross-attention map between the learnable empty context c. and the UNet features at timestep t and de denote the difference of adjacent cross- cots-ntttetionn maps. The boundary to for each sample is determined as follows,
(5) where th is hyperparameter. We interpolate the cross-attention maps of each layer to the same resolution, then average and normalize them across the spatial dimensions to get the final h, The attention maps mentioned later in this paper are obtained in similar manner.
= ||he+1 hell2. to min (de+1 > th) and (de < [hh]
3.4 ATTENTION INJECTION
To further enhance the quality of synthesized images by leveraging the inherent diversity of the empty context Ce. we propose an attention injection module in the second sampling stage to directly use the attention maps of c. to guide the conditional generation. Let h and hS represent the self-attention map of UNet features in conditional sampling and cross-attention map between UNet features and identity context c respectively. Similarly, let heery and hCrees denote the attention maps in unconditional generation. naive approach is to directly copy the corresponding attention maps. Nevertheless, experimental results indicate that directly substituting the cross- attss-attention map has a considerable impact on identity and quality of the generated images (Section (.3). Thus, we introduce a novel injection strategy to leverage the cross-attention map herass from c.. Specifically, we normalize Me using the mean and variance of ho along the spatial dimensions. The formula is as follows.
Pe, Fe = mean(hSross), std str((hooos); He Te, mean(h ste(h
(6)
Maross Pe x Joc + Pee
(7)

---

Published as a conference paper at ICLR 2025
Table I: Comparisons with state-of-the-art synthetic-based face recognition methods. Face
verification accuracy (%) on difference benchmarks.
Method
of imgs (IDs x imgs/lD)
LFW
CFP-FP
CPLFW
AGEDB
CALFW
Average
CASIA-Real
-0.5M( 10.5K 47)
99.43
97.27
90.18
94.78
93.58
95.05
SynFace
0.5M(10k x 50)
91.93
75.03
70.43
61.63
74.73
74,75
DigiFace
0.5M(10k x 50)
95.4
78.87
76.97
78.62
83.45
DCFace
0.5M(10K x 50)
98.55
85.33
82.62
89.70
89.56
IDiff-Face
0.5M(10K x 50)
98.0
85,47
80.45
86.43
88,20
IDa
0.5M(10K x 50)
97.68
86.84
82.77
91.0
90.37
89.80
CemiFace
0.5M(10K x 50)
99.03
91.06
87.65
91.33
92.42
92.30
Are2Face
0.5M(10K x 50)
91.87
85.16
90.18
91.73
UlFace
0.5M(10k
99.27
94.29
89.58
90.95
92.25
93.27
DigiFace
1.2M(10K 72. 100k
96.17
89.81
82.23
81.10
82.55
86.37
DCFace
1.2M(20K x 50 40k
98.58
88.61
85.07
90.97
92.82
91.21
CemiFace
L.0((0k x
99.18
92.75
88.42
91.97
93.01
93.07
Arc2Face
1.2M(20k x 50 40k
98.92
94.58
86.45
92.45
93.33
93.14
UlFacee
LOM(20K 50)
99.22
95.03
90.42
92.45
93.18
94.06
UlFaace
L.5M(3K 50)
99.38
95.96
90.67
93.28
93,43
94.54
As for the self-attention map, we adopt direct replacement as Weelf Seet/ because we found that the self-attention of UNet features affects identity-irrelevan attributes of the synthesized images. Experimental results demonstrate that the proposed attention injection module, which handles self and cross-attention differently, achieves identity preservation while further harnessing the inherent ability of diffusion model enhance the diversity of the synthesized images.
3.5 SYNTHETIC-BASER FACE RECOGNITION
To generate synthetic dataset with non-existent identities, first we follow the previous method (Boutros et al.) 2023) to generate non-existent face images using an additional unconditional face generation model and extract their identity contexts using the pre-trained FR model (Boutros et al. 2022ab. Then we adopt the strategy from Kim al. (2023) to filter out similar identity contexts using cosine distances to ensure inter-class discrepancy. Subsequently, we utilize these identity contexts conditions to generate face dataset using our method. Last, train new FR models on these synthetic data from scratch and report the final FR performance on real datasets.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Training and testing dataset. We train our diffusion model on CAISA-Webface Yi et al. 2014) dataset containing about 500k face images of 10575 real identities celebrities directly grabbed from the web, which show large variations of attributes such as pose, illumination, and facial ex- pressions. We evaluate the final byttheti-bbased FR models the five most commonly used real datasets, LFW (Huang et al., 2008), CFP-FP (Sengupta et al.] 2016). 6). CPLFW (Zheng & Deng. 2018 AgeDB (Moschoglou al.] (017) and CALFW (Zheng al.) 2017). These datasets include images with varying ages, poses, and facial expressions, which allow to comprehensively measure the generalization FR models.
Implementation details. Our diffusion model is trained using Adam (Kingma 2014) optimizer with learning rate of le-4 and batch size of 64. The training is conducted for a total 250k itera- tions. During training process, we randomly replaced with the empty context c, at probability of 20%. As for training synthetic-basee based FR models, We adopt an IR50 as backbone with ArcFace loss Deng al. 2019) for 40 epochs. The scale of classifier-free guidance is set to and th is 0.005.
4.2 COMPARISON WITH STATE-OF-THE- METHODS
In this subsection, compare method with other synthetic-based face recognition methods. The results are shown in Table [ As shown in the table, our method significantly outperforms pre- vious approaches, including the latest methods (Papantoniou al. 2024 (un al. 2024 Li et al.

---

Published as a conference paper at ICLR 2025
Table Ablation studies. Face verification accuracy (%) on difference benchmarks and diversity
metrics of synthetic datasets from diffent settings.
Method
LPIPS
ImRecall
LFW
CFP-FP
CPLFF
AGEDE
CALFW
Average
baseline
0.5270
53.99
98.98
92.57
87.0
88.42
90.77
91.53
buseline 2-stage-fixed
0,5302
57.92
98.83
92.94
88.37
89.1
91.15
92.08
baseline + 2-stage-adapiive
0.5346
62.35
99.05
94.13
88.93
89.92
91.48
92.70
baseline alln
99.17
92.73
baseline 2-stuge-fixed
0.5367
63.77
99.15
93.84
88.48
90.3
91.78
92.71
baseline + D-stage-adapivee + attn
0.5592
71.96
99.27
94.29
89.58
90.95
92.25
93.27
2024a) in the average metrics across the five datasets, Moreover, our method maintains superior per- formance when synthesizing fewer than half the number of face images compared to previous methods. Notably, previous state-of-the-art method Arc2Face (Papantoniou et al., (0224) even uses more datasets and higher resolution images to train the generative model. We attribute the superi- ority of our method to our two-stage sampling strategy and attention injection mechanism, which unleash the potential of diffusion model to synthesize images with enhanced diversity, thereby enabling the FR model to have better generalization capabilities on real datasets. By increasing the number of identities, approach even outperforms the FR model trained CAISA- -ebbface the CPLFW dataset and achieves comparable performance in the average accuracy.
4.3 ABLATION STUDIES
In this subsection we further investigate the effectiveness of each design of proposed UlFace in synthetic face recognition. The results are shown in Table 2
Impact of two-stage sampling. We first build baseline method based on the vanilla single-stage sampling strategy (baseline) for synthetic-bard based face recognition. To validate the effectiveness of proposed two-stage sampling, we first employ naive two-stage sampling process based on fixed stage partition (4n 500, 2-stage-fixed). As shown in Table the naive two-stage design with fixed partition has already demonstrated a significant performance improvement compared to the baseline in both the diversity of the generated dataset and the accuracy of the FR model. We attribute this to our observations Section so that such two-stage strategy can unleash the model's 's potential for sirra-llas-diivesiffed image generation, leading improved face recognition accuracy.
Impact adaptive partitioning strategy. Moreover, we validate the effectiveness of the proposed adaptive partition by incorporating into the baseline method (2- watte--daaptiie As shown in Table 2, the proposed adaptive partitioning significantly outperforms the fixed partitioning method. Such adaptive partition strategy only eliminates the need of manual hyperparameter tuning but also allows for the allocation of different stage boundaries for each sample, especially considering that the partition preferences may vary across different samples.
Figure 4: visualization results of different attention injection strategies (Top: unconditional genera-
tion; Middle: attention injection with vanilla replacement; Bottom: the proposed attention inject).
8

---

Published as a conference paper at ICLR 2025
Baseline
Ours
CASIA
9146
0.23
0.15
0.33
0.15
0.52
Figure 5: Genuine and imposter comparisons. (Lefft: baseline; Mid: UlFace; Right: CASIA).
Table UlFace with different diffusion sampling methods. Our method is agnostic to any spe- cific diffusion sampling method, and achieves performance improvements across different methods.
Method
Sampling
LFW
CFP-FP
CPLFW
AGEDB
CALFW
Average
baseline
DDIM
98.98
92.57
87.00
88.42
90.70
91.53
UlFace
DDIM
99.27
94.29
89.58
90.95
92.25
93.27
baseline
FPNDM
99.18
93.03
87.70
88.92
91.18
92.00
UlFace
FPNDM
99.20
94.66
89.75
91.5
92.86
93.59
Impact of the proposed attention injection. We first visualize the synthesized images using the proposed attention injection compared images generated using vanilla attention map replacement. As shown in Figure 4 directly replacing the attention map will greatly degenerate the quality and identity the synthesized images, while the proposed attention injection can effectively utilize the well as maintain
diversity from unconditional generation the baseline model with the proposed attention injection method (attn). A significant improvement be observed in Figure 2 in both the diversity of synthetic datasets and the accuracy of the FR model. This is because we enhance the intra-class variations with the model's inherent ability in unconditional diverse image generation. Combining our adaptive two-stage strategy and attention injection leads to the best diversity and performance improvement.
Then augment
Figure 6: Visualization results of IDiff-Face (odd rows) and UlFace (even rows).

---

Published as a conference paper at ICLR 2025
Discussion about identity consistency. Intuitively, the two-stage sampling strategy proposed in this paper may reduce the intra-class identity similarity of the generated images since the empty context ce stands for "random identities", which is also of great significance in synthetic face recognition. In fact, there is a trade-off between intra-class diversity and intra-class identity consistency in this task. And our designs, including the discussion in Section [.2] the use of classifier-free guidance in the second stage (Section 3.3) and the different treatment of self and cross-atees cross-attention maps in Section 3.4 have sought maximize intra-class diversity while preserving identity consistency much as possible. To demonstrate this, we conduct genuine and imposter comparisons, a common metric in face recognition that measures the similarity between data points of the same individual and different individuals. As shown in Figure 5 and Table 2 our method has improved both intra-class identity similarity and style diversity compared to the baseline method.
4.4 UIFACE WITH DIFFERENT DIFFUSION SAMPLING METHODS
Since motivation lies in different sampling preferences in different stages during inference, which is agnostic to any specific diffusion sampling method, our method can be applied to various diffusion-based generation methods. Here choose FPNDM (Liu et al. 2022) As demonstrated in Tablq3] UlFace achieves consistent improvements on both DDIM and FPNDM sampling methods. showing the effectiveness and generalization our method. The bascline the same as Section and the numbers of synthetic images are all 0.5M.
representative.
4.5 QUALITATIVE RESULTS
The qualitative results of IDiff-Face (Boutros et al. 2023) and proposed UlFace are shown in Figure 6 We randomly sample identity contexts and used both methods to generate synthetic face images. evident from the figure that the images generated by IDiff-Face exhibit lack of variations in attributes such expression, illumination and pose, which is caused by context overfitting as discussed in Section [ In contrast, our proposed UlFace employs a two-stage sampling strategy unleash the model s inherent capability, thereby enhancing intra-class diversity in the generated images (more expression changes, facial rotations and lighting variations).
5 CONCLUSION
We introduce UIFace, novel synthetic face recognition framework that leverages two-stage sam- pling strategy to unleash inherent model capability enhance intra-class diversity synthetic face dataset. Moreover, we propose an adaptive partitioning strategy and an attention injection method to further improve intra-class diversity while maintaining cennity-preserving.. Extensive experi- ments demonstrate that UlFace outperforms existing methods in multiple benchmarks even using less training data and fewer identity number.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, and Gang Hua. Towards open-set identity preserving face synthesis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6713-6722, 2018.
Fadi Boutros, Naser Damer, Florian Kirchbuchner, and Arjan Kuijper. Elasticface: Elastic margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR) Workshops, pp. 1578-1587, June 2022a
Fadi Boutros, Patrick Siebke, Marcel Klemt, Naser Damer, Florian Kirchbuchner, and Arjan Kuijper. Pocketnet: Extreme lightweight face recognition network using neural architecture search and multistep knowledge distillation. IEEE access, 10:46823 46833,,,633 2022b.
Fadi Boutros, Jonas Henry Grebe, Arjan Kuijper, and Naser Damer. Idiff-face: Synthetic-based face recognition through fizzy ceentity-conditione diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19650-19661, 2023.
Andrew Brock. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv: 2018.
Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and Andrew Zisserman. Vggface2: A dataset for recognising faces across pose and age. In 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018), pp. 67-74. IEBE, 2018
Yunjey Choi, Minje Choi, Munyoung Kim, Jung- Woo Ha, Sunghun Kim, and Jaegul Choo. Star- gan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8789-8797, 2018.
Ivan LeAndres-Tamme Ruben Tolosana, Pietro Melzi, Ruben Vera-Rodriguez, Minchul Kim, Chris- tian Rathgeb, Xiaoming Liu, Aythami Morales, Julian Fierrez, Javier Ortega-Garcia, al. Fresyn challenge at cvpr 2024: Face recognition challenge in the era of synthetic data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3173-3183, 2024.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4690-4699 2019.
Yu Deng, Jiaolong Yang, Dong Chen, Fang Wen, and Xin Tong. Disentangled and controllable face image generation via 3d imitative-contrastvy learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2020.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-877 8794, 2021.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(1 13(11):139-144 2020.
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb- Im: A dataset and benchmark for large-scale face recognition. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 1 11-14, 2016, Proceedings, Part III 14, pp. 87- 102. Springer, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Zecheng He, Bo Sun, Felix Juefei-Xu, Haoyu Ma, Ankit Ramchandani, Vincent Cheung, Siddharth Shah, Anmol Kalia, Harihar Subramanyam, Alireza Zareian, et al. Imagine yourself: Tuning-free personalized image generation. arXiv preprint arXiv:2409. 13346, 2024.

---

Published as a conference paper at ICLR 2025
Jonathan Ho and Tim Salimans. arXiv:2207.12553 2022.
Classifier-fre
diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-655 18510 2020.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 4700-4708, 2017.
Gary B Huang. Marwan Mattar, Tamara Berg, and Eric Learned- Labeled faces in the wild: A database forstudying face recognition unconstrained environments. In Workshop on faces in Real-Life Images: detection, alignment, and recognition, 2008
Yuge Huang, Yuhan Wang, Ying Tai, Xiaoming Liu, Pengcheng Shen, Shaoxin Li, Jilin Li, and Feiyue Huang. Curricularface: adaptive curriculum learning loss for deep face recognition. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5901- 5910, 2020.
Tero Karras. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint 17I0v 10196, 2017.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401-4410, 2019.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyz- ing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10-8119, 2020.
Ira Kemelmacher--hlizzenn
Steven M Seitz, Daniel Miller, and Evan Brossard. The megaface benchmark: 1 million faces for recognition at scale. In Proceedings of the IEEE conference computer vision and pattern recognition, pp.
2016.
Minchul Kim, Anil K. Jain, and Xiaoming Liu. Adaface: Quality adaptive margin for face recogni- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18750-18759, June 2022.
Minchul Kim, Feng Liu, Anil Jain, and Xiaoming Liu. Deface: Synthetic face generation with dual condition diffusion model. In Proceedings of the ieeelcvf conference on computer vision and pattern recognition, pp. 12715-12725, 2023.
Diederik Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Tuomas Kynkäääniemi,, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in neural information processing systems, 32, 2019.
Shen Li, Jianqing Xu, Jiaying Wu, Miao Xiong, Ailin Deng, Jiazhen Ji, Yuge Huang, Wenjie Feng, Shouhong Ding, and Bryan Hooi. Id3: synthetic face recognition. arXiv preprint arXiv:2409. 01044499.77777 17576, 2024a.
wwweservvinn--e--iirrisiiiii diffusion models for
Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming-Ming Cheng, and Ying Shan. Pho- tomaker: Customizing realistic human photos via stacked id embedding. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8640-8650, 2024b.
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. arXiv preprint arXiv:2202.0977 2022.
Ziwei Liu. Zhonggi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu. Large- scale long-tailed recognition in an open world. In Proceedings of the IEEE/CVF conference on computer vision and patlern recognition, pp. 2537-2546, 2019.
12

---

Published as a conference paper at ICLR 2025
Stylianos Moschoglou, Athanasios Papaioannou, Christos Sagonas, Jiankang Deng, Irene Kotsia, and Stefanos Zafeiriou. Agedb: the first manually collected, mn-the-wild age database. In proceed- ings of the IEEE conference computer vision and pattern recognition workshops, pp. 51-59, 2017.
Foivos Paraperas Papantoniou, Alexandros Lattas, Stylianos Moschoglou, Jiankang Deng, Bernhard Kainz, and Stefanos Zafeiriou. Arc2face: A foundation model of human faces. arXiv preprini arXiv:2403.11641, 2024.
Haibo Qiu, Baosheng Yu, Dihong Gong, Zhifeng Li, Wei Liu, and Dacheng Tao. Synface: Face recognition with synthetic data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10880-10890) 2021.
Protection Regulation. Regulation (eu) 2016/679 of the european parliament and of the council. Regulation (eu), 679:2016, 2016.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger. Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and computer-assisted intervention- MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings, part III 18. pp. 234 241. Springer, 2015.
Soumyadip Sengupta, Jun-Cheng Chen, Carlos Castillo, Vishal M Patel, Rama Chellappa, and David W Jacobs. Frontal to profile face verification in the wild. In 2016 IEEE winter confer- ence on applications of computer vision (WACV). pp. 1-9, IEEE, 2016
Yujun Shen, Ping Luo, Junjie Yan, Xiaogang Wang, and Xiaoou Tang. Faceid-gan: Learning symmetry three- player gan for preeeeving face synthesis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 821-830, 2018
Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. 2020a.
Yang Song. Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13445 2020b.
Zhonglin Sun, Siyang Song, Ioannis Patras, and Georgios Tzimiropoulos. Cemiface: Center-based semi-hard synthetic face generation for face recognition. arXiv preprint arXiv:2409. 2024.
Fei Wang, Liren Chen, Cheng Li, Shiyao Huang, Yanjie Chen, Chen Qian, and Chen Change Loy. The devil of face recognition is in the noise. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 765-780, 2018a.
Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5265-5274, 2018b.
Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and Anthony Chen. Instantid: Zero-shot identity- preserving generation in seconds. arxiv preprint arXiv:2401.05519 2024.
Xiaobo Wang. Shuo Wang, Jun Wang, Hailin Shi, and Tao Mei. Co-mining: Deep face recognition with noisy labels. In Proceedings of the IEEE/CVF international conference computer vision, pp. 9358-9367, 2019.
Dong Yi, Zhen Lei, Shengeai Liao, and Stan Z Li. Learning face representation from scratch. arXiv preprint ar arXiv:1411. 141.7999, 2014.
Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, and Manmohan Chandraker. Towards large-pose face frontalization in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3990-3999, 2017.
13

---

Published as a conference paper at ICLR 2025
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
Shilong Zhang, Lianghua Huang, Xi Chen, Yifei Zhang, Zhi-Fan Wu, Yutong Feng, Wei Wang, Yujun Shen, Yu Liu, and Ping Luo. Flashface: Human image personalization with high-fidelity identity preservation. arXiv preprint arXiv:2403.17000 2024.
Tianyue Zheng and Weihong Deng. Cross-pose Ifw: database for studying cross-pose face recog- nition in unconstrained environments. Beijing University of Posts and Telecommunications, Tech. Rep, 5(7):5, 2018.
Tianyue Zheng, Weihong Deng, and Jiani Hu. Cross-age Ifw: A database for studying cross-age face recognition in unconstrained environments. arXiv preprint arXiv:1708.008: 2017.
Zheng Zhu, Guan Huang, Jiankang Deng, Yun Ye, Junjie Huang, Xinze Chen, Jiagang Zhu, Tian Yang, Jiwen Lu, Dalong Du, et al. Webface260m: A benchmark unveiling the power of million- scale deep face recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10492-10500 2021.
14


Reviews:
[{"review_id": "rd6wFMnqLP", "reviewer": "Reviewer_TRrB", "summary": "This paper addresses the challenge of improving synthetic data generation for face recognition (FR) by tackling the issue of context overfitting, which leads to low intra-class diversity and diminished performance. The authors introduce a novel framework named UIFace, designed to enhance intra-class diversity while maintaining identity preservation. The approach begins by training a diffusion model capable of denoising under identity-conditioned contexts or a learnable empty context. The identity context generates images that maintain identity but lack variation, while the empty context leverages the model's intrinsic ability to produce diverse images, albeit with random identities.", "questions": "The authors are encouraged to include a theoretical analysis explaining how LDM learns different subjects at different training time steps.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. UIFace seems to be closer to the dataset diversity of real datasets than the previous method IDiff-face.\n2. better results can be achieved using less amount of synthetic data, and with increasing amount of data even approaching the performance of direct datasets", "weaknesses": "1. The main contribution of the paper lies in the observation that LDM learns different aspects during both pre-training and post-training. However, only the analysis based on Figure 2 is currently insufficient, and there is a lack of theoretical discussion. \n\n2. The training and testing times for the two-stage LDM should be provided for comparison with the one-stage methods.\n\n3. The font size of the axis labels in Figure 2 is too small, and it appears that the author has not used \\citep and \\citet correctly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uu0Qq9RFeS", "reviewer": "Reviewer_JbLs", "summary": "This paper tries to solve the problem of intra-class diversity with the generation of synthetic images in face recognition, and proposes a two-stage pipeline framework of DDPM with two improvements, including adaptive stage partition strategy and attention injection module. Solid results are achieved.", "questions": "1. In this abstract section, can authors explain how the methods raised in this paper preserve the privacy and legality. If this is not related to the research purpose, it is better to exclude it in case of misleading readers.  \n2. There were more methods proposed upon IDiff and had better performance, for example, “SDFR: Synthetic Data for Face Recognition Competition”.\n3. Line 308, ... by leveraging... Carefully check the grammar and typo errors.\n4. It is not very clear about timestep interval T analysis in Figure. The interval upperbound of the x-axis is incrementally increased or resets as well the interval lowerbound.\n5. For experimentation, different settings of fixed t_{0} should be examined, also \"baseline + 2-stage-fixed + attn\" should be conducted.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality: The method was proposed upon the previous DDPM with two tricks raised by the observation that early stage and later stage of denoising step can capture different semantic meanings of images.  \nQuality: The methodology is conceptually simple with massive experiments supported. The solid results have achieved on public benchmarks. \nClarify: The idea, albeit simple, has been clearly presented. It is better to re-structure introduce and related work slightly by minimizing the depiction of FR while maximizing the work related to data synthesis by generation methods.  \nSignificance: The authors declared that the proposed methods achieved sota performance of face recognition by training models with synthetic images. There is no future work being declared to improve the work.", "weaknesses": "1. The method is proposed upon the previous DDPM work by using identity context maps as conditions. Two improvements, including adaptive partitioning strategy and attention injection module, have been added to enhance the performance of the methods. \n2. The authors declared they proposed a framework of face image synthesis by generating methods. Can authors clarify if the framework is suitable for other image generation tech, besides DDPM.\n3. The paper is well-structured. However, there are certain grammar and typo errors.\n4.  For introduction and related work sections, the description of background can go more specific to synthetic data-based face recognition rather than face recognition. \n5. More recent face image synthetic methods should be investigated. \n6. The authors have implied that synthetic images can solve privacy and legal issues. Please clarify how your methods can avoid privacy and legal issues, since synthetic images are generated on real-world images. \n7. Extra experiments should be conducted to support the methods raised by this work. For example, different settings of fixed t_{0} should be examined, also \"baseline + 2-stage-fixed + attn\" should be conducted in the ablation experiment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uWvvecWV78", "reviewer": "Reviewer_zDSW", "summary": "The paper addresses the challenge of generating diverse synthetic face images for the purpose of training face recognition models. The authors introduce a novel framework named UIFace, which employs a two-stage denoising approach alongside an attention injection module. This combination is designed to increase intra-class diversity while preserving identity consistency. As a result, UIFace achieves performance that is competitive with models trained on real-world datasets.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The two-stage denoising strategy and the attention injection module work together to generate diverse images while maintaining identity preservation, effectively addressing the issue of context overfitting.\n2. The adaptive partitioning strategy dynamically identifies the optimal moment to transition between the two denoising stages, thereby enhancing both the quality and diversity of the generated images.\n3. UIFace demonstrates its effectiveness by achieving performance on par with models trained on real datasets.", "weaknesses": "1. What about the scalability of the model? Are there any potential benefits to further increasing the number of identities beyond the 20K shown in Table 1?\n2. While this paper focuses on enhancing the intra-class diversity for a specific identity, what about the inter-class discrepancies, which are also critical for training an effective face recognition model? How do the authors ensure this aspect is adequately addressed in their work?\n3. The details of the overall training pipeline could be more clearly explained. For instance, in Stage 2, how are the losses from the unconditional and conditional branches calculated and combined?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)