Towards Diverse and Faithful One-shot Adaption of Generative Adversarial Networks
Abstract
One-shot generative domain adaption aims to transfer a pre-trained generator on one domain to a new domain using one reference image only.
However, it remains very challenging for the adapted generator (i) to generate diverse images inherited from the pre-trained generator while (ii) faithfully acquiring the domain-specific attributes and styles of the reference image.
In this paper, we present a novel one-shot generative domain adaption method, i.e ., DiFa, for diverse generation and faithful adaptation.
For global-level adaptation, we leverage the difference between the CLIP embedding of reference image and the mean embedding of source images to constrain the target generator.
For local-level adaptation, we introduce an attentive style loss which aligns each intermediate token of adapted image with its corresponding token of the reference image.
To facilitate diverse generation, selective cross-domain consistency is introduced to select and retain the domain-sharing attributes in the editing latent W + space to inherit the diversity of pre-trained generator.
Extensive experiments show that our method outperforms the state-of-the-arts both quantitatively and qualitatively, especially for the cases of large domain gaps.
Moreover, our DiFa can easily be extended to zero-shot generative domain adaption with appealing results.
Code is available at https: //github.com/1170300521/DiFa .
Introduction
Generative adversarial networks (GANs) [8] have achieved remarkable progress in generating photorealistic and highly-diverse images [12, 13, 19].
However, GANs usually require a large number of samples for stable training, and suffer from severe mode collapse when trained with insufficient data ( e.g ., one image).
Recently, several works [2,11,28,30,35,38,40] have been proposed to train a GAN from scratch with only one or few images but are limited in generating high quality and diverse images.
In this paper, we resort to one-shot or few-shot generative domain adaption (GDA), i.e ., transferring a pre-trained generator on one domain to a new domain using one or few reference images (as shown in Fig. 1).
Thus, GDA can provide a new perspective to address the above issues by inheriting the generation ability and diversity of the pre-trained generator.
Many methods [17,20,22,26,32,34,36,39] have been proposed for one-shot GDA.
Nonetheless, domain-specific attributes and styles usually can be described by language, and thus can be well depicted by Contrastive-Language-Image-Pretraining (CLIP) [23].
Hence, CLIP-based one-shot GDA methods [7,16,33,42] have been proposed to adapt a pre-trained generator, e.g ., StyleGAN2 [13], to the target domain.
In particular, the domain-gap direction between source and target domains is first calculated in the CLIP embedding space.
Then, the pre-trained generator is transferred by aligning the CLIP direction between the source and adapted images with the domain-gap direction.
However, it remains very challenging for the adapted generator (i) to generate images as diverse as the pre-trained generator while (ii) faithfully acquiring the domain-specific attributes and styles of the
[Figure]
Notably, our DiFa is effective in inheriting the diverse generation ability of GAN
, while faithfully acquiring the representative characteristics of the reference image in domain B . reference image.
Firstly, although the domain-gap direction extracts the pronounced characteristics of the reference image, the detailed local styles and attributes are usually ignored in CLIP embedding.
Without considering these local styles and attributes, the adapted generator cannot faithfully acquire the domain-specific characteristics of the reference image.
Secondly, the domain-gap direction is the difference between the CLIP embedding of the reference image and source domain, which contains both domain-specific attribute shifts ( e.g ., thick eyebrows in second row of Fig. 1) and domain-sharing attribute shifts ( e.g ., gender).
Directly aligning the training sample-shift direction with the domain-gap direction introduces the unnecessary domain-sharing attribute changes to the adapted images, thereby being harmful to inheriting the diversity from the pre-trained generator.
Although [16,42] proposed to use the style mixing and editing direction preservation to address these issues, only limited improvements are achieved.
In this work, we present a novel one-shot GDA method, i.e ., DiFa, for diverse generation and faithful adaption.
In terms of faithful adaption, we consider both attributes and styles.
For global-level adaptation, we define the domain-gap direction as the difference between the CLIP embedding of reference image and the mean embedding of source images.
As for local-level adaptation, we introduce an attentive style (AS) loss on the intermediate layer of the CLIP image encoder.
For each intermediate token of an adapted image, it first finds the nearest token of the reference image, and then minimizes their difference to make GDA adapt to the target style.
In terms of diverse generation, selective cross-domain consistency (SCC) is introduced to select and retain domain-sharing attributes in the editing latent W + space to inherit the diversity of pre-trained generator.
In particular, we use a styleGAN inversion models [25,31] to invert the images from source and target domains into the
W + space.
Then, we compute the direction ∆ w between the two domains, where smaller values in ∆ w indicate that the corresponding latent variables in W + space are domain-sharing attributes.
Selective cross-domain consistency encourages an adapted image and its corresponding source image to be similar in domain-sharing attributes, and can be different in other attributes.
SCC allows the adapted generator to inherit from the pre-trained generator selectively.
Thus our DiFa can guarantee the diversity of adapted images without the sacrifice of decreasing domain adaption ability.
Quantitative and qualitative experiments are conducted on a wide range of source and target domains.
Evaluation results highlight the superiority of our DiFa compared against the state-of-the-art methods, especially for the cases of large domain gaps ( e.g .,
Cat → Tiger).
To illustrate the editing capabilities of the adapted latent space, we employ InterFaceGAN [29] to edit the real images in target domain.
Overall, our contributions are summarized as follows:
• We introduce a novel method namely DiFa, along with selective cross-domain consistency and attentive style losses, for diverse generation and faithful adaption.
• Extensive experiments show the effectiveness of our DiFa in acquiring the representative domain characteristics from the reference image, and inheriting the ability of pre-trained generator to produce diverse images.
• Our DiFa can be easily extended to zero-shot generative domain adaption with appealing results.
[Figure]
Related Work
Few-shot Domain Adaption of GANs.
Few-shot generative domain adaption aims to transfer a generator pre-trained on a source domain to a new target domain with very limited reference images.
Earlier studies [17,20,22,26,32,34,36,39] utilized the adversarial loss [8] to capture domain-specific information from given reference images.
To reduce mode collapse, these methods usually adopted fewer learnable parameters [20,26,32,36] or introduce regularization terms [17,22,34,39], but still produce images with insufficient diversity.
With the success of CLIP [23], recent works [7,16,33,42] leveraged the difference between the CLIP embeddings of the source and target domains to guide the attribute-level adaption, beating the methods with adversarial loss [8].
To better capture domainspecific styles, several methods [16,42] adopted the style mixing trick during inference time, however, it may bring undesired semantic artifacts when there is a significant shape discrepancy.
[16,33,42] attempted to generate diverse images by preserving the editing distance of input pairs, before and after adaption.
Nonetheless, they indistinguishably retain both domain-sharing and domain-specific attributes, which is conflicted with faithful adaption.
GAN Inversion.
GAN inversion aims to invert an image into its corresponding latent codes, which can be grouped into optimization-based and encoder-based methods.
Optimization-based inversion [5, 41] directly updates the latent code by minimizing the reconstruction error.
Albeit high-quality and accurate reconstruction can be obtained, it usually costs a few minutes for an image.
In contrast, encoder-based algorithms [1,25,31] directly embed a given image into latent codes, so that the inference can be completed in real-time, and the gradients of input images could also be passed backward.
Moreover, encoder-based algorithms also achieve considerable performance when handling out-of-domain images, and thus it is feasible to project adapted images into W + codes during training.
Proposed Method
In this work, we focus on one-shot generative domain adaption task, which aims to transfer a generator G A pre-trained on domain A to a new domain B using one reference image I tar only.
Specifically, we present a novel method termed DiFa to generate diverse images inherited from the pre-trained generator while faithfully acquiring the domain-specific attributes and styles of the reference image.
The overview of our DiFa is illustrated in Fig.
In this section, we first introduce the global-level adaption loss with an estimated domain-gap direction.
The attentive style loss and selective cross-domain consistency loss are then proposed for local-level adaptation and diverse generation, respectively.
Finally, we introduce the overall learning objective for training.
3.1 Global-level Adaption
Recent studies [7,16,42] have demonstrated the superiority of CLIP in transferring a generator in one domain to a new domain under the one-shot setting.
In comparison to the methods based on adversarial loss [17,20,22,26,32,34,36,39], CLIP-based methods are effective in describing domain characteristics and resulting in photo-realistic images.
Given a generator G A pre-trained on domain A and a target reference image I tar from domain B , CLIP-based methods first calculate the domain-gap direction between domain B and A :
[Equation]
where v tar =
E I ( I tar ) denotes the embedding of target domain B and E I is the CLIP image encoder.
v src represents the CLIP embedding of source domain A .
To transfer G A to domain B , they copy a new generator G B from G A and finetune it by aligning the sample-shift direction ∆ v samp with the domain-gap direction ∆ v dom :
[Equation]
where v B = E I ( G B ( z )) and v A
= E I
( G A ( z )) denote the CLIP embeddings of domain B and domain A samples.
z ∼ N (0 , I ) denotes the input noise.
After finetuning with the global-level adaption loss L global , the adapted generator G B can generate high-quality images of domain B .
Note that the embedding of source domain v src can be calculated in different ways.
Two-stage methods [16,42] find the image corresponding to I tar in domain A and treat its CLIP-space embedding as v src .
Nonetheless, the corresponding image in domain A inevitably contains domain-specific attributes of I tar , leading to ∆ v dom ignoring these domain-specific attributes.
One-stage methods [7] utilize the mean embedding of source images as v src .
Intuitively, the mean embedding usually represents the common attributes of source domain, and does not affect domain-specific attribute shifts in ∆ v dom .
Thus, in our experiments, we use the mean embedding of source images as the source domain embedding, i.e ., v src = v ¯ A =
E z ∼N (0 ,I )
[ E I ( G A ( z ))] .
3.2 Local-level Adaption
Albeit the domain-gap direction ∆ v dom captures the global-level representative domain characteristics of reference image I tar , the local attributes and visual styles are usually ignored in CLIP embedding.
Therefore, training G B with L global only cannot faithfully capture the local-level domainspecific characteristics of I tar .
For example, images generated by StyleGAN-NADA [7] fail to acquire the mane and stripes of tigers during the Cat → Tiger adaption (the first row in Fig. 6(c)).
[16,42] tried to inherit the detailed visual styles from I tar through the style mixing.
However, when there is a significant shape discrepancy ( e.g ., pose or cross-category) between I tar and the original adapted image I B , the mismatch of content and style in I mixB will lead to visible artifacts (the last row in
Fig. 5(b) and the first row in Fig. 6(b)).
To mitigate the above issue, we further present an attentive style loss L local to help G B faithfully acquire the local-level representative attributes and styles of I tar .
Inspired by content-style alignment in style transfer [15], L local is designed to encourage each part of I B to attentively align with its corresponding styles from I tar .
Specifically, we first extract the intermediate tokens of I B and I tar from the k -th layer of CLIP image encoder (shown in Fig. 4), and then align each of adapted tokens F B with its closest target token from F tar , where F B = { F 1 B , . .
, F nB } and F tar = { F 1 tar , . . .
, F mtar } are the extracted tokens.
The final attentive style loss is defined as,
[Equation]
where C is the cost matrix to measure the token-wise distances from F B to F tar , and each element of C is computed as:
[Equation]
[Figure]
L a y e r _ ...
L a y e r _
L a y e r _ ...
Image Encoder (ViT) 𝐸 !
[Figure]
3.3 Selectively Diverse Generation
The ability to generate diverse target domain images is also critical for one-shot generative domain adaption.
Recall that the domain-gap direction is the difference between the embedding of I tar and source domain, which contains both domain-specific and domain-sharing attribute shifts.
Training
G B with L local also introduces the unnecessary domain-sharing attribute changes to the adapted images, which hinders G B from inheriting the diversity of the pre-trained generator G A .
To facilitate diverse generation, we propose a selective cross-domain consistency loss to select and retain the domain-sharing attributes in W + space.
Intuitively, if an attribute is similar between domains A and
B during adaption, it is more likely to be a domain-sharing attribute.
According to this assumption, we can dynamically analyze and preserve the domain-sharing attributes.
Specifically, we first invert
G A ( z ) and G B ( z ) into W + latent codes w A and w B with an pre-trained inversion model ( e.g ., pSp [25] or e4e [31]) for each iteration.
Then, as shown in Fig. 3, we compute the difference ∆ w between the centers of a queue of W + latent codes X A and a queue of W + latent codes X B , where
X A and X B are dynamically updated with w A and w B during training.
According to ∆ w , we encourage w A and w B to be consistent in channels with less difference,
[Equation]
where α represents the proportion of preserved attributes and mask( ∆ w , α ) determines which channels to be retained.
Let | ∆ w s αN | be the αN -th largest element of | ∆ w | , and each dimension of mask( ∆ w , α ) is calculated as:
[Equation]
Note that fine layers of StyleGAN [12, 13] usually control color information, and constraining them may have a detrimental effect on obtaining styles of I tar .
Hence, we only use latent codes corresponding to coarse spatial resolutions ( 4 2 –8 2 ) and middle resolutions ( 16 2 –32 2 ) in L scc .
3.4 Overall Training Loss
Our overall training loss consists of three terms, i.e ., the global-level adaption loss L global , the attentive style loss L local for acquiring detailed style information and the selective cross-domain consistency loss L scc for inheriting the diversity:
[Equation]
In our experiments, we use λ local = 2 and λ scc = max(0 , n iter − n B N iter − n iter ) , where N iter and n iter denote the total number of training iterations and the n iter -th iteration of training, respectively.
That is, λ scc increases linearly as the training proceeds.
Experiments
In this section, we first introduce the experimental settings of our DiFa, including implementation details, datasets, and metrics (Sec. 4.1).
Both qualitative and quantitative experiments are conducted
[Figure] [Figure]
on a wide range of domains to demonstrate the superiority of our DiFa in generating diverse images and faithful adaption (Sec. 4.2 and Sec. 4.3).
Besides ablation studies are considered to evaluate the effects of our proposed two losses (Sec. 4.4).
Finally we also investigate the editing ability of the adapted generator and extend our DiFa to zero-shot generative domain adaption (Sec. 4.5)
4.1 Experimental Settings
Implementation Details.
In our experiments, we use StyleGAN2 pre-trained on FFHQ [12] and
StyleGAN-ADA [11] pre-trained on AFHQ-Cat [4] and employ e4e [31] and pSp [25] as their inversion models, respectively.
Following StyleGAN-NADA [7], we utilize both ViT-B/16 and
ViT-B/32 [6] models for CLIP-base losses.
For training, we use ADAM optimizer [14] with a learning rate 0.02 and set the batch size to 2.
We finetune the generator for 300 ∼ 400 iterations, which takes about 3 ∼ 4 minutes on an RTX 2080Ti GPU.
[Table] [Table]
Datasets.
For FFHQ adaption, the target images are collected from three datasets: (i) ArtstationArtistic-face-HQ (AAHQ) [18], (ii) MetFaces [11], and (iii) face paintings by Amedeo Modigliani,
Fernand Leger and Raphael [37].
Each of them contains 10 images.
For Cat adaption, we collect target images from the AFHQ-Wild validation dataset and divide them into Tiger, Fox, and Wolf datasets, which include 103, 53, and 46 images, respectively.
In particular, Amedeo Modigliani,
Fernand Leger, Raphael, Tiger, Fox and Wolf are used in quantitative experiments.
Metrics.
Following StyleGAN-ADA [11], we use Fr ´ echet Inception Distance (FID) [9] and Kernel
Inception Distance (KID) [3] to evaluate our DiFa quantitatively.
Both metrics measure the quality and diversity of the images, while KID is more suitable for the few-shot setting (only a few images in validation sets).
In all our experiments, both FID and KID are calculated between 5,000 synthesized images and each validation sets.
4.2 Qualitative and Quantitative Evaluation
Qualitative Results.
Fig.
5 shows the qualitative comparisons adapted from FFHQ [12].
As shown in the figure, Few-Shot Adaption [22] suffers from severe model collapse and generates similar images.
Due to StyleGAN-NADA [7] is trained by aligning the sample-shift direction ∆ v samp with domain-gap direction ∆ v dom , which contains the domain-sharing attributes ( e.g ., gender) shift, it also cannot inherit the sufficient diversity from the pre-trained generator.
For example, the gender of adapted images is changed to female in 4 ∼ 6-th rows in Fig. 5(c).
Mind The Gap [42] retains the local styles of reference image via style mixing.
However, it produces undesired semantic artifacts when there is a significant shape discrepancy between domains, e.g ., redundant noses and eyes in 3rd and last row of Fig. 5(b).
In contrast, with the proposed SCC and AS losses, our DiFa not only faithfully acquires the representative domain-specific attributes and styles from the reference image, but also produces images with high diversity inherited from the pre-trained generator.
Additionally, we also illustrate the qualitative results adapted from AFHQ-Cat [4] in Fig.
Our DiFa also captures sufficient domain-specific characteristics from the reference image in comparison to the competing methods ( e.g ., the mane and stripes of the tiger in first row), further demonstrating the superiority of our method.
More visualizations adapted from other domains are shown in Suppl .
Quantitative Results.
We also quantitatively compare our DiFa with competing methods [7,22, 42] under six settings, i.e ., FFHQ → { Amedeo Modigliani, Fernand Leger, Raphael } and Cat → { Tiger, Fox, Wolf } .
For each setting, we randomly sample an image from a target dataset to perform adaption, and report both Kernel Inception Distance (KID) [3] and Fr´echet Inception Distance (FID) [9] metrics.
To reduce random sampling error, we repeat it five times and use the mean value as final score.
The results are listed in Table 1 and Table 2.
One can see that our DiFa clearly outperforms the competing methods, which are consistent with qualitative results in Fig.
5 and Fig.
We observe
[Table]
[Figure]
that FSA [22] obtains better FID scores in Amedeo and Fernand datasets, which is inconsistent with above qualitative results (see Fig. 5(d)).
Note that FID cannot reflect the overfitting problem very well when target dataset is extremely small and biased [11].
Specifically, these two small datasets have different data biases with FFHQ, e.g ., gender bias.
8/10 images in the Amedeo Modigliani dataset and 9/10 images in the Fernand Leger dataset are female.
Due to our DiFa acquiring the diversity from the original generator which is trained on FFHQ, it generates male and female adapted images with similar probability.
In contrast, for FSA, the adapted images are all similar to the reference image.
When comparing on the above two datasets, FSA tends to generate images that have similar gender distribution to the validation dataset, thus achieving better FID results.
For the Raphael dataset, which has 5/10 images that are female, our DiFa achieves better FID results.
4.3 User Study
We further perform user study to compare our DiFa with the competing methods.
Specifically, we provide users a reference image, a source image, and two adapted images from different methods, and ask them to choose the better adapted image for each of three measurements: (i) image quality, (ii) style similarity with the reference and (iii) attribute consistency with the source image.
We randomly generate 1,050 samples for each comparison (3,150 in total).
There are 30 users.
We assignsamples for each of them, and give them unlimited time to complete the evaluation.
From Table 3, the users strongly favor our DiFa in all three aspects, especially from the perspective of image quality and attribute consistency.
Note that FSA [22] suffers from severe mode collapse and simply copies from the reference, hence, it is favored on style similarity but performs worse on the other aspects.
4.4 Ablation Study
Ablation studies are conducted to evaluate the effects of two critical components of our DiFa, i.e ., the selective cross-domain consistency (SCC) loss and the attentive style (AS) loss.
As shown in Fig.
7, the images from the baseline have very limited diversity and lack some representative characteristics of the reference image, e.g ., darker hue.
Benefited from SCC, the adapted generator begins to retrain the domain-sharing attributes ( e.g ., hair length and gender), thereby inheriting the diverse generation ability from the pre-trained generator.
When further adding AS, we observe that the adapted generator faithfully captures the domain-specific styles and local-level representative attributes from the reference image, e.g ., darker hue and purple hair.
More ablation studies about hyper-parameters are provided in the Suppl .
[Figure] [Figure]
4.5 Extensions
Latent Space Editing.
In Fig.
8, we illustrate the editing results performed on a real image adapted into a new domain.
Concretely, we employ InterfaceGAN [29] to discover some editing directions in domain A , and then leverage these directions to edit the adapted real image.
As can be seen, the directions from domain A still manage to control real images in domain B , indicating that the adapted generator maintains a similar ability in latent-based editing with the original generator.
Zero-shot Domain Adaption of GANs.
With minor modifications ( e.g ., removing the AS loss), our
DiFa can be easily extended to zero-shot GDA, i.e ., adapting to a target domain described by text only.
Fig.
9 shows the comparison between our DiFa and StyleGAN-NADA [7].
One can see that adapted images from our DiFa are more consistent with their corresponding source images, thereby inheriting more diversity from the pre-trained generator.
For example, when performing adaption from FFHQ to a target domain described by “ A sketch with black pencils ”, all eyes in StyleGAN-NADA results look to the left, which is inconsistent with the original eyes in source domain.
More visualizations and the implementation details are given in the Suppl .
Discussion
In this paper, we presented DiFa to address the diverse generation and faithful adaptation issues for one-shot generative domain adaption.
In particular, DiFa leverages the difference between the CLIP embedding of the reference image and the embedding of source domain to guide the global-level adaption.
To faithfully acquire local-level domain-specific characteristics, we introduce the attentive style loss to align each intermediate token of adapted images with its closest token of the reference image.
For highly diverse generation, the selective cross-domain consistency loss is proposed to select and retain the domain-sharing attributes in W + space.
Both qualitative and quantitative experiments show the superiority of our DiFa against state-of-the-arts under a wide range of settings, especially for the cases of large domain gap.
Furthermore, our DiFa can be easily extended to zero-shot generative domain adaption with compelling results.
Limitations.
When there are few domain-sharing attributes between source and target domains, e.g ., Church → Tiger, our DiFa cannot produce highly diverse images.
Fortunately, this issue may be largely alleviated by adaptively inheriting the prior knowledge from large-scale generators [21,24,27], which are pre-trained on sufficient variety of source domains.
Broader Impact.
Transferring a pre-trained generator with very limited data plays a crucial role in academia and industry.
More specifically, our DiFa provides insights on tasks in computer vision, e.g ., data augmentation and few-shot adaption.
Meanwhile, our DiFa also makes AI more accessible to the public.
On the one hand, users could leverage our method to create the artworks with any desired styles, even without adequate computing and data resources.
On the other hand, our work may bring potential concerns on the probability of producing fake images.
For example, someone may use our
DiFa to spoof other people’s portraits, to synthesize deceptive interactions, or even to impersonate public figures to influence political processes.
Albeit there are a few potential negative impacts, we believe that they could be well addressed with the development of DeepFake detection and proper protocols.
In particular, we could verify the authenticity, integrality, and source of images by adding digital watermarks or signatures.
Also, we may employ DeepFake detection technique to analyze the images without digital signatures.
Furthermore, our community should help the government to improve corresponding laws and regulations to avoid the abuse of image generation.
Acknowledgments and Disclosure of Funding
This work was supported in part by the National Key R & D Program of China under Grant No. 2021ZD0112100, and the National Natural Science Foundation of China under grant No.
U19A2073.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect
the paper’s
contributions
and
scope? [Yes] The main contributions can be summarized as: (
We introduce a novel method namely DiFa, along with Selective Cross-domain Consistency
Loss (SCC) and Attentive Style Loss (AS), for diverse generation and faithful adaption.
Extensive experiments highlight the effectiveness of our DiFa in acquiring the representative characteristics from the reference image, and inheriting the capability to produce high-diversity images from the pre-trained generator.
(3) Our DiFa can be easily extended to zero-shot generative domain adaption with appealing results.
(b) Did you describe the limitations of your work?
[Yes] See Sec.(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] See Sec.(d)
Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] We carefully read the ethics review and comply with them.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See https://github.com/1170300521/DiFa .
(b) Did you specify all the training details (e.g., data splits, hyper-parameters, how they were chosen)?
[Yes] See Sec. 3.4, Sec.
4.1 and https://github.com/1170300521/ DiFa (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Table 1 and Table(d)
Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Sec.4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] We cite papers that we used in Sec.(b) Did you mention the license of the assets?
[Yes] We use AFHQ, MetFaces, and Artistic-Faces datasets during training, and their license are included in our repository https://github.com/1170300521/DiFa .
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] We provide the URL of our source code in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[Yes] The AFHQ, MetFaces, and Artistic-Faces datasets are widely used in the community.
To our best knowledge, there is no inappropriate information in them.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[Yes] The datasets we used in this paper are popular benchmarks in computer vision and do not include any harmful information.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[Yes] See Sec. 4.
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[Yes] Each user is paid 10 dollars per hour and we spend about 300 dollars in total.