Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper considers the one-shot domain adaption problem, it decoupled adaptation into two parts: style and entity transfer. Unlike most previous works that mainly focus on style transfer, the proposed method utilises binary entity mask with concise manifold regularized GAN design. The author modifies the architecture of the original generator to decouple the adaption of style and entity, and proposes the variant Laplacian regularization to smooth the network. Extensive experiments are conducted on various references with and without entities.

strengths_and_weaknesses: This paper focuses on an interesting one-shot domain adaption problem, the idea of disentangling style and entity transfer is straightforward.

The motivation of this paper is to solve one-shot domain adaption problem. However, I have some concerns as follows:
1. About the problem setting: the paper only explores the domain adaption with a clear binary entity
mask on the target image, which does not always exist for the general one-shot domain adaption setting. 

2. The disentangled GAN structure has been well studied by many existing works, it is hard for the reviewer to identify the real contributions of this paper.

3. The experiments are insufficient. The evaluation size is too small, the main result Tabel.1, only uses Fig. 3's images for quantitative evaluation.

4. Since the proposed method focuses on manifold regularization, some related Manifold GAN methods [1][2][3] are missing in comparison.
[1]MR-GAN: Manifold Regularized Generative Adversarial Networks
[2]MMGAN: Manifold-Matching Generative Adversarial Network
[3]Manifold-preserved GANs
questions: How to get binary entity masks? The author just mentioned it in the abstract and fig.1, should add descriptions in the experiment part.

Why did this paper chose sliced Wasserstein distance?  Need more insight and theoretical depth.

In L139 'each w will be transformed into the style-fixed code...', how to do transform? Need more details. 

limitations: The authors adequately addressed the limitations.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper performs one-shot domain adaption of StyleGAN model and the model additionally supports adding new entities compared to previous methods. The proposed method is able to do so thanks to a handful of points. Among them, two techniques are interesting: 1) using sliced Wasserstein distance to compute the internal distribution distance between two images; 2) adding variant Laplacian regularization to alleviate cross-domain correspondence distortion. 
strengths_and_weaknesses: Strengths:

The experimental results are promising. Using sliced Wasserstein distance and variant Laplacian regularization make sense. The authors provide code and additional materials so the method seems reproducible. The learning speed of this approach is faster than previous methods.


Weaknesses:

The writing of this paper can be improved. From the abstract and introduction, it is hard to identify the contributions of the proposed approach. Fig. 2 is hard to follow. It seems weird to have Fig. 1 below the title. 
questions: What is the reason to produce a mask using UNet since we can directly use the mask provided by users (if I understand correctly)? The underlying motivation is unclear. Plus, the authors should conduct an experiment to explain the benefit of doing it.

The paper missed comparison with other internal distribution distances like mse between gram matrices. To me, entity loss and style loss can be directly implemented with MSE between the gram matrices of deep features from two images. The authors are advised to try these loss functions to show the advantage of using sliced Wasserstein distance.

Why do you give the task of this paper a new name--generalized one-shot GAN adaption? From my understanding, the method additionally can add entities but that doesn't mean it is generalized.
limitations: The authors provide a section to talk about the limitations of this work. I agree with the authors about these limitations. I think one viable solution for the entity position problem is to inquire the users for an extra mask representing the desired location of the entity in the synthesized image.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a manifold regularized GAN adaption framework to deal with the generalized one-shot GAN adaption problem (i.e. the target domain to transfer contains both artistic styles and entities). To tackle this novel task, the paper modifies the architecture of the original generator by adding an additional auxiliary network to facilitate entity generation. To reduce the domain gap, they employ the sliced Wasserstein distance to minimize the divergence of the internal distributions between the exemplar and synthesis. Besides, they propose to use the variational Laplacian regularization $L_VlapR$ to mitigate the content distortion during training by preserving the geometric structure of the source manifold.

strengths_and_weaknesses: The paper is well written and easy to follow. Both quantitative and qualitative experiments show that the proposed method has sufficient visual advantages over the competition. In addition, their image processing results look interesting, which I believe will be helpful for artistic creation.

My only concern is the definition of style. In my opinion, the geometric feature is also part of the style and not just the color. I feel the results for Disney and Zelda are not so successful. Domain adaptation should also include some geometric changes, but the Zelda results in the paper don't look like CG images, and the Disney results cannot keep the big eyes and exaggerated expressions.

questions: See Weakness
limitations: Adequate
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents a method for one-shot domain adaptation, where a pre-trained GAN is leveraged to fine-tune the generator on one target image. First, given a source image, a latent vector is obtained by GAN inversion. Then, the generator is trained using a reconstruction loss, style and entity losses and a Laplacian regularizer, in order to transfer the style content of the target image into the source image while retaining the entity of the latter. Moreover, the proposed method admits a binary mask to select parts of the target image and to blend them into the source image, which is a novel contribution. Quantitative and qualitative results are presented when transferring from a few source images to a few target images from AAHQ dataset, as well as qualitative results for a few source and target images on AFHQ dog dataset and some source and target images on LSUN church dataset, showing a successful one-shot style transfer with optionally transferring objects from the target image. 

strengths_and_weaknesses: **Strengths**

The work is well motivated and the introduction of a binary mask to optionally import objects from the target image is a novel contribution. It targets a useful application and can be of use to the community and for content creators. 

The usage of quantitative metrics to evaluate the results is a good addition to the user studies. Moreover, the two metrics used are reasonable in the context of face to face translation. 

The methodology is sound and the claims are adjusted to the results presented in the experimental section. 

**Weaknesses**

The paper lacks clarity and misses important information (more details in "Questions"). The related work is not well redacted and fails to situate the proposed method within other methods in the literature. 

I would have expected the quantitative metrics to be computed and averaged across multiple samples for the target and source domains. Instead, those are only computed on hand-picked source-target images and risks providing only a biased positive view of the results. 
The experimental setup is lacking  to properly quantitatively evaluate the method with respect to the baselines. Instead of computing them on a handful of samples, they should be computed over a dataset (or a randomly chosen subset of data) for the source and target domains in order to draw more robust conclusions.

questions: As it was written above, I would like to see averaged quantitative metrics over many source/target samples.


A sentence in l.176-177 claims that slice Wasserstein distance “leads to the same destination but with greater efficiency”, compared to the GAN loss. There is no reference cited nor I have seen an ablation experiment in the paper. Could the authors elaborate on this with empirical evidence or otherwise, remove the sentence altogether?

In l.35 it is mentioned that other methods produce artifacts when the entities are big. Could you point out where can this be observed in the figures?

How were the hyper parameters chosen? No discussion is provided with respect to that.



*Details on clarity*
* The task definition in l.106 is not clear and also fails to mention one of the most important details: the generator is fine-tuned on only one image from the target domain. As it stands, it appears as if a collection of target images were used instead. 
* The methodology section focuses on one particular case, pretrained on FFHQ. Instead, the methodology should be general and only introducing the datasets and particular examples in the experimental section and Figure 2 as an illustration. 
* Figure 2 is not clear, misses notation to quickly cross-reference it with the text. Moreover, the figure is not self-contained as there is some notation that is not explained in the caption. Additionally, this figure is referenced in the text before anything about the model is explained, which generates confusion. 
* The explanation of the auxiliary net is lacking, specially to understand how the feature map $f_{ent}$ and mask $m$ are obtained. Is the mask predicted on the feature maps from the style-gan architecture? And if so, did you use a pre-trained U-Net or you trained one from scratch? How did you adapt a pre-trained U-Net in this case?
* Equation (1) misses $f_{ent}$ and $m$ as inputs to $aux$.
* l.187: “layers from pre-trained LPIPS” this is a experimental detail that does not belong in the methodology section.
* l.187: “upsampled by m” what does hat mean?
* Paragraph in lines l. 158-164 would be better suited in the experimental section.

* Where is the source of the images “Sketch, Disney and Arcane” that are mentioned in l.241? No reference is provided. 


limitations: The limitations were discussed and the broader impact briefly touched upon.
review_title: null

Meta-review: metareview: This paper focuses on the one-shot domain adaption of GAN model. The idea of disentangling style and entity transfer is simple and effective. The meta-reviewer recommends acceptance of the paper, and the authors are encouraged to take the reviews into consideration when preparing a final version of the paper.
confidence: Certain
Decision: Accept