Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes to do one-shot domain adaption by fine tuning a state of the art network StyleGAN. The paper introduces a few techniques to do so. First, the paper proposes to add a lightweight attribute adaption layer in the beginning of StyleGAN to modify the latent code. An existing latent code can be modified to give a new latent code that is in the new domain. Second, the paper proposes a novel type of discriminator reuse. A discriminator has its features frozen, but a new classification head is added. The paper shows various visual results.
main_review: The paper is generally well written and understandable. The paper conveys the main ideas of the work to the reader and I believe I got a good understanding from what the authors did by reading the paper. It seems very doable to reproduce the results on the paper alone.

The two technical main ideas are simple, but somewhat novel and can be considered a contribution to the field. Most, I like the idea of reusing a pre-trained discriminator. I am not aware of similar work and this seems to be a creative idea.

On the downside, I do not like the idea of the adaptor layer. The adaptor does not seem powerful enough to match the complexity of the task. There are multiple concerns here. The layer is too early in the network and seems to manipulate z-space rather than w-space (or some of the other available latent spaces later in the network). The layer is not powerful enough to do a more complex computation. Also, naturally, domain transfer generally means that the generator does not have the capabilities to generate images in the new domain. What is this network really doing by only manipulating a z-vector? If the z-vector is not split, it is hard to imagine the generator has the range to produce the desired results. If the z-vector is split, this could be a bit better, but also carries more risks. As it is, we can only observe a limited form of domain transfer.

One concern regarding the results is Figure 4. Visually all faces look really nice. But the task is not that clear and it's really difficult to see a relationship between input and output. This is not a result that should be considered domain adaptation. I would strongly suggest that all results transferring faces to other faces (but both input and output are in the same domain) should be removed from the paper. While this uses the same software system, it is essentially a different task that has been addressed better by many other papers. For example, attribute-based GAN editing papers can do that as well and there are multiple of them. I am pretty sure the authors do not want to go down that route.
The challenging domain transfer results, e.g. Figure 5, are indeed what I was hoping to see more of. Unfortunately, these results do not look particularly good. We can mainly observe a slight adaptation of all results shifting to a particular color scheme. It would also suggest to synchronize the examples. I think this is would be more insightful if we can see the same inputs transferred to different domains. I also feel the results are hard to interpret in multiple figures (e.g. 3), because I do not understand where the corresponding images in the original domain are. Maybe the input changes a lot for the challenging examples. I am afraid that this is a considerable omission, because the images in the input domain are such an essential component of the results. How can anybody judge these results without seeing the input images? In addition, there should be corresponding results shown. E.g., the last two rows in Figure 5 should correspond to each other. I do like the result in Figure 2. In a future version of the paper it would be good to see more of these interesting results.

The evaluation of this type of work is primarily visual. The metrics are not especially revealing if the results are good. It's good to have them, but I feel the authors still need quite a bit of work to make the system function as desired. My recommendation would be to revisit the adaptor design and find something better.

summary_of_the_review: I am very positive about the potential of this work, but it needs a substantial improvement.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper studies the challenging problem of transferring a pre-trained GAN from the source domain to a target domain with only one example available. To achieve the source-to-target domain adaptation while being able to synthesize diverse samples, this paper proposed a novel method called GenDA. The key idea is to freeze the generator and discriminator backbones while fine-tuning the linear layer added on top. Convincing experiment results have been demonstrated on the standard face and building benchmarks.
main_review: Strengths:
* This paper tackles a challenging domain adaptation problem which is very interesting.
* This paper demonstrates convincing qualitative comparisons (e.g., realism and diversity) to the existing efforts including Mo et al., 2020 and Ojha et al. 2021.
* This paper also shows comparable and improved quantitative results to the state-of-the-art methods in the case of a one-shot and ten-shot adaptation, respectively.

Weaknesses:
* This paper discovered an interesting phenomenon that a lightweight attribute adaptor could generate surprisingly-good domain adaptation results. Although it has been explained in the paper, the reviewer would appreciate it if some sort of ablation study (both qualitative and quantitative results) can be included in the paper (at least in the supplementary material of the final version). For example, what happens if we use different architecture or operators for the attribute adaptor.  The same rule applies to the attribute classifier (though it might be overfitting).
* At a higher level, this paper reminds me of the Exemplar SVM published 10 years ago. The conclusion in the paper is that it is often sufficient to obtain a good decision boundary if you have one positive example and many negative examples. This is very similar to the attribute classifier proposed in the paper. The reviewer would suggest incorporating such discussions in the final version of the paper.
* The limitation of the paper is not clearly stated. For example, the reviewer would like to know if the one-shot learning method applies to the “car -> abandoned car” application studied in the Ojha et al., 2021. It seems to the reviewer that the proposed method could fail at “car -> truck” application when the inter-subclass variations are huge. Please comment on this in the rebuttal.
* Furthermore, the cross-domain experiment (see Figure 5) seems to be a special case of style transfer. In this sense, this paper should also include comparisons to the style-transfer methods.

References:
Ensemble of Exemplar-SVMs for Object Detection and Beyond, Malisiewicz et al., In ICCV 2011.
summary_of_the_review: Overall, this is a very interesting paper with convincing results. I believe the key idea from this paper can have a broader impact on the community (e.g., 3D and video synthesis) in the future. I recommend accepting this paper but would kindly ask the authors to incorporate the suggested discussions in the final version.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this work, the authors propose a framework for one-shot generation, which aims to tame a new GAN using a pre-trained GAN model as well as one target image. Here, the authors present three strategies to ensure the quality and diversity of generated images: introducing an attribute adaptor to map the latent space to the target attribute; presenting an attribute classifier in the discriminator, and applying the truncation trick for the training. Experiments are conducted on the face and building datasets. 
main_review: Pros:
1. The one-shot image generation is a novel and interesting task.

2. The overall idea and the proposed pipeline towards addressing it are easy to follow.

3. In the experimental, the claim about one-shot attribute-related results is being met.  

Cons:

While the authors aim to address a challenging and novel task, I believe some parts need more clarification (even after considering the supplementary material):

1. The biggest weakness is that the proposed method has limited novelty. While the authors propose a stacked pipeline to address the quality and diversity, the key contribution they made is unclear. 

a. The z+/w/w+/s space analysis and adaption has been widely conducted in the latest works [r1, r2, r3]. What are the differences between the proposed adaptor and these prior works? Why the proposed adaptor would like to perform better?

b. Related to the above, the attribute classifier has been used in StyleFlow [r2]. Why the proposed one is better? In addition, if I understand correctly, the attribute classifier only judges the output is real or fake, instead of predicting attribute labels, because some examples in Figures 2 and 3 should not have corresponding labels. If this classifier just outputs real or fake labels, why not just fine-tuning the final layer of the original discriminator?

c. I cannot buy the novelty of reusing truncation trick for diversity-constraint strategy. As mentioned by the authors, this trick is a normal one in the current generation code. The authors did not provide a new direction to sell this strategy. 

2. The impressive results in Figures 4 and 5 are interesting, but the analysis is not clear enough. It would like to be better to visualize the learning representation after the adaptor as in AgileGAN [r3].

a.  The results in the final two rows of Figure 4 are quite interesting. I guess the learned w space has been pushed to a narrow space (the final one will be more narrow). If the authors can visualize the learning representation distribution, it will be clearer to demonstrate such mapping in the representation domain. 

b. A similar situation is in Figure 5.  It is interesting to use the one-shot pipeline for style translation. As the pre-trained G and D have not been updated, I guess the learned w space has been pushed into a different distribution to the original w space. How to visualize such change and then demonstrate the swap of learning representation would be significant. 

[r1] Wu, Z., Lischinski, D., & Shechtman, E. (2021). Stylespace analysis: Disentangled controls for stylegan image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12863-12872).
[r2] Abdal, R., Zhu, P., Mitra, N. J., & Wonka, P. (2021). Styleflow: Attribute-conditioned exploration of stylegan-generated images using conditional continuous normalizing flows. ACM Transactions on Graphics (TOG), 40(3), 1-21.
[r3] Song, G., Luo, L., Liu, J., Ma, W. C., Lai, C., Zheng, C., & Cham, T. J. (2021). AgileGAN: stylizing portraits by inversion-consistent transfer learning. ACM Transactions on Graphics (TOG), 40(4), 1-13.
summary_of_the_review: My preliminary rating for this submission is marginal reject. While I like this novel task that aims to generate diverse images with high quality, the key novelty and contribution are unclear explained to the prior related works. In addition, interesting experimental results are provided, but they are not well analyzed in the representation domain. The realistically visual results cannot demonstrate the contribution perfectly.
review_title: null

Meta-review: This work was the subject of significant back and forth (between authors and reviewers, but also between reviewers & myself) due to the wide range of opinions. Two of the reviewers have found this work below the bar: they have provided multiple reasonings that I would rather not repeat here. The third reviewer found this work more compelling and argued for its acceptance. My attempts at reaching a consensus have yielding the following conclusions:

  * There's agreement that one-shot generation is indeed a challenging task
  * Some of the results are indeed impressive, but many results are not compelling.
  * The rebuttal addressed some of the concerns (e.g. visualization of latents), but some issues are unaddressed (e.g. more motivation, explanation of why the proposed method works better)
  * One of the reviewers has argued rather forcefully that the work doesn't quite do domain adaptation in the typically understood sense. Moving beyond definitions of domain adaptation, the same reviewer was not very convinced by the quality of the results themselves.
  * The reviewer most positive about this work agrees that this work only explores a limited form of domain transfer. They argued that some of the potential applications of this work do make the submission interesting. 

Fundamentally, the discussion did not necessarily resolve the differences in opinion one way or another. Ultimately, all 3 reviewers believe that it would fine if this work was not accepted to ICLR at this time, despite some of the interesting results and promise. Given the discussion and this mildest consensus, I am inclined to recommend rejection too. I do think there's a substantial amount of constructive feedback in the reviews that would make a subsequent revision of this work quite a bit better.
Decision: Reject