Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The work introduces a flexible method to distill the knowledge of unconditional GANs of images to various image translation tasks, including image-to-image, text-to-image, and audio-to-image translation. While prior work requires using the same architecture of the pre-trained GANs to train the downstream im2im tasks, the proposed method supports distillation on a wide range of X-to-image translation architectures (e.g. starGANv2). The authors have compared their methods with prior work and showed that the proposed method obtained better FIDs.
main_review: (Update: after reading the reviews and the authors' reply, I would recommend the paper to be accepted to the conference. I'm raising the score from 6 to 8.)

Strengths:
1. The proposed method obtains better image quality against prior work in various settings, as suggested by lower FIDs.
2. Strong results on X2I translation tasks are shown, indicating that the distillation method is effective for X2I tasks.
3. The method improves image translation performance when training on fewer labeled samples.

Weaknesses:
1. For clarification purposes, after the second stage of training of I2I models, can the model explicitly control domains/classes in the latent-guided synthesis, just as how starGANv2 is set up?
2. The clarity of writing can be improved. For example, it takes me a while to understand the setup for the entire training procedure. It might be better to mention more clearly that the distilled model is used as a pre-trained model for the downstream task.
3. Would it be possible to extend the ablation study (Figure 4) to more datasets and settings? It would be more convincing to the reader if the trend is similar in various settings.

Additional questions:
1. During training, I am wondering if the style-mixed triplets are pre-computed and stored, or randomly sampled on the fly. If it is the former case, maybe one can check if increasing the number of stored style-mixed triplets can eliminate the need for knowledge distillation.
summary_of_the_review: This work introduces a nice idea that improves image translation methods by distilling the knowledge of unconditional GANs. The setup is more flexible compared to prior work, where different architectures for image translation can be used. While the clarity of the writing can be improved, I think it is good to accept this paper to the conference.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors present a unifield learning method to investigate the knowledge transfer for X2I translation. Compared to existing methods, there are two advantages here: (1) this framework can be used for varying kinds of conditional image synthesis tasks; (2) it relieves the limitation for student generator to be the same as the pre-trained GAN. Specifically, the authors leverage the style mixing characteristics of high-quality GANs and introduce the semantic diversity loss to achieve it. Many experiments are conducted with good qualitative and quantitative results.
main_review: I think this submission is interesting. It is well written and clearly explains the ideas. I really like the figures which are clean and helpful in understanding the ideas of the paper and performance of the method. I am impressed by the variety of experiments, which answered the questions which arose to me when understanding the method. In the following part, I will detail my comments point by point.

PROS
1. The problem that this submission aims to solve, how to extend the application range of transfer learning for images, is important. And the authors make a step forward.
2. The proposed method is simple and easy to implement, which is also painless to understand.
3. The writting is quite good and I can follow it easily.

CONS

1. While the authors present a number of quantitative comparisons with existing methods, I find the corresponding qualitative results are limited. I can only see Fig. 3 in the submission file. Please attach more visual comparison results to highlight your advantages.
2. How to evaluate the results of each method is quite subjective. And there is not a canonical metric for quantitative evaluations. Thus I suggest the authors do a user study here to reduce the unwanted subjective influences.
3. Currently, this project performs the experiments on the Animal faces, Birds and Foods datasets, which are not the commonly used datasets in many I2I methods. However, with the default settings, they have the best performances on their chosen dataset. So the authors might need to do some experiments on those datasets with corresponding metrics for a fairer comparison.
4. There are also not qualitative comparison figures for tasks of audio2image and text2image.
5. In Fig. 3, I cannot see too much visual improvement especially for the bird figures. For example, StarGAN seems to generate better details. On the $4$-th row,  the feather colors of the proposed method are brighter than StarGAN. 
6. Also, another issues is that when the target dataset becomes bigger and more complex, the expected advantages become smaller and the developed method even behaves worse than previous methods (e.g. in Tab. 1 for Foods). This makes me hesitate the practical values of this method when working in the real world, which is very complicated.
7. Are there some examples to support the claim about the style mixing characteristics? Or can the authors present some citations to support it?
8. In Sec. 3.2 & Sec. 3.3, why should use the pre-defined discriminator to initialize the encoder and optimize it? Are there some specific motivations to do so? I assume more discussions are needed here to make it more self-contained.
9. I am looking forward to more special designs when trying to remove the constraint that requires the GAN architecture within the X2I system to be exactly the same as that of the pretrained GAN, instead of justing using several layers for dimension mapping.
summary_of_the_review: I think the paper could be strengthened by further analysis, experiments, and presentations fixing. However, I like the general idea and I am favorable to accept it considering a large number of work they have done and signifance of this problem.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presents a framework for knowledge distillation in image-to-image translation and arbitrary domain-to-image (e.g., text-to-image) translation. Authors claim following contributions: (1) a unified transfer learning method that leverages only synthetic images, (2) a semantic diversity loss, (3) a style-mixing triplets representing a tuple that includes an input image, a reference image, and a target image.
Authors provide results for image-to-image translation tasks and for text-to-image translation tasks. A limited comparison with several modern models is presented. Authors claim that the proposed transfer learning method outperforms modern state-of-the-art.
main_review: -------------------
Strengths
-------------------
- Authors evaluated their approach on multiple datasets and multiple tasks. Demonstrated results prove that qualitatively the model is comparable to modern state-of-the-art such as TransferI2I.
-------------------
Weaknesses
-------------------
- The proposed approach for transfer learning is obviously inspired by TransferI2I. A proper definition of main contributions to the TransferI2I model for the image-to-image translation problem will make the paper more sound.
- The provided comparison with the baselines is limited and doesn't present modern image-to-image translation models, e.g. [a,b]. 
[a] Schönfeld, Edgar, et al. "You only need adversarial supervision for semantic image synthesis." International Conference on Learning Representations. 2020.
[b] Zhou, Xingran, et al. "CoCosNet v2: Full-Resolution Correspondence Learning for Image Translation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
- The paper is hard to follow, and contains many vague sentences, e,g. Section 3.3, page 5:
However, when naively applying this for knowledge distillation the conditional information can still be ignored
Why conditional information can be ignored?
- The semantic diversity loss is declared to be novel in the abstract. Yet, equation (7) demonstrates that it is a small modification of a loss from paper  by (Yang et al., 2019). The modification is described as 'solely change the conditional terms'. A more detailed explanation why conditional terms must be changed, will significantly improve the description of the proposed semantic diversity loss.


summary_of_the_review: The paper would benefit from the following improvements:
- Clear definition of main technical contribution in the Introduction
- Evaluation of the semantic-to-image task in terms of mIoU (i.e. comparison of the input segmentation with the results of a segmentation model on a synthesised image)
- Extended explanation of the semantic diversity loss
- Extended evaluation of the semantic-to-image task with modern models such as [a,b]
review_title: null

Meta-review: This paper studies the problem of distilling the knowledge present in different GAN-based image generation tasks. The paper received mixed reviews. The reviewers had difficulty understanding some details regarding the approach, and requests for ablations and clarifications on existing empirical evaluation. The authors provided a strong thoughtful rebuttal that addressed many of those concerns. The paper was discussed and two reviewers updated their reviews in the post-rebuttal phase. Reviewers generally agree that the paper should be accepted but still have concerns regarding contribution and writing. AC agrees with the reviewers and suggests acceptance. However, the authors are urged to look at reviewers' feedback and incorporate their comments in the camera-ready.
Decision: Accept (Poster)