Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper:  
The paper provides interesting analysis and leveraging of GAN’s model alignment (i.e., transfer learning). Without custom architectures and losses, it demonstrates impressive performance in a diverse set of tasks (image translation and image morphing). It also demonstrates promising results for zero-shot image recognition by leveraging the shared latent space of aligned models. 

main_review: ##########################################################################

Pros:  
1. The paper performs the first detailed exploration of model alignment based on StyleGANs. Considering the wide-range applications of StyleGAN, I think the analysis is timely, insightful, and interesting!
2. To analyze the model alignment during transfer learning from the source domain to similar/distant target domains, the paper uses weight resetting and quantitative channel alignment measurement. These techniques probably inspire others to analyze their GAN models.
3. The paper demonstrates impressive and promising experimental results on image translation and image morphing between different domains. Besides, it demonstrates the benefits of the shared latent space by zero-shot recognition/regression. 
 
##########################################################################

Cons: 
1. I’m confused about the example of “Age” in Figure 2. Why does the “Age” change the identity, background, and color largely? It looks like another totally different face.
2. Many details of the results are missing. Take Figure 2 as an example, does the semantic control apply in overlap channels (e.g., 15 for eyebrow in FFHQ2MetFace) or all the distinct channels of the child model (e.g., eyebrow 35)? Are they in the same layer? How do the authors transfer the pose control from face to church and bedrooms in Figure 14? Please check all the results and provide the necessary details. 
3. Does the claim of “the hidden latent semantics” hold for the case of (parent FFHQ, child Church, grandchild FFHQ)?
4. I think most of the successful results can be attributed to the well-structured properties of the human face, animal face, and mega face. It would be helpful to provide further discussion on the “real distant domain”, e.g., face VS church. For example, is it helpful to transfer learning from face to church? Are there any other shared semantics between face and church except pose?

summary_of_the_review:  
Overall, I vote for accepting. I like the analysis of aligned GANs’ models. The paper also demonstrates impressive experimental results by leveraging the properties of aligned models for image translation, image morphing, and zero-shot classification. My major concern is about the clarity of the paper and some missing details. Hopefully, the authors can address my concern in the rebuttal period.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper provides an in-depth study of the properties and applications of the semantic alignment between the original parent StyleGAN model and its finetuned child model on another dataset. Specifically, the paper empirically demonstrates the semantical alignment of the two models. Then, based on the properties, the paper solves serval tasks like image translation, image morphing, zero-shot image editing and attribute classification. 
main_review: **Strengths:**
+ There are many works using aligned StyleGAN models but without deep analyses on the properties. This paper provides an in-depth study of the properties, which provides some insights on the aligned models and might inspire following more complex researches. 
+ The organization of the paper is good and the analyses are comprehensive. Several observations are studied with quantitative/visual analyses to prove key properties, followed by corresponding applications utilizing the properties, making the paper easy to follow and concrete.

**Weaknesses:**
+ While the part of the property study is comprehensive and interesting and provides insights, the part of the application is less insightful. The application involves the image translation, image morphing and zero-shot vision tasks. 
  - Image translation and image morphing are intuitive and well-studied in previous studies, as also pointed out in the paper, which might provide limited insights. Image morphing by interpolating aligned models has been studies in `[R1]`.
  - The zero-shot vision tasks involve utilizing the label of the parent domain to edit/classify the child domain, which gives some novel ideas but alone are not very thorough to me.  
  - I think it will be valuable to investigate the applications in terms of the specific properties. For example, only the conv parts of network change greatly for close domains, which supports translations. Can we just finetune the conv part and fix all other parts for better translations? (which is also discussed in `[R3]`) The paper discusses that the latent semantics are hidden rather than forgotten during finetune. It will be valuable to also discuss the potential application of this property.

**Some small issues:**
+ In Fig. 14, the authors demonstrate the semantic alignment between the human face and the church. However, only the church images are shown. The corresponding face images using the same latent z are suggested to be added to help the readers better find their correspondences.
+ In Sec. 4.2, the paper claims that the layer swapping performs the transition as a series of discrete steps, rather than continuously. In the implementation of Pinkney `[R2]`, the layers can be smoothly swapped through the parameter ` blend_width` instead of hard swap, which might be able to perform the transition continuously.

`[R1] 2019 CVPR Deep Network Interpolation for Continuous Imagery Effect Transition`

`[R2] https://github.com/justinpinkney/toonify/blob/master/StyleGAN-blending-example.ipynb`

`[R3] 2021 TMM Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network`

summary_of_the_review: This paper investigates an important topic of the semantic alignment of the parent model and finetuned child model. Despite the application part could be further enriched to better match the analyzed properties, the comprehensive analyses on the properties provides many insights and might arouse following researches. Therefore, I am positive.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work is about the task of transfer learning to tame a new "child" network using a pre-trained "parent" network. While the model and fine-tuning technology lack novelty, the shared semantic information in the generation network are interesting. Finally, the authors applied the proposed aligned model to multiple tasks, including image-to-image translation, cross-domain image morphing, and zero-shot classification and regression. The impressive results with shared semantic information are achieved. 
main_review: Pros:
1. The shared latent space for "child" and "parent" networks is interesting, and matches the learning representation goal. 

2. The overall idea and the proposed fine-tuning method towards analyzing the shared latent space is reasonable. The StyleGAN-based architecture is a powerful structure, which consists of a mapping network, an affine translation, and a generator. This is exactly suitable for exploring feature attributes at different levels.  

3. The paper provides many interesting visual results, such as in Figures 1, 2, 3, and 4, to show the effectiveness of shared information in different networks. These interesting conclusions may contribute to the generation community to design better generator works. 

4. The wide applications are being met by subtly using the shared semantic information between different domains. 

Cons:

While the impressive results are achieved, I believe some parts need more clarification (even after considering the supplementary material):

1. The key limitation of this work is the novelty. If I understand correctly, the authors just fine-tuning the different parts of the StyleGAN. This is very similar to the existing works AgileGAN and the shared attributes have also been mentioned in prior MUNIT. It takes me a hard time to buy the novelty of models, techniques, and theoretical insights. 

2. The authors claimed the "two W spaces are point-wise aligned" on page 4. While they showed some shared attributes in Figure 2 and Table 1, and they also demonstrated some attributes are hidden, I do not believe these attributes are "point-wise" aligned. Is this conclusion correct? How to demonstrate it?

3. A quantitative and qualitative comparison with the latest CUT and F-LSeSim is given in Figure 5. While the improved results are provided, the original CUT and F-LSeSim work for aligned shapes. Why not provide some aligned examples, for example, horse2zebra, night2day, apple2orange, every two domains share a similar shape, but different appearances? This would be more robust to demonstrate the effectiveness of aligned style on both aligned examples (Figure 5) and unaligned examples (Figure 6).

4. In Figure 6, the StarGAN2 seems to be able to provide more consistent results to the source image, such as the smiling mouth in the first row, and the same head poses for the third row. Except for the FID score, it seems the StratGAN2 provides better translated results (better shared content, such as pose and expression). 

5. The authors show abundant results in Figures 23, 24, 25, and 26. Similar results are also being met in the StyleGAN-based method. I do not fully understand the key challenge in such a situation.  
summary_of_the_review: As can be inferred from the balance in strengths/weaknesses, my preliminary rating for this submission is borderline accept. While the interesting results are shown in the paper and supplementary material, the key contribution for models, techniques or theoretical insights is unclear. Furthermore, some comparisons seem unfair in the paper. The authors should clearly interpret them during the rebuttal.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper undertakes extensive experimentation in the adaptability of latent space modifications from a base model, and a fine-tuned model for a secondary dataset. They demonstrate which areas of the models change most, and which information remains trained in the parameters, despite re-training. They go on to use their findings in downstream experiments, reaching state-of-the-art quality.
main_review: Weaknesses

	The main weakness for this paper is likely the limited novelty in the technical contributions, given that transfer learning is a fundamental, established technique. However, this paper is more of a study of this technique, applied to the particular domain of image GANs, and how their editability changes during the process.

	The qualitative results do seem high quality, but it is sometimes quite difficult to see where the resemblance to the reference images are (especially with Figure 6, for the dogs dataset). Perhaps the translation is not focused strongly enough on the reference images. Despite more artefacts being present, I'd say OverLORD images are a better transfer, for both cats and dogs.



Strengths

	The primary contribution of the paper is the extensive experimentation, around transfer learning in GANs, and they perform this very well. Their approach is very well documented, is very thorough, and convincing. The supplementary videos are helpful in further displaying their results.


	Although there are aspects that are both better and worse, the samples from their models compared to other models, are mostly good. The benefits arise mostly from the editability of the output, given their findings that the channels maintain their editing effects from previous datasets with more labels.
summary_of_the_review: The paper presents some novel analysis in a popular field of research, with implications that can help drive the field further, with minimal changes.
review_title: null

Meta-review: The paper provides an interesting analysis of aligned GAN models. The paper shows that when a model is obtained (fine-tuned) from another, then the corresponding hidden semantic spaces are aligned. The paper uses this property to show that without any additional architecture or training, the models can perform diverse tasks such as image translation and morphing. The paper also demonstrates that zero-shot tasks can be performed by learning in the parent domain and transferring to the child domain.

All reviewers agree that the paper presents an interesting analysis and findings and will make a valuable contribution to the field. The reviewers raised some particular concerns, which were addressed by the authors in their response.
Decision: Accept (Oral)