Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Summary: The authors use Domain Invariant Categorical Semantics to improve unsupervised domain translation (UDT). They learn these semantics in an unsupervised manner. They show how this can improve results on Semantic Preserving Unsupervised Domain Translation and Style Heterogeneous Domain Translation by doing experiments on MNIST<->SVHN (features traditionally learned are very different but digit identity could be the same) and Sketches->Reals (distinct styles) respectively.

---

Strengths:
- The visual results on both tasks/datasets are striking.
- The paper is simple to read and the idea is intuitive.
- Experiments are extensive, including an ablation on the losses and comparison against baselines.

Weaknesses: 
- More examples of Sketches->Real could have been shown;
- When does this method fail? Some failure cases would be good

review_title: Nice Results using Domain Invariant Categorical Semantics to Improve UDT

final_score: rating: 7: Good paper, accept
review_content: The paper addresses the domain translation problem and proposes a novel approach to translate images between domains in an unsupervised manner, by integrating unsupervised learning of domain-invariant semantic features between the two domains. The paper is well-written with a clear standing point and motivation, along with well-described contributions. The paper has an inclusive and sound theoretical comparison to related work. Evaluation is well-designed and includes previous work in the same context.
In overall, it is a good paper with an original and potentially inspiring idea and a convincing application of conditional GANs, would be an interesting read for many in the conference.

Typos
Page 1: 
showing that such a translation
two situations where unsupervised domain translation
Page 7: 
the representations of the real images

Comments
1. 'unsupervised domain translation methods work due to an inductive bias toward identity' in sections 1 and 3.2: need citation to support this statement
2. Considering clustering is one of the important parts of the contributions it may need more emphasis in the paper, it would be nice to restructure this part, extend the discussion of alternatives and an analysis to compare different approaches (also by moving some part of the discussion to the paper from the appendices).
review_title: An interesting and original idea

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary: This paper proposes to learn the categorical semantic features in an unsupervised manner to enhance the ability of image translation at preserving high-level features, and obtains some good results on Semantic-Preserving Unsupervised Domain Translation and Style-Heterogeneous Domain Translation problems. 


Major issues: 
- The proposed method seems to be a combination of current works. The main contribution of this work may be leveraging the unsupervised representation learning for semantic features extraction.  

- The quality of generated images is still not satisfactory with such rapid development of GANs.

- The experiment and qualitative evaluation are too limited. Only two image translation tasks are conducted for comparison, and little visual results are given. It will be preferred if some common I2I tasks results are given. Only FID is used, adding other metrics, such as LPIPS, NDB, and JSD, will be more convincing. 


Minor issues
- What are the essential differences between SPUDT and SHDT problem? How does the model solve the two problems according to their differences?


review_title: An interesting work but lacking promising results 

final_score: rating: 7: Good paper, accept
review_content: This paper presents unsupervised domain translation (UDT), considering two scenarios: Semantic Preserving Unsupervised Domain Translation (SPUDT) and is Style-Heterogeneous Domain Translation (SHDT). This study uses MNIST and SVHN datasets for demonstrating SPUDT and Sketches and Reals samples from the DomainNet dataset for demonstrating SHDT. Although the method uses different components depending on the scenario (or dataset), the presented architecture is essentially the same.

The method consists of the framework for learning invariant categorical semantics across domains (Section 3.1) and semantic style modulation to make SHDT generations consistent (Section 3.2). Section 3.1 consists of unsupervised representation learning, clustering, and unsupervised domain adaptation. However, this section did not describe the first two components in detail probably because the method uses different components depending on the datasets (described in Section 4). Unsupervised domain adaptation is realized by the minimization problem described in P4. Section 3.2 describes content encoders, semantics encoders (obtained in Section 3.1), mapping networks, style encoders, and generator. The section presents the loss functions (adversarial loss, cycle-consistency loss, style-consistency loss, style diversity loss, and semantic loss).

Section 4 compares the proposed method with UDT baselines under SPUDT and SHDT scenarios. The experiments on MNIST and SVHN show that the proposed method achieved high accuracy in domain translation and high quality in the generation (measured by FID). The experimental results on Sketches and Reals show that the proposed method yields high-quality generations. Overall, the experiments demonstrate the importance of incorporating the semantics category in UDT.

PROS

It was interesting to see that the performance of UDT was drastically improved by incorporating semantic categories induced by the clustering algorithms.

The presented method was reasonable and well designed.

CONS

The main part of this paper is dense. I had to go back and forth between the main body and the appendix a lot.

This paper does not seem to emphasize the technical novelty of this work. Although Section 3.1 was designed to present a novel framework, it does not explain the detail of the components. Methods for unsupervised representation learning and clustering are explained in Section 4. The component of unsupervised domain adaptation may be a new proposal in this work. However, this paper did not explain this component in the main body; I had to read Appendix B.5 to understand the formula presented in P4.

The model architecture described in Section 3.2 is mostly based on previous work (e.g., adversarial loss, cycle-consistency loss, etc.). The semantic loss presented in this section may be a new addition to UDT. But again, this is not emphasized in Section 3.2.

QUESTIONS

P4: "We describe those regularizers in more detail in Appendix B.5."
If this part is a proposal in this paper, it should be explained as the main content of this paper. If I do not read Appendix B.5., I have no idea how domain 2 is involved in the minimization problem.

MINOR COMMENTS:

P1: "That are mappings biased toward the identity."
"That" -> "Those"

It may be convenient to have equation numbers so that reviewers/readers can locate a formula directly.
review_title: Official Blind Review #2

Meta-review: This paper studies the problem of unsupervised domain translation. Here translation does not refer to language translation. Instead, it refers to the idea of transferring high-level semantic features. Specifically, the authors look at digit style transfer (between MNIST/postal address numbers and SVHN/street view house numbers) and Sketches to Reals. The visuals look very convincing and the empirical results are strong, too. There is one weaker review but the authors address the concerns in their response and the reviewer did unfortunately not respond despite promting.
Decision: Accept (Poster)