Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposes a method to disentangle the common and separate parts of two domains and to focus the attention of the underlying network to the desired part only, without reconstructing the entire target. The proposed method is also able to add or remove separate contents, and to enable weakly-supervised semantic segmentation of the separate part of each domain.
This work relates to the problem of content transfer between images. The proposed method consists of five networks: one encoder for common domain invariant features, one encoder for separate domain specific information, one network for mapping encodings from common features from both domains undistinguishable, a decoder that generates sample in the origin domain and a decoder that generates the image that combines content from the origin image and domain specific content from the target image. This last decoder also outputs a mask that focuses the attention of the model to the specific part.
The proposed model is trained using a combination of different losses: domain confusion loss, reconstruction losses, and cycle consistency losses. Ablation studies reported in the paper nicely show the contribution of each loss. The final loss is obtained by a weighted sum of the losses: how are the lambda coefficient chosen/learned?
The proposed method is evaluated on guided content transfer, out of domain manipulation, attribute removal, sequential content transfer, sequential attribute removal and content addition, weakly supervised segmentation of the domain specific content. Experimental results are clear, thorough and satisfactory, both quantitative and qualitative results are reported, as well as a user study. Presented results demonstrate the strengths and limitations of the proposed approach, and the analysis of the results helps understanding and emphasizing the contribution of the paper.
Information in appendix also enable reproducibility by providing parameters and architecture structure. 
Other comments: did you observe overfit for some choice of parameter? is your validation/test set large enough for evaluating results? did you observe biases in your method (e.g. to specific features/domain specific information)?
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This work proposed a mask based approach for instance-level unsupervised content transfer, which is an extension of the disentanglement work in (Press et al., 2019) and the attention guided translation (Chen et al., 2018, Mejjati et al., 2018). Unlike the disentanglement work, the introduced mask allows the adaptation to focus on the relevant content which substantially reduce the complexity of the generation. On the other hand, the proposed method extends the attention guided translation from the domain level to the instance level which allows more specific and diverse translations. Experiments on benchmark data shows both improved qualitative and quantitative results comparing to existing methods. It is really nice that the authors also considered the situation of generalization to out of domain images.

However, I would encourage the authors to spend more discussion on the "Method" and "Ablation Analysis" sections to give a better illustration. First is the choice of the L2 norm in all the reconstruction losses, which is different from L1 norm used in both (Press et al., 2019) and (Mejjati et al., 2018). What is the advantage of using L2 instead of L1 norm here? Does it work better with the mask generation? Second, the domain confusion loss. The presence of both equation (3) and (4) are quite confusing and the domain confusion loss (3) seems different from traditional ones. In Table 7, it shows that the learned mask is empty without any of the losses (3), (5), (7). But only loss (7) is directly related to the mask generation. How does the loss (3) or (5) impact the mask learning? It is also unclear why the losses introduced in (8) would encourage the mask to be minimal despite the quantitative results shown in Table 7. Actually, I am very curious about the performance of the loss introduced in (Press et al., 2019) on top of the network introduced in Figure 2.

Other comments:
- It would be nice to see the out of domain transfer in the "attribute" domain. Ideally, the network should be able to detect "difference" in the image from domain B and apply it to the image from domain A. For example, the model is trained on faces without and with glasses, but applied to faces without and with facial hair. Indeed, the introduction of mask alleviates the decoder to learn the attribute itself, and provides the ability to locate the place of difference.
- Please unify the citation style: there are both Press et al. (2019) and (Press et al., 2019) used.
- In Section 2 under "Mask Based Approaches", the authors argued that the existing attention guided translation "does not allow for the adaptation of the image information in the masked area". I do not think this is the case. The existing work also introduced adaptation of the image information in the masked area. For example, the equation (1) in (Mejjati et al., 2018).
- How is the binarized mask generated in inference? Specifically, how to determine the threshold?
- In Section 4.1, the authors argued that "without L_{Cycle} the masks produced include larger portions of the face". But this actually produces the second smallest mask in Table 7.
- What is the "L2 reg" in Table 7?
- It would be good to show the sensitivity of the lambdas in the overall loss.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper proposes a method for unpaired image-to-image translation, where the target domain explicitly contains some additional information than the source domain. The authors use auto-encoders to separate the common and specific representations and to generate masks, which seems to be related to [1]. The authors empirically show the proposed method can be used for image translation, attribute editing.

A small citation error:
"Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networkss. arXiv preprint arXiv:1703.10593, 2017a."
networkss -> networks
It is a published paper at ICCV, not just on arxiv.

[1] Domain Separation Networks, Bousmalis et.al, NIPS 2016
review_title: Official Blind Review #1

Meta-review: This paper extends the prior work on disentanglement and attention guided translation to instance-based unsupervised content transfer. The method is somewhat complicated, with five different networks and a multi-component loss function, however the importance of each component appears to be well justified in the ablation study. Overall the reviewers agree that the experimental section is solid and supports the proposed method well. It demonstrates good performance across a number of transfer tasks, including transfer to out-of-domain images, and that the method outperforms the baselines. For these reasons, I recommend the acceptance of this paper.
Decision: Accept (Poster)