From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Accept: poster
metareview:_summary,_strengths_and_weaknesses: This paper studies an interesting problem, unsupervised domain generalization (UDG), in which no paired data is required to bridge different domains. This problem is relatively under-explored, but it is meaningful in the DG context. The proposed approach is a complex combination of well-established methods, such as MAE, CycleGAN, and StyleMix. While the method is more engineering-oriented, the experimental performance is very good, which can be seen as the biggest merit of this paper. Reviews were with mixed ratings at the beginning. After rebuttal, all three reviewers acknowledged that their concerns were addressed reasonably well. AC read the paper and oversaw the reviewing process, and agreed that this paper has its merits that outweigh its flaws, and the current version is slightly above the acceptance threshold. Authors should incorporate the revised material into the future version.
justification_for_why_not_higher_score: The current paper is believed by the reviewers and the AC to be incremental in its technical solutions. While the empirical performance is good, which is acceptable, there is no evidence that it shall be presented as spotlights.
justification_for_why_not_lower_score: All three reviewers acknowledged that their concerns were addressed well. The paper has its merits in tackling a challenging UDG problem with a reasonable solution that does not rely on paired data. This is a solid empirical work that is a meaningful addition to the conference. Nonetheless, since this is a highly selective conference, it is not a great loss if this paper is rejected.
note_from_PC: if the above contains the word "oral" or "spotlight" please see: "oral" presentation means -> notable-top-5% and "spotlight" means -> notable-top-25%. As stated in our emails, we are disassociating presentation type from AC recommendations

From: ICLR.cc/2023/Conference/Paper783/Reviewer_QvTm
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper aims to address unsupervised domain generalization via a proposed cycle-consistent masked autoencoder, which includes a CycleGAN, MaskAutoEncoder, and contrastive learning. The authors claim that the poposed method can significantly improve state-of-the-art performances, which are demonstrated by the experimental results to some extent.
strength_and_weaknesses: # Strength

1. The motivation of introducing CycleGAN, MaskAutoEncoder, and contrastive learning sounds reasonable.

2. The experimental results look solid.

3. The writing is good, and the structure is easy to follow.

# Weaknesses

1. The novelty of this paper seems marginal. The main modules of the proposed method, including CycleGAN, MaskAutoEncoder, and contrastive learning, are all existing famous works. It seems that the authors simply combine these works into UDA.

2. The visualization results in Figure 5 are unsatisfied, especially the sketch ones. It is highly suggested to release more visualization results to demonstrate the effectiveness of the reconstrcution process.

3. I notice that the authors use Vit-small as the backbone network. Do all the other comparison methods use this network? If not, it is suggested to compare the backbones of different methods, including training time, model size, and inference speed. 

4. The proposed method needs to be pre-trained for 1000 epochs, which may easy to cause overfitting. It is suggested to add a set of experiments to fully prove the generalization of the proposed method.

clarity,_quality,_novelty_and_reproducibility: The quality, clarity and originality of the work are good.
summary_of_the_review: Please see [Strength And Weaknesses].
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
details_of_ethics_concerns: None
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper783/Reviewer_BvPn
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper proposes to use cycle consistency to help learn domain-invariant features for unsupervised domain generalization.
strength_and_weaknesses: Pro:

1.  Previous method DiVAE only considers the reconstruction loss ||y_m-x_m||. The proposed loss considers how to utilize the information of y_n and use cycle consistency to further help the task, i.e., adding a backward transformation process for other domain labels $n\neq m$.

2. The paper also notices that the generated images in other domains $$y_n$$ may be trivial such that all images are of the same style. To encourage diverse domain information, they propose a domain contrastive loss.

3. The results are promising and the paper is well-written.

Cons:

1.  Since the model adds a backward process compared to DiVAE, I'm wondering how much training time is increased. It would be good to have a running time comparison.

Comments:

The proposed domain contrastive loss encourages the outputs to have different domain styles but there is no supervised loss to encourage it to be exactly the domain style i. Is it possible that the outputs of different domain-specific decoders are out of the given domains and harm the performance? 


clarity,_quality,_novelty_and_reproducibility: The paper is well-written and novel.
summary_of_the_review: The paper proposes to use cycle consistency to help learn the domain-invariant features and the results are promising.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper783/Reviewer_7ZLc
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper focuses on the setting of unsupervised domain generalization, where the multiple source domains do not have label supervision and the target domain is unseen. Based on the previous work (i.e., DiMAE) that applies the masked-reconstruction-modeling as a self-supervised task, this paper further proposes two more objective functions: the cycle reconstruction loss and domain contrastive loss. The first one loss transforms the generated image back to original image domain and the second one loss performs a contrastive learning across the features from different decoders. The authors have conducted experiments on the DomainNet and PACS and achieves some improvements over the previous methods.
strength_and_weaknesses: Issues regarding the fair experimental comparisons.
+ To the reviewer’s knowledge, some previous works such as DiMAE use a supervised pre-trained model while this paper adapts an unsupervised pre-trained model on ImageNet. To this end, could the authors explain how much will the different pre-trained models affect the final performance? Meanwhile, pre-trained models using different pretraining recipes (e.g., number of training epochs) are also different starting points for downstream tasks. Overall, the reviewer hopes the authors to make sure the experimental comparisons are fair in terms of the pre-trained model.
***
Issues regarding the methods.
+ Could the authors clarify the extra training overhead they have introduced in their framework? Because more decoders and forward passes are appended upon previous approach.
+ As claimed in the paper that, the domain contrastive loss encourages more diverse domain-specific styles to be learned in the decoder. The reviewer is interested that why such constraints should be placed in the early layer of the decoder since the higher layer closer to the pixel space might be more responsible to encoder style information? 
+ The unsupervised domain generalization is a new setting in the field. The reviewer believes that reporting the performance in the unsupervised source domain is also of significance. Could the authors provide such information?

clarity,_quality,_novelty_and_reproducibility: + The paper is well-written and easy to follow.
+ If compared with previous work DiMAE, the novelty of this paper is somehow limited. Based on the previous reconstruction loss, this paper proposes to further include a cycle-consistent one. The similar story has repeated frequently in the field.
+ Since the unsupervised domain generalization setting is relatively new, the reviewer finds that most of the published methods do not have open implementations. To this end, the reviewer is not sure whether the experiments are conducted in a fair environment and whether there are some tricks hinder the reproduction. 

summary_of_the_review: This paper proposes to use a cycle-reconstruction loss and domain contrastive loss in the unsupervised domain generalization. Consistent performance gains are detected on two widely-used benchmarks. Currently, the reviewer’s concerns are two-fold. On the one hand, the authors should make sure that the experimental comparisons are fair and reproducible. On the other hand, the two new losses are kind of limited in the terms of novelty, which might be lower than the bar to be accepted to a venue like ICLR.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold