Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an approach to domain generalisation whereby they use Fourier to generate synthetic images which are then used as a worst-case target domain to improve the model's robustness and therefore its generalisation. This is reminiscent of a domain adaptation approach. The amplitude-based generator that creates the synthetic images is trained in an adversarial manner and is combined with the phase source domain data. Surprisingly, the method performs competitively compared to other methods proposed.
 
strength_and_weaknesses: Strengths:

a) generating target images for modelling worst-case scenario
b) repurposing mean classifier discrepancy to propose a supervised variant that leverages the labelled generated target domain data
c) competitive performance
d) relative simple implementation

Weakness:

a) not a weakness per se, but I am trying to get my head round why fourier amplitude for adversarially generating images.... is that to create bad but not too bad images deliberately to force a better generalisation performance? It sounds rather arbitrary to me (not in a negative way) so I would like to know what the thought process was to arrive to that approach vs trying other ways,
clarity,_quality,_novelty_and_reproducibility: The paper is well articulated and provides the extra context in the appendix.
The empirical part is good and the method is decent and relatively simple, albeit working surprisingly well across several datasets. 
I would say that pseudocode is just enough to reproduce it, but that assumes that the hyperparameters described in the main text are what one needs to reproduce the results rather than trying to magically grid search parameters. In addition, it would be nice to have a response from the authors as to whether they plan to release the full code.

****Authors please be consistent in the use of American vs British English - choose whichever you prefer but stick with it across the paper.****
summary_of_the_review: This is a good submission that seems to perform well. Authors have run several experiments some appearing in the main text and some others in the appendix. There is a good empirical component and the novelty is fair. I think this is a decent submission to be considered for acceptance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a new domain generalization (DG) method by synthesizing the virtual target domain samples in Fourier domain and exploiting the maximum classifier discrepancy (MCD) principle in source domain and generated target domain. It also gives a modified MCD loss, which is suitable for DG task. The experiments show the effectiveness of the proposed method.
strength_and_weaknesses: Strength
(1)	The proposed method is novel and well-motived. 
(2)	This paper gives a new perspective on DG by finding the worst-case ib an adversarial way and posing it as a domain adaptation (DA) task. 
(3)	Empirical results indicate that the proposed method greatly outperforms the other methods.
Weakness:
(1)	Even the generated target domain is the "worst case ", it is not as same as the true target domains. It would be better to discuss this in theoretical analysis.
(2)	Why do not directly minimize e^*(\mathcal{H} ; S, T) by one supervisor loss if you have known the label of generated target domain.
(3)	The results of CMNIST and RMNIST in the DomainBad benchmark are not provided in this paper. 
(4)	In this paper, there is no comparison with other methods [3,4] in different backbone networks, such as resnet18 in PACS.
(5)	Only the result of the leave-one-domain-out setting is provided. Can you give the result in single source generalization setting as EFDMix[4]?
[1] Minyoung Kim, Pritish Sahu, Behnam Gholami, and Vladimir Pavlovic. Unsupervised Visual Domain Adaptation: A Deep Max-Margin Gaussian Process Approach. CVPR2019.
[2] Zhihe Lu, Yongxin Yang, Xiatian Zhu, Cong Liu, Yi-Zhe Song, and Tao Xiang. Stochastic classifiers for unsupervised domain adaptation. CVPR2020.
[3] Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. ICLR2021.
[4] Yabin Zhang, Minghan Li, Ruihuang Li, Kui Jia and Lei Zhang. Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalization. CVPR2022.

clarity,_quality,_novelty_and_reproducibility: The clarity, quality, and novelty are pretty good. Not sure if the paper results can be reproduced.
summary_of_the_review: The paper proposes a novel and well-motived method for domain generalization. However, there are lacks of some experiments and more discussion about the theory and the relation between SMCD and other losses. If you solve the problems, I am willing to improve my rating.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper targets at solving domain generalization. The authors tackle the DG problem as DA task where the worst-case `target' domain is adversarially synthesized. Specifically, they generate Fourier amplitude images and combine them with source domain phase samples. MCD is further exploited to relate the target domain performance to the discrepancy of classifiers in the model hypothesis space. Bayesian hypothesis modeling is adopted to make adversarial MCD minimization feasible. Experiments show the effectiveness of the proposed method.
strength_and_weaknesses: Strengths:
+ This paper studies an important problem
+ This paper is well-written, it is enjoyable to read.
+ Experiments are comprehensive and show the effectiveness of the proposed method.

Weaknesses:
+ The contributions are a little limited considering that MCD is an existing approach.
+ This paper should concentrate more on true solutions in DG. More discussions about the true solution in DG should be provided.
+ Why choosing Bayesian hypothesis modeling? It should be clearly discussed.
+ More insightful discussions should be provided. For example, how to improve the generalization ability of deep models using other adversarial DA approaches (e.g. DANN and MDD)?
[1] Domain-adversarial training of neural networks
[2] Bridging theory and algorithm for domain adaptation.
clarity,_quality,_novelty_and_reproducibility: Clarity: Good. This paper is well-written, it is enjoyable to read.

Quality: Good. Experiments are comprehensive and show the effectiveness of the proposed method.

Novelty: A little marginal. MCD is an existing approach. And why choosing Bayesian hypothesis modeling is not clear.

summary_of_the_review: Strengths:
+ This paper studies an important problem
+ This paper is well-written, it is enjoyable to read.
+ Experiments are comprehensive and show the effectiveness of the proposed method.

Weaknesses:
+ The contributions are a little limited considering that MCD is an existing approach.
+ This paper should concentrate more on true solutions in DG. More discussions about the true solution in DG should be provided.
+ Why choosing Bayesian hypothesis modeling? It should be clearly discussed.
+ More insightful discussions should be provided. For example, how to improve the generalization ability of deep models using other adversarial DA approaches (e.g. DANN and MDD)?
[1] Domain-adversarial training of neural networks
[2] Bridging theory and algorithm for domain adaptation.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper tackles domain generalisation by synthesizing the hardest domain, and then using domain adaptation techniques. Domains are parametrized in fourier space using the amplitudes which the phases are taken from the original images. The hardest target domains synthesized by adversarial training. The classifier is learned via MCD, which is modified here as the target labels are known. Results seem better than mixup in fourier space.
strength_and_weaknesses: Strengths:
* The paper is clearly written and a pleasure to read.
* The main ideas are intuitive
* Augmenting MCD with the true labels is probably novel (as is an aspect of this setting)
* Results indeed outperform many previous methods

Weakness
* Synthesizing domains in fourier space clearly does not generate realistic images - and so the idea diesn't really work in that sense - even though it did generate improvements on the baseline. The method resolves some of the issue by using mixup, but this is obviously not the main idea the paper was selling. It makes the reviewer ponder if this is the fourier parametrization is the correct one for synthesising the target domain. 

* The method is only slightly novel compared to the previous baseline of fourier domain mixup. Most of the advantage comes form SWAD, which is orthogonal. 
clarity,_quality,_novelty_and_reproducibility: Clarity: the paper is well written and clear
Quality: the experiments seem extensive to me, however the poor visual quality of the target domains makes me question whether there is more to this story.
Novelty: Novelty is limited, to replacing amplitude mixup by adversarial training and the supervised MCD loss.
Reproducibly: The method is not too complicated, perhaps it can be reproduced but no code was released.
summary_of_the_review: This is a well written paper, which tells a nice story and presents strong results. The novelty is a little limited and the domain synthesis is raises some doubts. I am borderline on this paper and waiting for the rebuttal and other reviews.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper received contrasting reviews before and after authors' rebuttal, ratings are slightly above and below the threshold (6, 6, 5, 5), even if one of the positive reviewer seems not against rejection.
All reviewers seem to have arguments in favor and against this work.
Main issues regard the insufficient justification of the use of the Fourier domain, weak novelty (just for some reviewers), and missing experimental results. About the use of the Fourier domain there are also other comments, e.g., regarding the fact that the method does not generate realistic images, but this is not a big issue to my opinion.
Overall, all the raised comments seem to be well addressed by the authors in their rebuttal: the paper results sufficiently novel, the methodological contribution appears significant, the experimental analysis is comprehensive, also considering all the requests done by the reviewers, hence, my evaluation is that this paper is acceptable for publication to ICLR 23.

justification_for_why_not_higher_score: This paper is evaluated having contrasting comments, which have been fixed but still the reviewers were on the fence, so I think poster presentation is more suitable than higher scores.
justification_for_why_not_lower_score: The paper was accepted since, despite the initial criticisms, authors replied properly to all comments, and in these conditions I don't see reasons to reject it
Decision: Accept: poster