Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: ** Paper Summary **

This paper proposed a simple regularization technique for domain generalization tasks, termed MixStyle, based on the observation that domains are determined by image styles. By mixing styles of different instances, which generates synthesized domain samples while preserving the content features, the proposed method achieves the generalizability of the trained model. The MixStyle was applied to numerous applications, such as category classification, instance retrieval, and reinforcement learning, and attained the state-of-the arts. The MixStyle is relatively simple to implement, but effective. 
  
** Paper Strength **
+ Simple methodological design, so it is easy to implement.
+ Understanding the domain shift problems as a style variation makes sense.
+ Randomizing the styles might be the solution to alleviate the domain generalization problems, but searching all the possible styles and applying them would be challenging and not feasible. So, using different instance samples to extract the styles was nice.
+ It makes sense that introducing the \lambda to mix the styles itself and ones of different instances.
+ The paper is well organized and written.

** Paper Weakness **

I have no major comments on this paper, but minor comments as follows:
- Even though the authors have shown the ablation study to analyze the levels where the MixStyle should be applied, it is not clear for me yet. The authors applied the MixStyle after 1st, 2nd, and 3rd residual blocks for category classification problems, but applied the MixStyle after 1st and 2nd residual blocks for category classification problems for instance retrieval task. In 3.4 analysis, they only showed the ablation studies on the category classification. Thus, one think the optimal combinations may vary according to the applications. In addition, another combination, e.g., conv34, conv25, would be more interesting.
- Fig 4 is hard to understand; what do the corresponding style statistics mean? Why does (d) only represent different legends? 
- In Table 1, some experimental settings, e.g., Cartoon or Photo, have shown that MixStyle w/ random shuffle was better? The discussion on this might be interesting. 

review_title: Review for "domain generalization with MixStyle" 

final_score: rating: 6: Marginally above acceptance threshold
review_content: This work proposes a technique for domain generalization by mixing style of images from different domains. This work adopts a mix up style approach [A] for domain generalization. Different from [A], the paper proposes to conduct mix-up in the intermediate layers, in particular, instance normalization layers. The proposed approach diversifies the data implicitly and the experimental results show that the mix-style can improve domain generalization.

Overall the paper is well-written with plenty of details. I also appreciate the experimental analysis in Sec 3.4 and the variance reported in Table 1. However, I have several concerns regarding the paper:
- The technical novelty seems rather incremental. This method is an extension of [A] to the instance normalization layer. Similar strategies have been discussed in other works such [B] and [C]. However, these works are not discussed in terms of main similarities/differences.
- I also found the experimental validation not fully sufficient to grant publication. Currently the validation is only conducted on PACS, the improvement also seems limited. I believe validation on more datasets(such as Digits, Office-Home as used in L2A-OT) can further confirm the effectiveness of the proposed method.
- I suspect that  interpolating the style parameter might cause performance drop on the domains that have been seen during training. Would it be possible to report performance on the domains that have been seen in the training?

[A] Vikas Verma et al. Manifold Mixup: Better Representations by Interpolating Hidden States. In ICML 2019.
[B] Rui Gong et al. DLOW: Domain Flow for Adaptation and Generalization. In CVPR 2019.
[C] Seonguk Seo. Learning to Optimize Domain Specific Normalization for Domain Generalization. In ECCV 2020.

---
I have read authors' response and other reviews. Some of my concerns are addressed in the response. Especially the added discussion with related work is helpful. Thus I would increase my rating to 6.

review_title: Insufficient technical novelty and experimental validation

final_score: rating: 7: Good paper, accept
review_content: **Summary:** The paper proposes a simple method for domain generalization where multiple source domains are given for a certain task (like image classification) and testing happens on an unseen domain. The authors are inspired by normalization-based style-transfer techniques (Adaptive InstanceNorm) and propose to mix the styles of different source domains to effectively increase diversity of domains during training.

**Pros:**

- Overall, this is a well written paper with a clear idea that is simple but intuitive.
- The idea is well described, put into context of prior work and empirically validated to improve results over various baselines.
- It is good to see experiments outside of plain image classification to validate the proposed idea.
- The analysis where to apply MixStyle is good and makes intuitive sense.


**Cons:**

- The relation to MixUp needs to be explained in more details. While related to the proposed MixStyle, MixUp creates a convex combination of both input and output spaces. I can believe that MixUp as a standard data augmentation gives worse results than a vanilla CNN (Table 1) but I would not fully agree with the statement "... which demonstrates the advantage of mixing style statistics at the feature level over mixing images at the pixel level" from page 4. MixUp also interpolates the output label space, so the advantage cannot be only attributed the placement of the mixing within the network instead of at the pixel level.
- As an additional baseline, one could use MixUp with a sampled lambda that is larger than 0.5 in all cases (like in [FixMatch. Sohn et al. NeurIPS'20]) but keeping the label from sample $x$ rather than interpolating with $\hat{x}$.

- I do not understand why the suffix "_x" is added to the analysis in Table 3. Is MixStyle applied after each convolutional layer or after each block in a ResNet architecture? Specifically, for "conv234_x", how often is the MixStyle layer added? (3 times or 3 * num_convs_in_block times?)

- For the ReID experiments, I think it should be better highlighted that the cross-dataset setup is the key difference to evaluations in prior work. This somehow gets almost unnoticed because the default setting of ReID is already considered a valid domain generalization task due to the new label space and camera views. This left me a bit confused about how RandomErase can be a widely used data augmentation technique for ReID when it gives worse results in the experiments from Table 2. This became clear to me only after reading the discussion in the last paragraph of Section 3.2.

- I would not make the statement that "... mixing is CLEARLY better than replacing" on page 7 (see Table 4) while also stating that "... with alpha increasing from 0.1 to 0.4, the accuracy SLIGHTLY slides from 82.8% to 81.7%". That "slight" change is larger than the "clear" gap before.


**Other notes and open questions:**
- MixUp was used successfully as regularization for semi-supervised learning (SSL) [MixMatch. Berthelot et al. NIPS'19]. Can MixStyle also be used for SSL?

review_title: Simple but effective contribution to domain generalization

Meta-review: All three reviewers recommend acceptance after the rebuttal stage, and the AC found no reason to disagree with them. The proposed method is simple and effective, and the concerns raised about experimental validation and novelty seem well addressed in the rebuttal. 
Decision: Accept (Poster)