Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a DAT label smoothing approach. The results are backed by the theoretical evidences. However, the main novelty is minimalistic.
strength_and_weaknesses: Strength:
- The paper is well written.
- The empirical results are backed by theoretical evidences
- Simple solution

Weakness
- The paper used an already established idea with minor modifications and that is the major drawback of the work
- The paper showed label smoothing approach but the question is wouldn't it attribute to the catastrophic forgetting of the discriminator. 
Questions:
1) The contribution seems marginal as most of the proofs (Theorem 1 and 2) and  can be directly inferred from previous GAN based works. 
2) Proposition 1, do we need both the $\gamma(D_{S}-D_{T})$ and $\gamma(D_{T}-D_{S})$ 
3) Proposition 2, would $p_{s^{'}}$ be $p_{t}$? If proposition 2 have issues for $p_{s}\rightarrow 0$ then that would be similar for proposition 1, right?
4) Proposition 3, for multilabel scenario would the $D_{Mix}$ provide any meaningful information? Though theoretically we could show the use of it but I am curious whether there would be any practical implication of that.
5) Sec A.5, would the mixture provide any meaningful information or it would do only catastrophic forgetting to the discriminator?
clarity,_quality,_novelty_and_reproducibility: The paper shared the code and seems like the results would be reproducible though I haven't checked all of them. 
summary_of_the_review: The papers proposes a label smoothing idea for DAT. The novelty is limited as the idea of label smoothing is already established in so many previous papers.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Poor generalization on out-of-distribution data is a common problem among machine learning models. In classification tasks, one recently proposed solution is Domain Adversarial Training, which asks the model to predict the domain of the data sample in addition to its class, and encourages the model to learn domain-agnostic class-relevant features through an adversarial objective. However, this method is known to suffer from training instability. To that end, the authors proposed an extremely simple modification — label smoothing on the environment (i.e., domain) label — that improves not only the training stability but also robustness to label noise, convergence speed, and performance.
strength_and_weaknesses: Strengths
1. In general, I am in favor of simple-but-effective improvements.
2. The authors demonstrated via theoretical validations three advantages of the proposed method: (1) training stability, (2) robustness to label noise, and (3) non-asymptotic convergence speed. Further, the authors used empirical studies to complement the theoretical results. (1) is supported by Figure 2 and 5; (2) by Figure 4(a); and (3) by Figure 5.
3. I particularly like section 3.4, where the authors explicitly faced the unrealistic assumptions in their theoretical validation and analyzed the “empirical gap and parametrization gap”.
4. I am also glad about the fact that the authors intentionally organize the proofs and non-central results to the appendix, leaving the logic flow in the main text fluent and engaging.

Weaknesses
1. On the flip side of the first strong point I listed, one potential concern may be the lack of the so-called “novelty”. I fully acknowledge the thoroughness and comprehensiveness of this work, yet the technical contribution (apart from the analyses and insights) is among the least “innovative” I have seen. Afterall, label smoothing has been around for such a long time.

Minor Things
1. Would it be slightly more intuitive to write $D_S’ = D_T + \lambda(D_S - D_T)$ as $D_S’ = \lambda D_S + (1 - \lambda) D_T$ instead?
2. Typo. Page 2. “phenomena” instead of “phenomenons”.
3. Minor grammar thing. Page 3, under Proposition 2. “One **may** argue that adjusting the tradeoff weight λ can also balance AT and ERM**.** **H**owever, λ can only adjust the gradient contribution of AT part, …”
clarity,_quality,_novelty_and_reproducibility: The paper is of high clarity and quality. Novelty is slightly questionable since the building blocks are not new. 
summary_of_the_review: This paper introduced a seemingly trivial modification, label smoothing on environment label, to a popular approach for out-of-distribution generalization. Notably, the authors showcased the advantages of the label smoothing in three different aspects through a combination of theorical and empirical analyses. Overall, I find this a very solid piece of work and would recommend for acceptance of this submission.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the domain adversarial training problem and propose to use label smoothing for domain discrimination. Specifically, motivated by the observation that different domains from VLCS dataset show small difference, thus a soft domain label should be applied to domain adversarial training instead of hard label. By simply rewriting the cross-entropy loss, the proposed ELS can be easily implemented. Extensive theoretical and empirical analyses are provided to support the proposed method.
strength_and_weaknesses: Strength:
- This paper is well-organized and easy to follow. The general framework is pretty clear and follows rigorous logic.
- The theoretical analysis shows many helpful properties of ELS, which are further validated by effective experimental results. To my opinion, this is a very solid paper.
- The ELS method is very easy to implement. The details as well as codes are provided to ensure reproducibility.

Weakness:
- Although this paper is quite solid, I am still concerned about the motivation for introducing label smoothing. Indeed, the domains from the VLCS dataset are similar to each other. However, in other domain adaptation datasets containing significantly different domains (such as the Office-Home dataset), using smoothed domain labels might not improve the performance any further. As shown in the continuously indexed domain adaptation in Table 5, the improvement is 10%. Compared to tables 2 and 4, the improvement of ELS is less than 1%, especially in Table 4, ELS only surpasses the backbone method by 0.3%. So, is it possible that the proposed method is only effective on a dataset that has a small domain gap, and as the domain gap becomes smaller, the improvement would decrease?
- When the encoder gets better, the generated features from different domains are more similar. However, a good encoder is also based on the quality of the discriminator. By using label smoothing, the discriminator can not be as discriminative as vanilla DAT which leverage hard labels, thus the performance of the encoder could doubtful. Is there any analysis of the learned discriminator?
- Moreover, the improvements on the dataset Office-Home can be achieved by tuning the Hyper-parameter $\gamma$, whose optimal value is not provided in the parameter analysis in Figure 4 (b). In my opinion, by tuning $\gamma$, there is always a better classifier than the vanilla DAT.
- In the introduction, the authors are motivated by the observation that VLCS has similar domains, thus label smoothing (smaller $\gamma$) should be introduced, instead of the vanilla DAT ($\gamma$=1). However, in Section 4.2, “PACS can be easily discriminated”, thus the optimal $\gamma$ is smaller than VLCS. Is this a contradiction? Please give an explanation.
- Additionally, the selection of $\gamma$ is unclear to me. Do the authors conduct a grid search in the experiments? How to obtain the mean and std of $\gamma$ to produce Figure 4 (b)?
clarity,_quality,_novelty_and_reproducibility: This paper is great in clarity and quality. The proposed ELS method is simple but the technical contribution is limited. Implementation details and codes are provided to ensure reproducibility.
summary_of_the_review: I have carefully read the whole paper. This paper is quite solid and has excellent completeness, however, there are still some concerns (see weaknesses). If the authors can address my concerns, I will consider raising my score.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary:  
For the problem of out-of-distribution generalization, this paper presents a conceptually very simple yet effective method for domain adversarial training (DAT). Motivated by the intuition that some different domains show small difference, the proposed method utilizes soft domain labels for domain discrimination instead of the standard hard labels, in the form label smoothing. Extensive theoretical and empirical analyses are provided to support the proposed method. The proposed method is shown to boost generalization performance in 5 tasks, including 13 datasets and 10 backbones. Also notably, the authors showcased the advantages of the label smoothing in three different aspects through a combination of theorical and empirical analyses.

Strengths:  
1. The proposed method is surprisingly simple but effective. It can be implemented easily, and codes are provided to ensure reproducibility.
2. Theoretical foundations are given addressing three advantages of the proposed method: (1) training stability, (2) robustness to label noise, and (3) non-asymptotic convergence speed.
3. Extensive empirical evidences support those theories.
4. The paper is written and organized well.

Weaknesses:  
1. At a conceptual level, the method appears to be just an adaptation of the well-known label smoothing technique for DAT, posing the question of its novelty. 

justification_for_why_not_higher_score: As described in detail below, although the AC believes the paper deserves acceptance, its message to the audience is relatively simple: domain label smoothing works for DMT. It is useful for the domain adaptation/generalization community but may be not for the broader audience, so the AC supposes that poster is a suitable venue. 
justification_for_why_not_lower_score: This paper received the initial scores of 5,5,6. While the all reviewers acknowledged the simplicity and effectiveness of the method as well as its theoretical analysis, the main concern was its novelty since it is seemingly a trivial import of the popular label smoothing technique to this task. They also raised concerns regarding some details of the method, analysis and experiments. 
During the discussion period, the authors have presented additional results as well as the clarifications of the method, experiments, and theoretical contributions. The reviewers are mostly convinced by the new explanations and results, and have raised their scores to 6,6,8. (one update (5 to 6) is still not visible in OpenReview). 
Overall, the idea of the proposed method might be seemingly straightforward, but this paper is quite solid and well beyond a ‘just tried and it worked’ paper. It is deeply analyzed in terms of different interesting viewpoints supported by well-founded theories and convincing experiments. The results and findings of the paper will stimulate the domain generalization community that such a simple idea is indeed effective. The reviewers are unanimous for acceptance, and the AC also believes that the pros of the paper sufficiently overweigh the cons, thus recommends acceptance.
Decision: Accept: poster