Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The paper proposes use of Grad-CAM to identify positively aligned features for domain adversarial training to improve the downstream classification task in the target domain. 
main_review: I liked the paper overall. The idea is intuitive, it is well explained and compared against related work in literature. The evaluation is thorough in terms of comparisons with baselines, considering multiple scenarios and performing ablations. The visualization of the features are very useful to understand the concepts presented.

A few suggestions to improve the paper:
- Error bars have been shown in the appendix. It would be nice to have a pointer in the main paper noting that the proposed method is statistically better after considering variance across three runs with details in Appendix. 
- The Grad-CAM based methods proposed only applies to vision based models. The way the paper is written, it looks like the proposed solution is more broadly applicable. It will be good to call out this limitation and perhaps propose alternative approaches for other domains.


limitations_and_societal_impact: The broader impact section is left to Appendix instead of main paper
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper address the problem of Unsupervised Domain Adaptation. In particular, the authors investigate if aligning only task-oriented features could help to improve the results. They propose an approach that is mainly based on Grad-CAM. They use Grad-CAM to decompose a holistic feature into a task-discriminative feature and a task-irrelevant feature by softly selecting weights for the features. Later these features are integrated with an adversarial approach for domain adaptation. The authors demonstrate improved performance on several datasets. Additionally, in the supplementary material, the authors demonstrate feature response maps for the baseline and proposed approach on the source and target datasets. The method seems to indeed pick task-relevant features even on the target dataset, which was not necessarily the case for the baseline.
main_review: I find the proposed approach very interesting and original. The proposed approach is rather a heuristic in nature, but it is well-aligned with the line of works focusing on only aligning features that serve classification problems. To me, it looks like a good attempt to distinguish between causal and non-causal features for classification and align only causal ones. There are some works showing that having causal features would serve generalization (e.g. https://arxiv.org/abs/1907.02893), therefore I find this method for domain adaptation interesting. Even this link be more on an intuitive level in this work and demonstrated theoretically, I find experimental results quite convincing. 
limitations_and_societal_impact: The authors clearly discuss limitations in the main and in the supplementary text.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors propose ToAlign, a method which reweights the source features in an adversarial domain adaptation pipeline based on their relevance in predicting the source label. This method could be used in conjunction with any alignment-based domain adaptation approach and has shown the ability to improve performance by focusing alignment on relevant features. The authors show the results of experiments in multiple domain adaptation settings and consistently improve upon baseline performance.
main_review: The main ideas of the work are clearly presented; using ToAlign to reweight source features before aligning source and target distributions is shown to empirically improve performance on several actively explored domain adaptation tasks. The relevance of this proposed approach is demonstrated not just by its performance on challenging benchmarks, but also due to the fact that it can be easily combined with other state-of-the-art domain adaptation approaches to further improve performance. 

While the underlying math and experiments are clearly defined and discussed, elements of the motivation of this work are confusing or don't contribute to a better understanding of the work. Particularly the understanding from the meta-knowledge perspective, detracts from clearly understanding the work's motivations and intuitions. Is the meta-knowledge discussed in this section shared or accumulated across multiple examples or is instance-specific? If instance-specific is meta-knowledge the appropriate mechanism of conceptualizing this information?

While to the best of my knowledge the approach proposed in this work is original, the treatment of prior work leaves much to be desired. The work presents its as one of the only approaches to focus on task oriented alignment, but ignores several works on disentangled representations for domain adaptation which attempt the same goal. Positioning the works improvements and motivations solely on the focusing on task-oriented features and ignoring how it relates to other work which attempt the same goal may lead to inaccurate takeaways from the paper and is a substantial concern with the quality of the work. 
(Domain Agnostic Learning with Disentangled Representation - Peng et al)
(Learning Disentangled Semantic Representation for Domain Adaptation - Cai et al)
(DiDA: Disentangled Synthesis for Domain Adaptation - Cao et al)
limitations_and_societal_impact: The authors don't discussion potential negative societal impacts of their work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposed a method "To-Align" for alignment of the domains in subspaces that have relevant information embedded for a given task. For the classification task, the paper demonstrates that using Grad-CAM for obtaining subspaces is effective. Results on the single source and multi-source benchmarks show consistent improvements in the performance of UDA.

The motivation behind the usage of attention for domain adaptation has been well studied in the existing literature. To fully understand the benefits of the proposed method, more experimental studies would be necessary. However, in the current form, the paper is marginally acceptable.
main_review: The paper is of good quality and clarity. However, it is not up to the mark with regard to originality and significance (idea & results). Please see the below full review below.

1. Using attention for DA has already been explored in many papers [Ref 1,2]. The goal of this line of research is to identify the interesting aspects in the image for the given task. There have also been several methods that consider feature level manipulation (such as negative and positive features) for DA [Ref 3]. It is not clear why the proposed "ToAlign" performs better than any other attention-based works as the paper does not compare against them. 

2. Recent work on MUDA at NeurIPS [Ref 4] showed remarkable performance on DomainNet while being simpler than existing MUDA methods. Please comment on the performance of your method against this work. 

3. The experimental setup: The current set of experiments demonstrate improvement over a couple of baselines. How would the proposed "ToAlign" framework help with respect to other methods? Can any method exploit the benefits of "ToAlign"? What are the limitations? I would expect that 'ToAlign' will not improve over [Ref 1,2].

4. The method demonstrates only a marginal overall improvement (0.7%) on the SSDA dataset. However, the method is consistently improving the baseline as shown in Table  1 and Table 2. 

5. Are authors planning to release a reference implementation? 

6. Minor comments
 - L193:: Classifier C and #channels C have the same notation. 

-----
Please cite these related work and mention how these methods are different from yours:-

[Ref 1] Vinod Kumar Kurmi, Shanu Kumar, Vinay P. Namboodiri, "Attending to Discriminative Certainty for Domain Adaptation," CVPR'19

[Ref 2] Guoliang Kang, Liang Zheng, Yan Yan, and Yi Yang, "Deep Adversarial Attention Alignment for Unsupervised Domain Adaptation: the Benefit of Target Expectation Maximization", ECCV'18

[Ref 3] Jogendra Nath Kundu, Naveen Venkat, Ambareesh Revanur, Rahul M V, R. Venkatesh Babu, "Towards Inheritable Models for Open-Set Domain Adaptation" CVPR'20

[Ref 4] Naveen Venkat, Jogendra Nath Kundu, Durgesh Kumar Singh, Ambareesh Revanur, Venkatesh Babu R., "Your Classifier can Secretly Suffice Multi-Source Domain Adaptation", NeurIPS'20
limitations_and_societal_impact: Limitations:

* The authors point out that this approach works only with in domain adversarial settings and not in a pseudo-labelling settings. The * authors are encouraged to inlude limitations w.r.t to the usage of Grad-CAM in the feature selection process and how it may/may not improve other baselines.

Societal Impact:
* The authors should comment on any negative bias carried over by the Grad-CAM based feature selection.
* Does "ToAlign" ensure fairness if the source dataset consists of imbalanced classes? 
* Would "ToAlign" carry over source dataset bias in medical applications such as tumor segmentation?
review_title: null

Meta-review: This paper uses Grad-CAM to reweigh the latent features in order to align the source and target domains only with respect to task-oriented features. The method is simple and effective. The idea is clearly motivated in the context of relevant literature and the experimental results clearly demonstrate the effectiveness of the method. All reviewers, including myself, find the paper an interesting and solid contribution that is worth publication.
Decision: Accept (Poster)