Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a weight-sharing triple-branch transformer framework, or CDTrans for unsupervised domain adaptation. A two-way center-aware labeling method is proposed to provide better pseudo-labels. SOTA performances were achieved via the proposed method.
main_review: + The paper is well-written and easy to follow.
+ The methods achieve SOTA performances in various UDA benchmarks.
+ It's interesting to see the performances of transformers in UDA.
--------------------
- The source only transformers already achieve good baseline UDA performances which already outperforms some of other previous works. But the authors did not provide insights about why transformers has better generalization ability over ConvNets.
- Although the authors provide some ablation study over different pseudo-label generation methods, it seems it is just compared with the two-way's variants in source/target domain. The authors did not provide comparison with the pseudo-labeling methods as follows:
1. Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation, IJCV 2020
2. Confidence Regularized Self-Training, ICCV 2019
3. Two-phase Pseudo Label Densification for Self-training based Domain Adaptation, ECCV 2020
I suggest the authors to compare their methods to other pseudo-label generation method to see if the proposed pseudo-label generation improves over them.
- Lack a relevant reference of transformer based UDA:
TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation, arxiv. 
(Although it is an arxiv paper, it's also good to include it as it provides better UDA performances on office-31/Home)
summary_of_the_review: I hope the authors could provide some insights/intuition why transformer can generalize well from source to target. And the comparison with other pseudo-labeling approaches is still important to evaluate the contribution.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This submission proposes a transformer framework for unsupervised domain adaptive classification tasks. In this submission, they conduct an exploration about cross attention layer and found that the cross attention layer is robust to pseudo label noise. Inspired by this, they construct a three branches architecture in this submission, which includes a source transformer, target transformer, a source-target transformer. Due to the robustness of the cross attention layer, the source-target transformer acts as a teacher to guide the other two branches. 
main_review: Strengths:
-- an exploration about cross attention layer is conducted and it is discovered that that the cross attention layer is robust to pseudo label noise. 

-- a novel three branches architecture is proposed in this submission, and the source-target branch is utilized as a teacher to guide the other two branches during training. As far as I know, no similar work was proposed before on UDA classification tasks.

-- the experimental results seem promising, which demonstrate the efficacy of this method.

Weakness:
-- I am curious about the memory cost and the model size of the propose method. And inference speed is also expected.
summary_of_the_review: This submission proposes a transformer framework for unsupervised domain adaptive classification tasks. In this submission, they conduct an exploration about cross attention layer and found that the cross attention layer is robust to pseudo label noise. Inspired by this, they construct a three branches architecture in this submission, which includes a source transformer, target transformer, a source-target transformer. Due to the robustness of the cross attention layer, the source-target transformer acts as a teacher to guide the other two branches. 

As far as I know, no similar work was proposed before on UDA classification tasks. This work is of novelty and the experimental results are strong enough to demonstrate the efficacy of the proposed method.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this work, the authors propose a method for domain adaptation by introducing a new way of generating pseudo labels and a cross-transformer with classification and distillation losses. The authors used a cross-transformer where the queries come from the source domain and the values and keys come from the target domain, and try to minimize the output distribution difference between the cross branch and the target branch by a distillation loss. Since the transformers compare source and target in a patch-based manner, the authors find it is more robust to false-positive pairs. The authors do not focus on learning domain-invariant feature representation and mainly solve the domain adaptation problem by pseudo labeling. In four public datasets, the proposed method showed outstanding performance.
main_review: This paper has the following strengths.
- The problem addressed is interesting for the community. The study of pseudo labeling is under-explored compared with domain invariant feature learning.
- The figures describing the different modules are clear and stimulating.
- The method achieves outstanding performance on multiple datasets.

My main concerns are as follows:
- Comparison with other methods is unfair. From the experiments, we can clearly see the superiority of the adopted Transformer-based backbone against ResNet. Baseline-B even surpasses most previous works under the source-only setting. However, most previous works compared (e.g, SHOT, CGDM) use ResNet as the backbone. To better show the effectiveness of the proposed method, it is better to adopt the domain adaptation techniques of other works with the same backbone.
- The technical contribution of this work is limited. There are many works that have used cross transformers in computer vision and NLP tasks. However, not specifically in the UDA research landscape.
summary_of_the_review: Despite the concern on novelty, overall this paper lacks justification of their method to see whether the improvement is brought by the backbone change. The authors should compare other methods when replaced with the same backbone as the proposed method.
review_title: null

Meta-review: This paper makes use of cross-attention transformers to extract invariant features for unsupervised domain adaptation. Combined with pseudo-label approaches, the proposed method achieves state-of-the-art performance, possibly because the transformer features are more robust to the noise. In addition, a two-way centre-aware labeling method is proposed to produce more reliable pseudo labels. 

However, there are some concerns raised by reviewers. After the discussion period, there is still a concern that is not completely unresolved. The comparison with existing methods might not be fair. It is possible that the performance gain is caused by the generally better representation of transformers, which has been shown in supervised image classification.

Overall, the paper is novel and interesting.  I would recommend acceptance of this paper given its impressive performance, but I highly suggest the authors add more ablation studies, for example, compare the proposed transformer and ResNet on a supervised classification task, to further confirm that the performance gain is solely because the transformer is more robust to label noise. The results can be updated in the supplementary. Also, as promised in the discussion, I hope the authors could release their code as soon as possible. This is because the backbone in this paper is totally new, it will be hard for other researchers to achieve SOTA results if they still use CNNs.
Decision: Accept (Poster)