Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes an effective Deliberated Domain Bridging (DDB) for domain adaptive semantic segmentation (DASS). To this end, it takes advantage of two data mixing techniques, region-level mix and class-level mix, to train two corresponding teacher models, which eventually guide one student model on the target domain. It has been tested on several benchmarks (GTA5 to Cityscapes, GTA5 + Synscapes to Cityscapes, GTA5 to Cityscapes + Mapillary).

strengths_and_weaknesses: - Strengths:
1. It is a well-written paper that addresses the limitations of previous methods (e.g., global interpolation -> pixel-wise ambiguity), toy game to show the justification of using both coarse-grained and fine-grained DB, and proposes novel learning architecture with multi-teacher and single-source distillation method.
2. The proposed method is well proven to be effective with several benchmarks (e.g., single-source, multi-source, and multi-target settings) by widening the gap with the previous state of the arts in each benchmark.

- Weaknesses:
1. As the author illustrated in limitations, I am also a bit concerned with the training efficiency and complexity since the proposed method requires alternating optimization processes. One of the simple end-to-end optimizations is conducting EMA training on one model by combining region-level and class-level mixing techniques. It would be better to show a brief study about how the authors can extend this to end-to-end simple learning architecture.
2. Lack of ablation study on some hyperparameters: 1) alpha in equation 5: the authors suggested updating the teacher models with EMA to avoid a denoised pixel-wise pseudo label. To prove it, an ablation study on alpha needs to be explored. 2) x_aug in equation 9: need to empirically show justification using augmentation input for a student model.    
questions: Please answer the questions from the above weaknesses.
limitations: Yes, the authors addressed the limitation.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: To address UDA in semantic segmentation, this work uses two types of data mixing strategies to artificially create intermediate bridging domains between source and target. The paper starts with a detailed analysis comparing different data mixing strategies, either done globally (mixup[61]) or locally ( CowMix [13], FMix [17], CutMix [60] and ClassMix [42]). The analysis demonstrates favorable results when using local data mixing strategies for UDA in segmentation, in particular CutMix (coarse region-wise mixing) and ClassMix (fine class-wise mixing). 

Based on results of the analysis, this work proposes a simple way to combine the two mixing strategies CutMix and ClassMix. 
In the course of training, there are five models: two teacher models trained with CutMix and ClassMix, two EMA models of the two teachers, one student model trained using teachers' pseudo-labels.

Training is done in multiple rounds (fixed as 4 in the experiments). In each round:
- The two teachers are first trained separately with CutMix and ClassMix
- The student is then trained with pseudo-labels of two EMA models of the two teachers. Pseudo-label of a given target sample is determined as a weighted combination of softmax scores of the two EMA models (Eqn. 12). The weights have size $H \times W \ K$ with $K$ classes; at each spatial position, the weight vector over $K$ classes is the softmax over the feature distance to class centroids (Eqn. 11). Color jittering and gaussian blurs are used on target sample when training the student.
- The two teachers are initialized by the student.


strengths_and_weaknesses: ** Strengths **
Overall this is an interesting technical paper that combines multiple existing strategies, namely CutMix [60], ClassMix [42], mean teacher [42], prototypical weighting [62]  and pseudo-labelling [30]. Empirical results demonstrate better performance than previous SOTAs on comparable backbone (resnet101) and segmentation framework (deeplab-v2). Experiments are extensive. The paper is well-written and easy to follow.

** Weaknesses **
- My main concern is with the technical novelties of this work. The analysis comparing different mixing techniques, claimed as the first contribution, is somewhat interesting. However the main proposed approach is merely a mix of previous works. Actually there are no new insights that I could get from this work. 

- It's not clear to me how is the intermediate model selected at each stage. Is the target's validation set used to select the best model? If true, is there a risk of supervision leak from target validation set?

- Missing details for the multi-source and multi-target experiments.

I'm currently on the borderline, slightly leaning toward the positive side, thanks to the good results. My final decision will be adjusted based on the feedback from the authors and the discussion with other reviewers.

** Typos **
- L185: Eqn. 3 instead of Eqn. 2
- SuppMat: Algo.1 - L7 & L12: Eqn. 3 instead of Eqn. 2

========== Post-rebuttal
I thank the authors for being active during the rebuttal and addressing all of my concerns. I'm happy to increase my score.
questions: - DAFormer (CVPR'22) is a recent work demonstrating great DA performance thanks to the robustness of transformer-based Segformer model. As the proposed strategy is agnostic to the model choice, I'm wondering to which extent one can push the performance by combining the proposed and DAFormer
limitations: Limitation on training complexity is discussed in the supplementary material.
No discussion on potential negative societal impacts was given.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a deliberated domain bridging (DDB) method for domain adaptative semantic segmentation, where the target labels are not available during the training. In DDB, there are two parts: 1) a dual-path domain bridging step to train two teacher models with two intermediated domains using the coarse-wise and fine-wise, i.e., region-level and semantic-level, data mixing techniques. 2) a cross-path knowledge distillation step to adaptively transfer the knowledge from the two teacher models to a student model. The two steps are repeated for several rounds for a good performance. Extensive experiments on both single-source domain and multi-source multi-target domain settings are conducted to validate DDB’s superiority.
strengths_and_weaknesses: Pros:
1. This paper proposes an effective method to significantly boost the UDA segmentation performance in various settings.
2. The comprehensive ablations are done to clearly show 1) the complementarity between the two teacher models and 2) the effectiveness of the distillation step.

Cons:
1. Since GTA5 to Cityscapes and GTA5 + Synscapes to Cityscapes are done, what is the performance in Synscapes to Cityscapes? This experiment shows which dataset contributes more to adapt to the real dataset.
2. There are too many symbols, which makes the paper hard to follow. What do the numbers righter after the approach name in Tables 2, 3, and 4 mean? For example, ADVENT(19), BDL (19), FADA(20), etc.
3. The authors claim that soft distillation and hard distillation are compared in Table 5. However, the ‘soft distillation’ choice and the explanation are missing in that table, which is a bit confusing.
4. DDB requires two rounds for a good convergence. In each round, it needs to train three individual models and calculate two groups of category centroids by scanning the target training set for two teacher models respectively. This makes the approach cumbersome and may require more training time than others. The authors are encouraged to discuss the above issue with detailed analysis. Besides, this cumbersome training process seems in conflict with the stated ‘elegant’ method.
5. The following paper can be included for comparison since it also studies the data mixing technique in UDA semantic segmentation. Besides, the difference between DACS which also utilizes the data mixing technique in UDA is not well stated in the paper.

Dsp: Dual soft-paste for unsupervised domain adaptive semantic segmentation. Proceedings of the 29th ACM International Conference on Multimedia. 2021: 2825-2833.

questions: See the weakness.
limitations: The authors discuss the limitations of this paper in the supplementary material. No negative social impact has been discussed.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper is about unsupervised domain adaptation for the task of semantic segmentation. The paper argues for the importance of gradually bridging the domain gap, instead of a direct attempt to transfer a model from the source to the target domain. Motivated by an empirical analysis about different data mixing technologies, the paper explores two region-based mixing strategies (coarse regions and finer class-wise/mask regions) as domain bridges. Specifically, two models are trained on the two different domain bridges (Dual-Path-Domain-Bridge), which act as ensembled supervision for a single student model (Cross-path knowledge distillation). This student model can then initialize the teacher models for another round of these two steps. Experimental results in three different settings confirm the effectiveness of the proposed approach with state-of-the-art results on standard benchmarks.
strengths_and_weaknesses: ### Strengths
- The ablation studies in Tables 5-7 are great and showcase the impact of individual components
- The results are impressive, with a clear improvement on the standard benchmark (GTA5->CityScapes), as well as other settings (multi-source and multi-target)
- The experiments in Table 1 are a good motivation for the choice of data mixing strategies (CutMix and ClassMix)



### Weaknesses

- The writing, specifically the motivation and positioning with respect to prior work, needs improvement.
  - There may be alternative domain bridges than data mixing, like methods that rely on self-training and choose confident target pseudo labels as intermediate source domain.
  - The justification for exploring new domain bridges in lines 37-41 is vague and unclear: What are "unexpected artifacts in the global input space"? What "optimization constraints" are referred to?
- I do not see why the paper only evaluates two domain bridging strategies in the ensemble. One could also include more. Relating to ensemble methods, one could expect improvements if an additional data mixing strategy is "orthogonal". One recent successful example for a global mixing strategy is [A] and could be easily integrated.
- There is a related work on domain bridges for semantic segmentation that was not included: [B]
- In line 185, shouldn't the reference go to Eq. 3?
- I do not quite understand why the mixing weights in Eq. 11/12 help. Aren't the softmax values (i.e., scores) already an indication how far away a sample is from the decision boundary?
- It would be good to point the reader to the supplemental material for a detailed description of the training strategy.
- It's hard to understand and see details in Figure 1.


**References:**
- [A] FDA: Fourier Domain Adaptation for Semantic Segmentation. Yang and Soatto. CVPR'20
- [B] Domain Bridge for Unpaired Image-to-Image Translation and Unsupervised Domain Adaptation. Pizzati et al. WACV'20
questions: Please see few questions under "weaknesses" above
limitations: Potential societal impacts are not discussed. I think a paper on domain adaptation should include a discussion about potential biases that are carried over from source domains (specifically because these are often synthetic data which often contains some hand-crafted components, like object sampling distributions, etc.).
review_title: null

Meta-review: metareview: **Summary**: This paper proposes an effective Deliberated Domain Bridging (DDB) approach for domain adaptive semantic segmentation (DASS). It leverages two data mixing techniques: region-level mix and class-level mix, to train two corresponding teacher models, which then guide one student model on the target domain. It is evaluated on multiple benchmarks.

**Strength**: The paper is a well-written paper. It is well-motivated based on the limitations of previous methods. The proposed approach is novel, interesting, and effective. The experiments (with the toy game) are solid.

**Weakness**: Training efficiency and complexity. Lack of ablation study on some hyperparameters and design choices. Some missing references/comparisons; unclear positioning of the work w.r.t. prior work.

**Recommendation**: The paper receives consistently positive ratings. After rebuttal, most of the reviewers’ concerns are addressed and the paper clearly has strengths. The AC thus suggests acceptance. The AC strongly suggests that the authors incorporate their rebuttal (e.g., additional results) into their camera-ready version.

confidence: Certain
Decision: Accept