Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper measures predictive uncertainty on class assignments and the bounding box predictions. The uncertainty is further used to guide the self-training and adversarial learning for domain adaptive object detection. The experiments cover various domain shift scenarios and prove the approach improves upon existing state-of-the-art methods. 
main_review: Overall, this paper is easy to follow. The experiments validate the effectiveness of the method. 

1. However, my major concern about this paper is its novelty. Using uncertainty estimation to facilitate self-training and adversarial learning is a common operation in domain adaptation. The unique component of this paper is the way to measure uncertainty. As this is the core component of the method and the modifications are incremental, I hope to see more intuitions or conclusions that can help us guide the design of uncertainty estimation. More analysis about various uncertainty estimations are demanded to support the paper. While this paper only conducts a simple ablation study, which is not sufficient for the acceptance of the paper. 

2. As the paper also talks about the complementary between self-training (A) and adversarial learning (B), is it possible to observe more improvements with A + B than independently using the two components? (A bring ) What’s the difference between this synergy with [22]?

Joint considering the effective experiments and the limited novelty, I give my ranking as borderline reject. I am open to hearing from the authors to clear the main contributions of their paper vs. related works. 

Detailed comments:

There is only a Section 3.2.1 upon Section 3.2 which is not proper. 

limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: Authors address the issue of domain adaptation for object detection, to detect objects in images with same label space but drawn from different data distributions. To this end, the paper proposes to mine uncertain tiles including object-like regions for adversarial feature alignment, and mine certain pseudo-labels for self training. Authors argue that the uncertainty guided training strategy can lead to better calibration under domain shift and improve the model's generatlization. Experiments are carried out on street driving datasets and demonstrate the effectiveness of the proposed method.
main_review: Mining pseudo-labels with low uncertainty for training makes sense to me and should lead to more robust training on the target domain images. However, it's not very clear why it would be optimal to select tiles with uncertain detections. It would be better to provide more experiments about the adversarial alignment part, especially about different sample selection strategy (random selection, keeping all detection outputs, etc).

In Table 1 and Table 2, although those two-stage models should have more powerful detection performance, they generally are worse in the source only settings. It may suggest that those baselines are not well-tuned or powerful enough. It would be interesting to see if the proposed method can also be applied to two-stage methods and provide consistent performance gains.
limitations_and_societal_impact: Authors have adequately addressed the limitations and potential negative societal impact of their work, or there are no obvious issues regarding that direction.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary:  In this paper, authors propose an unsupervised domain adaptation method for object detection based on pseudo-labeling and adversarial training. The main idea is two-fold: 1) Monte-Carlo dropout is used to produce the detection uncertainty information that is used to generate pseudo-labels; and 2) regions surrounding uncertain detections are used for adversarial training against the regions from the source domain surrounding the ground truth bounding boxes. The proposed method is compared with the state-of-the-art single-stage and two-stage object detectors on Cityscapes-to-Foggy Cityscapes, KITTI-to-Cityscapes and Sim10K-to-Cityscapes datasets. Additionally, an ablation study examining the contribution of each of the proposed modules to the increased detection precision is presented. 

main_review: The paper is overall well-written, except the abstract which is a little bit hard to read.

The proposed idea of using spatial MC dropout to predict the detection and localization uncertainty is sufficiently novel and clever.  It is also intuitive to use the certain predictions as pseudo-labels. The only issue that I see is that the uncertain detections are used for adversarial training, as if they actually always correspond to the foreground objects. It is very much possible that some of the uncertain predictions are actually the false positives, and it is not clear to me how adversarially matching them to the source foreground objects would benefit the detection accuracy. 

The experimental design and the choice of baseline methods as well as an ablation study provide conclusive and convincing evidence that both of the proposed methods lead to increased detection precision of medium and large objects. 

While Figure 3 clearly illustrates that both the uncertainty and calibration error are minimized by the proposed solutions, the message behind the Figure 4 is unclear to me. It is difficult to understand what the authors were trying to show with these histograms, so perhaps a more detailed explanation in the caption with a brief conclusion would make it easier to comprehend. 

limitations_and_societal_impact: The limitations regarding the decreased detection accuracy of the small objects is discussed in the paper. However, the proposed method involves multiple stages of training and multiple additional modules, which is why a discussion of how much additional memory / trainable parameters / more training time is needed for this method compared to the others is needed.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: The work proposes a UDA pipeline for object detection based on self-training (via pseudo-labeling) and adversarial alignment. To this end, Monte-Carlo dropout inference is used to estimate detection confidence (or uncertainty). Certain detections (with  associated pseud-labels) are used for self-training while  uncertain ones are used to extract tile for alignment.


main_review: 1) The paper reads well and presents simple concepts, it is thus easy to understand. However, while the motivation behind using detections with low uncertainty for self-training is very intuitive, I could not grasp the reason behind exploiting low-confidence detections for adversarial aligment. I would accept it being verified experimentally, but no ablation in this direction is provided.

2) Novelty is quite limited, since all techniques employed are well known (MC- dropout, adversarial alignment, self-training via pseudo-labeling. Putting them together is somehow novel, but as I mention above, the motivation is unclear and it has not been proven nor verified experimentally.

3) Introduction, line 61: building over [47] cannot be claimed as a contribution

4) Related work, second paragraph can be omitted. Tiling is only marginally related to the work.

5) UGT paragraph (lines 202-211): here the authors try to motivate uncertainty-guided tiling as "Tiling anchored by uncertain object regions allows adversarial alignment to focus on potential, however, uncertain object-like region with context". Why not concentrating on those with high confidence or simply all of the detections. Since not motivation is brought forward, I would have expected an ablation study of this.

6) Ablation (especially table 3. and related paragraph - lines 268-280). Beside the missing ablation mentioned above, warmup seems to account for 2/3 of the total improvement over the baseline for mean AP (Baseline: 25.9, Ours w/o warmup: + 0.9%, Ours (UGPL + UGT)  + 2.9% )

7) Impact of object size (lines 281-284): no discussion on the (relatively) poor results for Small objects. Also you may want to discuss the reasons why warmup is detrimental for S(mall), almost irrelevant for M(edium) and essential for L(arge) objects.

Based on the points above, my overall score is below average. The missing ablation  (point 5) raises doubts about the soundness of the method, especially since there is no clear motivation for using only low confidence detections for adversarial alignment and being that one of the main claimed contributions.



limitations_and_societal_impact: I would argue that no discussion is needed on this topic. I cannot see any evident negative impact in adapting an object detector to a new domain.
review_title: null

Meta-review: Following the rebuttal and discussion phase two reviewers increased their scores leading to final recommendations of one clear accept, two leaning towards accept and one rejection rating. After considering all reviews and discussion between authors and reviewers, the AC agrees with the reviewer consensus that this work merits acceptance. The key concern prior to the rebuttal was whether the combination of prior approaches (such as MC-dropout, adversarial alignment, and pseudo-label training) were sufficiently new. The rebuttal provided sufficient clarification on this point, especially through new ablation studies. The authors should make sure to include these ablation studies as well as additional text clarifying the novelty of their contribution in the final version.
Decision: Accept (Poster)