SSAL: Synergizing between Self-Training and Adversarial Learning for Domain Adaptive Object Detection
Authors: Muhammad Akhtar Munir
Muhammad Haris Khan
M. Saquib Sarfraz
Mohsen Ali
Venue: NeurIPS 2021 Poster (2021)
Keywords: domain adaptation
uncertainty
deep learning

Abstract: We study adapting trained object detectors to unseen domains manifesting significant variations of object appearance, viewpoints and backgrounds. Most current methods align domains by either using image or instance-level feature alignment in an adversarial fashion. This often suffers due to the presence of unwanted background and as such lacks class-specific alignment. A common remedy to promote class-level alignment is to use high confidence predictions on the unlabelled domain as pseudo labels. These high confidence predictions are often fallacious since the model is poorly calibrated under domain shift. In this paper, we propose to leverage model’s predictive uncertainty to strike the right balance between adversarial feature alignment and class-level alignment. Specifically, we measure predictive uncertainty on class assignments and the bounding box predictions. Model predictions with low uncertainty are used to generate pseudo-labels for self-supervision, whereas the ones with higher uncertainty are used to generate tiles for an adversarial feature alignment stage. This synergy between tiling around the uncertain object regions and generating pseudo-labels from highly certain object regions allows us to capture both the image and instance level context during the model adaptation stage. We perform extensive experiments covering various domain shift scenarios. Our approach improves upon existing state-of-the-art methods with visible margins.

SSAL: Synergizing between Self-Training and Adversarial Learning for Domain Adaptive Object Detection
Muhammad Akhtar Munir Muhammad Haris Khan", M. Saquib Sarfraz"., Mohsen Ali
(akhtar www.mmniiiiee tu wed.ituuuuuuu.. edu-pk) (muhammad. hari s@mbzuai. ac. (muhammad. stw.razzk..a..a.a..a.aaaa.. .dd) (mchsen. a aliiitu.edu....................u
Information Technology University of Punjab, Mohamed bin Zayed University of Artificial Intelligence, Karlsruhe Institute Technology. Daimler TSS
Abstract
We study adapting trained object detectors to unseen domains manifesting signifi- cant variations of object appearance, viewpoints and backgrounds. Most current methods align domains by either using image or instance-level feature alignment in an adversarial fashion. This often suffers due to the presence of unwanted background and as such lacks class-specific alignment. common remedy to pro- mote class-level alignment is to use high confidence predictions on the unlabelled domain as pseudo labels. These high confidence predictions are often fallacious since the model is poorly calibrated under domain shift. In this paper, we propose to leverage model's predictive uncertainty to strike the right balance between ad- versarial feature alignment and class-level alignment. Specifically, we measure predictive uncertainty on class assignments and the bounding box predictions. Model predictions with low uncertainty are used generate pseudo-labels for self-supervission, whereas the ones with higher uncertainty are used to generate tiles for adversarial feature alignment stage. This synergy between tiling around the uncertain object regions and generating pseudo-labels from highly certain object regions allows us capture both the image and instance level context during the model adaptation stage. We perform extensive experiments covering various do- main shift scenarios. Our approach improves upon existing sfte-offtheearr methods with visible margins.
Introduction
Deep convolutional neural network based object detectors have shown promising results, through learning representative features from large annotated datasets 77, 32 I0]. However, like other supervised deep learning methods, object detection methods trained on the source domain do not generalize adequately to new target domain. This problem, known as domain shift [49 could be exhibited by change in style, camera pose, or object size and orientation, or the number or location of objects in the scene, among other things. Often, collecting large annotated dataset for fine-tuning the model to the target domain is expensive, error prone and in many cases not possible. Unsupervised Domain Adaptation (UDA) is a promising research direction towards solving this problem by transferring knowledge from a labelled source domain to an unlabelled target domain.
*Corresponding author, Intelligent Machines Lab, Department of Computer Science, Information Tech- nology University of the Punjab, Lahore, Pakistan. Email: akhtar w.htarr.uuiir....... cdu. pk Project Page: http://im. htt./mm.ito/ itu.uuue edu. wed..p//..... pk/synergi z zing-doma i in-maattttiinoo
35th Conference on Neural Information Processing Systems (NcurIPS 2021)

---

Many unsupervised domain adaptive detectors rely on adversarial adaptation or self-training tech- niques. Methods based on adversarial adaptation 4 discriminator for aligning features at image or instance levels. However, due to the absence of labels in target domain they suffer from the challenges of how to pick samples for the adaptation. Selecting uniformly, one ends up missing on infrequent classes or instances. Most importantly adversarial alignment do not explicitly incorporates the class discriminative information, resulting in non-optimal alignment for classification and object detection tasks [43 45) A potential solution to this problem is self-training based adaptation, however, it faces the challenge of how to avoid noisy pseudo-labels. Some methods choose high confidence predictions pseudo-labels (27 19 42 but the likely poor calibration of model under domain shift renders this solution inefficient [8]. Further, in the case of object detection, prediction probability not directly capture object localization inaccuracies.
[4 50
3 36], mostly rely on domain
We present a principled approach, dubbed as SSAL (Synergizing between Self-Training and Adversar- ial Learning for Domain Adaptive Object Detection), to achieve right balance between self-training and adversarial alignment for adaptive object detection via leveraging model's predictive uncertainty. To estimate predictive uncertainty of detection, we propose taking into account variations in both the localization prediction and confidence prediction across Monte-Carke dropout inferences [8]. Certain detections are taken as pseudo-labels for self-training, while uncertain ones are used to extract tiles (regions in image) for adversarial feature alignment. This synergy between adversarial alignment via tiling around the uncertain object regions and self self-training with pseudo-labet from certain object regions lets us include instance-lewe context for effective adversarial alignment and improve feature discriminability for class-specific alignment. Since we select pscudo-labels with low uncertainty and take relatively uncertain as potential, object-like regions with context (i.e. tiles) for adversarial alignment, we tend to reduce the effect of poor calibration under domain shift, thereby improving model's ' generalization across domains.
Our key contributions include the following: (I) We introduce a new uncertainty-guider framework that strikes the right balance between self-training and adversarial feature alignment for adapting object detection methods. Both pseudo-labelling for self-training and tiling for adversarial alignment are impactful due to their simplicity, generality and ease of implementation. (2) We propose a method for estimating the object detection uncertainty via taking into account variations in both the localization prediction and confidence prediction across Monte-Carlo dropout inferences. (3) We show that, selecting pseudo-labels with low uncertainty and using relatively uncertain regions for adversarial alignment, it is possible to address the poor calibration caused by domain shift, and hence improve model's generalization across domains. (4) Unlike most of the previous methods, we build on computationally efficient one-stage anchor-less object detectors and achieve state-of-the-at results with notable margins across various adaptation scenarios.
2 Related Work
Object detection. Deep learning based object detection algorithms can be classified into either anchor-based 40,
46 2 or anchor-free methods 126 6 47). Anchor-based methods, such as Faster RCNN [40] uses region proposal network (RPN) to generate proposals. Anchor-fice free detectors, on the other hand, skip proposal generation step and through leveraging fully convolutional network (FCN) [33 directly localize objects. For instance, [47] proposed per-pixel prediction and directly predicted the class and offset of the corresponding object at each location the feature map. In this work, we capitalize on the computationally inexpensive characteristic in anchor-free detectors to study adapting trained object detectors.
30,
Tiling for object detection. The process of cropping regions of an input image, a.k.a tiling, in a uniform [39], random, informed [51] 16, 29 fashion before applying object detection is typically used to tackle scale variation problem and improve detection accuracy over small objects. Informed tiling can be achieved by first generating set of regions of object clusters, and then cropping them
for subsequent fine detection [1].
The pioneering work of 4 on domain- domain-adaptive (DA) object detection proposed reducing domain shift at both image and instance levels via embedding adversarial feature adaptation into anchor- based detection pipeline. Global feature alignment could suffer as domains may manifest distinct scene layouts and complex object combinations. Several subsequent approaches attempted to achieve right balance between the global and instance-level alignments
Domain-adaptive object detection.

---

Source (5)
Training
Adversarial Leaminn
Lnter
UGPL Flow UGT flo
Fealure Extractor
Detection Head
Ldet
Training
by!
Uncertainty Guided Tiles (UGT)
Uncertainty Guided Pseudo Labels (UGPL)
Target (T)
MC
Dropout
Feature
Detaction Head
Extractor
N limes Inference
Overall architecture of our method (SSAL). Fundamentally, it is one-stage detector 47 with an adversarial feature alignment stage. We propose uncertainty-gut pscudo-labels (UGPL) and umcertainty-guidee adversarial alignment via tiling (UGT) (in dotted boxes). UGPL produces accurate pseudo-labels in target image which are used tandem with ground-truth labels in source image for training. UGT extracts tiles around possibly object-like regions in target image which are used with randomly extracted tiles around ground-trun -tuth labels in source domain for adversarial feature alignment.
Figure 1:
self training with
55 [50] Other methods [5 23
18
improved feature alignment in various ways e.g., through
exploiting hierarchical feature learning in CNNs (15). While above methods are built on two-stage pipeline, a few approaches have built domain adaptive detectors one-stage pipeline 22 17 17 proposed to predict pixel-wise objectness and center-aware feature alignment, building on to focus on the discriminative parts of objects.
Uncertainty for DA object detection. Exploiting model's predictive uncertainty and entropy opti- mization have remained subject of interest in prior cross-domain recognition [34] [4
35
4]
and
detection 212 37
works. For cross-domain recognition, [41] employed uncertainty for filtering
training data and aligning features in Euclidean space. For DA object detection, [12 proposed an uncertainty metric to regulate the strength adversarial learning for well-aligned and poorly-aligned samples adaptively.
Pscudo-labelling for DA object detection. DA object detection, pseudo-labelling aims at ac- quiring pseudo instance-level annotations for incorporating discriminative information. Inoue et al. [19 generated pseudo instance-level annotations by choosing the top-1 confidence detections. Sim- ilarly, 42 obtained the same using high- confidendee detections and further refined them using tracker's output. Towards refining (noisy) pseudo instance-level annotations, [1] employed auxiliary component and [22] devised criterion based supporting Rols.
Confidence- based pseudo-labe selection is prone to generating noisy labels since the model is poorly calibrated under domain shift, eventually causing degenerate network re-training.
Unlike most prior methods build computationally inexpensive one-stage anchor-free detector. Different to existing methods, we leverage model's 's predictive uncertainty, considering variations in localization and confidence predictions across MC simulations, to achieve the best of both self- training and adversarial alignment through mining highly certain target detections as pseudo-labels and relatively uncertain ones guides in the tiling process.
3 Proposed Method
In this section, we describe the technical details of our method. Fig. displays the overall architecture of our method. We propose leverage model's predictive uncertainty to strike the right balance between adversarial feature alignment and self-training. To this end, we introduce uncertainty -uidde pseudo-labels selection (UGPL) for self-training and uncertainty-guica alignment. The former allows generating accurate pseudo-labels to improve feature discriminability for class-specific alignment, while the latter enables extracting tiles on uncertain, object-like regions for effective domain alignment.
tiling (UGT) for adversarial

---

3.1 Preliminaries
Problem Setting. Let Let = (... be the labeled source dataset and D; = {aل1}= be the unlabeled target dataset. Where y = (bf, cf) set of bounding boxes by for the objects in the image 2 and their corresponding classes cf E (1..... C). The source and target domains share an identical label space, however, violate the i.i.d. assumption since they are sampled from different data distributions. Our goal is to learn domain-adaptive object detector, given labeled D, and unlabeled Dr capable of performing accurately in the target domain.
One-stage anchor- free object detection. Owing to the computationally inexpensive feature of one-stage anchor-free detection pipelines, we build our uncertainty-guie domain-adaplin detector on fully convolutional one-stage object detector (FCOS) [47]. Inspired from the fully convolutional architecture [33, FCOS incorporates per-pixel predictions and directly regresses object location. Specifically, it outputs C-dimensional classification vector, a 4D vector of bounding box coordinates, and centerness score. The loss function for training FCOS is:
1 Σ Coue Cu,o) + Npos Σ Lou (Bu,ey bu,.) Ldee (der bu,e)
1
N Npos
where Let is the classification loss (i.e. focal loss [1], and Lbow (i.e. IoU loss [52) is the regression loss. Cu.v. bu.e denotes class and bounding box predictions at location (u, v). Npos denotes the number positive samples.
Adversarial feature alignment. Several methods 43 align feature maps on the image-level to reduce domain shift via adversarial learning. It involves global discriminator Dade that identifies whether the pixels on each feature map belong to the source or the target domain. Specifically, let F E R #x XW be the K-dimensional feature map of spatial resolution H W extracted from the feature backbone network. The output of Dadu is a domain classification map of the same size as F. The discriminator can be optimized using binary cross-entropy loss:
g Eglo((Dass (F"" (u,)), +(1 q) log(1 (aat(F")us)) Lado (1" ,xx)x :x'==-
where g is the domain label E [0, 1]. We perform adversarial feature alignment by applying gradient reversal layer (GRL) 19 to source F and target EL feature maps, in which the sign of gradient is flipped when optimizing the feature extractor via GRL layer. Global alignment is prone to focusing on (unwanted) background pixels. We introduce uncertainty-guide tiling. that involves cropping tiles (regions with context) around object-like regions for effective adversarial alignment (sec. (32.1].
Self- Self-Training Self-training is process of training with pseudo-labels, which are generated for unlabelled samples in the target domain with a model trained on labelled data. Hard pseudo instance- level labels are obtained directly from network class predictions. Let Pj., be the probability outputs vector of trained network corresponding to detection y,,c. such that p5,, denotes the probability of class being present in the detection. With these probabilities, the pseudo-label can be generated for F as: Vi, = 1 1 2 a). where max.Pī, k- There could be a significant fraction of incorrectly pseudo-labelled detections used during training. A common strategy to reduce noise during training is to select pseudo-labels corresponding high-confidence detections [19 42 Let Ij.k be a boolean variable denoting the selection or rejection of Uj i.e. where Ii = 1 when Øj. selected otherwise. Formally, in confidence-baset selection, a pseudo-label Øj. is selected as: 9y.k [v 2 T]. where T is the confidence threshold. These high confidence detections are often noisy because the model is poorly calibrated under domain shift. Instead, we propose to select pseudo-labels utilizing uncertainty in both class prediction and localization prediction to mitigate the impact of poor network calibration (sec. (.2.1).
3.2
Uncertainty for Domain Adaptive Object Detection
The source model demonstrates poor calibration under target domain bearing sufficiently different superficial statisties and different object combinations 38 45). Although confidence-basee selection (typically highest confidence) improves accuracy, the poor calibration of the model under domain shift

---

PL
PL
Pseudo Labels
Uncertainty Guided Pseudo Labels Tiling
Regions for Tillng
Tiles with Connext Cropped
Figure 2:
illustration on which detections will be considered pseudo- labels and which for extracting tiles. More certain detections, such as pedestrians are taken as pseudo-label,, whereas relatively uncertain ones, like cars under fog, are used for extracting tiles.
makes this strategy inefficient. As a result, it could lead to both poor pseudo-labelling accuracy and incorrect identification of possibly object-like regions for adversarial alignment. Since calibration can be considered as the model's overall prediction uncertainty [25], we believe that through leveraging model's predictive uncertainty we can negate the poor effects of calibration. To this end, we propose to leverage uncertainty in detections to select pseudo-labels for self-training and choose regions for tiling in adversarial alignment.
Uncertainty in object detections. Assuming one stage detector, we perform the uncertainty estima- tion by applying Monte-Carlo dropout [8] (in particular, spatial dropout (48) the convolutional filters after the feature extraction layer. Given an image æ, we perform N stochastic forward passes (inferences) using MC dropout. Let nn.m = (bn,m, C,,m be the meh detection in inference, Cr. be the class label with highest probability Pn.m in the probability vector Pn.m. and bn.m F= is the predicted bounding box. We aim to capture the variations in both the localization prediction and confidence prediction across inferences. To this end, we define the uncertainty the object detection prediction mean class probability of the overlapping bounding boxes across individual inferences.
Specifically, for each Fn,m, create set T by including all ÿs.1, where * and is an arbitrary detection in hig MC forward pass, such that bn.m has IoU with b greater than a specific threshold and E,,m Ck.I.
T T Munnnn (br.1: Gr,), ToU(b,,, b..)) > y, Ō..1 = 2n m }
Where y is the IoU threshold to identify bounding boxes occupying same region (detected as same object). We use Tm,m to estimate uncertainty based both localization prediction and confidence prediction for yn.m as:
Pn.m
Dr,
(4)
1Tnm
where Fr.m is the class prediction confidence of esh. detection in Tu.m-
3.2.1 Uncertainty-Guidee Pseudo-Labelling and Tiling
We interpret the averaged confidences ((.) as a proxy (or indirect) measure of how uncertain (or certain) the model is in its class assignment and object localization information [41]. Under this definition, the model will be completely uncertain if ft.) has uniform distribution whereas it will be completely certain if P(.) be represented by Kronecker delta function.
encertaint-guiider pseudo-labelling for self-training. As discussed above, the calibration can be considered as measure of network's 's overall prediction uncertainty. To this end, we attempt to discover the relationship between calibration and individual detection uncertainties. We plot the relationship between the expected calibration error (ECE) score [13] and output detection uncertainties (Fig. 3. We see an existence relationship between the ECE score and detection uncertainties. When we select pseudo-labels with more certain detections, the calibration error goes down significantly for this selected set. We hope that for this selected set of pseudo-labels, high confidence detection will more likely result in a correct labudo-label.
5

---

Score
0.34
0.12
ECE
LLTT
Uncersiinty
0.08
ECE
615
0.04
0.00
Adaptation
Selectd Labels)
Figure 3: Left. ECE score as function of UGT, UGPL, and our method that achieves synergy between UGT UGPL, over the adaptation itcrations. Right. Selecting more certain object detection pscudo-labels results in significant improvement in ECE score for this selected set over the adaptation course.
In the light this observation, we propose to select the pseudo-label ý,,8 corresponding to detection ji by utilizing the uncertainty and detection consistency across N inferences:
9j.k 16 2 x1]1[17ga12 kg].
(5)
where K1 and K2 are uncertainty and detection consistency thresholds. Fig. Dillustrates some example detections that will be considered as pseudo-labely Once the pseudo-labels are selected using Eq. (5. we use them perform self- seff-training as:
(Ca, bu,r)
Σ u,v
20 (Cu Cu,,)) +
Σ 2la bboor (bu, Bu,,) a,,
Npos
Npos
(6)
where Eu,,, bu. represents the class label and bounding box coordinates of the (selected) pseudo- label. Compared to Eq. (T) in Eq. (6 we back-propagate classification loss only for (selected)
pseudo-label locations.
Uncertainty-guien guiddd tiling for adversarial alignment. Existing image and instance-level adversarial feature alignment suffer from interfering background and noisy object localization. We propose enveertaint-guuidee tiling for adversarial alignment; it mines relatively uncertain detected regions, possible object-like regions, for the tiling process. Tiling anchored by uncertain object regions allows adversarial alignment to focus on potential, however, uncertain object- region with context (see Fig. 2 Specifically, if 9j 0 corresponding uncertain detection ý,,x for extracting tile around Particularly, given 6 as the bounding box for detection YjA. we crop tile (region) T; of scale W times as that of the detected bounding box. For source image, we randomly extract a tile S, around the ground-truth bounding box. After resizing both T; and S; to the input image size, we perform the adversarial alignment as:
detection f, in Eq. 5). we consider it an
Cader(St, T --- Dedur (FF) + -2) log(1 Dadur(FF)aan
where Fiy. and Fj. are the feature maps for S; and T,, respectively.
Discussion. We analyze the impact on model's calibration through the adaptation phase after (1) selecting pseudo-labels with more certain detections (UGPL), (2) performing tiling on relatively uncertain detections (UGT), and (3) achieving the the synergy between UGPL and UGT (our method). Model's calibration can be measured with Expected Calibration Error (ECE) score. We compute ECE score by considering both the confidence and the regression branch of the detector (24) Fig. B reveals that UGPL results in decreasing ECE score, and similarly (UGT) allows reducing the same even further. Finally, the synergy between UGPL and UGT achieves the lowest ECE score, significantly alleviating the impact of poor model's calibration under domain shift.
Training objective. We combine Eq, I Eq. 6), and Eq 7 into joint loss as L Ldet + Lyt Lads and optimize it to adapt the source model to the target domain.
Description how ECE score is computed for detector is included in supplementary material.

---

Method
person
bicycle
mAP@0.5
Two Stage
Object
Detector
DAF 4
25.0
31.0
40.5
22.1
35.3
20.2
20.0
27.1
27.6
18.8/888
SW-DA 43
29.9
42.3
43.5
24.5
36.2
32.6
30.0
35.3
34.3
20.37 14.0
DAM 15
30.8
44.3
27.2
38.4
34.5
28.4
32.2
34.6
18.8 16.7
CR DA 55
32.9
43.8
49.2
27.2
45.1
36.4
30.3
34.6
37.4
22.07 15.4
CF-DA
43.2
37.4
52.1
34.7
34.0
46.9
29.9
30.8
38.6
20.8 17.8
IITC
33.2
47.5
47.9
31.6
47.4
40.9
32.3
37.1
39.8
20.3 19.5
UADA (36
34.2
48.9
52.4
30.3
42.7
46.0
33.2
36.2
40.5
20.3 20.2
SAPNet 28
40.8
46.7
40.7
40.9
20.3 20.6
One Stage Object Detector
Source Only
31.7
31.7
34.6
5.9
20.3
2.5
10.6
25.8
20.4
Baseline
17
38.7
36.1
53.1
21.9
35.4
25.7
20.6
33.9
33.2
18.47
EPM
41.9
38.7
56.7
22.6
26.8
24.6
35.5
36.0
18.47 17.6
Ours (SSAL)
45.1
47.4
59.4
24.5
50.0
25.7
26.0
38,7
39.6
20.4
19.2
Oracle
47.4
40.8
66.8
27.2
48.2
32.4
31.2
38.3
41.5
Table 1: Cityscapes + Foggy Cityscapes Our method achíeves an absolute gain of 19.2% over the source only model and out-performs most recent one-stage domain adaptive detector (EPM). SO refers to souree only. The best results are bold-faced.
Experiments
Datasets. Cityscapes (5) dataset features images of road and street scenes and offers 2975 and 500 examples for training and validation, respectively. It comprises following categories; person, rider, car, truck, bus, train, motorbike, and bicycle.
Foggy Cityscapes
dataset is constructed using Cityscapes dataset by simulating foggy weather utilizing depth maps provided in Cityscapes with three levels of foggy weather.
Sim10k 20 dataset is collection of synthesized images, comprising 10K images and their corre- sponding bounding box annotations.
KITTI Ii dataset bears resemblance Cityscapes it features images of road scenes with wide view of area, except that KITTI images were captured with a different camera setup. Following existing works, we consider car class for experiments when adapting from KITTI or Sim
Implementation Details. FCOS [7), fully convolutional one-stage object detector, is trained over the source domain. During the adaptation process, using the cource-trained model, we iterate two steps: UGPL and UGT (Sec. (3..1)) Following [56] [5] we define going over these two steps Domain Adaptation Round just Round. In all of the experiments for uniformity, three rounds. Since initially pseudo-labelling accuracy likely poor, following (53], we perform adversarial domain adaptation (using UGT), in round called RO. In next rounds, R1 and R2, we apply both the self-training and adversarial domain adaptation using UGPL and UGT, respectively. For extracting tile around uncertain detection, five times larger region cropped around the center location. Height and width are re-adjusted to make the extracted tile square, so that during the resizing in any later stage the aspect ratio of any object in tile remains unaffected.
We use batch size 3. Learning rate is set to x 10 during the training of source model and RO round training, and then reduced to 1 10 during the R1 and R2. R1 and R2 consists of 10K iterations, RO however is consists of 5.. IoU threshold 7 is set to 0.5. We use N = 10 MC-drop out inferences, with dropout rate (Quadro RTX 6000). Ki and K2, uncertainty and detection consistency thresholds, are both set to 0.5, indicating object same class prediction and location should occur at-least 50% of times. All training and testing images are resized such that their shorter side has 800 pixels.
10%. All experiments are performed using single GPU
4.1 Comparison with state-of-the-art
For all the domain adaptation experiments we compare both existing state-of-the-art, one-stage and two-stage object detectors using the same feature backbone. Results are compared terms of mAP(%), class-wise APs(%), and gain (%) achieved over source only model. To better understand
7

---

Sim10K
CS
KITTI
CS
Table 2: Sim10K -> Cityscapes: We outperform stage two-stage object detectors both terms of mAP(%) and gain obtained over source. For this case. baseline value was recomputed. KITTI + Cityscapes: Our method outperforms both EPM and existing state-of-the-art methods with considerable margin in terms of mAP. SO refers to source only. The best results are bold-
Method
Gain
SO/ Gain
Two Stage Object Detector
DAF 8
39.0
30.1 /8.9
38.5
30.2/8.3
SC-DA 55
43.0
34.0/9.0
42.5
37.4/5.1
MAF 15
31.17
41.0
30.27 10.8
CF-DA [54
35.0/888
HTCN 3
42.5
34.6/7.9
SAPNet 28
44.9
34.6 / 10.3
faced.
UADA 36
42.0
3..6/74
One Stage Object Detector
Source Only
38.0
34,9
Baseline (7
46.0
39.8/6.2
39.1
34.4/4.7
EPM 17
49.0
39.8/9.2
43.2
34.4 /88
Ours (SSAL.
51.8
38.07 13.8
45.6
34,9/ 10.7
the effect our algorithm, we also report results on Baseline, which is FCOS Tian et al. 47 with global-level feature alignment. We discuss each experiment below.
along
Weather Adaptation (Cityscapes - Foggy Cityscapes). Under same backbone and detection pipeline, our method outperforms the most recent one-stage domain adaptive detector (EPM) by an absolute margin of 3.6% and 1.6% terms of mAP and gain. We report (Tab. [ competitive performance against methods built on much stronger, two-stage anchor-based detection pipelines. In Fig. 5 compared to EPM [77, our method shows the capability of detecting objects of various sizes under severe climate changes.
Synthetic-to-real (Sim10K Cityscapes). Our method delivers a significant gain of 13.8% (Tab. 2 It exceeds existing state of the art methods, including ones built on stronger detection pipelines and feature backbones, by notable margin, that is 2.8% mAP over top- po--effooming oteestage adaptive detector (EPM) and 6.9% over two-stage object detection adaptation algorithm SAPNet (28).
Cross-camera Adaptation (KITTI Cityscapes). For this wide view camera setup the normal scenario we achieve 45.6% mAP, as compared to results reported by the existing state-of-the-aar. algorithms using one-stage and two-stage detection pipelines, 43.2% and 42.5% (Tab. 2).
4.2 Ablation Studies
Contribution of Components: To analyze the effectiveness of each individual component in our proposed method we perform Sim10K - Cityscapes adaptation in different settings. Results are detailed in Tab. We compare the impact on performance by training our model each time with (1.) confidence based pseudo labels only, obtained without our proposed uncertainty based selection. (UGPL) is used without the uncertainty-guider tiling procedure. and (3.) when relying only uncertainty-guudee tiling (UGT). Both UGPL and UGT show an increase 11.5% 12% in APQ0.5 over source only model and 3.5% 4.0% our Baseline. The non-trivial combination of UGPL and UGT, resulting in a synergy between them, produces& a further 1.8% increase in AP@0.5 over their individual performance contributions. Especially in case of AP our combined method reports 4.9 points improvement over the Baseline and more than 3 points improvement over UGPL and UGT, indicating that our method produces more accurate bounding boxes in the target domain.
(2.) when only eneertainty-uuidee
Impact of object sizes: In Table 3 we also include the impact on performance of different compo- nents w.r.t object sizes. We use MS-COCS evaluation metric (32 to understand method's behavior with respect to different object sizes categorized as small (S):< 32 pixels, medium (M): between 32 96 pixels and large (L.): > 96 pixels.
Uncertainty vs Confidence. We contrast between the proposed uncertainty- guideed balancing of pseudo-label (PL) selection and the tiling procedure and the confidence-guiie -uuiddd balancing of these two procedures (Fig. left)). Our approach resonates well with the fact that only when the model starts to become more certain of its detections, after round 1, the quantity of selected pseudo-labels should start to increase and so the number of regions being allocated to tiling should begin to decrease. This is not the case for the confidence based balancing. Through adaptive allocation of detection
8

---

21000
18000
15000
12000
Psenuto Labels
8000
Wof
6000
3000
Round
Round 2
Round
Round
Confidence based Labels
Psoudo Selection
Selecton
Figure 4: Left. Comparison of uncertainty guided vs the confidence guidee selection of and tiles. Right. Low mean accuracy of confidence based selected PL vs the uncertainty based PL indicates uncertainty based PL selection is less noisy over the adaptation rounds. As the adaptation process progresses pseudo labels (for both type of selection) increases but uncertainty hased PI. remains less erroneous than confidence based Pl..
Source Only
(SSAL)
Figure 5: Detections missed by the EPM and found by our method are shown in Blue. Compared to EPM 17 our method achieves better adaptation.
regions, in Fig. rright) we demonstrate that our approach also delivers improved pseudo-labelling accuracy in both rounds compared to confidence-based selection.
UGT Other Tile Selection Strategies. We analyze the impact of extracting tiles centered around the uncertain detections (UGT) for adversarial learning in comparison to different tile selection strategies along with the Uncertainty Guided Pseudo Labels (UGPL) in Tab. [4] Specifically, we chose full image, random tiles, and certain tiles in adversarial learning with UGPL instead of proposed
Methods
(mean)
@0.75
@M
Source Only
18.1
38.0
15.4
4.6
21,9
37.4
Baseline
25.9
46.0
25.5
5.7
28.8
52.2
Contident PL
21.8
43.2
19.8
4.7
27.5
42,9
Ours (UGPL)
27.6
49.5
26.9
6.7
31.2
55.0
Ours (UGT)
27.5
50.0
26.7
31.7
54.5
Ours (UGPL UGT)
28.9
51.8
30.4
6.4
32.7
58.7
Table 3: Ablation results on Sim10k Cityscapes. Combining the UGPL and UGT in principled way
results in most improvement than using them individually. Here, Bascline recomputed

---

Combinations
5
Fall Image + UGPL
48.1
UGPL
49.5
Random'Tilss + UGPL
49.8
UGT
50.0
Certain Tiles UGPL
50.2
UGT-UGPL
51.8
Table 4: Comparison of proposed UGT vs other tiling strategies, including full image, random and certain tiles in the adversarial learning. We observe that compared to other tile selection strategies with UGPL, our proposed UGT provides maximum AP with UGPL.
(intelligent) tile selection process (UGT). Note that, when using random tiles there are various parameters le.g..location size, and aspect ratio) involved in the tile selection process. So, we restrict the tile-selection space using the domain knowledge. Particularly, should have at least 60% of the image area. In case of certain tiles, tiling process is performed around the certain detections for the adversarial learning. We observe that compared to all three tile selection strategies with UGPL, our proposed UGT with UGPL provides maximum AP@0.5.
restrict that the tile selected
Impact of RO. To show how much RO round contributes to the final performance. we report the performance of the base model (source only) after different rounds of adaptation for all three datasets adaptation scenarios. We report AP@0.3 0.55 after RO and after RO+R 1 over the source model. As indicated Tab. [] performing both RI and R2 rounds (that include both UGPL+UGT) results in significant improvement over when only RO round (UGT) is performed.
Datasets
Source Only
Source + RD Momnee+R++R--e
CS to Foggy CS
20.4
27.4
39.6
Sim10K CS
38.0
46.3
51.8
KITTI to CS
34.9
38.5
45.6
Table 5: Impact of RO round. Performing both R1 and R2 rounds (UGPI. +UGT) results in significant improvement over when only RO round (UGT) is performed.
the existing SOTA algorithms based on both one-stage and two-stage object detection pipelines, our method still faces challenges when dealing with small objects depicted in Tab. 3 We plan overcome this limitation by studying relationship between uncertainty, object sizes and related contexts.
Limitation. Although report improvement
Conclusion
We propose leverage model's predictive uncertainty achieve the best of self-training and adversarial learning for domain-adaptive object detection. Specifically, we propose to measure uncertainty in object detections by considering the variations in both the localization prediction and confidence prediction across Monte-Carlo dropout inferences. Certain detections are considered as pseudo-labels for self-training, while uncertain ones are used to extract tiles (regions in image) for adversarial feature alignment. This synergy between the both allows us incorporating instance-level context for effective adversarial alignment and improving feature discriminability for class- specifcc alignment. Further, it helps to reduce the effect of poor calibration under domain shift, thereby improving model's generalization across domains. Under various domain shift scenarios our method obtains notable improvements over the existing wate-of-the the-hhhrrtt art methods.
References
[1] Cai, Q. Y. Pan, C.W.. Ngo, X. Tian, I.. Duan, and T. Yao (2019). Exploring object relation in mean teacher for cross-domain detection. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 11457-11466.
[2] Cai. Z. and N. Vasconcelos (2018). Cascade R-CNN: delving into high quality object detection. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22. 2018, pp. 6154-6162.
10

---

[3] Chen, C., Z. Zheng, X. Ding, Y. Huang, and Q. Dou (2020). Harmonizing transferability and discriminability for adapting object detectors. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition. pp. 8869-8878.
[4] Chen, Y., W. Li, C. Sakaridis, D. Dai, and L. Van Gool (2018). Domain adaptive faster r-cnn for object detection in the wild. In Proceedings of the IEEE conference on computer vision and patiern recognition, pp. 3339-3348.
[5] Cordts, M., M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele (2016). The eityscapes dataset for semantic urhan scene understanding. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[6] Duan, K., S. Bai, L. Xie, H. Qi. Q. Huang, and Q. Tian (2019). Centeret: Keypoint triplets for object detection. In Proceedings of the IEEE International Conference on Computer Vision, pp. 6569-6578.
[7] Everingham, M., L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman (2010). The pascal visual object classcs (voc) challenge. IJC 88(2), 303-338.
[8] Gal, Y. and Z. Ghahramani (2016). Dropout as a bayesian approximation Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059, PMLR.
[9] Ganin, Y. and V. Lempitsky (2015). Unsupervised domain adaptation by backpropagation. International conference on machine learning, pp. 1180-1189. PMLR.
[10] Geiger, A., P. Lenz, and Urtasun (2012a). Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354-3361. IEEE
[11] Geiger, A., P. Lenz, and R. Urtasun (2012b). Are we ready for autonomous driving? the kitti vision benchmark suite. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp- 3354-3361. IEBE.
[12] Guan, D., J. Huang. A. Xiao, Lu, and Y. Cao (2021). Uncertainty-awar unsupervised domain adaptation in object detection. arXiv preprint arXiv:2103.000 00236..
[13] Guo, C., G. Pleiss, Y. Sun, and K. Q. Weinherger (2017). On calibration of modern neural networks. In International Conference on Machine Learning, pp. 1321-1330. PMLR.
[14] Han, L., Y. Zou, R. Gao, L. Wang, and D. Metaxas (2019). Unsupervised domain adaptation via calibrating uncertainties. In CVPR Workshops, Volume
[15] Hc, Z. and L. Zhang (2019). Multi-adversarial faster-renn for unrestricted object detection. In Proceedings of the IEEE International Conference Computer Vision, pp. 6668 -6777
[16] Hong. S., S. Kang, and D. Cho (2019). Patch-level augmentation for object detection in aerial images. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp. 0-0.
[17] Hsu, C.-C., Y.H. Tsai, Y.-Y. Lin, and M.-H. Yang (2020). Every pixel matters: Center-aware feature alignment for domain adaptive object detector. In European Conference on Computer Vision. pp. 733-748. Springer.
[18] Hsu, H.-K., C.-H. Yao, Y.H. Tsai, W.-C. Hung, H.-Y. Tseng, M. Singh, and M.-H. Yang (2020). Progressive domain adaptation for object detection. In The IEEE Winter Conference on Applications of Computer Vision, pp. 749-757.
[19] Inoue, N., R. Furuta, T. Yamasaki, and K. Aizawa (2018). Cross-domain wcakly-supervised object detection through progressive domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5001-5009.
[20] Johnson-Roberson M., C. Barto, R. Mehta, N. Sridhar, K. Rosaen, and R. Vasudevan (2017). Driving in the matrix: Can virtual worlds replace human- generated annotations for real world tasks? In 2017 IEEE International Conference on Roboties and Automation (ICRA), pp. 746-753. IEEE.
[21] Khodabandeh, M., A. Vahdat, M. Ranjbar, and W. G. Macready (2019). A robust learning approach to domain adaptive object detection, In Proceedings of the IEEE International Conference on Computer Vision, pp. 480-490.
[22] Kim, S., J. Choi, T. Kim, and C. Kim (2019). Self-trainin and adversarial background regularization for unsupervised domain adaptive one-stage object detection. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp.

---

[23] Kim, T., M. Jeong, S. Kim, S. Choi, and C. Kim (2019). Diversify and match: A domain adaptive representation learning paradigm for object detection. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 12456-12465.
[24] Küppers, F.,. J. Kronenberger. A. Shantia, and A. Haselhoff (2020, June). Multivariate confidence calibration for object detection. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
[25] Lakshminarayanan, B., A. Pritzel, and C. Blundell (2016). Simple and scalable predictive uncertainty estimation using deep ensembles. arXiv preprint arXiv:1612.01472
[26] Law, H. and J. Deng (2018). Comnrmet: Detecting objects as paired keypoints. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 734-750.
[27] Lee, D.-H. et al. (2013), Pseudo-label: The simple efficient superviue learning method for deep neural networks. Workshop on challenges representation learning, ICMI, Volume 3.
[28] Li, C., D. Du, Zhang, Wen, T. Luo, Wu, and Zhu (2020). Spatial attention pyramid network for unsupervised domain adaptation. European Conference on Computer Vision, pp. 481 481-497. Springer.
[29] C.,
T. Yang. S. Zhu, C. Chen, and S. Guan (2020). Density map guided object detcction in acrial images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp- 190-191.
[30] Lin, T., P. Dollár, R. B. Ginshick, K. He, B. Hariharan, and J. Belongie (2017). Feature pyramid networks for object detection. In 2017 IEFE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolula, HI, USA, July 21-26, 2017, pp. 336-944.
[31] Lin, T.-., P. Goyal, R. Girshick, K. He, and P. Dollár (2018). Focal loss for dense object detection. IEEE transactions puttern analysis and machine intelligence.
[32] Lin, T.-Y., M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick (2014). Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer.
[33] Long, J., E. Shelhamer, and T. Darrell (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431-3440.
[34] Long, M., Z. Cao, J. Wang, and M. 1. Jordan (2018). Conditional adversarial domain adaptation. In NeurlPS.
[35] Manders, J.. T. van Laarhoven. and E. Marchiori (2018). Adversarial alignment of class prediction uncertainties for domain adaptation. arXiv preprint arXiv: arXiv:1804.0m 1804.04448.
[36] Nguyen, K.K W.L. Tseng, and H.-H. Shuai (2020a). Domain- adaptive object detection via uncertainty- aware distribution alignment. In Proceedings of the 28th ACM International Conference Multimedia, pp. 2499-2507.
[37] Nguyen, D.-K., W.-L. Tseng, and H.-H. Shuai (20206), Domain adaptive object detection via uncertainty- aware distribution alignment. In Proceedings of the 28th ACM International Conference on Multimedia, pp. 2499-2507.
[38] Ovadia, Y., E. Fertig, Ren, Z. Nado, D. Sculley, S. Nowozin, J. Dillon, B. Lakshminarayanan, and Snoek (2019). Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems, Volume 32.
[39] Onge Unel, F., B. O. Ozkalayci, and C. Cigla (2019). The power of tiling for small object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 0-0.
[40] Ren, S., K. He, R. B. Girshick, and J. Sun (2015). Faster R-CNN: towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pp. 91-99,
[41] Ringwald, T. and R. Stiefelhagen (2020). Unsupervised domain adaptation by uncertain feature alignment. arXiv preprini ar 2009 06483.
[42] Roy RoyChowdhury, A., P. Chakrabarty, A. Singh, S. Jin, H. Jiang. L. Cao, and E. Learned-Miller (2019). Automatic adaptation of object detectors to new domains using self-training, In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 780-790.
12

---

[43] Saito, K., Y. Ushiku, T. Harada, and K. Saenko (2019). Strong- weak distribution alignment for adaptive object detection. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 6956-6965.
[44] Sakaridis, C.,, D. Dai, and L. Van Gool (2018). Semantic foggy scene understanding with synthetic data. International Journal of Computer Vision 126(9), 973-992.
[45] Shu, R.,. H. H. Bui, H. Narui, and S. Ermon (2018). DIRT-T approach unsupervised domain adaptation. In 6th International Conference on Learning Representations, ICLR 2018, Vancouven BC, Canada, April 30 May 3. 2018, Conference Track Proceedings.
[46] Singh. B. and Davis (2018). An analysis scale invariance in object detection SNIP. In 2018 IEEE Conference on Computer Vision and Puttern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 3578-3587.
[47] Tian, Z., C. Shen, H. Chen, and T. He (2019). Fcos: Fully convolutional one-stage object detection. In Proceedings of the IEEE international conference on computer vision, pp. 9627-9636.
[48] Tompson, J., R. Goroshin, A. Jain, Y. LeCun, and C. Bregler (2015). Efficient object localization using convolutional networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 648 656.
[49] Torralba, A. and A. A. Efros (2011). Unbiased look at dataset bias. In CVPR 2011. pp. 1521-1528. IEEE.
[50] Xu, C.-D., X.-R. Zhao, X. Jin, and X.-S. Wei (2020). Exploring categorical regularization for domain adaptive object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11724-17733
[SI] Yang. F., H. Fan, P. Chu, E. Blasch, and H. Ling (2019). Clustered object detection in aerial images. In Proceedings of the IEEE International Conference Computer Vision, pp. 8311-8811 18320..
[52] Yu, Y. Jiang. Z. Wang, Z. Cao, and T. Huang (2016). Unitbox: An advanced object detection network. In Proceedings the 24th ACM international conference Multimedia, pp. 516-520.
[53] ZHANG, Q., J. Zhang, W. Liu, and D. Tao (2019). Category anchor-guided unsupervised domain adaptation for semantic segmentation. In Advances in Neural Information Processing Systems, Volume 32.
[54] Zheng, Y., D. Huang, S. Liu, and Y. Wang (2020). Cross-domain object detection through noarse-to-fine feature adaptation. In Proceedingy of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13766-13775.
[55] Zhu, X., J. Pang, C. Yang. Shi. and D. Lin (2019). Adapting object detectors via selective cross-domain alignment. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 687-696.
[56] Zou, Y., Z. Yu, X. Liu, B. Kumar, and J. Wang (2019). Confidence regularized self-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5982-5991.
[57] Zou, Y. Z. Yu, B. Vijaya Kumar, and J. Wang (2018). Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 289-305.
13


Reviews:
[{"review_id": "l4INrHcVX5X", "reviewer": "Reviewer_WDr2", "summary": "This paper measures predictive uncertainty on class assignments and the bounding box predictions. The uncertainty is further used to guide the self-training and adversarial learning for domain adaptive object detection. The experiments cover various domain shift scenarios and prove the approach improves upon existing state-of-the-art methods. ", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, this paper is easy to follow. The experiments validate the effectiveness of the method. \n\n1. However, my major concern about this paper is its novelty. Using uncertainty estimation to facilitate self-training and adversarial learning is a common operation in domain adaptation. The unique component of this paper is the way to measure uncertainty. As this is the core component of the method and the modifications are incremental, I hope to see more intuitions or conclusions that can help us guide the design of uncertainty estimation. More analysis about various uncertainty estimations are demanded to support the paper. While this paper only conducts a simple ablation study, which is not sufficient for the acceptance of the paper. \n\n2. As the paper also talks about the complementary between self-training (A) and adversarial learning (B), is it possible to observe more improvements with A + B than independently using the two components? (A bring ) What’s the difference between this synergy with [22]?\n\nJoint considering the effective experiments and the limited novelty, I give my ranking as borderline reject. I am open to hearing from the authors to clear the main contributions of their paper vs. related works. \n\nDetailed comments:\n\nThere is only a Section 3.2.1 upon Section 3.2 which is not proper. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "hUtZWaGgmC", "reviewer": "Reviewer_eqrv", "summary": "Authors address the issue of domain adaptation for object detection, to detect objects in images with same label space but drawn from different data distributions. To this end, the paper proposes to mine uncertain tiles including object-like regions for adversarial feature alignment, and mine certain pseudo-labels for self training. Authors argue that the uncertainty guided training strategy can lead to better calibration under domain shift and improve the model's generatlization. Experiments are carried out on street driving datasets and demonstrate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Mining pseudo-labels with low uncertainty for training makes sense to me and should lead to more robust training on the target domain images. However, it's not very clear why it would be optimal to select tiles with uncertain detections. It would be better to provide more experiments about the adversarial alignment part, especially about different sample selection strategy (random selection, keeping all detection outputs, etc).\n\nIn Table 1 and Table 2, although those two-stage models should have more powerful detection performance, they generally are worse in the source only settings. It may suggest that those baselines are not well-tuned or powerful enough. It would be interesting to see if the proposed method can also be applied to two-stage methods and provide consistent performance gains.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Authors have adequately addressed the limitations and potential negative societal impact of their work, or there are no obvious issues regarding that direction.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "HwaHdETmJU7", "reviewer": "Reviewer_dBi3", "summary": "The work proposes a UDA pipeline for object detection based on self-training (via pseudo-labeling) and adversarial alignment. To this end, Monte-Carlo dropout inference is used to estimate detection confidence (or uncertainty). Certain detections (with  associated pseud-labels) are used for self-training while  uncertain ones are used to extract tile for alignment.\n\n", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1) The paper reads well and presents simple concepts, it is thus easy to understand. However, while the motivation behind using detections with low uncertainty for self-training is very intuitive, I could not grasp the reason behind exploiting low-confidence detections for adversarial aligment. I would accept it being verified experimentally, but no ablation in this direction is provided.\n\n2) Novelty is quite limited, since all techniques employed are well known (MC- dropout, adversarial alignment, self-training via pseudo-labeling. Putting them together is somehow novel, but as I mention above, the motivation is unclear and it has not been proven nor verified experimentally.\n\n3) Introduction, line 61: building over [47] cannot be claimed as a contribution\n\n4) Related work, second paragraph can be omitted. Tiling is only marginally related to the work.\n\n5) UGT paragraph (lines 202-211): here the authors try to motivate uncertainty-guided tiling as \"Tiling anchored by uncertain object regions allows adversarial alignment to focus on potential, however, uncertain object-like region with context\". Why not concentrating on those with high confidence or simply all of the detections. Since not motivation is brought forward, I would have expected an ablation study of this.\n\n6) Ablation (especially table 3. and related paragraph - lines 268-280). Beside the missing ablation mentioned above, warmup seems to account for 2/3 of the total improvement over the baseline for mean AP (Baseline: 25.9, Ours w/o warmup: + 0.9%, Ours (UGPL + UGT)  + 2.9% )\n\n7) Impact of object size (lines 281-284): no discussion on the (relatively) poor results for Small objects. Also you may want to discuss the reasons why warmup is detrimental for S(mall), almost irrelevant for M(edium) and essential for L(arge) objects.\n\nBased on the points above, my overall score is below average. The missing ablation  (point 5) raises doubts about the soundness of the method, especially since there is no clear motivation for using only low confidence detections for adversarial alignment and being that one of the main claimed contributions.\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I would argue that no discussion is needed on this topic. I cannot see any evident negative impact in adapting an object detector to a new domain.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "DxVlZhS_HQk", "reviewer": "Reviewer_pTSg", "summary": " In this paper, authors propose an unsupervised domain adaptation method for object detection based on pseudo-labeling and adversarial training. The main idea is two-fold: 1) Monte-Carlo dropout is used to produce the detection uncertainty information that is used to generate pseudo-labels; and 2) regions surrounding uncertain detections are used for adversarial training against the regions from the source domain surrounding the ground truth bounding boxes. The proposed method is compared with the state-of-the-art single-stage and two-stage object detectors on Cityscapes-to-Foggy Cityscapes, KITTI-to-Cityscapes and Sim10K-to-Cityscapes datasets. Additionally, an ablation study examining the contribution of each of the proposed modules to the increased detection precision is presented. \n", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is overall well-written, except the abstract which is a little bit hard to read.\n\nThe proposed idea of using spatial MC dropout to predict the detection and localization uncertainty is sufficiently novel and clever.  It is also intuitive to use the certain predictions as pseudo-labels. The only issue that I see is that the uncertain detections are used for adversarial training, as if they actually always correspond to the foreground objects. It is very much possible that some of the uncertain predictions are actually the false positives, and it is not clear to me how adversarially matching them to the source foreground objects would benefit the detection accuracy. \n\nThe experimental design and the choice of baseline methods as well as an ablation study provide conclusive and convincing evidence that both of the proposed methods lead to increased detection precision of medium and large objects. \n\nWhile Figure 3 clearly illustrates that both the uncertainty and calibration error are minimized by the proposed solutions, the message behind the Figure 4 is unclear to me. It is difficult to understand what the authors were trying to show with these histograms, so perhaps a more detailed explanation in the caption with a brief conclusion would make it easier to comprehend. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations regarding the decreased detection accuracy of the small objects is discussed in the paper. However, the proposed method involves multiple stages of training and multiple additional modules, which is why a discussion of how much additional memory / trainable parameters / more training time is needed for this method compared to the others is needed.   \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Following the rebuttal and discussion phase two reviewers increased their scores leading to final recommendations of one clear accept, two leaning towards accept and one rejection rating. After considering all reviews and discussion between authors and reviewers, the AC agrees with the reviewer consensus that this work merits acceptance. The key concern prior to the rebuttal was whether the combination of prior approaches (such as MC-dropout, adversarial alignment, and pseudo-label training) were sufficiently new. The rebuttal provided sufficient clarification on this point, especially through new ablation studies. The authors should make sure to include these ablation studies as well as additional text clarifying the novelty of their contribution in the final version.
Decision: Accept (Poster)