Decoupled Adaptation for Cross-Domain Object Detection
Authors: Junguang Jiang
Baixu Chen
Jianmin Wang
Mingsheng Long
Venue: ICLR 2022 Poster (2022)
Keywords: Object Detection
Domain Adaptation
Object Localization
Deep Learning
Transfer Learning

Abstract: Cross-domain object detection is more challenging than object classification since multiple objects exist in an image and the location of each object is unknown in the unlabeled target domain. As a result, when we adapt features of different objects to enhance the transferability of the detector, the features of the foreground and the background are easy to be confused, which may hurt the discriminability of the detector. Besides, previous methods focused on category adaptation but ignored another important part for object detection, i.e., the adaptation on bounding box regression. To this end, we propose D-adapt, namely Decoupled Adaptation, to decouple the adversarial adaptation and the training of the detector. Besides, we fill the blank of regression domain adaptation in object detection by introducing a bounding box adaptor. Experiments show that \textit{D-adapt} achieves state-of-the-art results on four cross-domain object detection tasks and yields 17\%  and 21\% relative improvement on benchmark datasets Clipart1k and Comic2k in particular.

Published as a conference paper at ICLR 2022
DECOUPLED ADAPTATION FOR CROSS-DOMAIN OBJECT DETECTION
Junguang Jiang, Baixu Chen, Jianmin Wang, Mingsheng Long
School of Software, BNRist, Tsinghua University, China {jig20, chenbx18 8) @ma .. s.tainghua cr,
{ mwang, mi mingshngg/ttsinghuuee edu. cn
ABSTRACT
Cross-domain object detection is more challenging than object classification since multiple objects exist in an image and the location of each object is unknown in the unlabeled target domain. As a result, when we adapt features of different ob- jects to enhance the transferability of the detector, the features of the foreground and the background are easy to be confused, which may hurt the discriminability of the detector. Besides, previous methods focused on category adaptation but ig- nored another important part for object detection, i.e., the adaptation on bounding box regression. To this end, propose D-adapt, namely Decoupled Adaptation, to decouple the adversarial adaptation and the training of the detector. Besides, we introduce a bounding box adaptor improve the localization performance. Experiments show that D-adapt achieves sfute-of-the-r results on four cross- domain object detection tasks and yields 17% and 21% relative improvement on benchmark datasets Clipart Ik and Comic2k in particular.
INTRODUCTION
The object detection task has aroused great interest due to its wide applications. In the past few years, the development of deep neural networks has boosted the performance of object detectors [33; 15; 41]. While these detectors have achieved excellent performance on the benchmark datasets [11:31], object detection in the real world still faces challenges from the large variance in viewpoints, object appearance, backgrounds, illumination, image quality, etc. Such domain shifts have been observed to cause significant performance drop [8]. Thus, some work uses domain adaptation [39] to transfer a detector from source domain, where sufficient training data is available, to a target domain where only unlabeled data is available [8; 43]. This technique successfully improves the performance the detector on the target domain. However, the improvement of domain adaptation in object detection remains relatively mild compared with that in object classification.
The inherent challenges come from three aspects. Data challenge: what adapt in the object detec- tion task is unknown. Instance feature adaptation in the object level (Figure 1(a)) might confuse the features of the foreground and the background since the generated proposals may not be true objects and many true objects might be missing (Figure 5). Global feature adaptation in the image level (Figure i(b)) likely mix up features of different objects since each input image of detection has multiple objects. Local feature adaptation in the pixel level (Figure I(c)) can alleviate domain shift when the shift is primarily low-level, yet it will struggle when the domains are different at the semantic level. Architecture challenge: while the above adaptation methods introduce domain discriminators and gradient reverse layers [12] into the detector architecture to encourage domain- invariant features, the discriminability of features might get deteriorated [6; 5], which will greatly influence the localization and the classification of the detectors. Besides, where to place these mod- ules in the detection architecture has great impact on the final performance but is little tricky. Therefore, the scalability of these methods to different detection architectures is not so satisfactory. Task challenge: object detection is multi-task learning problem, consisting of both classification and localization. Yet previous adaptation algorithms mainly explored the category adaptation, and it's still difficult to obtain adaptation model suitable for different tasks at the same time.
To overcome these challenges, we propose a general framework D-adapt, namely Decoupled Adaptation. Since adversarial alignment directly the features of the detector might hurt its dis-

---

Published as a conference paper at ICLR 2022
Detectir
Detectre
Desecctor
RIN
Back
Upopsolc
Uaiqpery
Features
peopora
Autipece
Instace Disciiiirtto
Global Disrimiisate
Local Discimanter
tensoe flow
data flow
(a) Instance adapt
(b) Global adapt
(c) Local adapt
(d) Decouple adapt
Figure I: Comparisons among techniques. Most previous methods can be categorized into instance
adaptation [8], global adaptation [58], or local adaptation [43], which perform adaptation the
features of the detector. In decoupled adaptation, the adaptors are decoupled from the detector, and different adaptors are also decoupled. Decouple means that different parts have independent model parameters, independent input data distributions and independent training losses. Different parts are coordinated into some relationships through data rather than gradients, e.g., different adaptors form a cascading relationship while the detector and the adaptors form a eelf-feedback relationship.
criminability (architecture challenge), we decouple the adversarial adaptation from the training of the detector by introducing parameter-indeeendenn category adaptor (see Figure ((d).. To tackle the task challenge, we introduce another bounding box adaptor that's s decoupled from both the de- tector and the category adaptor. To tackle the data challenge, we propose adjust the object-level data distribution for specific adaptation tasks. For example, in the category adaptation step, we en- courage the input proposals to have IoU' close to 0 or 1 to better satisfy the low-density separation assumption, while in the bounding box adaptation step, we encourage the input proposals to have IoU between 0.5 and to ease the optimization of the bounding box localization task.
The contributions of this work are summarized three-fold. (1) We introduce D-adapt framework for cross-domain object detection, which is general for both two-stage and single-stage detectors. (2) We propose an effective method to adapt the bounding box localization task, which is ignored by existing methods but crucial for achieving superior final performance. (3) We conduct exten- sive experiments and validate that our method achieves state-of-the-ee performance four object detection tasks, and yields 17% and 21% relative improvement on Clipart kk and Comic2k.
2 RELATED WORK
Generic domain adaptation for classification. Domain adaptation is proposed to overcome the distribution shift across domains. In the classification setting, most of the domain adaptation meth- ods are based on Moment Matching or Adversarial Adaptation. Moment Matching methods [50; 36] align distributions by minimizing the distribution discrepancy in the feature space. Taking the same spirit as Generative Adversarial Networks [16], Adversarial Adaptation [12; 37] introduces domain discriminator to distinguish the source from the target, then the feature extractor is encouraged to fool the discriminator and learn domain invariant features. However, directly applying these methods to object detection yields an unsatisfactory effect. The difficulty is that the image of object detec- tion usually contains multiple objects, thus the features of an image have complex multimodal structures [20; 58; 5], making the image-level feature alignment problematic [58; 20].
Generic domain adaptation for regression. Most domain adaptation methods designed for clas- sification do not work well on regression tasks since the regression space is continuous with no clear decision boundary [22]. Some specific regression algorithms are proposed, including impor- tance weighting [54] or learning invariant representations [40; 38]. RSD [7] defines geometrical distance for learning transferable representations and disparity discrepancy [57] proposes an upper bound for the distribution distance in the regression problems. Yet previous methods are mainly tested on simple tasks while this paper extends domain adaptation to the object localization tasks.
Domain adaptation for object detection. DA-Faster [8] performs feature alignment at both image-level and instance-level. SWDA [43] proposes that strong alignment of the local features more effective than the strong alignment of the global features. Hsu et al. [20] carries out center-
The Intersection-over-Uniioo hetween the proposals and the ground-truth instance.
2

---

Published as a conference paper at ICLR 2022
aware alignment by paying more attention to foreground pixels. HTCN [5] calibrates the transfer- ability of feature representations hierarchically. Zheng et al. [59] proposes to extract foreground asymmetric tri way approach to account for the differences in labeling statistics hetween domains. CRDA [53] and auxiliary task to regularize the features. However,
regions and adopts coarse-to to-nee feature adaptation. ATF [19] introduces
MCAR [58] use multi-label classification although the auxiliary task of outputting Nomain-invariant features to fool a domain discriminator in most aforementioned methods can improve the transferability, it also impairs the discriminability of the detector. In contrast, we decouple the adversarial adaptation and the training of the detec- tor, thus the adaptors could specialize in transfer between domains, and the detector could focus on improving the discriminability while enjoying the transferability brought by the adaptors.
Self-training with pseudo labels. Pseudo-labeling [30], which leverages the model itself to ob- tain labels on unlabeled data, is widely used in self-training. To generate reliable pseudo labels, temporal ensembling [29] maintains an exponential moving average prediction for each sample, while the mean- weeantteacher [49] averages model weights at different training iterations to get teacher model. Deep mutual learning [56] trains a pool of student models with supervisions from each other. FixMatch [47] uses the model's predictions on weakly-augmented images to generate seeudo-labels for the strongly-auge auggeeted ones. Unbiased Teacher [35] introduces the teacher-student paradigm to Semi-Supervin
Object Detection (SS-OD). When some image-level labels exist, the perfor- mance can be further improved by encoding correlations between coarse-grained and fine- ane-ggrained classes [55], employing noise-tolerant training strategies [13], or learning mapping from weakly- supervised to fully-supervised detectors [24] in SS-OD. Recent works [21; 25; 26] utilize self- training in cross-domain object detection and take the most confident predictions as pseudo labels. MTOR [3] uses the mean teacher framework and UMT [10] adopts distillation and CycleGAN [60] in self-training. However, self-training suffers from the problem of confirmation bias [1: 4]: the performance of the student will be limited by that of the teacher. Although pseudo labels are also used in our proposed D-adapt, they are generated from adaptors that have independent parameters and different tasks from the detector, thereby alleviating the confirmation bias of the overly tight relationship in self-training.
PROPOSED METHOD
In supervised object detection, we have labeled source domain D, ((Xi, B', YO where X is the image, B3 is the bounding box coordinates, and Y is the categories. The detector G is trained with Cder, which consists of four losses in Faster RCNN [42]: the RPN classification loss f the RPN regression loss Leg the Rol classification loss Ltoi and the Rol regression loss froi
C Ex,, CTPe +
Cipa reg
Croi reg
In cross-domain object detection, there exists another unlabeled target domain D, (x:)= that follows different distributions from D,, The objective of Gddet is to improve the performance on D,,
3.1 D-ADAPT FRAMEWORK
To deal with the architecture challenge mentioned in Section 1, we propose the D-adapt framework, which has three steps: (1) decouple the original cross-domain detection problem into several sub- problems (2) design adaptors to solve each sub-problem (3) coordinate the relationships between different adaptors and the detector.
Since adaptation might hurt the discriminability of the detector, we decouple the category adaptation from the training of the detector by introducing a parameter-indes indepeeddenneeeeeeet category adaptor (see Fig- ure ((d).. The adaptation is only performed on the features of the category adaptor, thus will not hurt the detector's ability to locate objects. To fill the blank of regression domain adaptation in object detection, we need to perform adaptation on the bounding box regression. Yet feature visualization in Figure 6(c) reveals that features that contain both category and location information do not have an obvious cluster structure, and alignment might hurt its discriminability. Besides, the common category adaptation methods are also not effective on regression tasks [22], thus we decouple cate- gory adaptation and the bounding box adaptation to avoid their interfering with each other. Section 3.2 and 3.3 will introduce the design of category adaptor and box adaptor in details. In this section, will assume that such two adaptors are already obtained.
3

---

Published as a conference paper at ICLR 2022
To coordinate the adaptation different tasks, we maintain a cascading relationship between the adaptors. In the cascading structure, the later adaptors can utilize the information obtained by the previous adaptors for better adaptation, e.g. in the box adaptation step, the category adaptor will se- lect foreground proposals to facilitate the training of the box adaptor. Compared with the multi-task learning relationship where we need balance the weights of different adaptation losses carefully, the cascade relationship greatly reduces the difficulty of typer-paramedte selection since each adap- tor has only one adaptation loss. Since the adaptors are specifically designed for cross-domain tasks, their predictions on the target domain can serve as pseudo labels for the detector. On the other hand, the detector generates proposals to train the adaptors and higher-quality proposals can improve the adaptation performance (see Table 5 for details). And this enables the self-feedbuck relationship between the detector and the adaptors.
For good initialization of this self-feedbact loop, we first pre-train the detector Gdet on the source domain with C. Using the pre-trained Gdlet we can derive two new data distributions, the source proposal distribution D and the target proposal distribution Dr Each proposal consists of crop of the image x2, its corresponding bounding box budt, predicted category ydu and the class confidence d We can annotate each source-domain proposal X, E D with a ground truth bounding box be¹ and category label y.". similar to labeling each Rol in Fast RCNN [14], and then use these labels to train the adaptors. In turn, for each target proposal x, E D.P adaptors will provide category pseudo label ythh and box pseudo label b, to train the Rol heads,
Cdet = bf" Ifth bbth ED Lobl (x, b yythh + Ca (X,, b F0B yx
(ygh) C (X,, bast b
where lig is a function that indicates whether it is foreground class. Note that regression loss is ac- tivated only for foreground anchors. After obtain- ing a better detector by optimizing Equation 2, we can generate higher-quality proposals, which fa- cilitate better category adaptation and bounding box adaptation. This process can iterate multiple times and the detailed optimization procedures are summarized in Algorithm 1.
A Algorithm 1: D-adapt Training Pipeline.
input Source domain D. and target domain Di. number of iterations
output: Cross domain object detector G+
iiitaalzz the object detector t by optinizing with c for t 1 to do
generate proposals D! and Di for each sample in D, and De by Ctil. for eack mini maii-butch in Deme and Di do train the category adaptor (21h; end
Note that our D-adapt framework does not intro- duce any computational overhead in the inference phase, since the adaptors are independent of the detector and can be removed during detection. Also, D-adapt does not depend on a specific de- tector, thus the detector can be replaced by SSD [34], RetinaNet [32], or other detectors.
generate category label for each proposal in D generate foroground proposals D't and D from Dio and Domn
for eack mini-batch DJ and DD do I
train the bounding box udaptor (2"s; end
generate bounding box label for each proposal in D/C train the object detector G¹' by optimizing with Ct";
end
3.2 CATEGORY ADAPTATION
The goal of category adaptation is to use labeled source-domain proposals (X.. E Do to ob- tain a relatively accurate classification ytth of the unlabeled target-domain proposals x, E D.mup Some generic adaptation methods, such as DANN [12], can be adopted. DANN introduces do- main discriminator to distinguish the source from the target, then the feature extractor tries to learn domain-invariant invariant representations to fool the discriminator, which will enlarge the decision bound- aries between classes on the unlabeled target domain. However, the above adversarial alignment might fail due the data challenge- the input data distribution doesn't II satisfy the low-density sep- aration assumption well, i.e., the Intersection-ove -onnnnnoo of a proposal and a foreground instance may be any value between 0 and 1 (see Figure 2(a)) and explicit task-specific boundaries between classes hardly exist, which will impede the adversarial alignment [22]. Recall that in standard object detection, proposals with loU between 0.3 and 0.7 will be removed discretize the input space and ease the optimization of the classification. Yet it can hardly be used in the domain adaptation problem since we cannot obtain ground truth IoU for target proposals.
We use uppercase letters to represent the whole image, lowercase letters to represent an instance of object
4

---

Published as a conference paper at ICLR 2022
Density
Gete
confidence threshold
3.3
1 w(t)
D
pele
Tapa
fz
9t
0.0
0.5 1.0 IoU
Unddiaad
gradieni nevuree layer
Stop grndiut
(a) IoU distribution of proposals (b) Discretization
(c) Architecture of the category adaptor
Figure 2: Category adaptation (best viewed in color). (a) The IoU distribution of the proposals from Foggy Cityscapes. When we increase the confidence threshold from to 0.9, undefined proposals (proposals with IoU between 0.3 and 0.7) will decrease. (b) Proposals with lower confidence will
be assigned lower weight in the adaptation. (c) The discriminator D is trained to separate the source-domain proposals from the target-domain proposals for each class independently, while the feature extractor pels is encouraged to fool D.
To overcome the data challenge, we use the confidence of each proposal to discretize the input space, ie., when proposal has high confidence cdse being the foreground background. it should have a higher weight w (( in the adaptation, and vice versa (see Figure 2(b)). This will reduce the participation of proposals that are neither foreground nor background and improve the discreteness of the input space in the sense of probability. Then the objective of the discriminator D is,
max Cak = Ex D. (((,)) logl (fss, gx)] + R. w(()) log[1 ((fe,g:))
(3)
where both the feature representation f = [**(x) and the category prediction = (el(()) are fed into the domain discriminator D (see Figure 2(c)). This will encourage features aligned in conditional way [37]. and thus avoid that most target proposals aligned to the dominant category on the source domain. The objective of the feature extractor Fuls to separate different categories on the source domain and learn domain-invariant features to fool the discriminator,
min E DX. LCe (G (f v?) + AC G
(4)
where Lce is the cross-entropy loss, X is the trade-off between source risk and domain adversarial loss. After obtaining the adapted classifier, we can generate category pseudo label yı G Fel(x,) for each proposal E D.
3.3 BOUNDING BOX ADAPTATION
The objective of box adaptation is to utilize labeled source-domain foreground proposals (x., b E D to obtain bounding box labels b, of the unlabeled target-domain proposals E D/e Recall that in object detection, regression loss is activated only for foreground anchor and is disabled oth- erwise [14], thus only adapt the foreground proposals when training the bounding box regressor. Since the ground truth labels of target-domain proposals are unknown, we use the prediction ob- tained in the category adaptation step, i.e. D&d = [(x,, E DE
[Tre )y
Following RCNN [15], we adopt a class-specific bounding-boo box regressor, which predicts the bound- ing box regression offsets, e (th, gk t tk) for each of the K foreground classes, indexed by k. On the source domain, we have the ground truth category and bounding box label for each proposal, thus we use the smooth L, loss to train the regressor,
min C. pis,Cris C0
E (x. S........bo bib VE w.
smooth (t) - Us),
(5)
y is ground truth category, is the ground truth bounding box offsets calculated from bedh and bdlt However, it's hard to obtain satisfactory regressor with cleg on the target domain due to the domain shift.
where t Greg (x,) is the regression prediction,
Inspired by the lastest theory [57]. propose IoU disparity discrepancy method. As shown in Figure 3(a), we train a feature generator network F which takes proposal inputs, and two regressor networks G and G which take features from prg. The objective of the adversarial
5

---

Published as a conference paper at ICLR 2022
9
G
L
Tog
Free
Freg
Baav
reg adtv
gradicat reverse layer
stop gradicat
(a) Architecture of the bounding box adaptor
(b) Minimax on IoU
Figure 3: Bounding box adaptation (best viewed in color). Box adaptor has three parts: feature generator Fee, regressor Ge and adversarial regressor G G learns to maximize the target disparity by moving two predicted boxes far from each other while F learns to minimize the target disparity by making two predicted boxes overlap as much as possible.
regressor network Gav is to maximize its disparity with the main regressor the target domain while minimizing the disparity on the source domain to measure the discrepancy across domains. Then the objective of the adversarial regressor is
Teg adv =B (x (1) smoothr, (Gary (xTX G o F (x )x²
max C
(6)
E (x. smoothh, (G Fx (x,) )o G F (xx) )os)
Note that smooth,, on the source domain is only defined on the box corresponding to the ground truth category y and that on the target domain is only defined on the box associated with the pre- dicted category yrh. Equation guides the adversarial regressor predict correctly on the source domain while making as many mistakes as possible on the target domain (Figure 3(b)). Then the feature extractor F encouraged to output ancy,
features to decrease domain discrep-
min
CTT + reg pig
(7)
where 17 is the trade-off between source risk and adversarial loss. After obtaining the adapted re- gressor, we can generate box pseudo label th G o (x,( for each proposal x, D/E
4 EXPERIMENTS
4.1 DATASETS
Following six object detection datasets are used: Pascal [OC Clipart [21], Comic [21], Sim10k [23], Cityscapes [9] and Foggy Cityscapss [44]. Pascal VOC contains 20 categories of common real- world objects and 16, 551 images. Clipart contains 1K images and shares 20 categories with Pascal VOC. Comic2k contains 1k training images and 1 test images, sharing categories with Pascal VOC. Sim 1O has 10, 000 images with 58, 701 bounding boxes of car categories, rendered by the gaming engine Grand Theft Auto. Both Cityscapes and FoyyyCitsscpess have 2975 training images and 500 validation images with 8 object categories. Following [43], we evaluate the domain adap- tation performance of different methods on the following four domain adaptation tasks, VOC-to- Clipart, Co-ticCC Sim 10k-to to- to-Cityscapes, Cityscapes-tor Sorgsstooooooyyyycoppproo and report the mean average precision (mAP) with threshold of 0.5.
4.2 IMPLEMENTATION DETAILS
Stage 1: Source-domain pre-training. the basic experiments, Faster-RCNN [42] with ResNet-
101 [17] or VGG-16 [46] as backbone is adopted and pre-trained the on the source domain with learning rate of 0.005 for 12k iterations.
Stage 2: Category adaptation. The category adaptor has the same backbone as the detector but
simple classification head. It's trained for 10k iterations using SGD optimizer with an initial learning rate of 0.01, momentum 0.9, and a batch size of 32 for each domain. The discriminator D is three- layer fully connected networks following DANN [12]. is kept 1 for all experiments. w(c) is when > 0.5 and 0 otherwise.

---

Published as a conference paper at ICLR 2022
Table I: Results from PASCAL VOC to Clipart (ResNet101).
auto
beycle
bird
bua
bodil
bus
car
call
chair
cow
Lable
dog
Ins
bike
pean
plnt
sheep
sofa
unio
EV
mAP
Source Only
25.6
52.5
24.3
25.0
20.0
43.9
52.
10.7
30.6
11.7
13.8
6.0
36.8
43.9
45.7
41.9
16.5
7.3
22.9
52.0
27.8
DA-Fisterr [S]
15.0
54.6
12.4
11.9
193
21.1
23.2
3.1
22.1
26.3
10.5
10.0
19.6
39.4
34.6
29.3
1.0
17.1
19.7
24.8
19.8
BDC-Faader [43]
46.4
1.5
366
33.3
15.5
25.6
WST-BSRI 1271
64.5
25.9
30.5
7.9
54.5
36,4
57.4
43.5
35.7
SWDA [43]
26.2
48.5
32.6
33.7
58.5
$4.3
57.1
18.6
3488
58.3
17.0
12.5
33.
65.5
61.6
52B
9.3
24.9
54.1
49.1
38.1
MAF [18]
38.1
61.1
25.8
43.9
40.
416
40.3
92
37.1
48.4
26.2
13.4
36.4
52.7
52.0
52.5
18.2
24.3
52.9
39.3
36.8
SCT. [45]
500
55.6
26.3
47.
CRDA [53]
55.3
38.7
49.3
17.6
14.1
33.3
61.3
46.3
22.1
49.1
44.3
38,3
HICN 15]
33.6
58.9
34.0
23.4
45.6
57.0
998
12.0
39.7
51.3
21.1
20.1
39.1
72.
63.0
43.1
19.3
30.1
50.2
513
40.3
ATF [19]
41.9
67.0
274
364
41.0
48.5
42.0
13.1
30.3
75.1
334
79
41.
563
61.4
50.6
42.0
25.0
53.1
99.1
42.1
Unhinsed [35]
51.8
D-adape
03.2
77.0
44.3
58,4
31.4
24.5
47.1
69.3
43.5
27.9
60.7
64.0
VOC-Cllipn
VOC-Commi
Sim Citysscpee
Citysapes Cityscupes
Source only
BB
Amnolation
Ours
Figure 4: Qualitative results on the target domain.
Stage 3: Bounding box adaptation. The box adaptor has the same backbone the detector but simple regression head (two-layer convolutions networks). The training hyper-paramette (learning rate, batch size, etc.) are the same that of the category adaptor. 1j is kept 0.1 for all experiments. The input of the bounding box adaptor (the crops of objects) will be twice larger than the original predicted box, so that the bounding box adapter could access more location information.
Stage 4: Target-domain pseudo-label training. The detector trained on the target domain for 4k iterations, with an initial learning rate of 2.5 10-4 and reducing 2.5 10-5 exponentially.
The adaptors and the detector are trained in an alternative way for T = 3 iterations. We perform all experiments on public datasets using a 1080Ti GPU. Code is available at /tctts::/gith/ii com com/thuml// Saaaum//eecoupe
4.3
COMPARISON WITH SEATE-OF-THE--AE
Adaptation between dissimilar domains. We first show experiments dissimilar domains using the Pascal VOC Dataset as the source domain and Clipart as the target domain. Table shows that proposed method outperforms the state-of-the-art method by 6.9 points mAP. Figure 4 presents some qualitative results in the target domain. We also compare with Unbiased Teacher [35], the state-of-the-art method in semi-supervised object detection, which generates pseudo labels on the target domain from the teacher model. Due to the large domain shift, the prediction from the teacher detection model is unreliable, thus it doesn't 't do well. In contrast, our method alleviates the confirmation bias problem by generating pseudo labels from different models (adaptors).
We also use Comic2k as the target domain, which has very different style from Pascal VOC and lot of small objects. As shown in Table 2, both image-level and instance-level feature adaptation will fall into the dilemma of transferability and dis- criminability, and do not work well on this difficult dataset. In contrast, our method effectively solves this problem by decoupling the adversarial adapta- tion from the training of the detector and improves mAP by 7.0 compared with the ofth-atf-
Table 2: Results from VOC to Comic
1011), Oracle results are obtained by training on labeled data in the target domain.
Method
bike
bird
mAP
Source Only
32.5
12.0
21.1
10.4
12.4
29.9
19.7
DA [8]
15.5
12.4
3900
21.2
SWDA [13)
MCAR [SS]
47.9
20.5
37.4
20.6
24.5
53.6
33.5
Instance Adapt
39.5
17.7
26.5
27.3
22.4
48.4
30,3
Global Adapt
31.9
15.7
30.3
21.3
17.1
37.9
25.7
D-adapt
Oracke
40.9
70,9
44.6

---

Published as a conference paper at ICLR 2022
Adaptation from synthetic to real images. We use Sim10k the source domain and Cityscapes as the target domain. Following [43], we evaluate on the validation split of the Cityscapes and report the mAP on car. Table 3 shows that method surpasses all other methods.
Table Sim10k to Cityscapes.
Table 4: Results from Cityscapes to Foggy Cityscapes.
Method
Backhone Car
Method
Backhone pesn
rider
car
truck
hus
train
moyele boyele
MAP
Source Only
34,5
Source only
327
31.0
12.5
23.9
23.7
29.1
23.4
FA- [S]
38,9
Faster [8]
25.0
31.0
40.5
22.1
35.3
20.2
20.0
27.1
27.3
BDC (13]
31.8
BDC Fasser
26.4
37.2
424
212
29.2
12.3
22.6
28,9
27.5
SWDA
40.1
SW-DA [45]
36.2
35.3
43.5
300
29.9
24.3
34.3
[18]
VGG-16
41.1
Selective [61]
33.5
38.0
48.5
26.5
39.0
33.6
33.8
Selective DA [61]
43.0
DI-MRL" [28]
30.8
40.5
44.3
27:2
38.4
34.5
28.4
32.2
$4.5
CDN 1431
49.3
CADA [20]
41.9
38.7
56.7
22.6
41.5
26.8
24.6
35.5
30.0
HICN" [5]
42.5
CRDA [53]
32.9
43.
49.2
27.2
45.1
36.4
30.3
34.5
37.4
CERA [SP]
458
CPEA [59]
46.9
52.1
308
43.2
29.9
34.7
37,4
38.6
ATP 191
428
AIF
34.6
47.0
50.
23.7
43.3
38.7
33.4
38.8
58.7
CADA [20]
49.0
MCAR [58]
32.0
42.1
43.9
51.3
44.1
43.4
37.4
36.5
38.8
McGA-CDA
44.8
HICN" [20]
47.9
31.6
47.4
39.8
LMIT [10]
43.1
D-adapt
518
58.1
26.3
36.8
14.6
32.2
42,0
38.1
Deodage
51.3
D-adagt"
44.9
54.2
61.7
25.6
36.3
24.7
37.3
46.1
41.3
Ornele
69.7
Onacle
4774
40.8
66.8
27.2
48.2
32.4
31.2
38.3
41.5
Source only
41.8
Source only
33.8
34.8
39.5
18.6
27.9
6.3
18.2
25.5
25.6
CADA [20]
RosNet101
51.2
CADA [20]
ResNer101
57.1
29.4
44.9
39.7
29.0
36.1
4012
D adaps
51.9
D Ddgppt
42.8
48.4
56.8
31.5
428
37.4
35.2
42.4
42.1
D-adapt"
40.8
47.1
57.5
33.5
46.9
41.4
33.6
43.0
43.0
Oniele
70.4
Oracle
[44.7
43.9
64.7
31.5
48.8
44.0
31.0
36.7
43.2
Adaptation between similar domains. We perform adaptation from Cityscapes to FoggyCi- tyscape and report the results' in Table 4. Note that since the two domains relatively similar, the performance of adaptation is already close to the oracle results.
4.4 ABLATION STUDIES
In this part, we will analyze both the performance of the detector and the adaptors. Denote mty be the number of proposals of class predicted as class j. & be the total number of proposals of class i, and N be the number of classes (including the background), then we use mloUshs N 4.+ to measure the overall performance of the category adaptor. We use the intersection-over-niie between the predicted bounding boxes and the ground truth boxes, i.c., m to measure the performance of the bounding box adaptor. All ablations are performed on VOC + Clipart and the iteration T' is kept for a fair comparison.
E, Ru nn
Ablation on the category adaptation. Table 6(a) show the effective- ness of several specific designs mentioned in Section 3.2. Among them, the weight mechanism has the greatest impact, indicating the necessity of the low-density assumption in the adversarial adaptation. To verify this, we assume that the ground truth IoU of each proposal is known, and then we select the proposal with IoU greater than a certain threshold when we train the category adaptor: Table shows that as the IoU threshold of the foreground pro- posals improves from 0.05 to 0.7, the accuracy of the category adaptor will increase from 36.1 to 51.4, which shows the importance of the low-density separation assumption.
Table 5: Effect of pro- posals' quality.
boll threshold 0,55
03 0.5 0.7
mlot
36.1
38.2 46.7 51.4
Ablation on the bounding box adaptation. Table 6(b) illustrates that minimizing the disparity discrepancy improves the performance of the box adaptor and bounding box adaptation improves the performance of the detector in the target domain.
Ablation on the training strategy with pseudo-labels. In Equation 2, losses are only calculated on the regions where the proposals are located, and those anchor areas overlapping with the propos- als are ignored. Here, compare this strategy with the common practice in self-training filter out bounding boxes with low confidence, then label each proposal that overlaps with these boxes. Although the category labels of these bounding boxes also generated from the category adaptor, the accuracy of these generated proposals is low (see Table 6(c)). In contrast, our strategy is more conservative and both the mlo on the proposals and the final mAP of the detector are higher.
4.5 ANALYSIS
denotes this method utilizes CycleGAN to perform source-to-target translation.
8

---

Published as a conference paper at ICLR 2022
Table 6: Ablations on PASCAL VOC Clipart. Note that bounding box adaptation is adopted in (a) and (c) for a fair comparison. (a) Category adaptation. w/o condition: use a class-independent discriminator. w/o bg proposals: no background proposals added to source domain target domain or neither. w/o weight: remove the weight mechanism in Equation 3. w/o adaptor: remove the category adaptation step and directly use the labels generated from detector on the target domain as pseudo labels. (b) Spatial Adaptation. w/o DD: remove the disparity discrepancy in Equation 6. w/o adaptor: remove the bounding box adaptation step and only trains the classification branch of the detector. (c) Training strategy, In the standard training, if the confidence threshold increases, the number of false negatives will increase, otherwise the number of false positives will increase.
(a) Category adaptation
(b) Spatial Adaptation
(c) Training strategy
by proposall
metric
w/o aduptir
metric
standard
condition
snurce
target
weight
adapeor
mloU
0.631
0,598
0,531
confidence
threshold
0.1
0.7
0.1
mlot
172
17.6
17.1
16.3
38.2
mlvt
17.2
389
34.4
43.5
mAP
43.5
41.7
41.7
38.8
36.5
33.3
28D0
Error Analysis. Figure gives the percent of error of each model on VOC- following [2]. The main errors in the target domain come from: Miss (ground truth regarded as backgrounds) and Cls (clas- sified incorrectly). Loc (classified correctly but localized incorrectly) errors are slightly less, but still cannot be ignored especially after cate- gory adaptation, which implies the necessity of box adaptation in object detection. Category adaptation can effectively reduce the proportion of Cls errors while increasing that of Loe errors, thus it is reasonable to cascade the box adaptor after the category adaptor. Bounding box adaptation can reduce the proportion of Loc errors, revealing its effectiveness.
cvere 200 oappi 102
Figure 5: Error analysis.
Feature visualization. We visualize by t-SNE [51] in Figures 6(a)-6(b) the representations of task VOC - Comic2k (6 classes) by category adaptor with À 0 and category adaptor with À 1. The source and target are well aligned in the latter, which indicates that it learns domain-invarian features. We also extract box features from the detector and get Figure 6(c)-6(d). We find that the features of the detector do not have an obvious cluster structure, even on the source domain. The reason that the features of the detector contain both category information and location information. Thus adversarial adaptation directly on the detector will hurt its discriminability, while our method achieves better performance through decoupled adaptation.
(a) Adaptor (A = 0)
(b) Adaptor (A = 1) (c) Baseline (mAP:19.7) (d) Ours (mAP:40.5)
Figure 6: T-SNE visualization of features. (a) and (b) are features from the category adaptor. (c) and (d) are features from the Faster RCNN. (Orange: VOC; Blue: Comic2k).
DISCUSSION AND CONCLUSION
Our method achieved considerable improvement on several benchmark datasets for domain adapta- tion. In actual deployment, the detection performance can be further boosted by employing stronger adaptors without introducing any computational overhead since the adaptors can be removed dur- ing inference. It is also possible to extend the D-adapt framework other detection tasks, e.g., instance segmentation and keypoint detection, by cascading more specially designed adaptors. We hope D-adapt will be useful for the wider application of detection tasks.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGEMENTS
This work was supported by the National Megaproject for New Generation AI (2000AAA0109001),, National Natural Science Foundation of China (62022050 and 62021002), Beijing Nova Program (22010000088000/ and BNRist Innovation Fund (BNRR222 RC010002.
REFERENCES
[1] Eric Arazo, Diego Ortego, Paul Albert, Noel E O'Connor, and Kevin McGuinness. Pseudo- labeling and confirmation bias in deep semi-supervised learning. In IJCNN, 2020.
[2] Daniel Bolya, Sean Foley, James Hays, and Judy Hoffman. Tide: A general toolbox for iden- tifying object detection errors. In ECCV, 2020.
[3] Qi Cai, Yingwei Pan, Chong-Wai Wah Ngo, Xinmei Tian, Lingyu Duan, and Ting Yao. Exploring object relation in mean teacher for cross-domain detection. In CVPR, 2019.
[4] Paola Dascante-Bonilla, Fuwen Tan, Yanjun Qi, and Vicente Ordonez. Curriculum labeling: Revisiting pseudo-labeling for semi-supervised learning. In AAAI, 2021.
[5] Chaoqi Chen, Zebiao Zheng, Xinghao Ding, Yue Huang, and Qi Dou. Harmonizing transfer- ability and discriminability for adapting object detectors. In CVPR, 2020.
[6] Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. Transferability vs. discrim- inability: Batch spectral penalization for adversarial domain adaptation. In ICML, 2019.
[7] Xinyang Chen, Sinan Wang, Jianmin Wang, and Mingsheng Long. Representation subspace distance for domain adaptation regression. ICML, 2021.
[8] Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Domain adaptive faster r-cnn for object detection in the wild. In CVPR, 2018.
(9] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Ro- drigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR, 2016.
[10] Jinhong Deng, Wen Li, Yuhua Chen, and Lixin Duan. Unbiased mean teacher for cross-domain object detection. In CVPR, 2021.
[11] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisser- man. The pascal visual object classes (voc) challenge. In IJCV, 2010.
[12] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In ICML, 2015.
[13] Jiyang Gao, Jiang Wang, Shengyang Dai, Li-Jia Li, and Ram Nevatia. Note-renn: Noise tolerant ensemble renn for semi-supervised object detection. ICCV, 2019.
[14] Ross Girshick. Fast r-cnn. In ICCV, 2015.
[15] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 2014.
[16] Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. In NeurIPS, 2014.
[17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
[18] Zhenwei He and Lei Zhang. aultiaadversarial faster-renn for unrestricted object detection. In ICCV, 2019.
10

---

Published as a conference paper at ICLR 2022
[19] Zhenwei He and Lei Zhang. Domain adaptive object detection via asymmetric tri-way faster- renn. In ECCV, 2020.
[20] Cheng-Chur Hsu, Yi-Hsuan Tsai, Yen- Yu Lin, and Ming-Hsuale Yang. Every pixel matters: Center-aware feature alignment for domain adaptive object detector. In ECCV, 2020.
[21] Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa. Cross-domain weakly-supervised object detection through progressive domain adaptation. In CVPR, 2018.
[22] Junguang Jiang, Yifei Ji, Ximei Wang, Yufeng Liu, Jianmin Wang, and Mingsheng Long. Regressive domain adaptation for unsupervised keypoint detection. In CVPR, 2021.
[23] Matthew Rosaen, and Ram Vasudevan. Driving in the matrix: Can virtual worlds replace human- generated annotations for real world tasks? arXiv preprint arXiv: 161000961,00198,, 2016.
Charles Barto, Rounak Mchta, Sharath Nittur Sridhar, Karl
[24] Siddhesh Khandelwal, Raghav Goyal, and Leonid Sigal. Unit: Unified knowledge transfer for any-shot object detection and segmentation. CVPR, 2021.
[25] Mehran Khodabandeh, Arash Vahdat, Mani Ranjbar, and William G Macready. A robust learn- ing approach to domain adaptive object detection. In ICCV, 2019.
[26] Seunghyeon Kim, Jachoon Choi, Tackyung Kim, and Changick Kim. Self-training and adver- sarial background regularization for unsupervised domain adaptive one-stage object detection. In ICCV, 2019.
[27] Seunghyeon Kim, Jaehoon Choi, Taekyung Kim, and Changick Kim. Self-training and adver- sarial background regularization for unsupervised domain adaptive one-stage object detection. In ICCV, 2019.
[28] Taekyung Kim, Minki Jeong. Seunghyeon Kim, Seokeon Choi, and Changick Kim. Diversify and match: A domain adaptive representation learning paradigm for object detection. In CVPR, 2019.
[29] Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning, In ICLR, 2017.
[30] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In ICML, 2013.
[31] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects context. In ECCV, 2014.
[32] Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, 2017.
[33] Li Liu, Wanli Ouyang, Xiaogang Wang. Paul Fieguth, Jie Chen, Xinwang Liu, and Matti Pietikäinen. Deep learning for generic object detection: A survey. IJC,, 2020.
[34] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In ECCV, 2016.
[35] Yen-Cheng Liu, Chih-Yan Ma, Zijian He, Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, and Peter Vajda. Unbiased teacher for semi-supervised object detection. In ICLR, 2021.
[36] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan. Learning transferable fea- tures with deep adaptation networks. In ICML, 2015.
[37] Mingsheng Long, Zhangjic Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain adaptation. In NewrIPS, 2018.

---

Published as a conference paper at ICLR 2022
[38] Ramin Mikzaa-Langerodi. Werner Zellinger, Susanne Saminger-Platte and Bernhard A. Moser. Domain adaptation for regression under beer- lamber''sss law. Knowledge-Bond Based Systems, 210: 106447, 2020.
[39] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on Knowl- edge and Data Engineering, pp. 1345-1359, 2010.
[40] Sinno Jialin Pan, Ivor W. Tsang, James T. Kwok, and Qiang Yang. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks, 22(2):199-110, 2011.
doi: 10.1 20000TN.2000..0911111
[41] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Uni- fied, real-time object detection. In CVPR, 2016.
[42] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NeurlPS, 2015.
[43] Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko. Strong-weak distribution alignment for adaptive object detection. In CVPR, 2019.
[44] Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Semantic foggy scene understanding with synthetic data. In IJCV, 2018.
[45] Zhiqiang Shen, Harsh Maheshwari, Weichen Yao, and Marios Savvides. Scl: Towards accurate domain adaptive object detection via gradient detach based stacked complementary losses. arXiv preprint arXiv:
2019.
[46] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
[47] Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NeurIPS, 2020.
[48] Peng Su, Kun Wang, Xingyu Zeng, Shixiang Tang. Dapeng Chen, Di Qiu, and Xiaogang Wang. Adapting object detectors with conditional domain normalization. In ECCV, 2020.
[49] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In NeurIPS, 2017.
[50] Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell, Deep domain confusion: Maximizing for domain invariance. In CORR, 2014.
[51] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. In JMLR, 2008
[52] Vibashan VS, Vikram Gupta, Poojan Oza, Vishwanath A Sindagi, and Vishal M Patel. Mega- cda: Memory guided attention for category- unsupervised domain adaptive object detec- tion. In CVPR, 2021.
[53] Chang-Dong Xu, Xing-Ran Zhao, Xin Jin, and Xiu-Shen Wei. Exploring categorical regular- ization for domain adaptive object detection. In CVPR, 2020.
[54] M. Yamada, I.. Sigal, and Yi Chang. Domain adaptation for structured regression. Interna- tional Journal of Computer Vision, 109:126-11 145, 2013.
[55] Hao Yang, Hao Wu. and Hao Chen. Detecting 1k classes: Large scale object detection without fine-grained bounding boxes. CVPR, 2019.
[56] Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu. Deep mutual learning. In CVPR. 2018.
[57] Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. Bridging theory and algo- rithm for domain adaptation. In ICML, 2019.
12

---

Published as a conference paper at ICLR 2022
[58] Zhen Zhao, Yuhong Guo, Haifeng Shen, and Jieping Ye. Adaptive object detection with dual multi-label prediction. In ECCV, 2020.
[59] Yangtao Zheng, Di Huang, Songtao Liu, and Yunhong Wang. Cross-domain object detection through coarse-to-fine feature adaptation. In CVPR, 2020.
[60] Jun- Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-imas to-immae translation using cycle-consistent adversarial networks. In ICCV, 2017.
[61] Xinge Zhu, Jiangmiao Pang. Ceyuan Yang, Jianping Shi. and Dahua Lin. Adapting object detectors via selective cross-domain alignment. In CVPR, 2019.
13

---

Published as a conference paper at ICLR 2022
A MORE EXPERIMENT RESULTS
Results on Other Architecture. As shown in Tables 7, our method also applies to the one-stage detector RetinaNet [32] which improves the mAP by 17.5 on VOC - Clipart. The proposed D-adapt framework also surpasses both image-levell(b) and feature-levell(c) alignment as well as their combination by a considerable margin.
Table 7: Results from PASCAL VOC to Clipart (RetinaNet, ResNet1011.
Methud
boycle
bird
boat
bottle
bus
chair
table
dog
hrs
bike
ynn
pini
sofa
mAP
Sounce Only
30.1
40.8
21.7
153
28.4
51.6
31.1
13.1
34.5
142
29.6
16.2
31.4
53.1
374
30.3
6.9
24.8
31.8
42.1
28.8
Global
434
13.0
39.5
32.6
Adapt
42.2
184
33,4
54.6
48.7
404
Global + Local
37.5
50.4
25.3
28.8
45.0
31.7
45.
169
38.2
31.9
24.2
12.6
26.4
48.7
53.4
44.5
5.5
78.2
45.7
53.5
35
D-adage
47.4
65.0
331
37.5
56.8
61.2
85.1
27.
45.5
51.8
29.1
29.6
38,0
74.5
66.7
46.0
24.2
20.3
54,2
53,8
46.3
Results on VOC WaterColor: on WaterColor dataset.
As shown in Table 8, D-adapt also achieves strong performance
Table 8: Results from VOC to WaterColor (ResNet-101).
bike
bind
dog
prsn
Source Only
68.8
46.8
21.5
60.7
BDC-Faster [43]
48.3
21.7
60.5
DA-Fister [8]
75.2
40.5
48.9
31.5
20.5
60.0
46.0
WST-BSK [27]
75.6
45.8
49.3
34.1
30.5
64.1
49.9
SWDA [43]
82.5
55.9
35.5
66.7
X1F119
78.8
59.9
47.9
41.0
34.8
66.9
34.9
SCT.E-S]
82.2
55.1
51.8
39.6
38.4
64.0
55.2
MCAR [S8]
UMT [10]
88.2
55.3
51.7
39.8
43.6
69.9
Dedlsp
77.4
54.0
52.8
43.9
48.1
68.9
57.5
Orscle
48.5
54.7
41.3
36.2
52.5
74,6
51.3
Ablations the decouple strategy. adaptors is useful.
Further, we discuss whether the decoupling of different
In our original implementation, the input distributions of different adaptors completely different. In the category adaptation step, we encourage the input proposals to have loU close to 0 or to better satisfy the low-density separation assumption. In the bounding box adaptation step, encourage the input proposals to have IoU between 0.5 and 1 to ease the optimization of the bounding box localization task.
different adaptors are coupled, they must share the same input distribution. Table 9 shows that only sharing the inpul distributions will greatly damage their respective performance. Note that different adaptors still have independent architectures. And we can conclude that the decoupling of different adaptors is quite crucial.
Table 9: Ablations on the decouple strategy on VOC-Clipart.
Iaput Distribution
mioUsh
pruposals weight adoptors
proposals cupput detector)
all pruposals w/ weight (both adiptors use the propocals fel the original catepory adaptor)
33.3
0.319
fovegronnd preposals wright (hath adigtoes use the propocals fed the original hos adaptoe)
24.7
BJ631
Ours (different adaptors have different impui data distributioms)
33,3
0,631
Table 10: Ablations on the box adaptor when T varies.
Setting
(T=1)
(AP(T-2)
mAP (T=3)
without adaptor
with box adaptox (ours)
45.0
47.7
49.1
14

---

Published as a conference paper at ICLR 2022
Ablation on bounding box adaptor. Table 10 shows that the gain brought by box adaptation is consistent, for example when T 3, can still improve the mAP from 47.0 to 49.1.
B
MORE VISUALIZATION RESULTS.
Figure 7-10 gives more qualitative results on Faster RCNN.
Source only
Annotation
SUEKER
Ours
Figure 7: Qualitative results on VOC - Clipart.
Source only
Anmolation
Oracle
Ours
Figure 8: Qualitative results on VOC Comic.
15

---

Published as a conference paper at ICLR 2022
Source only
Ammolation
Oracle
Ours
Figure 9: Qualitative results on Sim10k Cityscapes.
Sourre only
Annolation
Omele
Figure 10: Qualitative results on Cityscapes - Foggy Cityscapes.
16


Reviews:
[{"review_id": "rYDMAoEQuy8", "reviewer": "Reviewer_HwgG", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The quality of the introduction is inferior to the rest of the paper, thus it can be improved.\n- As the paper presents the problem, it poses a \"Data challenge: what to adapt in the adversarial training is unknown.\" It is not clear at this point of the paper what is the relation between domain adaptation and adversarial training. For clarity, the connection that currently is made  on \"Related Work\" should be presented as part of the motivation presented in the introduction to situate the reader on the challenges presented.\n- The text poses \"Architecture challenge\" but it is also not clear on the following text, that should describe the challenge, how the argument is any related to architecture. The arguments presented in the text that follows (again without much context at this point) is that they are related to training policy and regularization, but not architecture specifically. Similar to the first challenge presented, the text is lacking a better context and justification.\n- The \"loss challenge\" will only be clear from section 3.1 on, as the introduction text defining it is not very informative. \n\nFor the sake of completeness and clarity, the main concepts related to domain adaptation and adversarial training, such as adaptors for domain adaptation, adversarial alignment and adversarial regressor, must be defined and referenced before its use and/or reformulated in the context of the paper. \n\nOn results comparison, it would be important to state any architecture difference (or state otherwise) between the proposal and compared approaches. Do they use same detector architecture?\n\nIt is also important to clarify what is the oracle used (and better justify its errors from Figure 5). \nIn that direction, the text should also clarify the results presented that surpassed the oracle results. This happens on Table 2 for \"bike\" and \"car\", and on Table 4, it is not clear how performance on \"rider\" and \"bicycle\" surpassed the oracle results for VGG backbone and \"truck\", \"motorcycle\" and \"bicycle\" for Resnet101. \n\nOverall, the experimental results are compelling, but it is necessary to better link the concepts used with pre-existent approaches on adversarial domain adaptation.", "summary_of_the_paper": "The paper focus on object detection domain adaptation to transfer a detector from a source domain, in the context where sufficient training data is available on a source domain but only unlabeled data is available in the target domain. It presents good results adapting from/to Pascal, Clipart, Comic, Sim10k, Cityscapes and FoggyCityscapes.\n", "summary_of_the_review": "The paper presents a clever combination of domain adversarial training for adapting the components of object detection. The experimental results obtain SOTA for some combinations explored. The weakest part is of the presentation is the introduction, and should be reviewed. Also,  although the set of references included have a good cover of the area, the concepts presented in the text are not connected to pre-existent concepts as it would be appropriate to be.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "Rfd62gdOCy6", "reviewer": "Reviewer_LmKN", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The paper is very well-written.\n2. The proposed idea is simple and intuitive, but turns out to be quite effective.\n3. The decoupled adaption is a great option for avoiding the hurt to the detector's discriminability.\n4. The decoupled adaption design enables the ease of incorporating the box regression adaption, which is novel.\n5. The experiments are extensive and persuasive.\n\nWeaknesses:\nWill the authors release their code? They do not mention it at all.", "summary_of_the_paper": "The authors propose a tailored method, named Decoupled Adaptation (D-adapt), for cross-domain object detection. The conventional domain adaptation techniques hurt the discriminability of the detector and ignores the adaptation on bounding box regression. In order to avoid hurting the discriminability of the detector, the authors propose to decouple the adaptation modules from the detector, which achieves the parameter independence. In addition, the authors propose the module for adapting the bounding box regression. Due to the decoupling property, the proposed D-adapt framework is compatible to many off-the-shelf detectors. The experimental results are very promising.", "summary_of_the_review": "Considering the simplicity, novelty and effectiveness, this paper is of high quality and above the acceptance bar.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "NNC0Btu2PDY", "reviewer": "Reviewer_Kz22", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Positives:\n+ The paper is well written and the problem is well motivated.  \n+ The experiments on the standard benchmarks are reported and comparisons are provided with recent works.\n+ The ablation study is formulated to reflect the claims.\n\nClarifications needed:\n- the bounding box adaptors take as input the foreground proposals, which are the crops of objects, as shown in figure 3. How does the regressor predict offsets if it has been given a close crop of the objects? The claim made is that the confirmation bias of the pseudo labels with the use of the adaptors. If the adaptors are trained to align the features such that the discriminator is confused, which means the source bounding box distribution needs to be matched with the target, hence biasing the regressor again.\n-T is set to be 3, is saturation seen after a point. Would different domain pairs have different saturation iteration? How was this value chosen? Shouldn't it be fair to also report mAP scores for T=1, in Table 3 and 4 .  \n-Table 1 is missing UMT[10] baseline which reports mAP 44.1 for VOC to Clipart which is a stronger baseline for the pseudo-label based techniques. \n-Table2 doesn't mention the backbone used in the experiment\n- The error analysis part should also include the bounding box adaptor part to highlight how much bias is removed using bounding box adaptor.\n- the line on page 9: \"The gain brought by box adaptation is consistent, for example when T = 3, it can still improve the mAP from 47.0 to 49.1.\" should it be 45.0 instead of 47.0?", "summary_of_the_paper": "This work provides a systematic way to adapt object detectors to an unseen target domain. Specifically, the object classification and bounding box regression heads are adapted one after the other, and ROIs are used to train a separate set of adaptors to provide foreground feature alignment without hurting the discriminablity of the detectors. ", "summary_of_the_review": "The approach highlights the shortcomings of the previous methods and systematically tries to resolve them. There are a few issues with the comparisons and clarification is needed for one of the claims.  Overall the writing and structure of the paper is good. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "98metX_5A9a", "reviewer": "Reviewer_PJVS", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "> Pros:\n- The approach is relatively simple and easy to understand\n- Performance of the proposed approach is strong, improving on the state-of-the-art\n\n> Cons:\n- Despite relative simplicity of the approach, writing obfuscates some of the details, making it more difficult to understand than necessary. Some improvements in exposition, particularly in Section 3, would improve overall readability.\n- Technical novelty of the proposed approach is relatively minor. Both adversarial feature adaptation and adaptive refinement of Faster RCNN heads are explored techniques. While the specific strategy proposed in the paper for doing so is novel, the core leveraged technical fundamentals are not. The novelty appears mostly in the use of proposal weighting within adaptation and sequential, rather then parallel, adaptation procedure for the two heads.\n- Paper overclaims its contributions. Specifically, it is claimed that \"previous methods mainly explored the category adaptation and ignored the regression adaptation\" (also Claim 2 of the contributions). This isn't really true. While I am not an expert on cross-domain object detection, a quick search reveals that regression adaptation has been utilized in other past works, e.g., for example in [23] see \"Bounding Box Refinement\", the pseudo-labeling approaches (e.g., \"Multi-Source Domain Adaptation for Object Detection\", Yao et al, ICCV) also refine classification and detection. Hence, this claim is at least miss-characterized, needs to be refined and overall contributions clarified.\n- For completeness, related work should also discuss approaches for object detection adaptation in semi-supervised learning (not specifically designed for cross-domain object detection, but leveraging techniques relevant to the approach). These include: (1) \"Note-rcnn: Noise tolerant ensemble rcnn for semi- supervised object detection\", Gao et al, ICCV, 2019 and (2) UniT: Unified Knowledge Transfer for Any-shot Object Detection and Segmentation, Khandelwal et al, CVPR 2021, among others.\n- Ablations show significance of the proposal weighting in the adaptation (good!), as well as significance of adaptation of regression for bounding boxes (also good!), however, fail to illustrate that decoupling and sequential nature of adaptors (Figure 1d) is important. Given that this is one of the core contributions, addition of such ablation would be very useful. \n- Minor: It is unclear why certain approaches were cited in related work, but do not appear to be compared against in the tables (e.g., [57]) ", "summary_of_the_paper": "Paper addresses the problem of cross-domain object detection. The formed source domain detectors (in the form of Faster R-CNN), learned in a supervised manner, are adopted to perform well on the target domain where no annotations are available. The key to the approach is separate adaptation of the classification head and the regression head within Faster RCNN. This leads to the proposed D-adapt, namely Decoupled Adaptation, that decouples the adversarial adaptation and the training of the detector. In addition, classification and detection heads are adopted in tandem. Experiments show that the proposed D-adapt strategy achieves state-of-the-art results on four cross-domain object detection tasks.", "summary_of_the_review": "Overall, the approach is sounds and experimentally performs well. However, the novelty and significance are somewhat limited. Further, exposition can be improved and claims (specifically claim 2) are (at least) imprecise and miss-characterize prior work. Finally, some ablations that would have been useful are missing and should be included. Given these concerns, overall, I view the paper as being slightly below bar for acceptance. I would be happy to revisit my assessment, and potentially increase the score, if authors can convincingly address the raised concerns in the rebuttal. \n\nAuthors have addressed a number of my comments in the rebuttal, and promised to addressed others (claims) in the camera ready revision. As such, I am raising my score and believe the paper should be accepted.  ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper decouples the adversarial training of a domain adaptation model with the detector learning process, and is able to disentangle the features of foreground and background when performing adaptation.  State of the art results on four different domains/tasks are presented with significant improvement. Reviewers are unanimous that the submission is acceptable.  Reviewer PJVS is the most authoritative and experienced reviewer, and notes the paper clarity is impaired, and that the paper is immodest in various places and overclaims what is otherwise supported by the results therein.  The AC concurs with this view.  However the paper is acceptable in present form for ICLR, and the AC advises the authors to revise as discussed in the rebuttal and response to the rebuttal.
Decision: Accept (Poster)