Decouple and Reconstruct: Mining Discriminative Features for Cross-domain Object Detection
Authors: Jiawei Wang
Konghuai Shen
Shao Ming
Jun Yin
Ming Liu
Venue: ICLR 2022 Submitted (2022)
Keywords: domain adaptation
object detection
discriminative feature mining

Abstract: In recent years, a great progress has been witnessed for cross-domain object detection. Most state-of-the-art methods strive to handle the relation between local regions by calibrating cross-channel and spatial information to enable better alignment. They succeed in improving the generalization of the model, but implicitly drive networks to pay more attention on the shared attributes and ignore the domain-specific feature, which limits the performance of the algorithm. In order to search for the equilibrium between transferability and discriminability, we propose a novel adaptation framework for cross-domain object detection. Specifically, we adopt a style-aware feature fusion method and design two plug-and-play feature component regularization modules, which repositions the focus of the model on domain-specific features by restructuring the style and content of features. Our key insight is that while it is difficult to extract discriminative features in target domain, it is feasible to assign the underlying details to the model via feature style transfer. Without bells and whistles, our method significantly boosts the performance of existing Domain Adaptive Faster R-CNN detectors, and achieves state-of-the-art results on several benchmark datasets for cross-domain object detection.

Under review as a conference paper at ICLR 2022
DECOUPLE AND RECONSTRUCT: MINING DISCRIMI- NATIVE FEATURES FOR CROSS-DOMAIN OBJECT DE- TECTION
Anonymous authors Paper under double-blind review
ABSTRACT
In recent years, a great progress has been witnessed for cross-domain object de- tection. Most state-of-the-ar methods strive to handle the relation between lo- cal regions by calibrating cross-channel and spatial information to enable better alignment. They succeed in improving the generalization of the model, but im- plicitly drive networks to pay more attention on the shared attributes and ignore the domain-speciffe feature, which limits the performance of the algorithm. In or- der to search for the equilibrium between transferability and discriminability, we propose novel adaptation framework for cross-domain object detection. Specif- ically, we adopt a style-aware feature fusion method and design two plug-and- play feature component regularization modules, which repositions the focus of the model on domain-specific features by restructuring the style and content of fea- tures. Our key insight is that while it difficult to extract discriminative features in target domain, it is feasible to assign the underlying details to the model via fea- ture style transfer. Without bells and whistles, our method significantly boosts the performance of existing Domain Adaptive Faster R-CNN detectors, and achieves state-of-the-ee results on several benchmark datasets for cross-domain object de- tection.
INTRODUCTION
Along with the advances in deep convolutional neural networks (Krizhevsky et al. 2012). remark- able progresses have been achieved in various visual tasks (He et al.) 2016] Long et al. 2015 Ren cet al. 2017). Such outstanding achievements heavily rely on the assumption that training and test data are drawn from an independent identical distribution. Nevertheless, due to the changes of en- vironmental condition in real-world, a phenomenon known as "domain shift" occurs and challenges this assumption, thus leading to significant performance decay. Although collecting more training data can alleviate such problems to some extent, it is non-trivial because of the high cost of image annotation.
An appealing solution to alleviate the shift is introducing domain adaptation (DA) methodology, which aims at bridging the gap between domains by learning invariant representations. In this paper, we focus on the unsupervised scenarios, called unsupervised domain adaptation (UDA), where the training set is formed from labeled source data and unlabeled target data.
Recently, a variety of UDA methods have been well-studied for classification
Tzeng et al. 2014
Ganin & Lempitsky 2015; Saito al.) 2018, Wang et al. 20200; Li al. 202lb) and semantic segmentation (Hoffman et al. 2018 Zou et al.] 2018: Dong et al. 2020 Li et al. 2020 Jaritz al. 202T), which are mostly based on minimizing domain discrepancy measurement or adversarial training. These strategies have made great progress but are hard to be applied to object detection directly because of the local nature of detection tasks (Zhu et al., 2019). To address the adaptation dilemma for detection tasks, instance-level feature alignment has been widely used in Chen et al. (2018b); Saito et al. ) Zheng et al. (2020). Most of them strive to handle the relation between local regions by calibrating cross channel and spatial information. However, it may implicitly drive networks to pay more attention on shared attributes and ignore the domain-speciff feature. As shown in Figure| T DA reduces the domain shift by feature alignment between source and target

---

Under review as a conference paper at ICLR 2022
Source Only
DA
DA+Ours
Source Features
Target Features
Illustration of the evolution process of source and target feature distributions. Source Only: feature distributions obtained by the model without domain adaptation. DA: aligned fea- ture distributions after domain adaptation. DA+Ours: refined feature distributions by applying our approach to the DA model. Compared to Source Only, DA reduces the domain gap and explores the domain-invariant features by adversarial training at the cost of discriminative features in source and target domain. Our supplementary framework highlights the target-like discriminative feature, which leading better adaptive performance.
Figure 1:
domain, which highlights the common components at the cost of domain- specificc features. We review the feature alignment of domain adaptation and suppose that image-level levell style and instance- level content play vital role in detection tasks. The style information should be globally consistent within own domain while local homogeneous content information between different domains ought to be analogous, and vice versa.
Building upon the above findings, we propose a novel adaptation framework for cross-domain object detection, which supplements the domain-specific representation by restructuring the content and style information. Specifically, we adopt the Domain Adaptive Instance Normalization (DAdaIN) on shallow layers generate synthesis features, which contributes encoding the domain and Nomain-invariaan features source and target domain. Furthermore, Global Style Alignment (GSA) module and Local Content Alignment (LCA) module are exploited regularize the consistency of style and content between multiple domains. GSA consists of two sub-modules, i.e., style similarity alignment and style consistency alignment, which are adopted in multi-layer align global style representation. Moreover, LCA is proposed to calibrate the content information of local regions, which enhances the model's s response to foreground objects of various styles.
In summary, the main contributions of this work are three-fold:
We review the feature alignment in domain adaptation and point out that it may implicitly drives networks to focus more public component and ignore the domain-specifie feature. Furthermore, we suppose that global style and local content informations play a vital role in detection tasks.
We propose novel adaptation framework for cross-domain object detection. A style-aware feature fusion method and two plug-and-play adaptation modules are designed to reposition the focus of the model to domain-specifer features by restructuring the style and content of feature.
We conduct extensive experiments adaptation between both similar and dissimilar do- mains to validate the effectiveness of our approach. The compelling experimental results and visualizations demonstrate that our method significantly boost the performance of ex- isting Domain Adaptive Faster R-CNN detectors under various domain-shift scenarios.
RELATED WORK
2.1
DOMAIN ADAPTATION
Domain adaptation Tzeng al. 2014 Gong et al. 2015: Sun & Saenko 2016 Zou et al., 2018)
with convolution neural network has emerged as a popular paradigm to boost the performance of trained model when training and test data do not satisfy the assumption of independent identical
2

---

Under review as a conference paper at ICLR 2022
distribution. They strive to learn domain- invariant features, and along this line the methods can be roughly divided into two categories: statistical matching and adversarial learning. The common idea
of statistical matching (Fernando et al.) 2014 Gong et al.] 2015: Zellinger et al. [017:] Peng et al.
2018) is to measure and minimize the distance between latent feature space of different domains by
appropriate metrics. The methods (Chang et al. 2019 Chen al. 2018a [Russo al.)
2017 Fu
et al. 2020] Xu et al.) (020) based on adversarial learning aim to learning domain invariann feature
by construct minimax optimization with the domain classifier. In addition. Li al. (2018) proposed
to modulate the statisties in all BN layers across the network. And He & Zhang (2019) applied a
progressive curriculum based adversarial training to boost fine-grained visual categorization. Be- sides, Jin et al. (202T) designed a SNR module to ensure both generalization and discrimination capability. Despite the great success of the methods mentioned above, most of them are hard to be applied to object detection directly because of the local nature of detection tasks.
2.2 DA FOR OBJECT DETECTION
Recently, an increasing number of researches (Chen et al.) 2018b: Zhu et al. 2019: Yang et al., 2020
Zhao et al., (020) have been proposed for cross-domain object detection. Domain Adaptive Faster R-CNN (Chen et al. 2018b) proposed to learn domain-invariant invariant features in image-level and instance-level by adversarial learning. Furthermore, Saito et al. (2019) applied strong-weak hier- archical feature alignment to alleviate non-transferable features, and Zhu et al. (2019) repositioned the focus of adaptation from global to local and introduces an additional module to reconstruct the patches. For the sake of more accurate feature alignment, Xu et al. (2020) integrated an image-level multi-label classifier upon the detection backbone to pay more attention on the foreground instances and re-weight the proposals to automatically hunt for the hard samples. Similarly, Zheng et al. (020)) adopted the attention mechanism to highlight the importance of the foreground regions, and then align instances with the same category between source and target domain by maintaining global prototypes. Besides. Chen et al. (2020) hierarchically calibrated the transferability of feature rep- resentations in inability. And |Vibashan et al. (2021) employed category-wise discriminators to ensure category- aware feature alignment. The above methods enable the network to extract domain-invariant features in virtue of refined feature alignment, and enhance the transferablility of the model. However, how to make better use of target-like lomain-specifer details is under-explored.
level, aiming to harmonize transferability and discrim-
2.3 STYLE TRANSFER
Style transfer (Ulynnov et al.) 2016 Yoo et al. 2019 Huang & Belongie 2017: Johnson et al. 2016) has been proven efficient the transformation of the object appearance. In general, style transfer is considered as a texture conversion issue. Some previous methods (Denton et al., 2015 Frigo et al., 2016) typicallyyyyyy rely on low-level statistics but fail to extract the semantic image content. Gatys al. (2016) pioneered the separation of image content information from its style and introduced new algorithm to perform image style transfer. Furthermore, Ulyanov et al. (211) propose light-weight feed-forward network. Surprisingly. Ulyanov et al. (2016) achieves remarkable improvement by re- placing BN with IN. Along this line. Dumoulin al. (2016); Huang Belongie (2017) go further and design alternative modules, which improve the quality and diversity of image stylization. And Kotovenko et al.) (202T) review the stylization process and concentrate on optimizing parameter- ized brushstrokes. All these methods motivate us to design a complementary module for mining target-like details domain adaptation. While previous works (Chen et al. 2020 Rodriguez & Mikolajczyk 2019) employed CycleGAN to generate interpolation samples, they faced critical limitation that the performance is greatly determined by image quality. In this paper, we propose to manipulate the content and style information simultaneously by feature reconstruction during the training phase, aiming to exploit more domain-specific features in an end-to-end fashion.
3 METHOD
3.1 PROBLEM FORMULATIONS
Unsupervised domain adaptation for object detection can boil down to the analysis of data distri- butions in two domains: source domain with adequate annotated image data and a target domain
3

---

Under review as a conference paper at ICLR 2022
Tlow
Tagge
Source Grecod
Festre
Donsiespedffe Lenming
Source Ground Truth
CBA
LGA
RPN
Source domain
Gi
Ens
Target domain
Learning
Figure 2: The Overview of our proposed adaptation framework. The main components of our model are DAdaIN, GSA and LCA. DAdaIN is inserted into the backbone network, which provides more domain-specifie details through feature reconstruction. GSA consists of two sub-modules, i... Style Similarity Regularize and Style Consistency Regularize. They constrain the similarity and consistency of global image style to improve the robustness of the model. LCA highlights the importance of local instance content, and achieves the alignment without destroying domain-specifi specific details.
with unlabeled images. Formally, we denote source domain as Ds = {(=] ((x])v())e v() where x; and vf = (bf, represent the i-th sample and label (class and bounding box) respectively. Similarly, target domain is defined as Df = {x] }. where N₂ is the amount of target samples. Ds and Dy have the identical categories tags but drawn from diverse scenarios with heterogeneous data distribution. Our destination is to generalize the detector from labeled Ds to unlabeled Dy.
3.2 FRAMEWORK OVERVIEW
The overview of proposed framework is illustrated in Figure It consists of three main components: DAdaIN, GSA and LCA. Previous works (Hsu et al. 2020 Chen al., 2020) exploited synthetic data by CycleGAN to narrow the domain gap, which cannot be trained in end-to-end fashion and sub-optimal for tackling the domain shift. In this paper, we design a style-aware feature fusion module called DAdaIN and insert it into the detector during the training phase aiming to improve the domain invariance of the model itself. Morcover, Global Style Alignment (GSA) and Local Content Alignment (LCA) are introduced to regularize the style and content information between multiple domains respectively. We suppose that image-level style and instance-leve content play a vital role in detection tasks. Extensive experiments verify the effectiveness of our method.
Our framework effectively increases the learning ability of the model for Nomain-specine features. Mcanwhile, the proposed module can be flexibly used supplement to existing SOTA methods and improve the generalization the model without any modification to the inference process.
3.3 STYLE- FEATURE FUSION
With the increase of domain gap, existing UDA methods usually exhibit instability or adaptabil- ity degradation. can be intuitively understood as the result of domain specific features missing- We suppose that low-level variations, e.g. color, texture and edges, constitute unique style which reflects the difference feature distributions between different domains. Inspired by Huang & Belongie (2017). we adopt the style-aware feature fusion method and design Domain Adaptive In- stance Normalization (DAdaIN) to provide extra target-like details. Our key insight is that while it is difficult to extract discriminative features in target domain. it is feasible to directly assign the underlying details to the model via feature style transfer, so as to alleviate the inconsistency between training and testing.
4

---

Under review as a conference paper at ICLR 2022
Specifically, denote fa IR NXXXXXX the feature map extracted by the detector in domain i E {Ds, Dr}. where H x W and C are the spatial dimensions and the number feature map channels, respectively. DAdaIN adaptively computes the affine parameters from the target feature ft to scale and shift the source feature fs:
Me
DAdaIN fi) 0
+ pr.
0x
fs
|,s + (ee pt))
d,
where μi RRN and o; NxC represent the mean and standard deviation of each channel for each sample fs.
derive that the output feature of DAdaIN depends the feature statisties (μ, o) computed across spatial dimensions. In order to enrich the diversity of synthesis feature and maintain the stability during training phase, we introduce K₁ control the translation scale between source and target domains. Meanwhile, we use Ri to balance the translation bias caused by the scale of parameter K1. The DAdaIN is adopted in the feature space of intermediate domain D.M- Eq with translation parameters can be re- re-written as:
From [
DAdaIN 81, K2) = K1
- fs + Ki 0s
Tt
T )os + K2(14 - (ka)
(2)
where кд and R2 obey the normal distribution (11,, 1).
Finally, the style-aware synthesis feature fm can be defined as following:
fm = A DA
where À is a set 1.
to control the degree of feature style transfer. In all experiments, we
Technically, we integrate DAdaIN module after relu4.2 2 of VGG16, then fuse the style and content informations to generate the synthesis features fm. worth noting that, all subsequent work is based on fm. Moreover, since the role of DAdaIN is to introduce domain-spectin details during training phase, it feasible remove such module for inference without affecting the performance.
By doing we can generate the synthesis features which encode the discriminative information of target domain in an end-to-end fashion. Besides, since the content of synthesis feature is shared with the source domain, the label information of the source domain can be better used.
3.4 DISCRIMINATIVE FEATURE MINING
Global Style Alignment. While the DAdaIN enrich the diversity of synthesis features, it also chal- lenges the model's invariance the rt-style features shallow layers. In order to further regularize the image style information across multiple domains, we propose Global Style Alignment (GSA) to constrain the style consistency with feature statistics.
GSA performs multi-level style alignment across different domains. It is composed by two parts: style similarity alignment and style consistency alignment. Style similarity alignment applied on the low-level features across the source and target domain, which aiming to improve training stability. Style consistency alignment is applied on the high-level features across the synthesis and target domain to constrain the consistency of image style. Considering Saito al. 2019) proposes that strong- weak feature alignment works well between both the similar and dissimilar domains, we construct different criteria to align the global style information at corresponding feature level.
Specifically, we define style similarity alignment as follows to measure the style similarity of shallow features between D. and Dy:
LSSA =
Σ
(4)
oE (E(1.T)
5

---

Under review as a conference paper at ICLR 2022
where the f!! and ft are output features of the G1. With the constraints shallow feature statisties, the domain gap is further reduced, so that the model can be stably trained even on dissimilar domains.
Besides, style consistency alignment adopt smooth L1 loss to align the mean and variance of high- level features between the DM and Dy:
LSCA =
Σ smoothy, (v(f)) w(f2) vE(u.o}
(5)
where the L and fe output features of the G2. In other words, it promotes the model extract stable feature in deep layers for the re-style instances by aligning the features of the target and intermediate domain. And result, the model's invariance shallow features improved.
Local Content Aligament. Attention to local regions which containing foreground objects is crucial the success of object detection. SCDA (Zhu al. 2019) repositions the focus the cross-domain adaptation process, from global to local, and shows impressive performance. However, we find that strong local feature alignment will cause instability during training. We suppose that it is due to mismatching of instance information. Strong local alignment will induce the model to pay more attention to common components. When the styles of instances in different domains differ greatly, the network will predict unexpected results and the training instability will be aggravated. A better way to enhance the invariance of the model to the local instance in various styles is aligning only the content information of the instance. Moreover, find that GSA tends to result in negative style transfer because of the poor-performing Convolutional Neural Networks, especially in the early stage. Motivated by the aforementioned findings, design Local Content Alignment(LCA) to further regularize the content of instances without destroying domain-specifie details.
Specifically, we train RPN in the intermediate domain DM and extract the instance feature of local regions in source and intermediate domain. Since DM is generated by decoupling and reconstructing the content and style Ds and Dr. the content representation ought be consistent between Ds and M. In order to achieve more accurate alignment of instance content information, we firstly re-style extracted instance feature in intermediate domain from synthesis to source:
fm DAdaIN fa)
(6)
Then, we use cycle consistency loss to regularize the content of instances with unified style:
Leycle
Σ
smoothy, (f (r).
TEGT
where (r) denotes the feature of foreground region r. Considering that the ground truth is shared between the intermediate and source domain, we use the ground truth GT to extract the foreground regions in Ds and D.M- By means of LCA, the consistency of instance content before and after feature style transfer is maintained, and the feature misalignment can be avoided.
3.5 TOTAL OBJECTIVE FUNCTION
We denote the objective of detection task as Ldets which consists of Lete and Creg- Lds aims to measure the classification accuracy, while Lreg is used to evaluates the location error. The DA Faster R-CNN series apply domain classifier in image and instance level to align the global and local feature. We denote the objective of domain classifier as LADV:
LADY = BI log Di(f) + Σ IE EE log(1 Di())
(8)
where domain discriminator Dj invariant feature extraction with respect to the base conv G shown in Figure
promotes domain-
We consider our approach as a complementary module which can be integrated into the existing DA Faster R-CNN series. The overall objective is:
Liotal = Laet + LADY + aLaSA + BBLICA
(9)
where and are trade-off factors for GSA and LCA module.

---

Under review as a conference paper at ICLR 2022
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
For all the experiments, we adopt the Faster R-CNN with VGG16 (Simonyan & Zisserman 2014) or ResNet-101 HHe al. 2016) backbone. The weights of the network is initialized with the pre- trained model on ImageNet The hyper-parameteny of the model is set following Chen et al. 20);;); Xu al. (2020); Zheng et al., (2020): [Chen et al.] (2020), and the shorter side of all training and test datasets is resized to 600 pixels. We use the stochastic gradient descent SGD (Bottou] 2010) as the optimizer with an initial learning rate of 10-3 in the training phase, which is reduced 10 10-1 after 50k iterations. We report the mAP with an loU threshold of 0.5 to evaluate the model in test procedure. All these experiments are implemented by the PyTorch framework.
4.2 COMPARISON RESULTS
We conduct quantitative comparsion on three cross-domain detection tasks by means of five datasets. Due the limit getting some papers open source code, we only employ SWDA (Saito al. 2019) and HTCN (Chen et al. 2020) as baseline models and straightly integrate our approach into them. For the sake of fairness, all of the SOTA models we cite are based on the Faster R-CNN with same backbone network. When comparing with them, all results we report are cited from original papers. Besides, we trained the model on the labeled source and target domain respectively, which are treated as the upper and lower bound of the algorithm.
Normal to Foggy. Changes in weather conditions pose a major challenge to the robustness of algo- rithm in real world. We thus perform the adaptation from Cityscape (Cordts et al. 2016) to Foggy Cityscape (Sakaridis et al., 2018) aiming to verify the domain invariance of algorithm to weather changes. Cityscapes contains 2,975 training images and 500 test images with dense pixel-level annotations. It is collected from the outdoor street scenarios under normal weather condition in Ger- man cities, while Foggy Cityscapes derives from Cityscapes and uses depth information to simulate foggy scenario. The results of the adaptation are reported in Table Our proposed method achieves state-of-the-arr mAP and boost the performance of SWDA and HTCN with 3.6% and 2.6% respec- tively. The compelling results demonstrate that our method effectively improve the generalization of the SOTA methods target domain by restructuring the style and content of feature representations.
Table 1: Results (%) of different methods on adaptation from Cityscapes to Foggy- Segyy-Cityscape.... "Source Only" denotes the Faster R-CNN model trained on the source domain only. "Oracle" rep- resents the model trained on the labeled target domain. VGG16 is used as the backbone network.
Method
bus bicyle
car meycle person rider train truck
mAP
Source Only
20.8
31.3
33.5
18.1
24.9
32.8
9.1
10.4
22.6
SCDA
(Zhu et al.
2019
39,0
33.6
48.5
28.0
33.5
38.0
23.3
26.5
33.8
CTF
iZhnng et al. 2020
43.2
37.4
52.1
34.7
34.0
46.9
29.9
30.8
38.6
DAA
[uu et al. 2020)
46.6
36.9
48.8
34.0
33.2
47.6
38.2
28.1
39.2
MeGA
Vibashan al.
2021
49.2
39.0
52.4
34.5
37.7
49.0
46.9
25.4
41.8
SWDA Saito et al.
2019
36.2
35.3
43.5
30.0
29.9
42.3
32.6
24.5
34.3
SWDA+ours
42.8
37.5
49.5
33.3
34.0
45.9
33.0
27.5
37.9
HTCN (Chen et al. HTCN+
2020
47.4
37.1
47.9
32.3
33.2
47.5
40.9
31.6
39.8
49.1
40.2
52.6
37.2
37.3
48.8
40.8
33.0
42.4
Oracle
53.7
41.4
53.3
39.1
38.6
49.5
38.7
35.8
43.8
Dissimilar Domains Adaptation. Adaptation between dissimilar domains has been difficult point in recent years. Models are always in trouble with the adaptation the domain shift increases. Followed Saito et al. (2019), we utilize Pascal VOC (Everingham et al. 2010) as the source domain and either the Clipartlk (Inoue et al. 2018) Watercolor (Inoue et al. 2018) target domain to further experiment on dissimilar domains. PASCAL VOC contains 20 common categories of images and their bounding box. Clipart K is cartoon image dataset which contains 1K images in total. It

---

Under review as a conference paper at ICLR 2022
has the same category PASCAL VOC, but there is large domain shift between the same category- Watercolor similar to Clipart which contains categories in common with PASCAL VOC and 2K painting images in total. ResNet101 (He et al. 2016) pre-trained ImageNet (Krizhessyy et al. 2012) is adopted as our backbone network.
1) Results Clipart. Table shows the results of adaptation from PASCAL VOC to C It can be observed that our method improves SWDA and HTCN by 4.5% and 3.0% respectively. The superiority of the proposed method demonstrate the effectiveness of feature fusion even dissimilar domains. In addition, we noticed that the performance of some classes is lower than baseline. We suppose it is mainly due to the difference of inter-class feature distribution. As the changes of object shape, size, texture, etc.. its challenging to use the image-level feature statistics to represent the discriminative information of all categories of objects accurately. But for most categories, our method is effective and can bring performance improvements.
Table 2: Results (%) on adaptation from PASCAL VOC -> Clipart Dataset. ResNer101 is used as the backbone network.
Method
Jerro beyele bird beat bottle
bus
car
cal
chair
cow
table dog
hus
bike
pron plst
sheep
sota
train
tv
ImAP
Only
33.6
24.3
20.0
43.9
10.7
30.6
11.7
36.8
7.3
27.8
SCL Dong el al. 2020
33.4
49.2
36.0
27.1
38.4
55.7
18.7
15.9
39.0
59.2
18.8
23.7
36.9
70.0
60.6
49.7
25.8
34,8
47.2
51,2
40.6
CRDA
28.7
31.8
40.1
63.6
49.3
33.3
61.3
22.3
24.3
DAA
eu
2020
35.0
59.5
34.6
30.2
38.1
60.2
40.2
20.5
39.3
58.5
26.4
22.8
33.8
82,9
64.4
48.8
18.0
28.6
57.6
46.2
42.3
SWDA
Saato et
26.2
32.6
38.5
54.3
34.8
58.3
17.0
33.8
52.0
9.3
249
38.1
SWIIA+our
40.1
62.6
27.0
39.1
42.9
65.7
39.6
17.1
36.3
62.1
18.0
24.2
44.7
85.9
64.2
41.7
33.3
22.8
35.9
48.9
42.6
HTCN
(Chen al. 2020
33.6
58.9
34.0
23.4
45.6
57.0
39.7
51.3
65.0
19.3
30.1
40.3
HTT.Nsmurr
43.6
58.8
32,1
39.6
39.8
68.5
45.3
11.4
38.1
62.4
20.9
30.3
46.7
76.5
62.9
39.3
33.6
23.5
48.
45.3
43.3
2) Results on Watercolor. According to Table 3 we observe that proposed method reaches the
mAP of 57.0% and 55.2% with a gain of 3.7% and 2.0% over the baseline on the adaptation from PASCAL VOC Watercolor. The fact indicates that our method help the baseline model capture the style information of Watercolor and converge to better performance.
Table 3: Results (%) on adaptation from PASCAL VOC to Watercolor Dataset. ResNet101 used as the backbone network.
Method
|eycle bird
cat
dog prsn
mAP
Source Only
68.8
46.8 37.2 32.7 21.3 60.7
44.6
DA-Faster
Chen et al.
2018b)
75.2
40.6
48.0 31.5 20.6 60.0
46.0
DAA
Fu
2020
85.1
56.6
46.2 39.9 36.9 65.6
55.1
CDG
ct al.
2021a
97.7
53.1
52.1 47.3
38.7 68.9
59.7
SWDA Saito al.
2019
82.3
55.9
46.5
32.7
35.5 66.7
53.3
SWDA+ours
84.8
51.6
51.4
47.9 38.9 67.3
57.0
HTCN (Chen et al. HTCN+
2020
77.4
52.2
56.1
36.3
33.5
63.4
53.2
76.2
52.2
52.2
45.9
37.5
66.9
55.2
4.3 FURTHER EMPIRICAL ANALYSIS
Ablation Study, In this session, we conducted ablation experiments to validate the effectiveness of proposed method. Firstly, we explore the role each component we proposed. Specifically, we append each module gradually and observe the performance of the model on Normal- The results are shown in Table It can found that the performance increases when any component appended, which proves that all component are designed reasonably and contribute to the adaptation. Secondly, in order to verify that discriminative feature mining helpful to improve the stability and invariance of the model, we adjust the weight À in Eq 3 to control the degree of feature style transfer. We change from 0 to 1 gradually and observe the performance
the model on Normal -oogy. As shown in Figure (a), we find that the mAP continuously increases with the style weight X, which proves the vahdity of DAdaIN and highlights the role of discriminative feature in domain adaptation. Thirdly, evaluate the effects of the parameter a and / in Eq 9 which balances the
8

---

Under review as a conference paper at ICLR 2022
Table 4:
Ablation of proposed method Cityscapes Foggy Cityscapes. DIN represents the proposed DAdaIN, and Ss, SC, LC indicate style similarity alignment, style consistency alignment, and local content alignment.
Method
DIN
SS
SC
I.C
mAP
Gain
HTCN
39.8
40.3
0.5
40.4
0.6
40.8
1.0
Proposed
41.2
1.4
41.5
1.7
41.7
1.9
42.4
2.6
APg50
ARg75
$1.544
KO. Maat
Model
(a) Discriminant control
(b) Parameter analysis
(c) Average discrepancy
Figure 3: Analysis of discriminative feature and hyper-parameee on multi task.
contribution GSA and LCA, respectively. As shown in Figure 3 (b), we change the parameters and compare method with the baseline HTCN different scenarios, which drawn with solid and dashed lines. We can find the performance first rises and then drops with the increase of a and 8, which proves that proper trade-off can help boost the performance.
Domain Discrepancy. To measure the cross-domain representation discrepancy, we introduce the proxy A-distance (Schikopf al.) which defined d,A 2(1 2e). The variable denotes the generalization error on the binary classification problem of discriminating the source and tar- get domain. As shown in Figure (c), compared to baseline model, our method further narrows the discrepancy between source and target domain, which demonstrate that discriminative features introduce key information that cannot brought by domain invariant features. Furthermore, we visualize the image features learned for the adaptation multi task using t-SNE (Van der Maaten Hinton 2008). Due to the limit of space, the results are shown in appendix.
Detection Examples. Figure 4 displays the qualitative detection results adaptation tasks. Our method outperform the Source Only and HTCN baseline method. Even for the corner cases, e.g. occlusion and small object, our method still makes accurate predictions and inhibits false positives, which proves that our approach successfully help the model focus on domain-speciff details.
CONCLUSION
this work, we propose a simple yet effective framework for cross-domain object detection. The proposed framework consists of three key components: DAdaIN, GSA and LCA. DAdaIN supple- ment the domain-specitn representation by restructuring the content and style information. GSA and LCA exploited to regularize the consistency of style and content between multi domains. Extensive experiments show that our method can be used as a complementary module to highlight target-like details and improve the performance of existing methods on several benchmark datasets.

---

Under review as a conference paper at ICLR 2022
(a) Source Only
(b) HTCN
(c) HTCN+OOrs
Figure 4: Qualitative detection results on the target domain. First and second rows: Normal-+Fogg! adaptation scenario. Third and fourth rows: BimlOK-CCtyssen adaptation scenario.
REFERENCES
Léon Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT pp. 177-186. Springer, 2010.
Woong-Gi Chang, Tackgeun You, Sconguk Sco, Suha Kwak, and Bohyung Han. Domain-specific batch normalization for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 7354-7367 2019.
Chaoqi Chen, Weiping Xie, Wenbing Huang, Yu Rong. Xinghao Ding, Yue Huang, Tingyang Xu, and Junzhou Huang. Progressive feature alignment for unsupervised domain adaptation. 2018a.
Chaoqi Chen, Zebiao Zheng, Xinghao Ding, Yue Huang, and Qi Dou. Harmonizing transferability and discriminability for adapting object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2020.
Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Lue Van Gool. Domain adaptive faster r-cnn for object detection in the wild. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 3339-3348, 2018b.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 22233,223, 2016.
Emily Denton, Soumith Chintala, Arthur Szlam, and Rob Fergus. Deep generative image models using a laplacian pyramid of adversarial networks. arXiv preprint arXiv: 2015.
Jiahua Dong, Yang Cong, Gan Sun, Yuyang Liu, and Xiaowei Xu. Cscl: Critical semantic-consisse learning for unsupervised domain adaptation. In European Conference on Computer Vision, pp. 745-762. Springer, 2020.
10

---

Under review as a conference paper at ICLR 2022
Vincent Dumoulin, Jonathon Shlens, and Manjunath Kudlur. A learned representation for artistic style. arXiv preprint arXiv:1610.0702. 2016.
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2): 303-338, 2010.
Basura Fernando, Amaury Habrard, Marc Sebban, and Tinne Tuytelaars. Unsupervised visual do- main adaptation using subspace alignment. In IEEE International Conference Computer Vi- sion, 2014.
Oriel Frigo, Neus Sabater, Julie Delon, and Pierre Hellier. Split and match: Example-based adaptive patch sampling for unsupervised style transfer. In IEEE Conference on Computer Vision & Pattern Recognition, 2016.
Minghao Fu, Zhenshan Xie, Wen Li, and Lixin Duan. Deeply aligned adaptation for cross-domain object detection. arXiv preprint (xXxx:0004000090 2020.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning. pp. 1180-1189, PMLR, 2015.
Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pp. 2414-2422, 2016.
Boqing Gong, Yuan Shi, Fei Sha, and Grauman. Geodesic flow kernel for unsupervised domain adaptation. In 2012 IEEE Conference Computer Vision and Puttern Recognition, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Zhenwei He and Lei Zhang. Multi-adversarial faster-renn for unrestricted object detection. 2019.
Judy Hoffman, Eric Tzeng, Taesung Park, Jun- Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In International conference on machine learning, pp. 1989-1998. PMLR, 2018.
Han Kai Hsu, Chun Han Yao, Yi Hsuan Tsai, Wei Chih Hung. and Ming Hsuan Yang. Progressive domain adaptation for object detection. In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), 2020.
Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance nor- malization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1501- 1510, 2017.
Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa. Cross-domain weakly- supervised object detection through progressive domain adaptation. In Proceedings of the IEÉE conference on computer vision and pattern recognition, pp. 5001-5009, 2018.
Maximilian Jaritz, Tuan-Hung Vu, Raoul de Charette, Émilie Wirbel, and Patrick Pérez. moosl learning for domain adaptation in 3d semantic segmentation. arXiv preprint arXiv:2101.07253, 2021.
X. Jin, C. Lan, W. Zeng, and Chen. Style normalization and restitution for domainneeneralizztion and adaptation. 2021.
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision, pp. 694-711. Springer, 2016.
D. Kotovenko, M. Wright, A. Heimbrecht, and B. Ommer. Rethinking style transfer: From pixels to parameterized brushstrokes. 2021.
11

---

Under review as a conference paper at ICLR 2022
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep volutional neural networks. Advances in neural information processing systems, 25:1097-1105, 2012.
Guangrui Li, Guoliang Kang, Wu Liu, Yunchao Wei, and Yi Yang. Content-consistent matching for domain adaptive semantic segmentation. In European Conference on Computer Vision, pp. 440-456. Springer, 2020.
Li, J. Q. Huang, X. Hua, and L. Zhang. Category dictionary guided unsupervised domain adaptation for object detection. 2021a.
S. Li, M. Xie, K. Gong. C. H. Liu, Y. Wang, and W. Li. Transferable semantic augmentation for domain adaptation. 2021b.
Yanghao Li, Naiyan Wang, Jianping Shi, Xiaodi Hou, and Jiaying Liu. Adaptive batch normalization for practical domain adaptation. Pattern Recognition, 80: 80:109-117, 2018.
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4):640-65 651, 2015.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. 2018.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis & Machine Intelligence,
2017.
Adrian Lopez Rodriguez and Krystian Mikolajczyk. Domain adaptation for object detection via style consistency. arXiv preprint 1XXi::9111110033 2019.
Paolo Russo, Fabio Maria Carlucei, Tatiana Tommasi, and Barbara Caputo. From source to target and back: symmetric bi-directional adaptive gan. 2017.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier dis- crepancy for unsupervised domain adaptation. In 2018 IEÉE/CVF Conference Computer Vi- sion and Pattern Recognition, 2018.
Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko. Strong-weak distribution align- ment for adaptive object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6956-6965, 2019.
Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Semantic foggy scene understanding with synthetic data. International Journal of Computer Vision. 126(9):973-992 2018.
Schlkopf, Platt, and Hofmann. Analysis of representations for domain adaptation.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. 2016.
Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprini arXiv:1412.3474 2014.
Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, and Victor S Lempitsky. Texture networks: Feed-forward synthesis of textures and stylized images. ICML, volume pp. 4, 2016.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Improved texture networks: Maximizing quality and diversity in feed-forwar stylization and texture synthesis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.
2017.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(1)), 2008.
12

---

Under review as a conference paper at ICLR 2022
V. S. Vibashan, P. Oza, V. A. Sindagi, V. Gupta, and V. M. Patel. Mega-cda: Memory guided attention for category-aware unsupervised domain adaptive object detection. 2021.
Hang Wang, Minghao Xu, Bingbing Ni, and Wenjun Zhang. Learning to combine: Knowledge aggregation for multi-source domain adaptation. In European Conference on Computer Vision, pp. 727-744. Springer, 2020.
Chang-Dong Xu, Xing-Ran Zhao, Xin Jin, and Xiu-Shen Wei. Exploring categorical regularization for domain adaptive object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11724-11733, 2020.
Siqi Yang, Lin Wu, Amold Wiliem, and Brian C Lovell. Unsupervised domain adaptive object detection using Computer Vision, 2020.
cyclic adaptation. In Proceedings of the Asian Conference on
Jaejun Yoo, Youngjung Uh, Sanghyuk Chun, Byeongkyu Kang, and Jung-Wor Woo Ha. Photorealistic style transfer via wavelet transforms. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9036-9043 2019.
Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschlger, and Susanne Saminger- Platz. Central moment discrepancy (emd) for domain-invarian representation learning. In Inter- national Conference on Learning Representations, 2017.
Ganlong Zhao, Guanbin Li, Ruijia Xu, and Liang Lin. Collaborative training between region pro- posal localization and classification for domain adaptive object detection. In European Confer- ence on Computer Vision, pp. 86-102. Springer, 2020.
Yangtao Zheng, Di Huang, Songtao Liu, and Yunhong Wang, Cross-domain object detection through coarse-to-fine feature adaptation. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 13766-13775, 2020.
Xinge Zhu, Jiangmiao Pang, Ceyuan Yang, Jianping Shi, and Dahua Lin. Adapting object detectors via selective cross-domain alignment. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 687-696, 2019.
Yang Zou, Zhiding Yu, BVK Kumar, and Jinsong Wang. Unsupervised domain adaptation for mantic segmentation via class-balances self self-training. In Proceedings of the European conference on computer vision (ECCV), pp. 289-305, 2018.
13

---

Under review as a conference paper at ICLR 2022
A
APPENDIX
A.1 MORE ABLATION STUDIES
In this session, we present the additional results with different settings of GSA and LCA in Table and 6 on the adaptation from Cityscape to Foggy Cityscape.
A.1.1 ANALYSIS OF GSA
As shown in Table 5 GSA-A applies smooth L1 loss on shallow features to regularize the style similarity. And GSA-B represents that the L2 loss is applied high-level features. We can find that the above methods achieve sub-optimal performance in the adaptation task.
Table 5: Results (%) on adaptation from Cityscape to Foggy Cityscape Dataset.
Method| bus bike car meycle prsn rider train truck mAP
Baseline 48.8 35.6 51.6
31.1
36.1 47.8 41.5 30.6
40.4
GSA-A 45.8 39.5 52.7
36.3
38.3 47.9 34.5
27.2
40.3
GSA-B 45.8 40.2 52.6
36.9
38.6
48.5
36.9
31.7
41.4
GSA 49.1 40.2 52.6
37.2
37.3 48.8
40.8
33.0
42.4
A.1.2 ANALYSIS OF LCA
Table d shows the results for three different variants of the LCA. LCA-A extracts the features in proposals generated by RPN rather than ground truth. LCA-B regularizes the content information after Rol alignment module. And LCA-C achieves content alignment based on the de-normalized features of instances between source and target domains.
It can be observed that the content alignment based on proposals will cause performance degradation. One possibility is that poor-performing networks causes RPN to produce more low- quality bounding boxes, which weakens the model's attention to the foreground instance. Moreover, the content alignment based on the de-normalized features can produce similar results to LCA, which prove the validity of local content alignment again.
Table 6: Results (%) on adaptation from Cityscape Foggy Cityscape Dataset.
Method| bus bike car mcycle prsn rider train truck imAP
Baseline 46.1 40.1 52.6
33.1
37.3 48.1 41.7 28.8
41.0
LCA-A
43.2 38.4 51.9
33.1
36.2 46.8 31.9 30.5
39.0
LCA-B
49.9
39.1 52.6
32.8
37.5
47.0
30.5
31.9
40.2
LCA-C
49.3 40.7
52.8
39.9
38.3
49.1
36.0
32.1
42.3
LCA
[49.1 40.2 52.6
37.2
37.3
48.8
40.8
33.0
42.4
A.2 MORE FEATURE VISUALIZATIONS
In order to further analyze the significance of characteristic statistics and the role of DAdaIN, we visualized the features and statisties in different adaptation scenarios. We applied global average pooling on the low-level features extracted by different models. As shown in [ our method ef- fectively reduce the domain gap from the Oracle, which proves our method can help the model to extract more domain-specifte features and improves the invariance of the model for domain shift. From Figure 6 we can observe that: 1) The distribution of the features is similar to their statisti- cal values. which proves that the statistics have certain statistical significance. 2) After applying DAdaIN, the difference of feature distribution between the intermediate and target domain is further optimized, which demonstrate the validity of DAdaIN.
14

---

Under review as a conference paper at ICLR 2022
Source only
HTCN
HTCN+Ours
Oracle
Figure 5: The t-SNE visualization of image features obtained from different models. Colors repre- sent different models. We can see that the difference of feature distribution is further optimized.
Source only
Tbrget only
Mine
Figure 6: Visualizations of shallow features and statistics obtained from different models. "Source only" denotes the baseline model trained and tested only in source domain without adaptation. "TTa- get only" represents the model trained and tested in target domain without adaptation. And "Mine" represents the model trained with proposed method. It should be noticed that, when testing method, we retain the DAdaIN module and extract the synthesis feature in intermediate domain to compare the feature distributions with source and target domain. First Row: The statistics of shal- low features. Second Row: The shallow features obtained by applying global average pooling to the shallow features. Each column (from left right) represents the model trained on Cityscape to Foggy Cityscapes, SIM10K to Cityscape, PASCAL VOC to Clipart, PASCAL VOC to Watercolor.
15

---

Under review as a conference paper at ICLR 2022
A.3 MORE DETECTION RESULTS
We show more detection results on PASCAL VoC-aClipart and PASCAL Wat--aatrrooo... As shown in Figure 7 our method promotes accurate results than others, which prove the gener- alization of our method under different tasks.
tor
persa
Cnale
(a) Source Only
(b) HTCN
(c) HTCN+Ours
Figure 7: Qualitative detection results the target domain. First and second rows: PASCAL
VOC CCC-pCl adaptation scenario. Third and fourth rows: PASCAL VOC-> Watercolor adaptation scenario.
16


Reviews:
[{"review_id": "qSoC9u7-A3", "reviewer": "Reviewer_5g9D", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "+: The key assumption of the proposed method is reasonable, however, it lacks comparison to other methods under similar assumptions.\n\n+: Experimental results show good performance of the proposed method.\n\n+: It provides ablation study to show the effectiveness of different modules.\n\n-: The core idea of the proposed method is to highlight the target-like discriminative feature, what is its difference to the self-training widely used in UDA, which after global aligning the source and target domains, selects pseo-samples for discriminative training in the target domain. Meanwhile, there is no discussion on related works of this aspect.\n\n-: Experiments could be improved. The proposed method improves over SWDA and HTCN, however, they are not the SOTA. If the proposed method can be implemented on MeGA, it would be better. As for comparison of DA methods, the authors missed the self-training based DA methods, which also address the issue of local alignment/target discriminative.\n\n-: The performance among different methods differs with respect to different adaptation settings, so, does the ablation study conducted on Normal -> Foggy is general enough for other adaptation scenarios? The difference can also be observed in Fig.3(b), where best parameter settings are different for different scenarios.\n\n-: Some statement needs further clarification or validation. \n(a) For the last sentence in the first paragraph of Section 1. it would also be helpful to show how many annotated images in the target domain are required (the same model trained on the source domain and finetuned in the target domain) to achieve the same performance as the best UDA methods. This will be a strong support to this argument. Otherwise, if the annotated images are few, the cost would not be high. Studying UDA for object detection would be less meaningful.\n(b) In the end of page one, it mentions \"it may implicitly drive networks to pay more attention on shared attributes and ignore the domain-specific feature\", this could also be the problem of DA for other tasks. What is the uniqueness of DA for object detection? In other words, why should we focus on DA for object detection with the proposed method?\n(c) the key assumption is that \"image-level style and instance-level content\" are important and can be decoupled. Is there any evidence to support this assumption. Another question is that  if it is still effective if we align instance-level and make the image-level keeping discriminative.\n\nminor issue\nthe illustration in Fig.1 between DA and DA+Ours is hard to understand and less informative.", "summary_of_the_paper": "This paper proposed a method for UDA object detection. The core idea is to increase the discriminative ability of learned features in the target domain while still globally aligning the feature distributions of the source and target domains. Under this motivation, the paper proposed modules of domain adapative instance normalization, global style alignment and local content alignment. Experiments are conducted on several benchmark adaptation scenarios, showing improvments on two existing methods.", "summary_of_the_review": "The idea of the proposed method seems to be effective as supported by the experiments, however, it lacks disccusion and comparison to self-training methods that are based similar motivation, i.e., globally and locally alignment should be achieved simultaneously to achieve good DA performance. There are also many unclear issues that require further validation. Therefore, the paper is currently with more weaknesses compared to its strengths.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "e-cZ-lROTjL", "reviewer": "Reviewer_VfW6", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n(1) The paper is well written and easy to understand.\n(2) The extensive experiments show the effectiveness of their proposed modules.\n\nWeakness:\n(1) The unclear motivation. The paper points out that image-level style and instance-level content matter in cross-domain detection. However, I do not understand the explanation that \"The style information should be globally consistent within its own domain while local homogeneous content information between different domains ought to be analogous, and vice versa\". In my opinion, the better extraction for the domain-specific features could lead to better decoupling the domain-invariance and domain-specific features. Thus, the domain-invariance feature could be enhanced. It is just my personal viewpoint and might be wrong. Please the author clarifies my questions.\n\n(2) Some missing illustrations for the framework. For example, what's G_1 and G_2? Where is the L_adv in the Figure2?\n\n(3) Is it wrong in the last row of Figure4? the nearest cyclist is not detected. ", "summary_of_the_paper": "This paper proposes a novel adaptation framework to balance transferability and discriminability for cross-domain object detection. Concretely, they focus on domain-specific features thus manage to reconstruct the style and content of features by AdaIN and some alignment techniques. Extensive experiments show the performance improvement by proposed modules. \n\n", "summary_of_the_review": "Overall, I am wondering about the entire motivation of this paper. I acknowledge the strength of the proposed modules. However, these modules lack some novelties in my opinion. They are like the combination of some typical modules in a generative model. Thus I tend to choose marginally below the acceptance threshold.\n\nI would like to upgrade the rating if the author addressed my concerns.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KimX2sPaxgQ", "reviewer": "Reviewer_FLN9", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper tackles a quite interesting and challenging problem that has received limited attention by the community in the recent past. The idea of building upon the concepts of style transfer to capture and encode domain-specific and domain-invariant features is interesting and sounding. I also appreciated the fact that the made choices seems to be properly motivated. Despite such positive aspects, the paper seems to have some consistent presentation issues that make it hard to read and properly follow. There are unclear notations and cryptic sentences that act against a proper understanding of the proposed approach. There are also important claims that are not properly supported by empirical evidence.\n\n\nPositive aspects:\n+ The paper tackles a quite interesting and challenging problem that has received limited attention. The problem of being able to perform UDA for detecting objects belonging to the same classes under different domains has severe impacts on many applications ranging from security to autonomous driving, etc. The motivations behind the work seem to be solid and properly defined.\n+ Building upon the style transfer ideas is an interesting approach. Leaving out more complex adversarial schemes to encode the domain-specific and domain-invariant features in source and target domain via an end-to-end trainable solution is quite appealing. The \"simplicity\" of the approach and the fact that it can be added to other existing methods are relevant points in favor of the proposed work.\n+ To show the benefits of the proposed approach, 3 benchmark UDA setups have been considered. The experimental results section report on the comparison with very recent works on the 3 setups, also considering the same/similar models to provide a fair comparison. \n\nNegative aspects:\n- First and foremost, the paper contains typos and grammar problems that make reading difficult, thus acting against a proper understanding of the key ideas and techniques proposed by the paper. The text is often leaving out relevant technical concepts, thus failing to provide clear explanations. The adopted notation is confusing. For instance:\n    - in section 3 the text refers to $G_1$ and $G_2$ whose definitions and descriptions are missing. There is evidence of these in Figure 2 but no clear explanation on what these are in practice. Same applies for the various $f(\\cdot)$'s. It would be quite helpful if a more appropriate description is provided by following figures that would help to understand \"what\" goes \"where\" and \"how\" it is used.\n   -  in section 3, when presenting the proposed DAdaIn layer, it seems that the function is depending on four inputs, with two of which represent the feature embedding from source and target, and the remaining two (i.e., $\\kappa$'s) being hyperparameter controlling the translation scale and bias. It is not clear how these two last ones are obtained.  \n    - When presenting the achieved results, there are unclear subdivisions in the Tables. What do the double lines splitting the works used for comparison mean? No description is present in the caption nor in the text. Please clarify.\n- There are claims in the paper that are not properly supported by experimental results or empirical evidence: while describing the local content alignment, the text states that \"we find that strong local feature alignment will cause instability during training.\". The text continues with: \"GSA tends to result in negative style transfer because of the poor-performing Convolutional Neural Networks.\". These are all strong, yet not properly supported, statements. Given the importance of these claims, it is highly recommended that appropriate experimental results are conducted (or results reported if already done).\n- An obscure part of the paper regards the applicability of the proposed approach to existing methods. The text mainly refers to the faster R-CNN model with a VGG/ResNet backbone, then to the HTCN and SWDA methods, that, per-se are already doing domain adaptation for the object detection problem. A more detailed description of how the proposed method works alongside these ones is fundamental.\n", "summary_of_the_paper": "The paper introduces a method to tackle the unsupervised domain adaptation problem, focusing on the object detection problem. Towards the final objective, an architecture building on the style transfer concepts is proposed. The main idea is to build upon the AdaIN ideas to encode the domain-specific and domain-invariant features in the source and target domain. Two additional modules are exploited to regularize the consistency of style and content between multiple domains. Experimental results on 3 benchmark UDA setups are conducted. Similar/better results than existing approaches are obtained.", "summary_of_the_review": "In light of the aforementioned consideration, this reviewer believes that the paper has some merits but, the lack of a proper presentation, and the fact that there are not enough experimental results to support all paper claims, result in a submission that is not solid enough to justify an ICLR publication. These are two very important weaknesses that are likely to require more than a single major review round before the paper can have a proper shape. Thus, this reviewer is considering the submitted work below the acceptance bar but would be happy to change the opinion in light of more/proper evidence.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "24WVrF4DV16", "reviewer": "Reviewer_1yU8", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n + This paper introduces a new direction for cross-domain object detection. This is an important issue given the fact that “domain shift” occurs widely in the real-life applications and annotations are very costly to get. The proposed method borrows idea from style transfer and tries to reconstruct the content and style information without the requirement of new annotations. I feel this idea is interesting. \n\n + The proposed method is technically sound. Global style alignment and local content alignment have been carefully considered. I don't find any significant technical flaws in the technical part.\n\n + The proposed method achieves promising results across multiple benchmarks. \n\nWeaknesses:\n\n-The weakness part is that the proposed method generally adopts existing techniques and combines them to deal with the cross-domain object detection. It is still hard for me to find some novel technical contributions. I like the key idea, motivations and writing, yet I feel its technical contributions are still incremental. For example, the style feature fusion is from Huang&Belongie(2017), the global style alignment seems borrowed from Saito et al(2019) and local content alignment shares the similar idea with Zhu et al(2019). It would be great if the authors can further highlight/enhance the novel part to make the paper more solid. \n\n-While the idea of learning domain-specific representation information is interesting and intuitively sensible, it seems speculative to me. It is difficult for readers to know what information has been captured. Some visualization or a more carefully targeted experiment showing what are in the domain-specific features are needed in order to justify the explanation that the authors give for their method.\n\n-The overall performance is acceptable yet not impressive. The proposed approach achieves only slightly better performance than other approaches as shown in Table 2, or lower accuracy than SOTA method (Table 3). Can the authors further provide the FLOPs, memory costs and parameters of all compared methods to better show the superiority of the proposed approach compared to other alternatives.", "summary_of_the_paper": "This paper presents a new method for cross-domain object detection, which employs the style-aware feature fusion method and two novel modules to build the model. The key idea of this paper is to assign the underlying details to the model by using techniques from style transfer to overcome the domain gap. The proposed method is evaluated on multiple datasets and achieves promising results.", "summary_of_the_review": "This paper introduces a new approach for cross-domain object detection but is somewhat weak in technical contributions.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a method for unsupervised domain adaptation, focusing on the object detection problem. Under this framework, the paper proposes modules of domain adaptive instance normalization, global style alignment and local content alignment. The proposed method is evaluated on multiple datasets.

Several reviewers have pointed out that the paper lacks discussion and comparison to related methods. The paper has some merits but, the lack of a proper presentation, and the fact that there are not enough experimental results to support all claims in the paper, result in a submission that does not meet the bar of ICLR publication. Hence, the current paper is recommended to be not published at ICLR.
Decision: Reject