Title: DA-Ada: Learning Domain-Aware Adapter for Domain Adaptive Object Detection
Venue: NeurIPS 2024
Accepted: yes
Keywords: Unsupervised domain adaptation; Object detection; Visual-language model
Abstract: Domain adaptive object detection (DAOD) aims to generalize detectors trained on an annotated source domain to an unlabelled target domain.
As the visual-language models (VLMs) can provide essential general knowledge on unseen images, freezing the visual encoder and inserting a domain-agnostic adapter can learn domain-invariant knowledge for DAOD.
However, the domain-agnostic adapter is inevitably biased to the source domain.
It discards some beneficial knowledge discriminative on the unlabelled domain, \ie domain-specific knowledge of the target domain.
To solve the issue, we propose a novel Domain-Aware Adapter (DA-Ada) tailored for the DAOD task.
The key point is exploiting domain-specific knowledge between the essential general knowledge and domain-invariant knowledge.
DA-Ada consists of the Domain-Invariant Adapter (DIA) for learning domain-invariant knowledge and the Domain-Specific Adapter (DSA) for injecting the domain-specific knowledge from the information discarded by the visual encoder.
Comprehensive experiments over multiple DAOD tasks show that DA-Ada can efficiently infer a domain-aware visual encoder for boosting domain adaptive object detection.
Our code is available at https://github.com/Therock90421/DA-Ada.

DA-Ada: Learning Domain-Aware Adapter for Domain Adaptive Object Detection

Abstract

Domain adaptive object detection (DAOD) aims to generalize detectors trained on an annotated source domain to an unlabelled target domain. As the visual-language models (VLMs) can provide essential general knowledge on unseen images, freezing the visual encoder and inserting a domain-agnostic adapter can learn domaininvariant knowledge for DAOD. However, the domain-agnostic adapter is inevitably biased to the source domain. It discards some beneficial knowledge discriminative on the unlabelled domain, i.e. domain-specific knowledge of the target domain. To solve the issue, we propose a novel Domain-Aware Adapter (DA-Ada) tailored for the DAOD task. The key point is exploiting domain-specific knowledge between the essential general knowledge and domain-invariant knowledge. DA-Ada consists of the Domain-Invariant Adapter (DIA) for learning domain-invariant knowledge and the Domain-Specific Adapter (DSA) for injecting the domain-specific knowledge from the information discarded by the visual encoder. Comprehensive experiments over multiple DAOD tasks show that DA-Ada can efficiently infer a domain-aware visual encoder for boosting domain adaptive object detection. Our code is available at https://github.com/Therock90421/DA-Ada.

1 Introduction

Object detection [52, 51, 41, 12] have achieved remarkable performance, but suffer severe performance drop when dealing with unseen data due to domain discrepancy. To alleviate this problem, domain adaptive object detection (DAOD) [7] is explored to transfer a detector trained on the labelled source domain to a unlabelled target domain. Traditional DAOD works [7, 68, 82, 65, 78, 37, 71, 25] generate the domain-aligned feature via fine-tuning the backbone, as depicted in Fig. 1(a). Nevertheless, it is easily biased towards the source domain since a considerable number of parameters need to be updated with the annotations only from the source domain.

Recently, applying prompt tuning [81, 80, 73] on visual-language models (VLMs) is widely used for two reasons: 1) few parameters need to be learned; 2) VLMs trained on large-scale image-text pairs can extract highly generalized features. Recent works [31, 57] have explored using prompt tuning to generate the domain-aware detection head for DAOD. However, they all extract the visual feature

finetune

Global-

LTask

LAdv

Agnostic

LTask

+

Specific

65.6

66.2

-65.8

w/oAdapter

LTask

LAdv

ADD

ADD

Learnable

Frozen

62.0

Domain-aware Adapter

Domain-agnostic Adapter

Adapter

-Oracle

Block Output

+

Block Output

Block Output

SUB

57.1

57.5

Backbone

VisualEncoder

Block

Adapter

VisualEncoder

Block

ADD

LAdv

52.6

53.8

Adapter

Invariant

BlockInput

BlockInput

BlockInput

SourceDomain

Target Domain

(a)Traditional Methods(b)Domain-agnostic Adapter

(c)Domain-awareAdapter

(d)mAP(%)Comparison

from the image with a frozen visual encoder, ignoring learning task-related knowledge and limiting the improvement of visual features' discriminative capabilities.

To inject the task-related knowledge into the visual encoder, some methods [24, 8, 5] insert an adapter module into the frozen backbone. Formally, a adapter shared across domains can be straightly introduced to learn the task-related knowledge with the annotations of source domain and domainaligned constraint, as shown in Fig. 1(b). However, this adapter is domain-agnostic and can only learn domain-invariant knowledge between the two domains under the domain-aligned constraint. Besides, the domain-invariant knowledge is inevitably biased to the source domain, since the annotations are only from the source domain. As shown in Fig. 1(d), compared with the original VLM, the domain-agnostic adapter brings significant improvement to the source domain, while the improvement of the target domain is limited. Summarily, the bias of the domain-invariant knowledge learned from domain-agnostic adapter limits the generalization to the unseen target domain.

Trained on large-scale data, the VLM provides essential general knowledge on unseen images, while the learned domain-invariant knowledge biased to the source domain shows limited improvement on the target domain. Consequently, when transferring essential general knowledge to the domaininvariant knowledge, the domain-agnostic adapter discards some beneficial knowledge on the target domain. Basically, it discards the domain-specific knowledge that distinguishes the target domain but is different from the domain-invariant knowledge. In summary, adding a complementary adapter to capture the target-specific knowledge from the discarded knowledge between the essential general knowledge and domain-invariant knowledge is an effective way to boost the performance of the VLM in DAOD task.

In this paper, we propose a novel Domain-aware Adapter (DA-Ada) to facilitate the visual encoder learning the domain-specific knowledge along with the domain-invariant knowledge. Formally, DA-Ada introduces a Domain-Invariant Adapter (DIA) and Domain-Specific Adapter (DSA) to exploit domain-invariant and domain-specific knowledge, respectively, as shown in Fig. 1(c). The DIA is attached to the block of the visual encoder in parallel and optimized by aligning the feature distribution of two domains to learn domain-invariant knowledge. The DSA is fed with the difference between the input and output of the block to recover the domain-specific knowledge discarded by the DIA. Since the difference represents the feature discarded by the block, the discarded knowledge between the essential general knowledge and domain-invariant knowledge is also hidden in the difference. Hence, the DSA can regain the domain-specific knowledge from the difference adaptively to improve the generalization ability on target domain, as shown in Fig. 1(d). Moreover, we propose the Visual-guided Textual Adapter (VTA), embedding cross-domain information learned by DA-Ada into textual encoder to enhance the discriminability of detection head. Overall, the proposed DA-Ada can inject domain-invariant and domain-specific knowledge into VLM for DAOD.

We conduct evaluations on mainstream DAOD benchmarks: Cross-Weather (Cityscapes → Foggy Cityscapes), Cross-Fov (KITTI → Cityscapes), Sim-to-Real (SIM10K → Cityscapes) and Cross-Style (Pascal VOC → Clipart). Experimental results show that the proposed DA-Ada brings noticeable improvement and outperforms state-of-the-art methods by a large margin. For example, DA-Ada reaches 58 . 5% mAP on Cross-Weather, surpassing the state-of-the-art DA-Pro [31] by 2 . 7% .

2 Related Work

Visual-Languague models Visual-language models (VLMs) [50, 33, 34] embed visual and text modalities into a shared space, enabling cross-modal alignment. Pre-trained with an astonishing scale of image-text pairs, they demonstrate comprehensive visual understanding. CLIP [50] simultaneously trains a visual encoder and a textual encoder with 400 million image-text pairs, showing promising performance on both the seen and unseen classes. Furthermore, [79, 19, 14, 61] distill the knowledge from the visual encoder of CLIP into the detection backbone and transform the textual encoder into detection head. Considering strong generalization, we apply RegionCLIP [79] as the detector.

Domain Adaptive Object Detection (DAOD) aims to adapt the object detector [52] trained on the labelled source domain to the unlabelled target domain. Previous approaches can be broadly divided into two orthogonal categories: feature alignment and semi-supervised learning. Feature alignment [45, 76, 46, 72, 56, 15, 47, 30, 76] aims to align the feature distributions of the two domains with domain discriminators [7], to generate domain-invariant knowledge in three levels: image-level [7, 68, 65, 40], instance-level [77, 53] and category-level [62, 25, 38]. To prevent knowledge unique to each domain from interfering with alignment, recent works [55, 2, 1, 32, 66] propose multiple extractors[42, 40, 63, 67, 27] and discriminators [69] to decouple the domaininvariant and domain-specific knowledge. In parallel, semi-supervised learning strives to augment training data with style transfer [43, 23, 10] and pseudo label [59, 39, 6, 11]. However, applying existing DAOD method to VLM would overfit the model to the training data, compromising the generalization of pre-trained models. To preserve the pre-trained knowledge, we opt to freeze the VLM and devise a novel domain-aware adapter to facilitate cross-domain adaptation. Compared with existing decoupling methods that only use domain-invariant features for detection, our method adopts a decoupling-refusion strategy. It adaptively modify domain-invariant features with domain-specific features to enhance the discriminability on the target domain.

Tuning method for VLM Adapting pre-trained VLM to downstream tasks via global finetuning is prohibitively expensive and easily overfitted to training datasets. To solve this issue, prompt tuning [81] replaces the hand-crafted prompts with the learnable tokens for the textual encoder. Conditions like categories [80, 74], human prior [73] and domain knowledge [31] are attached to attain robust performance on new tasks. However, they freeze the visual encoder, preventing it from learning cross-domain information for DAOD.In parallel, originated from Natural Language Processing (NLP)[24, 49, 64, 85, 21], adapter tuning inserts learnable small layers into the visual encoder so that the backbone can learn knowledge from new tasks. ViT-Adapter [8] and ConvAdapter [5] are proposed to efficiently transfer pre-trained knowledge to zero or few-shot visual tasks. [17] integrates the adapter into the CLIP model, and [58] further analyzes the components to be frozen or learnable. [48] combines self-supervised learning to enhance the ability to extract low-level features. Recent [70] explore injecting task-related knowledge into segmentation model SAM [29]. However, tuning the adapter directly on both domains will bias it towards the source domain and fails to distinguish domain-specific knowledge, leading to insufficient discrimination on the target domain. In this paper, we propose a novel domain-aware adapter that explicitly learns both domain-invariant and domain-specific knowledge to inject cross-domain information into the visual encoder.

3 Methodology

In this section, we present a novel Domain-aware Adapter (DA-Ada) tailored for DAOD. DA-Ada employs adapter tuning to introduce both domain-specific and domain-invariant knowledge into VLM. It is worth noting that the proposed method can be attached to any CNN-based detectors as a plug-and-play module. Without loss of generality, we take vanilla Faster-RCNN [52] as an example.

3.1 Overview

Inspired by adapter tuning, we can custom learnable adapters to inject cross-domain information into the visual encoder. Specifically, to enrich the extracted features with high domain generalization capabilities, an ideal adapter should satisfy conditions from the following two aspects. First, it can model the commonalities between the source and target domains, i.e. domain-invariant knowledge. Second, it can adaptively supply the unique attributes of each domain, i.e. domain-specific knowledge.

Both Domain

Source Only

Target Only

BBOX

SourceClassifier

Frozen

Learnable

Learnable

SourceProposal

Visual

Visual

Visual

CLIP

LabelledSource data

Stem

Encoder

Encoder

Encoder

VTA

Textual

Block 1

Block2

Block N

Encoder

TargetProposal

UnlabelledTargetdata

DA-Ada 1

DA-Ada 2

DA-AdaN

Target Classifier

(a)Overview of DA-Ada

F;(hj-1)

Laia

DIA

PS(vS)

hj-1

h

DSTA

CLIP

→P(vS)

Textual

P(vT)

Encoder

hi-1

S3

D

h

DITA

PT(vT)

DSA

DSTA

Ci

(b)Domain-aware Adapter Module (DA-Ada)

(c) Visual-guided Textual Adapter (VTA)

In this perspective, we design an effective Domain-Aware Adapter (DA-Ada) consisting of a DomainInvariant Adapter (DIA) and a Domain-Specific Adapter (DSA). As shown in Fig. 2(a), given input image x , we split the visual encoder into N blocks { F i } N i =1 by feature resolutions ( N = 4 in ResNet). Then we attach N blocks with DA-Ada modules { A i } N i =1 in Fig. 2(b):

where S denotes the stem layer. For the i -th DA-Ada module, we first feed the i -th block's input h i -1 into the i -th DIA module A I i to extract the domain-invariant features h I i . Then we attain the domain-specific features h S i from the subtraction of h i -1 and h I i by the DSA module A S i :

After that, we fuse h I i , h S i with spatial attention to output h i for i -th block:

where · denotes the element-wise Hadamard product. With N learnable adapters, we obtain visual embedding v = h N for subsequent detection. As the visual embedding contains sufficient crossdomain information, we propose the Visual-guided Textual Adapter (VTA), projecting the visual embedding to the textual encoder to enhance the discriminability of the detection head. As shown in Fig. 2(c), the visual-guided textual adapter uses the visual embedding v S , v T to infer textual embedding e S , e T on source and target domain, which is utilized for prediction. Overall, the proposed DA-Ada can inject domain-invariant and domain-specific knowledge into VLM to improve cross-domain generalization ability.

3.2 Domain-Invariant Adapter (DIA)

The DIA module is proposed to inject the domain-invariant knowledge into the visual encoder. As shown in Fig. 2(b), it applies a bottleneck to learn multi-scale domain knowledge, and the output distribution is aligned between domains for extracting domain-invariant knowledge. Specifically, for the i -th block of the visual encoder, we first forward the input feature h i -1 ∈ R b × c × h × w to the i -th DIA and filter domain-irrelevant information with embedding block E I :

After that, the embedding h E i ∈ R b × c × h × w is helpful for domain representation learning. Low channel-dimensional features have less information redundancy and are more suitable for domain adaptation than high-dimensional ones. Following this spirit, the embedding is encouraged to be down-projected to a low channel-dimensional vector h L i ∈ R b × r × h × w to extract domain-invariant

knowledge and filter redundant information. Formally, a down-projection C D is applied to reduce the dimension to r :

Considering that the scale of objects varies between domains, we introduce M down-projectors { C D i } i M =1 with different receptive fields, enabling it to capture various spatial features across multiple scales. Specifically, the embedding h E i = [ h E i, 1 , h E i, 2 , ..., h E i,M ] is first split up evenly in the channel dimension. Then, each partition is resized to different resolutions and down-projected. Therefore, the multi-scale version of Eq. (5) is expressed as:

Furthermore, the low-dimensional knowledge h L i is encouraged to be mapped back to the original dimensional feature space and supplemented to the pre-trained features. Typically, we apply the dimension-raising function C U on h L i to extract domain-invariant knowledge for the visual encoder.

where A I i ( h i -1 ) ∈ R b × c × h × w is output of the i -th DIA, and will be summed with F I i ( h i -1 ) to attain domain-invariant feature h I i in Eq. (2). To ensure DIA learning domain-invariant knowledge, the h I i is expected to be well aligned between the two domains. Therefore, N domain discriminator { D i } N i =1 is attached to each h I i to calculate adversarial loss L dia . We will introduce this loss in Sec.3.5.

With the combination of dimensional reduction-increase processes and the constraints of detection and adversarial loss, the DIA can extract domain-invariant features while reducing redundant features.

3.3 Domain-Specific Adapter (DSA)

Adapted with DIA, the essential general knowledge of the frozen VLM is transferred to domaininvariant knowledge. However, the knowledge learned only through the DIA is biased towards the source domain and appears less discriminative on the target domain. Considering the high generalization of essential general knowledge of the frozen VLM, we attribute this problem to the fact that the DIA discards some domain-specific knowledge that is highly generalizable on the unlabelled target domain. Since the difference between the input and output of the block denotes the discarded feature, the discarded domain-specific knowledge is also hidden in the difference. To this end, we propose the DSA module to recover domain-specific knowledge from the difference.

After the DIA injects the domain-invariant knowledge into the visual encoder, the domain-specific knowledge unique to the target domain is discarded by the output h I i . Therefore, we first obtain the feature h D i discarded by the visual encoder block from the difference of the input h i -1 and h I i :

where E S is similar with the embedding block E I . As domain-specific knowledge is hidden in the discarded difference h D i , a bottleneck architecture is employed for adaptive knowledge extraction:

where C D ′ , C U ′ follow the same configurations as C D , C U to perceive multi-scale domain-specific knowledge in bottleneck manner.

Generally speaking, domain-invariant knowledge dominates the process of transferring essential general knowledge of the VLM, while domain-specific knowledge fine-tunes this process based on the characteristics of each domain. To this end, it is a more reasonable way to adaptively supplement domain-specific knowledge with the extracted h S i through pixel-level attention rather than straightforward addition. Therefore, the injection of the whole DA-Ada is written as Eq. (3).

3.4 Visual-guided Textual Adapter (VTA)

With the domain-aware adapter to inject domain-invariant and domain-specific knowledge, the extracted visual feature shows rich discriminability, which can also be used to improve detection

TextualVectors

Encoder

Textual

VisualVectors

Encoder

Textual

Learnable

LearnableTextual

Visual-guided

Description

Adapter

Textual

Source

Visual

Detection

[Class]

Source

Target

Visual

Detection

[Class]

Image

Image

Encoder

Head

[Bbox]

Image

Image

Encoder

Head

[Bbox]

(a)

DA-Pro

(b)

head. Therefore, we introduce the VTA to exploit the cross-domain information contained in the visual features to enhance the textual encoder.

In order to fully exploit the domain-invariant and domain-specific knowledge extracted by the DAAda module, we equip two learnable components for VTA: the domain-invariant textual adapter P (DITA) and the domain-specific textual adapter P S , P T (DSTA) shown in Fig. 2(c). The DITA is shared across domains to encode visual domain-invariant knowledge into the input of the textual encoder, optimized by a domain discriminator D P . The DSTA is tailored to further supplement domain-specific knowledge for the source domain S and the target domain T . In practice, the structure of DITA and DSTA is a 3-layer MLP with a hidden dimension of 512, projecting visual embeddings into 8 tokens for the textual encoder. Formally, the VTA embeds visual information into the textual embedding,

where v S , v T , i and c i denote the visual embedding from domain S, T , the i -th class and its textual description. T denotes the textual encoder. e S i and e T i is the textual-level classifier embedding of i -th class from the source and target domains, respectively.

Our proposed VTA introduces discriminative visual features into the textual encoder, alleviating the problem of insufficient adaptation in plain textual tuning. Existing methods [31] only tune learnable textual descriptions for detection head, as shown in Fig. 3(a). However, textual descriptions are insufficient to describe certain inter-domain differences, e.g. , differences in fields of view, leading to a limited ability to learn cross-domain information. Different from them, VTA analyses domain-invariant and domain-specific knowledge from visual features, inferring an image-conditional detection head with high discriminability, as shown in Fig. 3(b).

3.5 Optimization Objective

We aim to insert the DA-Ada into the visual encoder to learn cross-domain information and further tune the prompt for discriminative textual representation with image conditions. On the one hand, we introduce domain adversarial loss to the DIA and DITA to guide the learning of domain-invariant information. Formally, we obtain the output features h I,S i , h I,T i for the source image x s and the target image x t of each DIA, and minimize the adversarial loss:

And the domain-shared DITA is expected to be aligned between domains:

where v S , v T denotes the source and target visual embedding.

On the other hand, we learn task-related domain-specific knowledge in a semi-supervised manner. For the source image, we calculate the cross-entropy for each visual embedding v S with its annotations y . For the target v T , we first obtain the prediction via the hand-crafted prompt "A photo of [class]" and filter out high-confidence pseudo labels y ′ , then minimize the cross-entropy as well:

where × denotes Matrix multiplication.

Meanwhile, to decouple domain-invariant and domain-specific knowledge, we maximize the distribution discrepancy between DIA and DSA.

where cos ( a , b ) = | a ⊤ · b | || a || 2 2 || b || 2 2 is absolute value of cosine distance, β is a threshold.

With the help of domain classifiers, DIA and DITA are encouraged to contain more domain-invariant knowledge. By minimizing L dec , the gap between DIA and DSA will be enlarged, which promotes DSA to extract more domain-specific knowledge. Overall, the optimization objective is:

where L reg is the regression loss, and λ dia , λ dita , λ dec are balance ratios.

4 Experiment

4.1 Datasets and Implementation

Weevaluate our method on four benchmarks: Cross-Weather(Cityscapes [9] → Foggy Cityscapes [54]), Cross-Fov(KITTI [18] → Cityscapes), Sim-to-Real(SIM10k [28] → Cityscapes) and Cross-Style(Pascal VOC [13] → Clipart [26]). Following [31], we adapt RegionCLIP(ResNet-50 [22]) with Faster-RCNN architecture as the baseline detector. We detail the datasets and implementation in Sec. 6.1and 6.3 of the Appendix.

4.2 Comparison to SOTA methods

We present representative state-of-the-art DAOD approaches for comparison, including feature alignment and semi-supervised learning methods.

Cross-Weather Adaptation Scenario Table 1 (C → F) illustrates that the proposed DA-Ada surpasses SOTA DA-Pro [31] by a remarkable margin of 2 . 6% , achieving the highest mAP over eight classes of 58 . 5% . Compared with existing methods, DA-Ada significantly improves seven categories ( i.e. person,

rider, car, truck, bus, train, and bicycle) ranging from 0 . 4% to 5 . 3% . The superior performance shows the remarkable effectiveness of the DA-Ada in the cross-domain generalization ability.

Cross-FOV Adaptation Scenario Table 1 (K → C) indicates a noticeable 5 . 3% improvement on the SOTA DA-Pro [31] by the DA-Ada. As K → C adaptation faces more complicated shape confusion than C → F, it requests higher discriminability of the model. Therefore, the considerable enhancement validates that the DA-Ada can efficiently learn robust visual encoder.

Sim-to-Real Adaptation Scenario We report the experimental results on SIM10k → Cityscapes benchmark in Table 1 (S → C). The proposed DA-Ada achieves the best results of 67 . 3% mAP, outperforming the previous best entry HT [11] 65 . 5% with 1 . 8% . The performance of DA-Ada is superior in the difficult adaptation task, which further demonstrates that our strategy is robust not only in appearance but also in more complex semantics adaptation tasks.

Cross-Style Adaptation Scenario Additionally, we assess DA-Ada on the more challenging CrossStyle adaptation, where the semantic hierarchy has a broader domain gap. DA-Ada peaks with 48 . 0% , outperforming all the SOTA methods presented in Table 2, demonstrating that injecting cross-domain information into the visual encoder could benefit the adaptation. Especially, DA-Ada exceeds all the compared methods on six categories (aeroplane, bike, bird, boat, bus, and sheep), which verifies the method is effective under challenging domain shifts and in multi-class problem scenarios.

4.3 Ablation Studies

Standard Adapter vs. Domain-aware Adapter We first compare the performance of the domainaware adapter with existing adapters, including source-only adapter and domain-agnostic adapter. As shown in Table 3, while the domain-agnostic adapter surpasses the source-only version by 3 . 4% ∼ 5 . 7% on four benchmarks, applying the domain-aware adapter further improves 2 . 4% ∼ 3 . 3% mAP. We further explored the reasons for this advantage, shown in Fig 1(d). Compared with oracle, the domain-agnostic adapter reaches similar performance on the source domain, but suffers severe performance drop of 3 . 7% on the target domain, indicating that it is biased towards the source domain. While improving the source domain with 0 . 4% , our method reaches the oracle on the target domain. The superioir performance indicates the domain-aware adapter not only aligns domain-invariant knowledge more accurately, but also utilizes domain-specific knowledge to improve the detector's discriminative ability on the target domain.

Ablation for Domain-aware Adapter We conduct comprehensive ablation studies on each component of the proposed method in Table 4. Only introducing DIA to the backbone attains an mAP of 53 . 8% , and optimizing each DIA with an independent discriminator L dia increases 1 . 0% . This indicates that learning domain-invariant adapters transfer task-related source knowledge to the target

domain. Moreover, the DSA boosts the DIA by 1 . 4% and 2 . 3% with the help of L dec , showing that learning domain-specific knowledge improves the discrimination of the target detection head.

Insertion Site We explicitly study the insertion site of DA-Ada, as shown in Table 5. When single adapter is applied, inserting the DA-Ada in the shallow block achieves better performance, e.g. DAAda with block 1 obtain 55 . 1% , surpassing all other insertion sites with block 2/3/4. And increasing the number of DA-Ada from 1 to 4 leads to steady improvements of 1 . 8% , 0 . 8% , 0 . 8% respectively.

Input and Injection Operation We analyze different input features and injection operations of DIA/DSA in Table 6. Directly inserting DIA into the visual encoder and directly adding to the output of each block attains 2 . 2% improvement, showing the effectiveness of learning domain-invariant knowledge. However, there is limited performance gain in sending the output F i ( h i -1 ) to DSA. It indicates that domain-specific knowledge is ignored during feature extraction of the visual encoder. To this end, inputting h i -1 -F i ( h i -1 ) to DSA receives 56 . 2% , exhibiting that the DSA can regain the domain-specific knowledge from the difference. As h I i = A I i ( h i -1 ) + F i ( h i -1 ) is updated to be domain-invariant, h i -1 - ( h I i ) removes domain-invariant parts and appears to be domain-specific. Therefore, we forward it to DSA and gain an improvement of 0 . 5% , demonstrating the efficacy of learning domain-specific knowledge. Additionally, we substitute cross-attention and pixel-level attention for the direct addition, and gains highest mAP of 57 . 0% and 57 . 1% . It reveals that domainspecific knowledge describes intra-domain properties and is more suitable for refining the extracted features. For efficiency, we adopt the simpler pixel-level attention as the fusion function.

Bottleneck Dimension We also conduct an ablation study in Table 7 to explore the optimal bottleneck dimension of the DA-Ada. As the dimension increases, the performance peaks 57 . 1% when the bottleneck dimension is 1 / 2 of the input and then appears to decline. We conclude that appropriate dimensional reduction can filter redundant features while extracting task knowledge.

Textual Tuning vs. Visual-guided Textual Adapter We compare the visual-guided textual adapter against existing methods, as shown in Table 8. Guided by visual conditions, VTA outperforms SOTA plain textual tuning methods by margins of 0 . 7% and 1 . 5% in two scenarios. Notably, VTA excels in the challenging Cross-FoV adaptation, suggesting that the visual modality effectively supplements the limitations of the textual encoder in describing domain attributes.

Ablation for Visual-guided Textual Adapter As shown in Table 9, learning DITA attains an mAP of 57 . 6% , and by introducing an additional adversarial loss, it achieves 57 . 9% . Moreover, with DSTA generating prompts for each domain, it exhibits a full adaptation performance of 58 . 5% . This shows that embedding image conditions into textual encoder can promote cross-domain detection.

Computational Efficiency As shown in Table 10, employing VLM yields a similar parameter scale while achieving a peak mAP of 58 . 5% . This indicates that the superior performance of DA-Ada does not arise from an increase in parameters. Furthermore, DA-Ada achieves the highest absolute gain of +8 . 0% with the training of only 1.794M parameters, demonstrating remarkable efficiency.

4.4 Detection Visualization

In Fig. 4, we present the comparison of the ground truth boxes (a) and the detection boxes of SOTA DA-Pro [31](c) and our methods (b)(d) on the target domain. (a.1)(b.1)(c.1)(d.1) are zoomed from the same region of images (a)(b)(c)(d) for a better view. Fig. 4(a.1) presents eight objects in the cropped region: 6 overlapped cars and a rider with a bicycle. The baseline model only detects two clear cars in Fig. 4(b.1). Failing to describe domain information, like weather conditions, it misses other objects hidden in the fog. In Fig. 4(c), the DA-Pro distinguishes the rider and the bicycle and improves 9 . 3% mAP with the domain-adaptive prompt. However, it ignores one car on the left of Fig. 4(c.1), suffering limited generalization ability due to insufficient domain representation learning in the visual encoder. Our proposed DA-Ada correctly detects the missing car (labelled in green) in the cropped region Fig. 4(d.1). By injecting cross-domain information into the visual encoder,

rider

bicycle

car 100%

(a)

Ground Truth

(a.1)

(c.1)

(c)

DA-Pro[31]

mAP:67.5

%/6J02

car 87%

100%

car 100%

rider 100%

rcle

CarH%

(b)

Baseline

(b.1)

(d.1)

(d)

DA-Ada (Ours)

mAP:54.6

mAP: 71.0

(d)DSA

the DA-Ada enables the model to detect more confidently on two bicycles ( 83% , 98% ) and one person ( 91% ), compared with DA-Pro's ( 79% , 95% ) and ( 89% ). These comparison results reveal the effectiveness of DA-Ada.

4.5 Feature Visualization

In Fig. 5, we visualize the output features of the traditional adapter, the domain-invariant adapter (DIA), domain-specific adapter (DSA) and the domain-aware adapter (DA-Ada). We sample image (a) a car and a person in the fog from Foggy Cityscapes. The traditional adapter (b) roughly extracts the outline of the car. However, affected by target domain attributes, such as fog, background areas are also highlighted in (b), and the person is not salient. DIA (c) mainly focuses on the object area and extracts domain-shared task information. DSA (d) mainly focuses on factors related to domain attributes besides the objects, such as foggy areas. By combining DIA with DSA, DA-Ada (e) extracts the car and person while reducing the interference of fog in the background. Compared with (b), objects are more salient in (e), indicating the effectiveness of DA-Ada.

5 Conclusion

In this paper, we propose a novel Domain-Aware Adapter (DA-Ada) for DAOD. As a small learnable attachment, it transfers highly generalized knowledge the visual-language model provides to crossdomain information for DAOD. Precisely, it consists of a Domain-Invariant Adapter (DIA) for learning domain-invariant knowledge and a Domain-Specific Adapter (DSA) for recovering the domain-specific knowledge from information discarded by the visual encoder. Extensive experiments over multiple DAOD tasks validate the effectiveness of DA-Ada in inferring a discriminative detector.

Acknowledgments and Disclosure of Funding

This work is partially supported by the National Key R&D Program of China(under Grant 2023YFB4502200), the NSF of China(under Grants 92364202, 61925208, U22A2028, 62222214, 62341411, 62102398, 62102399, U20A20227, 62372436, 62302478, 62302482, 62302483, 62302480, 62376268), 2022 Fundamental Disciplines Top Quality Student Training Program 2.0 Project, Strategic Priority Research Program of the Chinese Academy of Sciences, (Grant No.XDB0660200, XDB0660201, XDB0660202), Major Program of ISCAS(Grant No. ISCAS-ZD202402), Beijing Natural Science Foundation(4222039), CAS Project for Young Scientists in Basic Research(YSBR-029), Youth Innovation Promotion Association CAS and Xplore Prize.

References

Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. Domain separation networks. In NeurIPS , pages 343-351, 2016.

Woong-Gi Chang, Tackgeun You, Seonguk Seo, Suha Kwak, and Bohyung Han. Domain-specific batch normalization for unsupervised domain adaptation. In CVPR , pages 7354-7362, 2019.

Chaoqi Chen, Jiongcheng Li, Zebiao Zheng, Yue Huang, Xinghao Ding, and Yizhou Yu. Dual bipartite graph learning: A general approach for domain adaptive object detection. In ICCV , pages 2703-2712, 2021.

Chaoqi Chen, Jiongcheng Li, Hong-Yu Zhou, Xiaoguang Han, Yue Huang, Xinghao Ding, and Yizhou Yu. Relation matters: foreground-aware graph-based relational reasoning for domain adaptive object detection. TPAMI , 45(03):3677-3694, 2023.

Hao Chen, Ran Tao, Han Zhang, Yidong Wang, Wei Ye, Jindong Wang, Guosheng Hu, and Marios Savvides. Conv-adapter: Exploring parameter efficient transfer learning for convnets. arXiv preprint arXiv:2208.07463 , 2022.

Meilin Chen, Weijie Chen, Shicai Yang, Jie Song, Xinchao Wang, Lei Zhang, Yunfeng Yan, Donglian Qi, Yueting Zhuang, Di Xie, et al. Learning domain adaptive object detection with probabilistic teacher. In ICML , pages 3040-3055, 2022.

Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Domain adaptive faster r-cnn for object detection in the wild. In CVPR , pages 3339-3348, 2018.

Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2205.08534 , 2022.

Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, UweFranke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In CVPR , 2016.

Jinhong Deng, Wen Li, Yuhua Chen, and Lixin Duan. Unbiased mean teacher for cross-domain object detection. In CVPR , pages 4091-4101, 2021.

Jinhong Deng, Dongli Xu, Wen Li, and Lixin Duan. Harmonious teacher for cross-domain object detection. In CVPR , pages 23829-23838, 2023.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR , 2021.

Mark Everingham, SM Ali Eslami, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospective. IJCV , 111:98-136, 2015.

Mohammad Fahes, Tuan-Hung Vu, Andrei Bursuc, Patrick Pérez, and Raoul De Charette. Poda: Promptdriven zero-shot domain adaptation. In ICCV , pages 18623-18633, 2023.

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. JMLR , 17(1): 2096-2030, 2016.

Changlong Gao, Chengxu Liu, Yujie Dun, and Xueming Qian. Csda: Learning category-scale joint feature for domain adaptive object detection. In ICCV , pages 11421-11430, 2023.

Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. IJCV , 132(2):581-595, 2024.

Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR , 2012.

Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. In ICLR , 2022.

Dayan Guan, Jiaxing Huang, Aoran Xiao, Shijian Lu, and Yanpeng Cao. Uncertainty-aware unsupervised domain adaptation in object detection. TMM , 24:2502-2514, 2021.

Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. Towards a unified view of parameter-efficient transfer learning. In ICLR , 2021.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR , pages 770-778, 2016.

Mengzhe He, Yali Wang, Jiaxi Wu, Yiru Wang, Hanqing Li, Bo Li, Weihao Gan, Wei Wu, and Yu Qiao. Cross domain object detection by target-perceived dual branch distillation. In CVPR , pages 9570-9580, 2022.

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In ICML , pages 2790-2799, 2019.

Jiaxing Huang, Dayan Guan, Aoran Xiao, Shijian Lu, and Ling Shao. Category contrast for unsupervised domain adaptation in visual tasks. In CVPR , pages 1203-1214, 2022.

Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa. Cross-domain weaklysupervised object detection through progressive domain adaptation. In CVPR , pages 5001-5009, 2018.

Yifan Jiao, Hantao Yao, Bing-Kun Bao, and Changsheng Xu. Source-guided target feature reconstruction for cross-domain classification and detection. TIP , 2024.

Matthew Johnson-Roberson, Charles Barto, Rounak Mehta, Sharath Nittur Sridhar, Karl Rosaen, and Ram Vasudevan. Driving in the matrix: Can virtual worlds replace human-generated annotations for real world tasks? In ICRA 2017 , pages 746-753. IEEE, 2017.

Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In ICCV , pages 4015-4026, 2023.

Congcong Li, Dawei Du, Libo Zhang, Longyin Wen, Tiejian Luo, Yanjun Wu, and Pengfei Zhu. Spatial attention pyramid network for unsupervised domain adaptation. In ECCV , pages 481-497. Springer, 2020.

Haochen Li, Rui Zhang, Hantao Yao, Xinkai Song, Yifan Hao, Yongwei Zhao, Ling Li, and Yunji Chen. Learning domain-aware detection head with prompt tuning. NeurIPS , 36, 2024.

Haochen Li, Rui Zhang, Hantao Yao, Xin Zhang, Yifan Hao, Xinkai Song, and Ling Li. React: Remainder adaptive compensation for domain adaptive object detection. TIP , 33:3735-3748, 2024.

Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML , pages 12888-12900, 2022.

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pretraining with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597 , 2023.

Kai Li, Curtis Wigington, Chris Tensmeyer, Vlad I. Morariu, Handong Zhao, Varun Manjunatha, Nikolaos Barmpalios, and Yun Fu. Improving cross-domain detection with self-supervised learning. In CVPR , pages 4745-4754, 2023.

Wuyang Li, Xinyu Liu, Xiwen Yao, and Yixuan Yuan. Scan: Cross domain object detection with semantic conditioned adaptation. In AAAI , volume 6, page 7, 2022.

Wuyang Li, Xinyu Liu, and Yixuan Yuan. Sigma: Semantic-complete graph matching for domain adaptive object detection. In CVPR , pages 5291-5300, 2022.

Wuyang Li, Xinyu Liu, and Yixuan Yuan. Sigma++: Improved semantic-complete graph matching for domain adaptive object detection. TPAMI , 45(07):9022-9040, 2023.

Yu-Jhe Li, Xiaoliang Dai, Chih-Yao Ma, Yen-Cheng Liu, Kan Chen, Bichen Wu, Zijian He, Kris Kitani, and Peter Vajda. Cross-domain adaptive teacher for object detection. In CVPR , pages 7581-7590, 2022.

Chuang Lin, Zehuan Yuan, Sicheng Zhao, Peize Sun, Changhu Wang, and Jianfei Cai. Domain-invariant disentangled network for generalizable object detection. In ICCV , pages 8771-8780, 2021.

Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In CVPR , pages 2117-2125, 2017.

Dongnan Liu, Chaoyi Zhang, Yang Song, Heng Huang, Chenyu Wang, Michael Barnett, and Weidong Cai. Decompose to adapt: Cross-domain object detection via feature disentanglement. TMM , 25:1333-1344, 2022.

Rui Liu, Yahong Han, Yaowei Wang, and Qi Tian. Frequency spectrum augmentation consistency for domain adaptive object detection. arXiv preprint arXiv:2112.08605 , 2021.

Yabo Liu, Jinghua Wang, Chao Huang, Yaowei Wang, and Yong Xu. Cigar: Cross-modality graph reasoning for domain adaptive object detection. In CVPR , pages 23776-23786, 2023.

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In ICML , pages 97-105. PMLR, 2015.

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Unsupervised domain adaptation with residual transfer networks. NeurIPS , 29, 2016.

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Deep transfer learning with joint adaptation networks. In ICML , pages 2208-2217. PMLR, 2017.

Omiros Pantazis, Gabriel Brostow, Kate Jones, and Oisin Mac Aodha. Svl-adapter: Self-supervised adapter for vision-language pretrained models. arXiv preprint arXiv:2210.03794 , 2022.

Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2005.00247 , 2020.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML , pages 8748-8763. PMLR, 2021.

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In CVPR , pages 779-788, 2016.

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. NeurIPS , 28, 2015.

Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko. Strong-weak distribution alignment for adaptive object detection. In CVPR , pages 6956-6965, 2019.

Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Semantic foggy scene understanding with synthetic data. IJCV , 126(9):973-992, Sep 2018.

Sunandini Sanyal, Ashish Ramayee Asokan, Suvaansh Bhambri, Akshay Kulkarni, Jogendra Nath Kundu, and R Venkatesh Babu. Domain-specificity inducing transformers for source-free domain adaptation. In ICCV , pages 18928-18937, 2023.

Rui Shao, Xiangyuan Lan, and Pong C Yuen. Feature constrained by pixel: Hierarchical adversarial deep domain adaptation. In ACMMM , pages 220-228, 2018.

Mainak Singha, Harsh Pal, Ankit Jha, and Biplab Banerjee. Ad-clip: Adapting domains in prompt space using clip. In ICCV , pages 4355-4364, 2023.

Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. Vl-adapter: Parameter-efficient transfer learning for visionand-language tasks. In CVPR , pages 5227-5237, 2022.

Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. NPIS , 30, 2017.

VS Vibashan, Poojan Oza, and Vishal M Patel. Instance relation graph guided source-free domain adaptive object detection. In CVPR , pages 3520-3530, 2023.

Vidit Vidit, Martin Engilberge, and Mathieu Salzmann. Clip the gap: A single domain generalization approach for object detection. In CVPR , pages 3219-3229, 2023.

Vibashan Vs, Vikram Gupta, Poojan Oza, Vishwanath A Sindagi, and Vishal M Patel. Mega-cda: Memory guided attention for category-aware unsupervised domain adaptive object detection. In CVPR , pages 4516-4526, 2021.

Haoan Wang, Shilong Jia, Tieyong Zeng, Guixu Zhang, and Zhi Li. Triple feature disentanglement for one-stage adaptive object detection. In AAAI , pages 5401-5409, 2024.

Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Guihong Cao, Daxin Jiang, Ming Zhou, et al. K-adapter: Infusing knowledge into pre-trained models with adapters. arXiv preprint arXiv:2002.01808 , 2020.

Yu Wang, Rui Zhang, Shuo Zhang, Miao Li, Yangyang Xia, XiShan Zhang, and ShaoLi Liu. Domainspecific suppression for adaptive object detection. In CVPR , pages 9603-9612, 2021.

Aming Wu and Cheng Deng. Single-domain generalized object detection in urban scene via cyclicdisentangled self-distillation. In CVPR , pages 847-856, 2022.

Aming Wu, Yahong Han, Linchao Zhu, and Yi Yang. Instance-invariant domain adaptive object detection via progressive disentanglement. TPAMI , 44(8):4178-4193, 2021.

Aming Wu, Rui Liu, Yahong Han, Linchao Zhu, and Yi Yang. Vector-decomposed disentanglement for domain-invariant object detection. In ICCV , pages 9342-9351, 2021.

Jiaxi Wu, Jiaxin Chen, Mengzhe He, Yiru Wang, Bo Li, Bingqi Ma, Weihao Gan, Wei Wu, Yali Wang, and Di Huang. Target-relevant knowledge preservation for multi-source domain adaptive object detection. In CVPR , pages 5301-5310, 2022.

Junde Wu, Wei Ji, Yuanpei Liu, Huazhu Fu, Min Xu, Yanwu Xu, and Yueming Jin. Medical sam adapter: Adapting segment anything model for medical image segmentation. arXiv preprint arXiv:2304.12620 , 2023.

Minghao Xu, Hang Wang, Bingbing Ni, Qi Tian, and Wenjun Zhang. Cross-domain detection via graph-induced prototype alignment. In CVPR , pages 12355-12364, 2020.

Hongliang Yan, Yukang Ding, Peihua Li, Qilong Wang, Yong Xu, and Wangmeng Zuo. Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In CVPR , pages 2272-2281, 2017.

Hantao Yao, Rui Zhang, and Changsheng Xu. Visual-language prompt tuning with knowledge-guided context optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6757-6767, 2023.

Hantao Yao, Rui Zhang, and Changsheng Xu. Tcp: Textual-based class-aware prompt tuning for visuallanguage model. In CVPR , pages 23438-23448, 2024.

Jayeon Yoo, Inseop Chung, and Nojun Kwak. Unsupervised domain adaptation for one-stage object detector using offsets to bounding box. In ECCV , pages 691-708, 2022.

Xu Zhang, Felix Xinnan Yu, Shih-Fu Chang, and Shengjin Wang. Deep transfer network: Unsupervised domain adaptation. arXiv preprint arXiv:1503.00591 , 2015.

Yixin Zhang, Zilei Wang, and Yushi Mao. Rpn prototype alignment for domain adaptive object detector. In CVPR , pages 12425-12434, 2021.

Liang Zhao and Limin Wang. Task-specific inconsistency alignment for domain adaptive object detection. In CVPR , pages 14217-14226, 2022.

Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. Regionclip: Region-based language-image pretraining. In CVPR , pages 16793-16803, 2022.

Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In CVPR , pages 16816-16825, 2022.

Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. IJCV , 130(9):2337-2348, 2022.

Qianyu Zhou, Qiqi Gu, Jiangmiao Pang, Zhengyang Feng, Guangliang Cheng, Xuequan Lu, Jianping Shi, and Lizhuang Ma. Self-adversarial disentangling for specific domain adaptation. TPAMI , 45(7):8954-8968, 2023.

Wenzhang Zhou, Heng Fan, Tiejian Luo, and Libo Zhang. Unsupervised domain adaptive detection with network stability analysis. In ICCV , pages 6986-6995, 2023.

Yangguang Zhu, Ping Guo, Haoran Wei, Xin Zhao, and Xiangbin Wu. Disentangled discriminator for unsupervised domain adaptation on object detection. In IROS , pages 5685-5691, 2023.

Yaoming Zhu, Jiangtao Feng, Chengqi Zhao, Mingxuan Wang, and Lei Li. Serial or parallel? plug-able adapter for multilingual machine translation. arXiv preprint arXiv:2104.08154 , 2021.

6 Appendix

6.1 Datasets

Cross-Weather Cityscapes [9] contains diverse street scenes captured by a mobile camera in daylight. The regular partition consists of 2,975 training and 500 validation images annotated with eight classes. Foggy Cityscapes [54] simulates three distinct densities of fog on Cityscapes, containing 8,925 training images and 1,500 validation images. A standard configuration for cross-weather adaptation is to take the training set of Cityscapes as the source domain and the training set of foggy Cityscapes as the target domain, evaluating cross-weather adaptation performance on the 1500-sized validation set in all eight categories.

Cross-FoV KITTI [18] is a crucial dataset for self-driving that includes 7,481 photos annotated with cars. Collected by driving in rural areas and on highways, it provides data with a different Field of View (FoV). To fairly compare with other methods, we migrate KITTI to Cityscapes solely on the car category.

Sim-to-Real The synthetic dataset SIM10k [28] has 10,000 photos from the Grand Theft Auto V video game with labelled bounding boxes in the class car. We follow existing works to perform this sim-to-real adaptation and report the performance on the class car.

Cross-Style Pascal VOC [13] is a widely-used real-world dataset containing 2007 and 2012 subsets, annotated with 20 classes. Clipart [26] is collected from the website with 1000 comical images, providing bounding box annotations with same classes as Pascal VOC. Following the mainstream splitting, we use Pascal VOC 2007 and 2012 train-val split with a total of 16,551 images as the source domain and all Clipart images as the target domain.

6.2 Implementation Details

Following [31], we adapt RegionCLIP(ResNet-50 [22]) with a domain classifier [7] as the baseline. We use the Faster-RCNN [52] as the detector with the default configurations. Following [7], one batch of source images with ground truth and one batch of target domain images are forwarded to the proposed DA-Ada in each iteration to calculate the detection, adversarial and decoupling loss. The hyperparameter λ dia , λ dita , λ dec is set to 0 . 1 , 1 . 0 and 0 . 1 , respectively. We set the batch size of each domain to 8 and use the SGD optimizer with a warm-up learning rate. Mean Average Precision (mAP) with a threshold of 0.5 is taken as the evaluation metric. All experiments are deployed on 8 Tesla V100 GPUs.

6.3 Comparison to SOTA methods

We present representative state-of-the-art DAOD approaches for comparison, including feature alignment and semi-supervised learning methods.

Cross-Weather Adaptation Scenario Table 11 (C → F) illustrates that the proposed DA-Ada surpasses SOTA DA-Pro [31] by a remarkable margin of 2 . 6% , achieving the highest mAP over eight classes of 58 . 5% . Compared with existing methods, our method significantly improves seven categories ( i.e. person, rider, car, truck, bus, train, and bicycle) ranging from 0 . 4% to 5 . 3% . Compared with the SOTA decoupling method D 2 -UDA [84], the DA-Ada attains an improvement of 7 . 9% and promotes 3 . 0 ∼ 10 . 9% on all categories. The superior performance shows modifying domaininvariant knowledge with domain-specific knowledge could enhance the discriminative capability on the target domain.

Cross-FOV Adaptation Scenario Table 11 (K → C) indicates a noticeable 5 . 3% improvement on SOTA DA-Pro [31] by the DA-Ada, reaching an astounding peak of 66 . 7% mAP. As K → C adaptation faces more complicated shape confusion than C → F, it requests higher discriminability of the model. Therefore, the considerable enhancement validates that the proposed method can efficiently improve the discriminability of the visual encoder in new scenarios.

Sim-to-Real Adaptation Scenario We report the experimental results on SIM10k → Cityscapes benchmark in Table 11 (S → C). The proposed DA-Ada achieves the best results of 67 . 3% mAP, outperforming the previous best entry HT [11] 65 . 5% with 1 . 8% . The performance of DA-Ada is

superior in the difficult adaptation task, which further demonstrates that our strategy is robust not only in appearance but also in more complex semantics adaptation tasks.

Cross-Style Adaptation Scenario Additionally, we assess DA-Ada on the more challenging CrossStyle adaptation, where the semantic hierarchy has a broader domain gap. DA-Ada peaks with 48 . 0% , outperforming all the SOTA methods presented in Table 12. It demonstrates that injecting cross-domain information into the visual encoder could benefit the adaptation. Especially, DA-Ada exceeds all the compared methods on six categories (aeroplane, bike, bird, boat, bus, and sheep), which verifies the method is effective under challenging domain shifts and in multi-class problem scenarios.

6.4 Sensitivity on L dia

To select hyper-parameters for the adversarial loss in DIA, we perform experiments of different choices of the weight value λ dia . We conduct the experiment on DA-Ada on

Cityscapes → FoggyCityscapes adaptation scenarios, as shown in Table 13. Initialized with 0 . 01 , the DIA suffers from insufficient learning of domain-invariant knowledge, only attaining 57 . 1% mAP. When initialized with 0 . 05 ∼ 1 . 0 , the performance of DA-Ada is similar and achieves the best of 58 . 5% with λ dia = 1 . 0 . Increasing the λ dia to 10 . 0 sufferers significant performance degradation. We attribute this to the model focusing too much on alignment rather than detection.

6.5 Sensitivity on λ dita

We also explicitly study the sensitivity of weight value λ dita for the adversarial loss in visual-guided domain prompt, as shown in Table 14. As the weight value increases, the performance peaks with λ dita = 1 . 0 and then appears to decline. Since the hand-crafted token "A photo of [CLASS]" provides a solid prior, the learnable prompt is better initialized in the early stages of training. Therefore, the proposed model is more robust to the λ dita compared to λ dia .

6.6 Sensitivity on λ dec

We also evaluate the sensitivity of weight value λ dec for the decouple loss between DIA and DSA, as shown in Table 15. As the weight value increases, the performance rises rapidly until λ dec = 1 . 0 and then declines smoothly. L dec decouples domain-invariant and domain-specific knowledge by driving DIA to be orthogonal to the features extracted by DSA. Therefore, applying the decoupling loss with the same scale as the adversarial loss can optimize the goal relatively stably.

6.7 Ablation for Multi-scale Down-projector C D

Table 16 shows the impact of the different resolutions in multi-scale down-projector C D . The results indicate that while introducing various resolutions contributes to modeling multi-scale domain knowledge, inappropriate receptive fields may harm feature extraction performance. Concretely, we observed that the mAP on the target domain (Foggy Cityscapes) peaks when the number of down-projectors M is set to 4 , and the scaling ratios are { 1 , 1 2 , 1 4 , 1 8 } , respectively. And further applying 1 16 to C D results in slightly performance degradation. These experiments suggest that

applying a single convolution or introducing excessive distinct resolutions is unsuitable for learning domain knowledge, and the choice of resolution requires consideration of the difference in scale between the source and target domains.

6.8 Image or Instance-level Visual-guided Textual Adapter?

We explore whether to apply the visual-guided textual adapter at the image or instance levels. In DITA and DSTA, we replace the proposal embedding with the entire image as visual input, and it achieves 57 . 8% , suffering a performance drop of 0 . 7% . This indicates that instance-level alignment avoids the influence of background on learning domain knowledge in the foreground.

6.9 Evaluation on multiple Baselines

To properly evaluate the method, we introduce Da-Ada to two weaker baselines in 17. We inject DAAda into DSS and attain 7 . 2% improvement on mAP, showing great efficiency in feature-alignment methods. For further validation, we first adapt the classification model CLIP to Faster-RCNN to build a vanilla VLM detector with 42 . 8% mAP. Then we freeze the backbone and attach DA-Ada to the detector, achieving 52 . 6% mAP with an improvement of 9 . 8% . Experiments show that even with weak baselines, the proposed DA-Ada shows competitiveness to SOTA methods.

6.10 Performance and Computational Overhead

To verify the effectiveness of DA-Ada, we compare the performance and computational overhead with global fine-tune and DA-Pro on C → F adaptation. We initial the three methods with the same VLM backbone. Global fine-tuning has the largest time and memory overhead but only achieves the lowest performance, indicating the limitations of traditional DAOD methods in optimizing VLM. Compared with global fine-tuning, DA-Pro significantly reduces time and memory overhead while improving performance. Furthermore, DA-Ada significantly improves mAP with 4 . 9% while only using 6% of the time and 47% of the memory, showing great efficiency in adapting cross-domain information to VLM.

6.11 Failure Cases

We provide some examples of failure cases on the Cross-Weather adaptation scenario in Fig 6. We visualize the ground truth (a)(b) and the detection boxes of DA-Ada (c)(d). In (c.1), DA-Ada misses the car with its headlights on in the fog. Since the source data Cityscapes is collected on sunny days, few cars turned on their lights in the training set. Therefore, DA-Ada missed such out-of-distribution data. In (d.1), DA-Ada misses the bicycle and person blocked by other foreground objects. Since occlusion causes great damage to semantics, this type of missed detection is widely seen in object detection methods.

6.12 Limitation

Though effective, the proposed DA-Ada is specially designed for the domain adaptive object detection task, where a labelled source domain and a unlabelled target domain are needed. Currently, the

(a)

Ground Truth

a.

C

DA-Ada (Ours)

person

nerson)

person89%

car

bicycle

(b)

Ground Truth

(b.1)

(d.1)

(d)

DA-Ada (Ours)

method cannot deal with the setting of multiple source domains or no target domain. We plan to resolve these problems in our future research.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We state the contribution in the abstract and introduction, and provide experimental results in the last paragraph introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have discussed the limitation of the proposed method in Sec. 6.12 of the Appendix.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide source code in supplement material.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide source code and instructions in supplement material.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We detailed the setting of datasets and the implementation in Sec. 6.1 and 6.2 of the Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We provide error bars in Table 11, 12. The error bars are captured by multiple running with given experimental conditions.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide the compute works in Sec. 6.2, and discuss the computation overhead in Sec. 6.10 of the Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conforms with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: There is no societal impact of the work performed.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cite the original paper that produced the code package or dataset.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a Domain-Aware Adapter (DA-Ada) for DAOD based on VLMs, which aims to improve the model performance when applied to an unlabeled target domain. DA-Ada incorporates two types of adapters, including a  Domain-Invariant Adapter (DIA) to learn domain-invariant knowledge and a Domain-Specific Adapter (DSA) for injecting domain-specific knowledge. It also includes a Visual-guided Textual Adapter (VTA) to encode the cross-domain visual feature into a textural encoder to enhance detection. Experiments across various tasks show that DA-Ada significantly outperforms existing methods, demonstrating its effectiveness in eliminating the domain gap.\n\nSTRENGTHS:\n1. The proposed method can significantly improve DAOD performance, surpassing state-of-the-art works by a large margin.\n2. The general idea of using an adaptor with VLMs to assist DAOD is moderately interesting.\n3. The method is simple yet effective.\n\nWEAKNESSES:\n1. Some claims are confusing and lack sufficient justifications. \n2. Some designs are similar to existing works and lack sufficient technical contributions.\n3. The method is only deployed on Faster RCNN. The adaptability to more advanced detectors is unknown.\n4. The paper writing and organization need to improve. \n\nDetails are in Questions.\n\nQUESTIONS:\n1. In Line 139, are there any theoretical proofs or experimental justifications to support the claim that low-dimensional features have less information redundancy and are more suitable for domain adaptation? How can you ensure that low-dimensional representation is domain invariant? I don't think that the feature with less information redundancy means domain invariant.\n\n2. For the Visual-guided textual adapter, this design seems to follow CoCoop [1], a generic technique for improving model learning. The difference needs to be clarified.\n\n3. I am confused about the domain-specific adapter. The authors use the residual parts of the visual features to represent the domain-specific information. Is there any proof or literature that can justify the correctness of this assumption?\n\n4. Can this method be applied to more advanced detectors, such as DETR and CenterNet v2?\n\n5. The paper writing and organization need to improve. The authors propose DITA and DSTA but don't provide any information about them in the abstract. Additionally, the method descriptions and Fig.2/3 are not well matched, leading to the reading difficulty.  For example, in Line 184, where are DITA and DSTA in Fig. 2(b)?\n\n[1] Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022). Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 16816-16825).\n\nConsidering the significant performance gain of the proposed method, I'm happy to turn to a positive score if concerns are addressed.\n\nLIMITATIONS:\nNo potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a method to tackle the domain adaptive object detection (DAOD) task within the framework of visual-language models (VLM). The authors propose a Domain-Aware Adapter (DA-Ada) to enhance the visual encoder's ability to learn both domain-invariant and domain-specific features. DA-Ada consists of two components: the Domain-Invariant Adapter (DIA) and the Domain-Specific Adapter (DSA). The DIA learns domain-invariant features by aligning the feature distribution between the source and target domains, while the DSA recovers domain-specific knowledge from the differences between the input and output of the visual encoder blocks. Additionally, the Visual-guided Textual Adapter (VTA) embeds cross-domain information into the textual encoder to improve detection head discriminability. Experiments on various DAOD benchmarks indicate that DA-Ada improves performance compared to state-of-the-art methods.\n\nSTRENGTHS:\nThis paper is well-written and easy to follow. The motivation for explicitly learning additional domain-specific features to enhance cross-domain performance is clear and straightforward. The disentanglement of domain-invariant and domain-specific features proposed in the Domain-Specific Adapter, along with the regularization loss in Eq. 15, appears to be effective. The ablation study is comprehensive and detailed, and the experimental results indicate that the proposed model achieves significant performance improvements compared to other state-of-the-art methods across different domain shift scenarios.\n\nWEAKNESSES:\nDespite the paper’s strengths, certain aspects of the discussion could be further refined for precision and clarity:\n1. The performance of the source-only baseline in Table 3 outperforms (or achieves similar performance to) most of the non-VLM-based models in Table 1. For example, on the Cross-Weather domain shift, the source-only model achieves an mAP of 50.4, whereas methods like AT[35], which are backbone ImageNet-pretrained, show a performance of 50.9, and their source-only variants perform roughly >10 mAP lower. While using a VLM vision encoder to generate essential general knowledge is reasonable, it raises the question of whether this task should still be categorized as cross-domain (domain adaptive) learning or as transfer learning (or VLM-based domain adaptation). Additionally, it would be helpful to add a column in Tables 1 and 2 to indicate the pretraining.\n2. It would be beneficial if the authors could also provide the comparisons about the computational speed\n\nQUESTIONS:\n1. What are the regression losses used in DA-Ada? Are they the same as those used in [28]?\n2. The symbols of down-projectors in Eq.6 and Section 6.7 are different. Could you clarify this discrepancy? Additionally, for the down-up projections, which operate on both the channel dimension (Eq.5) and the spatial dimensions, a clearer explanation would be beneficial.\n3. Since DA-Ada modules are added to different visual encoder blocks, the claim in line 139 that “Low-dimensional features have less information redundancy…” seems inappropriate. In my opinion, “low-dimensional” features typically refer to features from the earlier layers. In Section 2, Eq. 5 seems more related to “channel dimension” condensation. Additionally, in “Bottleneck Dimension” in Section 4.3 (also Table 7), it would be helpful if the authors could directly indicate the input channel dimensions.\n4. It would be beneficial to add some visualizations for failure cases.\n\nSome minor problems:\n1.\tTypos in line 183-184?\n2.\tLegend missing in Figure 3 (the red symbols)\n\nLIMITATIONS:\nYes, the limitations are discussed in Sec 6.11.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work focuses on domain adaptive object detection (DAOD) with the vision-language models. The core idea behind this paper is the frozen visual encoder with a domain-agnostic adapter only captures domain-invariant knowledge for DAOD. To this end, this paper proposes a novel Domain-Aware Adapter (DA-Ada) to capture the domain-invariant knowledge and domain-specific knowledge. The experimental results on multiple DAOD tasks show the proposed method clearly outperforms existing DAOD methods.\n\nSTRENGTHS:\n1.\tDAOD is an important problem, especially, in the large vision-language era, we need to rethink and explore the new paradigm for DAOD.\n2.\tThe proposed method is reasonable, effective transfer performance should maintain both the transferability (domain-invariant) and discriminability (domain-specific) between source and target domains.\n3.\tThe experimental results over multiple benchmarks show the effectiveness of the proposed method. Extensive ablation studies have been conducted to investigate the proposed method.\n\nWEAKNESSES:\n1.\tAlthough the proposed method is reasonable, it still has limited novelty for the domain adaptation community. The domain-invariant and domain-specific knowledge have been exploited by many previous works. \n2.\tThe function of dec loss is to maximize the distribution discrepancy between DIA and DSA. Why is the cosine similarity calculated by $h_i^I$ and $h_i^I \\cdot h_i^S$ instead of $h_i^I$ and $h_i^I$.\n3.\tLack of details for VTA, what are the structures of DITA, DSTA, and DSTA. Besides, it should be added the text description c in Figure 2 (c), the CLIP Textual Encoder adopts both the projected visual feature and the textual feature for better clarity.\n4.\tThe author should add results of source-only baseline, i.e., RegionCLIP, for a fair comparison.\n5.\tThe Injection Operation has many ways in Table 6, can the authors provide some explanation between $h_i^I + h_i^S$ and $ h_i^I + h_i^I \\cdot h_i^S $\n6.\tIn line 184, the figure reference should be Fig.2(c) instead of Fig.3(b).\n\nQUESTIONS:\n1. Do we really need a source domain for adaptation when we already have a powerful vision-language detector that contains much general knowledge from large-scale data?\n\nLIMITATIONS:\nNothing.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article proposes a Domain-Aware Adapter (DA-Ada) tailored for the DAOD task. The key point is exploiting domain-specific knowledge between the essential general knowledge and domain-invariant knowledge. The DA-Ada framework consists of the Domain-Invariant Adapter (DIA) for learning domain-invariant knowledge and the Domain-Specific Adapter (DSA) for injecting the domain-specific knowledge from the information discarded by the visual encoder. Comprehensive experiments over multiple DAOD tasks show the effectiveness of DA-Ada.\n\nSTRENGTHS:\n1. The paper is well-written.\n2. The proposed method achieves significant performance improvement compared to several baselines on commonly used benchmarks. The proposed method not only improves the detection performance on the target domain, but also achieves improvement on the source domain.\n\nWEAKNESSES:\n1. The DIA module is a sequence of operations involving mapping, dimensionality reduction, slicing, and dimensionality-raising. It is challenging to understand why such operations can extract domain-invariant knowledge. The author should provide a stronger explanation to clarify the rationale behind this design.\n2. In line 251 of the text, could the author elaborate on how the source-only adapter and the domain-agnostic adapter each function? This would help readers better understand.\n3. Could the author design a quantitative experiment to further visualize which features represent domain-specific knowledge? Additionally, could they compare the traditional adapter with the method proposed in this paper to further demonstrate the effectiveness of the proposed method? The AP metric alone is insufficient to fully explain the motivation behind this method.\n4. Since this paper focuses on adapter work, could the author further review the related work on VLM (Vision-Language Models) adapters in more detail? This would help readers better understand the contribution of this paper.\n5. The paper mentions that the domain-agnostic adapter easily causes the model to bias towards the source domain. The latest work [1] also addresses the issue of source bias. In the related work and experiments sections, the author should include a discussion and comparison with the latest work.\n6. During this training process, are the original CLIP network components (including the visual encoder and the text encoder) completely frozen, with only the adapter and VTA parts being trained?\n7. minor error: There are several places between lines 91 and 102 where spaces are missing between sentences.\n\nReference:\n[1] DSD-DA: Distillation-based Source Debiasing for Domain Adaptive Object Detection[C]//Forty-first International Conference on Machine Learning.\n\nQUESTIONS:\nSee weaknesses\n\nLIMITATIONS:\nAdequately addressed", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}