Cloud Object Detector Adaptation by Integrating Different Source Knowledge
Authors: Shuaifeng Li
Mao Ye
Lihua Zhou
Nianxin Li
Siying Xiao
Song Tang
Xiatian Zhu
Venue: NeurIPS 2024 poster (2024)
Keywords: Object detection
Cloud model
Domain adaptation
Primary area: machine_vision
TL;DR: We propose to explore cloud object detector adaptation by integrating different source knowledge in a self-promotion way.

Abstract: We propose to explore an interesting and promising problem, Cloud Object Detector Adaptation (CODA), where the target domain leverages detections provided by a large cloud model to build a target detector. Despite with powerful generalization capability, the cloud model still cannot achieve error-free detection in a specific target domain. In this work, we present a novel Cloud Object detector adaptation method by Integrating different source kNowledge (COIN). The key idea is to incorporate a public vision-language model (CLIP) to distill positive knowledge while refining negative knowledge for adaptation by self-promotion gradient direction alignment. To that end, knowledge dissemination, separation, and distillation are carried out successively. Knowledge dissemination combines knowledge from cloud detector and CLIP model to initialize a target detector and a CLIP detector in target domain. By matching CLIP detector with the cloud detector, knowledge separation categorizes detections into three parts: consistent, inconsistent and private detections such that divide-and-conquer strategy can be used for knowledge distillation. Consistent and private detections are directly used to train target detector; while inconsistent detections are fused based on a consistent knowledge generation network, which is trained by aligning the gradient direction of inconsistent detections to that of consistent detections, because it provides a direction toward an optimal target detector. Experiment results demonstrate that the proposed COIN method achieves the state-of-the-art performance.

Cloud Object Detector Adaptation by Integrating Different Source Knowledge
Shuaifeng Li Mao Ye Lihua Zhou' Nianxin Li Siying Xiao Song Tang² Xiatian Zhu
University of Electronic Science and Technology of China
2 University of Shanghai for Science and Technology
University of Surrey
hotwindlsf@gmail com, cvlab. uestc@gmail com, xiatian. x ac.uk
https: htt:://ggttbb...... thub. com www.o///woogg/////ooooooooo COIN
Abstract
We propose to explore an interesting and promising problem, Cloud Object Detec- tor Adaptation (CODA), where the target domain leverages detections provided by a large cloud model to build target detector. Despite with powerful general- ization capability, the cloud model still cannot achieve error-free detection in specific target domain. In this work, we present a novel Cloud Object detector adaptation method by Integrating different source kNowledge (COIN). The key idea is to incorporate a public vision-language model (CLIP) to distill positive knowledge while refining negative knowledge for adaptation by self-promotion gradient direction alignment. To that end, knowledge dissemination, separation, and distillation are carried out successively. Knowledge dissemination combines knowledge from cloud detector and CLIP model to initialize target detector and CLIP detector in target domain. By matching CLIP detector with the cloud detector, knowledge separation categorizes detections into three parts: consistent, inconsistent and private detections such that divide-and- -nd--nuuuueerrrrrrr- strategy can be used for knowledge distillation. Consistent and private detections are directly used to train target detector; while inconsistent detections are fused based on a consistent knowledge generation network, which is trained by aligning the gradient direction inconsistent detections to that consistent detections, because it provides a direction toward an optimal target detector. Experiment results demonstrate that the proposed COIN method achieves the state-of-the tet-hhe-rtt art performance.
Introduction
The emergence of large language models like GPT-4 [45] heralds a future in which cloud model possesses remarkable capabilities tailored to specific tasks. But the performance always will be degraded for special target domain. Naturally, the problem Cloud Domain Adaptation (CDA) emerges, where the knowledge from cloud model is transferred to the target domain by cloud API requests. This work focuses on Cloud Object Detector Adaptation (CODA), a new problem setting in domain adaptation. This problem is training detector for any target domain under the conditions that there exists a large cloud detector offering API service while target domain samples have not any labels. As shown in Fig T(),, compared with previous settings, there open target scenarios and object categories; (2) without high domain similarity between domains.
main advantages: (1)
There have been some advances in related fields. Source-free Object Detection (SFOD) [35] 33 transfers a pre-trained model from the source domain to the target domain without considering the
'The corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Conditions
4000 on nAri
LD Backkoo
Good
Optimall turast moodi
dclodar
Scurce medel sccos
Closd API aoae
v V. X
X
X
consilent detections
Geadiagla
alignment
High denan simikrity
v
CLIP
Cloud
Ability
Fleshl: uechtecttree
Supery
Open Entegories
Carrent farget model
Gradiennt Incomildennt detections
(a) CODA other settings
(b) Knowledge Dissemination
(c) Knowledge Separation
(d) Knowledge Distillation
Figure I: (a) Setting comparison: Our CODA (Cloud Object Detector Adaptation), UDAOD (Unsu- pervised Domain Adaptive Object Detection), SFOD (Source-free Object Detection) and Black-box DAOD (Black-box Domain Adaptive Object Detection). (b) Knowledge dissemination initializes cloud detector and target detector. Then, detections are categorized to three parts: consistent, incon- sistent, private (cloud and CLIP) derections (c). (d) The gradients on consistent detections are used to guide decision fusion for inconsistent detections. Zoom in for best view.
model privacy issue. The existing methods usually resort three routes: pseudo label refinement (38) where more reliable labels are refined for self-training; knowledge distillation 51, where the Mean Teacher framework is used to distill knowledge; domain 53 10 61. where distribution alignment methods are used to learn domain-invariant
135
9
62
71
33
39
13
18 28
alignment 33
feature. Despite notable success has been achieved, these methods are not suited for CODA because cloud model is not accessible, and any target domain cannot be transferred because of category limitations.
Another related field is Black-box Domain Adaptive Object Detection (Black-box DAOD), which only offers detection results of several predefined categories without accessing source model and data [67]. There exists one work for Black-box DAOD, where three types of memory are used for label calibration [67]. More Black-box DA methods [36] 64 60
46
58 focus on classification to distill source domain to select representative samples
68
and fall into two categories. The first uses knowledge distillation 36
46
58
knowledge, while the second uses sample selection [64 60
68
57
for training. They also can not be applied to CODA. Unlike Black-box DAOD, CODA benefits from large training data and language modality, thus eliminating the trouble of finding tailored source domain containing all target categories and facilitating unrestricted target domain adaptation.
Despite being trained on large data, the powerful cloud model also cannot achieve error-free detection. So leveraging public auxiliary models with enough object categories to correct the error detections is natural choice. The public vision-language model such CLIP 47 pre- pre-trained image-text pairs, is a good model to help adaptation. Due to the lack of detection ability and domain shift, CLIP is first extended the knowledge from CLIP, as shown in Fig [b). However, the CLIP detector also has error detections. As shown in Fig[Tc), the detections of CLIP detector and cloud detector can be classified into three categories: consistent, inconsistent and private detections. Now, the problem is how to distill these detections to the target detector. The consistent and private detections can be easily distilled the target detector as the supervision signals. While for inconsistent detections, one potential route is to use consistent detections to help inconsistent detections, as shown in Fig d).
millions of
detector, abbreviated as CLIP detector, to inherit and further adapt
Based on the above analysis, we propose a novel adaptation method in divide-and -nd--quee manner, dubbed as COIN, where knowledge dissemination, separation, and distillation stages are carried out successively. Knowledge dissemination combines the knowledge from CLIP model and cloud model to initialize CLIP detector and target detector based on Faster R-CNN same time, prompt learning is performed to align the target domain for CLIP detector. Knowledge separation matches the detections from cloud detector and CLIP detector, categorizing three parts as consistent, inconsistent and private detections. Based on distillation regards CLIP detector and cloud detector as teachers while the target detector as student. Consistent and private detections are used as supervision; prompt learning is performed again to adapt CLIP model for target detector. A Consistent Knowledge Generation network (CKG) proposed to fuse inconsistent detections. Since the gradient direction for optimizing target detector based on consistent detections offers an optimization direction, for inconsistent detections, a gradient direction alignment loss is proposed to learn CKG under the situation without supervision labels. The Tean-he framework also updates CLIP detector based on target detector, thereby achieving better knowledge integration.
48 architecture. At the
framework, knowledge

---

Our contributions can be summarized as follows. (1) We propose to explore a promising problem CODA suited for real-world scenarios with large cloud models. A novel method COIN proposed in divide-and-conque
manner. An open auxiliary model (CIIP) is introduced to help adaptation. By carefully combining different source knowledge, the effect of one plus one being greater than two has been achieved. (2) A novel decision-level fusion strategy is proposed. The gradient direction alignment loss is proposed which fuses the conflicts by using consistence detections in a rational and self-promotion way. (3) Different prompt learning are performed for CLIP detector and target detector. For CLIP detector, the class text embeddings are aligned to CLIP visual feature class prototypes; while for target detector, the prototypes based on consistent detections are used since target detector combines different source knowledge.
2 Related Work
Domain adaptive object detection. Unsupervised Domain Adaptive Object Detection (UDAOD) assumes that source samples are freely accessible and labeled but target samples have no labels. Existing methods can be roughly classified into three categories. The first approach is based on
31 27 where the source and target domains aligned by adversarial learning, contrastive learning, etc. The second approach is the popular knowledge where the Mean-Teacher framework is used distill knowledge from teacher to student. The third approach is based on graph learning [ 34 41 [5), where graphs are constructed to achieve better adaptation. Source-free Object Detection (SFOD) assumes that only the pre-trained source domain model is accessible. Existing SFOD methods usually resort to three technical routes. The first pseudo label refinement based method [35 62 38), e.g., SED (35] seeks a confident threshold for filtering pseudo labels according to self-entropy descent. The second For example, LODS (33 enhances target domain style and then overlooks target domain style, resulting in an impressive two-way knowledge distillation. The third is based on domain alignment [53 33 61). For instance, IRG
domain alignment S
49 50
59
6
75
70
distillation [12 22 [4
20
knowledge distillation based approach [3 39
1
28
51
53
uses a contrastive loss to enhance the target representations by exploiting the object relations.
Black-box domain adaptation. Recently, black-box domain adaptation for image classification receives major attention. There are two routes. The first is knowledge distillation [36 46 (58)
For example, DINE (36] starts by training model using knowledge distillation and structural regularization, then further refines it for better adaptation; RAIN (46) introduces phase mixup and subnetwork distillation learn from both regularized data and subnetworks: AEM (58) first proposes to explore CLIP for -lo DA by introducing an adversaiial experts model. The second is based
on sample selection 64 60
57 For instance, IterL.N (64 follows learning with noisy labels
technique and estimates a noise rate to select confident target samples for training: BETA [60] divides
the target domain into two subdomains and leverages synergistic twin networks and subdomain augmentation for robust model learning; RFC [68] introduces selection training to pick samples from minority classes for reviewing forgotten classes, and employs neighborhood clustering for more balanced learning. There exists only one work for Black-box DAOD task, BiMem [67] refines the pseudo labels by constructing sensory, short-term and long-term memories, where a forward memory construction and backward label calibration performance achieved, the source model cannot be transferred to arbitrary target domains due to category restrictions and domain similarity, and require customizing the source domain model for specific target domain.
performed iteratively. Despite the great
3 Methodology
Problem statements. Cloud Object Detector Adaptation (CODA) assumes the unlabeled target domain D = { ] and C = {ce ) is a set of classes that need to be detected, where N, is the total number of target images, is the i-th class name and N₂ is the number of classes. There exists powerful cloud detector Feaay and the goal of CODA is to train target detector by the detection results yd via a cloud API. you consists of boxes bid and class probabilities D'd for any target domain image z², where class probabilities P'd are derived from class predictions, which can in the form of class-only, confidence score, or probability, depending on the cloud detector. Moreover, with the powerful cloud detector trained on large-scale scale image caption datasets, open target scenarios and even categories can be adapted, climinating the hassle of finding similar source domain.
3

---

(a) Knowledge Dissemination Stage
(b) Knowledge Separation Stage
Penso
Truck
A photo of (err'
Trmplater
A
Target image
Cloud detector
Car
Private Detoctions
muntchine
Backhone
Prompts:
CLIP detector
phose af at #P person. photo af o' tht muck.
BOJ Pouling
RPN
photoo af (PP
Detections
Strong augmentation
Feature extractor Co
Teat Encoder E
Tranformation Tramy
FEMA
Consistent Knowledge Generation
CKG
Per-class Embeddings
Box Headl
ClassHead
Targel
Box 6
Feature
Similarity Probabiliny P
(c) Knowledge Distillation Stage
Figure Overview of the proposed method COIN. (a) Knowledge dissemination stage. The architecture for CLIP detector and target detector is presented. (b) Knowledge separation stage splits detections from two detectors into three kinds. (c) Knowledge distillation stage trains target detector. A gradient direction alignment loss is proposed to fuse inconsistent detections in decision-level.
Overview. The proposed COIN method introduces Laioon-anguage model CLIP [47 to assist in domain adaptation of a freely chosen large-scale pre-trained cloud detector, like GDINO 40 consists of three stages, i.e., knowledge dissemination, knowledge separation and knowledge distillation, as illustrated in 2ig[2] (a) Knowledge dissemination stage first collects results from cloud detector and CLIP respectively, where the detection boxes from cloud and the classification results from CLIP are used to initialize a CLIP detector and target detector. Meanwhile, prompt learning is performed for CLIP detector. (b) Knowledge separation stage initially obtains and then matches the detections from cloud and CLIP detectors, resulting in the categorization of consistent, inconsistent (c) Knowledge distillation stage updates target detector. The Mean-Teacher framework is employed which regards cloud detector and CLIP detector as two teachers while the target detector a student. To enhance robustness, the student is fed into a strong augmentation version of target image, and consistent and private detections are directly used as pseudo labels. For inconsistent detections, a Consistent Knowledge Generation network (CKG) is designed to fuse them in decision-leved gradient direction alignment loss is proposed to optimize CKG; the target detector and CKG are updated mutually. Better knowledge integration is achieved by updating CLIP detector.
and private detections.
3.1 Knowledge Dissemination
In this section, the knowledge from cloud detector and auxiliary model (CLIP) are combined to get two object detectors in target domain, i.e., CLIP detector Fe..,, and target detector Før. An intuitive idea is to train Faster R-CNN based detector [48) using the predicted boxes from cloud detector and its corresponding labels from CLIP. However, there are two deficiencies. The first is that the knowledge from the auxiliary model has not been fully utilized; here the auxiliary model CLIP is supposed to be open-source and known. Another is domain shift existed between CLIP and target domain; the auxiliary model CLIP should be aligned with target domain. So the pre-traine pre-trained CLIP visual encoder used to build two detectors Fe... and F then domain-specifie prompts learned to align CLIP model to target domain for CLIPP detector; in the end, the detections collected to train CLIP detector. They are detailed as follows.
Detector architecture is based on Faster R-CNN framework shown in Fig 2la. The pre-trained CLIP visual encoder G is split into G₁ and G2 (the last block), which are used as backbone and ROI head feature extractor respectively. Because CLIP is pre-trained for classification task, the region feature of proposal r for target image x, fr = ())
can not be used for box regression. So. a transformation network Trans, composed of mean pooling and three dense layers, is used to endow the localization ability. Finally, a linear layer le and a linear layer l, are used to get the box feature and locations respectively, i.c, f (e(Trans(f..) and b ls(Trans(f..)) The class probability p; for the i-th category is calculated by computing the similarity with the i-th class embedding e'. The background is also considered to be class. It can be written
exp(sim(f,el) (f,')m
p
(1)
Σ :=

---

where -im(---) is the cosine similarity function and T = 0.01 is the fixed temperature following CLIP. The class embedding e' E(P) is obtained based on the frozen CLIP text encoder E and the prompt P3 wrapping the i-th class name into a later introduced prompt template PT.
Both Fo and F are based this architecture and are randomly initialized except that the back- bone and ROI head feature extractor are initialized by CLIP visual encoder G₁ and G2 respectively.
Prompt learning for CLIP detector. Since the simple prompt template, like "a photo of [CLS], has not target domain information, we use a trainable prompt template PT, like "a photo of { [CLS].", where M is fixed to and t' is a placeholder whose word embedding is randomly initialized. Prompt learning methods 774 73 29 69 adapt the class embeddings truths. To further adapt CLIP model to target domain, we propose to use visual features to align e.
{{') Ji=] with ground
Specifically, the visual feature class prototypes are used to learn prompt instead of matching all visual
features with the prompt embeddings. The prototypes Ep {e, J= N are updated by exponential moving average (EMA) as
e =
17 e (1- 17) E ERED
Σ 1(1= R
(2)
where R is the set of proposals for image is the label for the box feature f. and [(a b) an indicator function. 77 = 0.9996 is a fixed hyperparameter and ep are initialized as the original CLIP per-class embeddings e, for accelerating training. The following L1 loss is used to learn the prompt,
Calign = llep - e|l
Pre-training CLIP detector. Since CLIP do not predict boxes for target domain images, to train CLIP detector, we need to prepare supervision labels based on CLIP and cloud detector knowledge. For any target image z, the detection boxes baid are borrowed from cloud detector and the corresponding labels are obtained based on the CLIP model; they can be used as supervision signals. The detection boxes are easy to obtain, while obtaining the labels needs some specific operations. Suppose a box feature f is obtained by ROI pooling on the feature map output by CLIP visual encoder G. To obtain more accurate pseudo labels, as RegionCLIP 722, 81 kinds of prompt templates are used. If the j-th kind of prompt template is "a [target domain name] style rendering of the [CLS]", the j-th kind of class embedding for the i-th object class is e through CLIP text encoder E. The final i-th class embedding is e E, e /81. The class probability Pe of f is obtained by computing the similarity like [D using ez- The boxes predicted as "background" are removed. After the preparation of supervision signals, the CLIP detector is pre-trained via the following losses:
min CAPN + Lnor + XC1 olignn
Behop
where CRPN and CrOT are the standard detection losses. is a hyperparameter fixed as 10.
Remark. To inherit knowledge for better knowledge dissemination and accelcrate training, the visual encoder of the original CLIP model is directly used as the backbone and ROI Head fcature extractor. Compared to the previous CLIP based detector F-VLM [30] our backbone and ROI Head feature extractor are trainable with supervisions from CLIP and cloud detector to facilitate knowledge dissemination process. Moreover, in contrast to PromptSRC 22 and CLIP-GAP (52), which align the features to CLIP semantic space, our dynamically updated class prototypes align CLIP semantic space to target domain in an opposite way, thus capturing more target domain-specific attributes.
3.2 Knowledge Separation
Just like flipping two coins the same time, the detections from cloud detector and CLIP detector exhibit both consistency and conflicts due to different pretraining sources. It is obvious that consistent detection results can be used as ground truths, while inconsistent results pose obstacles to knowledge fusion. To integrate their knowledge into the target detector sensibly, we adopt a divide-and and-id--and strategy. Specifically, box matching is utilized to achieve knowledge separation by categorizing them into consistent, inconsistent, and private detections.
Given a target image v, suppose the detections based on cloud detector are Yetd (bald: Peld) containing R2 detected boxes and similarly the detections based on CLIP detector after NMS (Non- Maximum Suppression) are Yelip (belip: Petip] containing R2 detected boxes. To find the matched
5

---

boxes, an identification matrix I' is defined as follows. Fú, = 11 if the IoU 2 A between the i-th box from cloud detector and the j-th box from CLIP detector, otherwise F. O. x. is a fixed threshold set to 0.5 according to popular settings. For the i-th box of cloud detector, the label Ietd arg m Poidee while laiip = arg max. Petip.e is the label of j-th box of CLIP detector.
Then, as shown in Fig b)) the consistent detection set P and inconsistent detection set P are defined as follows,
P (Veld Vati)) [Tag = 1, Lad = 1 1. P (VVeia Velip) [Tg - 1, laid * 1 Pelipt
(5)
The unmatched detection set 2, also called private detections, is defined as
Q {yLid i = 0} J (V'dip T. = 0}.
(6)
T means the number of CLIP detector boxes that match the i-th cloud detector box; so does
For one pair of matched boxes bidd and b'dip from the consistent or inconsistent detections, an object located. Typically, the box with a higher score has more precise localization. Therefore, we merge them in probability-weighee manner to facilitate subsequent distillation, as the features extracted from two matched boxes exhibit slight inconsistencies. So the fused box 6%, is
6 max b'dd + max Peti..e b'dip
max maxPadd,e + max Petsp.c
After box refinements, the consistent detection set P and inconsistent detection set P can be denoted P (タ) and p {ÿ} respectively, where ÿ (b.. Peld Potip dm) and y
(bm Pold Pekp lotd leti)) is the inconsistent detections.
Remark. Detection conflicts is a core challenge here. Previous UDAOD method SSAL 44 performs sample selection within the same class, so boxes in the same region that are predicted as different classes may be selected for self-training, resulting in conflicts. While we address this issue by adopting a divide-and-come conquer strategy to separate conflicts here and solve them in the following text.
3.3 Knowledge Distillation
Mean-Teacher framework is utilized to distill the above three kinds of detections into target domain detector. The cloud detector and CLIP detector are two teachers while the target detector student.
Consistent and private detections knowledge distillation. For the consistent detections P. they are directly used as ground truths to train target domain detector. The consistency distillation loss is defined as Leon CRPN + LROI- For the private detections Q. because the prediction of private boxes is not accurate, only classification loss is calculated. By regarding the private boxes b, E [R[] as proposal boxes and feeding them into ROI Head, we obtain the classification probabilities p3 for student. Then standard distillation loss is employed to distill all private knowledge from both teachers to the target detector Cpri = Lu(( Py). where Lu((;-) is the Kullback-Leibler divergence and Py are prediction results from cloud detector or CLIP detector.
By integrating different source knowledge, the class embedding should be different from CLIP detector. Similar as previous prompt learning method, we also align per-class embeddings Cstu to visual class prototypes é, based on consistent detections computed as Eq 2 Latignn = ||ē, estul ll.
Inconsistent detections knowledge distillation. As shown in Fig 3 a Consistent Knowledge Generation network (CKG), noted as Fo,.,: is proposed to do decision-level fusion which refines inconsistent detections to consistent ones. Specifically, CKG takes the inconsistent box features fstu E R P|xc from target detector, inconsistent visual feature class prototypes geld and aclip, inconsistent probabilities Petd and Pelip as input. It outputs the consistent probabilities Poky as follows,
Pekg 5(W 4 Ped + Weisp C Petsp);
(8)
where wild CA (fstu, adid) and Welpp CA2(fstu adlig) are two adaptive weights generated by two cross-attention modules [19]. C represents the dlement-wise multiplication and 8(-) represents the softmax function. The architecture of cross-attention module is represented C'A(f

---

Potd
2dld
Cloud Detector Protetypes
Cloud Detector Probability
?
Multiply
Fenture
Sufunaa
Poky
Multiply
20
Class Profotypes
Proimbiliit
Polip
Figure 3: The structure of Consistent Knowledge Generation (CKG) network.
8(Q(fst)) & (())²) s V(X), where Q(-). K(-) and V(-) are the mapping functions, and 6 represents the matrix multiplication. With the cross-attention module, the features fstu are compared with the class prototypes through query and key, making the weights generation process more reliable.
Since there do not exist labels in the target domain, a gradient direction alignment is proposed to train CKG network in self-promotion way, which is also our key contribution. The idea is based on the following observation. Since the consistent detections can be regarded as ground truths in the target domain, they provide an optimization direction toward an optimal target detector. So the gradient direction from consistent detections is used as the supervised signal to train CKG network. Specifically, the gradients of consistent detections and inconsistent detections are computed using L2 loss as follows,
(9) where I(.) is the one-hot vector function, Peta are the predicted probabilities corresponding to the target detector features fstu on consistent detections; psta are the predicted probabilities correspond- ing to fetu on inconsistent detections. Then, the CKG network is optimized by aligning ğ to g by cosine similarity. Meanwhile, CKG should also be consistent on consistence detections, i.e.. Poky 6C (fstu e,) & Prid + CA2(fstus ep) @ Petip). is consistent with the label i, So the
g Ver P I((m))la, g
0z Pstu Pokg l|
total loss for training CKG network is
min Lekg (1 sim g)) + Lealee I((m)
(10)
9ckg
The CKG network and target detector are updated mutually. First, CKG is optimized based on target detector, then the output Peky is used in turn to update target detector. In order to avoid the interference of low-confidence predictions, we use a threshold я to filter out those low-confidence predictions, resulting in p.t. and Poky so target detector is optimized as follows:
Line
Laa(PStu Pakg).
(11)
Remark. Traditional decision-level fusion method (55) employs simple averaging to merge knowledge from various sources, where different sources share one RPN network to generate fully matched detections. contrast, we achieve decision-level fusion for two unrelated models based on divide- and-conquer strategy without ground truth. For inconsistent detections, our method uses gradient direction alignment to optimize the fusion network in a self prof-promotion manner.
Overall optimization. In each iteration, the CKG network is first updated via Eq. 10 Then we update target detector via the following objective function:
min feon + 71 Line + Ppr + alignn
(12)
87
where 71 and 722 are two hyperparameters. X is fixed as 10 as in Eq.(4). The CLIP detector is updated by Petip = 17 00pp + (1 7)) 0r, where 7) = 0.9996 in 2g. enabling the integrated knowledge in the target detector flows into the CLIP detector gradually, thus achieving better knowledge integration.
Remark. Although CLIP is utilized in COIN, CODA does not impose restrictions on the use of CLIP.
4 Experiments
Datasets. The problem CODA enables versatile target domain adaptation based on cloud detector, so there no limitation to transfer scenarios, unlike the problem settings of UDAOD, SFOD and
7

---

Table I: Results on Toggy-Cityseaper and BDD100K under GDINO. Object detection adaptation settings: U Unsupervised, SF Source-free free, BB Black- Box, - Cloud. det: detector.
#DD100K
Methoos
Type Tuck Cur Rder Pson Tain Mele Bele Bus mAP
Methods
Type Tuck Car Rder Pson
Mele Bele
Bus
mAP
MITR
g
U
21.9
44.0
41,4
30.6
40,6
28.3
35.6
38.6
35.1
SIGMA++
21.1
65.6
30.4
47.5
17.8
27.1
26.3 33.7
ICR-CCR
U
27.2
49,2
43,8
32.9
36,4
30.3
34.6
45.1
37,4
PT M
25,8
52.7
39.9
40.5
23.0
28.8
33.8
34,9
SED 85
SF
25.5
44.5
40.7
33.2
22.2
28.4
34.1
39.0
33.5
SED B5
SF 20.6
50.4
32.6
32,4
18.9
25.0
23.4
29.0
LODS 33
SF
27.3
48.8
45.7
34.0
19.6
33.2
37.8
39.7
35.8
PETS 39
SP
19.3
62.4
34,5
42.6
17.0
26,3
16.9
31.3
SFOD 10
SF
28.1
44.6
44.1
32.3
29.0
31.8
38,9
34.3
35.4
SFOD 1D
SF
33.2
36.3
50.2
26.6
28.2
24.4
22.5
31.6
IRG 53
SF
24.4
51,9
45.2
37.4
25.2
31.5
41.6
39.6
37.1
BT 13
SF
24.2
50.4
34,6
32.7
24.7
28.5
24.9
31.4
LPU 9
SF
24.0
55.4
50.3
39.0
21.
30.3
44.
46.0
38.8
LPU 9
SP
24.5
55.2
38.9
41.4
20,9
30.4
23.2
33.5
BiMem
+7
BB
23.4
56.9
42.5
28.5
32.4
39.7
38.4
DRU 28
32.5
36.6
Cloud det
+0
C
30.8
47.5
18.6
34,3
21,0
34.6
41.1
47.4
34.4
Cloud det
400
38.7
46.0
11.4
49.2
37.8
33.5
47.4
37.7
CLIP +
C
9.7
28.6
11.5
19.5
1.1
12.8
17.9
21.9
15.4
CLIP 4
23.6 31.1
4.4
6.7
18.0
11.4
27.7
17.5
CLIP det
C
8.2
46,9
27.5
34.1
16.5
24.9
31.5
36.2
28.2
CLIP det
34.3
53.4
14.1
31.7
28.7
24.6
36.7
31.9
COIN
C
27.4
57.9
42.3
41.6
25.9
32.7
43.1
39.0
COIN
32.0
33.0
40.6
39.7
Oracle
32,5 67.1 50.8
46.7 43.1
34.4
43.2
54.4 46.5
Oracle
54.0
70.6 42.3
51.4 35.8 41.5 53.2 49.8
Table 2: Results on Clipart under GDINO. Object detection adaptation settings: SF Source-free, U Unsupervised, Cloud. det: detector.
Methods
Type
Aero Bele Bird
Boat
Bod
Bus
Car
Cat
Chair
Cow
Thle
Dog
His
Bike
Pson
Pint
Shep
Sofa
Tain
Tv
mAP
MGADA
75
U
35,5
64.6
27.8
34,5
41.6
66.4
49.8
26.8
43.6
56.7
24.3
20,9
43.2
84.3
74,2
41.1
17.4
27.6
56.5
57.6
44.8
SIGMA++
34
U
36.3
54.6
40.1
31.6
58.0
60.4
46.2
33.6
44.
66.2
25.7
25.3
44.4
58.8
64.8
55.4
36.2
38.6
54,1
59.3
46.7
CIGAR 41
55.0
39.2
30.7
60.1
58.1
46.9
31.8
47.0
61.0
21.8
26.7
44.6
52.4
68.5
54.4
46.2
TFD 54
U
27.9
64.8
28.4
29.5
25.7
64.2
47.7
13.5
47.5
50.9
50.8
21.3
33.9
60.2
65.6
42.5
15.1
40.5
45.5
48.6
41.2
LODS 33
SF
43.1
61.4
40.1
36,8
48.2
45.8
48.3
20.4
44.8
53.3
32.5
26.1
40.6
86.3
68.5
48.9
25.4
33.2
44.0
56.5
45.2
IRG 53
SF
20.3
47.3
27.3
19.7
30.5
54.2
36.2
10.3
35.1
20.6
20.2
12.3
28.7
53.1
47.5
42.4
9.1
21.1
42.3
50.3
31.5
WSCoL
SF'
42.8
57.2
34.9
43.2
41.5
78.9
44.7
50.8
54.0
40.1
19.6
88.2
61.2
46.5
46.4
Cloud det
30
76.2
91.8
67.4
62.7
60.2
82.2
68.4
43.7
77.9
52.9
69.8
39.3
64.4
85.6
88.1
78.9
30.8
56.9
12.9
66.5
66.8
CLIP 17
62.3
70.1
42.5
42.7
50,9
50,0
44.8
47.8
22.8
59.5
28.6
34.2
43.7
51.4
61.1
59.8
24.1
28.1
50.4
50.5
46.3
CLIP del
61.4
56.5
46.9
48.8
57.4
54.1
49.7
40.2
32.7
48.7
16.6
33.8
51.4
50.4
62.8
60.6
25.7
28.8
43.9
52.6
46.2
COIN
82.0
87.6
70.1
58.1
63.7
68.7
55.2
70.5
59.0
58.8
68.6
82.9
88.0
67.3
68.5
Ornele
100 99.1 98.7
96.5
96.3
100
99.5
99.7
100
99.9
99.4
100
99.4 100
99.8
99.4
100
100
100
100 99.4
Black-box DAOD. Specifically, we validate the effectiveness of the proposed COIN method on six
object detection datasets, e.g., Cityscapes 11, Foggy-Cityscapes Clipart 25 BDD100K 63
KITTI [66 and Sim10K [26].
Implementation details. By default, we use the Swin-B 42 version of GDINO 40 as our cloud detector, where class predictions are provided in probability format. Additionally, in Appendix, we present results using the Swin-L 42 version of GLIP 332 as an optional alternative, which offers class predictions in the form of confidence score. SAM is not chosen here due to its need for weak supervision (65) For each dataset, both CLIP detector and target detector are based on the same version of CLIP visual encoder. Specifically, for Clipart, to be consistent with the compared methods ResNet101 (21) used. While for others, ResNet50 [21] is used. The hyperparameters 7/ 72 and π are set to 0, 0.1 and 0.7 by default. The shorter side of the image is resized to 600 during training and testing, and the reported mean average precision (mAP) based on an IoU threshold of 0.5. For more details about datasets, network architectures, algorithm et al, please refer to Appendix.
49 33
4.1 Comparison with -tate-of-the-arts
Since these do not exist any works on the CODA problem, we compare our method COIN with UDAOD, SFOD and Black-box DAOD methods, since their settings are closest to ours and the target domain is consistent. The performances of CLIP detector and cloud detector are also compared, which shows our method is better than both of them. UDAOD methods are DA -Asatte 8 MTOR (3]
SCL [50], ICR-CCR(59) SIGMA+ 33 PT 77. MGADA (75), CIGAR 41
TFD 54 MAF 23
ATF 24
SFOD methods are SED 35). LODS (33. A SFOD I0. IRG 53
PETS 39 LPU 9
BT 13. DRU 28 and WSCoL 61 Black box DAOD method is BiMem 67 CLIP represents the
original CLIP predictions with boxes from cloud detector. CLIP detector represents the detector after pre-training. Oracle represents target detector under supervision from ground truth. The results of compared methods in the tables are cited from their papers.
8

---

Table 3: Quantitative results on KITTI under GDINO. U Unsupervised, C Cloud. det: detector.
Type
Methods
AP of Car
Methods
AP of Car
Methods
AP of Car
Methods
AP of Car
DA-Faster 8
64.1
MAF
23
72.1
SCL [50]
72.7
ATF [24]
73.5
C
Cloud det 40
45.2
CLIP
62.1
CLIP det
79.9
COIN
80.8
Table 4: Quantitative results on Cityscapes and Sim10K under GDINO. C Cloud. det: detector.
Cityscapes
Sim1 OK
Methods
Type
Truck
Car
Rider
Person
Train
Mcycle
Beycle
Bus
mAP
Car
Cloud det 40
37.5
59.9
16.4
43.4
26.1
42.7
48.4
62.6
42.1
46.5
CLIP 47
C
15.9
36.9
15.5
27.8
0.9
15.7
20.5
31.8
20.6
46.4
CLIP det
C
11.3
55.8
35.1
39.1
33,8
32,0
33.7
44.7
35.7
60.0
COIN
C
26.9
64.3
47.5
47.0
26.4
44.4
46.9
52.8
44.5
62.4
Oracie
34.7
70.4
56.4
50.5
43.0
38.7
46.9
58.9
49.9
79.2
Sensitivity analysis of
Sensitivity analysis of (Y2
Sensitivity analysis y2 (Y100.1)
mAP(%))
MAP(%) (%)
MAP (%)
value
value V/
value
Figure 4: Hyperparameter analysis with respect क. 71 and 72 Foggy- -ityy--sssss under GDINO.
Quantitative results for GDINO [4) are shown Table and results for GLIP 32 are shown in Appendix. First, the existing methods are compared across four commonly used target domain BDD100K, Clipart, and KITTI. Specifically, our method COIN sig- nificantly outperforms cloud detector by +4.6% (from 34.4% to 39.0%) CLIP by +18.7% (from 62. 1% to 80.8%) on KITTI. This demonstrates that our COIN can identify valuable knowledge for adaptation, regardless of the performance of CLIP (bad while good on KITTI). And CLIP detector improves the mAP by a large margin of + +.2.8% on Foggy-Cityseune
datasets:
Cityscapes and
Dregy-Cityscapes
+14.4% BDD100K, and +17.8% KITTI compared with CLIP, strongly demonstrating the effectiveness of the knowledge dissemination stage. Moreover, GDINO and CLIP already achieve surprising performance of 66.8% and 46.3% on Clipart, proving the superiority of CODA compared to traditional adaptation settings.
Second, since CODA enables versatile target domain adaptation with open categories and scenarios, experiments on Cityscapes for all 8 categories and Sim 10K are conducted. Existing methods are not compared, for Cityscapes they the source domain. From Table 4 we see that the proposed COIN achieves the best performance. Specifically, for Sim10K, when cloud detector and CLIP perform similarly, COIN still brings significant improvement + +15.9% compared with cloud detector. The extensive quantitative results above not only demonstrate the wide applicability of CODA but also validate the effectiveness and robustness of our proposed method COIN.
only detect the category while Sim10K is usually used as
4.2 Further Analysis.
Ablation study. As shown in Table 5 ablation studies are conducted Foggy- Doggy-Cityscapes and Cityscapes. Specifically, Catign+ CLIP detector or Latign COIN represent prompt learning for CLIP detector target detector respectively; Leons Line and Cpri represent the distillation losses of consistent, inconsistent and private detection respectively. (1) For CLIP detector, prompt learning improves the performances from 27.4% and 35.1% to 28.2% and 35.7% Foggy- Foggy-Cityscape Cityscapes and Cityscapes respectively. (2) For the proposed COIN method, all proposed components are effective which demonstrates that method is able achieve judicious knowledge integration.
Ablation study for decision-level fusion. To further validate the effectiveness of decision-level fusion, our proposed COIN is compared with four experimental groups, as shown in Table 6 Using

---

Table 5: Ablation study on Foggy-Cityscen and Cityscapes under GDINO. det: detector.
Losses
mAP
Methods
Catign
Leon
Line
Ppii
Seegy-Ciyyccapes
Cityscapes
Cloud det 40
X
34.4
42.1
CLIP 47
X
X
X
X
15.4
20.6
X
X
X
27.4
35.1
CLIP det
V
X
X
X
28.2
35.7
X
V
X
X
36.7
41.7
V
V
X
X
37.1
42.4
37.5
42.9
38.4
43.8
COIN
39.0
44.5
Table 6: Ablation study for decision-level fusion of inconsistent detections on Fogey- Sogey-Cityscapes under GDINO. Detections are filtered by 0.7 for fair comparison. det: detector. probs: probabilities. avg: average, s-avg: score-weighted average.
Methods
Truck
Car
Rider
Person
Train
Meycle
Beycle
Bus
mAP
COIN w/ cloud det probs
25.1
56.1
45.3
40.1
20.5
33.7
41.3
39.3
37.7
COIN w/ CLIP det probs
22.1
56.4
44.5
39.5
26.8
32.4
40.4
42.4
38.1
COIN avg
24.8
55.8
44.1
39.9
21.7
32.8
43.7
38.0
COIN w/ s-avg
24.2
56.4
45.9
40.7
24.1
31.3
40.4
41.7
38.1
COIN w/ CKG
27.4
57.9
42.3
25.9
32.7
41.2
43.1
the cloud detector alone achieves mAP of 37.7%. Surprisingly, using the CLIP detector alone achieves an even higher mAP of 38,1%, attributed to the gradual parameter updates of the CLIP detector during training, allowing integrated knowledge to flow into it. Additionally, using both probabilities simultaneously with avg s-avg approaches yield similar results. While proposed CKG unsurprisingly achieves the best results, with a mAP improvement of +0.9% (from 38.16 % to 39.0%). This strongly demonstrates the effectiveness of our proposed decision-level fusion.
Hyperparameters sensitivity analysis. We conduct sensitivity analysis on n, ?1 7/2 on Foggy- Cityscapes, as shown in Fig For parameter our method achieves relatively stable results over wide range. For parameters 7 and 72, we first set "/2 to 0. and vary "i across six distinct values ranging from 0.1 to 1.0. Then, we reciprocate the process for 72. The outcomes are stable, with mAP oscillating within a band between 38.0% and 39.0%. This confirms the robustness of COIN.
Conclusion
We proposed novel method termed COIN for the proposed cloud object detector adaptation (CODA). The open source CLIP model is adapted to help distill knowledge in a divide-and -nd--uuu-- manner. To efficiently disseminate knowledge from CLIP and cloud detector, CLIP detector is designed and adapted to the target domain by prompt learning. Then, three kinds of detections are split and distilled target detector respectively. Consistent and private detections without loss of generality. Prompt leaning is applied again for target detector to fit target domain. To eliminate conflicts, a consistent knowledge generation network (CKG) is proposed for decision-level fusion. A gradient direction alignment loss is proposed to learn this network in a self-promotion way. Experimental results validated the effectiveness of our method. COIN it can also be utilized to other tasks, e.g., classification or semantic segmentation.
used supervision signals
limited detection task;
Acknowledgments and Disclosure of Funding
This work was supported by the National Natural Science Foundation of China (62276048, 62476169), Chengdu Science and Technology Projects (2023- YF06-00009-H2)) and Postdoctoral Fellowship Program of CPSF (GZC20233323).
10

---

References
[1] Hanoona Bangalath, Muhammad Maaz, Muhammad Uzair Khattak, Salman H Khan, and Fahad Shahbaz Khan. Bridging the gap between object and image-level representations for open-vocabulary detection. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 33781-33794. Curran Associates, Inc., 2022.
[2] Léon Bottou. Large-scale machine learning with stochastic gradient descent. In Yves Lecheval- lier and Gilbert Saporta, editors, Proceedings of COMPSTAT 2010, pages 177-186, Heidelberg, 2010. Physica-Verlag HD.
[3] Qi Cai, Yingwei Pan, Chong-Wan Wah Ngo, Xinmei Tian, Lingyu Duan, and Ting Yao. Exploring object relation in mean teacher for cross-domain detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[4] Shengcao Cao, Dhiraj Joshi, Liang-Yan Gui, and Yu-Xiong Wang. Contrastive mean teacher for domain adaptive object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23839 23848,, June 2023.
[5] Chaoqi Chen, Jiongcheng Li, Zebiao Zheng, Yue Huang, Xinghao Ding, and Yizhou Yu. Dual bipartite graph learning: A general approach for domain adaptive object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2703-27 27 12, October 2021.
[6] Chaoqi Chen, Zebiao Zheng. Xinghao Ding, Yue Huang, and Qi Dou. Harmonizing transfer- ability and discriminability for adapting object detectors. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2020.
[7] Meilin Chen, Weijie Chen. Shicai Yang. Jie Song, Xinchao Wang, Lei Zhang, Yunfeng Yan, Donglian Qi, Yucting Zhuang, Di Xie, et al. Learning domain adaptive object detection with probabilistic teacher. In International Conference on Machine Learning, pages 3040-3055. PMLR, 2022.
[8] Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Domain adaptive faster r-enn for object detection in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
[9] Zhihong Chen, Zilei Wang, and Yixin Zhang. Exploiting low-confidence pseudo-labels for source-free object detection. In Proceedings of the 31st ACM International Conference on Multimedia, MM '23, page 5370-5377 New York, NY, USA, 2023. Association for Computing Machinery.
[10] Qiaosong Chu, Shuyan Li, Guangyi Chen, Kai Li, and Xiu Li. Adversarial alignment for source free object detection. Proceedings of the AAAI Conference on Artificial Intelligence, 37(1):452-460, Jun. 2023.
[11] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), June 2016.
[12] Jinhong Deng, Wen Li, Yuhua Chen, and Lixin Duan. Unbiased mean teacher for cross-domain object detection. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4091 4091-4101, June 2021.
[13] Jinhong Deng, Wen Li, and Lixin Duan. Balanced teacher for source-free object detection. IEFE Transactions on Circuits and Systems for Video Technology, pages 1-1, 2024.
[14] Jinhong Deng, Dongli Xu, Wen Li, and Lixin Duan. Harmonious teacher for cross-domain object detection. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 23829-2822 23838, June 2023.

---

[15] Changlong Gao, Chengxu Liu, Yujie Dun, and Xueming Qian. Csda: Learning category- scale joint feature for domain adaptive object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 11421-11430, October 2023.
[16] A Geiger, P Lenz, C Stiller, and R Urtasun. Vision meets robotics: The kitti dataset. The Interaational Journal of Robotics Research, 32(1 12:111:1231 2013.
[17] Xiuye Gu, Tsung-Yi Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. In International Conference on Learning Representations, 2022.
[18] Yan Hao, Florent Forest, and Olga Fink. Simplifying source-free domain adaptation for object detection: Effective self-training strategies and performance insights. arXiv preprint arXiv:2407.07586. 2024.
[19] Yanchao Hao, Yuanzhe Zhang, Kang Liu, Shizhu He, Zhanyi Liu, Hua Wu, and Jun Zhao. An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge. In Regina Barzilay and Min-Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 221-231, Vancouver, Canada, July 2017. Association for Computational Linguisties.
[20] Boyong He, Yuxiang Ji, Zhuoyue Tan, and Liaoni Wu. Diffusion domain teacher: Diffusion guided domain adaptive object detector. In ACM Multimedia 2024, 2024.
[21] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
[22] Mengzhe He, Yali Wang, Jiaxi Wu, Yiru Wang. Hanqing Li, Bo Li. Weihao Gan, Wei Wu, and Yu Qiao. Cross domain object detection by target-perceived dual branch distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022.
pages
[23] Zhenwei He and Lei Zhang. Multi-adversarial faster-renn for unrestricted object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
[24] Zhenwei He and Lei Zhang. Domain adaptive object detection via asymmetric tri-way faster- renn. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer ECCV 2020, pages 309-324, Cham, 2020. Springer International Publishing.
[25] Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa. Cross-domald weakly-supervised object detection through progressive domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
[26] Matthew Johnson-Roberson, Charles Barto, Rounak Mehta, Sharath Nittur Sridhar, Karl Rosaen, and Ram Vasudevan. Driving in the matrix: Can virtual worlds replace numan-generated annotations for real world tasks? In 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2017.
[27] Mikhail Kennerley, Jian-Gang Wang, Bharadwaj Veeravalli, and Robby T. Tan. Cat: Exploiting inter-class dynamics for domain adaptive object detection. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16541-16550, June 2024.
[28] Trinh Le Ba Khanh, Huy-Hung Nguyen, Long Hoang Pham, Duong Nguyen-Ngoc Tran, and Jae Wook Jeon. Dynamic geraining-uudaatin mean teacher for source-free object detection. arXiv preprint arXiv:2407.16497, 2024.
[29] Muhammad Uzair Khattak, Syed Talal Wasim, Muzammal Naseer, Salman Khan, Ming- Miug-HHuann Yang, and Fahad Shahbaz Khan. Self-regulating prompts: Foundational model adaptation without forgetting. Proceedings of the IEEF/CVE International Conference on Computer Vision (ICCV), pages 15190-15200 October 2023.
12

---

[30] Weicheng Kuo, Yin Cui, Xiuye Gu, AJ Piergiovanni, and Anelia Angelova. Open- object detection upon frozen vision and language models. In The Eleventh International Conference Learning Representations, 2023.
[31] Haochen Li, Rui Zhang, Hantao Yao, Xin Zhang, Yifan Hao, Xinkai Song, Xiaqing Li, Yongwei Zhao, Ling Li, and Yunji Chen. Da-ada: Learinn domain- aware adapter for domain adaptive object detection. arXiv preprint arXiv:2410.0900 2024.
[32] Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, and Jianfeng Gao. Grounded language-imas pre- pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10965-10975 June 2022.
[33] Shuaifeng Li, Mao Ye, Xiatian Zhu, Lihua Zhou, and Lin Xiong. Source-free object detection by learning to overlook domain style. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8014-8023, June 2022.
[34] Wuyang Li, Xinyu Liu, and Yixuan Yuan. Sigma++: Improved semantic-complete graph matching for domain adaptive object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(7):9022-9088
2023.
[35] Xianfeng Li, Weijie Chen, Di Xie, Shicai Yang. Peng Yuan, Shiliang Pu, and Yueting Zhuang. A free lunch for unsupervised domain adaptive object detection without source data. Proceedings of the AAAI Conference on Artificial Intelligence, 35(10)88474-848) May 2021.
[36] Jian Liang, Dapeng Hu, Jiashi Feng, and Ran He. Dine: Domain adaptation from single and multiple black-box predictors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8003-8013, June 2022.
[37] Chuang Lin, Peize Sun, Yi Jiang, Ping Luo, Lizhen Qu, Gholamreza Haffari, Zehuan Yuan, and Jianfei Cai. Learning object-language alignments for open-vocabulary object detection. In The Eleventh International Conference on Learning Representations, 2023.
[38] Luojun Lin, Qipeng Liu, Xiangwei Zheng, and Zheng Lin. Slow-fast adaptation for source-free free object detection. In 2024 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6, 2024.
[39] Qipeng Liu, Luojun Lin, Zhifeng Shen, and Zhifeng Yang. Periodically exchange teacher-student for source-free object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 6414-6424, October 2023.
[40] Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303..m 055999 2023.
[41] Yabo Liu, Jinghua Wang, Chao Huang, Yaowei Wang, and Yong Xu. Cigar: Cross-modalle graph reasoning for domain adaptive object detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 23776-23786, June 2023.
[42) Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
[43] Zongyang Ma, Guan Luo, Jin Gao, Liang Li, Yuxin Chen, Shaoru Wang, Congxuan Zhang, and Weiming Hu. Open-vocabulary one-stage detection with hierarchical visual- visual-language knowledge distillation. Proceedings the the IEFE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 14074-14083, June 2022.
[44] Muhammad Akhtar Munir, Muhammad Haris Khan, M. Sarfraz, and Mohsen Ali. Ssal: Synergizing between self-training and adversarial learning for domain adaptive object detection. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 22770-22782. Curran Associates, Inc., 2021.
13

---

[45] OpenAl. Gpt-4 technical report, 2023.
[46] Qucheng Peng, Zhengming Ding, Lingjuan Lyu, Lichao Sun, and Chen Chen. Rain: regulariza- tion on ipput and network for black- box domain adaptation. In Proceedings of the Thirdy-Seoond International Joint Conference on Artificial Intelligence, pages 4118 4126, 2023.
[47] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learming, volume 139 of Proceedings of Machine Learning Research, pages $748-8763, PMLR, 18-24 Jul 2021.
[48] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun, Faster r-cnn: Towards real-time object detection with region proposal networks. C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28, Curran Associates, Inc., 2015.
[49] Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko. Strong-weak distribution alignment for adaptive object detection. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[50] Zhiqiang Shen, Harsh Maheshwari, Weichen Yao, and Marios Savvides. Scl: Towards accurate domain adaptive object detection via gradient detach based stacked complementary losses. arXiv preprint arXiv: (eXXv:9911.025553 2019.
[51] Simon Varailhon, Masih Aminbeidokhti, Marco Pedersoli, and Eric Granger. Source-free domain adaptation for yolo object detection. arXiv preprint vrXiv:2409.16555 2024.
[52] Vidit Vidit, Martin Engilberge, and Mathieu Salzmann. Clip the gap: A single domain gen- eralization approach for object detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 2219-3229, June 2023.
[53] Vibashan VS, Poojan Oza, and Vishal M. Patel. Instance relation graph guided source-free domain adaptive object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3520-3550 5520-3530, June 2023.
[54] Haoan Wang, Shilong Jia, Tieyong Zeng, Guixu Zhang, and Zhi Li. Triple feature disentangle- ment for one-stage adaptive object detection. Proceedings of the AAAI Conference Artificial Intelligence, 38(6):500-5549, Mar. 2024.
[55] Jiaxi Wu, Jiaxin Chen, Mengzhe He, Yiru Wang, Bo Li, Bingqi Ma, Weihao Gan, Wei Wu, Yali Wang, and Di Huang. Target-relevant knowledge preservation for multi-source domain adaptive object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1301-5310, June 2022.
[56] Size Wu, Wenwei Zhang, Sheng Jin, Wentao Liu, and Chen Change Loy. Aligning bag of regions for open- vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15254-15266 June 2023.
[57] Mingxuan Xia, Junbo Zhao, Gengyu Lyu, Zenan Huang, Tianlei Hu, Gang Chen, and Haobo Wang. A separation and alignment framework for black-box domain adaptation. Proceedings of the AAAI Conference on Artificial Intelligence,
Mar. 2024.
[58] Siying Xiao, Mao Ye, Qichen He, Shuaifeng Li, Song Tang, and Xiatian Zhu. Adversarial experts model for black-box domain adaptation. In Proceedings of the 32nd ACM International Conference on Multimedia, MM '24, page 1991, New York, NY, USA, 2024. Association for Computing Machinery.
[59] Chang- Dong Xu, Xing-Ran Zhao, Xin Jin, and Xiu-Shen Wei. Exploring categorical regular- ization for domain adaptive object detection. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
14

---

[60] Jianfei Yang, Xiangyu Peng. Kai Wang. Zheng Zhu, Jiashi Feng, Lihua Xie, and Yang You. Mitigating confirmation bias for domain adaptation of black-box predictors. In The Eleventh International Conference on Learning Representations, 2023.
Divide to adapt:
[61] Jiuzheng Yang. Song Tang, Yangkuiyi Zhang, Shuaifeng Li, Mao Ye, Jianwei Zhang, and Xiatian Zhu. Rethinking weak-to-strong augmentation in source-free domain adaptive object detection. arXiv preprint arXiv:2410.05557. 2024.
[62] Ilhoon Yoon, Hyeongjun Kwon, Jin Kim, Junyoung Park, Hyunsung Jang, and Kwanghoon Sohn. Enhancing source-free domain adaptive object detection with low-confidence pseudo label distillation. arXiv preprint wrXiv:2407.13524 2024.
[63] Fisher Yu, Haofeng Chen, Xin Wang. Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. Bdd100k: A diverse driving dataset for heterogeneous mul- titask learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern 2020.
recognition, pages
[64] Haojian Zhang, Yabin Zhang, Kui Jia, and Lei Zhang. Unsupervised domain adaptation of black-box source models. arXiv preprint arXiv:2101.0220
2021.
[65] Haojie Zhang, Yongyi Su, Xun Xu, and Kui Jia. Improving the generalization of segmentation foundation model under distribution shift via weakly supervised adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23385-23395, June 2024.
[66] Haotian Zhang, Pengchuan Zhang, Xiaowei Hu, Yen-Chun Chen, Liunian Li, Xiyang Dai, Lijuan Wang, Lu Yuan, Jenq-Neng Hwang, and Jianfeng Gao. Glipv2: Unifying localization and vision-language understanding. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 36067-360888 Curran Associates, Inc., 2022.
[67] Jingyi Zhang. Jiaxing Huang, Xueying Jiang, and Shijian Lu. Black- box unsupervised domain adaptation with bi-directional atkinson-shiffrin memory. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), pages 11771-11782, October 2023.
[68] Shaojie Zhang. Chun Shen, Shuai Lü, and Zeyu Zhang, Reviewing the forgotten classes for domain adaptation of black-box predictors. Proceedings of the AAAI Conference on Artificial Intelligence, 38(15): 1880-166877 Mar. 2024.
[69] Yabin Zhang, Wenjie Zhu, Hui Tang, Zhiyuan Ma, Kaiyang Zhou, and Lei Zhang. Dual memory networks: A versatile adaptation approach for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages June 2024.
[70] Liang Zhao and Limin Wang. Task-specific inconsistency alignment for domain adaptive object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14217-14226, June 2022.
[71] Sicheng Zhao, Huizai Yao, Chuang Lin, Yue Gao, and Guiguang Ding. Multi-source-free domain adaptive object detection. International Journal of Computer Vision, pages 1-33, 2024.
[72] Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. Regionclip: Region-based language-image pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16793-16803, 2022.
[73] Kaiyang Zhou, Jingkang Yang. Chen Change Loy, and Ziwei Liu. Conditional prompt learning for Dision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16825,16825, June 2022.
[74] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for lanion-l models. International Journal of Computer Vision, 133(9):2377-2348, 2022.
[75] Wenzhang Zhou, Dawei Du, Libo Zhang, Tiejian Luo, and Yanjun Wu. Multi-granularity alignment domain adaptation for object detection. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9581-9590, June 2022.
15

---

A Appendix
A.1 More Related Works
Integrating large language-visual models (c.g., CLIP) into object detectors becomes popular. Existing methods usually use the text encoder of CLIP as the classifier, which can be broadly classified into two routes. The first route is based on knowledge distillation 177 43 I. which aims to distill the knowledge of the CLIP model into closed-set detectors. For example, ViLD [17] utilizes instance-level visual-to-visuee knowledge distillation; HierKD [43 proposes hierarchical distillation method with global-level language-tor to-visual and instance-level visual-to-visse visual distillation. Since CLIP is trained images rather than object regions, the second route, region-text alignment 30 37 (6), aims to align image region features to the fixed text region features. For instance, RegionCLIP [72 aligns image regions with region-level descriptions using a contrastive loss: VLDet 37 formulates the alignment as set matching problem where a set of regions and a set of words are aligned. F-VLM (30] uses the CLIP vision encoder as the frozen backbone and combines the detection scores and CLIP predictions as the final output. Unlike these methods, our work combines the knowledge of CLIP to help adaptation of cloud detector. To fully explore the knowledge from CLIP, we also utilize CLIP vision encoder as backbone and ROI Head feature extractor. Moreover, prompt learning technique is embedded for adapting CLIP knowledge to target domain.
CLIP based detector.
Large detection model. In order to achieve success in open-set detection, large detection models leverage massive image-text pairs for training, breaking the constraints of categories and scenes while
attaining robust detection capabilities. GLIP [32]. CLIPv2 [66], and GDINO [40) are representative advances. Specifically, GDINO [40] integrates detection and grounding into a unified framework. By leveraging powerful detector pre-trained on multiple datasets, it delivers impressive performance across various downstream tasks. Without loss of generality, we choose GDINO and GLIP as the cloud detector in this work.
A.2 Methodological supplements
A.2.1 Network details
Due to the space constraints in the main text, we provide a detailed description here for the two designed networks: object detector and Consistent Knowledge Generation network (CKG).
Detector architecture is based on the two-stage Faster R-CNN 48 framework. Specifically, the ResNe150 [2I] or ResNet101 21 version of CLIP visual encoder G is split into G1 and G2 to be the backbone and feature extractor for ROI head following Faster R- R-CNN, where G2 is the last residual block. With input target image æ. the backbone G1 firstly produces output feature map E xxxx, where H. W and C₁ represent the height, width and dimension of the feature map. Then, based on g. RPN generates set of region proposals R RPN(g). For proposal r E RR ROI pooling and G2 are utilized to extract a region feature fr = G2( ROT(g, r)), where R fr E 7x7xxx and C2 is the feature dimension. Since CL.P is pre-trained for classification task, for can not be used for box regression, thus a transformation network Trans, composed of mean pooling and three linear layers (with two Leaky ReLUs), is used to endow the localization ability, To project feature into semantic space for final classification, a linear layer 4. is used to obtain the box feature f (T (a where f E RC and is the dimension of semantic space. While a linear layer 4, is used to get the box prediction b = 4o (TTrans(ffr)) where b E R Finally, the class probability p E IR N.++ of the box feature is calculated by computing the similarity with the per-class embeddings Specifically, the i-th class probability E R is calculated by
RRN++1)xC where background is also considered to be class.
exp(sim(f,et))
P
No+1 vrp(sim((f.ett)/ 1=
(13)
where sim(.,) is the cosine similarity function and 0.01 is the fixed temperature.
The i-th class embedding e' is obtained as follows. Since no target domain information wrapped in the simple prompt template, like "a photo of a [CLS].". the embedding generated with it does not fit the target domain. So, a trainable prompt template PT, "a photo of a [M),,, [CLS]." is designed to capture target specific attributes, where E' is placeholder and M is fixed to 4. By wrapping the
16

---

Table 7: The detailed 81 prompt templates for CLIP model. They are used to collect the classification probabilities to pre-train CLIP detector. zoom in for best view.
Number
Tamplates
1-2
"[larget domaain name] style [CLS),"
"a [Larget domain name)
style pluto of [CLS],"
3-4
"a [turget domain namn]] style had photo a [CLSS]"
"a [target domain name]
style photo of many [CI SJ"
5-6
[target domain name] style sculpture of [CLS].
"a [target domain name]
style photo of the hard to see [CLS],
7-8
"a [target domain name] style low resolution photo of the (CLS).
" [target domain name)
style rendering of a (CLS).
"[target domain name] style gratliti of [CLS],"
"a [Larget domain name)
style bad photo
[CLS)."
11-12
"a [turget domain name] style cropped photo of the [CLSS].
"a [target domain name] style tatton of [CL.S]."
13-14
"the [target domain name] style embroidered [CLSS]"
"a [target domain name]
style photo of hard to see [CLS]]
15.16
[turge domain name] style bright photo of [CLSS],
"a [target domain name] style photo of clean [CISS],"
17-18
"a [target domain name] style photo of dirty [CLSS]
"a [target domain name]
style dark photo of the [CI.S]
19-20
[target domain name] style drawing of [CLS]]"
"a [target domain name) style photo of my [CLS),
21-22
"the [aarett domain name] style plastic (CLS)."
"a [target domain name)
style photo of the cool [CLS)""
23.24
[targt domain namne] style close up photo [CLSS]"
"a [target domain name] style black and white photo
(CLS)"
25-26
" [turget domain name] style painting of the [CL.S]]"
"a [target domain name] style painting of [CLSS]"
27-28
aa [target domain name] style pixclated photo of the [CLS).
"a [target domain name] style sculpture of the [CLS]]
29:30
*. larget domain name] style bright photo of the (CLS),"
"a [arget domain name) style cropped photo of ICLS)."
31-32
" [target domain name] style plastic [CLS]]
"a [target domain name] style photo of the dirty [CLS]*
33-34
Itarget domain name] style jpeg corrupted photo of ICLS)."
"a [target domain name) style blurry photo of the (CLS).
35-36
" Iurget domain namne] style photo of the (CLS),
"a [target domain name) style good photo of the [CLS]."
37:38
[urget domain name] style rendering of the [CI.S]]
"a [target domain name] style [CL.S] in a video game."
39-40
[target domain name] style photo of one [CLS]]
"a [target domain name] style doodle of [CLS]]
41-42
" [target domain name]] style close-up photo of the (CLS)."
"the [target domain name) style origami (CLS)."
43-44
"the Iurget domain name] style [CLS] video game."
"a |arret domain name) style sketch of &CLSS
45-46
" (target domain name] style doodle of the [CLS]"
"a [target domain name] style origami [CLS)."
47-48
Itaret domain name) style resolution photo of [CLS)"
"the [tagett domain name) style toy (CLS)."
49:50
" [urget domain namn]] style rendition of the (CLS)."
"a [targei domain name] style photo of the cleun [CLS]]"
51-52
[trggt domain name] style photo large [CI SS."
"a [target domain name] style rendition of a [CLSS]
53-54
"a [target domain name] style photo of nice [CLS]."
"a [target domain name] style photo of weird [CLS],
55-56
"a [target domain namec] style blurry photo of [CLS),"
"a [target domain name) style cartoon (CLS))"
57-58
"urget domain name] style of (CLS))"
"a [target domain name) style sketch of the (CLS)."
59-60
" [target domain name] style embroidered [CLS)."
"a [target domain name) style pisclated photo of a [CLS)."
61-62
"Iarget domain name| style stap the (CLS)."
"a [Larget domain name! style joey corrupted photo the [CLS),"
63 64
"a [turget domain name] style good photo a [CLS),
"a [target domain name] style plushie (CLSS).
65-66
[target domain name] style photo
nice [CLS]]
"a [target domain name] style photo of the small [CLS]]
67-68
" [target domain name] style photo of the weird [CLSS)"
"the [target domain name] style cartoon (CLS)."
69-70
"targt domain name) style art of the (CLS),
"a Itarget domain name) style drawing of the [CLS)""
71:72
[turet domain name] style photo the large (CLS)."
"a [Larget domain name) style black and white photo of (CLS)."
73-74
"t Itrget domain namel style plushic [CLS)."
"a Iaaget domain name) style dark photo of (CLS)"
75-76
"urget domuin name] style stap of [CLSS).
"[urgel demuin name] style grafliti of the [CLS)."
77-78
"a [urget domain name] style toy (CLSS)"
"[urget domain name] style itap of my [CLS]
79-80
"a [target domain name] style photo a cool [CLS),
"a [target domain name] style photo of small [CLS].
81
Barget domain name] style upose-up photo of the (CLS)."
i-th class name like "car",, a prompt Ps, e.g.. "a photo of a t' f 13 8 car." is obtained using PT. Then, the tokens T; for the i-th class are obtained by projecting P. into word embeddings. and the embedding of t is randomly initialized. Finally, the class embedding e ETT,) is obtained based on the frozen CLIP text encoder E.
Consistent Knowledge Generation network. As shown in Fig 3 a Consistent Knowledge Genera- tion network (CKG), noted as Fe takes the inconsistent box features fecu E || from target detector as input and output the consistent probabilities Poky E (I++)(N++1),, where |P| represent the number of inconsistent boxes 6, from image x. A simple description is used here since features fat come from those proposals that matched to 6m in practice.
Specifically, to facilitate the generation process, features fat are compared with the inconsistent vi- sual feature class prototypes ēcld E (NN++1)xx and æçlip (NN++1)x for cloud and CLIP detectors respectively, resulting in two adaptive weights weld E (R|P1x(N++1) and Walip E TR (IN(+))))
Weld = CAi(fftur eqld), Welip = CA2 Ep eglip).
(14)
where CA₁ and CA2 are two randomly initialized cross-attention modules [19 with the same archi- tecture. Finally, the adaptive weights Wald and Weisp are multiplied with inconsistent probabilities Peid and Patip. resulting the consistent probabilities Pekg as follows,
Pokg = 5(wWatd ② Pedd + Welsp C Petip),
(15)
17

---

Algorithm Our proposed COIN method.
Input: Unlabeled target domain D, class names C, cloud detector Fo CLIP model Fo., hyperpa-
rameters x, and ?/2-
Output: Optimized target detector Föy-
1: function COIN(D, C, Fa Fe.: "1,72)
2: Build and randomly initialize CLIP detector Foatiy: target detector Fay and CKG Faaso
3: for = 0 - JerrN Nun do
4: Sample target image from D;
5: Knowledge Dissemination:
6:
Obtain detections Yeid (bold, Pata) through cloud detector Peari
7:
Obtain CLIP model probabilities Pe and boxes be through Fé, with cloud boxes bata: Update visual feature class prototypes cp: Optimize CLIP detector Fo.uup with pa. be and Cp;
8:
Eq.
9:
b Eq
end for
11:
for = - 'iterNum' do
Sample target image x from D; Knowledge Separation:
13:
Obtain detections Yetd through cloud detector Faasi
15:
Obtain detections Vetip through CLIP detector Fecss
Eq.
Separate Yetd and Yetip into P. P and Q.
Eq. and Eq.
17:
Knowledge Distillation:
18:
Update consistent feature prototypes ép. inconsistent feature prototypes epi and ep delip
>> Similar to Eq
19:
if f S W WarmUpNunm then Optimize CKG Fe.n, and target detector Foy; else
20:
Eq
21:
22:
Optimize CKG Fost.- target detector For and CLIP detector Feam end
Eq 18
23:
24:
end for
25:
return Fer
end function
where @ represents the element-wise multiplication and 8(.) represents the softmax function. The architecture of the attention module is represented
CA(fstu, X) = A(fstu, &)XV(X)
(16)
A(fatu X) = 6(Q(fscc)) K(X))),
where @ represents the matrix multiplication, A(fstu: X) is the attention map and X represents class prototypes eold or aclip, Q(-) and (-) are the linear mapping functions that map the input of dimension C' to dimension C" according to standard cross-attention, and V-() is the linear mapping function that maps the class prototypes of dimension C to weights of dimension N. + 1. Thus, the generation of final weights weid and Welip are supported by the attention map between inconsistent features and class prototypes, making more reliable.
A.2.2 Prompt templates for CLIP model.
Naturally, target tailored prompt templates encapsulate the relevant attributes of the target domain, and the integration of multiple prompt templates can yield more precise results. Therefore, RegionCLIP
772 we design 81 prompt templates to collect class predictions from the CLIP model, shown in Table 7 For example, the first prompt "Cityscapes style car." for class "car" and the target domain name "Cityscapes" is easily obtained by filling the first template "[target domain name] style [CLS].". With these prompts, a class embedding e. for the class and j-th template similarly calculated, just like calculating the i-th class embedding for CLIP detector from above. Then ensemble is used compute the mean of these 81 embeddings, resulting the final class embedding for the i-th class e = Σ, e/ /81.
18

---

Table 8: Results on Foggy-Cityscapes and BDD100K under GLIP. det: detector.
Foggy-Citysn Cityscapes
BDD100K
Methods
Tuck
Car
Rder
Pson
Tain
Mele
Bele
Bus
mAP
Methods
Tuck
Rder
Pson
Bele
Bus
mAP
Cloud det 32
23,9
23.9
14.3
13.9
6.1
21.0
22.1
39.8
20.6
Cloud det
32
33.1
24.3
13.5
21.0
30,0
29.8
40.1
27.4
25.4
CP det
10.0
33.7
28.2
26.0
14,1
25.0
24.9
38.1
25.0
CLIP der
38.5
39.2
16.7
27.1
26.3
20.7
34.9
29.1
COIN-GLIP
10.7
35.7
38.1
28,9
10.3
28.5
30.4
39.3
COIN-GLIP
39.3
22,9
36.4
26.8
29.9
33.5
Oracle
32,5
67.1
50.8
46.7
43.1
34.4
43.2
54.4
46.5
Oracle
54.0
70.6
42.3
514
35.8
41.5
53.2
49.8
Table 9: Quantitative results on Cityscapes, KITTI and Sim10K under GLIP. det: detector.
Cityacapes
KITTI
Sim10K
Methods
Truck
Car
Rider
Person
Train
Meycle
Beycle
Bus
mAP
Car
Car
Cloud det 3
31.5
24.0
8.8
13.2
8.2
27.2
23.0
55.7
24.0
26.6
17.1
CLIP 47
18.3
20.6
14.5
13.1
1.4
17.4
12.7
36.9
16.9
26.8
16.6
30.9
COIN-GI.IP
23.3
40.3
29.4
33.0
17.0
35.0
33.1
56.6
33.5
$6.8
37.1
Oraele
34.7
70.4
56.4
50.5
43.0
38.7
46.9
58.9
49.9
95.8
79.2
A.2.3 Optimization and algorithm
Due to the risk of disruption caused by the randomly initialized target detector the parameters of the CLIP detector through exponential moving average (EMA), we divide the training process into two stages. In the first stage, the CLIP detector is fixed, and updates are applied to the CKG and target detector as follows,
min Lekg Beky
(17)
min Loon + 72hprr a 87
which allows the CKG receive effective training before distilling inconsistent detections. In the second stage, as described the main text, updates are applied to the CKG, target detector, and CLIP detector as follows,
min Lekg, Beks
min Lcon + Line + 72hori + AC 8y
(18)
Ociip n Octip + n) Or.
where "/ and "/ are two hyperparameters. À fixed 10 and rj set to 0.9996. The update of CLIP detector enables the integrated knowledge in the target detector flows into the CLIP detector gradually, thus achieving better knowledge integration. The training process summarized in Algorithm
A.3 More Experiments
Detailed datasets. Cityscapes 11 consists of 2,975 training images and 500 testing images captured under normal weather with total classes, Foggy-Cityscaper [I] contains three levels of foggy images simulated by the images of Cityscapes. 2,975 training images and 500 testing images with foggy level of 0.02 are utilized for training and testing. Clipart (25) includes IK clipart-style images with 20 classes. Following [49 33), all 1K images are used for both training and testing. BDD100K (63] contains 100K videos of the scenes from New York, Berkeley, San Francisco and Bay Area. For comparison with existing methods,
follow [35 [4] and use 36,728 training images and 5,258 testing images with 7 classes for training and testing respectively. KITTI [66] contains 7,481 urban images with the car category. We use all the images for training and testing. Sim10K (26] contains 10K images collected from the computer game Grand Theft Auto with the car category. All images used for training and testing.
More implementation details. One 3090 GPU, a batch-size and a random seed 2024 are used for all experiments. SGD 22 is used as the optimizer where the initial learning rate is 0.001 and the weight decay is 0.0001. For pre-training CL.PP detector, we iterate 50K steps. For knowledge distillation, generally iterate 45K steps using Eq 17 and then iterate 20K steps using Eq The
19

---

Table 10: Effectiveness of COIN under different cloud detector output types Foggy-Cityscapes. Class-only and probability are compared. det: detector.
GDINO with class-only output type
Methods
Truck
Car
Rider
Person
Train
Meycle
Beycle
Bus
mAP
Cloud det 440]
6.5
41.1
16.0
29.7
20.3
24.2
29.3
22.8
23.7
CLIP 47
9.7
28.6
11.5
19.5
1.1
12.8
17.9
21.9
15.4
CLIP det
8.2
46.9
27.5
34.1
16.5
24.9
31.5
36.2
28.2
COIN
21.9
54.7
46.1
41.3
19.4
37.9
43.0
39.5
38.0
GDINO with probability output type (default)
Methods
Truck
Car
Rider
Person
Train
Meycle
Beycle
Bus
mAP
Cloud det 44
30.8
47.5
18.6
34.3
21.0
34.6
41.1
47.4
34.4
CLIP 47
9.7
28.6
11.5
19.5
1.1
12.8
17.9
21.9
15.4
CLIP det
8.2
46.9
27.5
34.1
16.5
24.9
31.5
36.2
28.2
COIN
27.4
57.9
42.3
41.6
25.9
32.7
41.2
43.1
39.0
Table 11: Ablation study for different prompt templates of CLIP model (boxes are borrowed from cloud detector). A simple template represents "a photo of a [CLS],"; A simple template w/ style represents "a [target domain name] style photo of [CLS]": 81 templates w/o style represents 81 templates where "[target domain name] style" is not added.
mAP
Templates
Cityscapes Clipart BDD100K KITTI Sim 10K Mean
A simple template
11.5
15.0
40.7
16.0
52.4
45.9
30.3
A simple template w/ style
13.8
18.5
45.2
16.7
61.0
42.1
32.9
81 templates w/o style
13.6
17.5
43.3
17.2
58.3
47.2
32.9
81 templates
15.4
20.6
46.3
17.5
62.1
46.4
34.7
training requires 18GB to 20GB of memory. Following [33 7. the target detector is used for final testing. For Eq.(9) of the main text, we use L2 loss since we find that other losses like L1 loss or Lut loss cannot backpropagate gradients for CKG. Additionally, reduce computation, the gradients of the transformation network are calculate rather than the entire target detector in (q.
Licenses. The models employed in this paper are available under open licenses: CLIP 47 and
GL.PP (32 are released under the MIT License, and GDINO 44 is under the Apache License 2.0.
The datasets employed in this research are released under various licenses: Cityscapes [1 and
Foggy-Cityscapes [11 are available under non-commereial license; BDD100K (63] is provided
under the BSD 3-Clause License for non-commercial use; KITTI []6] is published under the CC BY-NC-SA 3.0 License; Sim10K (26) available under a custom non-commercial license; and
Clipart 25 is intended for academic use, with specific licensing details to be confirmed with the authors.
A.3.1 More quantitative results.
Quantitative results under GLIP. In order to comprehensively evaluate the effectiveness of the proposed COIN across different cloud detectors, we conduct experiments under GLIP [3]] Since GLIP offers class predictions in the form of confidence score, we convert confidence score into probability by label smoothing. The results are shown in Tables - 9 Compared to GDINO [40], GLIP produces lower performance. However, our COIN still achieves significant improvements, such as +7.1 1 % (from 20.6% to 27.7%) on Foogy-Cityscapp
+6. 1 (from 27.4 33.5%) on BDD100K, +9.5% (from 24.0% to 33.5%) on Cityscapes, +30.0% (from 26.8% to 56.8%) KITTI, +20.0% (from 17.1% to 37.1 %) on Sim10K. The above results demonstrate the broad applicability COIN
across different cloud detectors.
20

---

Table 12: Ablation study for dual prompt learning on Foggy-Cityscapes. Tempate w/ t' represents "a photo of a t² 1 t [CLS]]". Tempate w/o t' represents "a photo of a [CLS],". Prototypes update represents the exponential moving average of them. COIN w/ CLIP det prototypes represents aligning to pre-trained CLIP detector prototypes, rather than collecting them with consistent detection. det: detector.
Components
Foggy-Cityscapes
Template w/ t
CLIP det Prototypes update
Methods
Lalign prototypes
Tuk Car Rdr Psn Tan Mel Bel Bus mAP
Cloud det 40
x
X
x
30.8 47.5 18.6 34.3
21.0 34.6 41.1 47.4 34.4
CLIP
X
X
9.7 28.6 11.5 19.5
1.1 12.8
17.9
21.9 15.4
X
X
X
4.8 46.3 23.1 33.1 13.6 30.2 38.5 27.0
X
X
X
7.3 26.2 8.8 27.0 30.7 38.4 27.4
V
V
X
5.9
44.8 25.2 32.9 20.7 24.9 29.9 37.6 27.7
CLIP det
V
V
8.2 46.9 27.5 34.1 16.5 24.9
36.2 28.2
V
V
V
29.7 57.5 37.9 40.8
22.0
42.0 42.5 38.3
COIN
V
V
X
V
27.4 57.9 42.3 41.6 25.9 32.7 41.741.244 43.1 39.0
A.3.2 More quantitative analysis.
Effectiveness under different cloud detector output types. To verify the effectiveness of our COIN under different cloud detector output types, we convert the probability outputs of GDINO to class-only format (converting probability to one-hot format) and conduct experiments on Foggy- Seeyy-Cttyscapes, as shown in Table TO Since confidence score is crucial for evaluating detector's performance, observe a performance deterioration of GDINO when the output type is class-only. In addition, COIN increases the mAP by +14.3% (from 23.7% to 38.0%) that is only 1.0% decrease compared to the probability format. This proves that our COIN is compatible with various cloud detector outputs, making it generally applicable.
blation study for prompt templates of CLIP model. As shown in Table TT we investigate the effectiveness of the proposed 81 prompt templates for CL.IP model across six datasets. Experiments are categorized into four groups based on the number of templates and whether style is incorporated. The reported mAPs are calculated by the classification probabilities from CLIP model and the boxes from cloud detector. Not surprisingly, our proposed 81 templates achieve a satisfactory victory. Furthermore, simple template with style outperforms 81 templates without style four datasets. This not only demonstrates the importance of target customized prompt templates for the classification of CLIP model but also proves the necessity of adapting CLIP target domain.
Ablation study for dual prompt learning. shown Table 12 to validate the proposed prompt learning, ablation studies are conducted on four main components. (1) For CLIP detector, training based solely on simple template photo of [CLS]]" achieves the mAP of 27.0%. While utilizing four randomly initialized placcholders improves results by +0.4% (from 27.0% to 27.4 27.4%), which suggests that our designed template can assist the CLIP detector in capturing more target domain-specific attributes. (2) When Carign is introduced to align the initial class prototypes class embeddings e. from CLIP, the mAP is further enhanced by +0.3% (from 27.4% to 27.7%), and when aligning the continuously updated prototypes based on EMA and visual features, the mAP increased by +0.8% (from 27.4% to 28.2%). As anticipated, the experimental results demonstrate that prototypes updated based on visual features capture more domain-specifet attributes compared to class embeddings e, calculated by the manually customized 81 CLIP prompt templates. These findings strongly support the effectiveness of our proposed prompt learning. (3) In the knowledge distillation stage, the target detector aligns to visual class prototypes collected based on consistent detections by L2 Lalign instead of visual prototypes trained in CLIP detector. This alignment ensures that the target detector aligns with the shared knowledge between the cloud detector and CLIP detector. In the last two rows of Table [2] these two scenarios are compared, and as expected, aligning the shared knowledge results in a mAP improvement of +0.7% (from 38.3% to 39.0%) compared to aligning the knowledge of the CLIP detector. This confirms the effectiveness of our second prompt learning.
21

---

Table 13: Ablation study for knowledge separation on Foggy-Cityscapes. Filter and distill represents the use fixed threshold achieve knowledge separation, resulting in only two kinds of detections for distillation. det: detector.
Poggy-Cityscapes
Methods
Threshold
Truck
Car
Rider
Person
Train
Meycle
Beycle
Bus
mAP
Cloud det 40
30.8
47.5
18.6
34.3
21.0
34.6
41.1
47.4
34.4
CLIP 47
9.7
28.6
11.5
19.5
1.1
12.8
17.9
21.9
15.4
CLIP det
8.2
46.9
27.5
34.1
16.5
24.9
31.5
36.2
28.2
0.1
17.7
46.4
23.1
31.0
19.1
25.4
31.7
34.9
28.7
0.3
18.8
49.4
31.3
35.2
14.8
26.8
33.3
39.4
31.1
0.5
20.6
50.1
33.8
35.1
12.1
32.7
34.6
41.0
32.5
0.7
10.5
52.4
36.8
35.7
22.3
27.9
36.2
39.3
32.6
Filter and distill
0.9
11.3
51.8
37.5
33.0
10.7
27.3
29.2
36.8
29.7
COIN
27.4
57.9
42.3
41.6
25.9
32.7
41.2
43.1
39.0
Table 14: Ablation study for knowledge dissemination Poggy-Cityscappe COIN w/o dissemina- tion represents directly utilizing detections from cloud detector and CLIP (not CLIP detector) for knowledge separation and distillation stages. det: detector.
Foggy- Sereyy-itysccppp
Methods
Truck
Car
Rider
Person
Train
Mcycle
Beycle
Bus
mAP
Cloud det 40
30.8
47.5
18.6
34.3
21.0
34.6
41.1
47.4
34.4
CLIP 47
9.7
28.6
11.5
19.5
1.1
12.8
17.9
21.9
15.4
COIN w/o dissemination
15.7
54.4
45.5
40.7
22.0
36.1
39.4
37.9
36.5
CLIP det COIN
8.2
46.9
27.5
34.1
16.5
24.9
31.5
36.2
28.2
27.4
57.9
42.3
41.6
25.9
32.7
41.2
43.1
39.0
Ablation study for knowledge separation. To verify the effectiveness of the knowledge separation stage, we design a simple comparative experiment called "Filter and distill" as shown in Table [3] A fixed threshold is used to filter detections. High-confideng detections from any detector are viewed as consistent detections (box fusion is no longer used), while low-confidence detections are viewed as private detections. To avoid the unfair comparison from specific threshold, we vary five thresholds overall results are not optimistic. The best performance
between and 1. As expected, is achieved when the threshold is set to 0.7, which means our COIN surpasses it by large margin of +6.4% (from 32.6% to 39.0%). This ablation study strongly demonstrates the effecttvvnness of proposed knowledge separation stage and meanwhile validates the power of box matching for separating knowledge from different detectors.
32.6%
Ablation study for knowledge dissemination. To demonstrate the significance of the knowledge dissemination stage, we conduct an ablation study as shown Table 14 (1) When directly using CLIP instead of training CLIP detector through knowledge dissemination, mAP of 36.5% is achieved. This represents an improvement of +2. % (from 34.4% to 36.5%) over cloud detector and +21.1% % (from 15.4% to 36.5%) over CLIP. This indicates that COIN can organically integrate the knowledge from both sources even without the knowledge dissemination stage. (2) When knowledge dissemination is included to train CLIP detector, the performance is improved by +12.8% (from 15.4% to 28.2%) compared to CLIP. With the CLIP detector, COIN further increases the mAP by +2.5% (from 36.5% to 39.0%) This highlights the effectiveness of the knowledge dissemination stage our designed detector fully utilizes the knowledge from CLIP, and prompt learning mitigates domain shifts, adapting CLIP the target domain.
Analysis for knowledge dissemination of both CLIP and cloud detector. Since knowledge dissemination for CLIP mitigates domain shift and enhances performance, we consider whether applying knowledge dissemination the cloud detector brings further improvements. To explore this, analyze the method of applying knowledge dissemination to both the cloud detector and CLIP.
22

---

Table 15: Analysis for knowledge dissemination of both cloud detector and CLIP. Cloud det* represents the pre-trained detector by knowledge dissemination of cloud detector, where detections from cloud detector are used as supervision. COIN w/ dual dissemination represents COIN, but separates and distills knowledge from cloud det and CLIP det. det: detector.
Citysapps
Methods
EMA role Truck Car Rider Person Train Mcycle Beycle Bus mAP
Cloud det 44)
30.8
47.5
18.6
34.3
21.0
34.6
41.1
47.4
34.4
CLIP +7
9.7
28.6
11.5
19.5
1.1
12.8
17.9
21.9
15.4
Cloud det*
18.8
56.5
39.9
41.1
22.1
37.4
43.7
42.7
37.8
CLIP det
8.2
46.9
27.5
34.1
16.5
24.9
31.5
36.2
28.2
Both
3.2
26.4
22.5
11.8
12.1
22.6
18.9
22.7
17.5
CLIP det
10.8
54.0
42.3
35.1
18.1
33.3
37.4
27.0
32.3
COIN w/ dual dissemination Cloud det
10.1
38.9
30.8
24.1
15.8
27.6
26.5
31.0
25.6
COIN
27.4
57.9
42.3
41.6
25.9
32.7
41.2
43.1 39.0
Table 16: Detection consistence of cloud detector GDINO and CLIPP detector on BDD100K, The average results over 1000 iterations are reported. CloPd()))LIP((N) means cloud detector is right while CLIP detector is wrong. So does Cloud(N)CLIP(P),
Inconsistent
Cloud())CLI((N)
ClCLI(N)/LIP(P)
CKG(P)
99.5
67.2
32.8
80.6
The results are shown in Table (1) Unsurprisingly, knowledge dissemination for the cloud detector results in +3.4% improvement (from 34.4% 37.8%), further demonstrating the broad applicability of the knowledge dissemination stage. (2) For the knowledge separation and distillation stages, since both the cloud detector and CLIP detector can update parameters through EMA, we list three settings in Table I3 However, the results in all three settings are not ideal. This is because both cloud d and CLIP detector are trained based on the same boxes (from the cloud detector), making them prone to similar false positives. This introduces significant noise into the consistent detections, leading to many incorrect predictions by the target detector. Nevertheless, our strategy of updating CLIP detector achieves the best results because updating its parameters improves its performance. In contrast, updating the cloud d results in performance degradation. While, when both are updated, EMA causes their parameters to gradually become similar, leading the target detector to get lost in the noise. We think these issues may be mitigated if CLIP is replaced with model inherently capable of detection ability. (3) Compared the best result in dual knowledge dissemination. COIN improves performance by +6.7% (from 32.3% to 39.0%), indicating the correctness and superiority of performing knowledge dissemination exclusively for CLIP.
Experimental analysis of the mechanism for gradient alignment. To demonstrate the rationality of the gradient alignment mechanism, we use the gradients generated by the ground truths of inconsistent detections as proxies to represent the direction for inconsistent detections towards the optimal target detector. Thus, we can verify the rationality of this mechanism by calculating the cosine similarity between the above gradients and the gradients of consistent detections. this end, we compute the aforementioned similarity for each iteration, obtaining average similarity of 0.527 across 1000 iterations on BDDI00K, The corresponding vector angle for this similarity is 58.2 degrees, indicating that the gradient direction of consistent detections has relatively small angle with respect to the direction of inconsistent detections towards the optimal target detector. This demonstrates the rationality of our gradient alignment mechanism.
Detection consistence cloud detector and CLIP detector. We analyze the consistence frequency between cloud detector GDINO and CLIP detector. For each iteration, we keep track of whether inconsistent detections occur and calculate the frequency of instances where the CLIP detector makes correct detections but the cloud detector does not, denoted as (CCLPP))) frequency of CILP)( and the frequency of correct detections by CKG, denoted as CKG(P). We then convert the frequencies into the probabilities and calculate the average results over 1000
well as the
23

---

Table 17: Model size and speed of target detector (ResNet50) and cloud detector on a 3090 GPU.
Model size
Speed
Models
Proposal num
Params
Space
Time
FPS
Target detector (testing)
1000
104M
325MB
0.081s
12.3
In testing (IT); In real world deployment (L))
Target detector (deployment)
1000
40M
155MB
0.077s
13.0
Target detector (deployment)
500
40M
155MB
0.047s
21.3
Target detector (deployment)
300
40M
155MB
0.034s
29.4
Target detector (deployment)
100
40M
155MB
0.023s
43.5
Cloud detector (Swin-
40
232M
895MB
0.109s
9.2
iterations. The findings are presented in [6 We find that inconsistent detections occur in almost every iteration (99.5%). with the probability of Cloud(N)CLIP(P) being 32.8% and CKG(P) being 80.6%. The above experimental results show that CLIP can indeed benefit knowledge distillation from cloud detector. Moreover, it also proves that CKG works in our knowledge integration process, as it achieves the best results.
Detection speed. In practical applications, the well-trained target detector is utilized in edge devices with relatively low computational power. As a result, the model size and detection speed significantly impact its practicality. As shown in Table we analyze the above two terms on Foggy-Cityscppes with an input size 600 x 1200, where the ResNet50 version of target detector is compared. (1) As for params, compared to testing, target detector embeddings for classification during deployment. So the text encoder utilized during testing is discarded at deployment, reducing target detector to 40M params, which is just 1/6 of the cloud detector. (2) As for detection speed, target detector can also reduce the number of proposals to accelerate detection during deployment (with negligible impact on accuracy). Compared the 9.5 FPS of cloud detector, target detector reaches a speed of 43.5 FPS, which further underscores the significance of our proposed CODA for real-world applications.
directly employ the well-trained class
Effectiveness across different versions of cloud detector. To verify the effectiveness of our COIN across different versions of cloud detector, the Swin-T version of GDINO [40] is selected as alternative to compare with our default selected Swin-B version, as shown in Table [8] Compared to the Swin-B version, the Swin-T version of the cloud detector performs slightly weaker, achieving 26.9% and 36.4% on Sregy-Cityscapes and Cityscapes respectively. Interestingly, the performance of CLIP (using boxes from cloud detector) is not significantly affected, suggesting that the Swin-T version of the cloud detector may not classify correctly due to fewer parameters compared to the Swin-B version. Moreover, our COIN still achieves the best results both datasets 33.6% Foggy-Cityscapes and 39.7% on Cityscapes. This demonstrates the robustness and versatility of our COIN across different versions of cloud detector.
Error bar. To facilitate the reproduction of experimental results, we use a fixed random seed of 2024 in all our experiments. To analyze the error bars introduced thereby, COIN is ran under four randomly generated seeds, and the mean and standard deviation of the results from all five seeds calculated, as shown in Table T9] Since cloud detector and CLIP are not retrained, they produce the same results under different seeds. For the CLIP detector and COIN, statistical results indicate that their performance conforms to the presupposed Gaussian distribution. This is evidenced by the 1- I-sigma error bars covering 60% and 80% the data points, respectively. The above shows that our method can achieve stable results under different random seeds.
A.3.3 More Qualitative analysis.
Qualitative Comparison. To qualitatively verify our methods, we visualize the detection results on six datasets, as shown Figure where cloud detector, CLIP (using boxes from cloud detector), CLIP detector and COIN are compared. (1) It is clear that COIN produces more true positives compared to the other three, indicating that our method achieves the best results. (2) By comparing the CLIP detector and CLIP, more ground truths detected, which proves the effectiveness of the

---

Table 18: Effectiveness across different versions of cloud detector on Cityscapes. Swin-T version of GDINO compared with Swin-B version. det: detector.
and
Cloud detector
GDINO
40
of Swin-B version (default)
Methods
Truck
Car
Rider
Person
Train
Meycle
Beycle
Bus
mAP
Cloud det 40
30.8
47.5
18.6
34.3
21.0
34.6
41.1
47.4
34.4
CLIP 47
9.7
28.6
11.5
19.5
1.1
12.8
17.9
21.9
15,4
CLIP det
46.9
27.5
34.1
16.5
24.9
31.5
36.2
28.2
COIN
27.4
57.9
42.3
41.6
25,9
32.7
41.2
43.1
39.0
Cloud detector GDINO
[40]
of Swin-T version
Methods
Truck
Car
Rider
Person
Train
Mcycle
Beycle
Bus
mAP
Cloud det 40
24.9
46.0
2.6
36.5
1.4
30.9
36.7
36.5
26.9
CLIP 47
12.0
29.6
10.8
18.1
0.9
13.4
16.1
23.5
15.6
CLIP det
10.9
49.1
22.8
31.1
5.3
29.1
29.6
34.5
26.6
COIN
16.8
56.6
29.8
39.9
13.4
36.3
34.5
41.3
33.6
Foggy- Seeyy-Cityscapes (11)
Cityscapes (1))
Cloud detector GDINO
440
of Swin-B version (default)
Methods
Truck
Car
Rider
Person
Train
Mcycle
Beycle
Bus
mAP
Cloud det 40
37.5
59.9
16.4
43.4
26.1
42.7
48.4
62.6
42.1
CLIP 47
15.9
36.9
15.5
27.8
0.9
15.7
20.5
31.8
20.6
CLIP det
11.3
55.8
35.1
39.1
33.8
32.0
33.7
44.7
35.7
COIN
26.9
64.3
47.5
47.0
26.4
44.4
46.9
52.8
44.5
Cloud detector GDINO [40]
of Swin-T version
Methods
Truck
Car
Rider
Person
Train
Mcycle
Bcycle
Bus
mAP
Cloud det 44
30.6
60.2
3.1
47.6
7.2
42.3
45.6
54.9
36.4
CLIP 47
16.4
40.1
14.7
24.4
0.7
16.5
20.1
33.1
20.8
CLIP det
11.2
57.4
28.9
37.7
26.1
33.1
31.4
44.8
33.8
COIN
18.8
64.2
36.3
44.6
21.2
37.8
45.8
49.0
39.7
Table 19: Error bars Foggy-Cityscaps. Five quantitative results from one default seed 2024 and other four randomly generated seeds are displayed. det: detector.
mAP
Methods
2024 (default) 36328971
59655772
26829060 4861658
Mean Standard deviation
Cloud det 40
34.4
34.4
34.4
34.4
34.4
34.4
0.0
CLIP 47
15,4
15.4
15.4
15.4
15.4
15.4
0.0
CLIP det
28.2
28.5
28.1
28.2
28.3
28.26
0.15
COIN
39.0
39.1
38.8
38.8
38.9
38.92
0.13
knowledge dissemination stage. (3) Furthermore, by comparing cloud detector, CLIP detector, and COIN, we find that COIN achieves ideal knowledge integration while also detecting some novel boxes, demonstrating the positive impact of our knowledge integration. (4) There are large number of false positives in the KITTI raw, but when upon magnifying the images for closer inspection, we find that they are not incorrect detections that our COIN even detects real objects that are not included in the annotation files! This not only proves the power of COIN but also once again highlights the significance of the proposed problem CODA.
actual existing objects with the car category. This means
25

---

Clipart
Cityscapes
Foggy
Cityscapes
SIM
KUTT
BDD100K
GDINO
CLIP
CLIP detector
COIN
Figure 5: Qualitative results Clipart, Green red and blue boxes represent true positives (TP), false negatives (FN) and false positives (FP), respectively. Zoom in for best view.
Cityscapes, SIM, KITTI and BDD100K.
A.4 Limitations
Although knowledge dissemination stage grounds detection capability to CLIP and mitigates domain shift, pre-training CLIP detector introduces additional training time overhead. Fortunately, our COIN is general method which is not limited to CLIP. When another auxiliary model with inherent detection capability is used, the domain shift can be alleviated with a few steps of fine-tuning, thus the issue of training time overhead is eliminated.
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: As shown abstract, introduction and contribution. The abstract and introduc- tion accurately outline the main claims, which are substantiated by the results presented in the paper, ensuring that the contributions and scope are clearly communicated.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We provide limitations at the end of appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
27

---

Answer: [NA]
Justification: The paper does not present any theoretical results; hence, the criteria for providing assumptions and proofs not applicable.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed model architecture, algorithm and dataset usage in ap- pendix. Moreover, detailed implementation details are provided in main text and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
28

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The test code is provided in supplementary material. The link the full code will be provided upon the publication this paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nnis.cc
Moolic//guides/Coeeeuubiiss wtw.eeeesuumiisioooooooooooo for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ wwtps/cccppplgguuuiaaeeeae
wetpeeee/oooSSeeesssSedeee.. www.ssssiicoooooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide dataset usage such as training and testing splits detailed datasets of appendix. Moreover, detailed implementation details are provided in main text and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We report error bar at the end of appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
29

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
The computer resources, including the GPU model and memory usage, are detailed in the implementation details of appendix. Detection speed (time and FPS) and model size (number of parameters and space occupied) are analyzed one table from appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttp::://oorrppp..cco///l//eeht
Answer: [Yes]
Justification: The research adheres the NeurIPS Code of Ethics in aspects, ensuring that ethical considerations are thoroughly addressed and integrated into the study.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
30

---

Given the purely academic nature of the research, which does not entail direct application or deployment, the discussion of broader societal impacts deemed not applicable.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not involve the release of data or models that are at high risk for misuse; therefore, the discussion of safeguards not applicable.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We provide the licenses for all used models and datasets in appendix. The version models are introduced in implementation details of main text and appendix.
Guidelines:
The answer NA means that the paper does not existing assets.
31

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces new assets, including test code and detection results, which are well documented as shown in supplementary material. The link the full assets, including full code and trained models with detailed document and license will be provided upon the publication of this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
32

---

Answer: [NA]
Justification: This paper does not involve crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as institution conducting the review.
33


Reviews:
[{"review_id": "9wMNrb6TBJ", "reviewer": "Reviewer_nLTh", "summary": "This paper introduces Cloud Object Detector Adaptation, in which a cloud model is responsible for detection in the target domain. The proposed framework, COIN, includes successive stages including knowledge dissemination, separation, and distillation. The target detector leverages a cloud detector and CLIP through a box matching mechanism, categorizing detections into consistent, inconsistent, and private categories. The consistent and private detections are utilized to train the target detector, while the inconsistent detections are refined using a consistent knowledge generation network. A gradient direction alignment loss is proposed to optimize consistent knowledge generation", "questions": "Please see the weakness section. I hope to see a good rebuttal, that majorly addresses the issues regarding how the optimization loss in this work addresses the challenge of identifying optimal hyperparameters for each dataset? Also what specific contributions does this work provide that distinguish it from existing techniques in domain generalization and domain adaptation?", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "+ This paper presents an interesting integration of dissemination, separation, and distillation processes for the given task.\n+ The organization and presentation of the paper are good, showcasing a clear and structured approach. \n+ The experimental evaluation, conducted on foggy Cityscapes, BDD100k, and other additional datasets, demonstrates the method's effectiveness", "weaknesses": "+ The overall optimization loss encompasses multiple values, making the task of identifying optimal hyperparameters for each dataset hard and challenging.\n+ While this paper discusses dissemination, separation, and distillation, these concepts are not novel. \n+ Instead, the work appears to integrate existing components to enhance performance without providing a strong contribution specific to this work. The methods employed have been previously utilized in domain generalization and domain adaptation literature, e.g., works like “CLIP the Gap: A Single Domain Generalization Approach for Object Detection” (CVPR 2023) and “SSAL: Synergizing between Self-Training and Adversarial Learning for Domain Adaptive Object Detection” (NeurIPS 2021).\n+ Could the authors clarify the principle differences between their approach and prior methodologies? It is essential to highlight the specific contributions of this work to distinguish it from established techniques in a similar area.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the very encouraging comments on the originality of CODA, the effectiveness of COIN, and the overall good presentation. We hope to provide satisfying answers to the concerns raised.\n\n**Q1: The challenge of identifying optimal hyperparameters.**\n\nA: (1) This number of hyper-parameters is typical for domain adaptation/generalization methods. We compare it with recent top-performance methods (Table 1) which confirms that our method is not more complex in design.\n\n(2) In practice, we tune the hyperparameters on Foggy-Cityscapes using a standard grid-search strategy and then apply the same settings to all datasets. This tuning is a one-time process, demonstrating the training generality of our method.\n\n**Q2: The concepts of dissemination, separation, and distillation are not novel.**\n\nA: We appreciate this observation. This paper indeed introduces a novel framework that uniquely integrates dissemination, separation, and distillation specifically for the adaptation of cloud object detector. These concepts' combined application in this particular context is unprecedented and innovative. This integration results in a synergistic effect, i.e., the collective impact significantly exceeds the sum of the individual parts. Please also refer to the contribution part in main text.\n\n**Q3: Without providing a strong contribution specific to this work.**\n\nA: (1) Please note that first we introduce a new meaningful problem of Cloud Object Detector Adaptation in the large models era, with the great potentials for unleashing pre-trained large detectors across a wider range of distinct application scenarios. Regarding our method novelty, we indeed introduce a novel framework that uniquely integrates knowledge dissemination, separation, and distillation, whose combined application in this particular context is unprecedented and innovative. This integration results in a synergistic effect, where the collective impact significantly exceeds the sum of the individual parts. This thanks to our decision-level fusion strategy and a fusion based gradient alignment algorithm newly introduced. Such a strategy is drastically different from previous fusion methods that usually simply discard inconsistent predictions and used self-supervised training of target domain detector based on consistent detections [8].\n\n(2) Thanks for suggesting both works and we will incorporate them. Comparing with CLIP-GAP: \n(i) Different problem settings. Our core innovation lies in the use of self-promotion gradient direction alignment to address knowledge conflicts between cloud detector and CLIP detector. CLIP-GAP [9] deals with a different problem setting nor considers this challenge at all.\n(ii) Different motivations and purposes. Whilst both methods align two feature spaces, they are purposefully and directionally different. CLIP-GAP aligns the features to CLIP semantic space in order to make the detector ignore target domain style. Aligning the CLIP model to the target domain in an opposite way, we instead aim to capture more target domain-specific attributes. Further, the capabilities of both cloud detector and CLIP would be then fused to improve the performance of target domain detector. \n\n(3) Comparing with SSAL [10]: \n(a) Different purposes. While similarly performing sample selection, SSAL does not consider and solve the detection conflicts issue, which is a core challenge in our problem.\n(b) Different technical strategies and problem settings: After sample selection, SSAL does not perform further fusion operations, resulting in multiple detection boxes in the same region. Therefore, they train the detector only using the classification loss. In contrast, we consider a proper detection problem by further fusing consistent detections to form ground truths and training both the classification and regression branches. Note this is crucial for CODA, as it does not have source domain labels as assumed in UDAOD, which SSAL focuses on.\n\n**Q4: Clarify the principle differences**\n\nA: Except the above responses, we further summarize the novelty and innovations in a whole picture: \n(1) As far as we know, this is the first attempt on adapting a cloud objector detector. \n(2) Further, we propose to leverage the pretrained language-visual model for tackling this new challenge. This echos/reflects the current trending in AI of leveraging large foundation models for dealing with a diverse of downstream tasks. In this context, this idea is still not straightforward to implement but challenging. To address that, we introduce a novel framework to integrate different source knowledge by innovatively integrating the concepts of dissemination, separation, and distillation.\n(3) Unlike previous methods only utilizing consistent detection results, our method can uniquely integrate inconsistent detections by aligning gradients with consistent detection results, achieving full utilization of knowledge from different sources.\n\n&nbsp;\n\nTable 1. Comparison of the number of hyperparameters in the overall loss function.\n|Methods|Loss terms|Adjustable number (W/O fixed value hyperparameters) | Actual number |\n|:-:|:-:|:-:|:-:|\n|SIGMA++ [1]|5|2|4|\n|TFD [2]|4|3|3|\n|CIGAR [3]|5|2|4|\n|LODS [4]|3|2|2|\n|LUP [5]|3|2|2|\n|IRG [6]|3|0|2|\n|BT [7]|4|1|3|\n|**Ours**|**4**|**2**|**3**|\n\n\n**References**\n\n[1] W. Li, et al, \"SIGMA++ ...\". TPAMI2023.\n\n[2] H. Wang, et al, \"Triple ...\". AAAI2024.\n\n[3] Y. Liu, et al, \"CIGAR...\". CVPR2023.\n\n[4] S. Li, et al, \"Source-free...\". CVPR2022.\n\n[5] Z. Chen, et al, \"Exploiting ...\". ACM MM2023.\n\n[6] VS Vibashan, et al, \"Instance ...\". CVPR2023.\n\n[7] J. Deng, et al, \"Balanced ...\". TCSVT2024.\n\n[8] S. Zhao, et al, \"Multi-...\". IJCV2024.\n\n[9] V. Vidit, et al, \"CLIP ...\". CVPR2023.\n\n[10] MA Muhammad, et al, \"SSAL...\". NeurIPS2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "969pnAdwyy", "reviewer": "Reviewer_s7Cc", "summary": "This paper proposes a novel task: Cloud Object Detector Adaptation (CODA). It discusses how to build a domain specific object detector with the help of a cloud detector, and local data from the target domain. Different from existing similar tasks, CODA does not have the full access of the cloud model, only detection outputs are used. Furthermore, it aims to transfer the model to a target domain with large domain gap.\n\nThe paper then presents a novel Cloud Object detector adaptation method by Integrating different source kNowledge (COIN). The key idea is to incorporate a public vision-language model (CLIP) to refine the knowledge for adaptation. Experiment results show that the proposed method achieve SOTA performance.", "questions": "The most critical part is the choice of cloud detector (Weaknesses 1). As a novel benchmark, it is better to study the problem with more settings.", "limitations": "Limitations are discussed in Appendix.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "1. This paper proposes a novel task: Cloud Object Detector Adaptation (CODA). It is different from existing tasks, by considering the model privacy issue.\n\n2. The pipeline consists of knowledge dissemination, separation and distillation stages. By including CLIP, the design seems reasonable, and novel. The final results show that the method is effective.", "weaknesses": "1. The cloud detector is discussed in general through the whole paper. And because the model is designed purely on the detection output of the cloud detector. It should work on different detectors. So the whole architecture should be validated on different cloud detectors. And the experiment section in the main paper is too short. It should focus more on different settings about this new benchmark.\n\n2. The methodology part is a bit confusing, specifically for the two detectors. CLIP and Target detectors are both introduced in Sec 3.1 without specific discussion of how they are used or different from each other. And the training of target detector is not further discussed until Sec 3.3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the very encouraging comments like the novelty of CODA and COIN as well as the effectiveness of COIN. We hope to provide satisfying answers to the concerns raised.\n\n**Q1: The whole architecture should be validated on different cloud detectors. It should focus more on different settings about this new benchmark.**\n\nA: Thanks for your suggestions. In Table 15 of the Appendix, we already presented the results with different cloud detectors $-$ GDINO [1] with Swin-B and Swin-T backbones. \nAs suggested, we further evaluated GLIP [2] as the cloud model (GLIP-L is used). The results as shown in Tables 1-4 indicate the good generality of our method across varying cloud detectors. We will add this test.\n\n\n\n**Q2: The methodology part is a bit confusing, specifically for the two detectors. CLIP and Target detectors are both introduced in Sec 3.1 without specific discussion of how they are used or different from each other. And the training of target detector is not further discussed until Sec 3.3.**\n\nA: Apologies for any confusion. We will further refine the presentation as simply summarized below:\nIn our main implementation (the could detector is replaceable), the CLIP and target detectors share the same architecture; CLIP detector is pretrained using Eq.(4) while target detector is randomly initialized (as stated in Lines 160-162 in main text). Both detectors are not updated until Section 3.3 (see Lines 271-275). \nSpecifically, the target detector is trained with Eq.(12), enabling the knowledge from the CLIP and cloud detectors to flow to the target detector, while CLIP detector is updated based on EMA (Line 274).\n\n&nbsp;\n\nTable 1. Results on **Foggy-Cityscapes**. GLIP-L is used as cloud detector. det: detector.\n|Methods|Truck|Car|Rider|Person|Train|Mcycle|Bcycle|Bus|mAP|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Cloud det|**23.9**|23.9|14.3|13.9|6.1|21.0|22.1|**39.8**|20.6|\n|CLIP|13.1|19.3|10.9|11.6|4.3|15.2|12.3|27.9|14.3|\n|CLIP det|10.0|33.7|28.2|26.0|**14.1**|25.0|24.9|38.1|25.0|\n|**COIN-GLIP**|10.7|**35.7**|**38.1**|**28.9**|10.3|**28.5**|**30.4**|39.3|**27.7**|\n&nbsp;\n\nTable 2. Results on **BDD100K**. GLIP-L is used as cloud detector. det: detector.\n|Methods|Tuck|Car|Rder|Pson|Mcle|Bcle|Bus|mAP|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Cloud det|33.1|24.3|13.5|21.0|**30.0**|29.8|**40.1**|27.4|\n|CLIP|25.4|19.9|4.9|5.4|20.1|11.4|28.9|16.6|\n|CLIP det|38.5|39.2|16.7|27.1|26.3|20.7|34.9|29.1|\n|**COIN-GLIP**|**39.3**|**41.3**|**22.9**|**36.4**|26.8|**29.9**|37.9|**33.5**|\n&nbsp;\n\nTable 3. Results on **Cityscapes**. GLIP-L is used as cloud detector. det: detector.\n|Methods|Truck|Car|Rider|Person|Train|Mcycle|Bcycle|Bus|mAP|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Cloud det|**31.5**|24.0|8.8|13.2|8.2|27.2|23.0|55.7|24.0|\n|CLIP|18.3|20.6|14.5|13.1|1.4|17.4|12.7|36.9|16.9|\n|CLIP det|13.8|37.6|**36.9**|29.5|**29.6**|29.6|27.2|43.2|30.9|\n|**COIN-GLIP**|23.3|**40.3**|29.4|**33.0**|17.0|**35.0**|**33.1**|**56.6**|**33.5**|\n&nbsp;\n\nTable 4. Results on **KITTI** and **Sim10K**. GLIP-L is used as cloud detector. det: detector.\n|Methods|KITTI|Sim10K|\n|:-:|:-:|:-:|\n|Cloud det|26.6|17.1|\n|CLIP|26.8|16.6|\n|CLIP det|55.9|35.8|\n|**COIN-GLIP**|**56.8**|**37.1**|\n\n&nbsp;\n\n**References**\n\n[1] S. Liu, et al, \"Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection\". arXiv2024.\n\n[2] Liunian Harold Li, et al, \"Grounded Language-Image Pre-training\". CVPR2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UYoOSpZDnV", "reviewer": "Reviewer_MA6n", "summary": "In this paper, the authors propose a Cloud Object Detector Adaptation framework, which leverages a strong detector in the cloud to extract discrimative knowledge of objects in the target domain. It is basically a mean teacher style for domain adaptive object detection.", "questions": "See the weakness section.", "limitations": "There is no potential negative societal impact of this work.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1 This paper considers an important object detection problem in the target domain.\n2 The experiments are extensive.", "weaknesses": "The design of this work basically follows the mean teacher style for self-distillation with EMA. Basically, the contributions are not convincing. \n\n1) Using CLIP model to build detector. It is a straightforward extension in the Faster RCNN style as shown in Fig 2 (a). \n\n2) Using a strong detector in the cloud. This allows to generate more discriminative box supervisions of objects in the target domain.  It is also straightforward to obtain. \n\n3) The only interesting part is knowledge seperation. The idea is also simple to discover consistent, private and inconsist boxes via matching. Distillation is operated on different types of these boxes.\n\nHence, the overall framework brings little new insightful design by the incremental addition of object detector in the cloud.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the very encouraging comments that confirm the importance of CODA and the extensive nature of our experiments.\n\n**Q1: The design of this work basically follows the mean teacher style for self-distillation with EMA. Basically, the contributions are not convincing. Hence, the overall framework brings little new insightful design by the incremental addition of object detector in the cloud. Using CLIP model to build detector is a straightforward idea. Using a strong detector in the cloud to generate discriminative boxes is straightforward.**\n\nA: We would like to summarize the significance and novelty of this work again (please also see our responses to reviewer nLTh):\n(1) As far as we know, this is the first attempt on adapting a cloud objector detector. \n(2) Further, we propose to leverage the pretrained language-visual model for tackling this new challenge. This echos/reflects the current trending in AI of leveraging large foundation models for dealing with a diverse of downstream tasks.\nIn this context, this idea is still not straightforward to implement but challenging.\nTo address that, we introduce a novel framework to integrate different source knowledge by innovatively integrating the concepts of dissemination, separation, and distillation.\n(3) Unlike previous methods only utilizing consistent detection results, our method can uniquely integrate inconsistent detections by aligning gradients with consistent detection results, achieving full utilization of knowledge from different sources. \nIn short, the overall technical innovation includes the introduction of CLIP, designing Consistent Knowledge Generation network (CKG) and its loss function.\nWe argue these form sufficient significance.\n\nTechnically, previous Mean-Teacher based source-free domain adaptive methods take as input differently augmented data into the teacher and student models, and then align them by such as consistency or contrastive learning to learn domain-invariant space for adaptation [1, 2]. In our problem, two teachers are involved with conflicting knowledge, and the model parameters of the cloud detector are inaccessible, rendering traditional alignment methods ineffective. \nTo address this, we specifically propose the self-promotion gradient direction alignment and the Consistent Knowledge Generation network to calibrate these conflicts, distinguishing our approach from previous Mean-Teacher methods.\n\n\n**Q2: The only interesting part is knowledge separation. The idea is also simple to discover consistent, private and inconsist boxes via matching. Distillation is operated on different types of these boxes.**\n\nA: Thanks and sorry for confusion. \nCompared to knowledge separation, the more important point with our model design is on how to exploit consistent knowledge to facilitate the fusion of inconsistent knowledge. This is made possible by developing a gradient direction alignment method to learn Consistent Knowledge Generation network (CKG). Please also see Q1. \n\n\n**References**\n\n[1] S. Li, et al, \"Source-Free Object Detection by Learning to Overlook Domain Style\". CVPR2022.\n\n[2] VS Vibashan, et al, \"Instance Relation Graph Guided Source-Free Domain Adaptive Object Detection\". CVPR2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lRANFuN6dY", "reviewer": "Reviewer_FRKx", "summary": "This paper proposes a new problem in the field of domain adaptation, called Cloud Object Detector Adaptation (CODA), where a cloud model is provided to help with target detector training. A novel method termed COIN is proposed to leverage CLIP for knowledge distillation in a divide-and-conquer manner. Sufficient experiments have proven the effectiveness of the proposed method.", "questions": "1. Can the cloud model be replaced by a large Visual Language Model (VLM), such as GPT-4V?\n1. The performance of state-of-the-art (SOTA) supervised methods on the validation dataset should be listed as the oracle performance.", "limitations": "This paper provides limitations at the end of appendix.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Good Presentation**. This paper is well-organized, with clear and effective writing, complemented by appealing figures and tables.\n- **Sufficient Experiments**. The paper conducts experiments on six validation datasets, demonstrating that the proposed method achieves state-of-the-art performance. Ablation studies further illustrate the effectiveness of the proposed components.", "weaknesses": "- **Method Generalization Ability**. The observation that both the CLIP detector and the target detector are based on Faster R-CNN, which utilizes the visual encoder of CLIP, is a valid point. While this design leverages the open-set capability of CLIP, it raises a fair concern that it may limit the method's applicability to DETR-based detection approaches.\n- **Detailed Problem Settings** The paper should provide more detailed problem settings for CODA, such as the differences in scenes and category gaps between the pretrained data used by the cloud detector and the validation set.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the very encouraging comments like the originality of CODA, the novelty of COIN, good presentation, and sufficient experiments. We hope to provide satisfying answers to the concerns raised.\n\n**Q1: Method Generalization Ability.**\n\nA: Great suggestions. COIN can be generalized to transformer based detector like DETR. As the rebuttal time is too limited, we will briefly describe here how to apply COIN to DETR and conduct the experiments for the revised paper:\n\n(1) For knowledge dissemination stage, CLIP can also be utilized to build CLIP detector and target detector upon DETR by replacing the feed forward network (FFN) with the transformation network (without mean pooling), the class head $l_c$ , the box head $l_b$ , and CLIP’s text encoder (see Lines 152-154 in main paper), as DETR requires a FFN to perform classification and box regression on the output embeddings from its decoder. \nMoreover, the CLIP detector (DETR version) can still use the same Eq.(4) for training, thereby adapting CLIP to the target domain. \nSo, using DETR during the knowledge dissemination stage does not cause significant changes. \n\n\n(2) As for knowledge separation stage, it is detector architecture general as box matching takes as input the detectors' predictions/outputs.\n\n(3) During knowledge distillation stage, the target detector (DETR version) outputs the predictions, which can be matched to consistent, inconsistent and private detections using the Hungarian algorithm as in DETR, with corresponding losses used for training. The Consistent Knowledge Generation network (CKG) can still be trained based on features from the class head $l_c$ and gradient direction alignment. \n\nThus, COIN is a general method not specific to the detector architecture. We will further clarify it in the final version.\n\n\n\n**Q2: Detailed Problem Settings.**\n\nA: Thanks for suggestions. \n\n(1) Cloud Object Detector Adaptation (CODA) defines a scenario where a large cloud detector, trained on extensive pre-training data, is deployed on the cloud to provide API services. Simultaneously, we have an unlabeled target domain locally awaiting training through the API. The gap between the target domain data and the cloud detector's pre-training data shall be within an acceptable range, and there should be an overlap between the categories of the target domain and those of the cloud detector's pre-training categories.\n\n(2) For the scene gap, it should not be too large to avoid the cloud detector producing completely incorrect results. As the cloud detector is a generally good model, this condition should often stand.\nFor the category gap, there should be an overlap between the target domain categories and the cloud detector's pre-training categories, allowing the cloud detector to detect some or all of the target domain categories.\n\n(3) Since cloud detectors are generally pre-trained on large-scale detection and image caption datasets such as COCO, Objects365, OpenImages, GoldG, Cap4M, and RefCOCO et.al. The above two conditions of scene gap and category gap can usually be met for most target domain scenarios.\n\nWe will clarify it in the final version.\n\n**Q3: Possibility of GPT-4V as the cloud model.**\n\nA: Great suggestion! Upon this comment, we have tested GPT-4V on a random image from Foggy-Cityscapes dataset (see Fig.1 in REBUTTAL PDF) to detect two simple categories: car and person.\nHowever, the detections obtained are entirely incorrect.\nWe find that GPT-4V is clearly inferior in object detection to GDINO [1], making it unqualified as a reasonable cloud detector. \nOnce it gets improved, we will further try with our pipeline.\n\n**Q4: Oracle performance on each dataset.**\n\nA: Thanks for suggestions. We train our target detector with standard Faster R-CNN losses and standard supervised data, and obtain the oracle performance on each dataset.\nOverall, the mAPs on the six benchmark datasets are: Foggy-Cityscapes (46.5\\%), BDD100K (49.8\\%), Cityscapes (49.9\\%), KITTI (95.8\\%), Sim10K (79.2\\%), and Clipart(99.4\\%). We will include these results in the final version.\n\n**References**\n\n[1] S. Liu, et al, \"Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection\". arXiv2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "awsyoBo0kX", "reviewer": "Reviewer_Ht4V", "summary": "This paper introduces CODA, a new problem, where the goal is to adapt object detectors for specific target domains using knowledge from cloud-based detectors. The paper uses CLIP to refine the knowledge from the cloud detector. A gradient alignment method is proposed to deal with the inconsistency between the detection outputs of the adapted CLIP detector and the cloud detector. Extensive experiments show that the proposed CODA approach improves detection performance in the target domain with high computational efficiency for edge devices.", "questions": "1. About the definition of the new problem “Cloud Object Detector Adaptation”. As my understanding, a cloud model like GPT-4 only releases its API to output its prediction, but confidence score or probability distribution of its prediction is probably inaccessible, while the proposed method needs a probability distribution. Black-blox DDOD does not need it. Do you think this will hinder practical application of the proposed method?\n2. About the gradient alignment method for decision-level fusion of inconsistent detections. Experiments in table 4 and table 5 show that CKG with gradient alignment method works. But it still lacks a deep experimental/visualization analysis or a theoretical support of this mechanism. Can you get a conclusion like “The gradient direction of consistent detections is the direction for inconsistent detections towards optimal target detector.” or “The gradient direction of consistent detections leads to the right detection prediction for inconsistent detections with high probability.”?\n3. About the performance degradation problem of cloud detectors for some categories in experiments. For example, in table 1, for 4 categories of BDD100K dataset, there is still a gap between COIN and the cloud detector. An adapted CLIP detector is utilized to refine the cloud detector, as my understanding, is this a sign that some wrong predictions from CLIP hinders the knowledge transferring from the cloud detector for some categories? I’m also curious about the analysis of why CLIP can benefit the knowledge distillation from the cloud detector. How often the inconsistence happens? How often the adapted CLIP detector can make the right detection but cloud detector cannot?", "limitations": "1. Clarity of the new problem definition.\n2. Lack of analysis of part of mechanism design and the idea.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Good writing and readability. \n2. Novelity of the method: Use CLIP to help refine the knowledge distillation from a cloud detector for a new problem CODA. \n3. Extensive experiments show the effectiveness of the method. \n4. Good computational efficiency for edge devices.", "weaknesses": "1. The definition of the new problem “Cloud Object Detector Adaptation” should be stated more precisely. \n2. The principle of the gradient alignment mechanism should be explained more clearly.\n3. Performance degradation problem of cloud detector for some categories in experiments should be elaborated more clearly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the very encouraging comments like the originality of CODA, the novelty of COIN, good writing, extensive experiments and good computational efficiency for edge devices. We hope to provide satisfying answers to the concerns raised.\n\n**Q1: The definition of the new problem “Cloud Object Detector Adaptation”, and the practical application of the proposed method.**\n\nA: (1) Our CODA can generalize to different output formats of the cloud detectors (such as class-only, confidence score, or probability). For instance, GDINO [1] uses probability output type, GLIP [2] gives confidence score output, and GPT-4V output class label alone (please refer to Fig.1 in REBUTTAL PDF). \nFor detection task, a good cloud detector should at least output confidence scores which are crucial for evaluating detector's performance; Using class-only outputs will negatively impact the cloud detector's performance on the target domain validation set.\n\n(2) Our COIN is compatible with various cloud detector outputs, making it generally applicable. For example, we convert the probability outputs of GDINO to class-only format and conduct experiments on Foggy-Cityscapes (see Table 1), resulting in a 14.3\\% improvement over GDINO with class-only output type $-$ that is only a 1.0\\% decrease compared to the probability format. For confidence score output type, we test the cloud detector GLIP [2]. As shown in Tables 1-4 for our responses to reviewer s7Cc, significant improvements can also be achieved. \n\n\nWe will carefully incorporate the above definition and experimental results into the final version. \n\n\n**Q2: A deep experimental/visualization analysis or a theoretical support of the mechanism for gradient alignment.**\n\nA: To demonstrate the rationality of the gradient alignment mechanism, we use the gradients generated by the ground truths of inconsistent detections as proxies to represent the direction for inconsistent detections towards the optimal target detector. Thus, we can verify the rationality of this mechanism by calculating the cosine similarity between the above gradients and the gradients of consistent detections. To this end, we compute the aforementioned similarity for each iteration, obtaining an average similarity of 0.527 across 1000 iterations on BDD100K. The corresponding vector angle for this similarity is 58.2 degrees, indicating that the gradient direction of consistent detections has a relatively small angle with respect to the direction of inconsistent detections towards the optimal target detector. This demonstrates the rationality of our gradient alignment mechanism.\nWe will clarify it in the final version.\n\n\n**Q3: About the performance degradation problem of cloud detectors for some categories in experiments. The analysis of why CLIP can benefit the knowledge distillation from the cloud detector. How often the inconsistence happens? How often the adapted CLIP detector can make the right detection but cloud detector cannot?**\n\n\nA: (1) CLIP detector is poor in detecting specific categories in the original form. \nThat results in inconsistent detections, likely weakening the target domain detector when the detection results are fused. \n\n(2) As for why CLIP can benefit the knowledge distillation from the cloud detector, \nthat is because CLIP and the cloud detector are complementary as they were trained differently in mange ways. This has been also validated in this work clearly (see Table 2 in main text).\n\n(3) We have now analyzed the consistence frequency between cloud detector and CLIP detector. For each iteration, we keep track of whether inconsistent detections occur and calculate the frequency of instances where the CLIP detector makes correct detections but the cloud detector does not, denoted as Cloud(N)/CLIP(P), as well as the frequency of Cloud(P)/CLIP(N) and the frequency of correct detections by CKG, denoted as CKG(P). We then convert the frequencies into the probabilities and calculate the average results over 1000 iterations. The findings are presented in Table 2. We find that inconsistent detections occur in almost every iteration (99.5\\%), with the probability of Cloud(N)/CLIP(P) being 32.8\\% and CKG(P) being 80.6\\%. The above experimental results show that CLIP can indeed benefit knowledge distillation from cloud detector. Moreover, it also proves that CKG works in our knowledge integration process, as it achieves the best results.\n\nWe will clarify them in the final version.\n\n\n&nbsp;\n\n\nTable 1. Verification of the applicability of the COIN to different cloud detector output types on **Foggy-Cityscapes**. det: detector.\n|Methods|Truck|Car|Rider|Person|Train|Mcycle|Bcycle|Bus|mAP|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Cloud det(class-only)|6.5|41.1|16.0|29.7|20.3|24.2|29.3|22.8|23.7|\n|Cloud det(probability)|**30.8**|47.5|18.6|34.3|21.0|34.6|41.1|**47.4**|34.4|\n|CLIP|9.7|28.6|11.5|19.5|1.1|12.8|17.9|21.9|15.4|\n|CLIP det|8.2|46.9|27.5|34.1|16.5|24.9|31.5|36.2|28.2|\n|**COIN(class-only)**|21.9|54.7|**46.1**|41.3|19.4|**37.9**|**43.0**|39.5|38.0|\n|**COIN(probability)**|27.4|**57.9**|42.3|**41.6**|**25.9**|32.7|41.2|43.1|**39.0**|\n\n&nbsp;\n\nTable 2. Detection consistence of cloud detector and CLIP detector on **BDD100K**. The average results are reported over 1000 iterations. Cloud(P)/CLIP(N) means cloud detector is right while CLIP detector is wrong. So does Cloud(N)/CLIP(P).\n|Inconsistent|Cloud(P)/CLIP(N)|Cloud(N)/CLIP(P)|CKG(P)|\n|:-:|:-:|:-:|:-:|\n|99.5|67.2|32.8|80.6|\n\n&nbsp;\n\n**References**\n\n[1] S. Liu, et al, \"Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection\". arXiv2024.\n\n[2] Liunian Harold Li, et al, \"Grounded Language-Image Pre-training\". CVPR2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate all the reviewers for the constructive and positive comments e.g., the originality of CODA (reviewers nLTh, s7Cc, MA6n, FRKx, and Ht4V), the novelty of COIN (reviewers s7Cc, FRKx, and Ht4V), good organization and presentation (reviewers nLTh, FRKx, and Ht4V), experimental effectiveness or extensiveness (reviewers nLTh, s7Cc, MA6n, FRKx, and Ht4V) and computational efficiency for edge devices (reviewer Ht4V).

Meta-review: Dear authors, 

Based on overall positive rating received by the authors, this draft is being accepted for the publication. 

regards
AC
Decision: Accept (poster)