DON’T NEED RETRAINING: A Mixture of DETR and Vision Foundation Models for Cross-Domain Few-Shot Object Detection
Authors: Changhan Liu
Xunzhi Xiang
Zixuan Duan
Wenbin Li
Qi Fan
Yang Gao
Venue: NeurIPS 2025 poster ()
Keywords: cross-domain few-shot object detection
foundation model
Primary area: applications
TL;DR: We propose a novel MOE-based architecture that combines the detector and the visual foundation model for cross-domain few-shot object detection.

Abstract: Cross-Domain Few-Shot Object Detection (CD-FSOD) aims to generalize to unseen domains by leveraging a few annotated samples of the target domain, requiring models to exhibit both strong generalization and  localization capabilities. However, existing well-trained detectors typically have strong localization capabilities but lack generalization, whereas vision foundation models (VFMs) generally exhibit better generalization but lack accurate localization capabilities. In this paper, we propose a novel Mixture-of-Experts (MoE) structure that integrates the detector's localization capability and the VFM's generalization by using VFM features to improve detector features. Specifically,  we propose Expert-wise Router (ER) that selects the most relevant VFM experts for each backbone layer, and Region-wise Router (RR) that emphasizes foreground and suppress background. To bridge representation gaps, we further propose Shared Expert Projection (SEP) module and Private Expert Projection (PEP) module, which align VFM features to the detector feature space while decoupling shared image feature from private image feature in the VFM feature map. Finally, we propose MoE module to transfer the VFM’s generalization to the detector without altering the detector original architecture. Furthermore, our method extend well-trained detectors for detecting novel classes in unseen domains without re-training on the base classes. Experimental results on multiple cross-domain datasets validate the effectiveness of our method.

DON'T NEED RETRAINING:
A Mixture of DETR and Vision Foundation Models for Cross-Domain Few-Shot Object Detection
Changhan Liu, Xunzhi Xiang, Zixuan Duan, Wenbin Li, Qi Fan 23 Yang Gao
School of Intelligence Science and Technology, Nanjing University, China https//ggttubbcoomm////l////ffDf
Abstract
Cross- Moos--Dmainn Few-Shot Object Detection (CD-FSOD) aims generalize to seen domains by leveraging a few annotated samples of the target domain, requiring models to exhibit both strong generalization and localization capabilities. However, existing well-trained detectors typically have strong localization capabilities but suffer from limited generalization, whereas vision foundation models (VFMs) generally exhibit better generalization but lack accurate localization capabilities. this paper, we propose novel Mixture-of-Exppe the detector's localization capability and the VFM's generalization by using VFM features to improve detector features. Specifically, (ER) that dynamically selects the most relevant VFM experts for each backbone layer, and Region-wise Router (RR) that emphasizes foreground and suppress background. To bridge representation gaps, we further propose Shared Expert Projection (SEP) module and Private Expert Projection (PEP) module, which align VFM features to the detector feature space while decoupling shared image feature from private image feature in the VFM feature map. Finally, we construct MoE module to transfer the VFM's generalization to the detector without modifying the original detector architecture. Furthermore, our method extend well-trained detectors for detecting novel classes in unseen domains without re-training the base classes. Experimental results on multiple cross-domain datasets validate the effectiveness of our method.
(MoE) structure that integrates
propose Expert-wise Router
Introduction
Few-Shot Object Detection (FSOD) aims to detect objects of novel classes with a few labeled support data. Existing FSOD methods [1, 2, 3, 4, 5, 6, 7, have achieved notable progress in generalizing to various in-domain novel categories. However, these methods often struggle with domain shifts, where training and testing data originate from different domains. This challenge underscores the significance of Cross-Domain Few-Shot Object Detection (CD-FSOD). CD-FSOD aims generalize object detection models to detect novel classes unseen domains by using few training samples. This challenging task typically requires model to combine strong generalization and accurate localization capability.
Existing well-trained object detection methods [9, 10, 11, 12, 13, 14, 15] usually excel at localizing and recognizing seen objects but struggle to generalize to unseen categories or knowledge. To address this problem, previous CD-FSOD methods [16, 17, 18] mainly adopt two-stage training paradigm that first involves base training on a large-scale dataset of base classes, follow by novel fine-tuning
indicates corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Other Specific-designed Detection Models
Base
Novel fineturing
Methods
training
36.
Our
Common-used Detection Model
Method
Novel finetuning
DAB-DET
AigrOSTA
DETA
3-8344
10-4740
Figure Existing CD-FSOD methods generally require time-consuming base training to adapt their specific-designed detection models. In contrast, our method directly extends the well-trained common-used detection models, e.g., DETR [25]. to CD-FSOD without retraining the model on base data. Our method significantly improves the cross-domain performance of well-trained object detection methods [21, 20, 19, 22]. Our method also outperforms the state-of-the-an method CD- ViTO [18].
on a small dataset of novel classes. However, these models are typically redesigned for CD-FSOD and retrained on base classes which brings substantial computational and time costs, as shown in Figure 1. Moreover, they fail to fully utilize the well-trained powerful object detection models, e.g. DETR-based methods [19, 20, 21, 22]. Thus, we propose novel CD-FSOD paradigm by extending well-trained in-domain object detection methods to detect unseen classes in new domains without retraining on the base classes. Our key idea is to leverage the powerful vision foundation models (VFMs) [23, 24] to equip well-trained DETR models with powerful domain generalization ability.
Vision foundation models [26, 27] have demonstrated excellent generalization to new domains, powered by sophisticated architectures and large-scale pretraining datasets. However, VFMs typically lack accurate localization capabilities due to the lack of bounding box annotations in their pretraining datasets. Consequently, directly applying VFMs to object detection generally results in suboptimal performance. To address this problem, existing VFM-based CD-FSOD methods [18, 28] usually modify the detector structure to integrate VFMs and retrain on the base data to achieve precise localization capabilities, which incurs substantial computational and time costs.
It is essential to fully utilize the VFM's generalization capabilities and the well-trained object detection model's localization capabilities while reducing computational and time costs. Thus, instead of altering the structure of existing detection models, we introduce a flexible framework based on a Mixture-of Mixterr-of-Expprts (MoE) [29] architecture to integrate detector with VFM. Our method enables well-traines detectors to achieve both strong localization and generalization capabilities, without retraining on base class. Specifically, we adopt Detection Transformer (DETR) as our detection framework due to its superior localization capabilities and leverage VFM features as experts to improve the detector's ' representations by aggregating both detector and VFM features.
Overall, our method focuses two parts: VFM expert feature selection and VFM-detector feature fusion. Since detector features at different layers contain varying levels of semantic information, each backbone layer generally requires distinct VFM features for effective guidance. Accordingly, we propose Expert-wise Router (ER) to select appropriate VFM features based on image features from different backbone layers. Mcanwhile, irrelevant background in the VFM feature map, we propose the Region-wise Router (RR) module which generates region-wiss gating weights to reweight different spatial regions in the VFM expert features. The ER and RR modules respectively generate expert- wwie and region-wise gating weights, dynamically prioritizing the most relevant VFM expert features for detector features at different layers and effectively suppressing irrelevant background in the VFM expert features.
highlight the foreground regions and suppress
After selecting expert features, it is essential ensure effective feature fusion between VFM expert features and detector features. Due to the differences in both channel and spatial dimensions between VFM expert features and detector features, is necessary to project VFM features into the detector feature space. However, assigning a dedicated projection layer to each expert feature will introduce a large number of additional parameters, which increases the computational cost. In contrast, using single shared projection layer fails to capture the diverse projection requirements of different expert features. To address this issue, we propose the Shared Expert Projection (SEP) and Private Expert Projection (PEP) modules to project the VFM features to the detector feature space with minimal parameters. Each expert feature contains shared image features, such as object shape. Additionally, since each expert feature may focus on different regions of the object, it also includes private image features. Based on this characteristic, we decouple the shared and private features of each expert feature and project them using separate modules. Specifically,
the SEP module to project

---

shared features across all expert features, while using the PEP module to project the private features that are unique each expert feature. This design minimizes the number of parameters required for feature projection and preserves detailed features from different object regions.
Compared with existing methods that adopt the foundation model learnable backbone [30, 31, 32] or frozen backbone [33, 34], our method offers the following advantages:
Maintaining strong localization capability. Our method uses VFM features as experts to enhance detector backbone features, rather than using the VFM as the detector backbone directly. This design avoids transferring the VFM's limited localization capability to the detector.
No retraining requirement base classes, We introduce only few trainable parameters to fuse VFM and detector features. Therefore, the model can be directly finetuned on downstream tasks without retraining base classes.
High extensibility. By treating the VFM as an expert model independent of the detector, our method avoids modifying the detector structure and can be easily transferred other well-trained object detection models.
2
Related Works
DETR and Its Variants. Detection Transformer (25T initially proposed an end-to-end object detector based on the Vision Transformer architecture. Many subsequent studies [35, 36] have improved DETR from various perspectives, such as accelerating convergence speed [37. 20, 39], improving the matching strategy [38, 40], and applying advanced query learning methods [41, 21, 42, 43]. Although these methods demonstrate strong performance, DETR and its variants typically struggle on CD-FSOD tasks because they are trained on base classes and thus fail to extract robust representations for target domain images, impairing DETR's 's generalization capabilities. In this work, we propose a MoE framework to integrate DETR with vision foundation model. Specifically, we leverage vision foundation model's features experts to improve the detector's features, enabling the detector to inherit the strong generalization of vision foundation model.
Few-Shot Object Detection. Pooss-Domain Few-Shot Object Detection (CD-FSOD) aims to enable object detection models to detect novel classes in unseen domains by using only a few training samples. Existing FSOD methods can be categorized into meta-learning- based [44, 34, 45], fine-tuning-base [46, 47, 48, 49], and data-enhanncement based [50, 51, 52, 53) approaches. Although these methods perform well in standard FSOD tasks, their performance generally degrades in CD- FSOD settings due to substantial domain gaps. Several methods [54, 55] specifically address CD-FSOD challenges. For instance, AcroFOD [16] uses domain-aware augmentation reduce domain gaps. OA-FSUI2IT [17] leverages a few unlabeled target-domain samples to generate cross-domain images for feature alignment. CD-ViTO [18] improves inter-class discrimination using DINOv2 prototypes. ETS [56] uses a two-step augmentation strategy for foundation model adaptation. CDFormer [57] introduces transformer-basee based modules distinguish object-backgrout backkround and object-object features. IFC [58] uses learnable instance feature caches for robust prototypes. These methods generally require modifying the original detector's s structure and retraining on the base classes. In this work, we propose novel Mixture-of off-Exerts (MoE) structure to aggregate VFM features and detector features with a few parameters. Our method treats the VFM as an expert model independent of the detector, avoiding modifications to the detector architecture and reducing the computational and time cost of retraining.
Muss--Domiin
Vision Foundation Model. Current vision foundation models (VFMs) are primarily trained under two paradigms: supervised learning and self-supervised learning. Supervised learning model [59, 60] commonly relies on high high-quality labeled datasets for pretraining. These models often exhibit strong generalization, demonstrating excellent performance even without fine-tuning based on downstream tasks. However, large-scale, high-quality labeled datasets are difficult to obtain due to the high cost of manual annotation. To address this problem, self- supervisdd learning models [61, 62, 63, 64, 65, 66] leverage techniques such as contrastive learning [67. 68] and masked image modeling [69, 70] to train on unlabeled data, reducing reliance on labeled datasets and improving image understanding through greater data diversity. However, the lack of accurate bounding boxes limits the localization capabilities of vision foundation models. When the vision foundation model directly used as the backbone for object detection tasks, its weak localization capability is transferred to the detector.
3

---

DINOV2
Mixture of Experts
Layer
Layer
Layer
Loyer
Expert fecture
Trainable
No
Expert-wee wiise Weight
Region-wiss Weight
Expert Projection M
Expert Projection
Mixture of Experts
Mixture Experts
( Times)
ER
RR
ER
RR
Loyer
Loyer
Transformer Encoder Decoder
Cls Head & Reg Head
Object Detection Backbone
Method overview. Our method consists of two main components: 1): The Expert-wise Router (ER) module generates expert-wise gating weight to select the appropriate VFM expert features for the detector's features at different layers. The Region-wise Router (RR) module generates region-wise gating weight to filter out the invalid background regions the VFM feature map. 2): Expert Projection (EP) module: The EP module primarily consists of shared expert projection (SEP) module and private expert projection (PEP) module. The SEP module projects the shared image feature contained in different expert features. The PEP module projects the private image feature contained in each expert feature.
Figure 2:
To overcome this limitation, we use VFM features as experts to improve the detector backbone features, rather than directly replacing the detector's backbone. This approach avoids modifications to the detector architecture and preserves the detector's localization capability. Additionally, since our method does modify the detector structure, can be easily applied other detector architectures.
3 Method
3.1
Method Overview
To fully exploit the VFM's generalization for CD-FSOD while preserving the detector's strong localization capability, we propose novel MoE framework that leverages VFM features to improve detector representations without modifying the original detector architecture, shown in Figure It consists of two key components: the expert routing module and the expert projection modules. Specifically, given VFM expert features {Fr J. J==1 Bx and detector feature FF R R xWW from the 1-th detector backbone layer, where B denotes the batch size, N corresponds to the number of expert features, C, Ca represent the channel dimension of detector feature and VFM expert feature, H x W, Ha x Wa represent the spatial size detector feature and VFM expert feature, the Expert-wise Router (ER) module is used to generate expert-wise gating weights base on F' to select suitable VFM expert features for current detector feature. Simultaneously. the Region -ise Router (RR) module is used to generate region-wise gating weights based on F' to suppress irrelevant background regions in the selected VFM expert feature. Then, the Shared Expert Projection (SEP) and Private Expert Projection (PEP) modules are applied to address the dimensional mismatch between VFM features and the detector features. The SEP module is used to project the shared feature contained in expert features. The PEP module is used to project the private feature contained in each expert feature. After applying expert projection, the projected VFM features are denoted as [F (F In=i E BBBxCxHxNN Wa, Finally, the MoE module fuses VFM features and detector features to obtain F/ BBXC*HXW as the input for the next layer the detector backbone.
3.2 Expert Routers
Expert-wise Router. Detector features at different layers contain different levels of image features. Shallow layers primarily focus on low-level feature such as edges and color, and deep layers focus on high-level semantics such object shape and category. Therefore, the importance of each VFM expert feature varies across different layers of the detector backbone. Inspired by the routing strategy of MoE, we propose the expert-wise router to select the most appropriate VFM expert features for detector features different backbone layers. Specifically, for the /-th layer detector feature F', we first apply

---

global average pooling along the spatial dimensions to aggregate spatial information of detector feature and obtain spatial- -aie aggregated feature vector f, = xxw Ei= " i',,,,ū, i, Then, the tensor f is fed to a learnable fully connected layer O. to aggregate channel information of detector feature and obtain channel-wise aggregated feature vector y - Og (f,)). Finally, a softmax function is applied to the channel- wise aggregated feature vector fy along channel dimension to obtain the expert-wise gating weight G. = softmax (f)).
The expert-wise router generates expert-wise gating weights to select the most suitable VFM expert features for different backbone layers. These VFM expert features typically exhibit high similarity to the detector features, indicating they are more likely to focus similar levels of image semantics. Therefore, the expert-wise router can effectively improve the detector's feature of different layers.
Region-wise Router. Different regions in the VFM feature maps exhibit varying importance for the detector. Foreground regions are typically more informative for object detection and background regions often contain more task-irrelevant image features. To filter out irrelevant background regions and highlight foreground objects, we propose the region-wise router to generate region-wise gating weights for different regions of the VFM expert feature maps. Specifically, the 1-th layer detector feature F' is fed to a learnable fully connected layer 0, along channel dimension to obtain projected tensor f, 0, (F'). The projected tensor f. is then normalized using softmax function to generate the wise gating weights G, = softmax (f,).
The region-wise router generates region- wise gating weights to reweight different regions of the VFM feature map. Compared with VFM, the detector can precisely localize foreground regions because of its stronger localization capability. Therefore, The region-wise router generates region-wise gating weights based on the detector features to highlight foreground regions.
3.3 Expert Projections
Since VFM expert features and detector features differ in both spatial and channel dimensions, it is necessary to project VFM expert into the detector feature space to align their feature dimensionality. However, due to the large number of VFM experts, assigning dedicated projection layer to each VFM expert feature would introduce substantial parameters and degrade training efficiency. In contrast, using single shared projection layer reduces parameter overhead but fails to accommodate the diverse projection needs across VFM expert features. To address this problem, we propose the Shared Expert Projection (SEP) module and Private Expert Projection (PEP) module. Each expert feature contains both shared information, egg., object shape, and private information, egg., fine-grained details from different object regions. The SEP module is used to project shared information across all VFM expert features, and the PEP module is used to project private information specific to each VFM expert feature.
Shared Expert Projection. The shared expert projection module enables the model to effectively project shared image feature across different expert features. Specifically, for the i-th VFM expert feature F we apply a shared linear projection layer 0, E RCa.C. to all expert features, where C. m C, m is hyperparameter that control the channel dimensions of the shared expert projection module transformation. This produces the shared projection component Fü F 0,,
Private Expert Projection. The private expert projection modules enable the model to project private image features contained in each VFM expert feature. Specifically, each expert feature applies its own private projection layer Or E RCaXCp to project its private image features, where C, LC The private projection component F can be obtained F Fj 0
The projected VFM expert feature Fi is constructed by concatenating F and Fý along the channel dimension. After projecting the VFM expert features along the channel dimension, we resize their spatial size via bilinear interpolation match the spatial resolution of 1-th detector backbone feature. The final VFM expert features i-th detector backbone layer are denoted as {F I--1-
3.4 Mixture of Experts
Given the projected expert features ( (F ) and their corresponding expert-wise and region- wise gating weights for the (-th detector backbone layer, the final feature fusion is performed by our MoE
5

---

Table 1: Comparison results the CD-FSOD benchmark. The models trained COCO dataset
and evaluated on six datasets with distinct domain shifts. The best results are highlighted with bold. f indicates that the methods fine-tuned six cross domain datasets.
Methods
Rackhone
ArTaxOr
Clipartik
DIOR
DeepFish
NEU-DET
UODD
Average
Distil-ceissdi [54]
ResNet50
DINO DETRF [20]
ReaNer50
2.9
13.6
6.9
11.6
4.5
2.8
7.1
VITDeTY [72]
VIT-B/14
6.1
12,9
Detic [73]
VIT.1114
11.4
2.2
1-bhot
Detief [73]
VIT-L114
3.2
15.1
4.1
9.0
3.8
4.2
6.6
DE-VIT [28]
VIT-L/14
DE-VITT [28]
VIT-L/14
10.5
13.0
14.7
19.3
0.6
2,4
10.1
[18]
VIT.114
Ourst
ResNer50
26.1
20.1
20.6
24.2
9.1
9.0
18.2
Distill-effodtt [54]
ReaNer50
12.5
23.3
19.1
15.5
16.0
12.2
16.4
DINO DETR? [20]
ReaNe150
21.2
12.3
16.2
9.6
ViTDeTY
B/14
20.9
23.3
23.3
13.5
11.1
16.9
Detic [73]
VIT:/14
11.4
0.1
0.0
0.0
2.2
5-bhot
Detief [73]
VIT-L/14
20.2
10.4
DE-VIT [28]
VIT-L/14
10.1
5.5
7.8
2.5
1.5
3.1
5.1
VIT-L114
CD-VITO= [18]
VIT-L114
47.9
41.1
26.9
22.3
11.4
6,8
26.1
Ourst
ResNet50
63.3
45.1
32.1
29.5
19.0
19.6
34.7
Distill-dffsott [54]
ReaNe150
27.3
26.5
14.5
20.5
DINO DETR [20]
ReaNet50
11.4
23.2
14.4
20.5
11.8
15.2
VITDeTY [72]
V/T-
25.6
29.4
15.8
19.4
10-ibot
Detic [73]
VIT-L/14
11.4
2.2
Detief [73]
VIT-L/14
12.0
22.3
15.4
17.9
16.8
14.4
16.5
[28]
DE-VIT+ [28]
VIT-L/14
49.2
40.8
25.6
21.3
8.8
5,4
25.2
CD-VITOF [18]
VIT-L/14
60.5
44.3
30.8
22.3
12.8
29.6
Ourst
ResNer50
71.3
49.9
37.8
34.1
23.7
22.1
39.8
module:
N =
(a @ &Fr Ga Σ
+ BB G. F
1=1
where @ denotes tensor broadcast multiplication, and and 8 are weighting factors, set to 0.5, 3 0.5 in our experiment. F'' denotes the output of the I-th detector's backbone layer. Simultaneously, it also serves the input to layer + 1. We repeat this process across all backbone layers until obtaining the final layer backbone features, which are then fed into the subsequent detector components to obtain the object detection results.
4 Experiments
We adopt the DINO detector [20] which is trained on the COCO source domain dataset as our baseline and use the self-supervised foundation model DINOv2 [27] as our expert model. Our method is directly applied to the publicly available DINO with ResNet50 [71] backbone, without any additional re-training the source domain dataset. For fine-tuning, employ the AdamW optimizer with learning rate of 2e-3. Following the benchmark in previous work [18], we evaluate our method six datasets with distinct domain shifts: ArTaxOr (insect images), Clipart thand-drawn cartoon image), DIOR (remote sensing imagery). DeepFish (underwater fish images), NEU-DET (industrial defect images), and UODD (marine organism images). For the 1/5/10 shot task settings, we train our model for 400, 800, and 1200 iterations, respectively. All experiments are conducted on four NVIDIA RTX 4090 GPUs. We rename the detector DINO DINO DETR in the following.
4.1 Quantitative Results
Comparison with State-of-The-Arts. As shown in Table 1, we compare our method with typical
CD-FSOD [18, 54], ViT-based OD [72], and open-set based OD/FSOD methods [73, 28]. Our method outperforms the baseline method DINO DETR, achieving improvements of 11.1/21.9/24. mAP on six cross domain datasets under the 1/5/10 shot task settings, respectively. Additionally, compared with the previous state-of-the-e ohethe--tttttttoo- cross cross-domain few-shot object detection method CD-VITO,

---

Table 2: Comparison results of method, MLLMs and OVMs under 10-shot task setting. The best results are highlighted in bold.
Methods
ArTaxOr
Cliparik
DIOR
DeepFish
NEU-DET
UODD
Average
Qwen model [74]
48.8
7.5
2.7
9.2
4.5
1.3
12.3
Ferret model [35]
5.5
8.5
0.8
5.0
0.6
1.4
3.6
YOLO-World [77]
10.5
37.5
3.1
29.5
0.1
0.2
13.5
Grounding DINO (Swin-B) B) [76]
12.8
49.1
4.5
28.6
1.2
10.1
17.7
DINO DETR (ResNet50) Ours
71.3
34.1
23.7
22.1
39.8
Table 3: Result of method extensibility. All models are trained on COCO. The best results on each baseline are highlighted in bold.
Methods
Backhone
ArTaxOr
Cliparik
DIOR
DeepFish
NEU-DET
LODD
Average
DAB-DETR [2i]
ResNer50
8.2
19.4
8.2
6.9
9.6
DAB-DETR Ours
ResNet50
68.7
45.2
31.8
27.5
20.1
22.1
35,9
[22]
ResNet50
23.4
11.6
DETA Ours
ResNet50
69.9
45.5
37.1
26.3
20.9
19.0
36.5
AlignDETR [19]
ResNe150
12.1
23.7
16.1
20.8
12.3
10.7
AlignETTR Ours
ResNet50
72.1
45.6
35.5
27.7
21.7
22.1
37,5
Table 4: Comparison results of different backbone. All models are trained on COCO. The best results are highlighted in bold.
Methods
Backhome
ArTaxOr
ClipartIk
DIOR
DeepFish
NEU-DET
UODD
Average
DINO DETR Ours DINO DETR Ours
ResNet50
71.3
49.9
37.8
34.1
23.7
22.1
39.8
Swin-B
75.4
56.7
39.5
35.1
23.2
23.1
42.2
DINO DETR Ours
VIT-L/14
75.8
60.3
42.0
37.2
25.1
25,9
44.4
our method achieves the improvements of 4.3/8.6/10.2 mAP on six cross domain datasets under the 1/5/10 shot task settings.
Comparison with MILMs and OVMs. As shown in Table 2, we compare our method with multimodal large language models (MLLMs) and open-vocabulary methods (OVMs). Using their
open-source code, we conducted fair comparisons on the same dataset. Qwen model [74] and Ferret model [75] obtain results through text- text-guided visual grounding. Grounding DINO [76] and YOLO-World [77] derive detection results via image-text matching. Our method achieves the highest performance compared with OVMs and MLLMs across six cross domain datasets. Compared with Grounding-DDin DINO and YOLO-World model, our method achieves the improvements of 22. 1/26.3 mAP. For Qwen model and Ferret model, our method achieves the improvements of 27.5/36.2 mAP. We argue that models such as Qwen, despite being trained on large-scale image-text datasets, have never seen the novel classes in the cross domain dataset, resulting in weak zero-shot performance. In contrast, our method adopts a cross-domain learning strategy that integrates with vision foundation models, achieving superior performance on cross-domain tasks.
Method Extensibility Analysis. To further validate the strong extensibility of our method, we adapt our method to other DETR models and evaluate their performance under the 10-shot task setting on all datasets. As shown in Table 3, the average performance of all models has been significantly improved. For instance, the performance DAB-DETR increased from 9.6 mAP 35.9 mAP. To validate the effectiveness of our method across different detector backbones, we apply our approach DINO DETR with various backbones and evaluate their performance under the 10-shot task setting on six cross domain datasets. As shown in Table 4, the experimental results demonstrate that our method achieves strong performance across different backbones. Specifically, when integrating our method with Swin-B [78] backbone, our method performance improves by 2.4 mAP compared to the ResNet backbone, while using a ViT-L [79] backbone leads to an improvement of 4.6 mAP.
Ablation Studies. To assess the contribution of each module, we conduct ablation studies under the 10-shot task setting across six cross domain datasets. As shown in Table 5, adding the SEP module raises the average performance to 34.6 mAP, surpassing the state-of-the-as art CD- ViTO. Building on this, adding the PEP module further improves performance private projection layer effectively retains rich private image features in the VFM expert features. After introducing the feature projection modules, we further evaluate the performance of the routing modules. Adding the expert-wise routing (ER) module improves the average performance to 37.2 mAP, highlighting the importance of selecting different VFM expert features to guide different detector layers. Adding the region- wise routing (RR) module further boosts performance to 38.2
35.7 mAP, demonstrating that the
7

---

Table 5: Results of ablation studies. "SEP" denotes the shared expert projection, "PEP" denotes the private expert projection, "ER" denotes the expert expert-wise wise router, "RR" denotes the region- wise router.
SEP
PEP
FR
RR
ArTaxOr
Clipartk
DIOR
DeepFish
NEU-DET
UODD
Average
x
11.4
23.2
14.4
20.5
11.8
9.9
15.2
x
x
62.1
43.8
34.5
26.4
21.2
19.3
34.6
63.2
44.1
35.8
26.0
21.5
35.6
X
65.1
44.6
34,8
27.4
22.0
20.3
35.7
X
X
69.0
47.2
36,1
32.2
20.9
21.8
37.9
35.2
23.2
22.0
X
67.1
46.7
37.5
28.5
23.2
22.5
37.6
K
68.8
46.8
36.3
27.8
21.8
21.5
37.2
X
35.2
314
22.1
38.2
X 70.3
V
49.1
35.8
32.5
22.3
22.7
38.8
V
37.1
V
71.3
49.9
37.8
34.1
23.7
22.1
39.8
Table 6: Comparison results of generalization. The numbers in parentheses on the right represent the decrease dAP values relative to those in the first row. The green font denotes the best performance. Under the 1/5/10-shot task setting, our method consistently achieves the lowest dAP values, demonstrating its strong generalization capability in addressing cross-domain tasks.
Method
1-hhot FPJ
S-shot FPJ
10-sho FP
DINO DETR
44.45
40.12
35.68
DINOv2
29.73 (-14.72)
27.70 (-12.42)
(-5.42)
Ours
26.47 (-17.88)
22.54 (-17.58)
17.98 (-17.70)
mAP, demonstrating that the RR module effectively highlights foreground regions in the feature maps and filters out irrelevant background information. Using ER and RR together achieves the best result of 39.8 mAP, confirming their complementarity.
Cross-Domaie Generalization Analysis. To validate that the VFMs have strong generalization and the detectors have poor generalization the cross domain tasks, we fine-tune DINO DETR, DINOv2 and our method on six cross-domain datasets and evaluate their performance. DINOv2 refers to replacing the ResNet50 backbone of original DINO DETR with DINOv2. We the average precision loss(dAP) caused by false positive samples as our evaluation metric. As shown Table 6, DINO DETR has the highest dAP values reach 44.45/40. 12/35.68 under the 1/5/10-shot task settings, respectively, indicating its poor generalization in cross domain tasks. contrast, DINOv2 has lower indicating the strong generalization capability. Our method leverages
dAP values of 29.73/27.7/30.2 a router module further to filter out irrelevant background region in VFM feature map. As a result, method achieves the lowest dAP values of 05.47/22.54/177 respectively. Notably, under the 10-shot task setting, our method reduces dAP by 12.28 compared with DINOv2, validating the stronger generalization capability of our approach.
under the 1/5/10-shot settings,
Cross-Domain Localization Performance Analysis. To validate that the detectors have strong localization capabilities and the VFMs lack accurate localization capabilities, we fine-tune DINO DETR, DINOv2 and our method on six cross-domain datasets and evaluate their performance under the 1-shot task setting. We use the decrease of AP75 relative AP50 to evaluate model's I localization capability. As shown in Table 7, DINOv2 shows the highest drop of 69.35%, indicating limited localization capabilities because its training dataset lacks accurate bounding box annotations. In comparison, DINO DETR exhibits better localization performance with a drop of 48.76%% 76%. Our method treats DINOv2 an independent expert model improve the detector's generalization rather than using DINOv2 directly as the detector backbone. Therefore, our method avoids the transfer of weak localization capabilities from the VFM the detector. Experimental results show that our method achieves an attenuation of only 40.41 %, demonstrating its effectiveness in maintaining the detector's localization performance.
4.2 Qualitative Visualizations
Method Analysis. To validate our method's effectiveness, we visualize the backbone feature maps. As shown in Figure 3, our method makes the model focus on foreground regions, showing that our router module can effectively filter out the useless background in the VFM feature maps. Additionally, use t-SNE [80] visualize backbone features on the DIOR dataset to demonstrate the model's

---

Table 7: Comparison results of localization capability under the 1-shot task setting. The arrows and values on the top-right denote the decrease in AP75 relative to AP50. Our method exhibits the smallest relative decrease, demonstrating strong robustness in localization performance.
DINO<2
AP50
AP75
AP50
AP75
AP50
AP75
12.51
6.41 4440
19.38
5.94
28.48
16.97
DINO DETa
DINO22
Figure 3: Visual comparison of backbone feature. The red regions in the figure represent show the model's focus regions.
DINO DETR
DINOv2
Figure 4: The t-SNE visualization of different detector features the DIOR dataset. Each point in the figure represents sample from the dataset, with color indicating its class. Our method effectively minimizes the distance between samples within the same class while maximizing the separation between samples of different classes
ArToxOr
Clparttk
bIOR
DeepFish
NEU-SET
U0b0
Shage
Expertveiee gettnn weihtt
Stope 2
Shoee 3.
Stowe
DINO export feotune index
Figure 5: Expert feature selection different layers of the backbone. The horizontal axis represents expert features, the vertical axis represents the expert-wise gating weights at different layers.
classification performance. As shown in Figure 4, method shows higher intra-class compactness and inter-class separability, further demonstrating the strong generalization of our model.
Expert-wist Routing Mechanism. To illustrate that the importance of expert features varies across backbone layers, we analyze expert-wise gating weights. As shown Figure 5, different backbone layers employ different VFM feature selection strategies. Deep detector features generally tend to

---

Stration
Figure 6: Heatmap visualization of region-wise gating weight, the brighter areas indicate higher levels of model attention.
select VFM expert feature from the deep VFM layer because they contain similar level image features. However, the deep detector features also select some shallow VFM expert features, showing that low-level image features, like object edges and colors, can also improve the high-level image feature. The VFM feature selection strategy at the same detector layer also changes on the different test datasets. These experimental results indicate that it is necessary to introduce a learnable module to dynamically adjust the VFM feature selection strategy. Therefore, inspired by the routing mechanism in MoE, we propose an expert-wise router method that generates expert gating weights based on detector features to select the VFM expert feature dynamically.
Region-wise Routing Mechanism. The information density is different different regions of the VFM feature map. The foreground regions typically have higher information density, and background regions tend to have lower information density. To make the model to focus on the foreground region, we propose region-wise router filter background region of the VFM feature map and highlight foreground regions. To demonstrate the effectiveness of our method, we visualize the region- gating weights by using heatmap. As shown in Figure 6, after applying our region-wise router, the model focuses on foreground regions and ignores the irrelevant background regions.
Conclusion
In this paper, we propose a novel Cross-Domain Few-Shot Object Detection (CD-FSOD) paradigm leveraging a Mixture of Experts (MoE) architecture to integrate the vision foundation model with the well-trained detection model. Our method transfers VFM's generalization to the detector without modifying the original detector structure retraining and region-wise routers to select VFM expert feature and filter irrelevant background regions in the VFM feature map. Additionally, projections module, which decouple the shared and private image feature projections and minimize the parameters introduced by the VFM feature projection. Extensive experiments demonstrate the effectiveness of our approach in improving cross-domain performance.
the base class. We introduce expert-wise
propose the shared expert projection module and private expert
Acknowledgements
This work is supported in part by the National Natural Science Foundation of China (62192783, 62276128, 62406140), Young Elite Scientists Sponsorship Program by China Association for Science and Technology (2023QNRC001) the Key Research and Development Program of Jiangsu Province under Grant (BE2023019) and Jiangsu Natural Science Foundation under Grant (BK20221441, BK20241200). The authors would like to thank Huawei Ascend Cloud Ecological Development Project for the support of Ascend 910 processors.
10

---

References
[1] Guangxing Han, Yicheng He, Shiyuan Huang. Jiawei Ma. and Shih-Fu Chang. Query adaptive few-shot object detection with heterogeneous graph convolutional networks. ICCV, 2021.
[2] Jiaming Han, Yuqiang Ren, Jian Ding. Yan, and Gui-Song Xia. Few-shot object detection via variational feature aggregation. In AAAI, 2023.
[3] Prannay Kaul, Weidi Xie, and Andrew Zisserman. Label, verify, correct: simple few shot object detection method. In CVPR, 2022.
[4] Bo Sun, Banghuai Li, Shengcai Cai, Ye Yuan, and Chi Zhang. Fsee: Few-shot object detection via contrastive proposal encoding. In CVPR, 2021.
[5] Chenchen Zhu, Fangyi Chen, Uzair Ahmed, Zhiqiang Shen, and Marios Savvides. Semantic relation reasoning for shot-stable few-shot object detection. In CVPR, 2021.
[6] Qi Fan, Wei Zhuo, Dhi-Keung Tang, and Yu- Wu-Wing Tai. Few-shot object detection with attention rpn and multi-relation detector. In CVPR, 2020.
[7] Qi Fan, Chi-Keung Tang, and Yu- Vu-Wing Tai. Few-shot object detection with model calibration. In ECCV, 2022.
[8] Qi Fan, Wei Zhuo, Chi MhinKe Tang, and Yu- Wi-Wing Tai. Fsodv2: A deep calibrated few-shot object detection network. IJC,, 2024.
[9] Joseph Redmon and Ali Farhadi. Yolo9000: better, faster, stronger. In CVPR, 2017.
[10] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real- ail-time object detection with region proposal networks. TPAMI. 2016.
[11] Ross Girshick. Fast r-cnn. ICCV, 2015.
[12] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yart Fu, and Alexander Berg, Ssd: Single shot multibox detector. ECCV, 2016.
[13] Yulu Gao, Yifan Sun, Xudong Ding, Chuyang Zhao, and Liu. Ease-detr: Easing the competition among object queries. In CVPR, 2024.
[14] Tahira Shehzadi, Khurram Azeem Hashmi, Didier Stricker. Muhammad Zeshan Afzal. Sparse semi-detr: sparse learnable queries for semi-supervised object detection. In CVPR, 2024.
[15] Mingqiao Ye, Lei Ke, Siyuan Li, Yu- ln-Wing Tai, Chi-Keung Tang, Martin Danelljan, and Fisher Yu. Cascade-detr: delving into high-quality universal object detection. In ICCV, 2023.
[16] Yipeng Gao, Lingxiao Yang. Yunmu Huang. Song Xie, Shiyong Li, and Wei-Shi Zheng. Acrofod: An adaptive method for cross-domain few-shot object detection. In ECCV, 2022.
[17] Lifan Zhao, Yunlong Meng, and Lin Xu. Oa-fsuiZit: A novel few-shot cross domain object detection framework with object-aware few-shot unsupervised image-to-iman translation. In AAAI, 2022.
[18] Yuqian Fu, Yu Wang, Yixuan Pan, Lian Huai, Xingyu Qiu, Zeyu Shangguan, Tong Liu, Yanwei Fu, Luc Van Gool, and Xingqun Jiang. Cross-domain few-shot object detection via enhanced open-set object detector: In ECCV, 2024.
[19] Zhi Cai, Songtao Liu, Guodong Wang, Zheng Ge, Xiangyu Zhang, and Di Huang. Align-detr: Improving detr with simple iou-aware bee loss. In BMVC, 2024.
[20] Hao Zhang, Feng Li. Shilong Liu, Zhang, Hang Su, Zhu, Lionel Ni, and Heung- Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. In ICLR, 2023.
[21] Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi. Hang Su, Jun Zhu, and Lei Zhang. Dab-detr: Dynamic anchor boxes are hetter queries for detr. In ICLR, 2022.
[22] Jeffrey Ouyang- Jang Hyun Cho, Xingyi Zhou, and Philipp Krähenbühl. Nms strikes back. arXiv preprint arXin:2212.066 221,122772. 2022.
[23] Shashanka Venkataramanan, Mamshad Nayeem Rizve, João Carreira, Yuki Asano, and Yannis Avrithis. Is imagenet worth video? learning strong image encoders from long unlabelled video. In ICLR, 2024.
11

---

[24] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. In International Conference on Learning Representations, 2021.
[25] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, 2020.
[26] Xinlei Chen, Zhuang Liu, Saining Xie, and Kaiming He. Deconstructing denoising diffusion models for self-supervised learning. arXiv preprint arXiv:2401. 144044 2024.
[27] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaacidin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07199 2023.
[28] Xinyu Zhang, Yuhan Liu, Yuting Wang, and Abdeslam Boularias. Detect everything with few examples. In CoRL, 2023.
[29] Robert A Jacobs, Michael I Jordan, Steven Nowlan, and Geoffrey E Hinton. Adaptive mixtures of local experts. Neural computation, 1991.
[30] Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Qiao. Vision transformer adapter for dense predictions. In The Eleventh International Conference on Learning Representations, 2022.
[31] Tianrun Chen, Lanyun Zhu, Chaotao Ding. Runlong Cao, Yan Wang. Zejian Li, Lingyun Sun, Papa Mao, and Ying Zang. Sam fails to segment Camouflage, shadow, medical image segmentation, and more. arXiv preprint arXiv:2304.09145 2023.
Adapting sam in underperformed scenes:
[32] Lingyi Hong, Shilin Yan, Renrui Zhang, Wanyun Li, Xinyu Zhou, Pinxue Guo, Kaixun Jiang, Yiting Chen, Jinglun Li, Zhaoyu Chen, et al. Onetracker: Unifying visual object tracking with foundation models and efficient tuning. In CVPR, 2024.
[33] Song Tang, Wenxin Su, Mao Ye, and Xiatian Zhu. Source-free domain adaptation with frozen multimodal foundation model. In CVPR, 2024.
[34] Guangxing Han and Ser-Nam Lim. Few-shot object detection with foundation models. In CVPR, 2024.
[35] Xiyang Dai, Yinpeng Chen, Jianwei Yang. Pengchuan Zhang. Lu Yuan, and Iei Zhang. Dynamic detr: End-to-end object detection with dynamic attention. In ICCV, 2021.
[36] Zhigang Dai, Bolun Cai, Yugeng Lin, and Junying Chen. Up-detr: Unsupervised pre-training for object detection with transformers. In CVPR, 2021.
[37] Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308. 10792, 2023.
[38] Qiang Chen, Xiaokang Chen, Jian Wang, Shan Zhang. Kun Yao, Haocheng Feng, Junyu Han, Errui Ding, Gang Zeng, and Jingdong Wang. Group detr: Fast detr training with group-wise one-to-many assignment. In CVPR, 2023.
[39] Dehua Zheng, Wenhui Dong, Hailin Hu, Xinghao Chen, and Yunhe Wang. Less is more: Focus attention for efficient detr. In ICCV, 2023.
[40] Shihua Huang, Zhichao Lu, Xiaodong Cun, Yongjun Yu, Xiao Zhou, and Xi Shen. Deim: Detr with improved matching for fast convergence. In CVPR, 2025.
[41] Feng Li, Hao Zhang, Shilong Liu, Jian Guo, Lionel M Ni, and Lei Zhang. Dn-detr: Accelerate detr training by introducing query denoising. In CVPR, 2022.
[42] Depu Meng. Xiaokang Chen, Zejia Fan, Gang Zeng. Houqiang Li, Yuhui Yuan, Lei Sun, and Jingdong Wang. Conditional detr for fast training convergence. In CVPR, 2021.
[43] Yifan Pu, Weicong Liang, Yiduo Hao, Yuhui Yuan, Yukang Yang, Chao Zhang, Han Hu, and Gao Huang. Rank-detr for high quality object detection. In NeurIPS, 2023.
[44] Guangxing Han, Jiawei Ma. Shiyuan Huang, Long Chen, and Shih-Fu Chang. Few-shot object detection with fully cross-transformer. In CVPR, 2022.
[45] Yuewen Li, Wenquan Feng. Shuchang Lyu, Qi Zhao, and Xuliang Li. Mm-fsod: Meta and metric integrated few object detection. CoRR, 2020.
12

---

[46] Hao Chen, Yali Wang. Guoyou Wang, and Yu Qiao. Lstd: A low-shot transfer detector for object detection. In AAAI, 2018.
[47] Karim Guirguis, Johannes Meier, George Eskandar, Matthias Kayser, Bin Yang, and Jürgen Beyerer. Niff: Alleviating forgetting in generalized sew-shot object detection neural instance feature forging. In CVPR, 2023.
[48] Xin Wang, Thomas Huang, Joseph Gonzalez, Trevor Darrell, and Fisher Yu. Frustratingly simple few-shot object detection. In ICML, 2020.
[49] Limeng Qiao, Yuxuan Zhao, Zhiyuan Li, Xi Qiu, Jianan Wu, and Chi Zhang. Defren: Decoupled faster r-cnn for few-shot object detection. In ICCV, 2021.
[50] Yuhang Cao, Jiaqi Wang, Ying Jin, Tong Wu, Kai Chen. Ziwei Liu, and Dahua Lin. Few-shot object detection via association and discrimination. In NeurlPS, 2021.
[51] Renrui Zhang. Wei Zhang, Rongyao Fang, Peng Gao, Kunchang Li, Jifeng Dai. Yu Qiao, and Hongsheng Tip-adapter: Training-free free adaption clip for few-shot classification. In ECCV, 2022.
[52] Jiaxi Wu, Songtao Liu, Di Huang, and Yunhong Wang. Multi-scale positive sample refinement for few-shot object detection. In ECCV, 2020.
[53] Weilin Zhang and Yu- Xi-ni Wang. Hallucination improves few-shot object detection. In CVPR, 2021.
[54] Wuti Xiong. Cd-fsod: A benchmark for cross-domain few-shot object detection. In ICASSP, 2023.
[55] Kibok Lec, Hao Yang. Satyaki Chakraborty, Zhaowei Cai, Gurumurthy Swaminathan, Avinash Ravichan- dran, and Onkar Dabeer. Rethinking few-shot object detection on a multi-domain benchmark. In ECCV, 2022.
[56] Jiancheng Pan. Yanxing Liu, Xiao He, Long Peng. Jiahao Li. Yuze Sun, and Xiaomeng Huang. Enhance then search: An augmentatiinn-saare strategy with foundation models for cross-domain few-shot object detection. CVPR, 2025.
[57] Boyuan Meng. Xiaohan Zhang, Peilin Li, Zhe Wu, Yiming Li. Wenkai Zhao, Beinan Yu, and Hui EtinLiang Shen. Cdformer: preprint arXiv:2505.00938 2025.
few-shot object detection transformer against feature confusion. arXiv
[58] Yali Huang, Jie Mci, Yiming Yang, Mi Guo, Mingyuan Jiu, and Mingliang Xu. Instance feature caching for cross-domain few-shot object detection. In CVPR, 2025.
[59] Hugo Touvron, Mathieu Cord, and Hervé Jégou. Deit ii:: Revenge of the vit. In ECCV, 2022.
[60] Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang. Jenq-Neng Hwang, al. Grounded language-image pre-training, In CVPR, 2022.
[61] Xinlei Chen, Saining Xie, and Kaiming He. empirical study training self-supervised vision transformers. In JCCV, 2021.
[62] Sara Atito, Muhammad Awais, and Josef Kittler. Sit: Self-supervised vision transformer. arXiv preprint arXiv:2104.0360 2021.
[63] Chunyuan Li, Jianwei Yang, Pengchuan Zhang, Mei Gao, Bin Xiao, Xiyang Dai, Lu Yuan, and Jianfeng Gao. Efficient self-supervised vision transformers for representation learning. In ICLR, 2021.
[64] Sukmin Yun, Hankook Lee, Jachyung Kim, and Jinwoo Shin. Patch-level representation learning for self-supervised vision transformers. In CVPR, 2022.
[65] Zhaowen Li, Zhiyang Chen, Fan Yang, Wei Li, Yousong Zhu, Chaoyang Zhao, Rui Deng, Liwei Wu, Rui Zhao, Ming Tang, et al. Mst: Masked self-supervised transformer for visual representation. In NeurIPS, 2021.
[66] Richard J Chen and Rahul G Krishnan, Self-supervised vision transformers learn visual concepts in histopathology. CoRR, 2022.
[67] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, 2021.
13

---

[68] Shashanka Venkataramanan. Mamshad Nayeem Rizve, João Carreira, Yuki Asano, and Yannis Avrithis. Is imagenet worth video? learning strong image encoders from long unlabelled video. In ICLR, 2024.
[69] Wenhui Wang, Hangho Bao. Li Dong, Johan Bjorek, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, al. Image foreign language: Beit pretraining for vision and visionlangguae tasks. In CVPR, 2023.
[70] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, 2022.
[71] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
[72] Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. ECCV, 2022.
[73] Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Krähenbühl, and Ishan Misra. Detecting twenty- thousand classes using image-level supervision. ECCV, 2022.
[74] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenhin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-v1 technical report. arXiv preprint arXiv:2502. 13923, 2025.
[75] Haoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du, Bowen Zhang, Zirui Wang. Liangliang Cao, Shih-Fu Chang, and Yinfei Yang. Ferret: Refer and ground anything anywhere at any gramularity. CoRR, 2023.
[76] Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chumyuan Li, Jianwei Yang, Hang Su, et Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In ECCV, 2024.
[77] Tianheng Cheng, Lin Song. Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In CVPR, 2024.
[78] Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021.
[79] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghami, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 1021.
[80] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 2008
[81] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021.
[82] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, al. Lora: Low-rank adaptation of large language models. ICLR, 2022.
[83] Shenghao Fu, Junkai Yan, Qize Yang. Xihan Wei, Xiaohua Xie, and Wei-Shi Zheng. Frozen-detr: Enhancing detr with image understanding from frozen foundation models. In NeurlPS, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly state the main contributions of our method.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [NA]
Justification: This paper has limitations, but those are not discussed in the paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the method was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the method. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their method to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [No]
15

---

Justification:
Our assumptions are validated through experimental results, without involving formal derivations or mathematical proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Our method is straightforward implement and highly reproducible.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: We will provide part of the code and links to the publicly available datasets used in this paper in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/ public/guides/ wttp://oooeSuuuueonnnne.
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https: CoodeSubmiis
httpsscccppublic//eec gomdeeSSSu ssi wswonnoliccyiooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
All experimental settings are thoroughly described the main text.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Our experimental results represent consistent and reliable findings confirmed through multiple independent trials.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have detailed the computational resources used in experiments in the Experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: stttps//neuripee htc/ppbllc/Ethhhshiiii del ines?
Answer: [Yes]
Justification: The research conducted in this paper fully complies with the NeurIPS Code of Ethics in all aspects.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our method significantly enhances the performance of well-trained object detection models on cross-domain tasks, making it more aligned with real-world scenarios.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [Yes]
Justification: Our method exclusively utilizes open-source ppen-source models and datasets, ensuring that there is no risk of misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Our method exclusively utilizes open-source models and datasets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [No]
Justification:
This paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [No]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [No]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https : h/tp://ies...//eeeeeeeeee http///ris..c//ooeeeeeeeeeeeeeee / for what should or should be described.
21

---

A Ablation Study on Different Vision Foundation Models
To evaluate the impact of using different VFMs as expert models in our method, we compare the performance using DINOv2 [27]. CLIP [81], and combination of both as expert models. As shown in Table 8, DINOv2 has better performance than CLIP on cross-domain tasks. We further explore integrating multiple vision foundation models simultaneously to boost feature representation, The integration of DINOv2 and CLIP exceeds the GPU memory limitation, forcing to reduce the batch size to 1. Experimental results demonstrate that integrating multiple vision foundation model leads to slight performance improvement, confirming the complementary of different vision foundation models. However, this integration significantly increases training time and memory consumption. Considering the trade-off between performance improvement and computational costs, we ultimately only select DINOv2 as the expert model.
Table Comparison results of combining different vision foundation models. The best results are highlighted with bold. bs denotes the batch size.
Method
ArTaxOr
ClipartIk
DeepFish
NEU-DET
UODD
Avg
Training time
Baseline (bs=2) 2)
11.4
23.2
14.4
20.5
11.8
15.2
0.8h
CIIP (bs
53.7
47.7
35.0
28.9
24.0
19.2
34.8
1.7h
+ .DINOv2 (bs = 2, Ours)
71.3
49.9
37.8
34.1
23.7
22.1
39.8
2.0h
-DINOv2&CLIP
71.8
50.2
33,9
39.9
3.6b
B Ablation Study on DINOv2 Models of Different Parameter Sizes
To evaluate the impact of VFMs with different parameter scales, we use DINOv2-smar DINOv2- base, DINOv2-large, and DINOv2-giant the 10-shot task setting. As shown in Table 9, DINOv2-giam exceeds the memory capacity of our GPU, forcing us reduce the batch size to during training. The performance of DINOv2-giant is suboptimal due to its large intermediate feature maps, which require processing a large number of additional parameters. These extra parameters not only hinder effective fine-tuning downstream tasks but also considerably increase training time. In contrast, DINOv2-large offers a balanced trade-off' between parameter size, training time, and performance improvement, delivering the best results within reasonable computational cost. Consequently, we select DINOv2-large as the expert model.
expert models and evaluate method permformance under
Table 9: Comparison results of DINOv2 Models of Different Parameter Sizes. Size denotes the parameter scales of different DINOV2 models.
Method
ArTaxOr
Clipartlk
DIOR
DeepFish
NEU-DET
UODD
Training time
Size
DINOv2-S (bs 2)
54.9
38.5
30.4
28.5
22.4
20.2
1.1h
21M
DINOv2-B 2))
67.5
45.9
35.6
30.4
21.6
21.3
1.5h
86M
Ours)
37.8
DINOv2 1bs=1)
69.1
48.7
36.7
33.1
22.0
21.4
4,5h
1100M
C Ablation Study on Hyperparameters Setting
the shared and private expert projection modules, we introduce the hyperparameters m, to control the parameter scales of the shared expert projection module and the private expert projection module. e.g., F = = -,, E IRB C, x H Fü = Fi Fi-0; E I BxCx X xW where C. = m C, C. CC To determine the optimal configuration, we conduct ablation study the values of m. As shown in Table 10, our method shows the best performance when m 16.
In the mixture of experts module, we introduce two hyperparameters, a and B, e.g., F Fl ΣN a Gra.l @ F + 3 G.d @ Fr. To evaluate the impact of these parameters, we perform an ablation study on and B. As shown Table 11, the results indicate that the optimal configuration 0.5 and 3 0.5.
22

---

Table 10: The 10-shot ablation results on hyperparameters m and n.
ArTaxOr
Clipartlk
DIOR
DeepFish
NEU-DET
UODD
Avg
2
71.9
49.6
36.6
31.1
22.8
18.2
38.4
4
70.0
49.0
37.3
32.5
21.1
21.4
38.6
8
70.3
48.9
35.7
32.3
22.4
21.4
38.5
(Ours)
37.8
22.1
32
69.5
48.5
36.4
33.2
23.1
22,9
38.9
Table 11: The 10-shot ablation results on hyperparameters a and |3.
&
ArTaxOr
Clipartik
DIOR
DeepFish
NEU-DET
UODD
71.1
22.9
0.3, 0.7
70.5
49.7
30.5
34.1
23.0
20.6
38.1
0.5, (Ours)
71.3
49.9
37.8
34.1
23.7
22.1
39.8
0.7. 0.3
70.1
49.2
37.6
31.9
21.7
18.1
38.1
0.9, 0.1
68.2
49.8
36.4
32.2
21.1
21.0
38.1
D Ablation Study on Fine-Tuning Strategy
To evaluate the impact of different fine-tuning strategies, we evaluate the performance four fine- tuning strategies: Full Finetune, LoRA [82], Partial Finetune and fine-tuning strategy. As shown in Table 12, All fine-tuning Strategies keep the vision foundation model frozen without updating its parameters. LoRA-based fine-tuning achieves the lowest number of trainable parameters but suffers from limited performance. Partially fine-tuning the model while excluding the backbone introduces moderate increase in trainable parameters and yields improved results. Full fine-tuning achieves the highest accuracy, yet comes at the cost of sigmificant training overhead and a heightened risk of overfitting few-shot scenarios. the classification head, regression head and the new proposed module, strikes an effective balance between computational efficiency and detection performance, demonstrating strong generalization in cross-domain few-shot object detection tasks.
contrast, our proposed strategy, which fine-tunes only
Table 12:
The 10-shot ablation results on different finetuning strategies. Full Finetune denotes fine- tuning all model parameters. LoRA denotes fine-tuning all model based on Low-Rank Adaptation. Partial Finetune denotes fine-tuning all components except the backbone. Ours denotes fine-tuning only the classification head, regression head, and the proposed module. Truin params indicates number of trainable parameters.
Fine- Tuning Strategy
ArTaxOr
ClipartIk
DIOR
DeepFish
NEU-DET
UODD
Avg
Full Finetune
55.2
39.0
32.6
22.3
20.8
19,3
31.5
LoRA
26.7
15.9
Partial Fineture
50.7
34.7
30.9
22.6
20.9
16.2
29.3
Ours
23.7
22.1
E
Ablation Study on Feature Aggregation Strategy
To validate the effectiveness and necessity of our proposed MoE-based feature aggregation strategy, compare method with four simpler feature aggregation strategies. Specifically, we design and evaluated four variants:
All Stages: Retain all stage PEP modules and replace them with simple MLPs, removing all other modules.
Final Stage: Retain only the final-stage PEP module and replace it with simple MLP, removing all other modules.
Linearly Adding: Remove all modules and aggregate features at the backbone stage via linear addition.
Learnable Weight: Introduce a learnable weight parameter aggregate features from different layers of the VFM.
23

---

As shown in Table 13, simple feature aggregation strategies lead to substantial performance decline, thereby validating the efficacy of our approach. Learnable Weight feature aggregation strategy introduces learnable weight parameters as simple routing network to select and aggregate VFM features, thereby maintaining the MoF. structure similar ours. contrast, the other three aggregation strategies (All Stages, Final Stage, Linearly Adding) simply aggregate the VFM features into the backbone features without selecting them. Therefore, it outperforms the other simple aggregation strategies. However, it still leads to decline in performance compared to method, highlighting the importance of backbone features in guiding VFM feature selection.
Table 13: Comparison results of our method and simple feature aggregation strategies.
Method
ArTaxOr
Clipartik
DIOR
DeepFish
NEU-DET
UODD
Avg
All Stages
56.3
44,4
34.1
26.4
16.8
17.6
32.6
Final Stage
48.8
43,8
30.7
24.1
17.1
17.4
30.3
Linearly Adding
4.7
Learnable Weight
62.5
47.4
31.6
31.1
19.1
20.7
35.4
Ours
37.8
34.1
39,8
F
Comparison with Recent Works
Comparison with ETS. We compare method with ETS [56] six cross domain datasets. For fairess,, we reproduce the experiments using the official ETS code. As shown in Table 14, our method can be directly applied Grounding DINO without any modification and achieves similar performance to ETS, even though is not specifically designed for Grounding DINO.
Table 14: Comparison results of ETS method and our method under the 10-shot task setting.
Method
ArTaxOr
Clipartik
DIOR
DeepFish
NEU-DET
UODD
Arg
implementation)
70.6
42.8
26.1
Grounding DINO (Swin-B) + Ours
73.8
60.3
40.2
37.0
27.9
29.2
44.7
Comparison with Frozen DETR. We compare method with Frozen DETR [83]. We adopt the same baseline structure as Frozen DETR. Frozen DETR follows a two-stage training paradigm of base training and finetuning. To align the experimental and dataset settings, we skip the base training step, finetuning only on the cross domain datasets. To validate the effectiveness of the method on in-domain datasets, we train our method on COCO for 12 epochs and compare of Frozen DETR reported in the original paper. As shown in Table 15, our method consistently outperforms Frozen DETR across six cross domain datasets and the COCO dataset.
the performance
Table 15: Comparison results of Frozen DETR and our method under the 10-shot task setting.
Method
ArTaxOr
Clipartlk
DIOR
DeepFish
NEU-DET
UODD
COCO
DINOrResNet50)
2.9
13.6
6.9
11.6
4.5
2.8
49.0
Frozen DETR
45.8
33,9
31.5
17.5
9.1
3.2
51.9
DINO(ResNet50) Ours
71.3
49,9
37.8
34.1
23.7
22.1
55.2
G Analysis of Routing Network Based on Detector Feature
To validate the feasibility of using detector features to route VFM features, we visualize the feature maps of detector and VFM using PCA. As shown in Figure the detector shows higher attention in the foreground regions, reflecting its strong localization ability. However, the foreground object feature are unclear, leading to weaker classification performance. feature representations but maintains high attention in multiple background regions, indicating its weaker localization ability. Therefore, we leverage the detector's localization ability to guide VFM feature maps in supplementing the missing key features in foreground regions.
contrast, the VFM extracts clear

---

Figure 7: Visual comparison of detector feature and DINOv2 feature. The red regions in the figure represent the areas of focus during feature extraction by the model.
25


Reviews:
[{"review_id": "yF6myWpITN", "reviewer": "Reviewer_ZRE6", "summary": "This paper presents  a novel framework for cross domain few shot object detection by combining the detection models with vision foundation model in a mixture of experts (MOE) based method. Leveraging VFM and DETR through MOE framework enables improved generalization without loss of localization. In particular, it introduces a plug-in strategy that keeps a well-trained DETR detector frozen and boosts its generalization by injecting features from a vision foundation model, DINOv2. The proposed strategy introduces expert-wise & region-wise routers dynamically weight which DINOv2 layers and spatial regions matter for each DETR backbone stage as well as projection layers (shared and private) for feature alignment. Experimental study on different domains show improved performance over the counterpart methods", "questions": "I have listed all my concerns in the \"Strength and Weaknesses section\". Please refer to \"Weaknesses\" for the raised concerns and questions.", "limitations": "N/A", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1)\tThe paper is well motivated and efficiently integrates vision foundation models with DETR architectures.\n\n2)\tThe introduced novelties are in plug-and-play format, making them generalizable to various DETR methods\n\n3) Experiments show consistent improvement over the baselines\n\nWeaknesses\n\n1)\tComputational latency: How does the computational latency of the proposed method compared to counterpart methods\n\n2)\tComparisons: How does the proposed method compared to open vocab methods(e.g. grounding-dino, yolo-world).\n\n3)   How does  the proposed method perfroms compared to multi modal LLMs (MLLMs)  such as Qwen models or Ferret models which are well known with their visual grounding.\n\n4)\tNo limitations are listed in the paper. What are the potential pitfall for proposed method.", "was_revised": true, "final_justification": "I would like to thank authors for effectively addressing my concerns. I have increased my score to 5.   Overall, I think this paper brings an important perspective, i.e. leveraging vision foundation models for enhanced downstream object detector performances without substantial computational sacrifice in inference time. I strongly encourage the authors to incorporate the arguments in the rebuttal, namely computational efficiency and comparisons with MLLMs and openvocab models.", "rebuttal": "We sincerely thank Reviewer ZRE6 for the constructive feedback, insightful questions, and valuable suggestions!\n\nBelow we address your concerns.\n\n>**Q1: Inference Speed Comparisons.**\n\n**R1**: Thanks for your insightful question. \n\nWe compare inference speed with YOLO‑World and Qwen Model using Frames Per Second(FPS). Our method achieved similar FPS like YOLO‑World while attaining the highest performance, as shown in Table 1.\n\n**Table 1: Comparison results of inference speed and performance. 'mAP' denotes the mean performance across the six cross‑domain datasets.**\n\n|      Model       |  FPS  |  mAP  |\n|:----------------|:-----:|:-----:|\n|   Qwen Model     |  0.4  | 12.3  |\n|   YOLO‑World     |  5.8  | 13.5  |\n| DINO DETR + Ours |  5.6  | 39.8  |\n\n>**Q2&Q3: Comparison to MLLMs and OVMs.**\n\n**R2&R3**:  Thanks for your constructive suggestion. \nWe compare our method with multimodal large language models(MLLMs) and open‑vocabulary methods(OVMs). Using their open‑source code, we conducted fair comparisons on the same dataset. Qwen model and Ferret model obtain results through text‑guided visual grounding. Grounding‑DINO and YOLO‑World derive detection results via image‑text matching. All codes are publicly available on their official websites.\n\nAs shown in Table 2, our method achieved highest performance compared to OVMs and MLLMs across six cross domain datasets. Compared to Grounding‑DINO and YOLO‑World model, our method achieves the improvements of 22.1mAP/26.3mAP. For Qwen model and Ferret model, our method achieves the improvements of 27.5mAP/36.2mAP. We argue that models such as Qwen, despite being trained on large‑scale image‑text datasets, have never seen the novel classes in the cross domain dataset, resulting in weak zero‑shot performance. In contrast, our method adopts a cross‑domain learning strategy that integrates with visual foundation models, achieving superior performance on cross‑domain tasks. This also points to an important future direction, where we plan to extend our cross‑domain learning approach to MLLMs to further improve their performance on cross‑domain tasks.\n\n**Table 2: Comparison results of our method, MLLMs and OVMs under 10-shot setting.**\n\n| Model                  | ArTaxOr | Clipart1k |  DIOR  | DeepFish | NEU-DET |  UODD  | Avg|\n|:----------------------|:-------:|:---------:|:------:|:--------:|:-------:|:------:|:------:|\n| Qwen model             |  48.8   |    7.5    |  2.7   |   9.2    |   4.5   |  1.3   |12.3|\n| Ferret model          |     5.5    |    8.5       |    0.8    |     5.0     |    0.6     |    1.4    | 3.6 |\n| YOLO-World             |  10.5   |   37.5    |  3.1   |  29.5    |   0.1   |  0.2   |13.5|\n| Grounding DINO (Swin-B)     |  12.8   |   49.1    |  4.5   |  28.6    |   1.2   | 10.1   | 17.7|\n| DINO DETR (ResNet50) + Ours | 71.3 |   49.9    | 37.8   |  34.1    |  23.7   | 22.1   |39.8|\n>**Q4: Analysis of method limitations.**\n\n**R4**: Thanks for your insightful question. \n\nA key limitation of our method is that our current approach is limited to purely visual tasks and does not yet support multimodal scenarios such as text-guided visual grounding or video synthesis. Nonetheless, the proposed enhancement strategy is readily adaptable to the visual backbones of multimodal models. Even without optimizing textual features, our method still achieves strong performance when combined with Grounding DINO, as shown in Table 3. We plan to explore in future work.\n\n\n**Table 3: Comparison results of original Grounding DINO and our method under 10-shot setting.**\n\n| Method                  | ArTaxOr | Clipart1k |  DIOR  | DeepFish | NEU-DET |  UODD  | Avg|\n|:----------------------|:-------:|:---------:|:------:|:--------:|:-------:|:------:|:------:|\n| Grounding DINO (Swin-B)   |  72.0   |    58.4   |  38.2   |   35.2    |   26.5   |  28.6   |43.2|\n| Grounding DINO (Swin-B) + Ours   |  73.8   |    60.3    |  40.2   |   37.0   |   27.9   |  29.2   |44.7|"}, {"review_id": "uWuC3RnlRg", "reviewer": "Reviewer_cjXu", "summary": "The authors presented a novel approach for Cross-Domain Few-Shot Object Detection (CD-FSOD) that effectively merges the localization abilities of the well-trained object detector with the strong generalized features learned by Vision Foundation Models (VFMs) to improve the detection performance. This is achieved by the Mixture-of-Experts (MoE) module that dynamically routes and modulates the integration of VFM features into the detector training. The MoE module includes two key components: 1)  a routing mechanism that assigns weights to VFM feature maps for each stage of the detector backbone, and 2) an expert projection (EP) module for aligning VFM features to those of detector features. The routing mechanism creates expert-wise weights and region-wise weights. The expert projection includes Shared and Private projections, which are concatenated to create VFM features aligned with backbone features. The aligned features are then processed by MoE with the help of routing weights. The method proposed in this paper can straightaway extend the capacity to detect novel classes without requiring re-training on the base data.", "questions": "1. In L90, the authors state, \"DETR and its variants struggle on CD-FSOD tasks, primarily due to poor classification accuracy\". Is there a reference to any papers or empirical results to justify this? \n2. Please provide results on other backbones for the results in Table 1, especially with VIT-L/14, as most of the methods listed are using that backbone.\n3. Why does the CLIP backbone give limited improvement compared to the DINO v2?\n\nMinor\n1. Repeated text about DINO DETR comparison in L239-243.\n2. dAP value mismatch between Table 4 and the corresponding text in L248.", "limitations": "The limitations are not discussed in the paper. However, the authors state that the paper has limitations. Please include them in the supplementary materials.\n\nOne obvious limitation is that when the number of layers of the detector and VFM increases, the routing computations increase quadratically.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths\n1. The paper presents a conceptually simple solution for CD-FSOD by integrating the strong generalizable features of VFMs with the localization capacity of trained detectors to facilitate novel object detection with few-shot training. \n2. Different from common FSOD methods, the technique proposed here doesn't need the detectors to be trained on the base classes. The authors used a pre-trained detector on the COCO dataset to adapt to a diverse set of new domains. \n3. The paper is well written and easy to follow. \n4. This method proposed is general purpose; any VFMs can be integrated into the standard detectors to perform FSOD using the proposed approach.  \n\n\nWeaknesses\n1. It would be great to see results on other detection models, though it is not a significant weakness. A recent paper [1] with the GroundingDINO detector has good detection performance on the same problem settings, but not straightforward to compare with the detector and backbone used in this study.\n2. It is not clear what is captured in the shared information and private information projection from the VFM features. The authors stated that the shared projection enables the aggregation of shared image information across different expert features. And the private projection module allows the model to learn tailored projection strategies for each expert feature. A good visualization is missing to understand this with more clarity.\n\n\n[1] Enhance Then Search: An Augmentation-Search Strategy with Foundation Models for Cross-Domain Few-Shot Object Detection, CVPRw 2025.", "was_revised": true, "final_justification": "Overall, this paper was a strong submission from the beginning. My concerns were minor, and the authors have addressed all of them. So I maintain my rating to accept this paper.", "rebuttal": "We sincerely thank Reviewer cjXu for the constructive feedback, insightful questions, and valuable suggestions!\n\nBelow we address your concerns.\n\n >**Q1: Comparison to ETS.**\n\n**R1**: Thank you for bringing ETS method [1] to our attention. \n\nWe compare our method with ETS on six cross domain datasets. For fairness, we reproduce the experiments using the official ETS code and also report the results from the original paper. As shown in Table 1, our method can be directly applied to Grounding DINO without any modification and achieves similar performance to ETS, even though it is not specifically designed for Grounding DINO. In future work, we will explore designs better tailored to Grounding DINO. We will include these experimental results in the revised main paper.\n\n**Table 1: Comparison Results of ETS method and our method under 10-shot setting.**\n\n| Method                  | ArTaxOr | Clipart1k |  DIOR  | DeepFish | NEU-DET |  UODD  | Avg|\n|:----------------------|:-------:|:---------:|:------:|:--------:|:-------:|:------:|:------:|\n| ETS (Paper report)   |  71.2   |    61.5    |  37.5   |   44.1    |   26.1   |  29.8   |45.0|\n| ETS (Our implementation)   |  70.6   |    60.8    |  37.5   |   42.8    |   26.1   |  28.3   |44.4|\n| Grounding DINO (Swin-B) + Ours   |  73.8   |    60.3    |  40.2   |   37.0   |   27.9   |  29.2   |44.7|\n\nReferences:\n\n[1] Enhance Then Search: An Augmentation-Search Strategy with Foundation Models for Cross-Domain Few-Shot Object Detection, CVPRw 2025.\n\n>**Q2: Analysis of SEP and PEP modules.**\n\n**R2**: Thanks for your valuable question. \n\nThe SEP module captures general saliency features that are common across all expert features. However, these feature maps lack the precise boundary details present in individual expert features. In contrast, our PEP module is tailored to extract those boundary cues unique to each expert, effectively filling in the missing boundary information. We validate our conclusion through PCA‑based feature visualizations. We will add visualization results in the revised main paper.\n\n>**Q3: Analysis of DETR’s classification capabilities.**\n\n\n**R3**: Thank you for your constructive question. \n\nWe have demonstrated DETR’s relatively weak classification performance in Table 4 of main paper. We use the AP drop(dAP) caused by false‑positive samples to reflect the model’s classification capability. We show the experimental results in Table 2 below for easier checking. We identify notable weaknesses in the classification performance of DINO DETR.Specifically,the dAP values reach 44.45/40.12/35.68 under the 1/5/10-shot settings. This is consistent with our paper’s conclusion that DETR has relatively weak classification capability.\n\n**Table 2: Comparison results of classification capability.**\n\n| Method  | 1-shot FP loss ↓  | 5-shot FP loss ↓ | 10-shot FP loss ↓ |\n| :------------|:---------------:| :-----:|:-----:|\n| DINO DETR   | 44.45 | 40.12 | 35.68 |\n| DINOv2      | 29.73 |   27.70 | 30.26 |\n| Ours | 26.47       |   22.54 | 17.98 |\n\n>**Q4: Comparison results on different detector backbones.**\n\n**R4**: Thank you for your valuable suggestions. \n\nWe conduct additional comparison experiments using differenet backbone architectures. A shown in Table 3, Applying our method to stronger backbones achieves better results than  result we report in the main paper. For example, integrating our method into DINO with a Swin‑B backbone improves the average performance by 2.4 AP, while using a ViT‑L backbone achieves a gain of over 4.6 AP. We will include these comparison experiments in the revised main paper.\n\n**Table 3: Comparison results of different backbone.**\n\n| Model                  | ArTaxOr | Clipart1k |  DIOR  | DeepFish | NEU-DET |  UODD  |Avg|\n|:----------------------|:-------:|:---------:|:------:|:--------:|:-------:|:------:|:------:|\n| DINO DETR(ResNet50)    |  71.3   |    49.9    |  37.8   |   34.1    |   23.7   |  22.1   |39.8|\n| DINO DETR(Swin-B)      |  75.4   |   56.7    |  39.5   |  35.1    |   23.2   |  23.1   |42.2|\n| DINO DETR(ViT-L)    |  75.8   |   60.3    |  42.0   |  37.2    |   25.1   | 25.9   |44.4|\n\n>**Q5: Analysis of different foundation models.**\n\n**R5:** Thank you for your valuable question. \n\nCLIP [4] uses large‑scale image text contrastive learning to capture high‑level semantics, excelling at zero‑shot classification and retrieval. However, it lacks the spatial granularity needed for region‑level tasks [1], making it less suitable for dense predictions such as detection and segmentation. In contrast, DINOv2 [5] uses self‑distillation with patch‑level reconstruction to learn fine‑grained, spatially coherent features. This leads to superior dense‑vision performance, as confirmed by studies [2,3] on visual grounding and segmentation. \n\nReferences:\n\n[1] Refining CLIP’s Spatial Awareness: A Visual‑Centric Perspective\n\n[2] From CLIP to DINO: Visual Encoders Shout in Multi‑modal LLMs\n\n[3] Learning Vision from Models Rivals Learning Vision from Data\n\n[4] Learning Transferable Visual Models From Natural Language Supervision (CLIP)\n\n[5] DINO v2: Learning Robust Visual Features Without Supervision\n\n>**Q6: Analysis of method limitations.**\n\nR6: Thank you for your valuable comment about our method limitation. \n\nThe increased computational cost caused by the larger number of detector layers and expert features is indeed a limitation of our method. This limitation can be easily mitigated. As the number of backbone layers increases, we can sparsely select a subset of layers for processing, which has little impact on performance. Moreover, the routing mechanism introduces only minor computational cost and has minimal effect on inference speed. Another limitation of our approach is that it is currently designed only for visual tasks and does not yet handle multimodal scenarios, such as text‑guided visual grounding or video synthesis. However, our enhancement strategy can be easily applied to the visual backbones of multimodal models. As shown in Table 4, our method achieves strong performance when combined with Grounding DINO, even without explicit optimization for textual features. We will further explore these directions in future work.\n\n\n**Table 4: Comparison results of original Grounding DINO and our method under 10-shot setting.**\n\n| Method                  | ArTaxOr | Clipart1k |  DIOR  | DeepFish | NEU-DET |  UODD  | Avg|\n|:----------------------|:-------:|:---------:|:------:|:--------:|:-------:|:------:|:------:|\n| Grounding DINO(Swin-B)   |  72.0   |    58.4   |  38.2   |   35.2    |   26.5   |  28.6   |43.2|\n| Grounding DINO(Swin-B) + Ours   |  73.8   |    60.3    |  40.2   |   37.0   |   27.9   |  29.2   |44.7|"}, {"review_id": "M0NYSmVZED", "reviewer": "Reviewer_Xh4w", "summary": "This paper introduces a cross-domain few-shot object detection method that does not require retraining on the source domain. It directly employs a mixture-of-experts (MoE) architecture, where a visual foundation model (DINOv2) provides expert features, and an object detector (DINO) serves as the router network, complementing each other in terms of generalization and localization capabilities. The router network includes expert-wise and region-wise modules that dynamically and iteratively aggregate coarse-grained and fine-grained visual information from the VFM into the object detector's backbone features at each stage. Expert features are projected from the VFM feature maps through shared and private projection modules, effectively modeling both common and expert-specific knowledge. Experiments show that the proposed method outperforms prior approaches on the CD-FSOD benchmark, demonstrates strong transferability across different object detection backbones, and validates the effectiveness of the introduced router and projection modules through ablation studies.", "questions": "Aside from the questions raised in the above section, I'd like to gain a deeper understanding of the intuition behind this architectural design. Fundamentally, why can an OD that's sensitive to domain shift reliably route VFM features? I know the authors show the effectiveness of the method through experiments, but theoretically, using the OD features to route VFM features seems to be counterintuitive and ill-posed. Would like to see further clarification/elaboration on this design in the paper.\n\nAlso, it would be great to hear authors' thoughst on why for DINOv2, dAP increase to 30.26 from 27.7 , for 5-shot to 10-shot according to the Table 4.", "limitations": "The authors didn't explicitly list the limitations; it would be helpful for future work in this direction if they could share any limitations they're aware of.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths: \n1) The motivation is clear: VFMs pretrained on large-scale data exhibit better generalization but lack accurate localization capabilities, while object detectors (OD) specialize in bounding box regression but are less robust to domain shifts. \n2) The design is straightforward yet effective, leveraging pretrained models without the need for retraining on base classes. This brings practical value to real-world scenarios, where downstream applications demand quick and performant adaptation. \n3) The method exhibits significant performance advantages over existing approaches on the CD-FSOD benchmark. It also showcases transferability across a variety of DETR architectures.\n\n\nWeaknesses:\n1. Clarity of ablation studies: It is unclear what it means for SEP and PEP to exist without ER and RR. How are the VFM features aggregated in this case? Is there still a gating network, or are the features aggregated in a way that is not conditioned on the backbone features?\n\n2. Comprehensiveness of ablation studies: What is the rationale for not enumerating all possible combinations? Since the projection methods and router mechanisms are orthogonal, there should be 3 (SEP only, PEP only, SEP+PEP) × 4 (ER + RR, ER only, RR only, neither) + 1 (DINO only) = 13 combinations.\n\n3. Lack of simpler baselines: Additional baselines are needed to show the necessity of the MoE structure. Table 3 shows a significant performance boost from DINO to SEP only, suggesting that much of the improvement may stem from the rich visual features provided by DINOv2. Did the authors explore simpler baselines, such as aggregating VFM features using a simple MLP (without conditioning on backbone features), or combining VFM features only at the final stage with the backbone features to justify the iterative design?\n\n4. Notations to be improved/typos to be corrected:\n\n    a) At the start of section 3.4, n seems to be overloaded, can be number of experts, also said to be a hyperpamter in expert   projection module; \n\n    b) F_D^l’, prime looks like indicate another layer l’, which I believe the intention here is still to indicate layer l;\n\n    c) Section 4.1, text description is not consistent with the table. Shouldn’t it be 24.6 mAP improvement on 10-shot for the     proposed method vs  DINO. Shouldn’t it be 9.8 improvement on 10-shot for the proposed method vs CD-ViTO\n   \n5. You may want to cite some more recent work:\n\n     [1] Pan, Jiancheng, et al. \"Enhance then search: An augmentation-search strategy with foundation models for cross-domain few-shot object detection.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n\n     [2]Meng, Boyuan, et al. \"CDFormer: Cross-Domain Few-Shot Object Detection Transformer Against Feature Confusion.\" arXiv preprint arXiv:2505.00938 (2025).\n\n     [3] Huang, Yali, et al. \"Instance Feature Caching for Cross-Domain Few-Shot Object Detection.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.", "was_revised": true, "final_justification": "The authors ran additional experiments showing the effectiveness of all modules introduced, further analyzed the experiments, and explained the rationale behind the design. My latest comment is intended to encourage the authors to further improve the completeness of the paper. Overall, this work is technically solid and worth more attention in this field, so I would increase the score to 5.", "rebuttal": "We sincerely thank Reviewer Xh4w for the constructive feedback, insightful questions, and valuable suggestions!\n\nBelow we address your concerns.\n\n>**Q1: Clarification of VFM feature aggregation without ER and RR.**\n\n**R1**: Thank you for your insightful comment. \n\nWhen the ER and RR modules are removed, we assign a unit weight to each VFM feature map. We then directly apply the SEP and PEP modules to all VFM features maps without backbone‑conditioned guidance or additional gating network.\n\n\n>**Q2: Additional ablation studies.**\n\n**R2**:   Thank you for your careful attention to our ablation studies. \n\nWe conduct comprehensive ablation studies, as shown in Table 1. We will include comprehensive ablation studies in the revised main paper.\n\n**Table 1: Results of ablation studies. 'SEP' denotes shared expert projection，'PEP' denotes the private expert projection，'ER' denotes the expert-wise router，'RR' denotes the region-wise router.**\n\n|  SEP  |  PEP  |   ER   |  RR   | ArTaxOr | Clipart1k |  DIOR   | DeepFish | NEU-DET |  UODD  | Avg |\n|:-----:|:-----:|:------:|:-----:|:-------:|:---------:|:-------:|:--------:|:-------:|:------:|:------:|\n|   ×   |   ×   |   ×    |   ×   |  11.4   |   23.2    |  14.4   |  20.5    |  11.8   |  9.9   | 15.2 |\n|   √   |   ×   |   ×    |   ×   |  62.1   |   43.8    |  34.5   |  26.4    |  21.2   | 19.3   | 34.6 |\n|   ×   |   √   |   ×    |   ×   |  63.2   |   44.1    |  35.8   |  26.0    |  23.0   | 21.5   | 35.6 |\n|   √   |   √   |   ×    |   ×   |  65.1   |   44.6    |  34.8   |  27.4    |  22.0   | 20.3   | 35.7 |\n|   √   |   ×   |   √    |   ×   |  63.1   |   45.2    |  36.1   |  29.5    |  23.3   | 17.3   | 35.8 |\n|   √   |   ×   |   ×    |   √   |  69.0   |   47.2    |  36.1   |  32.2    |  20.9   | 21.8   | 37.9 |\n|   ×   |   √   |   √    |   ×   |  66.2   |   45.3    |  35.2   |  30.1    |  23.2   | 22.0   | 37.0 |\n|   ×   |   √   |   ×    |   √   |  67.1   |   46.7    |  37.5   |  28.5    |  23.2   | 22.5   | 37.6 |\n|   √   |   √   |   √    |   ×   |  68.8   |   46.8    |  36.3   |  27.8    |  21.8   | 21.5   | 37.2 |\n|   √   |   √   |   ×    |   √   |  68.7   |   49.3    |  35.2   |  31.4    |  22.2   | 22.1   | 38.2 |\n|   √   |   ×   |   √    |   √   |  70.3   |   49.1    |  35.8   |  32.5    |  22.3   | 22.7   | 38.8 |\n|   ×   |   √   |   √    |   √   |  70.9  |   49.5    |  37.1   |  32.9    |  23.1   | 22.0   | 39.3|\n|   √   |   √   |   √    |   √   |  71.3   |   49.9    |  37.8   |  34.1    |  23.7   | 22.1   | 39.8 |\n\n>**Q3: Evaluation of simpler feature aggregation strategy.**\n\n**R3**: Thank you for your valuable suggestions. \n\nWe compare our method to three simpler feature aggregation strategies. Specifically, we designed and evaluated three variants:\n\n+ All Stages: Retain all stage PEP modules and replace them with simple MLPs, removing all other modules.\n\n+ Final Stage: Retain only the final-stage PEP module and replace it with simple MLP, removing all other modules.\n\n+ Linearly Adding: Remove all modules and aggregate features at the backbone stage via linear addition, \n\n As shown in Table 2, simple feature aggregation strategies lead to a substantial performance decline, thereby validating the efficacy of our approach. Moreover, aggregating features only at the final backbone stage performs worse than aggregating at every stage, highlighting the importance of our iterative design. We will include these comparison experiments in the revised supplementary material.\n\n**Table 2: Comparison Results of our method and simple feature aggregation strategies.**\n\n| Method                  | ArTaxOr | Clipart1k |  DIOR  | DeepFish | NEU-DET |  UODD  | Avg |\n|:----------------------|:-------:|:---------:|:------:|:--------:|:-------:|:------:|:------:|\n| All Stages    |  56.3  |   44.4      |  34.1   |   26.4    |   16.8   |  17.6   |32.6|\n| Final Stage      |   48.8    |  43.8   |  30.7   |  24.1    |   17.1   |  17.4   |30.3|\n| Linearly Adding    |  2.5   |   5.1    |  9.1   |  4.7   |   3.9   | 5.4   |5.1|\n| Ours   |  71.3   |   49.9    |  37.8   |  34.1    |  23.7   | 22.1   |39.8|\n\n>**Q4: About recent works.**\n\n**R4:** Thank you for pointing us to [1] ,[2] and [3]. \n\nWe acknowledge their relevance to our work and will cite these papers in the revised main paper.\n\n\n```\nETS[1] proposes a two-step augmentation and grid-search strategy to adapt foundation models for cross-domain few-shot object detection. CDFormer[2] introduces transformer-based OBD and OOD modules to mitigate object-background and inter-class feature confusion in cross-domain few-shot detection. IFC[3] enhances cross-domain few-shot detection by using learnable instance feature caches with an instance reweighting module for robust prototypes.\n```\n\nReferences:\n\n[1] Pan, Jiancheng, et al. \"Enhance then search: An augmentation-search strategy with foundation models for cross-domain few-shot object detection.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n\n[2]Meng, Boyuan, et al. \"CDFormer: Cross-Domain Few-Shot Object Detection Transformer Against Feature Confusion.\" arXiv preprint arXiv:2505.00938 (2025).\n\n[3] Huang, Yali, et al. \"Instance Feature Caching for Cross-Domain Few-Shot Object Detection.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n\n>**Q5: About notations and typos.**\n\n**R5:** Thank you for your careful attention to our paper.\n\nWe have carefully revised all the writing issues you pointed out and re‑checked the entire paper to ensure that similar mistakes will not occur again.\n\n\n>**Q6: Analysis of routing network based on the OD feature.**\n\n**R6**: Thank you for your insightful question. \n\nWe empirically observe that detectors possess strong localization and regression capabilities, accurately identifying object regions. However, due to domain shift, object features within these regions are often incomplete. This aligns with our paper’s conclusion that detectors have strong localization but weak classification. Therefore, we leverage the detector’s localization ability to guide VFM feature maps in supplementing the missing key features in salient regions so that the more generalizable VFM features can compensate for the information loss caused by domain shift. We will include the corresponding visualizations and discussions in the revised supplementary material.\n\n>**Q7: Analysis of abnormal variations in DINOv2’s classification capability.**\n\n**R7**: Thank you for your constructive feedback. \n\nWe argue that the increased sample size leads the model to prioritize localization regression at the expense of its classification accuracy under the 10‑shot task setting. As shown in table 3, We observe that although DINOv2 exhibits strong classification performance(dAP: 27.7) in the 5‑shot setting, its average performance(mAP: 23.3) is low. In contrast, despite weaker classification performance(dAP: 30.3), the average performance(mAP: 27.4) increases substantially in the 10‑shot setting, supporting our conclusion. \n\n**Table 3: Comparison Results of DINOv2 under 5/10 shot task settings.**\n\n| Model                  | ArTaxOr | Clipart1k |  DIOR  | DeepFish | NEU-DET |  UODD  | Avg |\n|:----------------------|:-------:|:---------:|:------:|:--------:|:-------:|:------:|:------:|\n| DINOv2(5-shot)   |  48.3  |   29.5      |  22.4   |   19.7    |   10.5   |  9.3   |23.3|\n| DINOv2(10-shot)    |   51.9    |  34.2   |  25.6   |  24.1    |   15.2   |  13.5   |27.4|\n\n\n\n\n>**Q8: Analysis of method limitations.**\n\n**R8:** Thank you for your constructive suggestion.\n\nOne major limitation of our approach is that it currently focuses solely on visual tasks and has not been extended to multimodal applications such as text‑guided visual grounding or video synthesis. Nevertheless, the proposed enhancement strategy can be easily applied to the visual backbones of multimodal models. As demonstrated in Table 4, our method still performs strongly when combined with Grounding‑DINO, even without explicit optimization for textual features. We intend to further explore this direction in future work.\n\n**Table 4: Comparison results of original Grounding DINO and our method under 10-shot setting.**\n\n| Method                  | ArTaxOr | Clipart1k |  DIOR  | DeepFish | NEU-DET |  UODD  | Avg|\n|:----------------------|:-------:|:---------:|:------:|:--------:|:-------:|:------:|:------:|\n| Grounding DINO(Swin-B)   |  72.0   |    58.4   |  38.2   |   35.2    |   26.5   |  28.6   |43.2|\n| Grounding DINO(Swin-B) + Ours   |  73.8   |    60.3    |  40.2   |   37.0   |   27.9   |  29.2   |44.7|"}, {"review_id": "IzFAOtNoEw", "reviewer": "Reviewer_KWMR", "summary": "The paper proposes incorporating the general representation capabilities of pre-trained vision foundation models as expert knowledge into object detectors, aiming to enhance their detection performance in unseen domains. Specifically, the output features from each layer of the VFM are treated as individual experts. A Mixture-of-Experts-based approach is introduced to dynamically route and modulate the VFM features. Experiments demonstrate that the proposed method achieves performance improvements in CD-FSOD tasks and can be used as a plug-in module for various detectors.", "questions": "Please answer the following questions:\n1. Addressing the ambiguities noted in Weakness 1.\n2. Please compare with Frozen-DETR on the CD-FSOD datasets.\n3. Is the proposed method effective on in-domain datasets with sufficient samples, such as COCO? Please supplement the experiments of the proposed method on the COCO dataset and compare the results with Frozen-DETR.\n\nSince the motivation of this paper is similar to that of Frozen-DETR, I am very concerned about the comparison and discussion between the proposed method and Frozen-DETR. This will affect my decision on adjusting the score.\n\nIn addition, I have the following suggestion:\nIn Table 1, † indicates that the methods were fine-tuned on the six datasets. In fact, Distill-cdfsod, CD-ViTO, and the proposed method (Ours) all require fine-tuning. The authors are advised to revise this description to avoid misunderstandings.", "limitations": "Limitations are not mentioned.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. The motivation is clear and meaningful.\n2. The proposed method is simple and effective, and it can be extended to various DETR-style detectors.\n3. The paper provides comprehensive analytical experiments.\n\nWeaknesses:\n1. The paper contains some unclear parts. For example, (1) The feature shapes of DINOv2 differ from those of the backbone (ResNet50) in DINO at various stages, yet the paper uniformly represents them using H×W, which may cause confusion. (2) According to Fig. 5, does the proposed method use all 24 output features from DINOv2-L as expert features but perform feature fusion only at the output features of the four stages of ResNet50? (3) What does DINOv2 refer to in Tables 4 and 5, and how does it work?\n2. There is a lack of comparison with related work. The motivation of the paper is similar to that of Frozen-DETR [1], which also transfers the general representations extracted by the frozen VFM to object detectors. It is recommended that the authors include a comparison with Frozen-DETR.\n3. The method proposed in this paper has been evaluated solely on the CD-FSOD datasets. Its effectiveness on general object detection tasks has yet to be demonstrated.\n\n[1] Fu S, Yan J, Yang Q, et al. Frozen-DETR: Enhancing DETR with Image Understanding from Frozen Foundation Models[J]. arXiv preprint arXiv:2410.19635, 2024.", "was_revised": true, "final_justification": "The authors have addressed all my concerns. In particular, the comparison with Frozen DETR demonstrates the advantage of using the proposed MoE-based fusion of VFM features over direct fusion. The results on the COCO dataset show that the proposed method not only improves performance in cross-domain few-shot tasks but also enhances general object detection.\n\nOverall, I have decided to raise my final rating to 5.", "rebuttal": "We sincerely thank Reviewer KWMR for the constructive feedback, insightful questions, and valuable suggestions! \n\nBelow we address your concerns.\n\n>**Q1: About some unclear parts in the paper.**\n\n**R1**: Thank you for your valuable questions. \n\nBelow is our detailed response to the three issues you raised in Question 1:\n\n+ R1.1: In DINOv2, the spatial dimensions of the feature maps differ from those of the ResNet backbone’s feature maps. In the revised main paper., we will use (H, W) to denote the spatial dimensions of backbone features and (Hd, Wd) to denote the spatial dimensions of DINOv2 features.\nWe will revise this phrasing in the revised main paper.\n\n+ R1.2: Our feature fusion strategy follows the illustration: We first process the output of the previous stage with the backbone layer. The processed features are then fused with the VFM features. Finally, the fused features are used as the input for the next layer.\n\n\n+ R1.3: In Tables 4 and 5, “DINO v2” refers to replacing the ResNet50 backbone of original DINO DETR  with DINO v2, while keeping the detection head and training settings unchanged. This ensures a fair experimental comparison.\n\n>**Q2: Comparison with Frozen DETR on cross‑domain datasets.**\n\n**R2**: Thank you for bringing Frozen DETR [1] to our attention. \n\nWe compare our method with Frozen DETR. We adopt the same baseline structure as Frozen DETR. Frozen DETR follows a two‑stage training paradigm of base training and finetuning. To align the experimental and dataset settings, we skip the base training step, finetuning only on the cross domain datasets. As shown in Table 1, our method consistently outperforms Frozen DETR across six crossdomain datasets. We will include these comparison experiments in the revised main paper.\n\n\n**Table 1: Comparison Results Against Frozen DETR under 10-shot setting.**\n\n| Method                | ArTaxOr | Clipart1k |  DIOR  | DeepFish | NEU-DET |  UODD  |Avg|\n|:----------------------|:-------:|:---------:|:------:|:--------:|:-------:|:------:|:------:|\n| Frozen DETR   |  45.8   |    33.9    |  31.5   |   17.5    |   9.1   |  3.2   | 23.5|\n| Ours     |  71.3   |   49.9    |  37.8   |  34.1    |   23.7   |  22.1   |39.8|\n\n**Discussion between our method and Frozen DETR.**\n\n+  Frozen DETR modifies the original DINO DETR architecture and introduces many additional parameters, thereby requiring base training followed by fine-tuning on novel classes to achieve strong performance. In contrast, our approach achieves best performance by fine-tuning only on novel classes, without any base training.\n+ Frozen DETR enhances detector features using only the final‑layer output of the VFM. Because it relies on self‑attention for feature enhancement, it incurs significant computational overhead. In contrast, our method employs the SEP and PEP modules, avoiding the \\(O(n^2)\\) complexity, which allows us to fuse multi‑layer VFM features efficiently.\n\nReferences:\n\n[1] Fu S, Yan J, Yang Q, et al. Frozen-DETR: Enhancing DETR with Image Understanding from Frozen Foundation Models[J]. arXiv preprint arXiv:2410.19635, 2024.\n\n>**Q3: Comparison with FrozenDETR on the COCO dataset.**\n\n**R3**: Thank you for your constructive suggestion. \n\nWe train and evaluate our method on the COCO dataset and compared it to the performance of Frozen DETR reported in the original paper. As shown in Table 2, our method outperforms Frozen DETR on COCO, further validating its general applicability. We will include these comparison experiments in the revised supplementary material.\n\n\n**Table 2: Comparison Results Against Frozen DETR on the COCO dataset.**\n\n| Method                | Backbone | Epochs |  AP  |\n|:----------------------|:-------:|:---------:|:------:|\n| DINO DETR              |  R50   |    12    |  49.0   |\n| Frozen DETR  |  R50   |    12    |  51.9   |\n|  DINO DETR + Ours  |  R50   |    12    |  55.2   |\n\n\n>**Q4: Analysis of method limitations.**\n\n**R4**: Thanks for your insightful comment.\n\nOne limitation of our approach is that it is currently confined to visual tasks and does not yet support multimodal applications such as text‑guided visual grounding or video synthesis. However, our enhancement strategy can be easily applied to the visual backbones of multimodal models. As shown in Table 3, the method achieves strong performance when combined with Grounding DINO, even without explicit optimization for textual features. Extending the approach to multimodal scenarios will be explored in future work.\n\n**Table 3: Comparison results of original Grounding DINO and our method under 10-shot setting.**\n\n| Method                  | ArTaxOr | Clipart1k |  DIOR  | DeepFish | NEU-DET |  UODD  | Avg|\n|:----------------------|:-------:|:---------:|:------:|:--------:|:-------:|:------:|:------:|\n| Grounding DINO (Swin-B)   |  72.0   |    58.4   |  38.2   |   35.2    |   26.5   |  28.6   |43.2|\n| Grounding DINO (Swin-B) + Ours   |  73.8   |    60.3    |  40.2   |   37.0   |   27.9   |  29.2   |44.7|\n\n>**Q5: About the symbol used in Table 1.**\n\n**R5:** Thank you for your constructive suggestion.\n\nDistill‑CDFSOD, CD‑ViTO, and our method all need to finetune on novel classes. We will revise this phrasing in the revised main paper."}]

Author rebuttal:

Meta-review: Dear authors, 

This draft has received all positive reviews. We strongly request authors to incorporate in the draft all the suggestions and required parts of discussion, that led to this decision. 
For example,  (ZRE6) "computational efficiency and comparisons with MLLMs and openvocab models.", results appreciated by cjXu, (Xh4w) additional experiments that showed "the effectiveness of all modules introduced, further analyzed the experiments, and explained the rationale behind the design", (KWMR) comparison with Frozen DETR etc.. 
Xh4w further requested to improve the completeness. 

Please provide clarify all the hyper-parameters, what they stand for, and what values were used in the experiments. 
In the abstract its written"eliminates the traditional two-stage paradigm of base training", however, in introduction its stated that"without retraining on the source domain".  Clarity will improve readability. 

We expect authors to provide essential and enough code that is necessary to reproduce their results, as we notice authors have promised "part of the code". Please note "The instructions should contain the exact command and environment needed to run to
reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.".


Congratulations 

Regards

AC
Decision: Accept (poster)