Test-Time Adaptive Object Detection with Foundation Model
Authors: Yingjie Gao
Yanan Zhang
Zhi Cai
Di Huang
Venue: NeurIPS 2025 poster ()
Keywords: object detection
test-time adaptation
foundation model
prompt tuning
Primary area: applications

Abstract: In recent years, test-time adaptive object detection has attracted increasing attention due to its unique advantages in online domain adaptation, which aligns more closely with real-world application scenarios. However, existing approaches heavily rely on source-derived statistical characteristics while making the strong assumption that the source and target domains share an identical category space. In this paper, we propose the first foundation model-powered test-time adaptive object detection method that eliminates the need for source data entirely and overcomes traditional closed-set limitations. Specifically, we design a Multi-modal Prompt-based Mean-Teacher framework for vision-language detector-driven test-time adaptation, which incorporates text and visual prompt tuning to adapt both language and vision representation spaces on the test data in a parameter-efficient manner. Correspondingly, we propose a Test-time Warm-start strategy tailored for the visual prompts to effectively preserve the representation capability of the vision branch. Furthermore, to guarantee high-quality pseudo-labels in every test batch, we maintain an Instance Dynamic Memory (IDM) module that stores high-quality pseudo-labels from previous test samples, and propose two novel strategies-Memory Enhancement and Memory Hallucination-to leverage IDM's high-quality instances for enhancing original predictions and hallucinating images without available pseudo-labels, respectively. Extensive experiments on cross-corruption and cross-dataset benchmarks demonstrate that our method consistently outperforms previous state-of-the-art methods, and can adapt to arbitrary cross-domain and cross-category target data. Code is available at https://github.com/gaoyingjay/ttaod_foundation.

Adaptive Object Detection with Foundation Model
Yingjie Gao Yanan Zhang Zhi Cail.2 Di Hu
State Key Laboratory of Complex and Critical Software Environment, Beihang University School Computer Science and Engineering, Beihang University
School of Computer Science and Information Engincering, Hefei University of Technology catuuaa.edu.com {gaoyingjie, caizhi97, dhuang) thuang)@buaa, @buaa. edu. cn, yananzhang@hfut. edu. cn
Abstract
In recent years, test-time adaptive object detection has attracted increasing attention due its unique advantages in online domain adaptation, which aligns more closely with real-world application scenarios. However, existing approaches heavily rely source-derived statistical characteristics while making the strong assumption that the source and target domains share an identical category space. In this paper, we propose the first foundation model-powered test-time adaptive object detection method that eliminates the need for source data entirely and overcomes traditional closed-set limitations. Specifically, we design a Multi-modaa Prompt- beaupt--aseddd Mean- Teacher framework for vision- language detector-driven test-time adaptation, which incorporates text and visual prompt tuning adapt both language and vision repre- sentation spaces on the test data in parameter-efficient manner. Correspondingly, we propose a Test-time Warm-start strategy tailored for the visual prompts to effec- tively preserve the representation capability of the vision branch. Furthermore, to guarantee high-quality pseudo-labels in every test batch, we maintain an Instance Dynamic Memory (IDM) module that stores high-quality pseudo-labels from pre- vious test samples, and propose two novel strategies- Memory Enhancement and Memory Hallucination-to leverage IDM's high-quality instances for enhancing orig- inal predictions and hallucinating images without available pseudo-labels, respec- tively. Extensive experiments demonstrate that our method consistently outperforms previous state-of-the-as methods, and can adapt to arbitrary cross-domain and cross-category target data. Code available at fhttps: : //t/ github. com/ gaayyiggay/ttaaaagaa http:///tttoooofootiiiooi..o foundation
cross-corruption and cross dataset benchmarks
Introduction
has made significant progress, while its performance drops dramatically when facing domain gaps. Although Unsupervised Domain attempts to mitigate domain differences in an offline manner, still struggles meet the real-time domain adaptation requirements in application scenarios such as
As a fundamental task in visual perception, object detection 23 12
Adaptation (UDA) technology 3
34
autonomous driving (33] and robotics [1 Consequently, Test-Time Adaptation (TTA) 226 28 25
has emerged, which operates real-time by adapting on the fly during inference.
Existing Test- Time Adaptive Object Detection (TTAOD) methods 2 31 29), predominantly built upon Faster R-CNN [23], leverage self-training or nource-target feature alignment strategies to achieve promising domain adaptation performance. However, as shown in Fig. [T]. there are two major issues: (1) requiring statistical characteristics (e.g., the mean and variance of feature maps) derived from sampled source domain data, which violates the source-free principle of TTA, and
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Source Domain
Minsa
Aeroplane
Prisss
Detector
Bicycle
V1. Deleclor
Cat. Person Mouse...
Bird
Statistical Charactetsitls
M(G,,,,)
Lnit
Closed-set
Init
Open-set
Leorable Prompt
CP
A emoplane
Aecoplane Bicycle. Bird.
Detector
Bicycle
Delectod
Bind
Targel Domein
Target Domain
(a) Traditional TTAOD methods
(b) Ours
Figure 1: (a) Traditional TTAOD methods require source domain statistical characteristics and limited to closed-set during adaptation. (b) Our method requires no source data while possessing open-vocabulary capability.
(2) assuming identical category spaces between source and target domains, which limits TTAOD's applicability in open scenarios.
Recent vision-language foundation models (VLMs) 22 [7]. pre-trained on large-scale datasets,
have demonstrated remarkable zero-shot generalization and open- vocabulary capabilities, motivating our exploration of introducing vision-language detectors (e.g., GLIP [2 and Grounding DINO (17)
into the TTAOD task address the aforementioned issues, shown in Fig. [b). However, how to adapt vision sasion-an detectors during 77A non-trivial. On one hand, full-parameter fine-tuning test data not only diminishes the pre- pre-trained detector's generalization capability
via self-learning but also amplifies sensitivity to noisy samples, exacerbating overfitting when target data is scarce. On the other hand, effective adaptation hinges high-quality pseudo-labels generated from target domain test data, yet consistently obtaining reliable pseudo-labels in every batch remains challenging even with advanced vision- Laiiuge detectors.
For the first point, a straightforward approach to perform parameter-efficient fin-ttnnig on the foundation model using text prompts, but empirical findings show that tuning only the text prompts is inadequate for effective adaptation. Therefore, we design a Multi-modal Prompt-baa based Reacher framework for vision- vision-language detectors to perform self self-training during TTA, which incorporates text and visual prompt tuning to jointly adapt both language and vision representation spaces on the test data. Correspondingly, to mitigate potential performance degradation of the teacher model due to suboptimal visual prompt initialization, we introduce Test-time Warm-start strategy that initializes the visual prompts by average pooling image tokens extracted from the first test sample.
For the second point, we first maintain an Instance Dynamic Memory (IDM) module for each category during test time to help preserve valuable knowledge acquired from prior test samples. Building upon IDM, we then propose two novel strategies: Memory Enhancement and Memory Hallucination. Memory Enhancement leverages high-quality instances stored in IDM to refine the original predictions of the current test image, while Memory Hallucination integrates instances sampled from IDM into test images that have no available pseudo-labels.
The main contributions this paper are summarized fourfold:
To the best of our knowledge, this is the first foundation model-powered test-time adaptive object detector, which eliminates the need for source data entirely and overcomes traditional closed-set limitations.
We design a Multi-modal Prompt-hased Mean-Teacher framework for vision-language detector- driven TTA, which incorporates text-visual prompts and Test-time Warm-start strategy to achieve effective parameter-efficient fine-tuning while preserving the pre-trained knowledge.
We introduce Instance Dynamic Memory module and propose two novel strategies-Memon Enhancement and Memory Hallucination effectively leverage high-quality pseudo-labels from previous test samples.

---

Extensive experiments on both cross- corsupoonn and cross-datasee benchmarks demonstrate that our proposed method outperforms the satte-of-the-aase approaches by large margins and can adapt to arbitrary cross-domain and cross-category target data.
2
Related Work
2.1 Test-Time Adaptive Object Detection
TTAOD extends TTA 7 26, 28
to the object detection task, aiming to adapt a detector pre- trained on a labeled source domain different unlabeled target domains an online manner. Early works [3] [7] employ self- training paradigm and perform multi-epoch offline adaptation on target domain data. STFAR ( utilizes self-training to generate pseudo-labeled objects on the fly and incorporates feature distribution alignment as regularization. CTTAOD [31] focuses continually changing test domains by introducing an adapter-based based adaptation approach that activates only when necessary. The latest work, Efficient TTAOD [29], proposes pruning sensitive channels to focus adaptation efforts solely on invariant ones. However, these methods require access to source data for computing statistical characteristics (e.g., mean and variance), which is impractical in real-world scenarios. Moreover, they inherently assume identical category spaces between source and target domains, which significantly limits the applicability of TTAOD. In this paper, explore leveraging foundation models to enhance TTAOD and overcome the above limitations.
2.2 Vision-Language Object Detection
Vision-language foundation models are trained on large-scale image-text pairs collected from the web, which establish connections between visual and textual representations and achieve impressive zero-shot performance on various downstream tasks. Early
object detection works 6 35] distill knowledge from pre-trained vision-langue Lsiiun-aa classification models (e.g., CLIP [22]
to student detector (e.g.. Faster R-CNN (23),, enabling the detection of novel categories beyond the training set. GLIP 12] introduces a grounded language-image pre-training framework that generates grounding boxes in self-training paradigm. achieving strong zero-shot performance across various object detection datasets. Grounding DINO [17] incorporates grounded pre-training into the Transformer- seeaserrer-baeedd detector DINO [32] using tight cross- moossttddlltyy fusion solution and demonstrates superior generalization ability. In this paper, we investigate how to adapt vision-language detectors during test time and propose effective and efficient test-time adaptive object detection approach based on Grounding DINO.
Methodology
3.1 Preliminary
Standard TTAOD approaches typically assume access to detector pre-trained on source domain
data Ds = {(x,,, y:) ) where x; ~ Ps(x) and yı = (bboz,, cs) consists of a set of bounding
boxes bbox, and their corresponding class labels E C. The goal of TTAOD is to adapt the detector to different unlabeled target domains Dr = {ar}}/r N. during testing, where Tj ~ Pr(x)) and Ps(x) + (x)).. Crucially, the source domain is unavailable during adaptation, and the target domains share the same label space with the source domain (i.e., a closed-set scenario). When vision- languala detectors introduced into TTAOD, their large-scale pre- nee-training enables superior generalization on target domains. Moreover, dision-language foundation models break the closed-set constraint, allowing adaptation to arbitrary cross-domaie and cross cross-category target data.
3.2 Overall Architecture
An overview of our method illustrated in Fig. 2 We build our approach on the pre-trained 3.3 we introduce a Multi-modal Prompt- based
vision-language model Grounding DINO. In Sec Mean-Teacher framework that enables parameter-efficient self-training on test data. To mitigate potential performance degradation when incorporating visual prompts, we propose a Test-time Warm- start strategy specifically for prompt initialization. See 3.4 we present an Instance Dynamic Memory module to store high-quality pseudo-labels extracted from the test stream, coupled with
3

---

TPT
(b) Memoor Enhancement Ong nal Prediction
Class
EMA
aroplane, Ricyole. Bird.
Text
Enooder
DINOVZ
Enbancement Prediction
Text tokens
Memory Prodiction
Image tokons
Insiance Dynasiic M Memory
Aoopllane
Image Encoder
VL Eucodes
Decoder
Updatte Bicyde
WenkAug
Binel
Xp-1
Sampling
Peoudle label
Lass
Hallocinabed Positive Image
Nogaraie Image
(c) Memory Hallucination
Image
Eacoder
Teacher
Tencher
Strong
Premmtis
Pipline
Leamable
Student
Studon:
Multi-modal Prompt-hased Mean-tracher
Prompts
Pipline
Prozum
Figure 2: Overview our method. It comprises two components: (1) the Multi-modie Prompt- based Mean- Teacher framework shown in (a), incorporating text prompt tuning ggreen-highlighted) and visual prompt tuning (blue-highlighted) with a Test-time Warm-start strategy; and (2) an Instance Dy- namic Memory module that stores high-quality pseudo-labels from previous test samples, integrating with Memory Enhancement (b) and Memory Hallucination (c).
Mean-Teacher
two novel strategies-Memrry Enhancement and Memory Hallucination refine the original predictions for test images and hallucinate positive samples for images lacking reliable pseudo-labes respectively.
3.3 Multi-modal Trompt-based Mean-Teacher
Although vision-language pre-trained detectors like GLIP and Grounding [IN demonstrate impressive zero-shot generalization, their performance often degrades under real-world distribution shifts. Fine-tuning vision-language detectors target domains during test time is therefore essential. To achieve effective and efficient adaptation while preserving valuable pre-trained knowledge, we propose Multi-modal Prompt-based Mean-Teachis framework, which primarily comprises three core components: Text Prompt Tuning, Visual Prompt Tuning, and Test-time time Warm-start.
Text Prompt Tuning, Text prompt tuning is of the most prominent approaches in parameter- efficient fine-tuning for VLMs 36, [2]. Accordingly, given a test label set C, Grounding DINO concatenates the class names in C' using dot symbols to form the text input (e.g., "aeroplane. bicycle. bird. ..."). The text encoder fr then maps the input description to a sequence of at most 256 tokens, Er. We further introduce learnable vector Pr the language branch, whose dimension matches that of Er. The modulated text tokens are computed as:
E, Er + Pr
(1)
This enriched representation Ey is subsequently fed into the vision-language feature enhancer fv
Visual Prompt Tuning. Observing that fine-tuning the text prompts alone fails to effectively adapt to test data. As shown in Fig. 3 b)) we further introduce m learnable tokens Pt, {P%, E IR k== at each image encoder layer L, in Grounding DINO, alongside the input image tokens EJ,,, where d; denotes the dimension L, Similar to [10, we concatenate the visual prompts P7 with the image tokens as the input of each Transformer layer:
Eral Li (IP[1,,0 Er,0))
(2)

---

Image Encoder
Image Encoder
Janndaama lager
Theeeforee waneerlgee leper
Phirh Pirkao
Trandforce Layar
X
Trandfomee
Tounsfomme
Mrgfiyy
poypbay
(a) Test- time Warm-start
(b) Visual Prompt Tunning
Figure 3: Different behaviors of visual prompts. (a) Init visual prompts by average-posn poolingg image tokens from the first test sample before TTA. (b) Insert visual prompts with image tokens for every test sample during TTA.
= (.P...,,, i= 2.3......
(3)
The visual prompt augmented image tokens E1,N then serve the visual input to JVL.
Test-time Warm-start. Here, challenge remains regarding the initialization of multi-modal prompts. For the text prompts, we simply set Pr = 0 so that the output tokens from the text branch remain unchanged. However, for the visual prompts, both zero initialization and random initialization inevitably degrade the representation capability of the vision branch. In extreme cases, the detector fails to detect any objects, resulting in catastrophic failure during test-time adaptation. As illustrated in Fig. [a), we propose a Test-time Warm-start strategy for the visual prompts. At the beginming of initialize the visual prompts using average pooling over the imput image tokens Er,; from the first image Xo at the i-th Transformer layer:
each test-time adaptation,
Pi. AugPool(E1,))
Based the above components, construct the Multi-modal Prompt-bascd Mcan-Teache frame- work, which comprises a set multi-modal teacher prompts P(r;,) and set of multi-modal student prompts P(r.i)- The teacher model, equipped with (TT,))' generates high-quality pseudo-labels on weakly augmented target domain test data to supervise the optimization of the student model, which uses P(r. and receives strongly augmented test data. prevent being misled by noisy pseudo-labele we set a classification score threshold thpt to filter them. The total optimization objective is:
Leotal = Lete + Lioe
(5)
where Lats and Lioe denote the contrastive classification loss and localization loss in Grounding DINO, respectively. Pseudo-labels generated by the teacher model are used ground-truth during test time. We only fine-tune P(T,r), while the majority of the pre-trained detector remains frozen during adaptation. And (Ir,1) are updated after each iteration via exponential moving average of P(r,1) follows:
PÍr. 7Pir. + (1 (1- ((r,))
(6)
where 7 E [0, 1] is the momentum coefficient.
3.4 Instance Dynamic Memory Enhancement and Hallucination
The mean-teacher framework primarily relies on high-quality pseudo-labels derived from target domain test data to facilitate test-time adaptation. However, obtaining reliable pscudo-labels in every test batch remains challenging, even with the assistance of the lisnun-language detector. To address this issue, we propose maintaining an Instance Dynamic Memory (IDM) module for each category during TTA to preserve valuable knowledge acquired from prior test samples.
Specifically, the IDM maintains dynamic queue Qe for each category e, which is initialized as empty. For the current test data Ii, one of its pseudo-labels is typically represented (bboz, 8, c). corresponding to the predicted bounding box, classification score, and category, respectively. For a high-quality pseudo-label with classification score s exceeding the given threshold thpt we construct a triplet (img, feat, s) as follows:
img = Crop(x bboz)
5

---

(8) where Cr)p(:..) is used to extract an instance crop from image T3, and DINOs2(.) generates the DINOv2[2] feature for the cropped instance. We then store the triplet in the dynamic queue Qu corresponding to its predicted category c. When Qe has not reached the maximum capacity Qq we directly insert this new triplet. However, if Q. is already full, we compare the classification score s of the current pseudo-label instance with the lowest score in Q.. 8 is higher, we replace the lowest-scoring triplet with the new one; otherwise,the new pscudo- instance is excluded, and the queue keeps unchanged.
feat = DINOO2(img)
IDM maintains progressively improving high-quality pseudo-labels through its iterative refinement mechanism. Based on IDM, we introduce two novel Praatgiess-Mmor Enhancement and Memory Hallucination-to more effectively leverage high-quality pseudo-labels from the test stream.
Memory Enhancement. Inspired by TDA[I]] we leverage the IDM refine the original predictions of the current test image through the Memory Enhancement strategy, as illustrated in Fig. bb. We first calculate the DINOv2 feature prototypes for each class by averaging all samples in Q denoted as lic. Then, for an original prediction (bbox,, 85, (y) of the current test image Tgg we compute its memory-based classification score 8, as:
5 = A(DINOv2(Crap(e,, bboo,, bboz,))
where A(z) = aezp(-8(11 =)) is the affinity function is weighting factor and is a sharpness ratio. The enhanced prediction is defined as ([bboxi, ()), where the final classification score is 8j 5j + 5,. and its predicted category is C argmarss
We compute similarity using the prototype Ve for each category e instead of all samples in Qes since the number of instances per category varies greatly in object detection. Directly using all samples from each category would cause categories with more instances to obtain higher classification scores. Given that the detector produces numerous predictions per image (e.g.. 300 in Grounding DINO), applying Memory Enhancement to every prediction would not only significantly reduce inference speed, but also degrade detection performance due to the enhancement of low-confidence predictions. Thus, we apply Memory Enhancement only to high-confidence predictions whose classification scores exceed thme.
Memory Hallucination. The threshold filtering in the mean-teacher framework may lead to no available pseudo-labels for adaptation on certain test data, a limitation overlooked by previous self- training based works Since these negative test samples still contain valuable target domain information, we propose a Memory Hallucination strategy, as shown in Fig. Cc. This strategy randomly samples high-quality instances from IDM and hallucinates positive samples by integrating them into the negative test data.
Specifically, for a negative image 2, and high-quality instance image img, sampled from IDM, we overlay imgy onto H; at random position to generate a hallucinated positive image ž; using mixing coefficient A, where À e [0, 1] is sampled from a Beta distribution. For each negative image. we commix at most three high-quality instance images. To prevent overlap among instances, we set an IoU threshold ohr if the IoU of the current instance image and any previously placed one exceeds thioU, we randomly reselect positions and retry up to 10 times. Additionally, to prevent the detector from overfitting high-quality instance images in the scale space, we apply random scaling to img before mixing.
4 Experiments
4.1 Datasets
We evaluate the effectiveness of our method across a variety of TTAOD scenarios, covering two benchmarks: the cross cross-corruppe corruptoont benchmark and the cross datasett benchmark. The cross-corruption benchmark is widely adopted in previous TTAOD works 31 [1 to assess model robustness, specifically including two datasets: Pascal-C and COCO-C. Pascal-C is constructed from the test set of Pascal VOC by applying an image corruption package 200, which consists of 15 types of corruptions. Each corrupted test set contains 4956 images spanning 20 classes. COCO-C generated from COCO [16), which contains 80 object categories. Following the same procedure as

---

Table 1: Test-time adaptive object detection results (AP50) Pascal-C.
Detectors
Mrthook
Noise
Blur
Weather
Digital
Arg
Impul
Defoc
7arm
Fog
1peg
7.4
RCNN (ResNer
X
11.0
13.8
SHOT Teacher
18.9
27.6
21.7
27.7
STWAR X
30.8
29.3
50.8
Grunding
(Swin-1)
Teacher
47.9
37.1
77.7
Table Test-time adaptive object detection results (mAP) on COCO-C. * indicates methods using Tefchea weights pre-trained on the COCO training set with extensive data augmentation.
Noise
Weather
Digital
AVE
Inpal
Defoc
Glues
Beii
Elast
3psy
Direct Test BN
8.2
10.0
9.1
12.9
4.7
9.1
19.8
24.0
38,9
38.4
22.9
16.5
15,9
[
0.8
5.8
8.3
13.6
15.2
3.4
7.3
TENT
0.5
0.5
5.6
6.4
3.5
5.6
5.0
2.2
3,6
Pister RCNN
(ReeNet:
Mean Teacher
SHOT
T3A
[N
11.0
4.6
13.0
58
12.1
52
14.4
14.7
83
3.7
3.2
3.1
11.0
588
3.5
25.1
22.0
13.8
26.7
27.0
13.2
41.5
28.9
40.9
28.8
37.8
26.6
15.9
19.7
11.3
9.7
19.3
16.4
9.0
18.6
11.0
STIAR
13.4
26.5
25.2
38.5
38.4
22.4
21.7
CTTOU*
31
15.9
14.1
13:7
7.5
25.5
23.6
39.4
38.8
26.3
24.8
22.5
STppD-Sbpp 31
13.2
7.3
24.0
58.9
25.4
23.7
Flster RCNN (Reshet 101)
Memn-Teache 50
Direct Tex
11.7
16.7
20.4
13.8
20.1
12.2
17.3
15.1
15.8
7.1
10.9
15.9
5.5
7.5
29.5
23.5
30.7
26.9
42.6
42.5
41.4
41.8
26,8
33.1
24.8
18.9
13.3
8.7
22.0
16.0
23.4
18.7
STFAR* B
20.1
19.3
20.7
17.0
16.6
17.1
8.5
30.5
31.2
42.1
41.7
33.8
29.6
26.1
25.3
25.3
Finter RCNN
(Swin-T)
crmn-Sép X
CT1O0 11
Direct Test
13.6
13.5
9.7
15.6
11.4
158
10.0
15.1
14.8
13.4
14.3
14.3
14.2
13.6
7.5
12.1
14.9
14.3
5.3
7.8
8.8
25.5 1
20.7
24.0
21.2
26.7
8
37.5
36.1
37.6
56.0
59.0
56.8
27.0
27.5
12.9
27.9
25.6
19.1
25.3
23.5
4.9
23.7
22.6
15.8
21.3
21.9
16.0
Grounding DINO (Swtn T)
Mean Teacher
Dieect Test
20
13.7
15.6
20.8
16.0
20.4
15.0
18.4
1688
11.0
7.5
13.6
16.6
6.7
7.6
27.5
30.9
32.5
34.6
44.2
48.3
44.1
44.8
21.9
25.8
22.5
25.8
12.3
5.3
21.1
25.7
24.1
2006
Our
20.2
22.0
21.4
17.8
14.5
16.9
7.9
31.1
34.7
45.1
44.9
30.6
29.9
23.6
29.2
36.0
Pascal-C, we construct COCO-C using the COCO va12017 set, which includes 5k images, to serve as the target domains.
We adopt the ODinW-13 datasets as novel cross-dataset benchmark to evaluate the detector's performance across 13 diverse object detection datasets, each representing a distinct domain with different categories. These datasets are labeled as Ae (Acrial Maritime Drone), Aq (Aquarium), Co (Cottontail Rabbits), Eg (Fgohands), Mu (Mushrooms), Pa (Packages), Pv (Pascal VOC), (Pistols), Po (Pothole), Ra (Raccoon), Sh (Shellfish). Th (Thermal Dogs and People), Ve (Vehicles). We perform test-time adaptation on the test sets of 13 sub-datasets, providing a comprehensive evaluation of the model's adaptability across varying class spaces.
4.2 Implementation Details
In this paper, our method is built upon Grounding DINO with Swin-Tiny 18 as the visual backbone. We use Grounding DINO tre-trained on Objects365124 24), GoldG[12), and [2) without any fine-tuning on source domain data before adaptation. Additionally, we employ DINOv2 with ViT-L as the feature extractor in the IDM module. For the cross-corruption benchmark, we set the learning rate of the AdamW W optimizer 0.02 for text prompts and 0.2 for visual prompts, while freezing all other parameters pre-trained on large-scale data. The batch size is set to 4. For other hyperparameters. we set Uhg to 0.3, time to 0.3, and Lar to 0.2. The momentum coefficient y in Eq. [6 is set to 0.999. The number y of visual prompts is set to 10, and the maximum capacity 1Qr/mox of IDM is set to 20. We set a = 5.0 and / 5.0 for Pascal-C, while a = 1.0 and 3 = 5.0 for COCO-C. All experiments conducted
single RTX 3090 GPU. Hyperparameters for the cross- dross-dataset benchmark are detailed in the appendix.
4.3 Comparisons with State-of-the-art
Results on the Cross-corruption Benchmark. We first compare our method with existing TTAOD approaches on Pascal-C. As shown in Table adapting BN TENT[28] and T3A [ to TTAOD leads to significant performance degradation compared to directly testing using Faster R-CNN trained on the source domain. contrast, self-training based methods such as SHOT(15), Mean- Teacher and STEAR yield performance gains. When tested directly, Grounding DINO achieves an
7

---

average AP50 of 44.8%, comparable to the previous tfe STFAR. This demonstrates the strong generalization capability of the pre-trained lision-lagguae detector. Furthermore, simply applying self-training to Grounding DINO results in 6.7% improvement in average AP50. In comparison, our proposed method achieves the highest average AP50 of 56. 36.2% across 15 corruption types, outperforming the previous Mate-of-the-ar STFAR by remarkable margin of 11.0%. Consid- ering the differences the adopted detectors, achieving a completely fair comparison challenging. Nevertheless, by conducting thorough comparisons with the strong Mean-Teache baseline built upon Grounding DINO (51.5% vs. 56.2%), we clearly demonstrate the consistent performance improvements achieved by our method.
We then report the mAP performance on COCO- C in Table 2 presenting comprchensive com- parisons with conventional TTAOD approaches across different backbones. Since CTTOAD and CTTAOD-Skip are specifically designed for continuous test-time adaptation (CTTA), we evaluate their performance under the discrete adaptation setting on COCO-C. Methods us- ing ResNet-50 and ResNet- 101 are initialized with SoftTeacher 30) weights pre-trained the COCO training set. Since SoftTeacher em- ploys extensive data augmentation during train- ing (e.g., brightness jitter, contrast jitter), demonstrates superior adaptation capability corrupted target domains. Our method achieves the highest average mAP of 26.0% without uti- lizing any source domain data, while attaining late-of-theeart performance out of 15 cor- ruption types.
Ve
Aq
17.4
Th
66.2
16.1
31.0
Co
55.6
802
55.2
61.0
30.0
79.4
54.8
A5.
49,2
88.8
62.0
94.2
60
56.5
24.8
86.4
Ra
Mu
25.2
66.6
25.6
73.2
57.5
06.3
73,6
58.4
Po
Pa
Direct Test
Mean-Reacher
Ours
Figure 4: Results the cross-dataset benchmark comprising 13 diverse object detection datasets.
Results the Cross-dataset Benchmark. Fig. Jshows our method's performance on 13 downstream datasets with different categorics. Our method reaches average mAP of 54.2%, representing a 1.4% improvement over Direct Test's 52.8%, while Mean-Teacher with Grounding DINO yields only 0.3% gain. Our method demon- strates consistent improvements almost all ODinW-13 sub-datasets except Mu and Pa. This phenomenon is primarily attributed to the extremely few test sanples available (5 for Mu, 4 for Pa), which prevents the vision-language detector from adequately adapting the target domains. Experimental results verify the effectiveness of our method in adapting to diverse class datasets during test time.
4.4 Ablation Studies
Effectiveness of Each Component. We investigate the contribution of each compo- nent Pascal-C across all 15 corruption types, with results presented in T Us- ing only Text Prompt Tuning (TPT) yields merely a 0.6% improvement in average AP50 compared to direct testing. When ap- plying Visual Prompt Tuning (VPT) alone for TTA. the performance even drops by 3.4%. We attribute this decline to subop- timal visual prompt initialization, which impairs the teacher model's performance and consequently undermines the adapta- tion capability of the lasion-language de- tector during test time. Comparing (2) and (3), our proposed Test-time Warm-start (TTWS) strategy effectively mitigates this issue. Furthermore, the text and visual prompts exhibit complementary effects. As a training-fres strategy, Memory Enhancement (ME) directly enhances the zero-shot performance of Grounding DINO test data. Comparing (4) and (6), Memory Hallucination
Table 3: Ablation study of each component on Pascal-C. The average AP50 across 15 corruption types is shown.
MPMT
IDM
Methods
Avg
TPT
VPT
TTWS
ME
MH
Direct Test
44.8
(I)
45.4
(
41.4
(3)
53.4
(4)
53.9
(5)
46.1
(6)
54.9
Ours
56.2

---

(MH) improves adaptation by hallucinating positive samples challenging images, enabling the Multi-modal Prompt-based Jlean-Teacher (MPMT) achieve an additional 1.0% performance gain. By integrating all components, our method achieves state-of-the-are performance.
Table Comparison on the Number of Visual Prompts.
# Prompts
2
4
10
15
20
30
50
AP50
42.6
44.0
45.0
45.3
45.4
45.2
45.3
45.1
44.2
About the Number of Visual Prompts. We evaluate the impact of the number m of visual prompts on the Gaussian noise corruption of Pascal-C. As shown T the visual-language detector achieves effective test-time adaptation with only few visual prompts. While increasing the number of visual prompts enhances the detector's adaptability, also raises the risk of overfitting. For example, performance begins to decrease when exceeds 30. Empirically, setting m to 10 provides an optimal balance.
About the Maximum Capacity of IDM. We analyze the influence of the maximum capacity (Qe lelma of IDM under Gaussian noise corruption on Pascal-C, which si- multaneously affects the effectiveness of both the Memory Enhancement and Memory Hallucination strategies. As shown in Fig. [] when 1Qelmar set too low, Memory Enhancement Tails to acquire sufficiently representative category prototypes ve, thereby limiting its ability to refine the original predictions. When set too high, noisy pseudo- labels may be included, compromising the quality of 6e and weakening the effectiveness of Memory Enhancement. Furthermore, experiments reveal that the introduction of category prototypes Ve confers robustness to Memory En hancement. For Memory Hallucination, a too-small (Qel maz leads to repeated use of few-shot images from IDM for negative image hallucination, causing the detector overfit and even degrading its performance. Conversely, large (Qel mag risks using noisy pseudo-labels during the hallucination process, misleading test-time adaptation. Considering both the performance of the two strategies and the storage cost, we set [Qel maz to 20.
# of IDM's Maximum Capacity
46
45
AP50 (%)
MH
32
10.lmax
Figure 5: Comparison on the Maximum
Capacity of IDM.
Original TP-FP Distribution
Enhancement Distribution
Precision Recall
TP
1.0
12.0
10.0
Precision
Densil
Densty
6.0
4.0
0.2
2.5
2.0
0.0
0.2
0.4
0.0
0.4
0.6
0.8
1.0
0.0
Classification Scores
Classification
Figure 6: Analysis of Memory Enhancement.
Analysis of Memory Enhancement. We observe that Memory Enhancement remains effective even when only classification scores are modified without changing the predicted labels. Therefore, conduct a comprehensive analysis of how Memory Enhancement improves the original predictions. In Fig. 6 plot the True Positive (TP) and False Positive (FP) distributions of the original predictions and enhancement predictions for the category "bicycle" on the Gaussian noise corruption of Pascal-C. It can be seen that Memory Enhancement optimizes the ordering of TP and FP predictions, ensuring that TP predictions are ranked higher than FP ones, thereby improving detection performance. The Precision- Recall curve for the 'bicycle' category further corroborates this result.
5 Conclusion
In this paper, we propose the first foundation model-powered test-time adaptive object detection method, which requires no source data while overcoming traditional closed-set limitations. The pro-

---

posed method employs ceatneett---ffcienn Multi- Prompt-basee baspt Mean-Teacher Teacher framework for adaptation, incorporating Test-time Warm-start strategy to preserve the teacher model's performance. Moreover, we introduce an Instance Dynamic Memory module, along with Memory Enhancement and Memory Hallucination strategies, to effectively leverage high-quality pseudo-labels from the test stream. Extensive evaluations on two benchmarks demonstrate that our method outperforms the state-of-the-art TTAOD approaches, with successful adaptation to arbitrary cross-domain and cross-category target data.
Acknowledgment
This work is partly supported by the National Key Research and Development Plan (024YFB3309302), the National Natural Science Foundation of China 11)00111), the Postdoctoral Fellowship Program of CPSF (GZC2025 1098). the Research Program of State Key Laboratory of Complex and Critical Software Environment, the Anhui Postdoctoral Scientific Research Program Foundation (2025C1 34) and the Fundamental Research Funds for the Central Universities.
References
[1] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. European Conference on Computer Vision, 2020.
[2] Yijin Chen, Xun Xu, Yongyi Su, and Kui Jia. Stfar: Improving object detection robustness at test- time by self-training with feature alignment regularization. arXiv preprint arXiv:2303.17937, 2023.
[3] Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Domain adaptive faster r-cnn for object detection in the wild. In IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2018.
[4] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
[5] Mark Everingham, SM Ali Eslami, Lue Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospective. International Journal of Computer Vision, 111(1):98-136, 2015.
[6] Xiuye Gu, Tsung- Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. In International Conference Learning Representations, 2022.
[7] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelcrating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, 2015.
[8] Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. In Advances in Neural Information Processing Systems, 2021.
[9] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Sung. Zhen Li, and Tom Duerig. Scaling up visual and languaee representation learning with noisy text supervision. In International Conference on Machine Learning, 2021.
[10] Menglin Jia, Luming Tang, Cor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision 2022.
[11] Adilbek Karmanov, Dayan Guan, Shijian Lu, Abdulmotaleb El Saddik, and Eric Xing. Efficient test-time adaptation of vision-language models. IEFF/CVF Conference on Computer Vision and Pattern Recognition, 2024.
10

---

[12] Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, al. Grounded language-image pre-training. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
[13] Shuaifeng Li, Mao Ye, Xiatian Zhu, Lihua Zhou, and Lin Xiong. Source- object detection by learning to overlook domain style. IEEE/CVE Conference on Computer Vision and Puttern Recognition, 2022.
[14] Xianfeng Li, Weijie Chen, Di Xie, Shicai Yang, Peng Yuan, Shiliang Pu, and Yueting Zhuang. free lunch for unsupervised domain adaptive object detection without source data. In AAAI Conference Artificial Intelligence, 2021.
[15] Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, 2020.
[16] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference Computer Vision, 2014.
[17] Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang. Qing Jiang, Chumyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference on Computer Vision, 2024.
[18] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In JEEE/CVF International Conference Computer Vision 2021.
[19] Haoxiang Ma, Ran Qin, Modi Shi, Boyang Gao, and Di Huang. Sim-to-real grasp detection with global-to-local rgb-d adaptation. In IEEE International Conference on Robotics and Automation, 2024.
[20] Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander S Ecker, Matthias Bethge, and Wieland Brendel. Benchmarking robustness object detection: Autonomous driving when winter is coming. arXiv preprint wrXiv:1907.0748/1 2019.
[21] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024.
[22] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning. 2021.
[23] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6): 137--1199 2016.
[24] Shuai Shao, Zeming Li, Tianyuan Zhang. Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: large-scale, high-quality dataset for object detection. In IEEE/CVF International Conference on Computer Vision, 2019.
[25] Yongyi Su, Xun Xu, and Kui Jia. Revisiting realistic test-time training: Sequential inference and adaptation by anchored clustering. In Advances Neural Information Processing Systems, 2022.
[26] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz. Hardt. Test-time training with self-supervision for generalization under distribution shifts. International Conference on Machine Learning, 2020.
[27] Vibashan VS, Poojan Oza, and Vishal M Patel. Instance relation graph guided source-free domain adaptive object detection. JEFF/CVF Conference on Computer Vision and Puttern Recognition, 2023.

---

[28] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2021.
[29] Kunyu Wang, Xueyang Fu, Xin Lu, Chengjie Ge, Chengzhi Cao, Wei Zhai, and Zheng-Jun Zha. Efficient test-time adaptive object detection via sensitivity- guiiddd pruning. In IEEE/CVF Conference Computer Vision and Pattern Recognition, 2025.
[30] Mengde Xu, Zheng Zhang, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, and Zicheng Liu. End-to-end semi-supervised object detection with soft teacher. In JEEE/CVF International Conference on Computer Vision, 2021.
[31] Jayeon Yoo, Dongkwan Lee, Inseop Chung, Donghyun Kim, and Nojun Kwak. What how and when should object detectors update in continually changing test domains? In /EEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[32] Hao Zhang. Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung- Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. In International Conference on Learning Representations, 2023.
[33] Yanan Zhang, Chao Zhou, and Di Huang. Stal3d: Unsupervised domain adaptation for 3d object detection via collaborating self-training and adversarial learning. IEEE Transactions Intelligent Vehicles, 2024.
[34] Yangtao Zheng, Di Huang, Songtao Liu, and Yunhong Wang. Cross-domain object detection through coarse-to-fint feature adaptation. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
[35] Yiwu Zhong, Jianwei Yang. Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. Regionclip: Region-based language-image pretraining. In IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2022.
[36] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learn- ing for vision vision-language models. In IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2022.
[37] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 13009)22337-2348, 2022.
12

---

A Appendix
A.1 More Implementation Details
Hyperparameters the Cross-dataset Benchmark. The cross-dataset benchmark is used to evaluate the de- tector's adaptability across both domains and categories, and consists of 13 object detection datasets: Ae (Aerial Maritime Drone), Aq (Aquarium), Co (Cottontail Rabbits), Eg (Egohands), Mu (Mushrooms), Pa (Packages), Pv (Pas- cal VOC), Pi (Pistols), Po (Pothole), Ra (Raccoon), Sh (Shellfish), Th (Thermal Dogs and People), Ve (Vehicles). Table presents the detailed statistics of these datasets.
Table 5: Datasets statistics of the cross- dataset benchmark.
Dataset
Classes
Test Size
Ae
5
15
Aq
7
127
Co
19
Eg
480
Mu
5
Pa
4
For the cross-dataset benchmark, we set thyt and thme to 0.3. The number of visual prompts is set to 5, and the maximum capacity [Qe/max of IDM set to 3. We 1.0 and 3 5.0 across all datasets. All other hyperparameters follows the settings in the main paper.
Py
20
3,422
Pi
297
Po
133
Ra
29
Sh
116
Data Augmentation. The strong augmentation consists of random resizing and one of color space transformation, se- lected from the following: ColorTransfin Equalize, Sharpness, Posterize, Solarize, Color, Contrast, and Brightness. Additionally, it integrates RandErase, which randomly erases several patches (fewer than 5) with fixed pixel values arbitrary locations to simulate occlusions. The weak augmentation consists solely random resizing, enabling the teacher model to generate more reliable pseudo-labels for the student model.
Th
2
41
Ve
250
AutoContrast,
A.2 More Experimental Results
Table 6: Full results about the ablation study of each component on Pascal-C.
Methods
MIMY
IDM
Natio
Blur
Whather
Digital
Augi
Gasss
Impui
Moto
Zoemn
Ipex
Direct Menn-Tracher
31.8
35.7
43.3
36.4
54.8
10.3
42.7
46.1
46.6
45.4
30,0
63.6
37.1
64.6
67.1
13.5
75.8
64.4
49.3
19.4
58.3
31.5
())
50.7
192
16.4
41.1
20.4
90.3
13.
6035
68.
73.4
$8.2
44.
8.8
407
45.4
(2)
9.1
364
37.1
13.6
41.7
31.4
63.4
65.2
68.5
75.3
37.6
433
0.1
51.4
41.4
(3)
a6.1
37.
55.6
10.7
(6)
45.9
50.9
50.2
47.0
36.6
46.7
38.4
65.9
67.3
73.
76.5
65.4
52.9
41.9
62.1
54.9
Onn
66.9
33.0
518
41.9
36.6
08.0
19.4
66.7
06.
77.7
778
06.5
54.2
42.3
03.7
56.2
Full Results about the Ablation Study for Each Component. In Table we report the contribution of each component across all 15 corruption types on Pascal-C, as well as the average AP50. The proposed components achieve consistent improvements across nearly all types of corruption. We observe that using only Visual Prompt Tuning can lead to catastrophic failure during TTA on certain corruptions (e.g. Gaussian noise, Pixelate). Our proposed Test-time Warm-star strategy effectively addresses this issue
Table 7: Comparison of the runtime cost and average AP50.
Tuned Params
Latency. (ms/img)
Mcmory! (GB)
Avg (%)
Methods
(M)
Full Fine-tuning
164.964
635.0
20.9
51.5
TPT
0.037
532.9
16.6
45.4
VPT
0.042
562.7
17.6
53.4
MPMT
0.079
582.9
18.0
53.9
Runtime Cost. In Table 7 we present the number of learnable parameters, per-image test-time adap- tation latency, peak GPU memory footprint, and average AP50 on Pascal-C, measured using an RTX 3090 GPU. Compared to Full Fine-tuning, proposed Multi-modal Prompt-based Mcan-Teacher
13

---

framework requires only 0.05% of the learnable parameters, enabling us to store corresponding multi-modal prompts for each target domain while sharing single copy of the pre-trained Grounding DINO weights. This advantage also facilitates easy extension to Continual Test-time Adaptive Object Detection (CTTAOD). In addition, our method demonstrates benefits in latency and peak GPU memory footprint compared to Full Fine-tuning, while achieving significant improvement in Average AP50.
About the Threshold of Memory Enhancement. Since Grounding DINO generates 300 predictions per image, applying Memory Enhancement to all predictions would significantly reduce inference speed. As shown in Fig. 7 we select the threshold thme. under Gaussian noise cor- ruption on Pascal-C. When thyme is below 0.2, the latency (blue solid line) increases notably compared to direct test- ing (blue dashed line). Moreover, we observe that when thme is below 0.2, Memory Enhancement leads to perfor- mance decline (green solid line) compared to direct testing (green dashed line). This is because, this threshold, there are a large number of noisy predictions, and refining noisy predictions causes greater confusion in the final results. Considering both latency and detection performance, we set it to 0.3.
Threshold of Memory Enhancement
AP50 (%)
28
Latency (ms))
24
16
0.1
0.2
0.3
0.4 0.5
0.6
0.7
0.8
0.9
thme
Figure 7: Analysis of the Memory En- hancement threshold.
Table 8: Sensitivity analysis of and B.
Sensitivity Analysis on the Weighting Factor and Sharpness Ratio. We conduct sensitivity analysis of the hyperparameters a and 3 on the Gaussian noise corrup- tion of Pascal-C. As shown in Table 8 we achieve the best AP50 when setting a to 5.0 and 3 to 5.0.
2.0
3.0
4.0
5.0
6.0
7.0
32.5
32.8
33.0
33.2
32.8
32.7
0.5
1.0
5.0
9.0
32.6
32.7
33.2
32.4
32.0
Table 9: Comparison of the inference cost and average mAP on COCO-C.
Latency (ms/img)
Detectors
Methods
Source Data
Avg
Direct Test
Pre-train
54.2
15.9
Mean- Mean-Teacher 330
Pre-train
296.1
18.6
Faster RCNN (Res-50)
STEAR 2
Pre-train Statistical Characteristics
327.3
21.7
CTTAOD 31
Pre-train Statistical Characteristics
143.9
22.3
CTTAOD-Skip 31
Pre-train Statistical Characteristics
84.9
21.7
Direct Test
213.1
20.6
Grounding DINO (Swin-T)
Teacher
30
635.0
24.1
Ours
693.8
26.0
Detailed Analysis of Inference Costs. TTA not only requires inference but also involves model adaptation based on the current test data. This process inevitably introduces additional latency compared to direct testing. As illustrated in Table although previous approaches demonstrate faster inference speeds, they require pre-training weights for each source domain and extract statistical characteristics from the source data-both of which incur significant time costs. Compared to the strong baseline Mean-Teacher we introduce only minimal additional inference time (693.8 ms/img versus 635.0 ms/img), and achieve nearly 2% performance improvement COCO-C.
On the other hand, some studies have begun specifically focus on improving efficiency. For example, CTTAOD-Skip (31) improves average inference speed by skipping some test samples, and Efficient TTAOD 22 accelerates inference through pruning. We attempted to integrate a relatively simple Skip strategy into our method, which reduces the latency to 387.7 ms/img while maintaining comparable performance with minimal performance drop.
A.3 Visualization
Examples of Memory Hallucination. Fig. shows examples Memory Hallucination. can be observed that Memory Hallucination effectively leverages high-quality instances from the Instance
14

---

Dynamic Memory, as well as negative images without available pseudo-labels to generate diverse positive samples.
Figure 8: Examples of Memory Hallucination.
present the changes in predictions before and after applying Memory Enhancement. Red boxes indicate false positive (FP) samples, while green boxes represent true positive (TP) samples. In the first row, the original classification scores of the two FP samples are higher than those of the two TP samples. After applying Memory Enhancement, as shown in the second row, the TP samples receive significant enhancement, resulting in their classification scores surpassing those of the two FP samples. The same trend is observed in the third and fourth rows. Memory Enhancement improves the recall high-confidence predictions, enabling the detector to produce predictions that include more true positives.
Examples of Memory Enhancement. In Fig.
Origioal
Origgiaal
Figure 9: Examples of Memory Enhancement.
Detection Results. We visualize detection results with all comparative methods based on Grounding DINO. Fig. shows the detection results on Pascal-C under Gaussian noise corruption. Our method alleviates misclassifications, as seen the first and second rows, missed detections, as shown the third row, and false positives, as illustrated in the fourth row. We also provide visualizations under different corruption types (Fig. [T] and across various corruption types on different datasets Fig. further demonstrating the effectiveness of our method.
15

---

Ground Truth
Direct Test
Mean-Teacher
Ours
Figure 10: Detection results under Gaussian noise corruption Pascal-C.
Ground TrathTTruth
Direct Test
Teacher
Ours
Figure 11: Detection results under JPEG compression corruption Pascal-C.
16

---

J. PM
M.P
J.M
Ground Truth
Direct Test
MeanhTeacher
Ours
Figure 12: Detection results under Shot noise corruption COCO-C.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the claims made, including the contributions made in the paper and the scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [NA]
Justification:
This paper provides detailed analysis and thorough experimentation, with no obvious limitations.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
18

---

Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have described the proposed method in Section 3 and the corresponding implementation details Section 4.2 Additionally, the code and models will be made publicly available.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
19

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The dataset open-access, and our code and models will be publicly released.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ praips.cc/publicc/uuieeee CoodeSumeeieeeeee ssi iessonPolli mollicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have described the implementation details in Section 4.2 Additionally, the code and models will be made publicly available.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: According to the convention in the field to which the paper belongs, there is usually need report error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
20

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have described the experiments compute resources Section 4.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:///aaaar...cpppoa.ccctataaa
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: We have described the potential positive societal impacts in Section I and there is no potential negative societal impact of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
21

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited the original paper that produced the code package or dataset. We will also include the acknowledgment and Licenses for existing assets when releasing the code.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
22

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: There are no new assets introduced in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
23

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
Declaration is not required in this paper.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy bottps://neurres /atps////ccooeeeeeeee////////// for what should or should be described.
24


Reviews:
[{"review_id": "iTFSMk8FU7", "reviewer": "Reviewer_E5Mt", "summary": "The paper proposes a novel test-time adaptive object detection approach based on the vision-language model. The proposed method employs a self-training framework that fine-tunes only the text and visual prompts. Specifically, the visual prompts are initialized using a test-time warm-start strategy to prevent performance degradation caused by inserting prompts into the visual branch. Additionally, the paper introduces an Instance Dynamic Memory module for storing high-quality pseudo labels, which are used to enhance original predictions and perform data augmentation for images without available pseudo labels. Experimental results demonstrate that the proposed method achieves superior performance across multiple benchmarks.", "questions": "See Weaknesses section above.", "limitations": "N/A", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n\n1.\tThis paper is well-motivated, addresses a more practical problem in TTAOD, and is the first to introduce a vision-language model into this field.\n2.\tThe proposed method is easy to understand and follow.\n3.\tThe proposed method demonstrates superior performance compared to existing Faster R-CNN based TTAOD methods and the VLM based baseline.\n4.\tThe experimental results on both cross-corruption and cross-dataset benchmarks are abundant, along with comprehensive ablation studies.\n\n\n**Weaknesses:**\n\n1.\tThe authors should compare their proposed multi-modal prompt tuning approach with other commonly used parameter-efficient fine-tuning methods, such as LoRA.\n2.\tThe authors use DINOv2 features when maintaining the Instance Dynamic Memory module, it is suggested that the authors provide an explanation for this choice.\n3.\tThe paper only conducts experiments based on Swin-T Grounding DINO. Would using larger model sizes bring further improvements?\n4.\tThe names Memory Reinforcement and Memory Hallucination proposed in the paper may cause confusion with concepts from reinforcement learning and LLM’s hallucination. It is recommended that the authors consider revising these terms.", "was_revised": true, "final_justification": "Thanks for the rebuttal comment. All of my concerns have been addressed. However, I still suggest that the authors supplement more details and analysis in the camera ready version. For example, as pointed out by yLfD02, the fair comparison and real-time inference are important for this task so the authors should add more detailed clarifications on them. Overall, I will keep my original rating.", "rebuttal": "**W1: Compare with other PEFT methods.**\n\nThank you for your suggestion!  Prompt tuning has been widely used for fine-tuning VLMs and has achieved impressive results. Following your advice, we compared our method with two PEFT approaches, LoRA and Adapter, based on the implementations provided in [1]. Both LoRA and Adapter show improvements over direct testing, but our method consistently achieves superior performance.\n\n|Methods|VOC-Gauss|\n|:--------:|:--------:|\n|Direct Test|31.8|\n|Mean-Teacher|42.7|\n|LoRA|43.5|\n|Adapter|39.6|\n|Ours|**46.9**|\n\n[1] Yin D, Hu L, Li B, et al. 5\\% $>$ 100\\%: Breaking performance shackles of full fine-tuning on visual recognition tasks[C]. In CVPR, 2025.\n\n**W2: Reasons for adopting DINOv2 for IDM.**\n\nDINOv2 is pretrained on large curated data from diverse sources, enabling highly generalizable feature representations. To determine the optimal feature extractor for the IDM module, we conducted a comprehensive comparison between DINOv2 and CLIP across various model sizes. Experiments demonstrate DINOv2's superior performance over CLIP, with model scaling further enhancing MR. Considering computational efficiency, we use DINOv2-L to maintain the IDM module.\n\n|Methods|Params (M)|VOC-Gauss|\n|:--------:|:--------:|:--------:|\n|DINOv2-B/14|86|32.3|\n|DINOv2-L/14|300|33.2|\n|DINOv2-g/14|   1,100    |33.7|\n|CLIP-B/32|87|31.9|\n|CLIP-B/16|86|32.0|\n|CLIP-L/14|304|32.4|\n\n**W3: Performance with large backbones.**\n\nConsidering computational cost, Swin-T is commonly used in tasks based on vision-language detectors [2]. Accordingly, we also adopt Swin-T for our experiments. In general, using larger models yields further performance improvements, which is also confirmed by the results shown in the table below. Moreover, our method consistently brings performance gains across Swin-T, Swin-B, and Swin-L.\n\n|Backbones|Methods|VOC-Gauss|\n|:--------:|:--------:|:--------:|\n|Swin-T|Direct Test|31.8|\n||Ours|46.9|\n|Swin-B|Direct Test|57.2|\n||Ours|64.9|\n|Swin-L|Direct Test|71.2|\n||Ours|76.2|\n\n[2] Deng J, Zhang H, Ding K, et al. Zero-shot generalizable incremental learning for vision-language object detection[J]. In NeurIPS, 2024.\n\n**W4: Rename Memory Reinforcement and Memory Hallucination.**\n\nThanks for your suggestion. We will revise \"Memory Reinforcement\" to \"Memory Enhancement\". As for \"Memory Hallucination\", in the context of few-shot learning [3], \"hallucination\" is commonly used to refer to the generation of new samples based on existing data. Therefore, we adopt this term. If you have a better suggestion, we would be glad to consider it.\n\n[3] Zhang W, Wang Y X. Hallucination improves few-shot object detection[C]. In CVPR, 2021."}, {"review_id": "las2LUbow4", "reviewer": "Reviewer_4Xpg", "summary": "The authors introduce a novel test-time adaptive object detection method powered by vision-language foundation models GDINO, which eliminates the need for source data and overcomes the traditional closed-set limitations. The proposed method adopts a Multi-modal Prompt-based Mean-Teacher framework, where learnable vectors are added to the token embeddings of both the text and image branches to enable effective prompt tuning. In addition, Test-time Warm-start strategy is employed to preserve the model’s representation capability during adaptation. The Instance Dynamic Memory IDM module stores high-quality pseudo-labels from previous test samples and supports two enhancement strategies: Memory Reinforcement and Memory Hallucination to improve prediction accuracy and generate positive samples for challenging images. Experiments conducted on multiple datasets demonstrate that the proposed method achieves state-of-the-art performance. Comprehensive ablation studies further highlight the individual contributions of each component in the framework.", "questions": "1. The change in mAP of the proposed method after each epoch should be reported.\n2. Present the ablation results showing the impact of using TPT, VPT, TTWS, and MR.\n3. Provide the actual latency of the proposed method compared to previous approaches.\n4. Include comparisons with recently proposed state-of-the-art (SOTA) methods.\n\nTypos etc:\nL117: use “Grounding” instead of “Groudning”; l289: “Tabel”; l521:”poistive”", "limitations": "No.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The proposed method leverages a vision-language foundation model (Grounding DINO) for test-time adaptive object detection without requiring any access to source data, and is capable of adapting to novel label sets beyond those seen during initial training (open-vocabulary adaptation)\n2. The multi-modal prompt-based Mean-Teacher framework, which incorporates both language and visual prompt tuning, enables the model to effectively adapt to target data by fine-tuning only a very small number of parameters, thereby reducing training time.\n\nWeaknesses:\n1. The use of Grounding DINO (Swin-T) significantly slows down the model’s inference speed and increases storage costs when Instance Dynamic Memory is employed. Please provide a detailed explanation for these limitations.\n2. The approach of adding a learnable vector to the image and text tokens may lead to representation collapse in the absence of source data. (see questions)", "was_revised": true, "final_justification": "The authors have clearly and thoroughly addressed my questions in the rebuttal, particularly clarifying the issues related to inference speed and storage cost. Therefore, I have changed my rating for this paper to 4.", "rebuttal": "**W1: About inference speed and storage costs.**\n\nThank you for pointing out the potential limitation. As shown in the table below, compared to the Mean-Teacher baseline using Grounding DINO, our approach achieves nearly a 2\\% performance improvement with only minimal additional inference time.  And our method overcomes the closed-set limitation and enables us to completely eliminate reliance on source domain data. Although previous approaches have faster inference speeds, they require pre-training weights for each source domain data and extract statistical characteristics from the source data—both of which also incur significant time costs. For example, pre-training takes about 2 hours for VOC and 15 hours for COCO on 8 V100 GPUs.\n\nIn the following table, CTTAOD-Skip* demonstrates excellent latency performance, primarily because it skips adaptation for some test samples based on the distribution gap. Recent works, such as Efficient TTAOD [1], also specifically focus on the efficiency issue. These techniques can also be integrated into our method to further improve inference speed. \n\n|   Detectors    |   Methods    |Use Source Data | Latency (ms/img) | COCO-C (Avg) |\n| :----------:| :----------: | :-----: | :--: | :--: |\n|Faster RCNN| Direct Test  |Pre-train|   54.2   |15.9|\n|(ResNet-50)| Mean-Teacher |Pre-train|   296.1  |18.6|\n||    STFAR    | Pre-train + Statistical Characteristics |  327.3  |21.7|\n||    CTTAOD    | Pre-train + Statistical Characteristics |  143.9  |22.3|\n|| CTTAOD-Skip* | Pre-train + Statistical Characteristics |  84.9    |21.7|\n|Grounding DINO| Direct Test  | No |  213.1  |20.6|\n|(Swin-T)| Mean-Teacher |No|   635.0  |24.1|\n||     Ours    | No |  693.8   |26.0|\n\nEmploying IDM does introduce an increase in storage cost, primarily due to loading DINOv2-L, which requires approximately 1 GB. The triplets maintained by IDM itself require only about 50 MB. In real-world deployments, techniques like quantization and pruning can be considered to reduce storage costs.\n\nThank you again. We will provide a detailed discussion of these limitations in the paper.\n\n[1]Wang K, Fu X, Lu X, et al. Efficient Test-time Adaptive Object Detection via Sensitivity-Guided Pruning[C]. In CVPR, 2025.\n\n**W2: Preventing representation collapse.**\n\nThis is a key issue that we have also noticed, and we have already discussed it in Section 3.3. Since the text prompt is added to the language branch in the form of a residual (Eq. 1), representation collapse can be avoided by initializing it with all zeros. For the visual prompts, the proposed TTWS is specifically designed to address this issue, as shown in Fig. 3.\n\nWe include results for directly testing with added learnable vectors in the following table. Using randomly initialized learnable vectors indeed leads to representation collapse, but this issue can be addressed by incorporating the proposed TTWS. Furthermore, representation collapse can cause catastrophic collapse of the detector during test-time adaptation. Our proposed TTWS effectively addresses this issue, as demonstrated in Tables 3 and 6 of the paper.\n\n|Methods|VOC-Gauss|VOC-Shot|VOC-Impul|\n|:--------:|:--------:|:--------:|:--------:|\n|Direct Test|31.8|38.6|35.7|\n|+Random Init|17.2|24.1|18.4|\n|+TTWS|33.1|39.1|36.4|\n\n**Q1: The change in mAP after each epoch.**\n\nFirst, there may be a misunderstanding. In TTA, we perform adaptation with only one pass over the test data, and the model learns while making predictions. The evaluation is conducted using the accumulated prediction results. We will emphasize this point in the paper. To more clearly demonstrate the evolution of mAP, we evaluated our method at different iterations, and the results show that as the number of iterations increases, the model's performance gradually improves, indicating a more thorough adaptation to the target domain.\n\n|Iters|500|1000|1238 (End)|\n|:--------:|:--------:|:--------:|:--------:|\n|VOC-Gauss|46.5|48.6|50.1|\n\n**Q2: About the ablation results.**\n\nWe have already investigate the impact of using TPT, VPT, TTWS, and MR in Table 3, and provide detailed results in Table 6. This might have been overlooked.\n\n**Q3: About the latency.**\n\nPlease refer to our response to W1.\n\n**Q4: Comparisons with recently proposed SOTA methods.**\n\nWe have compared our method with recent SOTA approaches, including STFAR, CTTAOD, and the strong baseline Mean-Teacher. We also noticed a recent work, Efficient TTAOD*[1], which primarily focuses on the efficiency issue in Continual Test-time Object Detection. Since its code has not been released, we are only able to compare with it on its first target domain of the first run. Additionally, we compare our method with several TTA approaches designed for image classification [2,3,4]. As illustrated in the following table, our method achieves SOTA performance.\n\n|   Methods   | Cityscapes | ACDC-snow  |\n| :---------: | :--------: | :------: |\n| Direct Test |    -    |    21.8    |\n|     Efficient TTAOD*     |    -    |   23.4   |\n| Direct Test |    30.5    |    33.9    |\n|     TPT     |    30.9    |   34.4    |\n|   DiffTPT   |    31.2    |   34.6   |\n|   HisTPT    |    31.9    |   35.6   |\n| Direct Test |    50.2    |    52.5    |\n|    Ours     |  **55.2**  | **54.1** |\n\n[2] Shu M, Nie W, Huang D A, et al. Test-time prompt tuning for zero-shot generalization in vision-language models[J]. In NeurIPS, 2022.\n\n[3] Feng C M, Yu K, Liu Y, et al. Diverse data augmentation with diffusions for effective test-time prompt tuning[C]. In  ICCV, 2023.\n\n[4] Zhang J, Huang J, Zhang X, et al. Historical test-time prompt tuning for vision foundation models[J]. In NeurIPS, 2024.\n\n**Typos:** Thank you. We have carefully proofread the paper and corrected the typos."}, {"review_id": "EJVkTBQXnb", "reviewer": "Reviewer_yLfD", "summary": "This paper proposes a multi-modal prompt based mean-teacher (MPMT) framework for the problem of test-time adaptive object detection (TTAOD), and utilizes a vision-language detector for self-training during TTA. The paper introduces an Instance DynamicMemory (IDM) module, and it is maintained for each category during test time. Based on IDM, they propose two schemes, namely, memory reinforcement (MR) and memory hallucination (MH), to utilize good quality pseudo-labels from previous examples. Experiments on cross-corruption and cross-dataset benchmarks demonstrate improved performance for the proposed approach compared to other approaches.", "questions": "1. How will the proposed approach perform when it encounters data from entirely new categories or when there is a significant distribution shift not represented in the pre-training data?\n2. Can the IDM module handle distribution drift in the continual test-time settings? Will the error accumulation due to noisy pseudo-labels cause a significant performance degradation in a continual scenario?\n3. What is the runtime latency compared to the direct test? Such comparisons of wall clock times will be helpful in understanding the computational overhead during inference.\n4. How sensitive are the results to the other hyperparameters, such as $th_{pl}$ and memory capacity? Whether the hyperparameter tuning is performed using the validation split?", "limitations": "* Some other limitations, such as scaling to drastic distribution shifts, may be included.\n* The trade-offs between accuracy and inference speed can be discussed.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n* This work utilizes a vision language foundation models (VLMs) for doing TTAOD.\n* Use of VLMs allows overcoming the closed-set limitation.\n* MPMT is a nice way to adapt the large foundation model in a parameter efficient manner.\n* Ablation study shows the contributions of different components of the model\n\n**Weaknesses**\n* The paper's claim of outperforming state-of-the-art methods is based on an unfair comparison. The proposed approach uses a powerful VLM (Grounding DINO, SWIN_T) as it's backbone, whereas, the compared prior works uses standard or less powerful architectures like Faster-RCNN (ResNet-50/101). Thus, it is impossible to isolate the actual improvement of the proposed adaptation approach compared to prior state-of-the-art approaches.\n* The quality of the pseudo-labels depends on the threshold $th_{pl}$. More exploration for this hyperparameter is lacking.\n* During test-time, the computational pipeline will have a forward pass involving a large detector, conditional feature extraction with another large model (DINOv2) and memory updates. This will limit its applicability in truly real-time and resource-constrained settings.\n\n**Typos**\n* Line 117: \"Groudning\" -> \"Grounding\"\n* Figure 2: \"Updata\"", "was_revised": true, "final_justification": "The authors have responded to most of the queries.\n\nAs already discussed in \"W1: About fair comparison\", a fair comparison is challenging, and a discussion and more thought about it are required.\n\nI will maintain a positive score.", "rebuttal": "**W1: About fair comparison.**\n\nPrevious methods rely on source domain data for fine-tuning or statistical computation, whereas our method has no access to source domain data. And considering the differences in the adopted detectors, achieving a completely fair comparison is challenging. However, we have made efforts to ensure a fair comparison with previous methods. For example, in Table 2, we compare with CTTAOD based on Faster R-CNN (Swin-T).  Its results are even worse than using ResNet-50/101 as backbones, because ResNet-50/101 use SoftTeacher weights pretrained on the COCO training set with extensive data augmentation.\n\nWe also conduct thorough comparisons with the strong baseline Mean-Teacher using Grounding DINO, which demonstrate the performance improvements achieved by our method.\n\n**W2: Ablation on the threshold $th_{pl}$.**\n\nThank you for pointing out this issue.  We select thethreshold $th_{pl}$ under Gaussian noise corruption on Pascal-C. When $th_{pl}$ is too high, only a small number of pseudo-labels are available for test-time adaptation, which hinders the model from effectively adapting to the target domain. Conversely, if $th_{pl}$ is set too low, noisy pseudo-labels may be introduced, which can also degrade the model's performance. Therefore, we set it to 0.3. \n\n\n|  $th_{pl}$  | 0.1  | 0.2  | 0.3  |0.4  |0.5  |\n| :-------: | :--: | :--: | :--: |:--: |:--: |\n| VOC-Gauss | 45.0 | 46.0 | **46.9** |45.6|41.2|\n\n**W3: About the applicability in real-time and resource-constrained settings.**\n\nThe latency of our method compared to directly testing with a vision-language detector is 693.8 ms/img vs. 213.1 ms/img. However, it brings significant performance improvements, with gains of +11.4\\% AP50 on Pascal-C and +5.4\\% mAP on COCO-C. For more analysis of inference speed and storage costs, please refer to our response to Reviewer 4Xpg W1.\n\n**Typos:** Thank you, we will revise the typos.\n\n**Q1: The performance under significant distribution shifts or unseen categories.**\n\nFor Pascal-C and COCO-C, we follow previous works and use the most challenging corruption level for evaluation.  Additionally, ODinW-13 contains 13 datasets with significant domain shifts (e.g. remote sensing, underwater, etc.).  These benchmarks ensure significant distribution shifts from the pre-training data. On the other hand, since the pre-training data of Grounding DINO includes grounding data, it is difficult to determine whether the categories in OdinW-13 are entirely novel. Our key contribution, by contrast, lies in introducing a vision-language detector to overcome the closed-set limitation. The experimental results in Table 1, Table 2, and Fig. 4 demonstrate that our method performs well under significant distribution shifts and in open-set scenarios. \n\n**Q2: About the IDM module for distribution drift in continual test-time settings and the error accumulation from noisy pseudo-labels in continual scenario.**\n\nOur method focuses on TTAOD, and continual test-time setting is beyond the scope of our paper. Theoretically, the IDM module can be extended to handle distribution drift in a continual test-time scenario. Like CTTAOD [1], we can detect sudden distribution shifts to identify the arrival of a new target domain. Then, we clear the triplets in the IDM module and re-maintains them on the new target domain to handle distribution drift. As for error accumulation, upon detecting the arrival of a new target domain, we can reinitialize new text-visual prompts in our MPMT to alleviate this issue. \n\n[1]Yoo J, Lee D, Chung I, et al. What how and when should object detectors update in continually changing test domains?[C]. In CVPR, 2024.\n\n**Q3: About the runtime latency compared to direct testing.**\n\nPlease refer to our detailed response to W3.\n\n**Q4: About the hyperparameters $th_{pl}$ and memory capacity.**\n\nThe ablation study of the threshold $th_{pl}$ is provided in W2, and the analysis of IDM’s memory capacity is presented in Fig. 5 and Section 4.4 of the paper. Regarding hyperparameter tuning, we strictly followed prior work by selecting hyperparameters under a single type of corruption (Gaussian noise in our paper).\n\n**Limitations:** Please refer to Q1 and W3. Thank you for the reminder. We will include a discussion on the detection performance and inference speed of our method in the limitations section."}]

Author rebuttal:

Meta-review: This paper introduces a framework for test-time adaptive object detection (TTAOD) by leveraging a vision-language foundation model. The ky contribution is a method that eliminates the need for source data and overcomes the traditional closed-set limitations of TTAOD, enabling adaptation to new domains and categories. This is achieved through a Multi-modal Prompt-based Mean-Teacher framework for parameter-efficient adaptation and an Instance Dynamic Memory module to ensure high-quality pseudo-labels. The reviewers unanimously recognized the work's novelty, strong empirical performance across multiple benchmarks, and thorough experimental validation. The primary weaknesses identified during the review process were the potential for an unfair comparison due to the use of a stronger backbone (Grounding DINO) than prior work and the increased computational latency. During the rebuttal period, the author addressed these concerns. They provided latency analyses, explaining the overhead as a common trade-off in TTA and demonstrating the method's strength over a baseline that uses the same detector. They also added new experiments comparing against other fine-tuning methods and analyzing hyperparameter sensitivity, which satisfied the reviewers. Therefore, AC recommends accept.
Decision: Accept (poster)