GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
Authors: Zhaohong Huang
Yuxin Zhang
JingJing Xie
Fei Chao
Rongrong Ji
Venue: ICML 2025 poster (2025)
Keywords: Test Time Adaptation
Vision-Language Models
Deep Learning
Primary area: deep_learning

Abstract: Recent advances in test-time adaptation (TTA) for Vision-Language Models (VLMs) have garnered increasing attention, particularly through the use of multiple augmented views of a single image to boost zero-shot generalization. Unfortunately, existing methods fail to strike a satisfactory balance between performance and efficiency, either due to excessive overhead of tuning text prompts or unstable benefits from handcrafted, training-free visual feature enhancement. In this paper, we present Global-Spatial Bias Learner (GS-Bias), an efficient and effective TTA paradigm that incorporates two learnable biases during TTA, unfolded as the global bias and spatial bias. Particularly, the global bias captures the global semantic features of a test image by learning consistency across augmented views, while spatial bias learns the semantic coherence between regions in the image’s spatial visual representation. It is worth highlighting that these two sets of biases are directly added to the logits outputed by the pretrained VLMs, which circumvent the full backpropagation through VLM that hinders the efficiency of existing TTA methods. This endows GS-Bias with extremely high efficiency while achieving state-of-the-art performance on 15 benchmark datasets. For example, it achieves a 2.23% improvement over TPT in cross-dataset generalization and a 2.72% improvement in domain generalization, while requiring only 6.5% of TPT's memory usage on ImageNet.

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
Zhaohong Huang'
Yuxin Zhang'
Jingjing Xie
Fei Chao
Rongrong Ji
Abstract
65.26
65.6
65.0
64.66
64.00
Accuracy (%)
Recent advances in test-time adaptation (TTA) for Laiiun-Lagguag Models (VLMs) have garnered increasing attention, particularly through the use of multiple augmented views of single image to boost zero-shot generalization. Unfortunately, existing methods fail to strike satisfactory bal- ance between performance and efficiency, either due to excessive overhead tuning text prompts or unstable benefits from handcrafted, training- free visual feature enhancement. In this paper, we present Global-Spatial Bias Learner (GS-Bias) an efficient and effective TTA paradigm that incorpo- rates two learnable biases during TTA, unfolded as the global bias and spatial bias. Particularly, the global bias captures the global semantic features of test image by learning consistency across augmented views, while spatial bias learns the semantic coherence between regions in the image's 's spatial visual representation. It is worth highlighting that these two sets of biases di- rectly added to the logits output by the pretrained VLMs, which circumvent the full backpropaga- tion through VLM that hinders the efficiency of cxisting TTA methods. This endows GS- GS-Bias with extremely high efficiency while achieving state-of-the-arr performance
64
GS- Bins
TPT
MTA
CLIP
63.5
Number of Augmented Views
(a) Accuracy vs. Number A Augmented Views
GS-Bins
1300 Mil
Modols
MTA
35%6
CLIP
758 MiB
2500
7500
10000
12500
17500
20000
Memory (MiB)
(b) GPU Memory Usage Compurison
Figure I. (a) Comparison of state-of-the-art TTA methods (Shu et al., 2022; Zanella & Ben Ayed, 2024) and our proposed GS-Bias 10 cross-datasets generalization benchmarks (Maji al., 2013; Fei-Fei et al., 2004; Krause et al., 2013; Cimpoi et al., 2014; Nils- back & Zisserman. 2008: Bossard al., 2014; Helber al., 2019; Sun et al., 2020; Soomro, 2012; Parkhi et al., 2012). (b) Compari- son of memory consumption of different TTA paradigms the ImageNet (Deng et al., 2009), where all methods are evaluated using augmented views test sample.
1. Introduction
15 benchmark datasets. For example, achieves 2.23% im- provement over TPT in cross-dataset generaliza- tion and 2.72% improvement in domain gen- eralization, while requiring only 6.5% of TPT's memory usage on ImageNet. Our code released at http:/iiiiibbooo/o//hmu/u//// Bias.
In recent years, the development of large-scale vision- language pre-trained models (VLMs) has significantly accel- erated progress within the computer vision community (Jia et al., 2021; Alayrac et al., 2022; Radford et al., 2021b). Notably, CLIP (Radford et al., 2021a), trained on 0.4 billion text-image pairs, have demonstrated remarkable zero-shot performance across a wide range downstream tasks. This success has ignited further research interest the transfer learning capabilities of VLMs. Prevailing efforts mainly falls tuning VLM prompt (Zhou al., 2022b; Chen et al., 2023; Fu et al., 2024) using visual adapters (Gao et al., 2024; Zhang et al., 2022b; Udandarao et al., 2023) to enhances the efficacy of VLMs on downstream tasks. However, two significant limitations are apparent (Zhang et al., 2022a; Wang et al., 2021; Chen al., 2024; Lin
KKey Laboratory of Multimedia Trusted Perception and Effi- cient Computing, Ministry of Education of China, Xiamen Uni- versity, 361005, P.R. China. Correspondence to: Rongrong Ji <ırjj@ emii.....du..n
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
e e
Entrgyy Lawr
Scleniin
Badkpnsagatioa
Fightwigh
AC
Fv
Ey
Local
Erga C Rhna
Fe 1
Augmented
(a) Time Prompt Optimizer
AQ
Ey
oost Proppt
2g
Chooning Opaiolationn
g
Augmented views
A(-)
Fy
Hond-engha
Aug R
Fug CR
P 1
Augusented views
(b) Time Visual Optimizer
Test-Time Globul-Spatial Leurner
Figure 2. Illustration of (a) Test-Time Prompt Optimizer, (b) Test-Time Visual Optimizer and (c) our proposed Test- Time Global-Spatial Bias Learner (ias). IS-Bias introduces two sets of learnable, independent global and spatial biases directly on the prediction distribution of a single image, ie., the output of CLIP. By optimizing both biases solely at the logits output stage, our GS- Bias achieves lightweight backward operations and eliminates overhead secondary inference.
et al., 2025): (1) the zero-shot generalization capabilities of VL.MS are severely compromised due to the substantial distribution shifts between the training data and downstream tasks, and (2) these approaches typically depend on access to high-quality, task-specific training data, which is often impractical in real-world scenarios.
MeanShift (Comaniciu & Meer, 1999) to jointly optimize visual features by leveraging internal scores and density modes across multiple augmented views. Compared to prompt learning, such visual optimizer demonstrated higher efficiency due to its training-free nature. However, its per- formance strongly depends on the quality of augmented views generated by the pre-trained model, as demonstrated in Figure (a), which limits its efficacy in zero-shot, com- plex real-world scenarios, such as those involving unseen classes.
In response to these drawbacks, recent works have shifted towards a new paradigm known as test-time adaptation (TTA) (Shu et al., 2022; Feng et al., 2023; Zanella & Ben Ayed, 2024; Yoon al., 2024; Liu et al., 2024). TTA addresses the above obstacles of domain shift and training data dependency by enabling VLMs to dynamically align with individual test samples on-the-fly fly. To achieve this, ex- isting methods mainly leverage multiple augmented views of a single test image and optimize either text prompts or image features for the sample-wise adaption, which we re- spectively discussed as follows.
To sum up, the two paradigms discussed above still fail to achieve optimal trade-off between efficiency and performance. Figure compares performance and effi- ciency of representative methods TPT (Shu et al., 2022) and MTA (Zanella & Ben Ayed, 2024) for prompt and vi- sual optimizers, respectively. Prompt optimizer exhibits superior performance by learning new knowledge specific to individual images. In contrast, visual optimizer provides higher efficiency through parameter-free optimization of in- ternal features, albeit with weaker performance constrained by the quality of the augmented images.
Test-Time Prompt Optimizer, depicted in Figure 2 (a), optimizes the VLM textual prompts by minimizing the marginal entropy of augmented views derived from the test sample (Yoon et al., 2024; Shu et al., 2022; Feng et al., 2023; Liu et al., 2024). Despite its promising performance, modifying textual prompts can lead to massive memory and computational overhead (see Figure 1 (b)). This due to the intensive backward propagation operations across the full network during the CLIP input-level optimization. Furthermore, the updated prompts require a second forward pass for inference, which is time-consuming (Zanella & Ben Ayed, 2024).
In response to the above drawbacks, this paper presents Global-Spatial Bias Learner (GS-Bias), novel TTA method that equips learnable biases directly alongside the output logits of VLMs. Particularly, GS-Bias contains two types of biases that plays unique role in improving the TTA per- formance, unfolded as the global and local bias. The global bias captures the global semantic knowledge of the test im- age by enforcing the consistency of logits across different views. For the local bias, we leverage the spatial visual rep- resentations from the visual encoder to provide contextual information (Miyai et al., 2024; Lafon al., 2024). Specifi- cally, select regions highly relevant to the downstream
Test-Time Visual Optimizer, as shown in Figure 2 (b), focuses on optimizing the internal visual features of VLMs. For example, MTA (Zanella & Ben Ayed, 2024) utilizes
2

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
task for spatial semantic consistency learning, which is used to update the spatial bias. As illustrated in Figure 2 (c), the cost of updating the biases is extremely low, as the back- ward propagation occurs only at the output side, eliminating the need to further propagate through the entire network as previous works do (Shu et al., 2022; Feng et al., 2023; Yoon et al., 2024; Liu et al., 2024). Consequently, as shown in Fig- ure 1. GS-Bias achieves a well-balanced trade-off between performance and efficiency of TTA. For example, with set- tings aligned to TPT (i.e., using the prompt "a photo of a [class].. and 64 augmented views), GS-Bias achieves leading performance across 10 cross-dataset bench- marks, while consuming merely 6.5% of TPT's memory overhead on ImageNet. Our contributions in this paper include:
able layers in both branches to modify feature distribution, while Tip-Adapter (Zhang et al., 2022b) uses memory mech- anism to perform training-frre few-shot setups. Despite their advantageous performance enhancements, these meth- ods necessitate considerable computational resources and training data, which hinders their practical deployment in real-world contexts. In this paper, we investigate test-time adaptation in VLMs, which does not require prior access to downstream training data.
2.2. Test-Time Adaptation for VLMs
scenarios, the inputs to VLMs are often hetero- geneous, underscoring the critical need for test-time adap- tation (TTA), particularly in contexts such as autonomous driving under adverse weather conditions (Lin et al., 2025) or the diagnosis of diseases during their progression (Chen et al., 2024). The most prevalent mechanism for achieving TTA involve minimizing entropy across either across a batch of test samples (Wang et al., 2021) or across multiple aug- mented views of single test sample (Zhang al., 2022a). As pioneering work, TPT (Shu et al., 2022) fine-tunes the textual context prompts based on the above objective. DifITPT (Feng et al., 2023) further introduces diffusion model to diversify the augmentation pool of the test images. Despite the efficacy, such prompt tuning can lead to mas- sive memory usage and computation resources due to the backward propagation operation through the entire network and secondary inference. In response, MTA (Zanella & Ben Ayed, 2024) transitions TTA from soft prompt learn- ing to direct visual optimization by refining the visual fea- tures through internal evaluations of augmented views, thus demonstrating greater adaptation efficiency. However, ex- isting TTA methods still fail to achieve favorable trade-off between computational efficiency and performance general- ization. In this work, we propose GS-Bias, which introduces lightweight global and spatial biases. These biases capture global semantics through augmented view consistency and align spatial information via regional consistency in visual representations. Crucially, the biases are optimized solely at the logit level, eliminating full-network backward propaga- tion and redundant secondary inference, which significantly reduces computational overhead while enhancing general- ization performance.
In
We propose Global-Spatial Bias Learner (GS-Bias), which unifies global semantics across augmented views via global bias and aligns regional semantics within spatial visual representations through spatial bias.
GS-Bias innovatively performs optimization at the logit level of network outputs, enabling lightweight infer- ence and drastically reducing the TTA overhead of existing methods.
We report comprehensive evaluations and compar- isons to the existing TTA method for VLMs across 15 datasets, demonstrating that GS-Bias achieves an excellent balance between performance and efficiency.
2. Related works
2.1. Contrastive Vision-Language Models
The core of contrastive visual-language models (Radford et al., 2021a; Jia et al., 2021; Radford et al., 2021b) lies in mapping text and images into a unified space, thereby enabling a more comprehensive understanding of visual con- cepts. Among these models, CLIP (Radford et al., 2021a) distinguishes itself through its strong zero-shot capabili- ties across wide range of downstream tasks. This suc- cess has spurred extensive interest in transferring its pre- trained knowledge to more complex tasks. A primary area of focus has been devoted to prompt tuning for few-shot learning scenarios (Zhou et al., 2022b;a; Chen et al., 2023; Lafon et al., 2024: Miyai et al., 2024: Fu al., 2024). For instance, CoOp (Zhou et al., 2022b) replaces hand- crafted prompts with learnable context tokens for task adap- tation, and CoCoOp (Zhou et al., 2022a) further incorporates image-specific conditional information into the optimiza- tion vectors. Differently, adapter-based methods (Gao et al., 2024; Zhang et al., 2022b; Udandarao et al., 2023) focus on designing extra lightweight modules to adapt VL.Ms. For example, CLIP-Adapter (Gao et al., 2024) introduces learn-
3. Method
3.1. Background
Vision Language Models. By aligning image and text within a unified embedding space, LanguuLLe Models (VLMs) such as CLIP (Radford et al., 2021b) mark sub- stantial advancements across broad range of vision-related tasks. Without loss of generality, We utilize CLIP, equipped with a Vision Transformer (VIT (Dosovitskiy, 2020)), as the
3

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
foundational model for our methodological demonstration. CLIP is composed of visual encoder Ey and a text coder ET, which independently extract information from images and text, subsequently mapping them into a shared
where y represents the learning rate for gradient descent, while V (j)) denotes the gradient of the entropy. The test-specific prompt V is then utilized as the text prompt for CLIP during secondary inference, generating the cor- responding test-time text embeddings ( (Fi-) Conse- quently, the final predictions of CLIP is refined as:
feature space. During training, a contrastive loss (Chen et al., 2020) is employed to maximize the cosine similarity between matched pairs from different modalities.
exp (cos (F gg FV.). (/7))/))))
For specific downstream task involving C' classes, each class is embedded into a hard text prompt to generate the corresponding text embeddings { where F? E Rd denotes the text feature of the class-specific text input. At test time, given a single input image z, the visual encoder extracts a global visual representation Fg = Ev(2z) E Rd, where d the feature dimension. By computing the cosine similarity between the visual feature and text embeddings, the prediction probabilities for the query image with re- spect to class Ve can be derived as:
Aupr(yells
(5)
De exp (cos (F -.v-//+
Despite the promising performance of such test-time prompt optimizers, learning the prompts requires backpropagating gradients through the entire network and involves redun- dant secondary inferences, posing significant challenges to computational efficiency. Recently proposed visual optimiz- ers (Zanella & Ben Ayed, 2024) offer lightweight solutions by directly integrating the augmented views using cluster- ing methods like Mean-shift (Comaniciu & Meer, 1999) to boost the robustness of visual embeddings. Unfortunately, the performance of such parameter-free optimization drasti- cally degrade when applied to domains outside of CLIP's pretraining knowledge, such as unseen classes.
exp (cos (F 9. Fi)/T) exp (cos (F g: Fi) /T)
PeLIE (Nelz))
(L)
where cos(-) calculates the cosine similarity between vec- tors, and F is the temperature of the softmax function.
3.2. Global-Spatial Bias Learner
Revisiting Test-time Adaption of VLMs. CLIP's output can be further refined through tuning text prompts visual adapters based on specific downstream datasets (Zhou et al., 2022b;a; Zhang et al., 2022b; Gao et al., 2024). However, these approaches require substantial training costs and data overhead, thus rendering it impractical in real-world see- narios. As more efficient way, test-time adaption adjusts the output of CLIP on-the-fly. This is typically achieved by using multiple augmented views of the test image, denoted where A(-) is the augmentation function,
In this work, we propose Global-Spatin Bias Learner (GS- Bias) as a way to address the above trade-off issue in per- formance and efficiency. The core contribution of GS-Bias lies in introducing two learnable bias terms Bg and B, during VLM test-time adaption. As illustrated in Fig. I (c), the global bias B, y enhances global semantic knowledge by adjusting the logit outputs of each augmented view, whereas the local bias B, refines the local spatial knowledge by selectively emphasizing the visual patches most relevant to class-speciffe objects. Notably, both B, and B, are opti- mized at the logits output stage, thus eliminating the need for entire network backpropagation and secondary inference. We detail the mechanism and implementation of the global and local bias terms of GS-Bias as below.
as (x)) and N denotes the number of augmented views, to adapt the model for more convincing output. For a representative example, TPT (Shu et al., 2022) tunes the text prompt V to reduce inconsistencies in the model's predictions across these diverse augmentations. Particularly, the augmented views are firstly leveraged to generate set of output logits as:
Global Bias Learner. As previously discussed, test-time prompt optimizers tunes text prompt to refine the logits entropy of augmented views (Shu et al., 2022). Although such global enhancement of semantic knowledge can lead to performance improvements, optimizing the text prompts at the model's input side clearly lead to efficiency bottleneck. We address this obstacle by directly adding a lightweight global bias learner to the model's output side. Specifically, to capture global bias knowledge within a single image, we begin by zero-initializing a learnable bias term B3 RIXC which is shaped to match the distribution of CLIP's final output. The learnable global bias is shared across all (3) augmented views and added to their respective logits, acting as a global logit to capture the image's global semantic
1 pN
P (Yel x)
I H (px) S 0] (p& (Me (x)))), (2)
where H(-) calculates the self-entropy of the predicted prob- ability distribution, and the indicator function [H (px) S 0] filters high-uncertainty logits smaller than 0. Afterwards, TPT updates the text prompt V by minimizing the distribu- tion entropy of top-p confident samples p (yelx: as:
H H = Σ po (yel log Py (ye x),
1
V V - - v
(4)
4

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
information, written as:
the concentration of relevance between the corresponding region and the classes in the downstream task. Intuitively, elements with higher concentration values are likely to be more significant. Thus, we employ straightforward Top-K selection strategy to filter out the most informative regions:
N Σ (H) PN
Po (velz)
S (p) Ai(x)) Bg)
(6)
Then, we follow TPT to calculates the self-entropy of the predicted probability distribution H (pg) as per Eq. 2. while perform very lightweight backpropagation to update the learablee global bias as:
I = {a (M,, i e (1, w x h)};
(11)
I denotes the set of indices corresponding to the selected spatial regions. Using these indices, we retrieve the selected probabilities from the regions-text probability set S, thereby obtaining refined set of predictions for the most relevant regions S = {p(ye F The set S contains the top K regions most likely to contain the class-specific object. For clarity, we denote this collection as S = (p(ye F We strategically incorporate shared spatial bias B, into S, thereby generating spatial logits f0s (yee x)) that serve as a foundation for exploring the fine-grained spatial knowledge within the test image:
B, B Bg - aV aVB H (pg),
where a > 0 controls the strength of the global consistency constraint. After updating, we directly add the global bias to the vanilla output logit as:
Pe(ulæ)= = (yl=) + Bg-
(8)
It is worth noting that the updating and incorporation of B, y occur directly at the output end of the text branch, thus eliminating the need for complete backpropagation and sub- sequent forward propagation post-update.
K
Ps (Mel2x)
1
Pk
ye
F
+ B.)
(12)
Spatial Bias Learner. We further present spatial bias learner to enhance to local spatial feature within the im- age branch of VLMs. In particular, spatial features F, E Rudxd refer to the intrinsic embeddings within the vision W. H and w = F. with H and W repre- senting the height and width of the input image, respectively, and P denoting the patch size. Recent advancements in prompt tuning (Miyai al., 2024; Lafon et al., 2024) have shown that enriching such spatial features brings signifi- cant performance gains. Despite this progress, the potential of spatial features remains untapped in TTA settings. To address this gap, we introduce a set of zero-initialized learn- able spatial biases B, E RIxC' to capture spatial knowledge specific to single image. Since the visual encoder maps both spatial features F, and text embeddings F; to a fea- ture space with the same dimensionality, we can directly compute the prediction probability for each spatial feature region with respect to class Ye. resulting in a set of spatial wexh defined as: probabilities S = {p(ve F)))x
K
Then, we enforce spatial consistency by calculating the entropy of ÿ. in the spatial branch as:
encoder, where h = F
H
- (P.)-
Σ Ps (Ue x)) log (Ve l x))
(13)
The underlying principle of the above learning target is that the selected high-information regions should encapsulate the majority of the target class information. At last, we update the spatial bias by computing the gradient of Eq. 9 as:
(14) where 8 represents the learning rate of the spatial bias. Based on the above, we have obtained the global bias B, summarizing the overall semantics, the spatial bias B, con- taining fine-grained details of the test image, and the pre- trained CLIP output distribution PeLie Pelle(y/z)) (v/z:) These compo- nents are combined to produce the final output of GS-Bias during the test phase, as follows:
B, + B 3V H (p.),
exp (cos F. Fi) /T)
p(ye1
F
(9)
Ee exp (cos (F', Fi)/T)
Pos-Bica (y)l))) = ( By + B,, (15)
Subsequently, we identify which spatial regions of the test sample are most relevant the target classes and desig- nate these regions as spatial views for further optimization. Specifically, we compute a category- spatial feature map M as
4. Experiment
4.1. Experimental Setup
1 M
Benchmarks. Following standard practices in Test-Time Adaptation (TTA) (Shu et al., 2022; Feng et al., 2023; Zanella & Ben Ayed, 2024), our primary results encom- pass two key benchmarks for model generalization: Cross- Datasets Generalization and Domain Generalization. For
Σ Softmax (F)))))),
(10)
C
M E R represents a set of spatial sequences enriched with category information, where each element indicates
5

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
Table 1. Comparison of GS-Bias in Cross-Datasets Generalization using ViT-B/16 (Dosovitskiy, 2020) as the backbone. Bold values indicate the best performance, while and $ refer to results reported in (Shu et al., 2022) and (Feng et al., 2023), respectively. E. denotes the use hand-crafted prompts suggested in CLIP (Radford et al., 2021a). BS represents the number of augmented views.
Method
Flowers102
DTD
Pets
Cars
UCF101
Caltech101
Foud101
SUN397
Aireraft EuroSAT
Average
CLIP-ViT-B/16
67.28
44.44
88.06
65,28
65.03
92.94
83,82
62.59
23.82
41.38
63.46
C.
71.34
44.39
89.10
65.91
66.67
94.12
86.01
66.29
24.84
47.72
65.64
CoOp"
68.71
41.92
89,14
64.51
66.55
93.70
85,30
64,15
18.47
46.39
63.88
CoCoOp"
70.85
45.45
90.46
64,90
68.44
93.79
83.97
66.89
22.29
39.23
64.63
TPT (BS = 64)
69.31
46.99
87.38
65.99
68.01
94.10
84.73
65,43
23.27
42.81
64,80
DifTPT+ (BS- 128)
70.10
47.00
88.22
67.01
66.69
92.49
87.23
65.74
25.60
43.13
65.47
MTA (BS = 64)
67.64
45.15
87,90
67.31
68.22
94.00
84.61
65.
23.91
41.35
64,43
MTA + E. (BS- = 64)
71.34
44.68
89.37
66.43
67.33
94.32
86,25
66.82
25.23
47.75
65.95
GS Bias (BS = 8)
68.86
45.10
88.58
66.77
65.74
94.16
85.67
64.78
25.30
43.63
64.86
GS-Bias + E. (BS- 8)
71.94
46.10
90.38
67.33
67.59
94.60
86.09
67.40
26.49
52.42
67.03
the Cross- Cross-Datasets Generalization benchmark, we assess performance on 10 diverse classification datasets, covering a broad spectrum of visual recognition tasks. These include datasets for plant and animal species (Flowers 102 (Nilsback & Zisserman, 2008) and OxfordPets (Parkhi et al., 2012)), transportation (StanfordCars (Krause al., 2013) and FGVC-Aircrat (Maji et al., al,, 2013)), food (Food101 (Bossard et al., 2014)), satellite imagery (EuroSAT (Helber et al., 2019)). human actions (UCF101 (Soomro, 2012)), tex- ture (DTD (Cimpoi et al., 2014)), scene recognition (SUN397 (Sun et al., 2020)), and general object classifica- tion (Caltech101 (Fei-Fei al., 2004)). the Domain Gen- eralization setting, we evaluate our approach on four out-of- distribution (OOD) variants of ImageNet (Deng et al., 2009): ImageNetV2 (Recht et al., 2019), ImageNet-Skktch (Wang et al., 2019), ImageNet-A (Hendrycks et al., 2021b), and ImageNet-R (Hendrycks et al., 2021a).
elaborate ensemble prompt described in (Radford et al., 2021a). It is important to highlight that due to the learnable nature of textual descriptions in the prompt optimizer, imple- menting an ensemble setup in this context is not straightfor- ward. Therefore, we incorporate the ensemble of prompts separately into our approach. Aligned with prior TTA works (Shu et al., 2022; Zanella & Ben Ayed, 2024), we adopt random cropping as the data augmentation strategy. Specifically, for the unseen cross-dataset generalization, we obtain a set of augmented views with BS = 8 and set the views selection rate to p = 0.5. For the out-of outtof-distribution domain generalization, the number of augmented views is increased BS = 64, and the views selection rate is ad- justed to p 0.3. For the learning of GS-Bias, the number of important spatial regions K in Eq. 12 is fixed at 16. These biases are optimized with 5 steps during test phase. The learning rates for the biases in Eq. 7 and Eq. 14 are set to a = 1 and B = 1 for cross-domain generalization, whereas for domain generalization, a 10 and = 1. Across all experiments, top-1 accuracy (%) is used as the evalua- tion metric, which is standard measure for classification performance.
Baselines. We compare our proposed method, GS-Bias, with several state-of-the -hetttr-rrt approaches across two bench- marks. The comparison includes zero-shot CLIP (Radford et al., 2021a), two representative training-time adaptation methods (CoOp (Zhou et al., 2022b) and CoCoOp (Zhou et al., 2022a)), and three test-time adaptation methods (TPT (Shu et al., 2022), DiffTPT (Feng et al., 2023), and MTA (Zanella & Ben Ayed, 2024)), all tailored for vision- language models. Notably, CoOp and CoCoOp are trained on the ImageNet (Deng al., 2009) dataset, with 16 views per class, and then transferred to downstream datasets for evaluation. In contrast, test-time adaptation methods per- form adaptation for each test sample without the need to access the training data.
4.2. Comparisons with State-of-the-art
Results on the Cross-Datasets Generalization. We re- port the quantitative results of various methods for cross dataset generalization on 10 benchmarks. Table 1 shows our GS-Bias achieves the highest average accuracy and demon- strates superior performance in 6 out of the 10 datasets. In particular, GS-Bias consistently outperforms CLIP across all 10 datasets, a performance trend not observed with other baselines. For example, TPT experiences a performance drop on the Aireraft dataset, while MTA faces similar issue on the Pets dataset. This highlights the potential risks of modifying the intrinsic visual and textual representations of pre-trained CLIP, which may lead to performance degra- dation. In contrast, our method enhances performance by
Implementation details. all experiments, we use the publicly available pre-trained CLIP model, with ViT-B/16 (Dosovitskiy, 2020) as the backbone and Transformer-based text encoder (Vaswani, 2017). For each test image, we evaluate two hand-crafted prompts: the ba- sic prompt "a photo of a [class]. and the more
6

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
Table Comparison of GS-Bias Domain Generalization using ViT-B/16 (Dosovitskiy, 2020) the backhone. Bold values indicate the best performance, while i and refer to results reported in (Shu et al., 2022) and (Feng et al., 2023), respectively. E. denotes the use of hand-crafted prompts suggested in CLIP (Rafford et al., 2021a). The two evaluation metrics, Average and OOD Average, are computed by calculating the mean accuracy across all five datasets, as well as the four OOD datasets, excluding ImageNet.
Method
ImageNet-A ImageNet
ImageNet-V2
ImageNet-R
ImageNet-S
Average
OOD Average
CLIP-ViT-B/EE
66.73
47.87
60.86
73.98
46.09
59.11
57.20
CLIP+E,
68.77
51.05
62.21
77.54
48.25
61.56
59.76
CoOpt
71.51
49.71
64.20
75.21
47.99
61.72
59.28
CoCoOp!
71.02
50.63
64.07
76.18
48.75
62.13
59.91
TPT
68.48
54,49
63.26
75.51
47.90
61.93
60.29
DiffTPT!
70.30
55.68
65.10
75.00
46.80
62.28
60.52
MTA
69.32
57.20
63.60
76.88
48.53
63.11
61.55
MTA B.
70.30
57.51
64.22
78.50
49.89
64.08
62.53
GS-Bias
69.02
54,55
63.37
76.64
48.21
62.36
60.69
GS-Bias + E. 70.57
56.61
64.62
80.49
50.33
64.52
63.01
Table 3. A comparison of the efficiency and effectiveness of our GS-Bias on ImageNet shown. * indicates results from repro- ducing the official released code, while represents results from our optimizations, including prompt ensembling and text encoder reuse. All experiments conducted single NVIDIA A800 GPU.
Table 4. Ablation study of global bias By and spatial bias B
B,
B,
ImageNet
10 Cross-Datasets
Average
68.77
65.64
67.21
/
69.10
65.77
67.44
70.45
66.07
68.26
70.57
67.03
68.80
Method
FPS(Image/Seccon)
Memory(Mii) Accuracy(5)
TPT' MIAA MTA E. 12.55
1.38
19997
68.48
son efficiency and effectiveness on ImageNet, evaluated on a single NVIDIA A800 GPU using the officially re- leased code. As shown Table 3, our GS-Bias benefits from lightweight backpropagation at the CLIP output logit level, achieving FPS approximately 10 times faster than TPT, with only 6.5% of the memory overhead. Notably, compared to the parameter-frre frreetter-free MTA, we achieve superior performance at nearly identical inference speeds.
8.59
3576
69.32
1448
70.30
GS-Bias
12.34
1308
70.57
supplementing bias knowledge while preserving the origi- nal CLIP output. Compared to the few-shot prompt tuning methods CoOp and CoCoOp, GS-Bias achieves substantial improvements 3.15% and 2.40%, respectively, without requiring access to training data. Compared to the prompt optimizer TPT, GS-Bias demonstrates superior performance. When combined with prompt ensembling. it further exceeds the performance of Dif which incorporates diffusion model. Furthermore, we achieve a 1.08% improvement over the visual optimizer MTA. These improvements verify that our GS-Bias can showcase stronger generalization on cross-dataset benchmarks.
4.3. Ablation Study
In this section, we provide a comprehensive analysis of the advantages of GS-Bias, examining the effectiveness of both global and spatial biases, and exploring the impact of different hyperparameter settings. All ablation studies conducted across 11 benchmark datasets, including Im- ageNet and 10 datasets in cross-dataset benchmark. The experimental setup is consistent with those outlined in Sec- tion 4.1.
Results on the Domain Generalization. We further assess the generalization capacity of GS-Bias across domains ImageNet and its four variants. As shown in Table 2, GS- Bias demonstrates sigmificant performance improvements across all scenarios. Compared to CoOp and CoCoOp trained on ImageNet, our method achieves improvements of 3.73% and 10% in OOD accuracy, respectively. When compared other TTA methods, GS-Bias leads in both average and OOD accuracy, highlighting the robustness of approach to domain shifts.
The effects of Global Bias and Spatial Bias. To verify the effectiveness of the proposed method, we examine the contributions global bias and spatial bias. As shown in Table 4, the introduced global bias captures global semantics by aligning the augmented views for consistency, signifi- cantly enhancing CLIP's performance. The spatial bias, on the other hand, facilitates semantic exploration at the re- gion level the test image, further improving performance, particularly on unseen cross-dataset benchmarks. The com-
Efficiency and Effectiveness. We also provide compari-

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Languant Models
20-57
70
02
Accuracy (%)
60
Accuracy
60
30
68
66 6788
Average
66 03
Avenge
ImageNet
55
ImageNet
10 Cross- Datasets
10 Cross-Dall Datasets
66
66
16
TTA Steps
Figure 3. Ablation study on the hyperparameter K in Eq. 12, where
Figure 4. Performance of GS-Bias at different TTA steps.
K 0 represents the performance of pa(y/ar (z) in Eq.
dataset generalization benchmarks, highlighting the bene- ficial role of spatial information in enhancing unseen class recognition. However, performance starts to decline when K' exceeds a certain threshold. This can be attributed to the fact that the foreground regions used for classification occupy only a portion of the test image, and selecting too many regions may introduce irrelevant background noise, which harms performance.
Table Ablation study on the hyperparameters a and / in Fq.7 and Eq. 14.
(8 =
5
10
15
20
ImageNet
70.08
70.51
70.57
70.55
70.54
10 Cross-Datasets
67.03
66.30
66.26
66.11
66.00
Average
68.56
68.41
68.42
68.33
68.27
B (a I)
0.5
1.5
ImageNet
70.04
70.08
70.12
70.17
69.97
The effects of TTA steps. We also studied the impact of TTA steps performance, where TTA steps refer to the number of backward propagation iterations. Unlike TPT, our method performs optimization solely at the CLIP output level, allowing for multi-step TTA with minimal overhead. In Figure 4, the accuracy on ImageNet remains stable after the first step of optimization. cross-datasets benchmarks, accuracy initially improves with increasing TTA steps, but begins to decline beyond certain point, indicating that excessive learning steps may lead to overfitting during TTA.
10 Cross-Datasets
66.15
66.48
67.03
66.97
66.67
Average
68.10
68.28
68.58
68.57
68.32
bination of both biases effectively enables high-performance TTA.
contrast, on
The effects of a and B. The test-time learning of ES-Bias controlled by a and 8, which represent the learning rates for global bias and spatial bias, respectively. To determine the effective configuration, we fixed one learning rate (i.e., = or 3 1) and observed how performance varied with changes in the other. As shown in Table 5, increasing the learning rate a leads to a continuous performance de- cline for GS-Bias on cross-dataset benchmarks, while on ImageNet, performance improves rapidly before stabilizing. This observation suggests that excessive capture of global information lead overfitting in unseen class scenarios, thereby impairing performance generalization. Conversely, learning more global semantics is crucial for general object classification. Furthermore, as learning rate 3 increases, GS- Bias exhibits consistent performance improvements across both benchmarks before eventually declining. This pattern indicates that learning spatial semantics enriches visual con- cepts, but excessive foeus spatial semantics can disrupt the understanding of global semantics.
5. Conclusion
In this work, we propose a novel Global-Spatial Bias Learner (GS-Bias) for test-time adaptation of vision- language models. Instead of modifying CLIP's textual prompts or visual features, our GS-Bias introduces two learnable biases directly at the output logits of CLIP. Partic- ularly, the global bias captures the overall semantic features of test image by aligning augmented views for consis- tency, while the spatial bias focuses on ensuring semantic coherence among regions within the spatial visual represen- tation of the image. Moreover, our optimization operates solely at the CLIP output level, making the approach very lightweight. Extensive experiments demonstrate that GS- Bias consistently outperforms several ttate-of-the-art meth- ods across all scenarios. We hope this study will inspire additional advancements in the field of multimodal learning and test-time adaptation.
The effects of K. We investigate the effect of K in eq. 12, which controls the scope of semantic consistency within the selected regions. As shown in Figure 3, recognition per- formance improves with increasing K, especially in cross-
8

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
Acknowledgement
Dosovitskiy, A. An image is worth 16x16 words: Trans- formers for image recognition scale. arXiv preprint urXiv22100111922 2020.
This work was supported by the National Science Fund for Distinguished Young Scholars (No. NN0.63055633, the Na- tional Natural Science Foundation of China (No. U21B2037,
Fei-Fei, L., Fergus, R., and Perona, P. Learning generative visual models from few training examples: An incremen- tal bayesian approach tested 101 object categories. In CVPR, 2004.
No. U22B2051, No. U23A20383, No. A2142047 No. 62176222, No. 62176223, No. 62176226, No. 62072386,
No. 62072387, No. 62072389, No. 62002305 and No. 62272401), and the Natural Science Foundation of Fujian Province of China (No. 2021J06003, No. 2022J06001).
Feng, C.-M., Yu, K., Liu, Y., Khan, S., and Zuo, W. Diverse data augmentation with diffusions for effective test-time prompt tuning. In ICCV, 2023.
Impact Statement
Fu, S., Wang. X., Huang, Q. and Zhang, Y. Nemesis: Normalizing the soft-prompt vectors of vision-language models. In ICLR, 2024.
This paper proposes GS-Bias, a Test-Time Adaptation (TTA) method for vision-language models that strikes balance between performance and efficiency. The TTA community may benefit from the insights and findings presented in our research. There are many potential socictal consequences of our work, none which we feel must be specifically high- lighted here.
Gao, P., Geng, S., Zhang, R., Ma, T., Fang. R., Zhang, Y., Li, H., and Qiao, Y. Clip-adapter: Better vision- saiiunlanguage models with feature adapters. I/CV, 2024.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR, 2016.
References
Helber, P., Bischke, B., Dengel, A., and Borth, D. Eurosat:
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lene, K., Mensch, A., Millican, K., Reynolds, M. et al. Flamingo: visual language model for few-shot learning. NeurlPS, 2022.
A novel dataset and deep learning benehmark for land use and land cover classification. IEEE Journal of Se- lected Topics in Applied Earth Observations and Remote Sensing, 2019.
Bossard, L.,, Guillaumin, M., and Van Gool, L. Food-101- mining discriminative components with random forests. In ECCV, 2014.
Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F,, Dorundo, E.,, Desai, R., Zhu, T., Parajuli, S., Guo, M., al. The many faces of robustness: A critical analysis of -iitrf--istributtion generalization. In ICCV, 2021a.
Chen, G., Yao, W., Song, X., Li, X., Rao, Y., and Zhang, K. Plot: Prompt learning with optimal transport for vision- language models. In ICLR, 2023.
Hendrycks, D.,, Zhao K., Basart, S., Steinhardt, J., and Song, D. Natural adversarial examples. In CVPR, 2021b.
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual rep- resentations. In Int. Conf. Mach. Intell., 2020.
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T. Parekh, Z., Pham, H., Le, Q., Sung, Y.H.,, Li, Z., and Duerig, T. Scaling up visual and vision- saiiunla representation learning with noisy text supervision. In Int. Conf. Mach. Intell., 2021.
Chen, Z., Pan, Y., Ye, Y., Lu, M., and Xia, Y. Each test image deserves a specific prompt: Continual test-time adaptation for 2d medical image segmentation. CVPR, 2024.
Karmanov, A., Guan, D., Lu, S., El Saddik, A., and Xing, E. Efficient test-time adaptation of vision-language models. In CVPR, 2024.
Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. Describing textures in the wild. In CVPR, 2014.
Krause, J., Stark, M., Deng, J., and Fei-Fei, L. 3d object rep- resentations for fine-grained categorization. In Proceed- ings of the IEFE international conference on computer vision workshops, 2013.
Comaniciu, D. and Meer, P. Mean shift analysis and appli- cations. In ICCV, 1999.
Lafon, M., Ramzi, E., Rambour, C., Audebert, N., and Thome, N. Gallop: Learning global and local prompts for vision- language models. ECCV, 2024.
Deng, J., Dong, W., Socher, R., Li, L.JJ, Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
Lin, H., Zhang, Y., Niu, S., Cui, S., and Li, Z. Monotta: Fully test-time adaptation for monocular 3d object detec- tion. In ECCV, 2025.

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
Liu, Z., Sun, H., Peng, Y., and Zhou, J. Dart: Dual-modal adaptive online prompting and knowledge retention for test-time adaptation. In AAAI, 2024.
Wang, H., Ge, S., Lipton, Z.., and Xing, E. P. Learning ro- bust global representations by penalizing local predictive power. NeurlPS, 2019.
Maji, S., Rahtu, E., Kannala, J., Blaschko, M., and Vedaldi, A. Fine-grained visual classification of aircraft. arXiv preprint 13x.vv3336..1.. 2013.
Yoon, S., Yoon, E., Tee, T. J., Hasegaaa-JJhhsse M. A., Li, Y., and Yoo, C. D. C-TPT: Calibrated test-time prompt tuning for vision-language models via text feature dispersion. In ICLR, 2024.
Miyai, A., Yu, Q., Irie, G., and Aizawa, K. Locoop: Few- shot out-of-distribution detection via prompt learning. NeurIPS, 2024.
Zanella, M. and Ben Ayed, I. On the test-time zero-shot generalization of vision-language models: Do we really need prompt learning? In CVPR, 2024.
Nilsback, M.-E. and Zisserman, A. Automated flower clas- sification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics image processing. 2008.
Zhang, C., Stepputtis, S., Sycara, K., and Xie, Dual prototype evolving for test-time generalization of vision- language models. NeurIPS, 2024a.
Parkhi, O. M., Vedaldi, A., Zisserman, A., and Jawahar, C. Cats and dogs. In CVPR, 2012.
Zhang, J., Huang. J., Zhang, X., Shao, L., and Lu, S. Histor- ical test-time prompt tuning for vision foundation models. NeurIPS, 2024b.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In Int. Conf. Mach. Intell., 2021a.
Zhang, M., Levine, S., and Finn, C. Memo: Test time robustness via adaptation and augmentation. NeurIPS, 2022a
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In Inl. Conf. Mach. Intell., 2021b.
Zhang, R., Zhang, W., Fang, R., Gao, P., Li, K., Dai, J., Qiao, Y., and Li, H. Tip-adapter: Training-free adaption of clip for few-shot classification. In ECCV, 2022b.
Zhou, K., Yang, J., Loy, C. C., and Liu, Z. Conditional prompt learning for vision-lanuuage models. In CVPR, 2022a.
Recht, B., Roelofs, R., Schmidt, L.. and Shankar, V. Do imagenet classifiers generalize to imagenet? In Int. Conf. Mach. Intell., 2019.
Zhou, K., Yang, J., Loy, C. C., and Liu, Learning to prompt for Saiiunla models. IJCV, 2022b.
Shu, M., Nie, W., Huang. D.-A., Yu, Z., Goldstein, T., Anandkumar, A., and Xiao, C. Test-time prompt tuning for zero-shot generalization in vision-language models. NeurlPS, 2022.
Soomro, K. Ucf101: dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:/212.0400 2012.
Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A., and Hardt, M. Test-time training with suff-superiii for gener- alization under distribution shifts. In Int. Conf. Mach. Intell., 2020.
Udandarao, V., Gupta, A., and Albanic, S. Sus-x: Training- free name-only transfer of vision-language models. In ICCV, 2023.
Vaswani, A. Attention is all you need. NeurlPS, 2017.
Wang D., Shelhamer, E.,, Liu, S., Olshausen, B., and Darrell, T. Tent: Fully test-time adaptation by entropy minimiza- tion. In Int. Conf. Mach. Intell., 2021.
10

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
A. Appendix
A.1. More Quantitative Analysis
Table Comparison of GS-Bias in Domain Generalization using ResNet50 (Hc ct al., 2016) as the backbone. Bold values indicate the best performance, while and refer to results reported in (Shu et al., 2022) (Feng et al., 2023), respectively. denotes the use hand-crafted prompts suggested in CLIP (Radford et al., 2021a). The two evaluation metrics, Average and OOD Average, are computed by calculating the mean accuracy across all datasets, as well as the four OOD datasets, excluding ImageNet.
Method
ImageNet
ImageNet-A
ImageNet-Y. V2
ImageNet-R
ImageNet-S
Average
OOD Average
CLIP-ResNet50
58.21
21.68
51.46
55.92
33.37
44.13
40.61
C.
60.33
23.49
53.31
60.59
35.50
46.64
43.22
CoOpT
63.33
23.06
55.40
56.60
34.67
46.61
42.43
CoCoOp
62.81
23.32
55.72
57.74
34.48
46.81
42.82
TPT
60.70
26.56
54.77
59.02
35.13
47.24
43.87
DiffTPT
60.80
31.06
55.80
58.80
37.10
48.71
45.69
MTA
60.37
27.35
52.72
58.63
34.45
46.70
43.28
MTA
61.44
28.02
54.88
61.88
36.37
48.52
45.29
SS-BBas
62.05
27.83
55.91
63.01
36.98
49.16
45.93
In this section, we report our GS-Bias along with quantitative analysis using ResNet50 (He et al., 2016) on domain generalization benchmark. To capture the spatial knowledge of ResNet50, we extract feature maps from the intermediate layer (with a size of 14214x1024). The experimental setup follows the same configuration outlined Section 4.1. The results in Table show that the GS-Bias, when equipped with ResNet50, achieves state-of-the-arr performance, further demonstrating the robustness of our method across different backbones.
A.2. Qualitative results
GT: Birman
PaLie (ylx)
B
B
Pes- Bias (v(x)
(0.4573) Ragdoll
(0.3416) Birman
(0.1338) Persiun
(0.7855) Birman
Pets
(0.391I) Birman
(0.2370) Ragdoll
(0.1312) Bombay
(0.7143) Ragdoll
(0.1193) Stamese
(0.1058) Basset Hound
(0.0580) Birman
(0.1599) Stamese
(Probability) Top-3
DHC-6
Pylx)
B
Pes-Bias(ylpp
(0.2725) DHC-8-100
(0.1222) DHC-1
(0.0465) DHC-6
(0.4020) DHC-6
Aircraft
(0.2642) DHC-6
(0.0913) DHC-6
(0.0248) DHC-1
(0.3593) DHC-1
(0.2123)DCC.
(0.0706) An-12
(0.0149) DHC-8-100
(0.2999) DHC-8-100
(Probabilit)) Top-3
GT: Axoloti
Paup Py|o
g
B
Pas (es-Bias(ylx)
ImageNet
(0.3250) Newt
(0.2251)
(0.0313) Axoloti
(0.3940) Axoledl
(0.2234) Fire Salamander
(0.082H)
(0.0211) Cradle
(0.3699) Newt
(0.1376) Axoloti
(0.0411)
(0.0110) Fire Salamander
(0.2387) Fire Salamander
(Probahility) Top-3
Figure 5. The inference examples of GS-Bias, which integrates the CLIP output Peize ()), global bias B, g. and spatial bias B..
We present several representative examples to demonstrate the effectiveness of GS-Ba Bias during inference. As illustrated in Figure 5, distribution shifts between training and test data often lead CLIP make biased predictions. In contrast, our method successfully rectifies such errors by incorporating both global bias and spatial bias. For instance, fine-grained

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
pet recognition task, CLIP misclassifies Birman Ragdoll. However, the global bias component confidently guides the model toward the correct prediction by emphasizing the high-level semantic discrepancy, thereby correcting the mistake. Similarly, in an aircraft recognition scenario, where the key distinguishing features are subtle and localized (e.g., fine-grained text on the fuselage), CLIP tends to fail due to its insufficient sensitivity to spatial cues. Our spatial bias module captures such fine-grained visual concepts effectively, leading to accurate recognition even under challenging conditions.
A.3. More A Ablation Studies
70-37
705
75
200
70
Accuncy (5)
Accmncy
69.0
63:03
-662
68.72
500
08.52
ImageNer
68.5
68
60
ImageNer-A
ImageNer-R
68.0
16
ImageNet-V2
ImageNer-8
Average
Average
ImageNNet
50
50,33
212
00D Avenage
07
Cross-Ditasets
10
20
40
50
32
128
Number o'Aggmented Views N
(a)
Figure 6. More ablation studies. (a) Ablation study the hyperparameter in Eq. for domain generalization. (h) Ablation study on the number of augmented views in Eq. 6.
The effects of a on domain generalization. In Figure 6 (a), we investigate the impact of a across multiple domain generalization datasets. We observe that as increases, different datasets exhibit distinct trends. For example, the performance ImageNet-A continuously improves, while that on ImageNet-S steadily declines. This suggests that the degree of global semantic learning varies across datasets. Adversarial datasets, for instance, require more extensive learning, while simpler datasets, like sketches, demand less.
The effects of N. Furthermore, we investigate how the number of augmented views affects performance. Intuitively, increasing the number of views enhances diversity and improves the overall representational quality. As shown in Figure (b), the performance of GS-Bias improves consistently both cross-dataset generalization and ImageNet N increases. The improvement ImageNet is especially notable. These results suggest that GS-Bias generalize well to unseen classes with only few views. However, a larger number of diverse views is crucial for recognizing the broader range of classes in ImageNet.
Table 7. The number of significant spatial regions across 11 datasets.
1lowers102
DID
Pets
Cars
UCH101
Caltech101
Food101
SUN397
Aireraft EuroSAT
ImageNet Average
K.
18.03
12.83
18.43
16.42
16.23
16.48
16.80
18.56
15.95
11.91
19.12
16.43
Analysis of spatial bias. In Table we analyze the number spatial regions that are significantly associated with the classification target across different datasets. Specifically, we normalize the similarity scores M Eq. 10 and define regions with scores greater than 0.1 as significant, denoted by K I. M, MM
mmin() 0.1). Furthermore, we compute the
average number of significant regions for each dataset, represented as Ka. The results indicate that the number of significant regions varies across different datasets. For example, the low-resolution satellite images EuroSAT appear blurry, making spatial information less distinguishable and resulting in fewer significant regions. In contrast, fine-grained datasets such as pet and flower recognition yield a greater number of significant regions. Furthermore, observe that the average number of significant regions across the 1l datasets is approximately 16. Notably, this setting also yields the best performance in Figure 3, making it reasonable and well-justified choice in our experiments.
12

---

GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
Table Comparison between our spatial views and small-scale crop-based augmented views.
Table 9. Quantitative analysis of the method equipped with histori- cal data stream.
Method
10 Cross-Datasets
ImageNet
Method
Memory
10 Cross-Datasets
ImageNet
Cropix1
66.02
70.49
TDA
1058M
67.96
69.50
Crop2x2
66.03
70.48
DPE
6560M
68.13
71.08
GS-Bias (w/o B.)
66.07
70.45
GS-Bias
1308M
67.03
70.57
GS-Bias
67.03
70.57
OS-Bias + TDA
1308M
68.30
70.96
Furthermore, we analyze the differences between spatial views and small-scale crop-based augmented views. this end, we generate 1 and 2 x 2 scale augmented views via random cropping and use them as substitutes for spatial views in the learning of spatial bias. As shown in Table our spatial bias effectively improves performance compared GS-Bias without spatial bias, especially in unseen cross-dataset generalization (66.07% vs. 67.03%), demonstrating its ability to capture visual concepts missed by global bias. In contrast, simply reducing crоp size does not achieve the same effect. This is because we learn spatial bias from the spatial features of the vision encoder, where each region inherently encodes rich contextual information. In comparison, image-level cropping produces isolated patches that lack spatial coherence. Moreover, our region selection is guided by the correlation between spatial regions and class descriptions, whereas cropping purely random.
Compatibility of GS-Bias with Historical Data Streams. Recent VLM-based TTA methods have leveraged historical data streams to achieve stronger performance. For instance, TDA (Karmanov et al., 2024) utilizes historical information to update dynamic queue for training-free TTA, while DPE (Zhang et al., 2024a) and HisTPT (Zhang et al., 2024b) further extend this paradigm by incorporating prototype learning and prompt tuning, respectively. In contrast, our GS-Bias operates solely on individual test samples without accessing historical data, making it more aligned with methods such as TPT (Shu et al., 2022) and MTA (Zanella & Ben Ayed, 2024). Notably, GS-Bias remains compatible with historical data streams, allowing for potential integration. In Table we report the performance of GS-Bias, TDA, and DPE across 11 datasets, along with their memory overhead ImageNet, where all methods adopt the same text prompts for fair comparison. We also present coarse combination of TDA and GS-Bias simulate the effect of incorporating historical data. The results show that our method achieves favorable balance between performance and efficiency. Moreover, when equipped with historical data, GS-Bias achieves further improvements without increasing memory consumption. This is because our bias learning is performed solely at the output level, effectively avoiding the need to store wradient-bearing objects in the queue.
A.4. Limitation
We further discuss the unexplored limitations of our proposed GS-Bias, which will be our future focus. Although GS-Bias achieves a favorable trade-off between performance and efficiency, one notable limitation is its reliance on empirically selected hyperparameters. Despite extensive ablation studies across diverse datasets indicating that GS-Bias relatively insensitive to these settings, hyperparameters still need to be reconfigured for different test samples. While this empirical strategy offers a practical baseline, it may not deliver optimal performance for every individual instance. Future work could explore dynamic, sample-specific hyperparameter adjustment mechanisms to further enhance the model's adaptability and generalization.
13


Reviews:
[{"review_id": "prycS1iZVm", "reviewer": "Reviewer_F3JS", "summary": "This paper introduces GS-Bias, a novel test-time adaptation (TTA) method for Vision-Language Models (VLMs). The approach aims to improve zero-shot generalization by learning two biases: a global bias that captures the global semantic features of a test image through consistency across augmented views, and a spatial bias that learns semantic coherence between regions in the image's spatial representation. GS-Bias adds these biases directly to the logits output of the pre-trained VLM, avoiding computationally expensive full backpropagation. The authors claim that GS-Bias achieves state-of-the-art performance on several benchmark datasets while being highly efficient in terms of memory usage.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: Provide a more comprehensive analysis of the number of spatial regions and the reasons for setting of the number of regions.**\n\n**A1:** Thank you for your insightful comments. In response, we have expanded our analysis by incorporating two new experiments:\n- **Exp1:** Computing the number of spatial regions that are significantly related to the classification target across 11 datasets.\n- **Exp2:** Extending Figure 3 by adding results for $K$ = 4, 128, and 196. (ViT-B/16 has 196 spatial regions)\n\nFollowing Eq.10 and 11 in the manuscript, we select the top-K regions for spatial bias learning by computing the similarity scores $\\boldsymbol{M}$ between all spatial regions and the class descriptions. Higher scores indicate stronger relevance to the classification target, while lower scores may correspond to irrelevant regions.\n\nIn **Exp1**, we normalize the similarity scores and consider regions with scores greater than 0.1 as significant, denoted by $\\tilde{K} = \\sum_{i} \\mathbb{1} \\left( \\frac{\\boldsymbol{M}_i - \\min(\\boldsymbol{M})}{\\max(\\boldsymbol{M}) - \\min(\\boldsymbol{M})} > 0.1 \\right)$. Furthermore, we compute the average number of significant regions for each dataset, represented as $\\tilde{K}_a$. The results indicate that the number of significant regions varies across different datasets. For example, the low-resolution satellite images in EuroSAT appear blurry, making spatial information less distinguishable and resulting in fewer significant regions. In contrast, fine-grained datasets such as pet and flower recognition provide a greater number of significant regions. Furthermore, we observe that the average number of significant regions across the 11 datasets is approximately 16.\n\n**Exp2** further confirms that as $K$ increases, performance initially improves, reaches a peak, and then starts to decline. This suggests that incorporating an appropriate number of spatial regions provides beneficial class-related information, whereas an excessively large $K$ (e.g., $K=196$) may lead to severe overfitting, causing the optimization to be trapped in irrelevant, misleading information.\n\nIn summary, we observe that $K$ = 16 achieved the best average performance and exhibited significant relevance, making it a reasonable and well-justified choice. We list the results as below.\n\n- **Exp1. The number of significant spatial regions $\\tilde{K}_a$ across 11 datasets.** \n|Method|Flower102|DTD|Pets|Cars|UCF101|Caltech101|Food101|SUN397|Aircraft|EuroSAT|ImageNet|Average|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|$\\tilde{K}_a$|18.03|12.83|18.43|16.42|16.23|16.48|16.80|18.56|15.95|11.91|19.12|**16.43**|\n\n- **Exp2. Results of different numbers of spatial regions $K$ across 11 datasets.** \n|$K$|0|4|8|16|32|64|128|196|\n|-|-|-|-|-|-|-|-|-|\n|10 Cross-Datasets|66.07|66.84|66.87|**67.03**|66.84|66.76|66.75|66.70|\n|ImageNet|70.45|70.52|70.51|**70.57**|70.48|70.42|70.38|70.35|\n|Average|68.26|68.68|68.69|**68.80**|68.66|68.59|68.57|68.53|\n\n**Q2: Comparison of efficiency in total computational time.**\n\n**A2:** Thank you for your valuable suggestions. To further demonstrate the practicality of our method, we have supplemented our analysis by reporting the total computation time of GS-Bias, MTA, and TPT across 11 datasets. Specifically, we set the augmentation batch size to 8 for cross-dataset generalization and 64 for ImageNet. All experiments were conducted on a single RTX 3090 GPU. The results indicate that GS-Bias achieves a significant speedup compared to TPT, while its total computational cost remains nearly identical to that of the parameter-free MTA. The detailed results are presented below.\n\n- **Comparison of efficiency in total computational time.**\n|Method|Flower102|DTD|Pets|Cars|UCF101|Caltech101|Food101|SUN397|Aircraft|EuroSAT|ImageNet|\n|-|-|-|-|-|-|-|-|-|-|-|-|\n|TPT|4min|2min|5min|23min|6min|4min|60min|103min|6min|10min|660min|\n|MTA|1min|1min|1min|3min|1min|1min|12min|9min|1min|3min|85min|\n|GS-Bias|1min|1min|2min|3min|2min|1min|12min|10min|2min|4min|88min|\n\n**Q3: How GS-Bias performs on more complex and fine-grained tasks？**\n\n**A3:** Thank you for your insightful suggestion. The idea of GS-Bias can be extended to other foundation models for various downstream tasks. For example, in segmentation tasks, a learnable bias $B \\in R^{1 \\times W \\times H \\times C}$ can be incorporated into the output segmentation mask $M \\in R^{1 \\times W \\times H \\times C}$, making it an updatable mask $\\tilde{M} \\in R^{1 \\times W \\times H \\times C}$. Applying a segmentation-specific test-time objective to $\\tilde{M}$ facilitates efficient bias learning.\n\nHowever, applying GS-Bias to more complex and fine-grained tasks requires designing a new test-time objective that aligns with the nature of the model and the specific downstream task. We plan to explore this direction further in future research.\n\nWe sincerely appreciate your profound comments and will incorporate the above discussion into the final version of the paper.", "claims_and_evidence": "The claims made in the paper are generally supported by the evidence provided.\n\nThe performance improvements over other TTA methods, as reported in Tables 1 and 2, seem consistent and significant, supporting the effectiveness claim.\n\nThe efficiency claim is supported by the memory usage comparison in Figure 1 (b) and the FPS comparison in Table 3.\n\nThe ablation studies in Table 4 and Figure 3, demonstrating the contributions of both global and spatial biases, are also convincing.\n\nHowever, the number of selected spatial region is limited. The authors could have more comprehensive analysis on the number of spatial region.", "methods_and_evaluation_criteria": "The proposed method, GS-Bias, is well-motivated. The idea of learning biases directly at the logit level is an efficient way to adapt VLMs during test time. The combination of global and spatial biases seems reasonable for capturing both overall semantics and local details.\n\nThe evaluation criteria are standard for this task. The paper uses several established benchmark datasets for cross-dataset generalization and domain generalization. Top-1 accuracy is a common metric for classification performance.", "theoretical_claims": "There are no theoretical claims.", "experimental_designs_or_analyses": "The experimental design appears sound.\n\nThe authors compare GS-Bias with several strong baselines, including zero-shot CLIP, training-time adaptation methods, and other TTA methods. \n\nAblation studies are conducted to analyze the contributions of different components of GS-Bias (global bias, spatial bias, hyperparameters).\n\nThe experiments cover a range of datasets and tasks, providing a comprehensive evaluation of the method's generalization ability.\n\nA direct comparison of efficiency in terms of total computational time is missing. Including this would enhance the practicality insight of GS-Bias.", "relation_to_broader_scientific_literature": "The paper does a good job of situating GS-Bias within the broader scientific literature.\n\nThe authors discuss the related works in prompt tuning, adapter-based methods, and test-time adaptation for VLMs.\n\nThey clearly explain how GS-Bias differs from and improves upon existing TTA methods by addressing the limitations of prompt tuning and visual optimization approaches.\n\nThe paper also cites relevant works on contrastive visual-language models and related techniques (e.g., MeanShift).", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n\n-The idea of learning biases at the logit level for test-time adaptation is novel and efficient.\n\n-The paper addresses an important problem (zero-shot generalization of VLMs) and proposes a practical solution that achieves state-of-the-art performance.\n\n-The paper is well-written and easy to understand, with clear explanations of the method and experimental results.\n\nWeaknesses:\n\n-The hyperparameter selections lack comprehensive analysis on the spatial region.", "other_comments_or_suggestions": "It would be interesting to see how GS-Bias performs on more complex and fine-grained tasks, such as object detection or semantic segmentation.", "questions_for_authors": "In the analysis of the selected regions, can you provide more reasons on the setting of the number of regions?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4pDYUTY1jz", "reviewer": "Reviewer_HBpX", "summary": "This paper introduces Global-Spatial Bias Learner (GS-Bias), a test-time adaptation method for vision-language models (VLMs). GS-Bias's main idea is to learn two biases at the output logits of CLIP:\n- Global bias that captures semantic consistency across augmented views of a test image\n- Spatial bias that learns semantic coherence between regions in the image's spatial representation\n\nExperiment results show that compared to previous methods (TPT, MTA), GS-Bias is more memory efficient and achieves better performance in general.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: A typo in the title of the paper.**\n\n**A1:** We sincerely appreciate your thorough and responsible review of our manuscript. We apologize for the typo caused by our oversight, and we will correct \"GB-Bias\" to \"GS-Bias\" in the final version. \n\n**Q2: Provide concrete inference examples to illustrate the effectiveness of the proposed method.**\n\n**A2:** Thank you very much for your insightful suggestions, which have encouraged us to present our work more intuitively. Following your advice, we have provided seven concrete inference examples, with images sourced from seven different datasets (ImageNet, ImageNet-A, ImageNet-S, Pets, Aircraft, Flowers102, and EuroSAT). For your convenience, we have placed these examples in the following anonymous link: [Inference Examples](https://github.com/anonymoussubmission74/Inference-Examples/blob/main/Inference_Examples.png).\n\nWe emphasize that GS-Bias consists of three key components: CLIP output, global bias, and spatial bias. To intuitively demonstrate the effectiveness of bias learning, we present the Top-3 probabilities and their corresponding categories for each component. The examples show that the combination of global and spatial biases effectively corrects the erroneous outputs of the original CLIP model.\n\n**Q3: The paper has limited discussion of potential failure cases or limitations.**\n\n**A3:** Thank you for your constructive comments, which has encouraged us to provide a more comprehensive discussion of GS-Bias. While GS-Bias achieves a well-balanced trade-off between performance and efficiency, one notable limitation is that the selection of hyperparameters is based on empirical choices. Although we performed ablation studies on the hyperparameters using as many datasets as possible and found that the model's performance is not sensitive to the hyperparameters, it is still necessary to reselect the hyperparameters for different test samples.\n\nFor instance, we set the learning rates of global and spatial biases to a fixed value of $ \\alpha = 1 $ and $ \\beta = 1 $ to achieve cross-dataset generalization. However, some samples may favor learning more from global information, while others may require a stronger focus on spatial information.  An empirically fixed setting might lead to suboptimal adjustments. To illustrate this more intuitively, we provide two failure cases (link: [Failure Cases](https://github.com/anonymoussubmission74/Inference-Examples/blob/main/failure_cases.png)), where fine-grained aircraft recognition tends to rely more on spatial information, whereas action recognition benefits more from global information.\n\nThus, we acknowledge that such empirical selection may not be optimal for every individual data sample, but it serves as a practical starting point. Future research could explore dynamic strategies for adjusting the balancing hyperparameters on a per-sample basis to further enhance model performance. \n\nOnce again, we sincerely appreciate your professional review. We will incorporate the discussion on potential failure cases and limitations into the final version and provide more concrete inference examples.", "claims_and_evidence": "The paper's claims about efficiency and performance improvements are well-supported by experiments.\n\n1. Efficiency claims: Figure 1 (b) and Table 3\n2. Performance improvements claims: Table 1 and Table 2.\n3. Besides, ablation studies effectively validate the contribution of both global and spatial biases: Table 4.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are appropriate for the task, as they are commonly used in the field.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The experimental design appears sound with appropriate controls and comparisons, validating the efficiency and performance improvements.", "relation_to_broader_scientific_literature": "I believe this work can benefit the field of VLMs as it provides a new approach for efficient test-time adaptation.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Weaknesses:\n1. There might be a typo (GB-Bias) in the title of the paper.\n2. The paper could benefit from showing concrete inference examples to illustrate the method's effectiveness, rather than purely relying on numbers.\n3. The paper has limited discussion of potential failure cases or limitations.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2TXzA2LQ0Y", "reviewer": "Reviewer_pUFB", "summary": "This paper introduces Global-Spatial Bias Learner (GS-Bias), a test-time adaptation (TTA) method designed to improve the zero-shot generalization of vision-language models (VLMs) like CLIP, while keeping computational costs low. The core innovation is the addition of two learnable biases—global bias and spatial bias—to the model’s output logits during testing, without the need of training data. Global Bias captures broad semantic patterns by aligning predictions across multiple augmented views of a test image, optimized via entropy minimization of high-confidence logits. Spatial Bias enhances local understanding by focusing on task-relevant regions within the image’s spatial features, ensuring the regional consistency. Both biases are applied directly to the pre-trained VLM’s logits, avoiding full-network backpropagation and making the method highly efficient.GS-Bias outperforms state-of-the-art TTA methods like TPT and MTA across 15 benchmark datasets, boosting cross-dataset generalization by 2.23% over TPT and domain generalization by 2.72%, while using only 6.5% of TPT’s memory on ImageNet. It excels in zero-shot and domain-shift scenarios, balancing performance and efficiency effectively. The method’s low memory footprint and fast inference speed make it practical for real-world use. In essence, GS-Bias offers a lightweight, powerful TTA solution that significantly enhances VLM generalization with minimal computational overhead.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate the reviewer’s encouraging and valuable comments on our paper. Below, we address the raised concerns.\n\n**Q1: The proposed global bias mechanism, which relies on global prediction consistency, is not a novel idea.**\n\n**A1:** We acknowledge that relying on global prediction consistency is not novel. However, our key innovation lies in introducing a **learnable global bias at the output level, which effectively addresses the efficiency bottleneck in prior methods**. Previous approaches require **modifying model inputs**, leading to substantial computational overhead. In contrast, our method operates **purely at the output level**, eliminating the need for expensive backpropagation while still ensuring effective test-time adaptation. Thus, our method strikes an effective balance between performance and efficiency. For instance, it improves cross-dataset and domain generalization by 2.23% and 2.72% over TPT, while using only 6.5% of its memory on ImageNet.\n\n**Q2: Could simply reducing the crop size yield a similar effect on spatial bias?**\n\n**A2:** Thank you for your question. In response, we conducted experiments on 11 datasets by reducing the crop size to $1\\times1$ and $2\\times2$ for spatial bias learning. The results show that our spatial bias effectively improves performance compared to GS-Bias without spatial bias, especially in unseen cross-dataset generalization (66.07% vs. 67.03%), demonstrating its ability to capture visual concepts missed by global bias. In contrast, simply reducing crop size does not achieve the same effect. \n\nThis is because we learn spatial bias from the spatial features of the vision encoder, where each region inherently encodes rich contextual information. In comparison, image-level cropping produces isolated patches that lack spatial coherence. Moreover, our region selection is guided by the correlation between spatial regions and class descriptions, whereas cropping is purely random.\n|Method|10 Cross-Datasets|ImageNet|\n|-|-|-|\n|$Crop_{1\\times1}$|66.02|70.49|\n|$Crop_{2\\times2}$|66.03|70.48|\n|GS-Bias (w/o $B_s$)|66.07|70.45|\n|GS-Bias|**67.03**|**70.57**|\n\n**Q3: Missed related works.**\n\n**A3:** We have conducted a comparison with state-of-the-art VLM TTA methods, including DPE, TDA, and HisTPT. As summarized below, TDA utilizes historical data streams to update a dynamic queue for training-free TTA, while DPE and HisTPT extend this by incorporating prototype learning and prompt tuning, respectively. In contrast, GS-Bias operates purely on a single image without accessing historical data, making it more aligned with TPT and MTA. Notably, GS-Bias remains compatible with TDA, allowing for potential integration. \n\nTo further clarify our advantages, we report the performance and memory cost of GS-Bias with TDA and DPE on ImageNet, and combine TDA and GS-Bias as a rough version with historical data flow. The results show that GS-Bias strikes a strong balance between performance and efficiency. Furthermore, when equipped with historical data flow, GS-Bias improves further without increasing memory usage. This is due to our bias learning operates solely at the output level, effectively avoiding the storage of gradient-bearing objects in the queue.\n|Method|Memory|Accuracy|\n|-|-|-|\n|TDA|1058M|69.5|\n|DPE|6560M|71.2|\n|GS-Bias|1308M|70.6|\n|GS-Bias-History|1308M|71.0|\n\n**Q4:  The performance of the original GS-Bias does not show substantial improvements over TPT and MTA.**\n\n**A4:** As shown in the table below, we aggregated the performance of the original GS-Bias across 15 datasets (Fig 1, Tab 1, and Tab 2 in the manuscript). With the same settings (BS=64), our method consistently outperforms others. More importantly, **GS-Bias requires only 6.5% of TPT’s memory on ImageNet and achieving a $10\\times$ speedup**, demonstrating much higher efficiency. Overall, even in its original form, GS-Bias achieves an substantial improvement for the trade-off between performance and efficiency compared to TPT and MTA.\n|BS|Accuracy|\n|-|-|\n|TPT|63.84|\n|MTA|63.99|\n|GS-Bias|**64.23**|\n\n**Q5: The positive effect of spatial bias is minimal?**\n\n**A5:** We elaborate on the effectiveness of spatial bias in **A2**. Regarding concerns about the performance gain, we clarify that spatial bias is designed to complement global bias, not to be used independently in CLIP. Since CLIP is trained on global features, its spatial distributions are smoother, leading to weaker gradient updates. Thus, applying spatial bias directly to CLIP has limited benefits.\n\n**Q6: More ablation studies on BS.**\n\n**A6:** We list all the results for BS=8,16,32,64, and 128 below. It turns out that a larger BS leads to better performance. GS-Bias outperforms both TPT and MTA even in the simplest setup.\n|BS|8|16|32|64|128|\n|-|-|-|-|-|-|\n|GS-Bias|64.86|64.93|65.26|65.16|65.38|\n|GS-Bias + E.|67.03|67.08|67.15|67.21|67.34|\n\nWe will include the added results and discussions in the final version as supplementary material.", "claims_and_evidence": "Supported Claims\nState-of-the-Art Performance:\nEvidence: Tables 1 and 2 show GS-Bias outperforming TPT, MTA, and training-time methods (CoOp, CoCoOp) across 15 benchmarks, with 67.03% average accuracy on cross-datasets and 68.80% on domain generalization.\nEfficiency Advantages:\nEvidence: Table 3 reports 12.34 FPS and 1,308 MiB memory usage, significantly better than TPT (1.38 FPS, 19,997 MiB).\nProblematic Claims or Gaps\n1. The concept of global prediction consistency, which underpins the global bias mechanism discussed in the contributions section, has already been extensively adopted in the TTA (Test-Time Adaptation) domain. Positioning this as a novel contribution lacks sufficient innovation, as it represents a well-established methodology rather than a substantive advancement in the field.\n2. The local bias mechanism proposed in the contributions section appears fundamentally equivalent to enforcing predictive consistency constraints at finer spatial scales. This raises a critical methodological question: Would comparable effects be achieved simply by reducing crop sizes in spatial bias augmentation operations, rather than implementing the described local bias paradigm? Given this equivalence, the claimed innovation of local bias demonstrates incremental advancement rather than substantiative methodological distinction.", "methods_and_evaluation_criteria": "The proposed method (GS-Bias) and its evaluation criteria are well-aligned with the problem and application of test-time adaptation (TTA) for vision-language models (VLMs). Here’s a breakdown of their rationale and suitability:\nProblem-Specific Design:\nCore Issue Addressed: Existing TTA methods struggle to balance performance (e.g., cross-dataset generalization) and efficiency (e.g., memory usage, inference speed).\nEfficiency vs. Performance Trade-Off:\nBy design, GS-Bias reduces memory usage to 6.5% of TPT and achieves 10× faster inference (12.34 FPS vs. 1.38 FPS), addressing the inefficiency of prompt optimizers and instability of visual optimizers.\nEvaluation Criteria and Benchmark Suitability\nThe paper evaluates GS-Bias on 15 datasets, covering two critical scenarios:\nCross-Dataset Generalization and Domain Generalization:Tests generalization to unseen classes and tasks, reflecting real-world scenarios where VLMs encounter novel distributions.Validates robustness to distribution shifts (e.g., adversarial corruptions, sketch-like inputs), critical for real-world deployment under dynamic conditions.\nEfficiency Metrics:\nMemory Usage and FPS are explicitly measured, aligning with practical constraints for edge or real-time applications.", "theoretical_claims": "While the study demonstrates that the merits in its application-oriented focus on Vision-Language Models (VLMs), this prioritization has resulted in a notable absence of formal theoretical propositions and their systematic substantiation. The methodological framework would significantly benefit from rigorous theoretical grounding to complement its empirical implementation, as current arguments remain predominantly heuristic rather than axiomatically derived", "experimental_designs_or_analyses": "Baseline Comparisons:\n\nStrengths: \nIncludes established methods (TPT, MTA, CoOp, CoCoOp) for fair comparison.\n\nWeaknesses:\n1. The manuscript lacks comparisons with the state-of-the-art VLM TTA methods, such as DPE, TDA, and HisTPT.  \n2. In Tables 1 and 2, the performance of the original GS-Bias method does not demonstrate substantial improvements over TPT and MTA; only after applying an ensemble with a hand-crafted template is a significant enhancement observed, which raises concerns about the intrinsic effectiveness of the proposed method.\n\n[1] DPE: Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models. NIPS 2024\n[2] TDA: Efficient Test-Time Adaptation of Vision-Language Models. CVPR 2024\n[3] HisTPT: Historical Test-time Prompt Tuning for Vision Foundation Models. NIPS 2024", "relation_to_broader_scientific_literature": "The paper's key contributions lie in introducing lightweight, learnable global and spatial biases that act directly on the CLIP model's output logits to achieve test-time adaptation without the heavy full-network back-propagation or complex visual feature optimization seen in methods like TPT, DiffTPT, and MTA. The global bias leverages the idea of multi-view semantic consistency to enhancing textual prompts and strengthen overall semantic representation, while the spatial bias utilizes local region information from the visual encoder to focus on target classes, thus boosting generalization across domains and unseen classes. This approach builds on and extends earlier research in zero-shot and cross-domain adaptation, improving both efficiency and robustness.", "essential_references_not_discussed": "There are indeed a few related works that, while not discussed in the paper, provide important context for its key contributions. For instance, although the paper emphasizes output-level adaptation via learnable biases, methods such as TENT (Wang et al., 2021) show that adapting only batch normalization parameters through entropy minimization can effectively counter domain shifts with minimal network changes. Furthermore, several recent VLM TTA methods that emphasize both accuracy and efficiency, such as DPE, TDA, and HisTPT, have not been discussed. These approaches exhibit significant improvements in performance and efficiency compared to earlier methods like TPT and DiffTPT, warranting more comprehensive discussion and comparative analysis.", "other_strengths_and_weaknesses": "Strengths:\n1. The paper is well-organized and is easy to read. Evaluation includes 2 set of experiments on 15 dataset which is comprehensive.\n2. Efficiency through Logit-Level Adaptation: The paper contributes a significant efficiency gain by restricting the adaptation to the logit outputs of the pre-trained model (CLIP). This approach is inspired by the desire to minimize computational overhead while still leveraging the benefits of test-time adaptation.\n\nWeaknesses：\n1. The proposed global bias mechanism, which relies on global prediction consistency, is not a novel idea since it is already widely used in the TTA domain. \n2. The local bias mechanism seems equivalent to enforcing consistency at a finer spatial scale, raising the question of whether simply reducing crop sizes would yield similar effects.\n3. The manuscript lacks comparisons with the state-of-the-art VLM TTA methods, such as DPE, TDA, and HisTPT.  \n4. In Tables 1 and 2, the performance of the original GS-Bias method does not demonstrate substantial improvements over TPT and MTA; only after applying an ensemble with a hand-crafted template is a significant enhancement observed, which raises concerns about the intrinsic effectiveness of the proposed method.", "other_comments_or_suggestions": "No", "questions_for_authors": "1. In Table 4, the ablation study indicates that spatial bias yields only about a 0.2% improvement on average. Does this suggest that the positive effect of spatial bias is minimal, or that it merely acts as a smaller-scale version of global bias? The authors should include additional experiments to substantiate the method's effectiveness.  \n2. In Table 1, GS-Bias is evaluated with a batch size of 8, while TPT and MTA experiments use a batch size of 64. The absence of BS=64/BS=128 results in the ablation study is concerning; the authors should provide further results or justification to address this discrepancy.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No"}}]

Author rebuttal:

Meta-review: This paper proposes GS-Bias, a novel test-time adaptation (TTA) method for Vision-Language Models (VLMs). The core idea of GS-Bias is to learn two types of biases at the output logits of CLIP: a global bias, which captures semantic consistency across augmented views of a test image, and a spatial bias, which models semantic coherence between regions in the image's spatial representation. Experimental results demonstrate that GS-Bias is memory-efficient and delivers strong performance.
Following the rebuttal, the paper received all positive recommendations (one Accept and two Weak Accepts). Concerns related to reduced crop sizes and the intrinsic effectiveness of the proposed method were adequately addressed by the authors. The Area Chair concurs with the reviewers' assessments and recommends accepting the paper.
Decision: Accept (poster)