Self-Bootstrapping for Versatile Test-Time Adaptation
Authors: Shuaicheng Niu
Guohao Chen
Peilin Zhao
Tianyi Wang
Pengcheng Wu
Zhiqi Shen
Venue: ICML 2025 poster (2025)
Keywords: test-time adaptation
Primary area: general_machine_learning->transfer_multitask_and_metalearning
TL;DR: This work seeks to develop a unified test-time adaptation (TTA) objective for a variety of tasks—classification and regression across image-, object-, and pixel-level predictions.

Abstract: In this paper, we seek to develop a versatile test-time adaptation (TTA) objective for a variety of tasks — classification and regression across image-, object-, and pixel-level predictions. We achieve this through a self-bootstrapping scheme that optimizes prediction consistency between the test image (as target) and its deteriorated view. The key challenge lies in devising effective augmentations/deteriorations that: i) preserve the image’s geometric information, e.g., object sizes and locations, which is crucial for TTA on object/pixel-level tasks, and ii) provide sufficient learning signals for TTA. To this end, we analyze how common distribution shifts affect the image's information power across spatial frequencies in the Fourier domain, and reveal that low-frequency components carry high power and masking these components supplies more learning signals, while masking high-frequency components can not. In light of this, we randomly mask the low-frequency amplitude of an image in its Fourier domain for augmentation. Meanwhile, we also augment the image with noise injection to compensate for missing learning signals at high frequencies, by enhancing the information power there. Experiments show that, either independently or as a plug-and-play module, our method achieves superior results across classification, segmentation, and 3D monocular detection tasks with both transformer and CNN models.

Self- Self-Boossrapping for Versatile Test-Time Adaptation
Shuaicheng Niu 12
Guohao Chen Peilin Zhao
Tianyi Wang'
Pengcheng Wu Zhiqi Shen'
Abstract
remains long-existed challenge (Hendrycks & Dietterich, 2019; Koh et al., 2021), drawing substantial interest from both research and industry. To address this, numerous meth- ods have been explored, including training- approaches like domain generalization (Shankar et al., 2018; Dou et al., 2019) and data augmentation (Hendrycks et al., 2020; Yao et al., 2022), as well as test-time techniques such as source- free domain adaptation (Qiu al., 2021) and test-time adap- tation (TTA) (Liang al., 2023), name just new.
In this paper, we seek develop a versatile test- time adaptation (TTA) objective for variety of tasks - classification and regression across image-, object-, and pixel-level predictions. We achieve this through a self-bootstrapping scheme that optimizes prediction consistency between the test image (as target) and its deteriorated view. The key challenge lies in devising effec- tive auggeenations//dteriraatinn that: i) preserve the image's geometric information, e.g., object sizes and locations, which is crucial for TTA on object/pixel-eveel tasks, and ii) provide suf- ficient learning signals for TTA. To this end, we analyze how common distribution shifts affect the image's 's information power across spatial fre- quencies in the Fourier domain, and reveal that wow-frequeccy components carry high power and masking these components supplies more learn- ing signals, while masking high-frequency com- ponents can not. In light of this, we randomly mask the low-frequency amplitude of an image in its Fourier domain for augmentation. Mean- while, we also augment the image with noise in- jection to compensate for missing learning signals at high frequencies, by enhancing the information power there. Experiments show that, either in- dependently or as a plug-and-play module, our method achieves superior results across classifica- tion, segmentation, and 3D monocular detection tasks with both transformer and CNN models.
Among these approaches, TTA (Sun et al., 2020; Niu et al., 2023; Iwasawa & Matsuo, 2021; Bartler et al., 2022; Liang et al., 2023; Wang et al., 2024) has emerged as a rapidly advancing research area. By adapting to each data point immediately after inference, TTA achieves minimal over- head for model updates, making it highly appealing to a broad audience in real-world applications. However, current TTA solutions remain limited in scope, supporting only a narrow range of tasks due to constraints in their underlying self-supervised learning objectives, as depicted below.
Self-leerming/eenrppy--seed TTA methods (Wang et al., 2021; Niu et al., 2022a; 2023) and prototype-based meth- ods (Iwasawa & Matsuo, 2021) focus on minimizing the entropy of predicted logits or maintaining class-wise proto- types for discriminative models, e-g., image classification, making them unsuitable for regression tasks. Alignment- based approaches (Mirza et al., 2023; Lin et al., 2023) con- duct adaptation by aligning feature statistics, e.g., mcan and variance, between the target and source data. While this ap- proach is more general and applicable a broader of tasks, it requires pre-calculating source statisties with access to source data, which raises data privacy concerns (Liang et al., 2020; Wang et al., 2021). Moreover, this method regular- izes statistics to align with the source but overlooks direct learning from test data, leading to limited performance in more complex scenarios, see Tables and 3.
1. Introduction
Despite deep learning evolving at an incredible speed, its ability to generalize to out-of-distribution (OOD) domains
Consistency-based methods (Zhang et al., 2022; Shu et al., 2022) are another major category of TTA, such as MEMO (Zhang et al., 2022), which optimizes prediction consistency across different augmented views of the input image. Their augmentation strategy typically follows the well-established practice in contrastive learning methods like MoCo (He et al., 2020) and SimCLR (Chen et al., 2020a), relying on techniques such as random cropping
'Equal contribution 1 'Corresponding author 'College of Comput- ing and Data Science. Nanyang Technological University, Singa- pore "Joint WeBank-NTU Research Institute on Fintech, Singapore School of Artificial Intelligence, Shanghai Jiao Tong University, China. Correspondence to: <shuaicheng.nin .nceeennnnnnuuee niu@@iiu ntu..du.sg>.
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

SPA: Self-Bootstrapinng for Versatile Test-Time Adaptation
and resizing. However, these augmentations can disrupt the overall content of the image, i.e., the image's geomet- ric obructure-objects sizes, locations, relative layouts, and etc. While this may not impact image-level predictions like classification, it shall fail on more fine-grained tasks, such as object detection, which require the precise predictions of each object's coordinates and sizes.
model has higher prediction confidence on the target strong image view than the deteriorated weak image view.
Main Novelty and Contributions 1) We propose a sim- ple yet effective active boltsbootsrappinn learning frame- work for TTA, which general to be used for classifica- tion and regression across imagefobjectppixe-leeee tasks, showing broad applicability. 2) We analyze how common domain shifts manifest in the Fourier frequency domain and, based on this analysis. propose geometry- augmenttations-lov fro-ruuuee amplitude masking and high-frequency noise injection. These augmentations supply learning signals for our self- bootstrpttgg adaptation across all spatial frequencies, significantly enhancing adaptation performance. 3) Extensive experiments across classifica- tion, segmentation, and 3D monocular detection with both transformer and CNN models demonstrate our superiority.
In this paper, we aim to develop new fully TTA method to adapt an arbitrary trained model, while requiring access to source data or altering the original training process. We build on the general idea of consistency-bant based learning, but extend it to a more unified, architecture aggnostic, and -agnosicc self- bootstrapping TTA approach, so that it is applicable to classification and regression tasks across image- object-, and pixel-level prediction models.
To be specific, in our self- cootsrootstrapinng TTA (SPA) frame- work, we use the predictions of the original image (strong view) as the target, which provide supervision to guide the model learning in making consistent predictions on a de- teriorated view (weak) of the same image. This process enhances the predictions of the weak image view, which, in turn, feeds back and improves the original predictions through shared model parameters. Here, the key challenge of making this self bootsttrappiigg learning scheme applica- ble to fine- tne-grained object- or pixel-level tasks, is designing effective augmentations-wwe which need to preserve the main image's 's geometric structure while introducing sufficient differences (learning signals) to deteriorate a given image.
2. Preliminary and Problem Statement
We briefly revisit fully TTA in this section for the conve- nience of our method presentation and put detailed related work discussions into Appendix A due to page limits.
Fully Test-Time Adaptation (TTA) In this paper, we fo- cus on the problem of fully TTA. Formally, given any model [( 0) trained on source data Dtrain = [(xi; yt) J.W i= fully TTA adapts f:: f) to testing data Drest (x3 j=1 with po- tential distributions shifts from Dtrain on the fly using some unsupervised learning objectives (Wang et al., 2021) L, i.e., ming Lf 0)), where @ C 0 are learnable parameters dur- ing TTA. Here, test samples arrive in an online data stream. The fully TTA process does not alter the original model training process or require access to source data, making it practical and easy to implement in real-world applications.
y:)]
To the ahove end, we propose randomly masking amplitudes in the image's s Fourier frequency domain for augmentation. Specifically, we analyze how common distribution shifts manifest in the frequency domain by comparing the radially averaged power spectral density (RAPSD) ( (Van der Schaaf & Van Hateren, 1996) of shifted and source domains in Figure 2, where RAPSD reflects image information power across spatial frequencies. We observe that original images typically exhibit low RAPSD (i.e,, low information power) at high frequencies, with several domain shifts further re- ducing it. Thus, masking figh-frequency amplitudes tends to provide limited learning signals (see Table 6). In contrast, images show high RAPSD at low frequencies, indicating that there will be a larger RAPSID difference before and after masking there, and thus can provide richer learning signals. Therefore, we only mask the low- fwwfereuency com- ponent of the amplitude. Moreover, to compensate for the lack of learning signals at high frequencies, we augment the image by injecting random Gaussian noise into it, which enhances RAPSD in the high-frequency range, thereby sup- plying learning signals across all frequencies. Lastly, to make SPA more stable and reliable, we introduce an active self-bootstrapping learning scheme-for tasks including classification heads, we perform adaptation only when the
Problem Statement and Motivation Existing fully TTA methods often suffer a limited application scope and are not general enough. For instance, entropy- based meth- ods ( (Wang et al., 2021; Niu et al., 2022a) are restricted to classification and not compatible with regression tasks. Aug- mentation consistency-brid based methods (Zhang et al., 2022) work well for image-level recognition but may tend to fail in object- and pixel-level prediction tasks, e.g., object detec- tion, where precise coordinates and dimensions of objects shall be destroyed by their commonly used augmentations like random cropping and resizing. In real-world applica- tions, tasks are typically various, and selecting or redesign- ing working TTA solutions for different tasks or models can be inconvenient or impractical. Therefore, in this paper, we aim to develop a more versatile fully TTA method that is task-agnostic, supporting both classification and regression across image-, object- and pixel-level prediction tasks.
2

---

SPA: Self-Bootstrapinng for Versatile Test-Time Adaptation
similarity loss, Eqn. (1)
low-frequency component
amplitude
gradient
92
gradient
FFT
random IFFT mask
sg
prediction head
(b) augmentation: low-frequency amplitude mask
test-time model
(1
+
x
aug (b) aug (c)
(a) cooetroottt TTA
(c) augmentation: high-frequency noise injection
Figure/ 1. Illustration of SPA method. (a) We conduct self- self-bootstrapping learning for TTA by maximizing prediction consistency from the weak augmentedddettriioaated views to the strong original image view. The augmentations are designed to preserve geometric structure by (b) randomly masking low-frequency components of the image's 's amplitude in the Fourier domain, and (c) injecting Gaussian noise into the original image to enhance the information intensity on high frequency. 'sg": : stop gradient. (I)FPT: (Inverse) Fast Fourier Transform.
3. Approach
signals are essential for guiding the prediction adaptation; otherwise, it risks degrading the model's performance.
We achieve the goal of task-agnostic and architecture- agnostic versatile TTA by establishing framework of self-bootstrapping adaptation with geometric structure- preserving weak-to-strong learning, namely SPA. Here, we begin with belf-bootstrapping which introduces a general idea of refining the model predictions, using its own outputs as training targets. To broaden its compatibility with object- /pixel-level downstream tasks requiring dense prediction, we propose to optimize the prediction consistency between the original image (strong), as targets, and a corresponding geometric structure-pu prreetrrrpr deteriorated image (weak). In SPA, we aim to create a weak image view that loses as sufficient signals
To make this self-bootstrapping TTA applicable to object- /pixel-level tasks, we define geometric structure-preserve random augmentations T. This differs from heavy geomet- ric augmentations required by prior self-supervised meth- ods, such as random cropping and resizing, which signif- icantly alter pixel-level content (e.g.. object locations and sizes) from the original image, making them unsuitable in our weak -t--ng bofostrapttttnggggggggg learning. For pre- sentation coherence, leave the augmentation details in Sect. 3.2. Given test image x, SPA creates augmented view v t(x) with t ~ T. We take v as the weak view and the original x as the strong view. We then minimize a similarity loss L. between the predictions of x and v using a confidence-aware selection function S(x, v) (for tasks including classification heads), which actively determines whether to perform optimization w.r.t given sample or pixel to mitigate the influence of unreliable supervisions. Formally, our seff-bootstrapping TTA formula is given by:
much information as for bofttpoottrr learning, while preserving the overall structural content of the image, such size and relative lo- cation- cation-making it suitable for finer object/pixel-
tasks. We depict the self- bootstraatgg scheme in Sect. 3.1 and augmentations in Sect. 3.2. The over- all details of SPA are illustrated in Figure and Algorithm
woometry-preserveeee
min S(v, x) (x): = t(x), t t~ T. (I) 0
3.1. Active Seff-Bootsrrapping Learning
We design SPA with inspiration from prior self-supervised contrastive methods like BYOL (Grill et al., 2020) and DINO (Caron et al., 2021), as their core learning scheme, bootstrapping, is not limited classification regression.
Here, Ø c 0 denotes the learnable model parameters. For the similarity loss L, (.))),, we adopt KI. divergence for clas- sification heads and L1 loss for regression heads. We also insert a new learnable projector before the final prediction head, following BYOL (Grill et al., 2020). prevent the model from converging to trivial solutions in the image-level classification task. This projector is initialized as identity mappings to enable warm start for the fully TTA process.
Unlike BYOL DINO which maximize consistency be- tween two randomly augmented views for representation learning, we extend this framework for TTA by introducing weak-to-strong learning, i.e., from a randomly deteriorated view (weak) to the original image (strong). This adjustment necessary because, in TTA, we are adapting well-trained model to new, out-of-distribution domains where reliable
The confidence-awand selection S(v,x) in Eqn. (1) is designed for when the given task comprises classifica- tion heads. For image classification tasks, let f(x, 0)
3

---

SPA: Self-Bootstrapinng for Versatile Test-Time Adaptation
(a) Noise
(b) Blur
(c) Weather
(d) Digital and Rendition
(e) Augmentation (ours)
Gausdan
10a
ahot
ispass
10a
10*
10a
10t
orginal
RAPSD
10a
Frer ani mank
10
100
10a
glam
aefoc.s
10a
dassc
10a
Noh. ficg
10
10
400m1
100
orghtrica
100
souree
rensilan
10
weat- at ong
10
1
2
10
10
Frequency
Frequency
Frequency
Frequency
Frequency
Figure 2. (a-d) Changes of radially averaged power spectral density (RAPSD) (Van der Schaaf & Van Hateren, 1996) under domain shifts. (e) SPA's 's coometry-preserviee augmentations reduce the RAPSD at low frequencies and enhance at high frequencies create deteriorated images for our self-bootstrapping learning. We separately select 512 images from the Source, ImageNet-R, ImageNet-CC (15 corruptions), to perform FFT, and visualize their mean RAPSD based on the spectrum amplitude.
max 0)) and (4,+()) (+) be an indicator function, S(v, x) aims to select samples whose prediction confidence of the strong view is higher than the weak view for optimization:
supply sufficient learning signals, as in Table 6. However, this is still not optimal. To further improve, we first analyze how amplitude changes when domain shift occurs below.
S(v, x) = II f(vee) (V;x),
(2)
How Common Domain Shifts Manifest in Frequency Do- main? We compare the radially averaged power spectral density (RAPSD) (Van der Schaaf & Van Hateren, 1996) differences between the out-of-distribution test samples and the in-distribution source samples. Here, RAPSD calculates the average spectral amplitude of an image across a concen- tric rectangle (within F (x) in Eqn. 3) at each frequency, reflecting the distribution of image information power across all frequencies, i.e., the higher RAPSD, the higher informa- tion power. From Figure 2 (a-d), common domain shifts typically exhibit the following changes of RAPSD: high- frequency increase (e.g., noise corruptions) or degradation (e.g., blur corrpptions, low-frequency degradation (e.g.. contrast), or minimal change across all frequencies (e.g-- rendition). In the following, we seek to devise general aug- mentation strategies for various types of domain shifts and based on the observations from Figure 2 (a-d), we draw the following designing motivations.
For segmentation and object detection tasks, the predictions are often made on the pixel-level f(x/v; (0.... this case, we conduct selection at the pixel level, and calculate overall similarity loss (:s-) by averaging on all selected pixels.
3.2. Preserrvn Augmentation
The core idea in SPA is to devise image geometric structure- preserving augmentations to enable self- celf-bootstrapping learn- ing on finer object-/pixel-leeel tasks. To this end, one can di- rectly exploit conventional image structure- preeterrir aug- mentations, e.g., contrast adjustment, brightness changes, and grayscale conversion. However, as in Table 6, these augmentations, either individually or in combination, failed to provide sufficient learning signals, were sensitive to the type of domain shifts, and even led model collapse.
To ensure the augmentations are geometric-preservie while still providing as much learning guidance as possible for TTA, we propose augmenting the image by randomly mask- ing its amplitude in the Fourier frequency domain. Let F()) and F (-) be the Fast Fourier Transform (FFT) and inverse Fast Fourier Transform (IFFT) operations, respectively. We denote the amplitude and phase component after FFT by
1) Low-frequency masking contributes more for SPA. Im- ages generally exhibit high RAPSD in the low-frequency range. Thus, the RAPSD difference before and after ap- plying low-freq mask is relatively large, making random low-freq masking an effective way to supply sufficient learn- ing signals, regardless of whether amplitude shifts degrade or remain unchanged, i.e., for all types of domain shifts.
(-)() and pP( (.). Then, using random amplitude mask to augment an image x is defined as
2) High-frequency masking yields limited gains in SPA or even slightly hinder adaptation. Images typically display low RAPSD at high frequencies, especially with further RAPSD degradation there, e.g., blur corruptions. In these cases, masking high-freq components fails to provide suffi- cient learning signals, since the absolute RAPSD difference before and after applying a high-freq mask is relatively small. From Table 6, it can easily lead to trivial solutions if we only mask the high- freq amplitude. Moreover, masking high-freq components (which lowers RAPSD) runs counter
F F- (m) (x),, FP (x))).
(3)
Here, o is element-wise multiplication, M (m) produces a random 0-1 matrix with the same size for masking. The proportion of 0 in M (m) is defined as a mask ratio m.
Applying Eqn. (3) to generate a deteriorated view to per- form weak-to-strong self-bootstrapping learning in Eqn. (1) already yields much better performance compared to con- ventional geometry- prreerrip augmentations that fail to
4

---

SPA: Self-Bottstrapinng for Versatile Test-Time Adaptation
Algorithm 1 PyTorch-style pseudocode of SPA.
High-Frequeney Noise Injection We inject Gaussian noise into the original image to further supply learning signals at high frequencies. Note that although noise in- jection contrasts with the amplitude shift direction of some domain shifts like blur corruptions-where high-frequency RAPSD degrades, we still apply it in these cases and ob- serve promising performance, see Table 6. This is because, for such corruptions, augmentations like high-frequent amplitude masking that align with the RAPSD degradation direction fail to provide effective learning signals. Instead, the availability of sufficient learning signals is more critical than maintaining consistency with the degradation direc- tion. Consequently, noise injection remains an effective and viable option. Formally, the augmentation is given by
1f, hf: low-frequancy, high-Ereyuancy
f a, m: \alpha for 1f size, mask ratio, Eqn. (4) # hyper-param \gamma for noise injection, Eqn. (5)
optimizer
learnalle params, 1r, momentum)
predictions =
for in rest_data_loader: # load data in online manner #tzucture-preservvnn random augmented views x1 = 1f =1g_ aug(x,
x2 - hf_aug (x, 9) m) # if amplituda mask, Eqn. (4) $ hf noise injection, Eq).
# poraaad-poopggate
y, yz, y2 = model (x), model (x1), model (x2)
predictions.
# active loss calculation, Eqn. (1)
use KL/L1 similarity for /le. heade y dedeach () stop gradient
loss = active Sasiee_smilarittyl loss (yl, y)
loss active similarity Lossyy___osssgg (y2, y)
loss backward)) I back-propagate optimizer. (tep()
update
optimizer.zere zero zeroggrr. (raa()
V/ = (1 7)) x + where c c~ (O, 1). (5)
return predictions
Here, (0, 1) is a multivariate standard normal distribution with the same size of image x, y is constant for injection.
to the direction of domain shifts that raise RAPSD (e.g.. noise-based corruptions), thereby slightly hindering adapta- tion performance, as results on noisy corruptions in Table 6.
4. Experimental Results
Datasets and Models For classification, we conduct exper- iments on four benchmarks, i.e., ImageNet-C (Hendrycks & Dietterich, 2019) (corrupted images in 15 types of main categories, with the most severe corruption level 5), ImageNet-R (artistic renditions of 200 Im- ageNet classes) (Hendrycks et al., 2021a), ImageNet- Adversarial (Hendrycks et al., 2021b) and ImageNet- Sketch (Wang et al., 2019). We use ViT-base (Dosovit- skiy et al., 2021), trained on ImageNet by timm repos- itory (Wightman, 2019), as the source model. For 3D monocular object detection, we follow MonoTTA (Lin et al., 2024) evaluate all methods on KITTI-C, con- structed from validation set of KITTI (Geiger et al., 2012) through the incorporation of 13 distinct types of data cor- ruptions (Hendrycks & Dietterich, 2019). Each corruption has 3,769 images by following the original training and validation split of MonoFlex (Zhang et al., 2021). We use the model trained on KITTI by MonoFlex (Zhang et al., 2021) the source model for TTA. For segmentation, we use the Segformer-B5 (Xie al., 2021) model trained on Cityscape dataset (Cordts et al., 2016) as the source model and perform TTA on ACDC dataset (Sakaridis et al., 2021).
3) Noise injection is an effective option for creating learn- ing signals at high frequencies for SPA. Though amplitude masking doesn't 't work at high frequencies, one can still aug- ment the image to generate deteriorated views through noise injection to supply learning signals at high frequencies for self-bootstrapping learning. This is because noise injection, which is able to preserve the core geometry information of given image, increases the information power/RAPPD at high frequencies, as in Figure 2 (a), thereby creating larger RAPSD differences before and after augmentation compared with high-freq amplitude masking.
Inspired by the above motivations, we derive our overall geometry- preeeeriip augmentation from both low- and high- frequency perspectives. to supply sufficient learning signals for our self- eeff-bootsrrappng TTA framework.
Low-Frequency Amplitude Mask Let r(.) represent a shift (sort) operation that moves the low-frequency com- ponents to the center of the amplitude, and r(-)-t be the inverse operation that returns the components their orig- inal positions. Based on Eqn. (3), augmentation with a low-frequency amplitude mask becomes
Compared Methods We compare SPA with: batch norm (BN) Adapt (Schneider et al., 2020); entropy-based: TENT (Wang et al., 2021), EATA (Niu et al., 2022a), SAR (Niu al., 2023) and DeYO (Lee et al., 2024); ActMAD (Mirza et al., 2023) aligns feature statisties between target and source data; COTTA (Wang et al., 2022) and DePT (Gao et al., 2023) adapt a given model via augmentation based consistency maximization and a teacher-student learning scheme, ROID (Marsden et al., 2024) and CMF (Lee Chang, 2024); MonoTTA (Lin
Vi
= v
(M (),
F (x))), (4)
where ((x m) is a random 0-1 low-frequency mask with the same size as x E ghxw", In ((, m). the center area of size ah x aw is a random 0-1 matrix with a mask ratio m and a is always set to 0.2, while the surrounding area is padded with Other notations are the same as in Eqn. (3).
5

---

SPA: Self-Bootstrapinng for Versatile Test-Time Adaptation
Noise
Blur
Weather Frost
Digital Elastic
Average
Method
Gauss.
Shot
Impul.
Defoc
Glass
Motion
Zoom
Snow
Fog
Brit.
Contr.
Pixel
JPEG
Acc.
Source
56.8
56.8
57.5
46.9
35.6
53.1
44.8
62.2
62.5
65.7
77.7
32.6
46.0
67.0
67.6
55.5
TENT
60.3
61.6
61.8
59.2
56.5
63.5
59.2
54.3
64.5
2.3
79.1
67.4
61.5
72.5
70.6
59.6
COTTA
63.6
63.8
64.1
55.5
51.1
63.6
55.5
70.0
69.4
71.5
78.5
9.7
64.5
73.4
71.2
61.7
EATA
62.2
63.4
63.4
60.5
61.2
66.0
63.5
70.3
68.4
73.1
79.8
67.0
69.7
75.2
73.4
67.8
SAR
59.2
60.5
60.7
57.5
55.6
61.8
57.6
65.9
63.5
69.1
78.7
45.7
62.4
71.9
70.3
62.7
ActMAD
61.3
62.8
63.2
55.9
55.7
62.7
61.7
70.8
68.8
73.5
80.8
62.3
67.8
74.8
73.0
66.3
DeYO
59.8
61.5
61.1
57.4
59.0
64.5
61.9
69.1
66.7
69.5
78.9
65.3
69.6
74,0
72.3
66.0
ROID
63.0
64.3
64.0
60.1
61.6
65.2
63.5
71.9
70.4
73.8
80.5
60.5
71.8
75.8
74.0
68.0
CMF
63.5
65.2
64.7
59.3
63.3
67.2
66.0
73.1
71.2
72.7
80.9
65.6
73.6
76.6
74.6
69.2
SPA (ours)
64,0
65,5
65.2
61.0
63.6
69.1
67.9
74.1
72.7
75.3
80.9
65.2
74.0
77.6
75.0
70.1-a.1
+ ActMAD
64,8
66.5
66.0
62.2
64.6
70.3
69.7
75.1
73.4
76.7
81.6
67.3
75.0
78.4
75.7
71.2_n.2
+ TENT
65.2
67.0
66.4
63.7
65.7
70.9
70.4
75.2
73.5
77.2
81.7
67.8
75.5
78.4
76.0
71.6-0.0
Table 1. Comparisons with state-of-the-an methods on ImageNet- C (severity level 5) with BaT- regarding Accuracy (%).
et al. 2024) and VDP (Gan et al., 2023) are TTA methods specified for 3D monocular detection and segmentation.
Method
R
A
Sketch
Avg. Acc. (96.7)
Source
59.5
50.5
44,9
51.6
TENT (Wang et al., 2021)
63.9
5288
49.1
55.3
CoTTA (Wang al., 2022)
63.5
52.2
50,0
55.2
Implementation Details We set the mask ratio m to 0.2 for all experiments. The noise factor 7 is set 0.4 for classification and 0.1 for segmentation and 3D detection. Following CoTTA (Wang et al., 2022) and TTnоTTA (Lin et al., 2024), we apply SGD on classification and 3D de- tection, and Adam on segmentation, using the learning rate of 10 100 10----0-//66 3/6 10 We only update norm layers following TENT. More details of SPA and details of base- line methods are put in Appendix B.2. The cource code is availiable at
EATA (Niu et al., 2022n)
67.5
54,3
52.1
58.0
AcIMAD (Mirza et al., 2023)
60.2
50.3
46.2
52.2
De (Lee al., 2024)
68.7
55.0
50.3
$8.0
SPA (ours)
68.2
55,4
53.4
59.0
EATA
70.4
55.0
55.0
60.1
Table 2. Comparisons on ImageNet-R/A/Skete with VTT-Base.
alignment, allowing it to be integrated with these approaches to enhance performance further, as demonstrated by SPA incorporating ActMAD and TENT. At last, from the results on ImageNett R/A/Skethh in Table 2, our SPA also achieves the best performance, further suggesting our effectiveness.
4.1. Image Classification
In this section, we validate our SPA on image classification. From results in Table SPA outperforms all considered baselines consistently on all corruptions of ImageNet-C,, highlighting its superiority. To be specific,
4.2. 3D Monocular Object Detection
This section validates SPA on 3D monocular object detec- tion (Zhang al., 2021), challenging task that involves detecting objects in single-camera images using 3D bound- ing boxes. This task comprises classification, to identify objects within each 3D bounding box, and regression, to predict the bounding box coordinates, dimensions, depths, and angles. We perform weak-to so-tonn self-bootstrapping learning to align the classification head (via KL loss) and align regression heads of bounding box coordinates, dims, and depths predictions (via L11 loss).
1) Compared to entropy-based methods such as TENT, EATA, SAR, and DeYO, SPA improves the state-of-the- art by 2.3% in average accuracy, achieving 70.1% 1% compared to EATA's 67.8%. Additionally, SPA offers the flexibility to handle regression tasks (as shown in Table 3) while the entropy- -based objectives can not; 2) Compared to COTTA, which also applies consistency learning between the original image and its augmented views, our SPA achieves signifi- cantly greater gains. This result highlights the effectiveness of our bolesbbotstrapiigg learning framework and demon- strates that our structure-preservingg augmentations are able to provide richer learning signals within this framework; 3) Compared to ActMAD, our method, SPA, does not require access to the source training data for calculating source statistics to achieve alignment, and thus is more general to be used in case of source data are unavailable. Despite this, SPA achieves higher performance than ActMAD, im- proving average accuracy from 66.3% to 70.1%; 4) The learning objective of SPA is in parallel and not conflict with existing objectives like entropy minimization and feature
From Table 3, SPA achieves the best average APan R40 over 13 corruptions across different object difficulty levels (casy, moderate, hard) for each class, suggesting our effectiveness. Even compared to MonoTTA (Lin et al., 2024), the lat- est TTA method tailored for 3D monocular detection, SPA shows clear performance gains, with improvements such as 1.4% for Cars and 2.4% for Pedestrians. This largely benefits from the generality of SPA, which introduces TTA loss to regression heads in addition to classification heads, unlike previous methods, MOnOTTA, TENT, and EATA, which focus solely on classification heads. Moreover, the 3D monocular detection task is highly imbalanced, with
6

---

SPA: Self-Bootstrapinng for Versatile Test-Time Adaptation
Car, AP@0.7, 0.5, 0.5
Pedestrian, AP@0.5, 0.25, 0.25
Cyelist, AP@0.5, 0.25, 0.25
Method
Easy
Mod
Hard
Average
Easy
Mod
Hard
Average
Easy
Mod
Hard
Average
Source
16.4
12.1
10.5
13.0
5.0
4.2
3.5
4.3
6.5
3.3
3.0
4.3
BN Adapt (Schneider et al., 2020)
33.2
23.0
19.2
25.1
9.7
8.1
6.7
8.2
12.9
6.6
6.0
8.5
TENT (Wang et al., 2021)
36.1
25.2
21.4
27.6
10.2
8.5
7.1
8.6
13.3
6.8
6.1
8.7
EATA (Niu el al., 2022a)
36.7
25.5
21.8
28.0
10.4
8.7
7.2
8.8
13.4
6.8
6.2
8.8
ActMAD (Mizza et al., 2023)
33.3
23,2
19.4
25.3
9.0
7.6
6.3
7.6
12.6
6.6
6.0
8.4
MonoTTA (Lin et al., 2024)
42.1
29.5
25.6
32.4
11,3
9.4
7.7
9.5
13.6
6.9
6.2
8.9
SPA (ours)
43.7
30.9
26.9
33.8
14.3
11.8
9.7
11.9.02
14.3
7.4
6.8
9.5.a1
* MonoTTA
45.7
32.8
28.6
35.7
20.1
14.9
12.3
10.1
12.4
14.7
7.5
6.9
9.7202
Table Comparisons 3D monocular object detection w.r.t. the average precision of 3D bounding boxes, denoted as APapl t).
The results are averaged over 13 corruptions of KITTI-C (e.g., Fog and Snow, see Appendix B.1 for more details) with MonoFlex (Zhang al., 2021) as the source model. The Intersection over Union (IoU) thresholds are set to 0.7, 0.5, 0.5 for Cars and 0.5, 0.25, 0.25 for Pedestrians and Cyclists, respectively.
Time: 1
Round Rain
Round 2
Round 3
Round 1-3
Method
Fug
Night
Snow
Avg.
Fog
Night
Rain
Snow
Avg.
Fog
Night
Rain
Snow
Avg.
Average
Source
69.1
40.3
59.7
57.8
56.7
69.1
40.3
59.7
57.8
56.7
69.1
40.3
59.7
57.8
56.7
56.7
TENT (Wang et al., 2)21)
69.1
40,2
60.0
57.3
56.7
68.4
39.1
60.0
56.4
56.0
67.6
37.9
59.7
55.3
55.1
55.9
CoTTA (Wang el al,, 2022)
70,9
41.1
62.4
59.7
58.5
70.9
41.0
62.5
59.7
58.5
70.9
40.8
62.6
59.7
58.5
58.5
DePT (Gao al., 2023)
71.0
40.8
58.2
56.8
56.7
68.2
40.0
55.4
53.7
54.3
66.4
38.0
47.3
47.2
49.7
53.6
VDP (Gan et al., 2023)
70.5
41.1
62.1
59.5
58.3
70.4
41.1
62.2
59.4
58.3
70.4
41.0
62.2
59.4
58.3
58.3
SPA (ours)
68.7
42.9
62.0
59.8
58.3
69.7
44.6
63.3
61.1
59.7
70.0
43.2
63.8
61.7
59.7
59.2.mn1
+ CoTTA
71.2
42.7
65.2
62.1
60.3
72.5
43.1
66.0
62.2
61.0
72.5
42,9
66.0
62.1
60.9
60.7_a.1
Table 4. Comparisons on segmentation under continual TTA. We report mloU (%, 1) on Cityscape-o--ACDC with Segformer-Bi BB5.
Pedestrian and Cyclist as minority objects. Methods that only target the classification head often over-optimize the majority class (e.g., Cars) while neglecting minority ones. Thus, these classification-nnly methods yield very marginal gains on Pedestrian and Cyclist. In contrast, SPA is some- how free from this limitation, and it can mitigate this im- balanced issue, achieving higher gains on minority classes, e.g. 2.4% higher AP aa on Pedestrian over MOnOTTA.
ImageNet.CC C.
KITTI-Fog (AP3o Ruo, %. f)
Acc. (%. f)
Car
Pedestrian
Cyclist
No Adapt Full SPA
55.5
7,8
2.0
3.6
70.1
34.4
11.1
9.7
w/o weak-to-strong Le., stop grad
4.9
24.0
7.3
wlo active selection, Egn. (2)
69.4
28.0
9.9
7.7
Low-frre amplitude mask
65.6
7.2
wio High-frey noise injection
67.7
31.4
10.3
9.4
Table 5. Ablation effects of components in SPA. We use VIT- Base/MonoFlex as the source model on ImageNet-C/KITTT--O
4.3. Image Segmentation
2022) that maximize prediction feature consistency be- tween two augmented views, SPA adopts weak-to-strong self self-bootstrapping learning paradigm, which learns from a weak (augmented) view to strong (original) view. This uni- directional learning plays a crucial role in our TTA approach, as it provides more reliable learning signals, which are es- sential in TTA contexts. Without this weak-to-strong mech- anism, bidirectional consistency learning results in drastic performance degradation. Second, The activation selection strategy in Eqn. (2) also helps to filter out partially unre- liable supervisions, based on the premise that predictions from the original image are more reliable than those from the weaker augmented views, and thus results in promising performance gains. Third, both our low-frequency ampli- tude mask and high-frequency noise injection augmentation strategies are effective individually, yet achieve the best performance when applied together.
We compare SPA with prior TTA methods on image seg- mentation in a continual adaptation setting (Wang et al., 2022). In this setup, the target domains of ACDC (Sakaridis et al., 2021) progress through an ordered sequence of -ii-NiggttRRin--SSSSS
repeated across rounds, to simulate the environmental changes encountered in real-life driving scenarios. Table 4 shows that SPA alone outperforms CoTTA, which requires 29 augmentations per sample, while SPA achieves better results with only two augmentations, highlighting its effectiveness and efficiency. Furthermore, when combined with COTTA, SPA improves the average mloU from 58.5% to 60, further demonstrating its supe- riority plug-ande play module.
4.4. Ablations
Effects of Components in SPA We ablate the effects of each component within SPA in Table 5. First, unlike previ- ous SSL methods (Chen et al., 2020a; He et al., 2020) and augmentation consistency-basee based TTA methods (Zhang et al.,
Effects of Different Seometrie-Preseevvnng Preserving Augmenta- tions in SPA The core idea in SPA is to devise image geometric cruuctue--peservvie augmentations to boost its applicability for -rainedd tasks like object detection
7

---

SPA: Self-Bootstrapping for Versatile Test-Time Adaptation
Noise
Blur
Weather Frost
Digital Elastic
Average
Aug. Choice
Gauss.
Shot
Impul.
Defoc
Glass
Motion
Zoom
Snow
Fog
Brit.
Contr.
Pixel
JPEG
Acc.
Greystyle
59.7
59.8
61.4
58.7
59.0
65.5
63.6
70.3
69.4
72.5
76.5
0.4
71.0
75.5
71.4
62.3
Brightmess
38.4
60.5
60.8
9.7
55.5
35.5
60.9
69.2
70.3
4.9
79.7
0.8
66.0
74.3
70.7
50.5
Contrast
34.9
61.3
61.6
45.0
58.2
65.0
61.5
68.1
70.0
72.2
78.7
0.4
66.9
74.5
70.9
59.3
d) Gaussian Blar
3.2
8.7
2.9
20.9
1.7
30.6
30.4
57.9
41.8
73.9
80.2
6.5
14.6
2.0
6.6
25.4
a) + c)
59.8
60.5
61.3
60.0
60.5
66.6
64.1
70.5
69.2
73.7
76.5
0.4
71.4
75.8
71.7
62.8
e) Freq Mask
60.8
62.5
61.7
57.8
59.4
66.5
63.9
69.9
68.0
74.8
79.6
65.6
70.7
75.3
72.7
67.3
D High- Free Mask
4.3
5.9
5.4
1.4
6.6
10.5
16.0
63.7
21.3
22.9
73.4
2.3
11.0
2.5
4.7
16.8
g) Low-Freq Mask
62.6
64.3
63.4
57.6
59.2
66.9
64.4
70,6
69.0
74.8
79.6
65.7
71.0
75.9
72.6
67.8
h) Noise Injection
63.6
65.5
65.2
57.5
61.2
66.5
63.9
73.6
72.1
68.4
81.0
21.6
71.7
77.1
74.8
65.6
g) + h) (ours)
64.0
65.5
65.2
61.0
63.6
69.1
67.9
74.1
72.7
75.3
80.9
65.2
74.0
77.6
75.0
70.1
Table 6. Effects of different image geometric penetuurepprserving augmentation choices under our self- coltssootttrapping learning framework. We report Accuracy (%) on ImageNet-C (severity level 5) with ViT-
ImageNet-C, Acc. (%,
KITTI-Fog, Aaau
(%.
ImageNet-C: Acc. (%.. 1)
KITTI-Fog (AP (APaojuuo %.
64
Method
Car
Pedestrian Cyclist
0.5
62.6
63.1
63.1
62.9
63
0,5
18.6
37.7
17.0
15.8
14.0
18
No Adapt
55.5
7.8
2.0
3.6
Mask ratio
0.4
62.9
63.3
63.4
63.3
63.2
62
Mask 1a
0.4
19.0
37.9
16.8
162
15.1
BN Adapt (Schneider et al., 2020)
Self-hootstrapiing lnarning of SPA using augoenntatins of:
n/a
23.3
8.6
9.7
0.3 63.4
63.7
63.7
63.6
63.5
61
0.3
18.7
18.0
16.9
16.2
14.6
16
Eqns. (4) and (5) (ours)
70,1
34.4
11.1
9.7
0.2-636
63.8
63.9
64.0 63.8
0.2
18.4
37.6
16.9
16.0
14.7
Random image mask
64.9
9.7
4.3
2.0
60
14
Aix
63.5
0.1 63.8 64.0 64.1 64.0 64.1
0.1
10.4
27.4
16.5
15.3
14.2
SimCLR augmentations
68.1
25.0
5.7
6.6
0.1 0.2 0.3 0.4 0.5
59
0.1
0.2
0.3
0.4
0.5
MoCu augmentations
64.0
27.7
6.5
6.7
Noise ratio y
Noise ratio V
Table 7. Comparisons with SPA using conventional augmentation strategies that do not preserve geometric structure. We use ViT- Base/MonoFlex as the source model on CraggNet-CCKKTTII-
Figure 3. Sensitivity of amplitude mask ratio m in Eqn. (4) and noise injection ratio y in Eqn. (5). We use ViT-Base for ImageNet- C (Gaussian Noise) and MonoFlex for KITTI-Fog. The source model Acc.AP on ImageNet- is 55.5%
strictly require content invariance, allowing for a higher y to provide richer learning signals. In contrast, 3D monoc- ular detection involves dense predictions where high noise levels could significantly disrupt the original image content, making it challenging for our self- self-bootstrapping learning.
and segmentation. Here, we compare our augmentation strategy with existing geometric-presservnng ones, including grayscale, brightness, contrast ( (ColorJittr),, and Gaussian blur. However, as in Table 6, none of them, individually or combined, effectively provide TTA with rich learning signals. These augmentations are also often sensitive to the corruption type and struggle to perform stably across all cor- ruptions, leading to limited overall performance. Moreover, as discussed in Sect. 3.2, SPA masks only the lowffrequeccy amplitude while keeping the high-frequency components unchanged, since the high-frequency range exhibits low RAPSD and masking there provide limited learning signals (whereas low frequencies are quite the opposite). To verify this, rows f) and g) in Table 6 show that solely masking high frequencies tends to yield trivial solutions, i.e., model collapse, while masking at low frequencies perform stably.
Comparison with Self-Bootstrapping Learning using Augmentations of Image Mask (He et al., 2022; Gan- delsman et al., 2022b), AugMix (Hendrycks et al., 2020), MoCo (He et al., 2020) and SimCLR (Chen et al., 2020a) We compare our proposed augmentation strategy with prior augmentations used in SSL and consistency-baset TTA methods under our self-bootstrapping TTA framework in Table 7. Results show that while prior augmentations achieve considerable performance on image-level classi- fication (ImageNet-C), they fall short for 3D monocular detection, where they often perform worse than BN Adapt. This arises because these augmentations like random crop- ping&resiizng disrupt the image's 's geometric structure, mak- ing them unsuitable for finer dense prediction tasks. In contrast, our augmentations are geometric-preserve and supply rich signals for weak-to-strong self- ceff-bootsrappping learning, achieving improved adaptation performance.
Parameter Sensitivity We evaluate SPA with different am- plitude mask ratio m (in Eqn. (4)) and noise ratio 7 (in Eqn. (5)) selected from (0.1, 0.2, 0.3. 0.4, 0.5). From Fig- ure 3, SPA works well across a wide range of m s 0.5 and 7 S 0.5 for image classification, showing its insensi- tivity. However, for 3D monocular detection, although m performs well within broad range from 0.1 to 0.5, the optimal range of noise ratio y is narrower than that in image classification, i.e., 7 & 0.2. This difference arises because, in ImageNet-C, the task is at the image level and does not
4.5. Further Results and Discussions
Comparisons with Methods Beyond Fully TTA Our SPA approach can adapt any pre-trained model off- thelf with- out any other requirements, which falls into the category of
8

---

SPA: Self-Bootstrapinng for Versatile Test-Time Adaptation
Model Method
Gauss.
Fog
Pisel
Snow
Contr.
Averae
Method
DTD
UCFI01
Aireraft Avg. Acc. (%)
Customized VIT-1/16 classifier
17.1
38.7
47.1
35.6
6.9
29.1
Source
44.3
65.1
23.8
44.4
+ TTT-MAE
37.9
51.1
65.7
56.5
10.0
44.2
VLM TTA methods:
VIT-B/32
39.5
35.9
55.0
30.0
31.5
38.4
TPT (Shu ct al., 2022)
46.7
67.3
23.4
45.8
+ Diffusion-TTA
46.5
56.2
64.7
50.4
33.6
50.3
+ SPA (Ours)
49.4
56.9
66.7
49,9
51.5
54.9
C-TPT (Yoon et al,, 2024)
46.0
65.7
24.0
45.2
Vision 77A methods:
Table Comparison with TTT-MAE and Diffusion-TTi on ImageNet-C regarding accuracy (%).
Tent
45.2
66.0
23.4
44.9
ETA
44.7
66.1
23.7
44.8
SAR
44.6
66.5
23.4
44.8
Method
Source
TPT C-TPT ETA SAR
DeYO
SPA
SPA+ETA 78.2
DeYO SPA (ours)
44.2
66.0
22.7
44.3
45.4
66.2
23.6
45.1
Ace (%)
74.0
77.1
76.0
76.9
75.6
76.6
77.2
Table 10. Results on CLIP-ViT-B for cross-dataset gross-dataseeennnrrizatioo
Table 9. Comparisons with baselines on ELIP-ViT-B for OOD robustness. We report results on ImageNet-R.
fully TTA. Here, we further compare SPA with methods be- yond fully TTA, Le., TTT-MAE (Gandelsman et al., 2022a) that requires modifying the original model training process and Diffusion-TTA (Prabhudesai et al., 2023) that relies on an additional pre-trained generative diffusion model and updates both the diffusion model and original discriminative model at test time. As shown in Table 8, our SPA, as fully TTA method, still achieves superior performance, further demonstrating our effectiveness.
5. Conclusion
In this paper, we aim to develop new versatile fully TTA ap- proach to support various tasks cooks-claassffiatiinn or regression across image-, object,, and pixel-level predictions. To this end, we re-establish a consistency-based TTA framework with active weak (augmented image)-to-strong (original im- age) supervisions, termed SPA. In SPA, by analyzing how domain shifts manifest in the Fourier frequency domain, we devise two Fourier-based augmentations: low-frequency am- plitude masking and high-frequency noise injection. These augmentations preserve the geometric structure of images (e.g., object locations and sizes), making them applicable for consistency learning fine-grained dense tasks while also supplying rich learning signals for TTA. Extensive ex- periments on classification, 3D monocular object detection, and segmentation verify the generality and superiority of SPA, both as a standalone approach and as a and-play module to enhance existing methods.
Effectiveness on Vision-Language Model (VLM) for both OOD and Cross-Datase Generalization We freeze the text encoder of the CLIP (Radford et al., 2021) model and treat it as fixed classifier, and then apply SPA to the image encoder. SPA primarily focuses on addressing the OOD issue in the visual modality through our deterioration-driven self-bootstrapping learning, by improving the visual fea- ture representation. Therefore, as in Table 9, SPA achieves superior performance on OOD benchmarks.
However, for commonly used cross-dataset benchmarks (like DTD (Cimpoi et al., 2014), UCF101 (Soomro et al., 2012) and Aircraft (Maji et al., 2013)) on the CLIP model, the image distributions are often relatively more stable com- pared to OOD datasets, and the image encoder already pro- vides more semantical representations. The performance bottleneck instead often lies in the text modality (i.e., the text encoder)- the quality of the classifier formed from text embeddings. This is also supported by 1) prior VLM TTA methods, such TPT (Shu et al., 2022) and C-TPT (Yoon et al., 2024), which focus on adapting the text branch to improve cross cross-dataset generalization, and 2) prior visual fodulit TTA methods (like Tent and ETA) achieve limited gains in this cross-dataset scenario.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning, especially for Test-Time Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Acknowledgment
This research was supported, in part, by the Joint WeBank- NTU Research Institute on Fintech, Nanyang Technological University, Singapore, and Ministry of Education, Singa- pore, under its Academic Research Fund Tier 1.
Therefore, the performance gain of our SPA method on cross-datased scenarios with CLIP is not as competitive its improvement on OOD scenarios but it still provides benefits (see Table 10)-as as we do not adapt the text branch. Extending our method to also adapt the text branch to further boost performance in cross- dataset scenarios is an interest- ing and promising direction. We leave this for future work.
References
Bartler, A., Bühler, A., Wiewel, F., Döbler, M., and Yang, B. Mt3: Meta test-time training for self-supervised test- time adaption. In International Conference on Artificial Intelligence and Statistics, pp. 3080-3090. PMLR, 2022.

---

SPA: Self-Bootstrapping for Versatile Test-Time Adaptation
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, 2021.
with second-order degradation and reconstruction. In Ad- vances in Neural Information Processing Systems, 2024.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
pp.
Chen, D., Wang, D., Darrell, T., and Ebrahimi, S. Con- trastive test-time adaptation. In IEEE Conference Computer Vision and Pattern Recognition, pp. 295-305, 2022.
Dou, Q. Coelho de Castro, D., Kamnitsas, K., and Glocker, B. Domain generalization via model-agnostic learning of semantic features. In Advances Neural Information Processing Systems, pp. 6447-6458, 2019.
Chen, G., Niu, S., Chen, D., Zhang, S., Li, C.,, Li, Y., and Tan, M. Cross-device collaborative test-time adaptation. In Advances in Neural Information Processing Systems, 2024a.
Gan, Y., Bai, Y., Lou, Y., Ma. X., Zhang, R., Shi, N., and Luo, L. Decorate the newcomers: Visual domain prompt for continual test time adaptation. In AAAI Conference on Artificial Intelligence, volume 37, pp. 7595-7603, 2023.
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual rep- resentations. In International Conference on Machine Learning, pp. 1597-1607. PMLR, 2020a.
Gandelsman, Y., Sun, Y., Chen, X., and Efros, A. Test- time training with masked autoencoders. In Advances in Neural Information P Processing Systems, volume 35, pp. 29374 29374-29385, 2022a.
Chen, X., Fan, H., Girshick, R., and He, K. Improved baselines with momentum contrastive learning. arXiv preprint ar arXiv:2003.04297 2020b.
Chen, Y., Niu, S., Wang, Y., Xu, S., Song, H., and Tan, M. Towards robust and efficient cloud-edge elastic model adaptation via selective entropy distillation. In Interna- tional Conference on Learning Representations, 2024b.
Gandelsman, Y., Sun, Y., Chen, X., and Efros, A. Test- time training with masked autoencoders. In Advances in Neural Information Processing Systems, volume 35, pp. 29374-29385, 2022b.
Choi, S., Yang. S., Choi, S., and Yun, S. Improving test-time adaptation via shift-agnostic weight regularization and nearest source prototypes. In European Conference Computer Vision, pp. 458. Springer, 2022.
Gao, Y.. Shi, X., Zhu, Y., Wang. H., Tang, Z., Zhou, X., Li, M., and Metaxas, D. N. Visual prompt tuning for test-time domain adaptation. In International Conference on Leurning Representations, 2023.
Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. Describing textures in the wild. In IEEE Conference Computer Vision and Puttern Recognition, pp. 3606-3613, 2014.
Geiger, A., Lenz, P., and Urtasun, R. Are ready for autonomous driving? the kitti vision benchmark suite. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354-3361. IEEE, 2012.
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. The cityscapes dataset for semantic urban scene under- standing. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3213-3223, 2016.
Gong, T., Jeong, J., Kim, T., Kim, Y., Shin, J., and Lee, S.-J. Note: Robust continual lest-time adaptation against temporal correlation. In Advances Neural Information Processing Systems, volume 35, pp. 27253-27266, 2022.
Cubuk, E. D., Zoph, B., Mane, D., Vasudevan, V., and Le, Q. V. Autoaugment: Learning augmentation strategies from data. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 113 123-123,220 2019.
Goyal, S., Sun, M., Raghunathan, A., and Kolter, Test time adaptation via conjugate pseudo-labels. In Advances in Neural Information P Processing Systems, volume 35, pp. 6204-6218, 2022.
Cubuk, D., Zoph, B., Shlens, J., and Le, Q. Ran- daugment: Practical automated data augmentation with reduced search space. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 702-703, 2020.
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., et al. Bootstrap your own latent-a new approach to self-supervised learning. In Advances in Neural Information Processing Systems, volume 33, pp. 21271-21284, 2020.
Deng, Z., Chen, Z., Niu, S., Li, T., Zhuang, B., and Tan, M. Efficient test-time adaptation for super-resolution
10

---

SPA: Self-Bootstrapinng for Versatile Test-Time Adaptation
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. Mo- mentum contrast for unsupervised visual representation learning. In IEEE Conference Computer Vision and Pattern Recognition, pp. 9729-97338 2020.
Lee, J., Jung, D., Lee, S., Park, J., Shin, J., Hwang. U., and Yoon, S. Entropy is not enough for test-time adap- tation: From the perspective of disentangled factors. In International Conference on Learning Representations, 2024.
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. Masked autoencoders scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000-16009, 2022.
Lee, J.-H. and Chang, J.-H. Continual momentum filtering on parameter space for online test-time adaptation. In International Conference on Learning Representations, 2024.
Hendrycks, D. and Dietterich, T. Benchmarking neural network robustness to common corruptions and perturba- tions. In International Conference on Learning Represen- tations, 2019.
Liang, J., Hu, D., and Feng, J. Do we really need to access the source data? source hypothesis transfer for unsuper- vised domain adaptation. In International Conference on Machine Learning, pp. 6028 0038-603,-400 2020.
Hendryeks, D., Mu, N., Cubuk, E. D., Zoph, B., Gilmer, J., and Lakshminarayanan, B. Augmix: A simple data processing method to improve robustness and uncertainty. In International Conference on Learning Representations, 2020.
Liang, J., He, R., and Tan, T. A comprehensive survey test-time adaptation under distribution shifts. arXiv preprint arXiv:2303./536/ 2023.
Lim, H., Kim, B., Choo, J., and Choi, S. TTN: A domain- shift aware batch normalization in test-time adaptation. In International Conference on Learning Representations, 2023.
Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al. The many faces of robustness: A critical analysis of dittr---istribution generalization. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 8340- 8349, 2021a.
Lim, S., Kim, I., Kim, T., Kim, C., and Kim, S. Fast autoaugment. In Advances in Neural Information Pro- cessing Systems, volume 32, pp. 6665 00655-665,570 2019.
Hendryeks, D., Zhao, K., Basart, S., Steinhardt, J., and Song, D. Natural adversarial examples. In IEEE Conference Computer Vision and Pattern Recognition, pp. 15262- 15271, 2021b.
Lin, H., Zhang, Y., Niu, S., Cui, S., and Li, Z. Monotta: Fully test-time adaptation for monocular 3d object detec- tion. In European Conference on Computer Vision, pp. 96- 114. Springer, 2024.
Hu, X., Uzunbas. G., Chen, S., Wang, R., Shah, A., Neva- tia, R., and Lim, S.-N. Mixnorm: Test-time adaptation through online normalization estimation. arXiv preprint arXiv:21 10. 1478, 2021.
Lin, W., Mirza, M. J., Kozinski, M., Possegger, H., Kuehne, H., and Bischof, H. Video test-time adaptation for action recognition. In IEEE Conference Computer Vision and Pattern Recognition, pp. 22952-22961, 2023.
Iwasawa, Y. and Matsuo, Y. Test-time classifier adjust- ment module for model-agnostic domain generalization. In Advances in Neural Information Processing Systems, volume 34, 2021.
Liu, Y., Kothari, P., van Delft, B., Bellot-Gurlet, B., Mordan, T., and Alahi, A. Ti+++; When does self-supervised test-time training fail or thrive? In Advances in Neural Information Processing Systems, volume 34, 2021.
Kalibhat, N., Morningstar, W., Bijamov, A., Liu, L., Singhal, K., and Mansfield, P. Disentangling the effects of data augmentation and format transform in self-supervised learning of image representations. In European Confer- ence on Computer Vision Workshop, 2023.
Maji, S., Rahtu, E., Kannala, J., Blaschko, M., and Vedaldi, A. Fine-grained visual classification of aircraft. arXiv preprint 10XXvv1306.511 2013.
Marsden, R. A., Döbler, M., and Yang, B. Universal test- time adaptation through weight ensembling, diversity weighting, and prior correction. In Proceedings of the IEEF/CVF Winter Conference on Applications of Com- puter Vision, pp. 2555 2255-2565,22 2024.
Khurana, A., Paul, S., Rai, P., Biswas, S., and Aggarwal, G. Sita: Single image test-time adaptation. arXiv preprint arXiv:21 2021.
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L.,, Gao, I., et al. Wilds: A benchmark of in-the- wild distribution shifts. In International Conference on Machine Learning, pp. 5637-5664, 2021.
Mirza, M. J., Soneira, P.J., Lin, W., Kozinski, M., Possegger, H., and Bischof, H. Actmad: Activation matching to align distributions for ses-time-training In IEEE Conference on Computer Vision and Pattern Recognition, pp. 24152- 24161, 2023.

---

SPA: Self-Bootstrapiing for Versatile Test-Time Adaptation
Mummadi, C. K., Hutmacher, R.,, Rambach, K., Levinkov, E.,, Brox, T., and Metzen, J. H. Test-time adaptation to distribution shift by confidence maximization and input transformation. arXiv preprint arXiv:2106. 199.9999 2021.
common corruptions by covariate shift adaptation. In Advances in Neural Information Processing Systems, vol- ume 33, pp. 11539-11551, 2020.
Shankar, S., Piratla, V., Chakrabarti, S., Chaudhuri, S., Jyothi, P., and Sarawagi, S. Generalizing across domains via gross-gradient training. In International Conference on Learning Representations, 2018.
Nado, Z., Padhy, S., Sculley, D., D' Amour, A.. Lakshmi- narayanan, B., and Snoek, J. Evaluating prediction-time batch normalization for robustness under covariate shift. arXiv preprint arXiv:2006. 10963, 2020.
Shorten, C. and Khoshgoftaar, T. M. A survey on image data augmentation for deep learning. Journal of big data, 6(1 ((1): 1-11148222 2019.
Niu, S., Wu, J., Zhang, Y., Chen, Y., Zheng. S., Zhao, P., and Tan, M. Efficient test-time model adaptation with- out forgetting. In International Conference on Machine 2022a.
Learning, pp.
Shu, M., Nie, W., Huang, D.-A., Yu, Z., Goldstein, T., Anandkumar, A., and Xiao, C. Test-time prompt tuning for zero-shot generalization in vision- laiiugl models. Advances in Neural Information Processing Systems, volume 35, pp. 14274-14289, 2022.
Niu, S., Wu, J., Zhang, Y., Xu, G., Li, Zhao, P., Huang. J., Wang, Y., and Tan, M. Boost test-time performance with closed-loop inference. arXiv preprint arXiv:2203.18851 2022b.
Soomro, K., 7amir, A. R., and Shah, M. Ucf101: A dataset of 101 human actions classes from videos in the wild, arXiv preprint arXiv: /212. 02..222.4222222 2012.
Niu, S., Wu, J., Zhang, Y., Wen, Z.,, Chen, Y., Zhao, P., and Tan, M. Towards stable test-time adaptation in dynamic wild world. In International Conference Learning Representations, 2023.
Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A., and Hardt, M. Test-time training with self-supervision for generaliza- tion under distribution shifts. In International Conference on Machine Learning, pp. 9229-92455 2020.
Niu, S., Miao, C.,, Chen, G., Wu, P., and Zhao, P. Test-time model adaptation with only forward passes. In Interna- fional Conference on Machine Learning, 2024.
Tan, M., Chen, G., Wu, J.. Zhang, Y., Chen, Y., Zhao, P., and Niu, S. Uncertainty-calibratee test-time model adaptation without forgetting. arXiv preprint arX1v:2403.11491, 2024.
Prabhudesai, M., Ke, T.WW., Li, A., Pathak, D., and Fragki- adaki, K. Diffusion-tta: Test-time adaptation of discrim- inative models via generative feedback. In Advances in Neural Information Processing Systems, volume 36, pp- 17567-17583, 2023.
Van der Schaaf, V. A. and Van Hateren, J. v. Modelling the power spectra of natural images: statistics and informa- tion. Vision Research, 36(17) (6(17))2759-2770 1996.
Qiu, Z., Zhang, Y., Lin, H., NIu, S., Liu, Y., Du, Q., and Tan, M. Source-free domain adaptation via avatar pro- totype generation and adaptation. In International Joint Conference on Artificial Intelligence, 2021.
Wang, D., Shelhamer, E., Liu, S., Olshausen, B., and Darrell, T. Tent: Fully test-time adaptation by entropy minimiza- tion. In International Conference on Learning Represen- tations, 2021.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 8748-8763. PmLR, 2021.
Wang, H., Ge, S., Lipton, Z., and Xing, E. P. Learning ro- bust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems, pp. 10506-10518, 2019.
Recht, B., Roelofs, R., Schmidt, L.., and Shankar, V. Do imagenet classifiers generalize imagenet? In Interna- tional Conference on Machine Learning, pp. 5389-5400, 2019.
Wang, Q., Fink, O., Van Gool, L., and Dai, D. Continual test-time domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition, 2022.
Sakaridis, C., Dai, D., and Van Gool, I.. Acde: The ad- verse conditions dataset with correspondences for seman- tic driving scene understanding. In IEEE International Conference Computer Vision, pp. 10765-10775, 2021.
Wang, Z.,, Luo, Y., Zheng, I.., Chen, Z.., Wang, S., and Huang, Z. In search of lost online test-time adaptation: A survey. International Journal of Computer Vision pp. 1-34, 2024.
Schneider, S., Rusak, E., Eck, L., Bringmann, O., Bren- del, W., and Bethge, M. Improving robustness against
Wightman, R. Pytorch image models. GitHub repository, 2019. doi: (0.5281/zenodo.44 441661..
12

---

SPA: Self-Bootstrapping for Versatile Test-Time Adaptation
Xie, E., Wang, W., Yu, Z.,, Anandkumar, A., Alvarez, J. M., and Luo, P. Segformer: Simple and efficient design for semantic segmentation with transformers. In Advances in Neural Information Processing Systems, pp. 12077- 12090, 2021.
Xu, Q., Zhang, R., Fan, Z., Wang, Y., Wu, Y.-Y,, and Zhang, Y. Fourier-based augmentation with applications to do- main generalization. Pattern Recognition, 139:109474, 2023.
Yang, Y. and Soatto, S. Fda: Fourier domain adaptation for semantic segmentation. In IEEE Conference Computer Vision and Pattern Recognition, pp. 4085 4095, 2020.
Yao, H., Wang, Y., Li, S., Zhang, L., Liang, W., Zou, J., and Finn, C. Improving out-of-distribution robustness via selective augmentation. In International Conference Machine Learning, 2022.
Yoon, H. S., Yoon, E., Tee, J. T. J., Haseguwa-Johnson, M., Li, Y., and Yoo, C. D. C-tpt: Calibrated test-time prompt tuning for vision-language models via text feature dispersion. In International Conference on Learning Representations, 2024.
Zhang, H. Mixup: Beyond empirical risk minimization. In International Conference Learning Representations, 2017.
Zhang, M. M., Levine, S., and Finn, Memo: Test time
robustness via adaptation and augmentation. In Advances in Neural Information Processing Systems, 2022.
Zhang. Y., Lu, J., and Zhou, J. Objects are different: Flexi- ble monocular 3d object detection. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3289- 3298, 2021.
Zhang, Y., Mehra, A., and Hamm, Ot-vp: Optimal gransport-guides visual prompting for test-time adapta- tion. In IEEE/CVF Winter Conference on Applications of
Computer Vision (WACV), pp. 1122-1132. IEEE, 2025.
Zhao, B., Chen, C., and Xia, S.-T. DELTA: Degradation-fre free fully test-time adaptation. International Conference on Learning Representations, 2023.
13

---

Self- Self-Boostrapppin for Versatile Test-Time Adaptation Supplementary Materials
A. Related Work
Contrastive-Based TTA (Zhang et al., 2022; Liu et al., 2021; Shu et al., 2022; Chen et al., 2022) is another pop- ular and promising direction. It is mainly based on self- supervised learning methods that seek to learn robust repre- sentations by contrasting different augmentation views of the same image to enforce consistency, such as MoCo (He et al., 2020; Chen et al., 2020b), SimCLR (Chen al., 2020a), BYOL (Grill et al., 2020), and DINO (Caron et al., 2021). Building on this, many test-time adaptation (TTA) methods incorporate contrastive learning objectives to adapt models to ontoof-distribution data during testing, show- ing promising performance. TTT+ (Liu et al., 2021) ex- ploits the SimCLR (Chen et al., 2020a) loss, while Ada- Contrast (Chen et al., 2022) incorporates MoCo (He et al., 2020)'s s momentum update scheme with the InfoNCE loss, which conduct contrastive learning at the feature level. MT3 (Bartler et al., 2022) explores BYOL (Grill et al., 2020) loss for TTA, applying contrastive learning directly on model predictions. However, as a TTT-series method, MT3 still requires modifications to the model training process. In contrast, MEMO (Zhang et al., 2022) and TPT (Shu et al., 2022) extend this prediction-level consistency learning into fully TTA approach by maximizing prediction consistency across various augmented views.
We categorize related Test-Time Adaptation (TTA) works based on their learning objectives and discuss their general- ity across model architectures and tasks. Following that, we relate our method with data augmentation techniques.
Learning Objective-Free TTA mainly relies on adapting batch norm statistics (Nado et al., 2020; Schneider et al., 2020; Gong et al., 2022). Building on this, various methods have been developed to extend its applicability to diverse test scenarios, e.g., label shifts (Niu et al., 2023; Gong et al., 2022) and single sample (Khurana et al., 2021; Niu et al., 2023), using methods like data augmentation (Khurana et al., 2021), statistics mix-up (Hu et al., 2021; Lim et al., 2023), re-normalization (Zhao et al., 2023), etc. These methods are general for various tasks (e.g., image-/object-level) but are limited to batch norm norm-equipped models.
Cearning-Based TTA (Sun al., 2020; Wang al., 2021; Liang et al., 2023; Niu et al., 2022b; 2024; Deng et al., 2024; Chen et al., 2024a; Zhang et al., 2025) explicitly learns from test data by updating model parameters using self- or unsupervised learning, achieving much better gains. Unlike learning-free TTA, in learning-base TTA, the designed objectives typically support various models, e.g., CNN or ViT, but may be restrictively applicable to different tasks:
Nevertheless, the success of contrastive learning (He et al.,
2020; Chen et al., 2020a; Grill et al., 2020; Caron et al., 2021) and their integration in TTA (Zhang et al., 2022; Shu et al., 2022: Chen al., 2022; Bartler al., 2022) stems from crucial motivation: they depend heavy and strong augmentations to supply adequate learning signals across different views. Their commonly used strong augmenta- tions, egg., random cropping, resizing, and masking, often disrupt the overall content and geometric structure of the image, making them work well for image-level tasks, but infeasible for object-/pixll-leeel TTA tasks. In this work, we aim to develop new versatile fully TTA method to enable model- and task-agnostic applications, by augmenting an image in the Fourier domain for seff-bootstrapping.
Entropy/ / Self-LLeanning-baae based methods (Wang et al., 2021; Mummadi et al., 2021; Goyal et al., 2022; Chen et al., 2024b) are among the most popular fully TTA approaches. These methods, which optimize prediction entropy or cross- entropy loss using pseudo-labels, have become foundational techniques in the TTA community. Building on them, sev- eral methods have been further developed: EATA (Niu et al., 2022a; Tan et al., 2024) and SAR (Niu et al., 2023) intro- duce selective entropy minimization strategies for improved efficiency and sharpness-aware optimization for stable TTA in the wild; SLR (Mummadi et al., 2021) and Conjugate PL (Goyal et al., 2022) design advanced loss functions to better utilize pseudo-labels in TTA, etc.
Data Augmentation (Shorten & Khoshgoftaar, 2019) aims to increase the diversity of training data, i.e., enlarging train- ing data distribution, by applying transformations such as rotation, flipping. and cropping, helping models generalize better and reduce overfitting. As widely used technique in deep learning, data augmentation has evolved significantly over time. Starting from the simple, manually designed
Prototype-Based TTA methods (Iwasawa & Matsuo, 2021; Choi et al., 2022) enhance TTA performance by main- taining class-specific prototypes and making predictions based on feature similarity with these prototypes.
However, all these methods rely on predicted class probabil- ities, making them unsuitable for regression tasks.
14

---

SPA: Self-Bootstrapiing for Versatile Test-Time Adaptation
transformations, it has progressed more advanced ap- proaches like RandAugment (Cubuk et al., 2020), AutoAug- ment (Cubuk et al., 2019), FastAutoAugment (Lim et al., 2019), Mixup (Zhang, 2017), AugMix (Hendrycks et al., 2020) and etc. These methods have demonstrated remark- able effectiveness in enhancing generalization at training time in supervised manner, and also have been adopted by self-supervised (He al., 2020; Chen et al., 2020b;a; Grill et al., 2020; Caron et al., 2021) methods and contrastive TTA methods (Chen et al., 2022; Zhang et al., 2022; Bartler et al., 2022) to generate diverse augmentation views for effective learning, as we depicted above.
ImageNet-R contains 30,000 images featuring artistic ren- ditions of 200 ImageNet classes. These images are mainly sourced from Flickr and filtered by Amazon MTurk.
ImageNet-A comprises 7,500 images covering 200 Ima- geNet classes. These images are naturally existing samples that lead to a notable degradation in classifier performance.
ImageNet- consists of 50,899 images represented as black and white sketches, covering 1000 ImageNet classes.
KITTI-C consists of various versions of corruption applied to 3,769 validation images from KITTI (Geiger et al., 2012). It encompasses 13 corruption types of 4 main groups, includ- ing Gaussian noise, shot noise, impulse noise, defocus blur, glass blur, motion blur, snow, frost, fog. brightness, contrast, pixelation, and saturation. Each corruption is characterized by 5 different levels of severity. We utilize severity level I for all evaluations per MonoTTA (Lin et al., 2024).
In the work, we develop a new versatile fully TTA method, termed SPA, to enable model- and task-agnostic applica- aug-
tions. SPA is also built on data augmentation. ments/deteriorates an image in the Fourier domain gener- ate a weak view for self- seff-bootstraaping learning. Here, we would like to point out that although recently there are some works introducing Fourier-based augmentations (Xu et al., 2023; Yang& & Soatto, 2020; Kalibhat et al., 2023), they tar- get training-time generalization, unsupervised domain adap- tation and self-supervised learning, exploring techniques like amplitude swap/mixup, and phase shift. And these methods still overlook and disrupt the geometric structure of images after augmentation, or rely on paired source and target data, making them incompatible with our context of a versatile fully TTA framework. In this paper, we ana- lyze how common distribution shifts manifest in the Fourier domain and, based on our findings, design specific augmen- tation strategies for low and high frequencies, respectively. Our augmentations aim to preserve the core geometric struc- ture of the image while providing as much as possible learn- ing signals for effective and versatile TTA.
ACDC contains four categories of images collected in ad- verse conditions, including fog. night, rain, and snow. Fol- lowing CoTTA (Wang et al., 2022), we use 400 unlabeled images from each adverse condition for continuous TTA.
B.2. More Evaluation Protocols
We use the ViT-Base (Dosovitskiy et al., 2021) model trained on ImageNet by timm (Wightman, 2019) as the source model for classification, the MonoFlex (Zhang et al., 2021) model trained on KITTI (Geiger et al., 2012) for 3D monocular detection, and the Segformer-B3 B55 (Xie et al., 2021) model trained on CityScapes (Cordts et al., 2016) for semantic segmentation. We introduce the implementation details of the involved methods below.
SPA (ours) For classification, we set the mask ratio m to 0.2 and the noise factor 7 to 0.4. We insert a new learn- able projector before the final predictions of the augmented (weak) views following BYOL (Grill et al., 2020). This projector is initialized as identity mappings and updated via SGD optimizer with a learning rate of 0.05 and a momen- tum of 0.9. The affine parameters of the norm layers are also updated via SGD, using a learning rate of 0.01 and a momentum of 0.9. When integrated with ActMAD (Mirza et al., 2023) and Tent (Wang et al., 2021), the learning rate of the norm layers is set to 0.005 following ActMAD, For 3D monocular object detection, we set the mask ratio m to 0.2 and the noise factor y to 0.1. We align the classi- fication head via KL loss and regression heads of bound- ing box coordinates, dims, and depths prediction via LI loss, where we apply a rescaling factor of 0.01 on regres- sion losses for balancing. We also adopt threshold of 0.4 on f(x; 0) to filter unreliable predictions inspired by MonoTTe (Lin et al., 2024). The norm layers are updated using SGD, with a learning rate of 0.005 and a momentum of 0.9. When integrated with MonOTTA, the loss com-
B. More Implementation Details
B.1. More Details on Datasets
We conduct experiments on six datasets to evaluate the OOD generalization. Specifically, 1) for classification: we use ImageNet-C (Recht et al., 2019), ImageNet-R (Hendrycks et al., 2021a), ImageNet-A (Hendrycks et al., 202 lb), and ImageNet-Sketee (Wang et al., 2019); 2) for 3D monocular object detection: we use KITTI-C per MonoTTA (Lin et al., 2024); and 3) for segmentation: we use ACDC (Sakaridis et al., 2021) per CoTTA (Wang et al., 2022); encompassing 35 distribution shifts in total, shown in Figure 4.
ImageNet-C consists of various versions of corruption ap- plied to 50,000 validation images from ImageNet. The dataset encompasses 15 distinct corruption types of 4 main groups, including noise, blur, weather, and digital. Each corruption is characterized by different levels of severity. We specifically utilize severity level 5 for all evaluations.
15

---

SPA: Self-Bootstrapping for Versatile Test-Time Adaptation
ImageNet
ImageNet-C
ImageNet-A
ImageNet-R
ImageNet-S
Classification
Original
Noise
Blur
Adversarial
Rendition
Sketch
Original
Weather
Digital
Adversarial
Rendition
Sketch
KITTI
KITTI-C
aD
Original
Impulse Noise
Motion Blur
Frost
Saturate
Cityscapes
ACDC
Segmentation
Original
Fog
Night
Rain
Snow
Figure Visualizations of partial images in ImageNet, ImaggeNctCCAA/R/kkethh KITTI, KITTI-C, Cityscapes and ACDC.
ponents from SPA are further rescaled by factor 0.5. For semantic segmentation, when incorporated with COTTA (Wang al., 2022), we the mask ratio to 0.2 and the noise factor y to 0.1, where our augmentation strate- gies are applied on the student model. We adopt the Adam optimizer with the learning rate of 6 x 10 to optimize all the parameters in Segformer-BS. When SPA works as a standalone method, we further apply a confidence threshold of 0.2 on the original (strong) views select reliable predic- tions inspired by CoTTA (Wang al., 2022), and adopt the Adam optimizer to update the norm layers with learning rate of x 44
that are set in Tent unless it does not provide. For clas- sification, we use the SGD optimizer, with a momentum of 0.9 and a learning rate 0.001. For 3D monocular detection,
the SGD optimizer per MonoTTA (Lin et al., 2024), with a momentum of 0.9 and learning rate of 0,0005, and entropy loss is applied on the classification head. For segmentation, we use the Adam optimizer per CoTTA (Wang et al., 2022) with learning rate 0.00006/8. Trainable parameters are the parameters of the norm layers.
SAR (Niu et al., 2023) We follow all haper-parameters that set in SAR unless it does not provide. Specifically, we use SGD as the update rule, with a momentum of 0.9, batch size of 64, and learning rate of 0.001. The threshold Fg is set to 0.4x In C, where C' is the number of classes. The trainable parameters are the affine parameters of the layer normalization layers from blocks ViT-Base (Dosovitskiy et al., 2021).
We make noise injection learnable for image classification tasks by dividing the images into patches, with a patch size of 16, and learn patch-specific noise factor {7/ 1 where P is the number of patches. 7 is updated to maximize the test-time objective via SGD with learning rate of and momentum of 0.9, under the constraint mean(y) 0.4.
blocks 8 in
EATA (Niu et al., 2022a) We follow all hyper-parameters
TENT (Wang et al., 2021) We follow all hyper-parameters
hhtt:://giiaabcccmmmmmegggnnnii http://gghua..co/m/megggg//////
"bttss//githubb.om//qquuu Wang/tent
16

---

SPA: Self-Bootstrapping for Versatile Test-Time Adaptation
Noise
Blur
Weather Frost
Digital
Average
Method Gauss.
Shot
Impul.
Defoc
Glass
Motion
Zoom
Snow
Fog
Brit.
Contr.
Elastic
Pixel
JPEG
Acc.
Source
56.8
56.8
57.5
46.9
35.6
53.1
44.8
62.2
62.5
65.7
77.7
32.6
46.0
67.0
67.6
55.5
TENT
60.3
61.6
61.8
59.2
56.5
63.5
59.2
54.3
64.5
2.3
79.1
67.4
61.5
72.5
70.6
59.6
EATA
62.2
63.4
63.4
60.5
61.2
66.0
63.5
70.3
68.4
73.1
79.8
67.0
69.7
75.2
73.4
67.8
SPA
64.0
65.5
65.2
61.0
63.6
69.1
67.9
74.1
72.7
75.3
80.9
65.2
74,0
77.6
75,0
70.1
SPA-I
62.7
64.3
64.2
58.9
61.9
67.7
65.9
72.6
71.7
75.1
80.5
66.3
71.9
76.8
73.6
69.0
Table 11. Effects of combined vs. separate augmentation strategy SPA on ImageNet--C (severity level 5) with ViT-Base w.r.t. Accuracy (%). ). SPA-I applies low-frequency amplitude mask (Eqn. (4)) and high-frequency noise injection (Eqn. (5)) in a single image simultane- ously, obtaining one augmented image. While SPA augments an image using Eqn. (4) and Eqn. (5) separately, generating two augmented images for test-time self-hootstrapping learning.
that are set in EATA unless does not provide. Specifically, for image classification, we use SGD as the update rule, with a momentum of 0.9 and a learning rate of 0.001. The entropy threshold Eo set to 0.4 X In C, where C' is the number of task classes. We use 2,000 samples to estimate the importance of each parameter. The trainable parameters are all affine parameters of layer normalization layers in ViT- Base (Dosovitskiy et al., 2021). For 3d monocular detection, we follow the ayper-parametee specified by MonoTTA (Lin et al., 2024). In particular, we optimize the affine parameters of the batch norm layers using SGD, with a momentum of 0.9 and learning rate of 0.0005. The entropy threshold Eo is set to In C/2 - and we also apply confidence threshold of 0.2 to filter out potentially unreliable predictions.
CoTTA6 6 We follow all hyper-parameters that are set in COTTA unless it does not provide. For classification, we use SGD as the update rule, with a momentum of 0.9, and batch size of 64. We consistently set the learning rate to 0.001 and the augmentation threshold Ptu. to 0.1 given the optimal accuracy observed in Table 1. For images below the threshold, we conduct 32 augmentations including color jitter, random affine, Gaussian blur, random horizontal flip, and Gaussian noise. For segmentation, the threshold Pt is set to 0.69. We apply a range of image resolution scale factors [0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0] and horizontal flip as the augmentation of teacher model input. We use Adam as the update rule with learning rate of 0.00006/8. In both cases, the trainable parameters are all the parameters in the student model, and the teacher model is updated via the exponential moving average with a moving factor of 0.999. The restoration probability is set to 0.01.
De (Lee et al., 2024) We follow all hyper-parameters that are set in YO unless does not provide. Specifi- cally, we use SGD as the update rule, with a momentum of 0.9 and a learning rate of 0.001. The entropy thresh- old Fg is set to 0.4x In C and the entropy factor TEm is set to 0.5x In C, where C' is the number of task classes. The Pseudo-Label Probability Difference (PLPD) threshold TPLPD is set to 0.2. Trainable parameters are the affine pa- rameters of the layer normalization layers from blocks I to blocks in ViT-Base (Dosovitskiy et al., 2021).
MonOTTA? (Lin et al., 2024) We follow all hyper- parameters that are set in MonoTTA. Specifiiill,, the initial object detection threshold y is set to 1, the decay coefficient 3 for threshold updating is set to 0.9, and the threshold n for low-score object filtering is set to 0.05. The affine pa- rameters of batch norm layers are updated via SGD, with a learning rate of 0.0005 and a momentum of 0.9. We also follow the hyper- prperpparaeetes of other baseline meth- ods, including TENT, EATA, and ActMAD, as specified by MOnoTTA for the 3D monocular object detection task.
ActMADs (Mirza et al., 2023) The implementations of ActMAD ViT-Base (Dosovitskiy et al., 2021) for classifi- cation and MonoFlex (Zhang et al., 2021) for 3D monocular detection are inspired by FOA. For classification, we cal- culate the source training statistics with the validation set of ImageNet and align the test statistics per batch, using the SGD optimizer with a learning rate of 0.005 and a mo- mentum of 0.9. For 3D monocular detection, we calculate the source training statistics with 15.168 samples from the validation set of KIITI and adopt the SGD optimizer with a learning rate of 0.0005 and a momentum of 0.9. Trainable parameters are affine parameters in norm layers.
C. More Discussions and Results
Effects of Combined rs. Separate Augmentations in SPA In SPA, we augment a given image using low-frequency am- plitude mask in Eqn. (4) and high-frequency noise injection in Eqn. (5) to generate two distinct augmented views for weak-to-strong self-bottstrapingg learning. This separates the learning process for high and low frequencies and helps the model better learn low- or high-frequency preserving features, aiming to maximize the efficiency of utilizing the
hhaps://gttuh..co///yyu 7/DeYO " httpps///ghtubb.oo//mmmmmmaaaaai
httt:://jihhb..comm/mnnnnrrggoo hhtp:://ghhhbboooo///ooogn////nn
17

---

SPA: Self-Bootstrapping for Versatile Test-Time Adaptation
KITTI-Fog (AP sDieun %,
KITTI-Fog (AP anjrm- %. ")
Method
Car
Pedestrian
Cyelist
Avg.
Method
Car
Pedestrian
Cyelist
Avg.
No Adapt
7.8
2.0
3,6
4.5
No Adapt
7.8
2.0
3.6
4.5
BN Adapt (Schneider et al., 2020)
23.3
8.6
9.7
13.9
BN Adapt (Schneider et al., 2020)
23.3
8.6
9.7
13.9
TENT (Wang et al., 2021)
26.5
8.7
10.5
15.2
TENT (Wang et al., 2021)
26.5
8.7
10.5
15.2
EATA (Niu et al., 2022a)
27.9
8.7
10.5
15.7
EATA (Niu et al., 2022a)
27.9
8.7
10.5
15.7
MonoTTA (Lin et al., 2024)
32.0
9.2
9.7
17.0
MonoTTA (Lin et al., 2024)
32.0
9.2
9.7
17.0
SPA-1 SPA
32.6
10.1
8.8
17.2
Self-bootsrappping learning of our SPA by aligning:
34.4
11.1
9.7
18.4
Reg. heads
32.4
8.7
8.7
16.6
Cls. heads
32.2
10.7
9.3
17.4
Table 12. Effects of combined vs. separate augmentation strat- egy in SPA on 3D monocular detection with MonoFlex source model. SPA-I applics low- tiefrequenccy amplitude mask (Eqn. (4)) and high-frequency noise injection (Eqn. (5)) in singlc image simultaneously, obtaining one augmented image. While SPA aug- ments image using Eqn. (4) and Eqn. (5) separately, generating two augmented images for test-time self-hootstrapingg learning.
Reg. heads & Cls. heads
34.4
11.1
9.7
18.4
Table 13. Effects of SPA aligning different heads' predictions (clas- sification and regression heads) in 3D monocular object detection. We use MonoFlex as the source model.
Method
Average Accuracy (%)
Source
31.4
CoTTA (Wang et al., 2022)
34.0
EATA (Niu et al., 2022n)
44,4
DeYO (Lee et al., 2024)
45.9
constructed weak-to-strong learning signals at different fre- quency ranges independently. In this section, we further compare SPA with SPA-I, which simultaneously applies two augmentation strategies in single image. From results in Tables 11 and 12, SPA-I performs slightly worse than SPA but still achieves better or comparable performance compared to prior SOTAs, suggesting its superiority.
ROID (Marsden al., 2024)
46.8
CMF (Lee & Chang, 2024)
48.1
SPA (ours)
49.2
SPA + Tent (ours)
50.9
SPA + EATA (ours)
52.5
Table 14. Comparisons ImageNet-C (level 5) with ResNet-50.
Adaptation Efficiency Notably, though SPA involves one more forward and backward propagation. it remains effi- cient and operates in real-time, achieving 79 FPS (vs. SPA- I: 125 FPS) single A100 GPU with ViT-Base and ImageNet-C. Here, SPA-I is the variant that applies two deteriorations within a single image (1 augmentation) and achieves similar ImageNet-C accuracy: SPA (70.1%) vs SPA-I (69.0%) as in Table 11. SPA is more efficient than prior augmentation--asse methods such as MEMO (Zhang et al., 2022), CoTTA (Wang al., 2022), and TPT (Shu et al., 2022), which require 64, 34(or 2), and 63 augmenta- tions per sample, respectively. It also matches the efficiency of entropy- SAR. The FPS on ImageNet-C with ViT- Base (on a single 100 GPU) are: SPA-T (125)= > SAR (102)= > SPA (79) > CoTTA (36).
tency of regression predictions, offers rich learning signals at test time. As shown in Table 13, SPA achieves comparable performance to existing methods using only regression self- supervision, e.g., with an average AP3 R440 of 16.6% (SPA with Reg. Heads Aligmment) vs. 15.7% (EATA). When fur- ther incorporating classification supervision, our SPA is able to surpass the existing state-of-the-arr method, MonoTTA, which focuses only the classification head, by average of 1 1.4%. These results collectively highlight the effective- ness of SPA, and underscore the importance of exploiting both information from regression and classification predic- tions to design more general and effective TTA solution.
Effectiveness of SPA with Different Model Architectures In our experiments, we use the ViT-Base model for image classification, MonoFlex (Zhang et al., 2021) for 3D monoc- ular object detection, and Segformer-Bi B5 (Xie et al., 2021) for segmentation tasks. Here, ViT-Base and Segformer-B3 B5 employ transformer-based architectures, while MonoFlex is based on convolutional neural networks. Our results in Tables 2, 3 and 4 demonstrate that SPA performs well all three backbone models, demonstrating its generality across different types of model architectures.
Effectiveness of Aligning Regression Heads in SPA for 3D Monocular Object Detection The 3D monocular de- tection task comprises both classification heads to identify object class within each 3D bounding box, and regression heads to predict the 3D bounding box coordinates, dimen- sions, depths, and angles which provides a comprehensive spatial understanding of each detected object. However, existing TTA methods (Schneider et al., 2020: Wang et al., 2021; Niu et al., 2022a; Lin et al., 2024) for 3D MonoDet focus on designing TTA loss classification heads while overlooking the regression heads with rich predictions. In contrast, SPA is task-agnostic, making it applicable to both classification and regression heads seamlessly. In SPA, we demonstrate that leveraging the regression heads (with rich spatial information) for TTA, i.e, aligning prediction consis-
More Results on Classification with ResNet-50 In this section, we provide more experiments on ResNet-50 with ImageNet-C. As shown in Table 14, the improvements ob- served on ResNet-50 are consistent with our experiments on Vit-Base. This further underscores SPA's effectiveness both standalone method and as a plug-and-pla module to enhance existing methods
18


Reviews:
[{"review_id": "XL4qw6Y7B5", "reviewer": "Reviewer_61cM", "summary": "This paper proposes an image augmentation strategy utilizing the Fourier domain for randomly masking the low-frequency amplitude of an image. Further, it augments the image with noise injection to account for the lack of learning signals at high frequencies.\nThe paper reports experimental results on classification, segmentation, and 3D monocular detection tasks to show the effectiveness of the proposed approach.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your valuable feedback and constructive comments on improving the quality of our paper. We would like to address your questions below.\n\n>Q1. More classification results on ResNet-50.\n\nThanks for your suggestion. We conduct additional experiments on ResNet-50 and compare with more SOTA approaches. As in the table below, the improvements observed on ResNet-50 are consistent with our experiments on ViT-Base. This further underscores SPA's effectiveness both as a standalone method and as a plug-and-play module to enhance existing methods.\n\n|Method|Avg Acc (%) over 15 corruptions of ImageNet-C|\n|-|:-:|\n|NoAdapt (ResNet-50)|31.4|\n|CoTTA|34.0|\n|EATA|44.4|\n|DeYO|45.9|\n|ROID|46.8|\n|CMF|48.1|\n|SPA (ours)|49.2|\n|SPA+Tent (ours)|50.9|\n|SPA+EATA (ours)|52.5|\n\n>Q2. Differences from FDA [A].\n\nThank you for pointing out this related work. While FDA also utilizes the Fourier augmentation, SPA's primary contributions lie in the overall design of a versatile TTA scheme. SPA comprises several key components, including an active, deterioration-driven self-bootstrapping scheme (distinct from BYOL), and geometry-preserving augmentations inspired by our Fourier analysis across domain shifts. All these innovations are non-trivial and collectively contribute to SPA’s effectiveness and versatility. Compared with FDA, which focuses on unsupervised domain adaptation, our SPA method mainly differs in the following aspects.\n\n- **Different Augmentations:** FDA augments a source domain image by replacing the low-frequency spectrum of the source image with that from a target image. This augmentation relies on paired source and target data, which is infeasible in TTA as only a single unlabeled target sample is available at test time. Unlike FDA, SPA randomly masks the low-frequency spectrum and injects noise for a single target image, without requiring the source images.\n- **Different Methods:** FDA uses Fourier augmentation to transfer the source image style to the target domain, creating a target-style labeled source dataset $D^{s\\rightarrow t}$. It then jointly trains the model offline using both $D^{s\\rightarrow t}$ and the original target dataset $D^{t}$ to mitigate domain shifts. In contrast, SPA targets the more general and challenging setting of fully online TTA. It establishes a versatile self-bootstrapping framework that performs active weak-to-strong learning from a deteriorated view to the original image, for each given target test sample.\n- **Different Design Motivations:** The key motivation of FDA is style transfer（$s\\rightarrow t$）for cross-domain training, by swapping the spectrum in Fourier domain. Unlike FDA, the online unsupervised learning in SPA is highly unstable, and thus our key motivation is how to design effective deterioration/augmentation strategies to conquer this. To this end, we analyze how information power shifts across frequencies under domain shift and leverage these insights to design augmentations that consistently deliver informative signals across all frequencies for SPA, meanwhile maintaining stability.\n\nWe will include the above discussions in the revised paper.\n\n[A] Fda: Fourier domain adaptation for semantic segmentation. CVPR 2020.\n\n>Q3. How is the noise factor $\\gamma$ hyperparameter tuned?\n\nWe did not carefully tune $\\gamma$ on each individual test set as at test time we do not have ground truth labels. In main experiments, we briefly set $\\gamma$ to 0.4 for all classification datasets, including ImageNet-C (15 corruptions, ImageNet-R/Sketch/A), and $\\gamma=0.1$ for all fine-grained tasks of detection and segmentation, including KITTI-C (13 corruptions) and Cityscape-to-ACDC. \n\nWe also show the sensitivity of $\\gamma$ in Figure 3 on a single corruption (test set) of ImageNet-C(Gaussian) and KITTI-C(Fog). From the results, $\\gamma\\in[0.1,0.5]$ works stably on classification, while the stable range of $\\gamma$ on mono 3D detection is relatively narrower, with performance slightly declining when $\\gamma>0.2$—though it still remains significantly better than without adaptation. This difference arises because, for classification, the task is at the image level and does not strictly require content invariance, allowing for a higher $\\gamma$ to provide richer learning signals. In contrast, 3D detection involves dense predictions where high noise levels could significantly disrupt the original image content, making it challenging for our self-bootstrapping learning. \n\nWe thank you for appreciating our contributions. We sincerely hope our clarifications above have addressed your questions and can improve your opinion of our work.", "claims_and_evidence": "The claims made in the submission are supported by the improved performance in the experiments.", "methods_and_evaluation_criteria": "The benchmark datasets make sense but it would better a more fair comparison if experiments on ImageNetC is on ViT, and not ResNet-50 like prior works such as CoTTA [1], PETAL [2], EcoTTA [3], etc.\n\n**References**\n1. Wang, Qin, et al. \"Continual test-time domain adaptation.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n2. Brahma, Dhanajit, and Piyush Rai. \"A probabilistic framework for lifelong test-time adaptation.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n3. Song, Junha, et al. \"Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n4. Yang, Yanchao, and Stefano Soatto. \"Fda: Fourier domain adaptation for semantic segmentation.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.", "theoretical_claims": "This work does not involve theoretical claims or proofs.", "experimental_designs_or_analyses": "Refer to Questions and Weaknesses.", "relation_to_broader_scientific_literature": "The key contributions of this paper can lead to better test-time adaptation in general. However, the novelty seems limited, as pointed out in \"Essential References Not Discussed\" and \"Questions\".", "essential_references_not_discussed": "A Fourier transform for domain adaptation that also involves masking has already been proposed by [1].\nThis limits the novelty of the proposed approach utilizing Fourier transform with masking for TTA, which is a setting related to domain adaptation.\n\n**References**\n1. Yang, Yanchao, and Stefano Soatto. \"Fda: Fourier domain adaptation for semantic segmentation.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.", "other_strengths_and_weaknesses": "**Strengths**\n* Performing image augmentation by utilizing Fourier transform for TTA is an interesting idea\n* Experimental gains demonstrate the effectiveness of the proposed approach\n\n**Weaknesses**\n* Fourier transform for domain adaptation that also involves masking has already been proposed by [4].\n* Experiments on ImageNetC is on ViT, and not ResNet-50 like prior works such as CoTTA [1], PETAL [2], EcoTTA [3], etc.\n\n**References**\n1. Wang, Qin, et al. \"Continual test-time domain adaptation.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n2. Brahma, Dhanajit, and Piyush Rai. \"A probabilistic framework for lifelong test-time adaptation.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n3. Song, Junha, et al. \"Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n4. Yang, Yanchao, and Stefano Soatto. \"Fda: Fourier domain adaptation for semantic segmentation.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.", "other_comments_or_suggestions": "Please include the reference and highlight the novelty or difference between [1] and the proposed approach in the paper (appendix if the space is limited).\n\n**References**\n1. Yang, Yanchao, and Stefano Soatto. \"Fda: Fourier domain adaptation for semantic segmentation.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.", "questions_for_authors": "1. How is the noise factor γ hyperparameter tuned? Is the performance on the same test data monitored to get the best hyperparameters?\n2. A Fourier transform for domain adaptation that also involves masking has already been proposed by [1]. This limits the novelty of the proposed approach utilizing Fourier transform with masking for TTA, which is a setting related to domain adaptation. Can the authors elaborate on how the proposed approach is a non-trivial extension of this prior work?\n\n**References**\n1. Yang, Yanchao, and Stefano Soatto. \"Fda: Fourier domain adaptation for semantic segmentation.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p2Z9J4uvAZ", "reviewer": "Reviewer_DdZn", "summary": "The paper introduces Self-Bootstrapping for versatile Test-Time Adaptation (SPA), a novel framework that enables TTA across multiple tasks—classification, segmentation, and 3D detection. The authors propose a geometry-preserving augmentation strategy using low-frequency amplitude masking and high-frequency noise injection in the Fourier domain. This approach maintains spatial structure while providing sufficient learning signals for adapting models at test time without source data access.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your valuable feedback and constructive comments on improving the quality of our paper. We would like to address your questions below.\n\n>Q1. Comparison with TTT-series methods. \n\nThank you for your suggestion. We follow the comparison setting used in Diffusion-TTA and directly compare our results with the TTT methods reported in their work, by implementing our method using their adopted model and experimental environment. This is because TTT-series methods modify the model training process, making them relatively difficult to implement under our diverse environmental setups. As shown in Table A, our SPA, as a fully TTA method, still achieves superior performance, further demonstrating its effectiveness. We will discuss all mentioned TTT-series methods in the revision.\n\nTable A. Comparisons with TTT-series methods on ImageNet-C w.r.t. acc (%).\n|Model+Method|gauss|fog|pixel|snow|contrast|Avg|\n|-|:-:|:-:|:-:|:-:|:-:|:-:|\n|Customized ViT-L/16 classifier|17.1|38.7|47.1|35.6|6.9|29.1|\n| + TTT-MAE|37.9|51.1|65.7|56.5|10.0|44.2|\n|ViT-B/32|39.5|35.9|55.0|30.0|31.5|38.4|\n| + Diffusion-TTA|46.5|56.2|64.7|50.4|33.6|50.3|\n| + SPA (Ours)|49.4|56.9|66.7|49.9|51.5|54.9|\n\n>Q2. Why are conventional augmentations not good?\n\nConventional augmentations can be broadly categorized into two groups based on whether they preserve geometric information.\n\n- **Geometry-preserving augmentations** (e.g., grayscale conversion, brightness, contrast, blur) typically adjust the global distribution of images—for instance, changing brightness by adding the same constant across all pixels. Such augmentation patterns are relatively simple and lack diversity or uniqueness for different samples, thus providing limited learning signals for our self-bootstrapping learning pipeline. As in Table 6, none of these augmentations, individually or combined, effectively provide TTA with rich learning signals. These augmentations are also often sensitive to the corruption type and struggle to perform stably across all corruptions, leading to limited overall performance.\n- **Non-geometry-preserving augmentations** (e.g., random resizing, cropping, image masking, SimCLR, MoCo, AugMix) introduce randomness for different samples, which randomly deteriorate the local information of different images, thus can provide more diverse learning signals for SPA. However, these augmentations shall disrupt the image’s geometric structure and directly desert the potentially useful image information. As in Table 7, SPA with these augmentations achieves considerable performance on image classification but performs poorly on finer dense prediction tasks like 3D monocular detection. In contrast, our Fourier augmentations preserve the overall information and geometry to support dense prediction tasks and also introduce sufficient diversity for different samples, supplying rich signals for SPA and achieving improved adaptation performance.\n\n>Q3. Results on more realistic distribution shift scenarios.\n\nThank you for your valuable suggestion. We have conducted additional experiments on ObjectNet in Table B, based on the codebase of VisDA-2021 challenge. Furthermore, we also evaluate SPA under more 'realistic settings', including continual adaptation (Table C) and mixed domain adaptation (Table D). The results across all these tables further validate the effectiveness of SPA, both as a standalone method and as a plug-and-play module to enhance existing approaches.\n\nTable B. Results on ObjectNet with ResNet-50.\n|Method|Source|Tent|ETA|SAR|DeYO|ROID|SPA(ours)|SPA+ETA(ours)|\n|-|-|-|-|-|-|-|-|-|\n|Acc (%)|27.2|27.5|26.0|28.6|29.8|28.7|30.9|32.3|\n\nTable C. Results of continual adaptation on ImageNet-C (level 5) with ViT-Base. We report avg acc of 15 corruptions.\n|Method|Avg Acc (%)|\n|-|-|\n|Source|55.5|\n|EATA|67.3|\n|SAR|61.3|\n|ROID|67.9|\n|ActMAD|59.9|\n|SPA (ours)|68.6|\n|SPA+ActMAD (ours)|71.5|\n|SPA+ActMAD+Tent (ours)|71.8|\n\nTable D. Results on mixture of 15 corruptions of ImageNet-C (level 5) with ViT-Base.\n|Method|Acc (%)|\n|-|-|\n|Source|55.5|\n|EATA|63.7|\n|SAR|60.7|\n|DeYO|55.1|\n|ROID|62.0|\n|SPA (ours)|67.1|\n|SPA+ActMAD (ours)|68.2|\n|SPA+ActMAD+Tent (ours)|68.5|\n\n>Q4. Integration with foundation models.\n\nWe briefly test our method on CLIP-ViT model and report results in Table E. With CLIP, our method still works, further suggesting the versatility of SPA.\n\nTable E. Results with CLIP-ViT-B on ImageNet-R.\n|Method|Source|TPT|C-TPT|ETA|SAR|DeYO|SPA (ours)|SPA+ETA (ours)|\n|-|-|-|-|-|-|-|-|-|\n|Acc (%)|74.0|77.1|76.0|76.9|75.6|76.6|77.2|78.2|\n\nWe thank you for appreciating our contributions. We sincerely hope our clarifications above have addressed your questions and can improve your opinion of our work.", "claims_and_evidence": "- The core claim of versatility across task types is well-supported through experiments on ImageNet-C/R/A/Sketch, KITTI-C, and ACDC datasets.\n- Results demonstrate consistent improvements over baseline methods across different architectures (ViT, CNN) and tasks.\n- The ablation studies effectively validate design choices, particularly regarding frequency-domain augmentation strategies.", "methods_and_evaluation_criteria": "- The proposed frequency-domain analysis and augmentation approach is novel and well-motivated.\n- Benchmarks cover a good range of distribution shifts and tasks.\n- Evaluation methodology follows established protocols in the field.", "theoretical_claims": "No formal proofs are presented.", "experimental_designs_or_analyses": "- Experimental setup is sound with appropriate baselines.\n- Missing comparison with certain TTA methods: Notable omissions include MT3, TTT_MAE, and Diffusion-TTA, which would strengthen the evaluation.\n- The analyses of how different domain shifts manifest in the frequency domain are particularly insightful.\n\n\n\n\nDiffusion-TTA - https://diffusion-tta.github.io/\nTTT-MAE - https://papers.neurips.cc/paper_files/paper/2022/file/bcdec1c2d60f94a93b6e36f937aa0530-Paper-Conference.pdf", "relation_to_broader_scientific_literature": "The paper builds upon and extends several research directions in domain adaptation and self-supervised learning. It draws conceptually from self-supervised contrastive methods like BYOL and DINO but adapts these approaches specifically for test-time adaptation. The authors' frequency-domain augmentation strategy connects to a growing body of work on Fourier-domain analysis in computer vision, though they uniquely apply it to preserve geometric structure for dense prediction tasks. The paper sits at the intersection of consistency-based TTA methods (like MEMO), entropy-based approaches (TENT, EATA), and structure-preserving adaptation—extending beyond prior work by creating a unifying framework applicable to a wider range of tasks.", "essential_references_not_discussed": "Diffusion-TTA - https://diffusion-tta.github.io/\nTTT-MAE - https://papers.neurips.cc/paper_files/paper/2022/file/bcdec1c2d60f94a93b6e36f937aa0530-Paper-Conference.pdf", "other_strengths_and_weaknesses": "Strengths:\n\n- The approach's versatility across regression and classification tasks is impressive\n- Geometry-preserving augmentations address a key limitation in prior work\n- Functions well as both standalone method and plug-and-play module\n\nWeaknesses:\n\n- No comparison with TTT-series methods that might be competitive, i would prefer if the authors could do apples to apples comparision against MT3, so that we can use directly the numbers reported in their paper\n- I'm still missing a good intution/analysis as to why conventional augmentations are not good, although Figure 2 shows RAPSD changes with different shifts, i'm still struggling to understand why standard augmentations such as Random Masking or Blur are not sufficient?", "other_comments_or_suggestions": "Nothing in particular", "questions_for_authors": "- Why were TTT methods like MT3, TTT_MAE and Diffusion-TTA not included in the comparisons? These seem like competitive approaches worth benchmarking against.\n- How does SPA perform on more realistic distribution shifts beyond those tested? like for instance objectnet\n- Have you explored potential integration with foundation models?\n\n\n[1] Objectnet - https://papers.nips.cc/paper_files/paper/2019/hash/97af07a14cacba681feacf3012730892-Abstract.html", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fMjMLttfsj", "reviewer": "Reviewer_QKc5", "summary": "The paper explores the impact of typical distribution shifts on the information content of images across different spatial frequencies in the Fourier domain. It highlights that low-frequency components dominate in terms of information power, and removing these components provides more effective learning signals compared to masking high-frequency components. Based on this observation, the authors propose a data augmentation strategy that involves randomly masking the low-frequency amplitudes of an image in the Fourier domain. Additionally, they inject noise into the image to enhance the information power of high-frequency components, compensating for the loss of learning signals at lower frequencies. Experimental results demonstrate that this method, whether applied on its own or as a plug-and-play module, improves performance in tasks such as classification, segmentation, and 3D monocular detection across both transformer-based and CNN-based models.\n\n## update after rebuttal\nThanks to the authors for their response. I do not have any further queries.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your valuable feedback and your recognition of the novelty and contributions of our work for designing a challenging versatile fully TTA framework. Our SPA incorporates several components, including an active, deterioration-driven self-bootstrapping scheme (distinct from feature-level BYOL), as well as carefully crafted insights into geometry-preserving augmentation strategy design. The overall design of SPA with these innovations is challenging and non-trivial based on our analysis. These innovations collectively ensure the stability and effectiveness of SPA for online fully test-time adaptation. Thank you again for your insightful suggestion. We will make the novelty and contributions of our work clearer in the revised version.", "claims_and_evidence": "The main claim of this paper centers around the validity of the proposed data generation scheme, supported by experiments. Table 1 provides evidence for classification tasks, Table 3 validates the approach on object detection, and Table 4 focuses on segmentation. Additionally, Table 5 includes an ablation study that appears to substantiate the points made, sufficiently supporting the conclusions drawn. However, the novelty of the SPA approach may be somewhat overstated in the paper; it presents ideas that are not particularly new.", "methods_and_evaluation_criteria": "The TTA benchmark is well-established, and this paper runs most of the convincing datasets, with a sufficiently large testing scale to support its claims.", "theoretical_claims": "The augmentation scheme proposed in this paper relies more on empirical experimental validation. While Figure 2 provides some insightful theoretical basis, it is not sufficiently theoretical to fully support the proposed method.", "experimental_designs_or_analyses": "The experiments are comprehensive, with no gaps in the ablation studies. All argued applications have been validated through experimentation.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "This paper is well-written.", "other_comments_or_suggestions": "Please emphasize augmentation more than SPA in future versions.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y1jLg14mdQ", "reviewer": "Reviewer_tvoR", "summary": "This paper proposes Self-bootstrapping for versatile test-time adaptation, a general TTA framework that adapts models across classification, regression, and dense prediction tasks without requiring source data. SPA introduces weak-to-strong self-bootstrapping learning, ensuring adaptation by aligning a deteriorated (weak) view’s predictions with the original (strong) view using Fourier-based augmentations—low-frequency amplitude masking and high-frequency noise injection.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">Q1. Results on CIFAR-100 and ResNet\n\nIn the submission, we compare SPA on ImageNet-C/R/Sketch/A as they are large-scale, more challenging than CIFAR-100, and commonly used. We now report more results on CIFAR100-C below to further validate SPA. **Pls see Reviewer 61cM's Q1 for more ResNet results**.\n|Method|Avg Acc on CIFAR100-C|\n|-|:-:|\n|CoTTA|69.6|\n|EATA|72.1|\n|ActMAD|74.0|\n|DeYO|74.0|\n|ROID|76.3|\n|SPA(Ours)|75.4|\n|SPA+ActMAD+Tent|77.0|\n|SPA+ROID|77.4|\n>Q2. Clarification on \"rounds\" in segmentation exps\n\n“Round” refers to the number of repeated sequence of datasets to simulate long-term continual TTA (e.g., Fog-Night-Rain-Snow-Fog-…). This is a standard evaluation protocol in continual TTA following CoTTA. We will make this clearer.\n>Q3. Comparison with more baselines\n\nAs you suggested, we compare SPA with more baselines in the table below on classification, showing our efficacy. For detection, we acknowledge that the suggested baselines are designed for 2D tasks, while we focus on more challenging 3D monocular object detection. Due to time limits, it is non-trivial to implement them in our 3D setting. Here, we would like to clarify that MonoTTA (ECCV'24) is tailored for 3D detection and is the current SOTA (more latest than suggested ones), making it more relevant for comparison. Thus, we believe the current thorough comparisons with MonoTTA have fully validated SPA's effectiveness. We will discuss all mentioned methods in the revision.\n\nTable. More results on ImageNet-C with ViT-B\n|Method|Avg Acc|\n|-|-|\n|ROID|68.0|\n|ViDA|60.8|\n|CMF|69.2|\n|SPA(Ours)|70.1|\n|SPA+ActMAD|71.2|\n|SPA+ActMAD+TENT|71.6|\n>Q4. The method's complexity is limited, ...\n\nSPA's main contribution lies in its overall design for versatile online fully TTA. It comprises an active, deterioration-driven self-bootstrapping scheme and geometry-preserving augmentations inspired by our Fourier analysis of domain shifts.\n\nUnlike prior methods enforcing consistency across different augmentations with a teacher-student scheme, our SPA generates a deteriorated image to create an information gap, enabling self-distillation *in a single model* from strong to weak predictions. The distilled knowledge is then directly fed back to the strong branch via shared parameters, forming a closed-loop TTA process.\n\nThough simple in form, SPA’s insights and overall design are non-trivial and challenging, reflected in the result of extensive preliminaries where direct consistency learning or traditional augmentations failed to deliver satisfactory performance. We also believe such method simplicity helps enhance both usability and impact.\n>Q5. The experiments lack a clear focus, ...\n\nOur goal is to design a versatile TTA method, so we evaluate SPA across multiple tasks. Actually, our validations are thorough: we verified our superiority as standalone or as a plug-and-play module on different tasks, e.g., outperforming prior SOTAs like DeYO (ICLR'24) in classification and MonoTTA (ECCV'24) in 3D detection. We also provided extensive ablations and analyses for different tasks. As you suggested, we also compare with more baselines to verify our efficacy, pls see Q3 and Q9.\n>Q6. The method could benefit from stronger empirical analysis to validate its effectiveness\n\nIn Figure 2, we empirically analyze how domain shifts manifest in Fourier domain, and derive insights of which image deteriorations can supply sufficient learning signals for our self-bootstrapping. Extensive experiments (Tables 5,6,7) further validate the conclusions drawn from Figure 2. These analyses are also positively acknowledged by other reviewers, e.g., *“analyses…particularly insightful”* [DdZn] and *“sufficiently supporting the conclusions drawn”* [QKc5].\n>Q7. Adaptation efficiency\n\nOur SPA is much more efficient than augmentation-based CoTTA, which uses 2 or 34 augmentations per sample, while SPA needs only 1–2. It also matches the efficiency of entropy-based SAR. FPS on ImageNet-C with ViT-Base (on A100) are: SPA-I(125)>SAR(102)>SPA(79)>CoTTA(36). Here, SPA-I is a variant that applies two deteriorations within a single image (1 aug) and achieves similar ImageNet-C acc: SPA(70.1%) vs SPA-I(69.0%). Partial results were in Appendix C and we will make this clearer.\n\n>Q8. Efficacy on VLM\n\nOur SPA works on VLM, pls see Reviewer DdZn's Q4.\n>Q9. Results under continual or mixed-domain TTA in classification\n\nOur SPA works under such scenarios, pls see Reviewer DdZn'Q3.\n>Q10. Computation cost of augmentation\n\nOur augmentations are efficient: 1) Fourier masking uses PyTorch's FFT/iFFT, accelerated via parallelized GPU operations; 2) Noise injection is a trivial element-wise addition. Augmentation takes only a tiny (negligible) fraction of the total adaptation time, and the FPS of SPA is almost the same whether using our augs or MoCo/SimCLR’s.\n\nWe deeply appreciate your constructive comments. We sincerely hope our clarifications above have addressed your questions and can improve your opinion of our work.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Almost make sense.\n\nThis paper aims to demonstrate that the proposed Fourier-based augmentations work across various image-based tasks, including classification, segmentation, and 3D detection. However, the evaluation could benefit from a clear primary focus, such as image classification, to better align with existing TTA benchmarks. Expanding the evaluation to include CIFAR-10-C and CIFAR-100-C datasets and incorporating additional backbones like WideResNet and ResNet would strengthen the comparison with prior TTA methods and provide a more comprehensive assessment of generalizability.", "theoretical_claims": "No theoretical claim and proofs.", "experimental_designs_or_analyses": "Many new TTA works on image classification were published in 2024, and comparing SPA with them on CIFAR-10-C, CIFAR-100-C, DomainNet, and additional architectures (e.g., WideResNet, ResNet) would strengthen its evaluation.\n\nFor segmentation, the use of \"rounds\" is unclear—if the same test samples appear multiple times, this is not true online TTA; if not, the sample split criteria per corruption type need clarification.\n\nFor object detection, the MemCLR (WACV 2023) method might be considered to provide a more complete comparison.", "relation_to_broader_scientific_literature": "This paper extends TTA by adapting test samples into a weak-to-strong self-bootstrapping framework for stable adaptation. SPA introduces Fourier-based augmentations—low-frequency masking and high-frequency noise injection—to provide stronger adaptation signals while preserving geometric structure. This augmentation is general across classification, segmentation, and detection tasks, and might benefit to supervised learning (not verified)", "essential_references_not_discussed": "A lot of TTA method for image classification missed:\n\n1. Roid, published in WACV 2024;\n2. ViDA, published in ICLR 2024;\n3. A Versatile Framework for Continual Test-Time Domain Adaptation: Balancing Discriminability and Generalizability, published in CVPR 2024;\n4. CMF, published in ICLR 2024;\n5. SLWI, ICML 2024, etc.\n\nFor object detection:\n1. IoU-filter published in CVPRW 2024\n2. Memclr published in wacv 2023", "other_strengths_and_weaknesses": "Strengths:\n\n- The Fourier-based augmentation strategy has potential for broader applications in various image-based tasks beyond TTA.\n\n- The paper evaluates the method across different image-based tasks (classification, segmentation, and detection), demonstrating its generalizability.\n\nWeaknesses:\n\n- The method's complexity is limited, as it primarily modifies the augmentation strategy within an existing consistency-based TTA framework, which may not be a major conceptual leap.\n\n- The experiments lack a clear focus, spreading across multiple tasks but not providing in-depth validation on any single one.\n\n- The high-/low-frequency augmentation approach could benefit from stronger empirical analysis or theoretical justification to validate its effectiveness.\n\n- Adaptation efficiency and latency are not discussed, which are crucial for real-world TTA deployment.\n\n- Additional comparisons with recent TTA methods (e.g., 2024 TTA classification works, MemCLR for detection) would strengthen the evaluation.", "other_comments_or_suggestions": "See the comments.", "questions_for_authors": "Have you explored the effectiveness of this augmentation strategy on Vision-Language Models (VLMs) like CLIP and SigLIP?\n\nYour experiments include continual adaptation for segmentation—does the strategy also work for continual or mixed-domain TTA in image classification?\n\nHow does the computational cost of the proposed Fourier-based augmentations compare to standard TTA augmentations? Will it slow the adaptation time?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper is recommended for unanimous acceptance by all reviewers, three of whom engaged with the authors’ responses. The work presents a novel and unified approach to both classificaiton and regression in test-time adaptation (TTA), with extensive evaluation performed on different tasks and architectures. The AC checked the paper and all reviewing material and concurred with acceptance.
Decision: Accept (poster)