PAID: Pairwise Angular-Invariant Decomposition for Continual Test-Time Adaptation
Authors: Kunyu Wang
Xueyang Fu
Yuanfei Bao
Chengjie Ge
Chengzhi Cao
Wei Zhai
Zheng-Jun Zha
Venue: NeurIPS 2025 poster ()
Keywords: Continual Test-Time Adaptation
Decomposition
Domain-Invariant
Primary area: applications
TL;DR: We show that the pairwise angular structure of pre-trained weights serves as a domain-invariant semantic anchor for continual test-time adaptation.

Abstract: Continual Test-Time Adaptation (CTTA) aims to online adapt a pre-trained model to changing environments during inference. Most existing methods focus on exploiting target data, while overlooking another crucial source of information, the pre-trained weights, which encode underutilized domain-invariant priors. This paper takes the geometric attributes of pre-trained weights as a starting point, systematically analyzing three key components: magnitude, absolute angle, and pairwise angular structure. We find that the pairwise angular structure remains stable across diverse corrupted domains and encodes domain-invariant semantic information, suggesting it should be preserved during adaptation. Based on this insight, we propose PAID (Pairwise Angular Invariant Decomposition), a prior-driven CTTA method that decomposes weight into magnitude and direction, and introduces a learnable orthogonal matrix via Householder reflections to globally rotate direction while preserving the pairwise angular structure. During adaptation, only the magnitudes and the orthogonal matrices are updated. PAID achieves consistent improvements over recent SOTA methods on four widely used CTTA benchmarks, demonstrating that preserving pairwise angular structure offers a simple yet effective principle for CTTA. Our code is available at https://github.com/wangkunyu241/PAID.

PAID: Pairwise Angular-Invariant Decomposition for Continual Test-Time Adaptation
Kunyu Wang, Xueyang Fu, Yuanfei Bao, Chengjie Ge, Chengzhi Cao, Wei Zhai, Zheng-Jun Zha' University Science and Technology of China
(kunyuwang, baoyuanfei, cjge, chengzhi cao @mail wut.uuucc.....o .dd. (xyfu, wzhai056, zhazj @ustc. edu.cr cn
Abstract
Continual Test-Time Adaptation (CTTA) aims to online adapt pre-trained model changing enviromments during inference. Most existing methods focus exploiting target data, while overlooking another crucial source of information, the pre-trained weights, which encode underutilized domain-invariant priors. This paper takes the geometric attributes of pre-trained weights as a starting point, systematically analyzing three key components: magnitude, absolute angle, and pairwise angular structure. We find that the pairwise angular structure remains stable across diverse corrupted domains and encodes domain-invarianl semantic information, suggesting it should be preserved during adaptation. Based on this insight, we propose PAID (Pairwise Angular-Invariant Invariant Decomposition), prior- driven CTTA method that decomposes weight into magnitude and direction, and introduces a learnable orthogonal matrix via Householder reflections to globally rotate direction while preserving the pairwise angular structure. During adaptation, only the magnitudes and the orthogonal matrices are updated. PAID achieves consistent improvements over recent SOTA methods on four widely used CTTA benchmarks, demonstrating that preserving pairwise angular structure offers a simple yet effective principle for CTTA. Our code is available at atthtps:: httttps///gthub com/ comm/vangkumyyu241/PA11
1
Introduction
Deep Neural Networks (DNNs) have achieved remarkable success in various computer vision However, when there exist domain discrepancies between training and 57 58), directly applying source pre-trained model may cause significant performance degradation, particularly when the target distribution is unpredictable and continually changing over time. This challenge in real-world scenarios has motivated the emergence of Continual Test-Time Adaptation (CTTA) 60), which aims to adapt source pre-trained models during inference to evolving test data, making it especially suitable for practical applications.
tasks 337 38
73
80
testing environments
14
47
In CTTA, models rely two key sources of information: the prior knowledge encoded in pre-traie traitra source weights and the streaming data from the target domains. Most existing methods 26 62 63
focus adapting the target data, while treating pre-trained weights static initialization, with their potential largely overlooked. However, these weights, learned from large-scale supervised training 4 (48). may encode transferable priors that remain invariant across domains. We posit that leveraging such invariances in parameter space can help address core challenges in CTTA, including catastrophic forgetting and error accumulation. This work explores this direction by uncovering
*Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

the
leyate
Sepule
Valen
twen
14
Sckiom
Magnitude Varlation
(2) Absolute Angle Varlation
(3) Pairwise Angular Structure Varlation
Figure 1: (Experiment 1) Visualization of cross-domain variation of three geometric properties. Pairwise angular structure remains stable under corruption but varies under semantic shift, suggesting it encodes semantic-relevant domain-invariant information. In contrast, magnitude and absolute angle fluctuate irregularly across domains, reflecting domain-specific shifts.
Nomain-invarrinn components in pre-trained weights, offering a new perspective on knowledge retention and transfer for continual adaptation.
1.1 Motivation
Recent studies in hyperspherical learning [31 52 weights, rather than their magnitudes, primarily encodes discriminative semantic information crucial for visual recognition. Similar observations have been made in generative models [46], where the pairwise angular structure among neurons effectively preserve semantic consistency after fine-tuning on different domains. These findings suggest that beyond treating neural weights as individual scalars, the geometry of weight space, particularly the angular components, may encode invariant semantic priors derived from pre-training. Motivated by this, we ask: can the angular configuration of pre-trained weights serve stable semantic anchor during continual adaptation? To investigate this hypothesis, we design three sets of targeted experiments that isolate and examine the respective roles of magnitude and angular components in the adaptation process.
reveal that the angular component of neuron
Specifically, neuron weights, as exemplified by the linear projection matrix = [wi, ......... wk] Rdxk, can be decomposed into a magnitude M E Rlxk and unit-length direction W e Rdxx
W = M WW where
||w,|| 0is i
=
Wi
11
Building on this view, we identify three geometric attributes of the weight space that are relevant to adaptation: (1) the magnitude, which determines the scaling feature responses; (2) the absolute angle, referring to the orientation of each unit direction vector in the feature space, which may rotate during adaptation; (3) the pairwise angular structure, defined as the set of angles between all pairs of unit direction vectors, capturing how weight vectors are arranged relative one another.
Experiment 1 examines the domain invariance of geometric attributes in weight space by analyzing their variation under different conditions. We use a ViT-Base model pre-trained on ImageNet and conduct two types of experiments: (1) Corruption, where the model performs test-time adaptation on each of the 15 corrupted domains in ImageNet-C; and (2) Semantic shift, where the model is fine-tuned on CIFAR-100 to induce semantic changes, serving as reference. quantify the average variation in geometric attributes, we define three metrics: (1) magnitude variation A.,, (2) absolute angle variation AA, and (3) pairwise angular structure variation ÁS, quantified using hyperspherical energy 31 46 which refers to the sum of hyperspherical similarity between all pairwise neurons:
AM(W), 3 = 1
AA(W1, W2) = E (1 --oos( cos( w))) |
(2)
AS(Wi,W²'
Σ
|0
|
Eity i
1|: - -x03 1-| where i,j E (1....,A
... k}, (3)
MMor experimental details can be found in Appendix

---

Y
B (2) Direction Adjustment
a
Pre-trained Weight Vectors
(I) Magnitude Adjustment
(3) Preserving Pairwise Angular Structure
Figure 2: (Experiment 2) Illustration of three update strategies. (1) Scaling the magnitude of each
vector; (2) Independently rotating each vector, altering their absolute angle; (3) Jointly rotating all
vectors while preserving their pairwise angular structure.
Goussian
Gouzeian
Pisel
Inguise
Pixel
Impuise
Mor Eme Rate
Elastic
Deficus
Elastic
Deficrs
Conrast
Glaes
Contrast
Glass
Mem Eme Rote Mi
Bright
Motion
Rright
Motion
Fog
Zoom
Fog
Zoom
14.0%
Magnitude- Dircction
Magnitude only
Direction only
Magnitude + Direction
Direction only
[Preserving pairwise structure)
(Preserving pairwise
Figure 3: (Experiment 2) Radar and bar charts showing classification error rates across 15 corruption domains and their mean. Comparisons between settings (5 vs 3; 5 vs 4; 3 vs 2) show that pairwise angular structure is domain-invartie when constrained by fixed angular structure, are domain-specific and beneficial for adaptation.
component worth preserving, while magnitude and direction,
As shown in Fig. I the pairwise angular structure remains consistently stable across all corruption domains, with minimal changes in hyperspherical energy. contrast, exhibits substantial variation under semantic shift. This contrast reveals that pairwise angular structure is well preserved under per- turbations and varies response semantic changes. In contrast, both magnitude and absolute angle exhibit irregular variations across domains, indicating that their changes are driven by adaptation to domain-specific statistical differences rather than semantics. These findings provide strong empiri- cal support that pairwise angular structure encodes semantic-relevant, domain-invarian properties, whereas magnitude and absolute direction are more responsive to comain-specific statistical shifts.
Experiment 2 conducts an ablation study to clarify the contribution of geometric attributes to adaptation. Based on the decomposition, we design three update strategies, as shown in Fig. - (1) magnitude adjustment, enabling learning of the magnitude matrix M; (2) direction adjustment, enabling learning of unit direction matrix W; and (3) direction adjustment while preserving pairwise angular structure, implemented via orthogonal rotation [5] 16. 69) direction matrix. Note that changing the pairwise angular structure inherently changes absolute angle, but not vice versa. Thus, (3) represents constrained subset of (2). Using these strategies, we construct five settings for continually adapting ViT-Base model (pre-trained ImageNet) to the 15 ImageNet-C domains: (I) magnitude only; (2) direction only; (3) direction only while preserving pairwise structure; (4) magnitude direction; and (5) magnitude + direction while preserving pairwise structure.
As shown in Fig. setting (5) outperforms settings (1) and (3), while settings (2) and (4), both of which alter the pairwise angular structure, lead to clear performance drop compared to (3) and (5). This contrast underscores the role of pairwise angular structure as a domain-invariant component that should be preserved during adaptation. In contrast, magnitude and direction, when adjusted under the constraint of maintaining relative angular geometry, serve as domain-specific components that are adaptable and beneficial for enhancing performance in CTTA.

---

Tuput Images
Magnitude Direction
Dérection onlv
Brightmess
Magaitude- (preserving pairwise structure)
(preserving pairwise structure)
Figure 4: (Experiment 3) Attention map visualizations for two setting pairs (5 vs 4; 3 vs 2). The comparison further supports the semantic relevance and domain invariance of pairwise angular structure in CTTA.
Experiment further substantiates the earlier conclusions from a perceptual perspective by visu- alizing attention maps under the five settings in Experiment 2. As shown in Fig. 4 preserving the pairwise angular structure allows the model to focus on crucial semantic regions. In contrast, settings (2) and (4) that alter this structure lead to diffused and misaligned attention, with the model failing to capture core object information. This comparison reinforces the critical role of preserving pairwise angular structure for achieving cross-domain invariance.
1.2 Contribution
These three sets of -tatinnttttttttttcicaaal analysis, functional validation, and visual interpreta- colleceleyyyvely reveal key insight: the pairwise angular structure of neural weights encodes semantically relevant and Nomain-invarian prior derived from pre-training, which should be preserved during CTTA. In contrast, the magnitude and absolute angle, when adjusted under the constraint of preserving the pairwise angular structure, serve as domain-speciff components that enable effective adaptation to the target domains.
Building on this insight, we propose Pairwise Angular-Invaree Invariatn Decomposition (PAID) for CTTA. PAID explicitly decomposes pre-trained weights into magnitude and direction matrices. To maintain the pairwise angular structure during adaptation, we introduce a learnable orthogonal matrix con- structed via Householder transformations, enabling global rotation of directions without altering their relative angular configuration. Leveraging this orthogonality, we freeze the original direction matrix and update only the magnitude and the injected orthogonal matrix, ensuring structure- preservnng adaptation. In summary, the contributions can be summarized as follows:
We identify the pairwise angular structure of pre-trained weights as semantic prior that should be preserved during CTTA, supported by statistical, functional, and visual analyses.
We propose PAID, novel prior-driven CTTA method that preserves the pairwise angular structure of weights while enabling controlled adaptation their magnitudes and directions through orthogonal transformations.
PAID achieves consistent improvements over recent SOTA methods on four standard CTTA benchmarks, demonstrating strong effectiveness and generalizability.
2 Related Work
Continual Test-time Adaptation (CTTA) aims to online adapt source pre-trained model to handle
a sequence of target domains. Existing learning paradigms can broadly categorized into three
types [26] (66) 62 63) pptimization-based methods aim to adjust pre-trained models by designing new
objectives, including statistics calibration [I] 68, consistency regularization [29] [1 66),
entropy minimization 41 55 78), and pscudo-labeling
20 70). Data-based methods focus

---

enhancing data diversity or mitigating the impact of distributional shifts. Typical strategies include data augmentation 36 74 and memory bank [2] Model-based methods enhance adaptation
by modifying or extending the model architecture, including module addition [20 29], module
substitution [19], and prompt-based mechanisms [9 TO 50 76]. While similar form model- based methods, our method introduces a novel perspective by leveraging transferable priors encoded in pre-trained weights. We exploit the pairwise angular structure as a Nomain-invariann prior to enhance CTTA.
Carameter-Efficient Fini-Tuning (PEFT) reduces adaptation cost by freezing most large model parameters and updating only a small, task specifc subset. Representative methods include adapter- based methods (15), low-rank adaptation [6] 75] and prompt tuning [22] 45). Among them, DoRA [30] and OFT [46] are particularly insightful: DoRA decomposes weights into magnitude and direction, applying low-rank updates solely to the direction for near full-tuning expressiveness; OFT adopts layer-shared orthogonal transformations to fine-tune only the direction of weights, preserving the angular structure critical for semantic consistency generative models. Inspired by these, we introduce the idea of weight decomposition into the field of test-time adaptation, rethinking from the lens of generalization. Through systematic analysis of pre-trained weight space, we find that the pairwise angular structure remains stable across domains and encodes domain-invariant semantics. This observation, supported by statistical, functional, and visual evidence, motivates us to explicitly preserve angular structure during CTTA.
Methodology
3.1 Preliminaries
Given model fa pre-trained on the source domain D, = y. our goal is to adapt this model to a sequence of continually changing target domains {D), D?...... ...... ). In online setting, the model fo processes a sequence of test data batches [B, ) with each batch B, arriving at time step t. Consistent with prior work (60), assume that all sam- ples in batch B, come from the same tar- get domain, though the domain identity is unknown. At each time step I, CTTA aims to adapt the model parameters from 0, 9.41 by learning from the current batch B, thereby enhancing performance subse- quent batches.
Weight
Merge Weight W E R
W, E BR
Decompose Merge W M off - (xx
Preserving Pairwise Angalar
Structure
Pre-trained Weight
a
Global Rotation
Maznitude
M M
Decompose
Geometry
A Vision Transformer model consists of multiple encoder layers, each containing a Multi-Head Attention (MHA) block and a Feed-Forward Network (FFN) block. The MHA block utilizes three key linear weights W, for the query, Wa for the key, and W. for the value compute atten- tion scores and aggregate weighted values from normalized feature representations. In addition, linear weight W, combines the outputs of all attention heads. In the FEN block, the input is processed through two linear weights Wm, and Wm, with a GELU function applied between them. This work focuses on decomposing the pre-trained weights of these linear layers W,, Wx, W., W., Wmy and Wyg preserving their -nnvariat priors to enhance CTTA task. Fig. provides overview of our method.
Magaltude
M R
Frozen
Birentia
# 3'
Trainable
x
Direction
Orthogonal Mairis
O 2
Figure 5: PAID decomposes pre-trained weights into magnitude and direction. To preserve pairwise angular structure, we introduce a learnable orthogonal matrix, enabling global rotation. Only magnitude and orthogo- nal matrices are updated during adaptation.
5

---

3.2 Pairwise Angular-Invariant Decomposition (PAID)
Following Eq. linear projection matrix W [10,, w.....,, | R can be decomposed into a magnitude matrix M E R and a direction matrix W [101, u2...... wpg] E IR as:
W = M O W. where
| wis.
|| =1.
(4)
Empirical evidence suggests that the pairwise angular structure of pre-trained weights, defined as the angles between all pairs of unit direction vectors, encodes semantically meaningful and domain-invarian prior. This structure should be preserved during the CTTA process. In contrast, magnitude and absolute angle, when adapted under the constraint of preserving this structure, serve as domain-specific components that support effective adaptation. Accordingly, we make the magnitude matrix M learnable while freezing the original direction matrix W. To enable directional adaptation without disrupting angular structure, we introduce a learnable orthogonal matrix R which performs global rotations while preserving pairwise angular structure:
O W + Mo ( (W.O).
(5)
real square matrix Ri orthogonal if it satisfies O T. Such matrix represent distance-pressrvingg linear transformation, including rotation and reflection. For any vector e H the transformation æ + Oz preserves the Euclidean norm, that is, |Ox||| = ||=||] For any pair of vectors (z, y). it also preserves inner products: (Or, Oy) geometry among the vectors remains unchanged. Therefore, applying an orthogonal transformation to set of vectors performs a global rotation without altering their pairwise angular structure.
(æ,, y) As result, the relative
To construct the orthogonal transformation, we adopt the Householder reflection formulation 5 16 699 A Householder reflection is a linear transformation that reflects a vector across hyperplane perpendicular to a unit vector. Given u E R with ||u||2 = 1, the corresponding Householder matrix is defined as:
= 1=1_ T,
(6)
where E lx k is orthogonal and symmetric, satisfying H H and dct (H) = 1, indicating reflection. The transformation preserves Euclidean norms and inner products, i.e.,
|axl2 = ||0||2 (Hv,Hy) = (z. Vz R**
thereby maintaining both vector lengths and pairwise angles. Since Householder matrices are orthogonal, and the product of orthogonal matrices remains orthogonal, a general orthogonal matrix O E R* can be constructed as a chain of Householder reflections:
0 I H. II 2u;u,), uy E SK- E=1
This parameterization is expressive: when k, it can represent any element in the orthogonal group O(k), while smaller r yields a trade-off between representational capacity and efficiency. Therefore, we adopt such chain to construct the Iearnable orthogonal transformation used for structure-preserving adaptation and r denotes orthogonal matrix coefficient.
3.3 Optimization Objective
In line with prior CTTA works 39 between the source and target domains. The core idea is to reduce the domain shift by aligning the first and second moments (i.c., mean and standard deviation) of features extracted from both domains. To ensure consistency, all features are extracted from the same location in the backbone: the CLS token output after the final layer normalization and before the classification head in ViT-Base. This representation used for both source and target domains throughout the CTTA process.
40 77), we adopt feature distribution alignment strategy
We pre-compute the source domain statistics (p., O.) using randomly sampled subset of 500 images from the source domain D.. This process is performed offline, and no further access to source data needed at test time. The resulting statistics are compact and stored for use during adaptation. At each test-time step new target batch B.' arrives, where the subscript t indicates the target domain. The features of Bd" denoted as 2f_ are extracted from the same CLS token position. We compute

---

the batch-wise mean pf and standard deviation of of the target features. To align the source and target distributions, we define the following objective:
L
||, / 1l Allo2 - of 112,
where À is a weighting hyper-parameter that balances the contribution of the two terms.
3.4 Intuitive Explanation of PAID
To explain the intuition for why PAID works, draw analogy from the frequency domain decomposition images. In the Fourier transform [43], an image can be decomposed into magnitude and phase spectrum. The magnitude captures energy distribution, contrast, and intensity, and is highly sensitive to domain-specific variations such as noise and style. In contrast, the phase encodes angular alignment with Fourier bases, which determines the structural layout and semantic content. The separation between semantic structure and coomain-specifer appearance has been widely 67]. For instance, APR [2 reveals that networks are more sensitive to magnitude perturbations, while phase is crucial for retaining semantic information and achieving robust recognition. FACT
explored in frequency-based domain generalization [8 21 25 27
[65] verifies the donain-invariant property of phase by showing that transferring phase yields better generalization than amplitude. This perspective supports our design PAID. The pairwise angular structure, akin the phase spectrum, captures semantics and remains invariant across domains. PAID preserves the angular structure while adapting the magnitude and direction via constrained orthogonal transformations, thus achieving structure- preeturvi-r adaptation across domains.
3.5 Theoretical Justification of PAID
To justify the design of PAID, analyze the effect of domain shifts the angular structure of weights using simplified linear classification model. This reveals why a shared orthogonal transformation suffices for adapting domain corruptions but not semantic shifts.
Setup. Let x E IR be an input vector and w. IR the weight vector for class c (I,..., CJ. The predicted label is:
j(x) = arg max w
Case 1: Corruption (Class-agnostic Transformation). Suppose all inputs are transformed by the same differentiable function x' d(x). Using first-order Taylor expansion around clean data:
d(x) = Sx +
where S E _axa is the Jacobian matrix, assumed invertible and independent of class Under this assumption, the optimal target-domain classifier that preserves original class boundaries satisfies:
W = S Wez
Ve.
We write S RH (polar decomposition), where is orthogonal (R R= 1) and H is symmetric positive definite. The transformed weights are:
w/ =H=wHa
The cosine similarity between any two transformed weights is: w. H2wa
cos <(w' w')) = WeW HHall
Near-isotropic assumption. We assume 122 RS 721 for some scalar 7 > 0. This holds when the degradation has approximately equal effect in all directions, such Gaussian blur, light noise, or JPEG compression. Under this mild and physically reasonable assumption, pairwise angles are approximately preserved:
cos L(w', w')) x cos <(we, wa).
Case 2: Semantic Shift (Class-dependent Transformation). Now suppose each class undergoes different transformation:
x' = do(x) xe S.x + ge: S. GL(a).
7

---

Table 1: Mean classification error rate (%) and gain (%) on ImageNet-to-lmageNee- using ViT-Base. Bold indicates the best performance. aen-ggrined performances are shown Appendix C
Method
Pacudo
TENT [55]
VDP
SAR
RoTTA
EcoTTA [1]]
VIDA
C-MAE
Mean, 55.8
66.8
$1.0
54.8
50.0
45.6
48.2
48.0
43.4
42.5
42.2
Gaint 0.0
-11.0
+4.8
+1.0
+10.2
+7.6
+7.8
+1244
+13.3
+33.6
Table 2: Mean classification error rate (%) and gain (%) on CIFARI00-to -EEARI00-o-CIFAAROOO using ViT-
Base. Bold indicates the best performance. Fine-graines performances shown in Appendix [
Method
Source
Pseudo (24]
TENT [SS]
CoTTA [60]
VDP 9
ViDA 29
C-MAE [28]
Ours
Mcan Gaint
35.4
33.2
32.1
34.8
32.0
27.3
26,4
24.9
0.0
+2.2
+3.3
+0.6
+3.4
+8.1
+9.0
+10.5
The new classifier becomes:
w/ = S. we-
Since S. varies across classes, there is generally no single matrix 7 such that w" = -wwe for all c, and therefore:
cos <ww": wx) * cos (w wa).
Semantic shifts necessarily change the pairwise angular structure.
Conclusion. When domain shifts are class paass-ggnsticc and locally differentiable, they can be represented by a shared invertible matrix S, and the optimal classifier corresponds to applying the inverse transpose S-T to all weights. This transformation preserves angles up to near-isotropic assumption, justifying why PAID only needs learn shared orthogonal rotation and magnitude adjustment. When the shift is class-dependent, this shared structure no longer exists, and angular geometry must change.
4 Experiments
we compare our method with state-of-the-art approaches on both classification and segmentation CTTA benchmarks. Section 4.3 analyzes the effectiveness of our design choices and investigates the influence of the injected matrix coefficient, test batch size, number of source examples, and computational overhead. Additional ablations, including the effect of the loss coefficient, the choice of injection layer, and 10-round classification CTTA evaluation, and experiments on convolutional backbones, are provided in Appendix and D
In Section 4.2
4.1 Experimental Setup
Datasets. We evaluate our method on three classification CTTA benchmarks: CIFARIO-IC CIFARIOC, CIFARIO0-to-CIEE
100C (23), and ImageNee-too-maagee ImageNet-C [14. In the classification tasks, we follow the sequential adaptation process described in [51]. where the pre-trained source model adapts to each of the 15 target domains, each defined by the highest corruption severity. Online prediction results are immediately assessed after processing the input. For segmentation CTTA, we method
CCyCC where Cityscapes (3] serves the source domain and ACDC 449 as the target domains, which includes images captured under four distinct unobserved visual conditions: Fog, Night, Rain, and Snow. To simulate continual environmental changes, cyclically iterate through the same sequence of target domains (Fog - Night - Rain - Snow) three rounds, reflecting real-world scenarios.
Methods Compared. We compare our method against several strong CTTA baselines, including Source, Pesudo (24), TENT (55), COTTA (60], DePT [10), VDP 9. SAR (42). RoTTA 168), EcoTTA [5T], ViDA (2), and C-MAE (28). "Source" represents the use the pre-trained model for adaptation without any specific method. The selection of comparison methods is based on their open- source availability and representativennss, with implementations and results drawn from publicly available codebases, paper descriptions, and established benchmarks.
Implementation Details.
classification CTTA tasks, we use ViT-base 7 as the backbone
model, resizing input images to 384x384 for CIFAR 10-C and CIFAR 100-C, and to 224x224 for the ImageNet-C benchmark. For the segmentation CTTA task, we employ Segformer-B5 [64 pre-trained on the Cityscapes dataset as the source model, down-sampling input images from 1920x 1080 to

---

Table 3: Mean classification error rate (%) and gain (%) on CUFARRO-to-CIARRIU using ViT-Base.
Bold indicates the best performance. Fine-grained performances are shown in Appendix C
Method
Source
Pseudo 24
TENT 55
CoTTA
66
VDP 9
ViDA 29
C-MAB 28
Ours
M 28.1
26.9
23.5
24.6
24.1
20.7
12.6
11.0
Gaint 0.0
+1.2
+4.6
+3.5
+4.0
+7.4
+15.5
+17.1
Table Mean mloU (%) and gain (%) on Cityscapes-t--ACDD (3-round average) using Segformer- B5. Bold indicates the best performance. Fine-grained performances are shown in Appendix
Metric
Source
TENT 55
CoTTA 660
DePT 10
VDP
SAR [42
BcoTTA 51
VIDA 29
C-MAE 28
Ours
Meant
56.7
55.7
58.6
53.4
58.2
57.0
55.8
61.9
61.8
62.2
Gaint
0.0
-1.0
+1.9
-3.3
+1.5
+0.3
-0.9
+5.2
+5.1
+5,5
optimizer is used with parameters (3), B2) (0.9, 0.999). Hyper- parameters for CIFARIO-C, CIFAR100-C, ImageNet-C, and ACDC are set as follows: batch size (64, 64, 64, 1). orthogonal matrix coefficient {12. 12, 12, 12). and loss coefficient {1.0, 1.0, 0.1, 1.0). To initialize the learnable parameters, we perform warm-up iterations on classification datasets. For linear layer selection, block (q. k, V, o) and the MLP block (m). The number of source examples is set to 500. The experiments are conducted the NVIDIA RTX 3090 GPU. All ablation studies are conducted ImageNet-to-IImgeeNrr unless otherwise specified.
960x540. The AdamW 35
inject all linear layers, including those in the multi-head attention
4.2 Results on Benchmark Datasets
rrepors classification eπrоr rates ImageNet- under corruption severity level 5. Our method achieves the lowest mean error of 42.2%, outperforming strong baselines such as SAR (45.6%), ViDA (43.4%), and C-MAE (42.5%) Compared to the source model, our method yields substantial improvement of + +13.6%. In Table 2 we present results on CIFAR 100-C. Our method achieves the best overall performance with mean error of 24.9%, surpassing recent leading methods including ViDA (27.3%) and C-MAE (26.4%). The performance gain over the source model reaches + +10.5% Table 3soww the results on the simpler CIFAR dataset. Our method achieves mean error of 11.0%, which is a +17.1 % improvement over the source model and also outperforms the best baseline (C-MAE 12.6%). This confirms the effectiveness of our adaptation mechanism even on small-scale datasets. For the segmentation task, Table Sabble/reppo reports the average mloU on the Cityscapes ACDC benchmark over three consecutive adaptation rounds. Our method achieves the highest average mloU of 62.2%. showing robust performance across rounds and delivering a consistent +5.5% improvement over the source model.
4.3 Ablation Analysis
Effect of Each Component. We conduct ablation studies to validate the effectiveness of key design choices in PAID, as shown in Tab. 5 A central finding is that preserving the pairwise angular structure during adaptation is essential for strong CTTA performance. Directly adjusting the direction matrix, which allows each weight vector rotate independently, disrupts the relative geometry and leads to performance degradation. In contrast, PAID freezes the original direction matrix and applies learnable orthogonal transformation rotate all vectors jointly. This enables adjustment of absolute angles while preserving their relative angular structure. We also compare two general parameter- efficient tuning methods: LoRA [17]. which adds low-rank matrices to linear layers, and DoRA [30], which decomposes linear weights into magnitude and direction and applies LoRA to the directional component. Both methods perform sub-optimally in our setting, suggesting that generalization in CTTA demands tailored, customized designs.
Effect of Injected Matrix Coefficient. We perform an ablation study on the number of orthogonal matrices used in the Householder transformation chain (denoted as the coefficient r), as shown in Fig. (a). While increasing the number of matrices enhances the model's expressive capacity, we observe that CTTA performance does not improve monotonically with the number of matrices. A possible reason is that the corrections required for domain perturbations do not demand overly complex transformations, and higher complexity may negatively affect model convergence in online adaptation.

---

Table 5: Ablation on design choices. "Adjust Magn." denotes whether the magnitude ma- trix is updated, "Adjust Dir." denotes whether the direction matrix is updated, and "Inject Orth." indicates whether the direction matrix frozen and rotated via orthogonal matrices.
Table 6: Computational analysis of different meth- ods. "#Param." denotes the number of learnable parameters, while "#Extra Param." refers to addi- tional parameters introduced during CTTA. "#FP" and "#BP" indicate the forward and backward prop- agation times, respectively. "Time" represents the relative computation time (normalized by TENT).
Adjust Magn.? Adjust Dir.? Inject Orth.? Mean, Gaint
Baseline
0.0
Method
#Parmm.
EExtra Param.
dEP
ABP
Time
For. Mean]
K
46.0
+9.8
X
51.3
+4.5
CoTTA
S1
86.57M
11.7
3.6
54,8
+3.6
VDP
1800
1.5
50.0
X
44.1
+11.7
VIDA 29
7.13M
11
2.8
43.4
+13.6
1.24M
42.2
LoRA
17
49.7
+6.1
DoRA
30
48.1
+7.7
Mean Error Raie (%)
45.0
.14
Maan Eiror Rate (%)
46.0-457
Mean Rate (%)
Tai
45.0
44.0-
To0
VIDA
CUTTA
PAID
45.0
42.7
43.5
43.0
42.
42.7
800
423
4211
43.1
$2.0
42.0
12
24
32
10
100
200
303
500
1000
3002
5000
255
(a)
(b)
Figure 6: Ablation on (a) the coefficient of the injected orthogonal matrix, (b) the number of source domain samples, and (c) the test-time batch size
Our experiments reveal that the best performance is achieved when the number of matrices is set to 12, indicating an efficient trade-off between capacity and stability.
Effect of Number of Source Examples. We investigate the sensitivity of PAID to the amount of source data by varying the number of images used pre-compute source-domain statistics from 0 to 5,000. As shown in Fig. (b), our method achieves strong performance with as few as 500 source images. worth emphasizing that these statistics are computed once prior CTTA and are not involved in the online adaptation process. Moreover, storing the computed statistics incurs negligible memory cost. These results demonstrate that PAID requires minimal source-domain information to function effectively, making it practical for real-world scenarios.
Effect of Test Batch Size. To comprehensively evaluate the impact of test-time batch size, compare various CTTA methods under batch sizes ranging from to 256. As shown in Fig. (c). consistent trend emerges across all methods: regardless of their objective functions, performance remains stable with sufficiently large batches but deteriorates as the batch size decreases. In the extreme case of sagle-sample adaptation, all methods suffer substantial performance drops. In addi- tion, our method maintains decent performance as long as the batch size exceeds 4, and consistently outperforms all comparison methods beyond this threshold.
A naayysi of Computation, Parameter, and Latency. We analyze the computational complexity of different methods in Tab. comparing the number of learnable parameters, the number of forward and backward passes, and relative runtime. While achieving the best overall performance, our method maintains relatively small number of learnable parameters and avoids the repeated forward passes required by methods such as CoTTA, VDP, and ViDA. As result, the increase in computational cost remains moderate, striking good balance between efficiency and effectiveness.
Conclusion
This work demonstrates that the pairwise angular structure of source pre-trained weights encodes the domain-invariant semantic prior, supported by statistical analysis. functional validation, visual evidence, intuitive explanation, and theoretical justification. Leveraging this insight, we decompose weights into magnitude and direction, allowing magnitudes to adapt freely while constraining directional updates to global rotations via chained Householder transformations. This preserves the intrinsic angular structure during adaptation. Extensive experiments validate its efficacy.
10

---

Acknowledgements
This work was supported by the National Natural Science Foundation of China (NSFC) under Grants 62225207, 62436008, 62422609 and 62276243.
References
[1] Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 295-305, 2022.
[2] Guangyao Chen, Peixi Peng, Li Ma, Jia Li, Lin Du, and Yonghong Tian. Amplitude-phase recombination: Rethinking robustness of convolutional neural networks in frequency domain. In Proceedings of the IEEE/CVF international conference on computer vision, pages 458-467,
2021.
[3] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 3-233, 2016.
[4] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. leee, 2009.
[5] Wei Dong, Yuan Sun, Yiting Yang, Xing Zhang, Zhijun Lin, Qingsen Yan, Haokui Zhang, Peng Wang, Yang Yang, and Hengtao Shen. Efficient adaptation of pre-trained vision transformer via householder transformation. arXiv preprint arXiv:2410.22952, 2024.
[6] Wei Dong, Dawei Yan, Zhijun Lin, and Peng Wang. Efficient adaptation of large vision transformer via adapter re-composing, Advances in Neural Information Processing Systems, 01552248-525661 2023.
[7] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x 16 words: Transformers for image recognition at scale. arXiv preprint 1:Xiv:2010.11929 2020.
[8] François Fleuret et al. Test time adaptation through perturbation robustness. In NewrlPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
[9] Yulu Gan, Yan Bai, Yihang Lou, Xianzheng Ma, Renrui Zhang, Nian Shi, and Lin Luo. Decorate the newcomers: Visual domain prompt for continual test time adaptation. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 7595-7603, 2023.
[10] Yunhe Gao, Xingjian Shi, Yi Zhu, Hao Wang, Zhiqiang Tang, Xiong Zhou, Mu Li, and Dimitris Metaxas. Visual prompt tuning for test-time domain adaptation. arXiv preprint arXiv:2210.04833 2022.
[11] Taesik Gong, Jonghcon Jeong, Taewon Kim, Yewon Kim, Jinwoo Shin, and Sung-Ju Lee. Note: Robust continual test-time adaptation against temporal correlation. Advances in Neural Information Processing Systems, 35:27253-27266 2022.
[12] Taesik Gong, Yewon Kim, Taeckyung Lee, Sorn Chottananurak, and Sung-Ju Lee. Sotta: Robust test-time adaptation on noisy data streams. Advances in Neural Information Processing Systems, 2023.
[13] Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403. /4608, 2024.
[14] Dan Hendryeks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint
2019.

---

[15] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly, Parameter-efficiee transfer learning for nip. In International conference on machine learning, pages 2790-2799. PMLR, 2019.
[16] Alston S Householder. Unitary triangularization of a nonsymmetric matrix. Journal of the ACM (JACM), 5(4):339-342, 1958.
[17] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li. Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. JCLR, 1(2):3, 2022.
[18] Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu. Fsdr: Frequency space domain randomization for domain generalization. In Proceedings of the IEEF/CVÉ conference on computer vision and pattern recognition, pages 6891-6902, 2021.
[19] Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. Advances Neural Information Processing Systems, 34:2427-244
2021.
[20] Minguk Jang. Sae- Young Chung, and Hye Won Chung. Test-time adaptation via self-training with nearest neighbor information. arXiv preprint arXiv:220.110792, 2022.
[21] Scogkyu Jeon, Kibeom Hong, Pilhyeon Lee, Jewook Lee, and Hyeran Byun. Feature stylization and domain-avee aware contrastive learning for domain generalization. In Proceedings of the 29th ACM International Conference Multimedia, pages 22-31, 2021.
[22] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European conference on computer vision, pages 709-727. Springer, 2022.
[23] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[24] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, page 896. Atlanta, 2013.
[25] Sangrok Lee, Jongseong Bae, and Ha Young Kim. Decompose, adjust, compose: Effective normalization by playing with frequency for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pages 11776-1 1785, 2023.
[26] Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey test-time adaptation under distribution shifts. International Journal of Computer Vision, 133(1):31-64 2025.
[27] Shiqi Lin, Zhizheng Zhang, Zhipeng Huang. Yan Lu, Cuiling Lan, Peng Chu, Quanzeng You, Jiang Wang. Zicheng Liu, Amey Parulkar, et al. Deep frequency filtering for domain generalization. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1 1199-118807 2023.
[28] Jiaming Liu, Ran Xu, Senqiao Yang, Renrui Zhang, Qizhe Zhang, Zehui Chen, Yandong Guo, and Shanghang Zhang. Continual-mas -mae Adaptive distribution masked autoencoders for continual test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28653-28663 2024.
[29] Jiaming Liu, Senqiao Yang, Peidong Jia, Renrui Zhang, Ming Lu, Yandong Guo, Wei Xue, and Shanghang Zhang. Vida: Homeostatic visual domain adapter for continual test time adaptation. arXiv preprini arXiv:23066 00344, 2023.
[30] Shih- Yang Liu, Chien-Yi Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation. In Forty- first International Conference Machine Learning, 2024.
Shih-Yang
[31] Weiyang Liu, Rongmei Lin, Zhen Liu, Lixin Liu, Zhiding Yu, Bo Dai, and Le Song. Learning towards minimum hyperspherical energy. Advances neural information processing systems, 31, 2018.
12

---

[32] Weiyang Liu, Rongmei Lin, Zhen Liu, James M Rehg, Liam Paull, Li Xiong, Le Song, and Adrian Weller. Orthogonal over paraaaaameter training. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 7251 2021.
[33] Weiyang Liu, Zhen Liu, Zhiding Yu, Bo Dai, Rongmei Lin, Yisen Wang, James M Rehg, and Le Song. Decoupled networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2771-2779, 2018.
[34] Weiyang Liu, Yan-Ming Zhang, Xingguo Li, Zhiding Yu, Bo Dai, Tuo Zhao, and Le Song. Deep hyperspherical learning. Advances in neural information processing systems, 30, 2017.
[35] Uya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprini 17Xiv: 2017.
[36] Robert A Marsden, Mario Döbler, and Bin Yang. Universal test-time adaptation through weight ensembling, diversity weighting, and prior correction. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2555-2565, 2024.
[37] José Maurício, Inês Domingues, and Jorge Bernardino. Comparing vision transformers and convolutional neural networks for image classification: A literature review. Applied Sciences, 13(9):5521, 2023.
[38] Shervin Minace, Yuri Boykov, Fatih Porikli, Antonio Plaza, Nasser Kehtarnavaz, and Demetri Terzopoulos. Image segmentation using deep learning: A survey, IEEE transactions on pattern analysis and machine intelligence, 44(7):3523-355 -342,, 2021.
[39] Muhammad Jehanzeb Mirza, Pol Jané Soneira, Wei Lin, Mateusz. Kozinski, Horst Possegger, and Horst Bischof. Actmad: Activation matching to align distributions for test-time-trainn training.. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24152-24161, 2023.
[40] Shuaicheng NIu, Chunyan Miao, Guohao Chen, Pengcheng Wu, and Peilin Zhao. Test-time model adaptation with only forward passes. arXiv preprint arXiv:2404. 0.65000 2024.
[41] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International confer- ence on machine learning, pages 16888-16905. PMLR, 2022.
[42] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. arXiv preprint arXiv:2302. /2400, 2023.
[43] Henri J Nussbaumer and Henri Nussbaumer. The fast Fourier transform. Springer, 1982.
[44] Junyoung Park, Jin Kim, Hycongjun Kwon, Ilhoon Yoon, and Kwanghoon Sohn. Layer-wise auto-weighting for non-stationary test-time adaptation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1414-1423, 2024.
[45] Guanghui Qin and Jason Fisner. Learning how ask: Querying Ims with mixtures of soft prompts. arXiv preprint arXiv:2104.06599 2021.
[46] Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Schölkopf. Controlling text-to-image diffusion by orthogonal finetuning. Advances in Neural Information Processing Systems, 36.79320-79362, 2023.
[47] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International conference machine learning, pages 5389-5400. PMLR, 2019.
[48] Tal Ridnik, Emanuel Ben-Barude Asaf Noy, and Lihi Zelnik-Manor. Imagenet-21k pretraining for the masses. arXiv preprint arXiv:2104.10972 2021.
[49] Christos Sakaridis, Dengxin Dai, and Lue Van Gool. Acde: The adverse conditions dataset with correspondences for semantic driving scene understanding. In Proceedings of the IEFF/CVF international conference computer vision, pages 10765-10775, 2021.
13

---

[50] Manli Shu, Weili Nie. De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in -ainule models. Advances in Neural Information Processing Systems, 35:14274-14289, 2022.
[51] Junha Song, Jungsoo Lee, In So Kweon, and Sungha Choi. Ecotta: Memory--fficient continual test-time adaptation via self-distilled regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11920-11929, 2023.
[52] Cheng Tan, Zhangyang Gao, Lirong Wu, Siyuan Li, and Stan Li. Hyperspherical consistency regularization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7244-7255, 2022.
[53] Devavrat Tomar, Guillaume Vray, Behzad Bozorgtabar, and Jean-Phillipes Thiran. Tesla: Test- time self-learning with automatic adversarial augmentation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages
2023.
[54] Devavrat Tomar, Guillaume Vray, Jean- Shan-Philippe Thiran, and Behzad Bozorgtabar. Un-mixing test-time normalization statistics: Combatting label temporal correlation. arXiv preprint arXiv:2401.08328 2024.
[55] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006. 10726, 2020.
[56] Kunyu Wang, Xueyang Fu, Chengzhi Cao, Chengjie Ge, Wei Zhai, and Zheng-Jun Zha. Towards better de-raining generalization via rainy characteristics memorization and replay. arXiv preprint arXiv:2506.0244 024777 2025.
[57] Kunyu Wang, Xucyang Fu, Chengjie Ge, Chengzhi Cao, and Zheng-Jun Zha. Towards gen- eralized uav object detection: A novel perspective from frequency domain disentanglement. International Journal of Computer Vision, 132(1 1):5410: 2024.
[58] Kunyu Wang, Xueyang Fu, Yukun Huang, Chengzhi Cao, Gege Shi, and Zheng-Jun Zha. Generalized uav object detection via frequency domain disentanglement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1064-1073, 2023.
[59] Kunyu Wang, Xueyang Fu, Xin Lu, Chengjie Ge, Chengzhi Cao. Wei Zhai, and Zheng-Jun Zha. Efficient test-time adaptive object detection via sensitivity-guiian -uided pruning. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 10577-10586. 2025.
[60] Qin Wang, Olga Fink, Lue Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 7201-7211, 2022.
[61] Shuai Wang. Daoan Zhang, Zipei Yan, Jianguo Zhang, and Rui Li. Feature alignment and uniformity for test time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20050-20060, 2023.
[62] Zixin Wang, Yadan Luo, Liang Zheng, Zhuoxiao Chen, Sen Wang, and Zi Huang. In search of lost online test-time adaptation: A survey. International Journal of Computer Vision, pages 1-34, 2024.
[63] Zehao Xiao and Cees GM Snoek. Beyond model adaptation at test time: A survey, arXiv preprint arXiv:2411.03688 2024.
[64] Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems, 34:12077-12090, 2021.
[65] Qinwei Xu, Ruipeng Zhang. Ya Zhang, Yanfeng Wang, and Qi Tian. A fourier-based framework for domain generalization. In Proceedings of the IEEF/VV conference on computer vision and pattern recognition, pages 14383-14392, 2021.
14

---

[66] Xu Yang. Xuan Chen, Moqi Li, Kun Wei, and Cheng Deng. A versatile framework for continual test-time domain adaptation: Balancing discriminability and generalizability. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23731-23744 2024.
[67] Yanchao Yang and Stefano Soatto. Fda: Fourier domain adaptation for semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4085-4095 2020.
[68] Longhui Yuan, Binhui Xie, and Shuang Li. Robust test-time adaptation in dynamic scenarios. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pullern Recognition, pages 15922-15932, 2023.
[69] Shen Yuan, Haotian Liu, and Hongteng Xu. Bridging the gap between low-rank and orthogonal adaptation via householder reflection adaptation. arXiv preprint arXiv:2405.17484 2024.
[70] Longiin Zeng, Jiayi Han, Liang Du, and Weiyang Ding. Rethinking precision of pseudo label: Test-time adaptation via complementary learning. Pattern Recognition Letters, 177:96-102, 2024.
[71] Wei Zhai, Yang Cao, Jing Zhang, Haiyong Xie, Dacheng Tao, and Zheng-Jun Zha. On exploring multiplicity of primitives and attributes for texture recognition in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(1) (1:1))40333)))) 2023.
[72] Wei Zhai, Yang Cao, Jing Zhang, and Zheng-Jun Zha. Exploring figure-ground assignment mechanism in perceptual organization. Advances in Neural Information Processing Systems, 15:17030-17044 2022.
[73] Wei Zhai, Pingyu Wu, Kai Zhu, Yang Cao, Feng Wu, and Zheng-Jun Zha. Background activation suppression for weakly supervised object localization and semantic segmentation. International Journal of Computer Vision, 132(3):750-775. 2024.
[74] Marvin Zhang, Sergey Levine, and Chelsea Finn. Memo: Test time robustmess via adaptation and augmentation. Advances in neural information processing systems, 35:38629-3888 -36622, 2022.
[75] Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karampatziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adalora: Adaptive budget allocation for parameter- efficient fine-tuning, arXiv preprint arXiv:2303. 10512, 2023.
[76] Yunbei Zhang, Akshay Mehra, and Jihun Hamm. Dynamic domains, dynamic solutions: Dpcore for continual test-time adaptation. arXiv preprint arXiv:2406. 10737, 2024.
[77] Yunbei Zhang, Akshay Mehra, and Jihun Hamm. Ot-vp: Optimal transport-guided visual prompting for test-time adaptation. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 1122-1132. IEEE, 2025.
[78] Bowen Zhao, Chen Chen, and Shu-Tar Xia. Delta: degradation fully test-time adaptation. arXiv preprint 2rXIv:2301.13018 2023.
[79] Zhilin Zhu, Xiaopeng Hong, Zhiheng Ma, Weijun Zhuang, Yaohui Ma, Yong Dai, and Yaowei Wang. Reshaping the online data buffering and organizing mechanism for continual test-time adaptation. In European Conference on Computer Vision, pages 415-433. Springer, 2024.
[80] Zhengxia Zou, Keyan Chen, Zhenwei Shi, Yuhong Guo, and Jieping Ye. Object detection in 20 years: A survey. Proceedings of the IEEE, 111(3):257-22) 2023.
15

---

A Additional Details of Three Motivation Experiments
In Experiment compute weighted average of three statistics (mean, variance, and hyperspheri- cal energy) across all linear layers, followed by min-max normalization across different cross-domain settings to produce the visualization in Fig. [ The TTA results correspond our proposed method, PAID, applied under non-continual setting w where all linear layers are adapted. The fine-tune baseline refers to supervised fine-tuning of the pret-rained model on the CIFAR- dataset for 3 epochs, using warm-up and cosine annealing learning rate schedules. All linear layers and the classification head are learnable during this process. Experiment the orthogonal rotation is implemented using the chained orthogonal matrices described later in the paper. All hyperparameter settings are aligned with those specified for ImageNet-C: in our implementation details. For the attention map visualization in Experiment 3, we use the attention-rollout codebase
Rate (%)
41.0
Mean Error Rate (16)
32.0
(%)
240
24.1 230
43.5
43.5
43.3
430
47.7
26.6
42.5
20.0
U.0 JLL
42.0
ImageNet-CC
CIFARIOI-C
CIFARIOCC
Figure 7: Ablation on loss balancing coefficient
Table 7: Ablation the selection of linear layer injection
Ablation Gaussian
Shot
Impulse
Defocus
Glass
Motion
Zoom
Snow
Frost
Fog
Bright
Contrast
Elastic
Pixel
JPBG] Mean]
51.1
44.2
44.5
60.0
58.3
50.3
50.8
40.8
41.7
49.4
25.3
59.0
43.4
37.1
35.8
46.1
qk
52.3
48.5
47.2
66.2
67.7
56.9
60.1
46.1
44.7
58.8
28.1
77.8
49.8
40.1
39.4
50.7
44.5
44.2
62.8
59.1
53.0
53.5
43.4
43.4
54.1
27.0
64.1
43.8
37.5
37.3
47.9
glve
48.8
43.0
43.2
58.5
56.8
51.6
50.0
41.0
42.
48.5
26.7
57.8
44.0
38.6
37.5
45.9
48.8
43.7
44.4
49.4
49.6
47.3
44.2
37.5
39.4
42.1
25.2
50.0
39.3
35.5
36.5
42.2
Table 8: 10-Round CTTA classification results on ImageNet-C, CIFARI00-C, and CIFARIO-C.
Dataset
Setting
Gaussian
Shot
Impulse
Deficus
Glass
Motion
Zoom
Snow
Frost
Fog
Bright
Contrast
Elastic
Pixel
JPEG
Mean]
I-Round
48.8
43.7
44.4
49.4
.9.6
47.3
44.
37.5
39.4
42.1
25.2
50.0
39.3
35.5
36.5
42.2
ImageNet-C
10-Round
47.0
43.8
43.5
48.6
50.6
46.2
42.
39.5
38.5
40.5
25.5
47.2
39.2
34.5
36.4
41.6
CIFARTOB-C
10-Round
I-Round
35.5
31.9
31.9
20.4
18.6
19.5
35.7
25.1
16.7
20.5
18.1
19.3
21.5
12.1
19.8
19.5
26.2
29.7
35.2
24.9
24.3
I-Ronnd
22.9
11.8
16.7
10.8
7.4
74
6.6
11.4
4.5
9.3
12.8
9.4
14.5
11.0
114AR10-C
10-Round
10.3
B Additional Ablation Studies
Effect of Loss Balaneing Coefficient. Unlike the orthogonal matrix coefficient r, which shows consistent optimal value across the three classification benchmarks, the optimal value of the loss balancing coefficient exhibits outliers. Fig. show that ImageNet-C achieves optimal accuracy with much smaller À than CIFAR100- and CIFAR 10-C. ImageNet-C keeps its native resolution of 224 224, and most corruptions simply shift global intensity, so aligning feature means removes most of the domain shift and only small weight
variance alignment is needed. By contrast, CIFAR images are first enlarged from 32 x 32 to 384 x 384, which spreads each pixel and magnifies local artifacts, making second-order statistics more important; therefore, larger À that emphasizes variance alignment is required on the CIFAR variants.
Effect of Linear Layer Injection. We examine where to place the orthogonal update in Tab. ViT-Base contains five types of linear layers: q. k, v. o (attention output), and m (MLP). We find that the best performance is achieved when all linear layers are updated (gkvom), indicating that
16

---

Table 9: Classification error rate (%) for SeggeNettt--ImagNNeeee evaluated on ViT-Base with
corruption severity level Bold indicates the best performance.
Method
Gaussian
Shot
Inpalse
Defocus
Glass
Motion
Zoom
Snow
Frost
Fog
Bright
Contrast
Elastic
Pixel
IPEG
Mean| Gainf
Source
53.0
51.8
52.1
68.5
78.8
58.5
63.3
49.9
54.2
57.7
26.4
91.4
57.5
38.0
36.2
55.8
0.0
Pscudo
24
45.2
40.4
41.6
51.3
53.9
45.6
47.7
40.4
45.7
93.8
98.5
99.9
99.9
98.9
99.6
66.8
-11.0
TENT
55
52.2
48.9
49.2
73.0
54.5
5844
44.0
47.7
50.3
23.9
72.8
55.7
34.4
51.0
44.8
CoTTA
60
52.9
51.6
51.4
68.3
78.1
57.1
62.0
48.2
52.7
55.3
25,9
90.0
56.4
36.4
35.2
54.8
+1.0
VDP 9
52.7
51.6
50.1
58.1
70.2
56.1
58.1
42.1
46
45.8
23.6
70.4
54.9
34,5
36.1
50.0
+5.8
SAR 4
45.8
45.9
47.7
63.7
46.2
50.9
40.3
42.4
41.
24.4
53.4
53.6
36.6
45.6
+10.2
RoTTA 68
51.5
51.7
58.7
52.6
54.8
42.8
25.9
48.8
+7.6
Ecо [1
48.1
45.6
46.3
56.5
67.1
50.4
57.1
41.3
44.5
43.8
24.1
71.6
54.8
34.1
34,8
48.0
+7.8
VIDA 29
47.7
42.5
42.9
52.2
56,9
45,5
48.9
38,9
42.7
40.7
24.3
52.8
49.1
33.5
32.3
43.4
+12.4
C-MAE 28
46.3
41.9
42.5
54.9
43.3
40.7
34.2
35.8
64.3
23.4
60.3
37.5
29.2
31.4
42.5
+13.3
Ours
48.8
43.7
44.4
49.4
49.6
47.3
44.2
37.5
39.4
42.1
25.2
50.0
39.3
35.5
36.5
42.2
+13.6
Table 10: Classification error rate (%) for 1EFARI00-t-CCFARRII evaluated on ViT-Base with
corruption severity level 5. Bold indicates the best performance.
Method
Gaussian
Sbot
Impatise
Defocus
Glass
Mation
Zsom
Snow
Frust
Fog
Bright
Conrast
Elastic
Pixel
JPEG
Mcan Gain
Source
55.0
51.5
26.9
24.0
60.5
29.0
21.
21.1
25.0
35.2
34,8
43.2
56.0
35,9
35.4
Pscudo
53.8
48.9
25.4
23.0
58.7
27.3
20.6
23.4
313
39.6
52.3
33,9
33.2
+2.2
TENT 55
53.0
47.0
24.6
22.3
58.5
26.5
19.0
21.0
23.0
30.1
11.8
25.2
39.0
47.1
33.3
32.1
+3.3
CoTTA [6]]
55.0
$1.3
25.8
24.1
59.2
28.9
21.4
21.0
24.7
34.9
11.7
31.7
40.4
55.7
35.6
34.8
+0.6
VDP 99
54.8
51.2
25.6
24.2
59.1
28.8
21.2
20.5
23.3
33.8
7.5
11.7
32.0
51.7
35,2
32.0
ViDA
50.1
40.7
22.0
25.6
27.1
C-MAE 28
48.6
30.7
18.5
21.3
38.4
22.2
17.5
19.3
18.0
24.8
13.1
27.8
31.4
35.5
29.5
26.4
+9.0
Ours
40.7
319
20.4
19.8
35,9
23.0
16.3
20.5
18.2
25.3
12.6
19.8
29.4
28.2
31.3
24.9
+10.5
distributing the orthogonal correction across both attention and feed-forward paths is essential. This suggests that corruption- corruption-induced induced degradation affects the model in layer-wise and cumulative manner, and only full-layer adaptation can effectively counteract its impact.
10-Round Classification CTTA. To further evaluate the stability and effectiveness of our method, conduct 10-round CTTA experiments on ImageNet-C, CIFAR 100-C, and CIFAR O-CC. Specifically, cycle through the fifteen corruption domains ten times. As shown in Tab. 8 our method remains stable throughout and even achieves slight performance improvements as adaptation progresses. The values reported in the "I0-round" row of Tab. represent the average results over the rounds.
C Fine-grained CTTA Performance
In this section, we extend the classification and segmentation results reported in our submission by presenting detailed, fine- tne-grained performance analysis. Specifically, evaluate classification error rate and average mloU score across distinct corruption types. To complement the summary results shown in Tab. to Tab. we provide additional detailed results in Tab. to 'ab.. [2] These comprehensive evaluations demonstrate the robustness and effectiveness of our approach in various CTTA scenarios, including weweeeettto-mmeeeee-eee CIFAR-IO-to-CIFAL 10-C. CIFAR- 100-t-- LIFAR-100-CE and CoySSC.
D Extension to Convolutional Backbones
Beyond Vision Transformers, the core idea behind PAID, i.e., preserving the pairwise angular structure between weight vectors, can be naturally extended to convolutional networks, since each convolutional layer is also linear operator with well-defined geometric structure.
In a standard convolutional layer, each output channel corresponds filter of shape (Cin- kis kin. We flatten this into vector of length D) Cjn kn. kw. treating each filter as vector in D. Stacking
17

---

Table 11: Classification enror rate (%) for CIEARI0++-CIFARIOC, evaluated on ViT-Base with corruption severity level Bold indicates the best performance.
Method
Gaussian
Shot
Impalie
Defocus
Glass
Motion
Zoom
Snow
Frust
Fog
Bright
Contrast
Elastic
Pixel
JPE
Mean
Gainf
Source
60.1
53.2
38.3
19.9
35,5
22.6
18.6
12.1
12.7
22.8
533
49.7
23.6
24.7
23.1
28.1
0.0
Pscudo
59.8
52.5
37.2
19.8
35.2
21.8
17.6
11.6
12.3
20.7
5.0
41.7
21.5
25.2
22.1
26.9
+1.2
TENT 55
57.7
56.3
29.4
16.2
35.3
16.2
1499
4.7
22.5
15.9
29.1
19.5
+4.6
CoTTA
60
58.7
51.3
33.0
20.1
34.8
20.0
15.2
11.1
11.3
18.5
4.0
34.7
18.8
19.0
17.9
24.6
+3.5
VDP
9
57.5
49.5
31.7
21.3
35.1
19.6
15.1
10.8
10.3
18.1
4.0
27.5
18.4
22.5
19.9
24.1
+4,0
ViDA 29
52.9
47.9
19.4
11.4
31.3
13.3
7.6
7.6
9.9
12.5
3.8
26.3
14,4
33.9
18.2
20.7
+7.4
C-MAB
30.6
10.4
13.9
8.8
8.5
Ours
22.9
11.8
9.9
9.1
16.7
10.8
7.4
7.4
6.6
11.4
4.5
9.3
12.8
14.5
11.0
+17.1
Table 12: Average mIoU score (%) for Cityscapes-to-AG
evaluated on Segformer-Bi The same target domains are repeated three rounds. Bold indicates the best performance.
Round
Round 2
Round
Method
Meunt
Gaint
Fog
Night
Rain Snow
Meant
Fog
Night
Rain
Snow
Mcant|
Fog
Night
Rain
Snow
Meant
Source
69.1
40.3
39.7
57.8
56.7
69.1
40.3
59.7
57.8
56.7
69.1
40.3
59.7
$7.8
56.7
56.7
0.0
TENT 65
69.0
40.2
60.1
57.3
56.7
68.3
39.0
60.1
56.3
55.9
67.5
37.8
59.6
55.0
55.0
55.7
-1.0
CoTTA 60
70.9
41.2
62.4
59.7
58.6
70.9
41.1
62.6
59.7
58.6
70.9
41.0
62.7
59.7
58.6
58.6
+1,9
DePT 10
71.0
40.8
58.2
56.8
56.5
68.2
40.0
55.4
53.7
54.3
66.4
38.0
47.3
47.2
49.7
53.4
-3.3
VDP
70.5
41.1
62.1
59.5
58.3
70.4
41.1
62.2
59.4
58.2
70.4
41.0
62.2
59.4
58.2
58.2
+1.5
SAR 42
69.0
40.2
60.1
57.3
56.7
69.0
40,3
60.0
67.8
59.3
67.5
37.8
59.6
55.0
55.0
$7.0
+0.3
Eco' TA [1
68.5
35.8
62.1
57.4
56.0
68.3
35,5
62.3
57.4
55.9
68.1
35.3
62.3
57.3
55.8
55.8
-0.9
ViDA
29
71.6
43.2
66.0
63.4
61.1
73.2
44.5
67.0
63.9
62.2
73.2
44.6
67.2
64.2
62.3
61.9
+5.2
C-MAE
28
71.9
44.6
67.4
63.2
61.8
71.7
44,9
66.5
63.1
61.6
72.3
45,4
67.1
63.1
62.0
61.8
+5.1
Ours
169.6
45.5
68.0
60.7
61.0
72.3
45.2
66.8
62.5
61.7
72.6
46.9
68.4
63.8
62.9
62.2
+5.5
these column-wise yields a weight matrix W E PxC which we decompose as:
W MOW,
where W contains unit- norm columns and M E L stores the magnitudes. During adaptation, we freeze W and update M along with a learnable orthogonal matrix O E Gue Con, constructed via Houscholder reflections. The updated weights become:
W M (. O..)
Here, O operates along the output channel dimension. The spatial structure within each filter remains intact because we do not rotate across spatial positions. Notably, flattening convolution filters and treating them as vectors is a standard practice in low-rank adaptation and model compression literature. Reshaping back after rotation ensures that spatial locality is fully preserved during convolution.
To validate this extension, we follow standard CTTA setups used in prior works: WideResNet-28 on CIFAR-10-C, ResNeXt-29 on CIFAR-100-C, and ResNet-50 on ImageNet-C. We select representative and open-sourced baselines for comparison, using benchmark results that have been consistently reported and validated across multiple CTTA studies. As shown in Tab. [3] PAID achieves consistent performance gains across all convolutional backbones, indicating that PAID generalizes beyond ViTs and is also applicable to CNNs.
To further explain why PAID remains valid in convolutional networks, we revisit several prior studies on the geometric structure of convolutional filters. DCNet (33] shows that the convolution operation can be reformulated as inner product and decomposed into magnitude and angular: the magnitude better models intra-class variation, while the angular captures semantic differences. SphereConv [34] further demonstrates that learning only the angular component of convolutional filters sufficient for semantic classification. These findings indicate that the angular structure in CNNs also carries domain-invaree semantics. Therefore, applying PAID's strategy to convolutional networks not only formally valid, but also empirically supported by prior work.
18

---

Table 13: Extension of PAID to convolutional backbones. Test error (%) on three CTTA benchmarks.
Dataset
Model
Source
TENT [55]
CoTTA 660
AdaCon [
CRG
LAW
Ours
ImageNet-C
ResNet-50
82.0
62.6
62.7
65.5
59.1
60.1
58.4
CIFAR 100-C
ResNeXI-29
46.4
60.9
32.5
33.4
29.0
30.9
28.1
CI-ARIO-C
WideResNet-28
43.5
20.7
16.1
18.5
15.9
15.7
15.0
E
Intuition Behind the Performance Drop with Small Batch Sizes
As shown in Fig. (c), most existing CTTA methods, including ours, exhibit notable performance drop as the batch size decreases. This behavior is not specific to our approach, similar trends have been reported in prior works such as [68] (Fig. 4(d)) and (51) (Tab. 5), where small-batch scenarios consistently lead to degraded adaptation performance.
For our method in particular, the primary factor is the reliability of target feature statistics. In Eq. 9 we perform alignment between source and target distributions using the mean and standard deviation computed from the current test batch. When the batch size is small, these estimates become highly variable and noisy, which results unstable gradients and impairs the optimization process. This instability ultimately hinders adaptation effectiveness.
In contrast, larger batch sizes produce more reliable and smoother estimates of distributional statistics, effectively reducing variance in the adaptation signal. This leads to more stable parameter updates and improved learning dynamics. Empirically, we observe that performance becomes acceptable when the batch size exceeds 4, and saturates when the batch size is greater than 32.
F
Limitations
While PAID shows consistent gains under diverse corruptions, is built upon a central hypothesis: the angular structure learned by source pre-trained models is potentially generalizable across diverse target domains. This assumption has been preliminarily validated through experiments, yet its applicability boundaries remain unclear, particularly under extreme conditions. Moreover, hyper- parameter tuning remains a common challenge in the CTTA community, underscoring the need for automated tuning strategies to enhance practical usability. Finally, extending the idea of angular structure preservation to tasks such as few-shot learning, and continual learning presents a promising direction, where maintaining the geometric stability of weights may likewise prove essential.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please see Introduction 1
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please see Conclusion and Limitations
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
20

---

Justification:
For each theoretical result, we provide the full set of assumptions and complete and correct proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Please Implementation Details in Experiments 4
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
21

---

Answer: [Yes]
Justification:
Please see Datasets and Methods Compared in Experiments
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
caiiioooooyyyyl for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ //ippe.cc/pullic/guuii wwwpc//ggus////eee CoodeSubeiiieeeec ssi asionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please see Experimental Setup in Experiments
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: Our work does not involve specific error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
22

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please Implementation Details in Experiments 4
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https:
Answer: [Yes]
Justification: Our paper conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: We only use public available datasets.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
23

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All the creators or original owners of assets including code, data, and models used in the paper properly credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwwprspipetattteeeeee thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets

---

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
This paper does not involve crowd-sourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowd-sourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact core methodology, scientific rigorousness, originality of the research, declaration is not required.
25

---

Answer:
[NA]
Justification:
Only for writing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non standadd components.
Please refer to our LLM policy (https: /tttps://neurpps. cc//oomferenccs/222551/11 for what should or should not be described.
26


Reviews:
[{"review_id": "c6XaOheqG1", "reviewer": "Reviewer_KZqM", "summary": "This paper tackles the problem of Continual Test-Time Adaptation (CTTA) for vision models. Building on an empirical finding that pairwise angular relationships among weight vectors remain stable across diverse corruption domains while changing primarily with semantic shifts, the authors propose PAID (Pairwise Angular-Invariant Decomposition). PAID factorizes each linear weight into a magnitude term and a direction term, then freezes the original pairwise angles and adapts only through a global orthogonal rotation—implemented via chains of Householder reflections—plus magnitude updates. This design preserves the geometry believed to encode domain-invariant semantics while allowing flexibility to absorb distribution drift. Extensive experiments on ImageNet-C, CIFAR10/100-C, and Cityscapes demonstrate the effectiveness of the method.", "questions": "Please refer to the Weaknesses.", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Positive Points:\n1. The work approaches CTTA from the weight-space geometry perspective, identifying pairwise angles as a domain-invariant signal and building an algorithm tightly linked to that observation.\n2. Extensive experiments demonstrate the effectiveness of the proposed method.\n\nNegative Points:\n1. The details regarding the feature alignment process described in Section 3.3 require further clarification. Specifically, it is unclear whether Z_t^T denotes features extracted from layer T for sample t within a batch. Additionally, it would be helpful to specify whether source domain features are extracted in the same manner (i.e., from the same layer) or if only the final layer features are used. \n2. The authors mainly conduct experiments on the Vit-Based model, does this method can be applied on ResNet models on the ImageNet-C, Cifar-100-C and Cifar-10-C datasets?\n3. The authors employ Householder reflection to construct the orthogonal transformation matrix. It would be beneficial to clarify the motivation behind choosing this particular formulation. Are there specific advantages it offers over alternative methods for generating orthogonal matrices? Furthermore, it would be useful to analyze whether the performance of the method is sensitive to the choice of orthogonal transformation strategy.", "was_revised": true, "final_justification": "The authors have adequately addressed my concerns by providing additional experiments using the ResNet backbone, as well as offering further clarification of their proposed approach. I will maintain my score of acceptance.", "rebuttal": "Thank you for your valuable and constructive comments. We appreciate your recognition of our work's novel perspective on CTTA from the view of weight-space geometry and the tight algorithmic design based on this insight. Below, we address your concerns and questions individually:\n\n## **Q1: Clarity about the $Z_{t}^{T}$ in Eq. (9) and the procedure for computing source and target statistics**\nThank you very much for pointing out this ambiguity. We agree that our notation in Eq. (9) was unclear and could lead to confusion.\n\nTo clarify: in $Z_{t}^{T}$, the subscript \\(t\\) indicates that the features are from the target domain (in contrast to source domain \\(s\\)), and the superscript \\(T\\) refers to the test-time step, meaning the index of the currently arriving target batch in the continual adaptation process.\n\nFor each incoming batch $B_{t}^{T}$, we extract features using the **CLS token output after the final layer normalization and before the classification head** in the ViT-Base. This CLS token serves as the feature representation for each image in the batch.\n\nThe source domain statistics $(\\mu_s, \\sigma_s)$ are computed in exactly the same way: we randomly sample 500 source images prior to test-time, extract their CLS token features from the same location, and compute their mean and standard deviation. These statistics are pre-computed once and reused throughout adaptation, with no further access to the source data.\n\nWe will revise Section 3.3 to explicitly clarify these. We will also release our code on GitHub so that the details can be better understood.\n\n## **Q2: Extension to convolutional networks**\nThank you for your forward-looking suggestion. The key insight of PAID, preserving the pairwise angular structure between weight vectors to retain domain-invariant semantics, can be **formally** extended to convolutional networks, where each convolutional layer is also a parameterized linear transformation.\n\nIn a standard convolutional layer, each output channel corresponds to a filter of shape $(C_{\\text{in}}, k_h, k_w)$. We flatten this into a vector of length $D = C_{\\text{in}} \\cdot k_h \\cdot k_w$, treating each filter as a vector in $\\mathbb{R}^D$. Stacking these column-wise yields a weight matrix $W \\in \\mathbb{R}^{D \\times C_{\\text{out}}}$, which we decompose as:\n\n$\nW = M \\odot \\hat{W},\n$\n\nwhere $\\hat{W}$ contains unit-norm columns and $M \\in \\mathbb{R}^{1 \\times C_{\\text{out}}}$ stores the magnitudes. During adaptation, we freeze $\\hat{W}$ and update $M$ along with a learnable orthogonal matrix $O \\in \\mathbb{R}^{C_{\\text{out}} \\times C_{\\text{out}}}$, constructed via Householder reflections. The updated weights become:\n\n$\nW' = M \\odot (\\hat{W} \\cdot O).\n$\n\nHere, $O$ operates along the output channel dimension. The spatial structure within each filter remains intact because we do not rotate across spatial positions.\nNotably, flattening convolution filters and treating them as vectors is a standard practice in low-rank adaptation and model compression literature. Reshaping back after rotation ensures that spatial locality is fully preserved during convolution.\n\nTo **validate** this extension, we follow standard CTTA setups used in prior works: WideResNet-28 on CIFAR-10-C, ResNeXt-29 on CIFAR-100-C, and ResNet-50 on ImageNet-C. We select representative and open-sourced baselines for comparison, using benchmark results that have been consistently reported and validated across multiple CTTA studies. The results show **consistent performance gains** across all convolutional backbones, indicating that PAID generalizes beyond ViTs and is also applicable to CNNs.\n\n| Dataset     | Model          | Source | TENT[31]  | CoTTA[9] | AdaCon[36] | CRG(1) | LAW(2) | Ours |\n|-------------|----------------|--------|-------|-------|--------|--------|------|------|\n| ImageNet-C  | ResNet-50       | 82.0   | 62.6  | 62.7  | 65.5   | 59.1   | 60.1   | **58.4** |\n| CIFAR100-C   | ResNeXt-29    | 46.4   | 60.9  | 32.5  | 33.4   | 29.0   | 30.9   | **28.1** |\n| CIFAR10-C  | WideResNet-28   | 43.5   | 20.7  | 16.1  | 18.5   | 15.9   | 15.7   | **15.0** |\n\nTo further **explain** why PAID remains valid in convolutional networks, we revisit several prior studies on the geometric structure of convolutional filters. DCNet [18] shows that the convolution operation can be reformulated as an inner product and decomposed into magnitude and angular: the magnitude better models intra-class variation, while the angular captures semantic differences. SphereConv [15] further demonstrates that learning only the angular component of convolutional filters is sufficient for semantic classification. These findings indicate that the angular structure in CNNs also carries domain-invariant semantics. Therefore, applying PAID’s strategy to convolutional networks is not only formally valid, but also **empirically supported** by prior work.\n\n## **Q3: Motivation and sensitivity analysis of using householder reflection for orthogonal transformation**\n\nWe thank the reviewer for this constructive question. We choose to use the Householder reflection primarily due to **computational efficiency** considerations. Below, we elaborate on our motivation through a comparison with alternative implementations and an analysis of sensitivity.\n\n### Alternative: cayley transformation\nAnother common mathmatics tool for constructing orthogonal matrices $ R \\in \\mathbb{R}^{d \\times d} $ is the cayley transformation, which maps a skew-symmetric matrix $ A \\in \\mathbb{R}^{d \\times d} $, where $ A^\\top = -A $, to an orthogonal matrix:\n\n$\nR = (I + A)(I - A)^{-1}.\n$\n\nThis formulation underlies two recently proposed methods:\n- **OFT** [20]: Constructs an orthogonal matrix by applying the Cayley transformation to multiple learnable skew-symmetric blocks.\n- **BOFT** (3): Builds a butterfly-style composition of sparse Cayley-transformed blocks for efficient approximation.\n\n### Comparison of parameter and computational complexity\nWe refer to the analysis in Section 3.2 and Table 1 of [21], which compares these two types of methods:\n- **Parameter complexity**: Both Cayley-based methods and the Householder-based method require approximately $ \\mathcal{O}(d^2) $ parameters.\n- **Computational complexity**: Cayley-based methods involve matrix inversions and dense multiplications, resulting in **$ \\mathcal{O}(d^3) $** complexity. In contrast, the Householder-based method only involves vector inner products and scale-vector multiplications, yielding a lower cost of **$ \\mathcal{O}(d^2) $**.\n\nThis **one-order-of-magnitude improvement** in computational complexity makes Householder reflection a more efficient choice for constructing orthogonal matrices.\n\n### Sensitivity analysis\nWe construct orthogonal matrices following [20] and (3), and conduct a comparative experiment on the ImageNet-C benchmark. All methods perform similarly, with only a slight performance drop, suggesting that the overall performance is **not sensitive** to the specific choice of orthogonalization strategy.\n\n| Method        | Type         | Mean Error on ImageNet-C  |\n|---------------|--------------|-----------------------------|\n| OFT [20]       | Cayley      |             42.9            |\n| BOFT (3)      | Cayley       |             42.4            |\n| Ours          | Householder  |             42.2            |\n\nThat said, we recommend the Householder reflection approach. Given its **comparable expressiveness and parameter complexity** to Cayley-based methods, its **lower computational cost** makes it more suitable for CTTA, where efficient online adaptation and reduced overhead are essential.\n\n(1) Reshaping the Online Data Buffering and Organizing Mechanism for Continual Test-Time\nAdaptation, ECCV 2024.\n\n(2) Layer-wise Auto-Weighting for Non-Stationary Test-Time Adaptation, WACV 2024.\n\n(3) Parameter-efficient Orthogonal Finetuning via Butterfly Factorization, ICLR 2024."}, {"review_id": "lenULWKdb9", "reviewer": "Reviewer_jqAQ", "summary": "This paper uses the geometric properties of pre-trained weights for the task of Continual Test-Time Adaptation (CTTA). The key insight is that the pairwise angular structure of weight vectors is domain-invariant and encodes important semantic information. Based on this, the paper proposes Pairwise Angular-Invariant Decomposition (PAID) that decomposes the weights of linear layers into magnitude and direction. During adaptation, it keeps the original direction vectors frozen but allows their magnitudes to be updated. PAID also introduces a learnable orthogonal matrix, parameterized by Householder reflections, which applies a global rotation to all direction vectors simultaneously. The method is evaluated on several benchmarks for image classification and segmentation and shows consistent performance improvements over existing methods.", "questions": "1. The performance drop with small batch sizes is significant. Are there any intuitions?\n2. How does PAID perform on CNN architectures like ResNet?\n3. The proposed method is sensitive to hyperparameter. Did you also tune parameters for baselines to achieve the best performance for each of them?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper addresses the Continual Test-Time Adaptation (CTTA) problem, which is a challenging and practical scenario.\n2. The paper is well written and easy to follow.\n3. The paper provides extensive experiments (ImageNet-C, CIFAR-10/100-C, ACDC), showing the effectiveness and versatility of the proposed method.\n4. Although the idea of decomposing weights into magnitude and direction has been explored in other contexts (e.g., DoRA, OFT), it is relatively new in CTTA domain.\n\n\nWeaknesses\n1. The proposed method needs a batch of data for adaptation. As shown in Figure 6 (c), performance degrades significantly with smaller batch sizes, especially for single-sample adaptation. \n2. The learning rate and lambda in eq (9) vary across datasets, suggesting that manual tuning is needed for new datasets and tasks, which could be a practical limitation.\n3. The method is limited to ViT backbone and not validated on CNNs.\n4. The improvement is somehow limited compare to the best baseline. For instance, only 0.3 gain in Table 1 compared with C-MAE and only 0.3 gain in Table 4 compared with ViDA.", "was_revised": true, "final_justification": "The author addressed all my concerns and I am satisfied with the answer. The paper can bring interesting insights to CTTA domain.", "rebuttal": "Thank you for your valuable and constructive comments. We appreciate your recognition of the importance of the CTTA setting, the clarity of our writing, the thorough experimental validation across multiple benchmarks, and the novelty of applying weight decomposition in the CTTA context. Below, we address your concerns and questions individually:\n\n## **Q1: What is the intuition behind the performance drop with small batch sizes**\n\nAs shown in Figure 6 (c), most existing CTTA methods, including ours, exhibit a notable performance drop as the batch size decreases. This behavior is not specific to our approach, similar trends have been reported in prior works such as [24] (Figure 4 (d)) and [28] (Table 5), where small-batch scenarios consistently lead to degraded adaptation performance.\n\nFor our method in particular, the primary factor is **the reliability of target feature statistics**. In Equation (9), we perform alignment between source and target distributions using the mean and standard deviation computed from the current test batch. When the batch size is small, these estimates become highly variable and noisy, which results in unstable gradients and impairs the optimization process. This instability ultimately hinders adaptation effectiveness.\n\nIn contrast, larger batch sizes produce more reliable and smoother estimates of distributional statistics, effectively reducing variance in the adaptation signal. This leads to more stable parameter updates and improved learning dynamics. **Empirically, we observe that performance becomes acceptable when the batch size exceeds 4, and saturates when the batch size is greater than 32.**\n\nTo mitigate this limitation, we also explore a simple enhancement: applying an Exponential Moving Average (EMA) to the target statistics. This temporal smoothing helps stabilize the alignment objective under small-batch regimes and partially alleviates the performance drop when the batch size is 1 or 2. The corresponding results are provided below:\n\n| Batch Size | w/o EMA | w/ EMA |\n|------------|---------|--------|\n| 1          |   85.0    |  69.2    |\n| 2          |   78.7    |  65.1    |\n\n## **Q2: Clarification and Discussion on the Fairness of Baseline Comparisons and Hyperparameter Tuning**\n\nWe sincerely thank the reviewer for raising this practically relevant concern, which prompted us to further reflect on the challenges faced by the CTTA community in real-world deployment.\n\n### Clarification on the fairness of baseline comparisons\nThe baseline results reported in our experiments are all based on benchmarks widely adopted in prior CTTA studies. These results have not only been repeatedly validated in multiple papers, but also follow the hyperparameter settings **specified** in the original papers or open-source code provided by the respective authors. For example, ViDA uses a learning rate of 1e-4 on CIFAR-C, but 5e-7 on ImageNet-C; C-MAE adopts 1e-5 on CIFAR-C, while using 1e-3 on ImageNet-C. Therefore, we ensure that all baseline comparisons are conducted under reasonable and recommended settings, which guarantees the fairness and reliability of our experimental results.\n\n### Discussion on hyperparameter sensitivity and manual tuning  \nWe agree with the reviewer that manual tuning may present a limitation in real-world deployment. In fact, sensitivity to hyperparameters such as learning rate and regularization coefficients is a widely recognized challenge across CTTA community, as discussed in the prior work (1). We believe that improving the robustness of CTTA methods to hyperparameters, or exploring automated tuning strategies, is one of the most important directions for future research.\n\nThat said, we also note that manual tuning still holds practical value at the current stage and offers useful empirical guidance for deployment. Below we share our observation regarding hyperparameter selection in our method:\n\n- **Loss balancing coefficient λ**: We found that image resolution significantly affects statistical variations. For low-resolution images (e.g., CIFAR with original size 32×32), upsampling to 384×384 amplifies local noise, making variance alignment more important. For high-resolution images (e.g., ImageNet-C with native resolution 224×224), global intensity shifts dominate, so aligning feature means becomes more crucial. This observation provides an empirical basis for choosing λ under different pre-processing conditions.\n\nIn addition, we observed that when using the same learning rate settings for both CIFAR-C and ImageNet-C, our method still achieves competitive performance. This demonstrates that our method does not rely heavily on excessive tuning.\n\n| lr      | CIFAR10-C | CIFAR100-C | ImageNet-C |\n|---------|-----------|------------|------------|\n| 8e-5    |   *11.0*    |      *24.9*   |     42.5    |\n| 2e-4    |   11.5    |      24.9   |     *42.2*    |\n\n\n\n## **Q3: Extension to convolutional networks**\nThank you for your forward-looking suggestion. The key insight of PAID, preserving the pairwise angular structure between weight vectors to retain domain-invariant semantics, can be **formally** extended to convolutional networks, where each convolutional layer is also a parameterized linear transformation.\n\nIn a standard convolutional layer, each output channel corresponds to a filter of shape $(C_{\\text{in}}, k_h, k_w)$. We flatten this into a vector of length $D = C_{\\text{in}} \\cdot k_h \\cdot k_w$, treating each filter as a vector in $\\mathbb{R}^D$. Stacking these column-wise yields a weight matrix $W \\in \\mathbb{R}^{D \\times C_{\\text{out}}}$, which we decompose as:\n\n$\nW = M \\odot \\hat{W},\n$\n\nwhere $\\hat{W}$ contains unit-norm columns and $M \\in \\mathbb{R}^{1 \\times C_{\\text{out}}}$ stores the magnitudes. During adaptation, we freeze $\\hat{W}$ and update $M$ along with a learnable orthogonal matrix $O \\in \\mathbb{R}^{C_{\\text{out}} \\times C_{\\text{out}}}$, constructed via Householder reflections. The updated weights become:\n\n$\nW' = M \\odot (\\hat{W} \\cdot O).\n$\n\nHere, $O$ operates along the output channel dimension. The spatial structure within each filter remains intact because we do not rotate across spatial positions. Notably, flattening convolution filters is a standard practice in low-rank adaptation and model compression literature. Reshaping back ensures that spatial locality is fully preserved during convolution.\n\nTo **validate** this extension, we follow standard CTTA setups used in prior works: WideResNet-28 on CIFAR-10-C, ResNeXt-29 on CIFAR-100-C, and ResNet-50 on ImageNet-C. We select representative and open-sourced baselines for comparison, using benchmark results that have been consistently reported and validated across multiple CTTA studies. The results show **consistent performance gains** across all convolutional backbones, indicating that PAID generalizes beyond ViTs and is also applicable to CNNs.\n\n| Dataset     | Source | TENT[31]  | CoTTA[9] | AdaCon[36] | CRG(2) | LAW(3) | Ours |\n|-------------|--------|-------|-------|--------|--------|------|------|\n| ImageNet-C  | 82.0   | 62.6  | 62.7  | 65.5   | 59.1   | 60.1   | **58.4** |\n| CIFAR100-C  | 46.4   | 60.9  | 32.5  | 33.4   | 29.0   | 30.9   | **28.1** |\n| CIFAR10-C   | 43.5   | 20.7  | 16.1  | 18.5   | 15.9   | 15.7   | **15.0** |\n\nTo further **explain** why PAID remains valid in convolutional networks, we revisit several prior studies on the geometric structure of convolutional filters. DCNet [18] shows that the convolution operation can be reformulated as an inner product and decomposed into magnitude and angular: the magnitude better models intra-class variation, while the angular captures semantic differences. SphereConv [15] further demonstrates that learning only the angular component of convolutional filters is sufficient for semantic classification. These findings indicate that the angular structure in CNNs also carries domain-invariant semantics. Therefore, applying PAID’s strategy to convolutional networks is not only formally valid, but also **empirically supported** by prior work.\n\n\n## **Q4: Limited performance improvement in Table 1 and Table 4**\n\nWe sincerely thank the reviewer for the rigorous examination of the results. In response to the concern, we would like to clarify its significance from two perspectives: **empirical consistency** and **research value**.\n\n### First, the performance improvements are consistent across datasets and tasks, rather than accidental fluctuations.\n\nOur method achieves a +1.5% gain on the CIFAR100-C benchmark, and a +1.6% gain on CIFAR10-C. On the ImageNet-C benchmark, which is the largest and most diverse, our method maintains a 0.3% lower mean error than C-MAE. Moreover, on the ACDC segmentation benchmark, our method outperforms ViDA by +0.3% in mIoU. These **consistent improvements across different datasets and tasks** demonstrate the **robustness** and **generalizability** of our method, suggesting that its effectiveness does not rely on any data distribution from a particular benchmark.\n\n### Second, the core contribution of this paper lies in proposing a novel direction for CTTA research, which we believe carries substantial value.\n\nWhile most existing methods focus on exploiting target stream data, PAID is the **first** to explore **pre-trained weights** as more than static initializations. By preserving the pairwise angular structure of weight directions during adaptation, PAID enables a structured update that respects the semantic organization encoded during pre-training. This perspective **establishes a new design paradigm** for CTTA, with the potential to **inspire** a range of principled adaptation strategies and lay a solid foundation for future performance improvements.\n\n(1) On Pitfalls of Test-time Adaptation, ICLR 2023.\n\n(2) Reshaping the Online Data Buffering and Organizing Mechanism for Continual Test-Time Adaptation, ECCV 2024.\n\n(3) Layer-wise Auto-Weighting for Non-Stationary Test-Time Adaptation, WACV 2024."}, {"review_id": "zgRopLGe1p", "reviewer": "Reviewer_e8tG", "summary": "This paper presents a new approach for Continual Test-Time Adaptation (CTTA) by analyzing and leveraging the geometric properties of pre-trained neural network weights. The authors find that the pairwise angular structure of weights is stable across different domains and encodes domain-invariant semantic information, whereas magnitude and absolute angle are more domain-specific. Based on this insight, they propose PAID, a method that adapts only the magnitudes and applies a global orthogonal transformation to directions, strictly preserving the pairwise angular structure. The method consistently outperforms state-of-the-art baselines on several classification and segmentation benchmarks.", "questions": "1. Can the authors provide additional theoretical justification for why pairwise angular structure is robust to domain-specific shifts but sensitive to semantic changes?\n\n2. Have the authors investigated whether the pairwise angular invariance observed in ViT models also holds for convolutional layers or non-transformer architectures? If not, could some pilot experiments or discussion be provided to address the generality of the method?", "limitations": "Yes, the authors have adequately discussed the limitations of proposed method in the supplementary material.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The key empirical finding that the pairwise angular structure of pre-trained weights is a domain-invariant semantic prior provides a fresh direction for CTTA.\n\n2. The approach is well-motivated. The motivation is supported by comprehensive experiments (statistical analysis, functional ablation, visualization), leading directly to the method’s design.\n\n3. The paper is well-organized and clearly written, with intuitive explanations and detailed appendices.\n\nWeaknesses:\n\n1. Limited Theoretical Analysis. While the empirical evidence for pairwise angular structure invariance is strong, the theoretical basis for why it encodes domain-invariant semantics is largely motivated by analogy and experiment rather than formal analysis.\n\n2. The method is evaluated primarily on ViT architectures and transformer-based segmentation models. It is unclear whether the same invariance properties and adaptation mechanisms apply as strongly to convolutional or non-transformer architectures.\n\n3. Typo in Figure 5: wectors.", "was_revised": true, "final_justification": "The authors have addressed my concerns, including clarifying the scope of their approach beyond ViT-based models and providing additional theoretical foundations. Given these clarifications, and the overall strength of the paper's motivation and experimental design, I will maintain my original rating.", "rebuttal": "Thank you for your valuable and constructive comments. We appreciate your recognition of our key empirical finding, as well as your positive feedback on our well-motivated approach, the comprehensive supporting experiments, and the clarity and organization of the paper. Below, we address your concerns and questions individually:\n\n## **Q1: Theoretical analysis for why pairwise angular structure is robust to domain-specific shifts but sensitive to semantic changes**\nTo address your question, we provide a formal analysis using a simplified linear classification model. Specifically, we prove that:\n\n- **As the domain-specific shift applies the same transformation to all input samples regardless of class**, the optimal classifier in the target domain can be written as a shared linear transformation of the source classifier, under mild and mathematically defined conditions. This transformation preserves all pairwise angles between class weights.\n- **In contrast, semantic changes induce class-specific input transformations**, which break this shared structure and necessarily alter the pairwise angular geometry.\n\nThis supports the core design of PAID: using a orthogonal rotation and magnitude adjustment is sufficient for adapting to domain-specific corruptions, but insufficient for adapting to semantic shift, where the angular structure must be re-learned. We now present the mathematical proof in detail.\n\n---\n\n### Mathematical Formulation\n\n#### Linear Classification Model\n\nLet $ x \\in \\mathbb{R}^a $ be the input vector, and for each class $ c \\in \\{1, ..., C\\} $, let $ w_c \\in \\mathbb{R}^a $ be the classifier weight vector.  The predicted class is:\n\n$\n\\hat{y}(x) = \\arg\\max_{c} w_c^\\top x.\n$\n\nWe compare two settings:\n\n---\n\n### Case 1: Domain-specific Degradation (Class-agnostic input transformation)\n\nWe assume the degraded input is obtained by applying the same transformation to all inputs:\n\n$\nx' = d(x),\n$\n\nand that $ d $ is differentiable near each $ x $. We approximate $ d(x) $ by its first-order Taylor expansion around clean data:\n\n$\nd(x) \\approx Sx + q,\n$\n\nwhere $ S \\in \\mathbb{R}^{a \\times a} $ is the Jacobian matrix at $ x $, assumed to be invertible and independent of class $ y $.  We formalize this assumption as:\n\n> **Assumption 1 (Class-agnostic linear perturbation):**  \n> There exists an invertible matrix $ S \\in \\mathrm{GL}(a) $ such that for all $ x $,  $\\quad d(x) = Sx + q, \\quad \\text{with } S \\text{ independent of class } y$.\n\nWe define the optimal classifier in the target domain as the one that operates on $ x' = Sx $ but maintains the same class boundaries in the original space. This gives:\n\n$\nw'_c = S^{-\\top} w_c, \\quad \\forall c.\n$\n\nNow we examine the effect of this transformation on pairwise angles. We perform the polar decomposition:\n\n$\nS^{-\\top} = RH,\n$\n\nwhere $ R \\in \\mathbb{R}^{a \\times a} $ is orthogonal $( R^\\top R = I )$, and $ H \\in \\mathbb{R}^{a \\times a} $ is symmetric positive definite. We now examine the transformed weights:\n\n$\n\\tilde{w}_c = RH w_c.\n$\n\nWe compute the cosine similarity between any two transformed weights:\n\n$\n\\cos \\angle(\\tilde{w}_c, \\tilde{w}_d)\n= \\frac{(RH w_c)^\\top (RH w_d)}{\\|RH w_c\\| \\cdot \\|RH w_d\\|}\n= \\frac{w_c^\\top H^2 w_d}{\\|H w_c\\| \\cdot \\|H w_d\\|}.\n$\n\nTo ensure angles are preserved, we introduce:\n\n> **Assumption 2 (Near-isotropic perturbation):**  \n> The matrix $ H $ satisfies $ H^2 \\approx \\gamma^2 I $ for some scalar $ \\gamma > 0 $.  \n> This holds when the degradation has approximately equal effect in all directions (e.g., blur, light noise, JPEG), and ensures angle preservation:\n\n$\n\\cos \\angle(\\tilde{w}_c, \\tilde{w}_d) \\approx \\cos \\angle(w_c, w_d).\n$\n\nThus, under Assumptions 1 and 2, **pairwise angular structure remains invariant under domain-specific shifts**.\n\n---\n\n### Case 2: Semantic Change (Class-dependent input transformation)\n\nSuppose the semantic shift modifies each class's input differently:\n\n$\nx' = d_y(x), \\quad \\text{with } d_y(x) \\approx S_y x + q_y,\n$\n\nwhere $ S_y \\in \\mathrm{GL}(a) $ varies with class $ y $. Then the optimal classifier becomes:\n\n$\nw''_c = S _{y=c} ^{-\\top} w_c.\n$\n\nIn this case, each weight undergoes a different transformation. Unless all $ S_y $ are identical up to scaling or rotation, **there exists no single matrix** $ T \\in \\mathbb{R}^{a \\times a} $ such that $ w''_c = T w_c $ for all $ c $. Hence:\n\nThere is no shared transformation that preserves the entire angular structure. The pairwise angle matrix must differ from that in the original domain. This proves that **semantic changes necessarily break the pairwise angular structure**.\n\n---\n\n### Final Conclusion\n\n- When domain shifts are class-agnostic and locally differentiable, they can be represented by a shared invertible matrix $ S $, and the optimal classifier corresponds to applying the inverse transpose $ S^{-\\top} $ to all weights.\n- This transformation preserves angles up to a near-isotropic assumption, justifying why **PAID only needs to learn a shared orthogonal rotation and magnitude adjustment**.\n- When the shift is class-dependent, this shared structure no longer exists, and angular geometry must change.\n\nThis provides a direct theoretical justification for your question and explains the observations of our paper.\n\n## **Q2: Extension to convolutional or non-transformer architectures**\nThank you for your forward-looking suggestion. The key insight of PAID, preserving the pairwise angular structure between weight vectors to retain domain-invariant semantics, can be **formally** extended to convolutional networks, where each convolutional layer is also a parameterized linear transformation.\n\nIn a standard convolutional layer, each output channel corresponds to a filter of shape $(C_{\\text{in}}, k_h, k_w)$. We flatten this into a vector of length $D = C_{\\text{in}} \\cdot k_h \\cdot k_w$, treating each filter as a vector in $\\mathbb{R}^D$. Stacking these column-wise yields a weight matrix $W \\in \\mathbb{R}^{D \\times C_{\\text{out}}}$, which we decompose as:\n\n$\nW = M \\odot \\hat{W},\n$\n\nwhere $\\hat{W}$ contains unit-norm columns and $M \\in \\mathbb{R}^{1 \\times C_{\\text{out}}}$ stores the magnitudes. During adaptation, we freeze $\\hat{W}$ and update $M$ along with a learnable orthogonal matrix $O \\in \\mathbb{R}^{C_{\\text{out}} \\times C_{\\text{out}}}$, constructed via Householder reflections. The updated weights become:\n\n$\nW' = M \\odot (\\hat{W} \\cdot O).\n$\n\nHere, $O$ operates along the output channel dimension. The spatial structure within each filter remains intact because we do not rotate across spatial positions.\nNotably, flattening convolution filters and treating them as vectors is a standard practice in low-rank adaptation and model compression literature. Reshaping back after rotation ensures that spatial locality is fully preserved during convolution.\n\nTo **validate** this extension, we follow standard CTTA setups used in prior works: WideResNet-28 on CIFAR-10-C, ResNeXt-29 on CIFAR-100-C, and ResNet-50 on ImageNet-C. We select representative and open-sourced baselines for comparison, using benchmark results that have been consistently reported and validated across multiple CTTA studies. The results show **consistent performance gains** across all convolutional backbones, indicating that PAID generalizes beyond ViTs and is also applicable to CNNs.\n\n| Dataset     | Model          | Source | TENT[31]  | CoTTA[9] | AdaCon[36] | CRG(1) | LAW(2) | Ours |\n|-------------|----------------|--------|-------|-------|--------|--------|------|------|\n| ImageNet-C  | ResNet-50       | 82.0   | 62.6  | 62.7  | 65.5   | 59.1   | 60.1   | **58.4** |\n| CIFAR100-C   | ResNeXt-29    | 46.4   | 60.9  | 32.5  | 33.4   | 29.0   | 30.9   | **28.1** |\n| CIFAR10-C  | WideResNet-28   | 43.5   | 20.7  | 16.1  | 18.5   | 15.9   | 15.7   | **15.0** |\n\nTo further **explain** why PAID remains valid in convolutional networks, we revisit several prior studies on the geometric structure of convolutional filters. DCNet [18] shows that the convolution operation can be reformulated as an inner product and decomposed into magnitude and angular: the magnitude better models intra-class variation, while the angular captures semantic differences. SphereConv [15] further demonstrates that learning only the angular component of convolutional filters is sufficient for semantic classification. These findings indicate that the angular structure in CNNs also carries domain-invariant semantics. Therefore, applying PAID’s strategy to convolutional networks is not only formally valid, but also **empirically supported** by prior work.\n\n## **Q3: Typo in Figure 5**\nThank you very much for pointing out this typo. We will correct this error in the revision.\n\n(1) Reshaping the Online Data Buffering and Organizing Mechanism for Continual Test-Time\nAdaptation, ECCV 2024.\n\n(2) Layer-wise Auto-Weighting for Non-Stationary Test-Time Adaptation, WACV 2024."}, {"review_id": "hW8QXYmyYv", "reviewer": "Reviewer_tGtE", "summary": "The authors focus on the often-overlooked pre-trained weights and empirically discover that the pairwise angular structure among weight vectors remains stable across a wide range of corrupted domains, capturing domain-invariant semantic information. Motivated by this finding, they propose PAID (Pairwise Angular-Invariant Decomposition), a method that adapts the model during test time by updating only the weight magnitudes and applying a learnable orthogonal matrix—constructed via Householder transformations—to preserve the relative angular structure. Extensive experiments on four widely-used CTTA benchmarks validate the effectiveness and generalizability of the proposed approach.", "questions": "1. Can the proposed method be applied to other network architectures beyond Transformers?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\n1. The paper tackles the TTA problem from a novel and insightful perspective, focusing on the geometric structure of pre-trained weights.\n2. The writing is generally clear, and the authors effectively convey their motivation through well-designed experiments and illustrative figures.\n3. The experimental evaluation is relatively comprehensive, covering multiple classification and segmentation benchmarks with thorough ablation studies.\n\n**Weaknesses:**\n\n1. The method is only evaluated on Transformer-based architectures. Its applicability to other network types, such as CNNs, remains unexplored and should be discussed or validated.\n2. Some of the text in Figure 5 is blurry.", "was_revised": true, "final_justification": "The authors’ additional experiments adequately resolve my concerns and I have no other questions. So I retain my original score.", "rebuttal": "Thank you for your valuable and constructive comments. We appreciate your recognition of our novel and insightful perspective on CTTA based on the geometric structure of pre-trained weights, as well as your positive feedback on the clarity of our writing, the motivation conveyed through well-designed experiments and figures, and the comprehensive evaluation across multiple benchmarks with detailed ablation studies. Below, we address your concerns and questions individually:\n\n## **Q1: Extension to other network architectures beyond Transformer**\nThank you for your forward-looking suggestion. The key insight of PAID, preserving the pairwise angular structure between weight vectors to retain domain-invariant semantics, can be **formally** extended to convolutional networks, where each convolutional layer is also a parameterized linear transformation.\n\nIn a standard convolutional layer, each output channel corresponds to a filter of shape $(C_{\\text{in}}, k_h, k_w)$. We flatten this into a vector of length $D = C_{\\text{in}} \\cdot k_h \\cdot k_w$, treating each filter as a vector in $\\mathbb{R}^D$. Stacking these column-wise yields a weight matrix $W \\in \\mathbb{R}^{D \\times C_{\\text{out}}}$, which we decompose as:\n\n$\nW = M \\odot \\hat{W},\n$\n\nwhere $\\hat{W}$ contains unit-norm columns and $M \\in \\mathbb{R}^{1 \\times C_{\\text{out}}}$ stores the magnitudes. During adaptation, we freeze $\\hat{W}$ and update $M$ along with a learnable orthogonal matrix $O \\in \\mathbb{R}^{C_{\\text{out}} \\times C_{\\text{out}}}$, constructed via Householder reflections. The updated weights become:\n\n$\nW' = M \\odot (\\hat{W} \\cdot O).\n$\n\nHere, $O$ operates along the output channel dimension. The spatial structure within each filter remains intact because we do not rotate across spatial positions.\nNotably, flattening convolution filters and treating them as vectors is a standard practice in low-rank adaptation and model compression literature. Reshaping back after rotation ensures that spatial locality is fully preserved during convolution.\n\nTo **validate** this extension, we follow standard CTTA setups used in prior works: WideResNet-28 on CIFAR-10-C, ResNeXt-29 on CIFAR-100-C, and ResNet-50 on ImageNet-C. We select representative and open-sourced baselines for comparison, using benchmark results that have been consistently reported and validated across multiple CTTA studies. The results show **consistent performance gains** across all convolutional backbones, indicating that PAID generalizes beyond ViTs and is also applicable to CNNs.\n\n| Dataset     | Model          | Source | TENT[31]  | CoTTA[9] | AdaCon[36] | CRG(1) | LAW(2) | Ours |\n|-------------|----------------|--------|-------|-------|--------|--------|------|------|\n| ImageNet-C  | ResNet-50       | 82.0   | 62.6  | 62.7  | 65.5   | 59.1   | 60.1   | **58.4** |\n| CIFAR100-C   | ResNeXt-29    | 46.4   | 60.9  | 32.5  | 33.4   | 29.0   | 30.9   | **28.1** |\n| CIFAR10-C  | WideResNet-28   | 43.5   | 20.7  | 16.1  | 18.5   | 15.9   | 15.7   | **15.0** |\n\nTo further **explain** why PAID remains valid in convolutional networks, we revisit several prior studies on the geometric structure of convolutional filters. DCNet [18] shows that the convolution operation can be reformulated as an inner product and decomposed into magnitude and angular: the magnitude better models intra-class variation, while the angular captures semantic differences. SphereConv [15] further demonstrates that learning only the angular component of convolutional filters is sufficient for semantic classification. These findings indicate that the angular structure in CNNs also carries domain-invariant semantics. Therefore, applying PAID’s strategy to convolutional networks is not only formally valid, but also **empirically supported** by prior work.\n\n## **Q2: Blurry text in Figure 5**\nThank you very much for pointing out this typo. We will replace the figure with a clearer one and use a larger font in the revision.\n\n(1) Reshaping the Online Data Buffering and Organizing Mechanism for Continual Test-Time\nAdaptation, ECCV 2024.\n\n(2) Layer-wise Auto-Weighting for Non-Stationary Test-Time Adaptation, WACV 2024."}]

Author rebuttal:

Meta-review: Pairwise Angular-Invariant Decomposition (PAID) examines the source model parameters when applied to continual adaptation problems for target data and contributes a new prior / regularizer for test-time adaptation methods. The choice of prior—pairwise angular structure—is empirically explored and well motivated across shifts with clear notations and visualizations to support this choice and check alternatives (the magnitude or absolute angle). The experiments show improvement on standard benchmarks for continual test-time adaptation (ImageNet-C, CIFAR-100-C for classification and Cityscapes/ACDC for segmentation) with current models (ViTs, Segformer).

- Positives: the parameterization focus in concept/experiment/theory is complementary to existing work on test-time adaptation (KZqM, jqAQ, e8tG, tGtE), the scenario is practical and the experiments extensive (KZqM, jqAQ), the exposition is clear (jqAQ, e8tG, tGtE).
- Negatives: the work focuses on transformers like ViTs which are current but is not as comprehensive as it could be given the number of existing results for ResNets which are relevant for their efficiency (KZqM, jqAQ, e8tG, tGtE), the results suffer as batch size decreases and the method only addresses batch-wise and not input-wise adaptation (jqAQ), insufficient clarity in the presentation of the method (KZqM), sensitivity to hyperparameters and the need for tuning (jqAQ), and limited theoretical support (e8tG) although this is common for the topic.

The reviewers all converge to ratings of 4: borderline accept and their are broad points of agreement on the positives and the negatives. In particular all reviewers agree on the positive that the parameter geometry perspective is informative and complementary to existing work on continual test-time adaptation and all reviewers agree on the negative that the paper narrows its evaluation too much w.r.t. model architecture. The rebuttal and discussion phases confirmed and clarified the core contribution of PAID on parameter geometry analysis and regularization and resolved the negative about ViT exclusivity with multiple results on convolutional architectures. Reviewers KZqM, jqAQ, e8tG, and tGtE all confirm that the rebuttal has addressed their points and either raise or maintain their score. The area chair sides with acceptance due to the universal positive ratings and agreement on the complementary and informativeness of the main contribution about parameter geometry and variation vs. invariance across shifts. The potential arguments for rejection, like the narrow selection of models for experiment or points of clarity, have all been resolved by the rebuttal as confirmed by the reviewers. The additional results and details should be incorporated into the revision and related work like DoRA and OFT (jqAQ) should be cited as similar in technique if not in purpose and experimental scope.
Decision: Accept (poster)