Avoiding Structural Pitfalls: Self-Supervised Low-Rank Feature Tuning for Graph Test-Time Adaptation
Authors: Haoxiang Zhang
Zhuofeng Li
Qiannan Zhang
Ziyi Kou
Juncheng Li
Shichao Pei
Venue: Accepted by TMLR ()

Abstract: Pre-trained graph neural networks (GNNs) have demonstrated significant success in leveraging large-scale graph data to learn transferable representations. However, their performance often degrades under distribution shifts, particularly in real-world scenarios where test labels are unavailable. To address this challenge, we propose Graph Optimization via Augmented Transformations (GOAT), a novel self-supervised test-time tuning paradigm that adapts pre-trained GNNs to distribution-shifted test data by focusing exclusively on node feature transformations. By avoiding complex and often suboptimal graph structure transformations, GOAT overcomes the limitations of existing data-centric methods.
To further address the issue of transformation collapse, where feature transformations converge to trivial solutions—such as when the test-time learned data-centric transformation degenerates into a constant or identity mapping across different inputs, we introduce a parameter-efficient low-rank adapter that generates diverse transformations tailored to individual input graphs. This design not only enhances adaptation performance but also improves interpretability by avoiding modifications to the graph structure. Through extensive experiments on six real-world datasets with diverse distribution shifts, we demonstrate that GOAT achieves consistent performance improvements across different pre-trained GNN backbones, outperforming state-of-the-art test-time adaptation methods.

Published in Transactions on Machine Learning Research (11/2025)
Avoiding Structural Pitfalls: Self-Supervised Low-Rank Feature Tuning for Graph Test-Time Adaptation
Haoxiang Zhang'
haz1400ucsd edu
Hakcroglu Data Science Institute University of California, San Diego
Zhuofeng Li
2huofengl12345 (tt.24444444mm.m. com
Independent
Qiannan Zhang
pizz0005@mee..crmnell. www.://ww..oomo..oommmmmmmmoml.m edu
Weill Cornell Medicine College Cornell University
Ziyi Kou
zkou@nd. edu
Department of Computer Science and Engineering University of Notre Dame
Juncheng Li
jeliBcs.ccmu.o edu.
School of Computer Science and Technology East China Normal University
Shichao Peil
shichao.peiee www.wpmbi.mm.m edu
Department of Compuler Science University of Mossachusells Boston
Reviewed on OpenReview: https: openreview. net/ forum? id= yd=yoS6442LLt
Abstract
Pre-trained graph neural networks (GNNs) have demonstrated significant success in leverag- ing large-scale graph data to learn transferable representations. However, their performance often degrades under distribution shifts, particularly in real-world scenarios where labels of test data are unavailable. To address this challonge, we propose Graph Optimization via Augmented Transformations (GOAT), a novel self-supervised test-time tuning paradigı that adapts pre-trained GNNs distribution-shin node feature transformations. By avoiding complex and often suboptimal graph structure transformations, GOAT overcomes the limitations of existing data-centric methods. To further address the issue of transformation collapse, where feature transformations con- verge to trivial solutions - such as when the test-time learned data-centric transformation degenorates into a constant or identity mapping across different inputs, we introduce a parameter--fficient low-rank adapter that generates diverse transformations tailored to dividual input graphs. This design not only enhances adaptation performance but also improves interpretability by avoiding modifications the graph structure. Through exten- sive experiments six real-world datasets with diverse distribution shifts, demonstrate that GOAT achieves consistent performance improvements across different pre-trained GNN backbones, outperforming state-of-the-ar test-time adaptation methods.
test data by focusing exclusively on
"Equal contribution.
i "oorrsppondigg author.

---

Published in Transactions on Machine Learning Research (11/2025)
Dataset
OGB-ArXiv
Elliptic
Description
wornd datasct of academic papers, the graph evolves new papers are cited.
A datasct of transactions labeled as licit or illicit,
influenced by market conditions.
Split
Year Slice
Accuracy
Degrade
Time Slice
F1 score
Degrade
Train
before 2011 2011
47.88%
7th 11
90.12%
Val
2011 2014
44.46%
-9.66%
12th 17th
78.75%
-39.17%
Test
2014 2020
38.92%
17th 49th
50.95%
Table 1: The showcase indicates significant decrease in the node classification performance of the pre-trained GCN on the OGB-ArXiv (Hu et al. 2020) and Elliptic (Pareja et al. (020) datasets in an OOD setting where graph data is generated from different time intervals. For OGB-ArXiv, the year ranges from before 2011 to 2020; for Elliptic, from the 7th snapshot (when the dark market crackdown occurred) to the 49th snapshot. Performance degrades noticeably during validation and testing as time progresses.
1 Introduction
Graph pre-training has become a powerful technique for capturing and preserving information from large- scale upstream graph data (Kipf & Welling 2022
Hamilton et al. 2017 Veličković et al.) 2018), enabling graph neural networks (GNNs) to learn rich representations that can be effectively transferred to variety of
downstream graph tasks. However, the performance of pre-trained GNNs is often hindered by distribution shifts (Yehudai et al. 2021 Li et al.] 2022a; Song & Wang 2022 Zhu et. al., 2021), particularly in real-world scenarios where the labels of the test data are unavailable. This presents a significant obstacle to the practical
deployment of GNNs, as their performance can degrade substantially under such conditions. For example, Table | ralustrates that the performance of pre-trained GNN deteriorates as the distribution of the test data changes over time.
Adapting pre-trained GNNs to such unlabeled and distribution-shifte test seenarios is therefore a crucial yet. under-addressed problem. Several methods have been proposed to deal with dist shift at training time, including invariant risk minimization (Arjovsky et al. 2019; Wu et al. 2023). (Muandet et al.] 2013 Li et al. 2022b), and invariant representation learning (Wu et al. 2022 Chen et al. 2022b). These approaches attempt learn features that remain stable across environments, assuming access to labeled data in the target domain during training. Unfortunately, in many practical settings, test-time data is both unlabeled and unavailable at training time, rendering these methods inapplicable for adapting pre-trained models al Lest lime.
learning
Test-time adaptation (TTA) offers promising alternative. Model-centric TTA methods, such as updating classifier heads (Wang et al. 2022) or finc-tuning the full model (Zhang et al., 2024 Wang et al. 2021), aim to leverage the generalization capabilities of pre-trained models. However, these approaches often struggle to adapt effectively to unseen or shifted distributions, as they heavily depend on parameters and statistics learned from the training data (Hendrycks & Dictterich 2019 Arjovsky et al. 2019). Morcover, finc-tuning the entire model can be computationally expensive, making it impractical in many real-world resource- constrained scenarios. In contrast to model-centric approaches, data-centric TTA methods (Jin et al.) 2023 Chen et al.) 2022a Zhang et al. 2024) shift the focus from updating model parameters to refining the test graph itself. By transforming node features and graph structure, these methods aim to better align the test data with the representation space of the pre-trained model. This strategy has shown considerable promise in improving model performance under distribution shifts.
Despite their effectiveness, data-centric methods typically rely simultaneous transformations of both node features and graph structures, as adapting node features alone often leads to suboptimal performance. However, this requirement introduces two key challenges. First, identifying and optimizing appropriate graph structure transformations is inherently difficult due to the vast and continuous search space. Inaccurate structure modifications can result in poorly refined test graphs, ultimately degrading adaptation performance. Sccond, altering the graph structure reduces the interpretability of the adaptation process by obscuring the relationship between the original and transformed graphs, making it difficult to trace how specific structural changes influence model predictions.
2

---

Published in Transactions on Machine Learning Research (11/2025)
While one might consider avoiding structure transformations by focusing exclusively on node features, this approach introduces its own difficulties. In particular, it risks both reduced performance and transformation collapse, where the transformation function becomes ineffective, often converging to a constant transforma- tion or identity mapping that fails to facilitate meaningful adaptation.
Present work. To address these limitations and overcome the associated challenges, we propose a novel self-supervised test-time tuning paradigm Graph Optimization via Augmented Transformations (GOAT) that enables the pre-trained GNN to dynamically adapt to unseen test distributions without requiring access to test labels, source training data, or training details. 1) To avoid modifying the test graph structure, our node feature transformations. However, naively transforming node
method GOAT focuses exclusively features is insufficient for effective adaptation. To overcome this, we introduce a self-supervised strategy that estimates optimal node feature transformations by leveraging set of augmented test graphs. By measuring the embedding discrepancies between augmented graphs before and after transformation, our method learns a generalizable transformation function tailored to the test distribution. 2) To address the transformation collapse issue, we propose a low-rank adapter that generates unique transformations for different input graphs while effectively utilizing the knowledge encoded in the pre-trained GNN. This approach ensures that, during each learning iteration, the augmented graphs undergo diverse transformations by the dynamic adapter, preventing uniform transformations across all graphs and preserving the diversity of the augmented data. With the learned low-rank adapter, the model's interpretability becomes more intuitive compared to interpreting transformations of the graph structure. To summarize, our main contributions are as follows:
Our work centers on data-centric test-time adaptation and introduces novel self-supervised test-time tuning paradigm, GOAT, that leverages node features exclusively, bypassing the complexity of optimizing for structure transformations.
We propose a parameter-efficient low-rank adapter to address the transformation collapse issue while enhancing model interpretability.
Extensive experiments on real-world datasets with diverse distribution shifts demonstrate consistent performance improvements across various backbones.
2 Related Work
Distribution Shift on Graphs. Graph-structured data often exhibits distribution-shiff phenomena (Song
& Wang 2022 Li et al., 2022a). To tackle this challenge, researchers have proposed methods for learning
invariant representations (Wu et al.
2023 Arjovsky et al.) 2019 Wu et al. 2022 Chen et al. 2022b Li
et al. b022b Muandet et al. 2013). generalizing pre-trained GNNs (Zhu et al., 2021 Li al. 2022a Song
& Wang 2022 Hu et al.) Zhao et al.
2021), detecting OOD instances (Zellinger et al. 2022 Guo et al.
2023 Huang et al. 2024h. Most of these approaches often require access to multiple source domains, rely
on specific model architectures and train-time paradigıns, or may lead to performance degradation. For a
thorough review, we refer the readers to two recent surveys [Wu et al. 2024 Liu & Ding 2024).
Graph Test-time Adaptation. Graph test-time adaptation (GTTA) aims to adapt pre-trained models to the test distribution without requiring labeled data or modifying the model's parameters (Chen et al. 2022a Existing proposed methods can be broadly categorized into the following two classes: data-centric and model- centric. (1) Model-centric. These approaches center on the learning process or the design of the graph model. SLAPS Fatemii et al.) 2021), TTT (Sun et al.) 2020), TTT++ (Luu et al. 2021), and Tent (Wang et all 2021) update the whole pre-train model or some specific layers. In addition, GraphTTA (Chnn et al, 2022al and GT3 (Wang et al.) 2022) tunc new classifier head expecting better prediction. However, these approaches have linitations such as reliance on specific architectures, over-smoothing, or being preoccupied with how to select negative samples for contrastive learning. These issues direct our attention to data-centric approaches. (2) Data-centric. These recently emerging approaches emphasize the manipulation of input graphs. GTHANS (Jin et al., (2023) both modify the adjacency matrix and node feature to empower graph representation learning. GraphCTA (Zhang et al.) 2024) uses memory bank to keep the best neighbor
3

---

Published in Transactions on Machine Learning Research (11/2025)
Traisable Fixed
Node Embedding
Node Embedding
Adapter
Adapter
Adaptation Nodes
Node Feature X E R
X E R
AX & R-
AX R Mid
Sampled G
Pred
CAzA
Sampled G2
Go X)
Pre-truined GNN
On (A.
Gy (4.
Pre-trained GNN
Test-time Tuning
Test-time Inference
Figure Overview of the proposed method GOAT under two augmented views. In the test-time tuning, multiple augmented views of single test graph are generated by extracting induced subgraphs. These views passed through the fixed pre-trained GNN, with their imput node features modified by an adaptation term AX. self- supervised loss Laza is used to train the adapter, encouraging alignment between the embeddings of one view with node feature transformation and another without. During the test-time inference phase, all parameters are frozen. The node feature and its embedding after certain layers of the pre-trained GNN will be fed to the adapter. The best adaptation AX generated from the adapter is added to the imput features of the test graph and passed through the pre-trained GNN, resulting improved prediction performance.
structure. Whereas, GTTRANS requires the transformation of graph structure, leading to the suboptimal structure refinement, while GraphCTA still needs to update the whole pre-trained model's parameters. Graph prompt tuning (Fang et al., 2024) inspires succinct dircction; however, emphasizes the task discrepancy rather than the distribution shift and relies on the availability of label sets for downsst tasks. These challenges guide us toward a more universal test-time tuning paradigm.
3 Methodology
In this section, we delve into our proposed paradigm GOAT and Figure provides the framework overview,
3.1 Self-supervised Graph Test-time Adaptation from Augmentations
A graph is represented by Ç with a set of nodes V and a set of edges E. For any input graph data G = (A, X), the adjacency matrix derived from a graph is denoted by A E N If oe Uyy and (v, Uy) E E, then Ai is one, otherwise it is zero and X (a E V) is the node features. Apart from these, each node in the graph has label yo, and the label of the whole graph can be represented define an induced graph Gs Xs), where S C V, As [Amu. c, u E S), and Xs (a, v E S}. Its corresponding label is V/s.
vector V. In local view,
Graph Pre-train. During train time, a GNN fe(g) parameterized by 0 is optimized by the input-target pairs (9s, sampled from training data Der (Gir, Verk which minimizes the empirical risk:
0* arg min E 8 (Go.J's)
(fo(s))))
(1)
Distribution Shift. It's key challenge in real-world applications that the training Dir and test graph Gie together with its label Die are drawn from different probability distributions, Formally, if denote the source distribution Per (G, and the test distribution as Pte (Dr (re), distribution shift is defined via divergence measure d(.,,) as:
A dP. Pte (Ie(le)))))) >
Here, assumed to be relatively large threshold, meaning that the distributional gap is substantial enough to hinder direct generalization of the source-trained model. This mismatch means model optimized

---

Published in Transactions on Machine Learning Research (11/2025)
the source distribution may not generalize effectively. The problem is particularly acute when test labels are unavailable, which makes determining the true test-time distribution more difficult and is a core challenge we aim to address.
Graph Test-time Adaptation. At test time, for arbitrary graph Gie, when the label Vie available, following the data-centric paradigm (Jin et al. 2023 Zhang et al. 2024), an extra transformation of the input graph data should satisfy the following objective:
2 arg min Ey Lie (oo V (Gr..hee)
(2)
denoted that for parameter efficiency, the parameters 6* of the pre-trained GNN f(g) is fixed during tuning the graph transformation ((g) C + G parameterized by v and the number of parameters |+| should be significantly smaller than 10*). During the prediction stage, Lie represents the likelihood loss, while in the representation learning stage, Lie can be defined the L2 norm. Since the pre-trained 0 are fixed at test-time and the test graph is the focus of this paper, we drop the subscript in Ste and for to simplify nolations in the rest of the paper.
When test labels Vie are unavailable, we reframe the original task of predicting 3he as estimating the optimal reprosentation 2" (9re), where f is the fixed pro-trained GNN encoder, and gv is a transformation function applied test time. To estimate the optimal representation z", we use prozies, strategy akin to ostimating an unknown quantity (e.g., room temperature) by averaging multiple noisy measurements. According to the law of large numbers, this can yield reasonably accurate approximation of the ground truth. Since we do not modify the pre-trained model or access ground truth labels, we optimize Jø using prozy representations å obtained from augmented versions of the test graph, denoted Sie:
2 = f(ic).
(3)
Using these proxies, we formulate the objective to learn Iv as:
v arg min B(g....) (1(( =11?;
(4)
where the Le norm is used the loss Lie in the representation learning stage. Since we use L2 loss, it suffices that the proxy & satisfics Eg[²] = E.[=1]
To ensure generalization, Ie must be capable of transforming both the original test graph Gte and its various augmented versions G.e. To this end, we train By Sie and generate proxies from gie- Importantly, to prevent Iv from collapsing to the identity function, the input graph passed to Je must be different from the one that generates Specifically, lot Daug = 92,,,,......... 961 be a set of |r] augmented graphs derived from Gie. We train Io to transform each 9p Dougs while selecting different graphs Gy Daug Gp as the proxies. The optimization objective becomes (proofs can be found in Appendix ??):
lrl
C.(1) =E B.
(((0((99) ((9))
(5)
(
p=1 p=1g=1 1=1
47p
subject to the constraint:
E, [(g) 0, VG, E D Daug.
(6)
To efficiently construct D Daug. We apply subgraph sampling strategy, similar in principle to methods RRog ee al.] 2020 Hamilton et al. 2017], to generate sufficient sub-views of augmentation of the test graph. Note that the graph transformation gy here is applied exclusively to the node features of these subgraphs. The loss L9 G2) is symmetric with respect to its inputs, swapping 9, and 9, leaves the loss unchanged. This symmetry ensures that the adaptation process invariant to the input order, thereby enhancing the robustness of the data-centric optimization approach.
5

---

Published in Transactions on Machine Learning Research (11/2025)
3.2 Node Feature Transformation with Low-rank tross-attention Adapter
During optimizing the transformation 9e with L. in Eq. 5), collapse mapping occurs when each augmented graph receives the same transformation solution, leading to a failure in producing meaningful input-specific adaptations. We define this phenomenon as transformation collapse, failure mode where the adapter converges to a trivial function that fails to generate distinct outputs for different inputs. This includes two primary forms: a) Constant Collapse: Where the data-centric transformation I6 maps all different imputs to the same output (gp( = G", vgp). b) Identity Collapse: Where By simply returns the input (gw = 9p). To address this issue, design a low-rank cross-attention adapter to ensure that distinct inputs & produce unique outputs (g),, thereby preventing all augmented graphs from being mapped to the same point in the pre-trained GNN's representation space from data-centric perspective.
For an arbitrary graph G = (A, where A E RNN represents the adjacency matrix encoding the edge relationships among nodes, and X' E Rxx denotes the node feature matrix with d-dimensional features for each node, data-centric graph adaptation assumes the existence of a transformed graph G = Gø- (9), which ensures that the graph achieves the best possible performance when applied to the pre-trained GNN f(), by bridging the gap between the test graph and the knowledge learned by f(-)- Specifically, graph transformation that only transforms node features is defined as:
( = (A, X e AX),
where AX Rg.xd denotes continuous learnable parameters with the same shape as the node features. Here, @ denotes element-wise addition.
However, simply adding AX cannot solve the collapse mapping issue. To better solve this issuc and integrate global and local information of test graphs into AX while fully leveraging the knowledge learned by the pre-trained GNN, we design the following adapter incorporating the pre-trained GNN to generale AX dynamically.
In GNNs, learning of node representations typically relies on the aggregation of neighboring nodes. Given a graph 9 (A, an I-layer GNN can be represented
H = (A, 1=
(8)
where HT)) E Nxd" represents the node representation at the 1-th layer in pre-trained GNN with H = X. (GG is the aggregation function at the I-th layer, and a the activation function. In our experiments, the ReLU activation function is employed: (z)) = +
Although GNNs effectively capture local structural information through neighborhood aggregation, they face two key limitations. First, they primarily focus on local interactions and lack mech- anisms to model global structural context. Second, modeling long-range dependencies requires stacking multiple layers, which significantly increases computational cost and often leads to over- smoothing. To address these challenges, we propose the Low- Rank Adapter (LRA), lightweight module designed incorpo- rate global context efficiently via attention over compact set of virtual nodes.
X
(1)
H(Q
Lincar
Linear
Lew-rank
Low-rank
Rdind
Risia
B ghia
EK
Softmax
Specifically, the LRA operates on the k-th output of the first layers of the GNN with the original node features to obtain an k- hop neighbor-aware node representation (A) E R (d Then we intrueeee Low-Rank projection : Nxx'' + Rinixd (usually In| < N), which a low-rank projection function that projects nodes into a latent space represented by few virtual nodes, striv- ing for maximal rank density and minimal computational overhead, where d' is the attention dimension LRA. Let F, F E R denote the key and value's low-rank transition matrices of the virtual nodes. The
goa
AX
Figure Structure overview of Low-Rank Adapter.

---

Published in Transactions on Machine Learning Research (11/2025)
core of LRA is self-attention mechanism, which enables each node to attend to globally-aware summary of the graph via virtual nodes. The computation process of LRA is:
QYEKY (FV))Wx.
AX =
(Softmax
(9)
Vd
where Wx Rd is learnable output projection matrix, and Q,K, V are obtained via:
Q Q= K= H V Wv.
(10)
xa' and Wy R xd' are learnable weight matrices. By introducing the low-rank transition matrices E and F, LRA can compute long-range dependencies wet nodes with linear time complexity and generate AX E IR ngxx that contain local-global contextual information. The time complexity the LRA, assuming & = 1, is O(Nd'.
Here, We Rdxd WK Rd xd'
With LRA, both interpretability and robustness are improved. The output AX can be directly analyzed, making it more interpretable than structural graph transformations. Additionally, in the test-time adaptation setting, LRA helps prevent transformation collapsc: when applied to different graphs g(G,) and g(G,), their outputs are less likely to degenerate to the same representation, thus maintaining representational diversity.
3.3 Optimization
The constraint in Eq. requires the same input graph for both f(g (-)) and f(-), which easily result in the mapping collapse issue. As transformation g(g) can be formulated as (A, & + AX), the constraint is equivalent to:
By + - f(A, - f(A, AX) s.t. Ey( =0,
AX
(11)
the transformation node embeddings. In this form, it can be seen encouraging Is to learn an isomorphic mapping with ensuring that the adapter's transformations are consistent with those of the GNN, i.c., Eg o ~ fogs- This consistency facilitates the rapid optimization and performance consistency of the transformation Iø across different designs of f. With further relaxing the constraint in Eq.[11 [1], we derive the following consistency loss and a regularization loss. Optimizing the two loss functions jointly can achieve the same effect as optimizing the constraints in Eq. with the same |r| augmented graphs in Eq. (5:
where ((A,X) serves as regularization. We define go ((A,))
Co(b)
Ey IsHsir|
Σ Ilf(ow 9 (f(G))))
CR(0)= = Ey ISPSIR
Σ II((Ap, |
(12)
where Ap is the adjacent matrix of 9p E Dangr AX, is generated from LRA with the input of 9p-
The overall loss function used for training is defined as follows:
LazA = a + L.) + a)-a)
(13)
where a and X are hyperparameters that control the importance of each objective. a is hyperparameter in the range (0, 1); X is a positive hyperparameter.
4 Experiments
4.1 Adaptation on Data with Distribution Shift
Datasets. We evaluate GOAT 's performance on three types of distribution shifts across six benchmark datasets, following the experimental settings of FERM (Wu et al. 2023 The dataset. statistics, along with

---

Published in Transactions on Machine Learning Research (11/2025)
Table 2: Average classification performance (%) on the test graphs. 'he best performance on each dataset with a specific backbone is indicated in bold, the second-best method is underlined, and " "Avg." indicates the average ranking of same method across six datasets under the same backbone. OOM indicates an out-of-memory error on a 24 GB GPU. f denotes that GOAT significantly outperforms GTTran and ERM using the Wilcoxon signed-rank test at the significance level of p < 0.05, respectively, across all datasets.
Datasots
Artificial Transformation
Temporal Evolution
Cross- Donss-DDmaan
Backbone
Method
Amz-Photo
Cora
Elliptic
OGB-Arxiv
Twitch-E
PB-100
Avg.
GCN
ERM
92.78+1.31
03.92200.66
$4.1321.18
36.8920.67
56.81=1.13
53.9520.77
4.7
EERM
94.2410.40
87.36.1.0.86
53.15:10.01
OOM
37.25=0.42
54.0310.80
5.2
Tent
03.84+
91.64422.77
39.34442.76
54.11+1.50
4.2
GCTA
91-43+ 1.74
93. 08.13322.000 02
55.82+33.00 50
37.27
60.10=0.95
14.11+1.49
3.7
GTRANS
94.32+1.11
94.764:1 94
35.07=3.66 61
10.15=1..76
60.37=1-44
51.17#1.23
1.8
GOAT
1..3
94.7911.30
55.83=13.81
*39.44:2202
*60.13=1.30
54.19:2..04
1.3
SAGE
ERM
87.7921.71
62:60.20000
50.11=0.30 39
37.5210.66
59.20=0.11
51.09=0..10
5.2
FERM
96.76+0.11 11
99.70--0.21
60.43++0.28
OOM
60.09=0.25
0OM
5.2
Tent
95,23+1.52
99. 19.71:10.17
20.2533.22
39.56#1 .19
62.05=0..2
55.11
3.0
GCTA
16.8611.11
99.85200.06
66.92:42.35
33.67:3,25
62.05=0.24
50.11
3.2
GTRANS
97.09:11.13
99 89.
6..04:16.39
39. 49.74411.14
61.97=0.34
55.07:10.59
2.5
GOAT
*92.54+2.51
#99.89::0.10
56.
$39.52+1.00
*61.91-0.28
*55.61 =0.30
2.5
GAT
ERM
94.3312.33
95.99:10.888
49.49:11.51
37.92:10.68
07.36=0.30
18.25
3.8
FRRM
94.07+1.32
70.35-+88.00
34.2742.42
OOM
56.27-0.37
62.46422.02
3.7
Tent
94.96+0.88
93 03.54433.00
55.2945.22
37.1+5.200
08.93-1.50
01.2221.99
5.3
GCTA
04.72+11.75
06.03:11.770
56.00+10011
37.8 64-2.117
58.83-1.59
51.22++1.98
3.2
GTRANS
95.14=00.70
95 16-446:1.90
42.56
37.6242268
18.84=1499
61.27#1.91
2.5
GOAT
91.69:00633
91.72:42.88
86..33:+1.83
1+11-18:11.00
58.95+1.50
51.20011.00
2.3
GPR
ERM
84.8143.71
83. 63.98 1.72
48. 18.96-t1.00
40.91#0.228
87.25=0.66
64.36+0022 27
4.1
BERM
90.87:10.52
87.16:1.2.39
60.08:10.03
OOM
58.75=0.29
54.21:1.42
4.0
Tent
GCTA
91 91.96+0778 75
92. 75+2 48
66.3643.06
14.4+0.770
09.97-0.62
54.63+0.77
2.6
GTRANS
91.97:084
92. 70:2 46
68.54=5.50
45. 15.6110.61
39 09.8
54.4820.66
2.4
GOAT
0..98
192.79+22.0
*66.47-66.4
*44. 74.009
*60.00==0.65
55 035.23300.43
1.7
cannot applied to models that
normalizatioo layers.
Table 3: Summary of the experimental datasets that entail diverse distribution shifts.
Distribution
#Nodea
Train/Val/Ti Split
Adapted
Actificial Treasformation
Demain- Level
Ascuracy
Yaang
2018
Phoxo Phasko
7,860
119,081
Domain-
Ascuracy
Shelaur
2018
Elliptic
Temporal Evolution
208,769
234.365
Timo-Aware
Beors
2020
OGB-ArKiv
3443
1,166,243
Time:Aware
Acouracy
2020
Twitchh Explicis Crow- Domain Transfor 1,912 9,498
31,209
163, 188
Domein-Level
ROC-AUC
Rosomberoski
2021
Pacaboole-100
789
41,536
16,858
1,5900608 855
Domatn- Leval
Accuracy
Traud al.
2012
breakdown of these three distinct types of distribution shifts, are presented in Table 3 (1) Artificial
Transformation for Cora (Yang et al. 2016) and Amazon-Photo (Shchur et al.) 2018), where node features are replaced by synthetic features. (2) Dooss-Domann transfers for Twitch-E (Rozemberczki et al. 2021
and FB-100 (Trad et al. 2012), involving graphs from different domains. (3) Temporal Evolution for
Elliptic (Pareja al., 2020) and OGB-ArXiv (Hu al.) 2020),, utilizing dynamic datasets with natural evolving characteristics. The datasets split into training/ training/validatiinn valiivaiooiiieessstseeseseesssee sets with ratios of: 1/1/8 for Cora and Amazon-Photo;; 1/1/5 for Twitch-F; 3/2/3 for FB-100; 5/5/33 for Elliptic; and 1/1/3 for OGB-
ArXiv.
Baselines. GOAT is compared with four baselines: empirical risk minimization ERM, test-time training method Tent WWng et al.) 2021), memory-bank--asee method GrapHCTA(GCTA) (Zhang et al. 2024 the train-time state-of-the-art method EERM (Wu et al. 2023 which is exclusively developed for graph
OOD issues, and the test-time graph transformation state-of-the-art method GTRANS (Jin et al. 2023 All methods are evaluated with four popular GNN backbones: GCN (Kipf & Welling 2022), GraphSAGE
Hamilton et al. 2017). GAT /Veličković et al.) 2018), and GPR. (Chien et al. 2021). Their default setup follows that FERM More implementation details of the baselines and GOAT can be found in Appendix
Adjustments have only been made the hidden dimensions GAT to ensure consistency in the parameter across all four backbones.
8

---

Published in Transactions on Machine Learning Research (11/2025)
Table 4: Efficiency comparison. GOAT is more time- memoory-flleient than EERM on large graphs and comparable to GTRANS.
0.8
Accuracy
0.7
0.6
Rumming Time (s)
GPU Memory (GB)
0.5
Photo Elliptic
ArXiv
Photo
Elliptic ArXiv
EERM
413.4
629.6
10.5
12.8
24+
T6
GTRANS
1.9
6.8
12.2
1.6
1.3
4.1
Figure 3: Results on Elliptic under OOD, GOAT im-
GOAT
5.5
0.5
0.3
1.5
1.3
5.0
proves SAGE on most test graphs.
390000
300000
Sinquar Value 10-12
Stogiar Value
293333
10-1-
10
10"4.
10-25
150000
10:M
9390
(a)
(b)
(c)
Figure (b) Visualization of the low-rank property of matrix AX the LRA module of a GAT backbone trained the two largest test graphs on OGB-ArXiv(1693433 nodes) and FB-100(11554 nodes) under OOD settings. The singular values, obtained via SVD, show a rapid decay, indicating that node embeddings can be effectively compressed into virtual nodes of the units digit. (c) Visualization of the distribution of generated AX obtained after training GOAT on test graphs in Cora with Gaussian KDE. The x-axis represents the sum of feature values the nodes, and the y-axis represents the density of bias on each node within that value range. The further the mode of the distribution is from '0', the greater the degree of distribution shift.
All experiments are repeated 8 times with different random seeds. Due to page limits, additional basclines and backbones such as SR-GNN (Zhu et al. 2021). UDA-GCN (Wu et al. 2020). and GTN (Yun
et al. 2019 are included in Appendix D
Overall Comparison. Table reports the averaged performance over the test graphs for each dataset as well as the average rank of each algorithm. From the table, we conduct the following observations: (a) Overall Performance. The proposed framework consistently achieves strong performance across the datasets: GOAT achieves average ranks of 1.3, 2.5, 2.3, and 1.7 with GCN, SAGE, GAT, and GPR, respectively, while the corresponding ranks for the best baseline GOAT are 1.8, 2.5, 2.3 and 2.4. Furthermore, in most cases, GOAT significantly improves the vanilla baseline (ERM) by large margin. Particularly, when using SAGE as the backbone, GOAT outperforms ERM by 9.8%, 18.5%, and 3.9% on Cora, Elliptic, and OGB-ArXiv, respectively. These results demonstrate the effectiveness of GOAT in tackling diverse types of distribution shifts. (b) Comparison other baselines. Both GraphCTA and EERM modify the model parameters improve model gencralization. Nonctheless, they are less effective than GOAT, as GOAT takes advantage of adapting the pre-trained GNN to the environment of test graphs. As test-time training methods, Tent and GTRANS also perform well in cases. However, Tent is ineffective for models that do not incorporate batch normalization. On the other hand, GTRANS not only modifies node features but also alters edges, which can backfire if the edge modifications are not carefully chosen, potentially leading to misrepresentation of the graph structure.
We further show the performance on each test graph on Elliptic with SAGE in Figure 3 and the results for other datasets are provided in Figure g in Appendix. We observe that GOAT generally improves over individual test. graphs within each dataset, which validates the effectiveness of GOAT.
9

---

Published in Transactions on Machine Learning Research (11/2025)
Quey
Surogetee
Origin
Surrogale
Arwaonn Lenazo-Ploolo Photo
Cara
AX Embodding
Node Embedding
Figure 5: t-SNE' visualizations of AX and node feature embeddings Amazon- Photo, Cora, and datasets. The embeddings are obtained after a pre-trained GCN, using AX generated by our method and surrogate loss proposed by GTRANS.
Efficiency Comparison. In Table 4 we compare the computational time and GPU usage on the largest graph of each dataset for our GOAT, EERM, and GTRRANS methods. Unlike EERM, which increases pre- training gencralization through extensive environment augmentation during train time, GOAT optimizes efficiency by minimizing reliance computationally expensive data augmentations and parameter tuming, In contrast to GTRANS, which adjusts based on the proportion of edges modified on the graph, GOAT requires sampling only a minimal number of two augmented input graphs per training epoch. These features ensure that GOAT not only conserves GPU resources but also accelerates the adaptation process during test time, showcasing substantial efficiency improvements over both train-time methods and other test-time methods
4.2 A Node-Level Jow-Rank Perspective on Adaptation
After tuning the parameters of LRA on validation sets and obtaining the optimal results through test- time tuning, we further investigate the principal components of the low-rank matrix AX RR in the N-dimensional space Figure 4 (b) shown. By performing Singular Value Decomposition (SVD), we obtained the singular values sorted in descending order and compared the major eigenvalues, showing significant decline compared to the others. In almost all large graphs, the adaptation graph AX exhibits low rank along its N-th dimension. This phenomenon differs from the low-rank attention applied to the node feature across each node's dimensions. This finding further demonstrates that the input test graph can be adapted using a low-rank additive representation. To claborate, in Eq. Eq.77. AX can also take the form AX R with N' nodes (N' & N), selected from a predefined node dictionary. This provides empirical evidence for setting our |n] hyperparameter to small constant that is independent of the number of nodes. In comparison, with full-node attention, the time cost will surge to 4x or even cause an 00M error.
4.3 An Embedding View on Adaptation AX
We further visualize the t-SNE distribution of different AX generated by our paradigm and GTRANS different test graphs. From Figure 5] it can be observed that the AX obtained through our paradigm, i.e., optimized with LRA module and LAZA, forms low-dimensional manifold M after being embedded by the pre-trained GNN. This implies that AX itself belongs to a function manifold, encouraging gv(-) to transform each node feature along the local tangent space of M or along the curvature directions of M.
Detailed early-stop procedures are shown in Appendix
10

---

Published in Transactions on Machine Learning Research (11/2025)
Table 5: Ablation study of the overall loss function Caza comparison the Elliptic datasel under OOD, Two-view sampling under a test environment shows improvement in GCN average performance test graphs with the addition of each LAaA constraint component, demonstrating the effectiveness of each part the loss function and the choice of the number of samples.
Fl scome
A=0.11 A. o-Se 4
Configuration
Performance
1e-3
Se.3
Se-2
0.3
Pre-train
Le
L
One sample
Two samples
Figure 6: a Parameter Study, We compared the parameter sitivity under two commonly used settings: 0.0005 and X 0.1
10.00%
10.00%
-1.51%
-1.51%
v
/
-1.62%
+-4.49%
/
-0.18%
+5.28%
Graph
Greph
Graph.2
Srrch
0raph-3
Gragh
Greph44
Crech
Graph5
Density
6mphh7
Grach
Greph-3
Crah
Graph
Srahh 26
0rphh11
Grgg
Gmph:11
Greh
0mph.12
6m
Grsph15
Chalh
Greph 1i
0-
08
00.44
-0.02
0.00
0.04
Figure 7: Visualization of the distribution of generated AX obtained after training GOAT 32 test graphs in Elliptic with Ganssian KDE.
Furthermore, we found that although the AX generated by the GTRANN method still adheres to the clustering distribution of classification task labels, which, to some extent, even distorts the original topological structure learned by the GNN. A notable observation that the performance of GTRRANS correlates with the degree of distortion introduces. The distortion, in effect, allows AX to generate a 'spurious' graph that potentially overshadows the importance of the original node features for the classification task. Upon examining the node embeddings before and after transformation by AX,, it becomes evident that our method significantly enhances performance while maintaining largely consistent distribution. This further demonstrates that the AX generated by our approach represents the degree of distributional shift, rather than conforming to the classification and clustering observed during pre-training.
4.4 Distribution Shift Quantification
We utilize Kernel Density Estimation (KDE) to visualize the distribution of AX generated by our adapter obtained through GOAT on Cora and Elliptic as Figure 4(c) and Figure 7 shown, by aggregating each node's feature dimensions distribution should be Due to the varying degrees of distribution shifts in different graphs, after tuning by GOAT our adapter can effectively capture the discrepancy between the current test graph and the pre-trained GNN. Adding the generated A.X can be seen as the mapping from the current test graph to the distribution to which the original training graph belongs. Therefore, the farther the represcntation deviates from the origin, the more severe distribution shift the graph has, whether observed from the perspective of the entire graph or an individual node's 's perspective. The distribution shifts in the time-evolving graph could be more intuitive as time flows. Furthermore, we show AX's distribution in other datasets and compare with the central moment discrepancy (CMD) (Zellinger al.) 2022) measurement in the Appendix.Figurg Figure highlighting the interpretability of our designed adapter.
AX. As the initialization of the AX is zero, the mean and mode of the initial
11

---

Published in Transactions on Machine Learning Research (11/2025)
01325
GCN
SAGE
GMR
SAGE+GOAT
0:94
GPR+COAT
accuacy
0.900
accurntyy
0.055
0.97
0:90
0.800
0.96
0.825
0.00
0.05
0.30
0.35
0.20
0.25
0.00
0,3
0.30
0.15
0.20
0.25
4.00
0.05
0.10
0.35
0.20
0.2
Nuise Structure ralio
Nolse structure rob
Noise structure radio
Figure 8: Performance of GCN, GraphSAGE, and GPR on Cora under structural attacks (edge
4.5 Ablation Studies and Parameter Study
Overall Loss Function LAZA. The ablation study shown Table 5 demonstratts the effectiveness of the different components in our proposed loss function. Optimizing C. alone may lead to instability and mode collapse, which empirically proves our proposed target in Eq. (5) and Eq. 12 The target Ln is affected by whether the pre-trained GNN has sulficient generalization capability. Therelore, optimizing Le alone is also sufficient to meet the requirements of (g. The best performance was achieved when all three components were jointly optimized. Additionally, sampling at least two distinct graph augmentations in Daug is essential to prevent biased learning, which empirically supports choice of enforcing (q a p) in Eq.(5) We show more detailed loss component ablation study with backbone GraphSAGE in Appendix E.1
Adapter LRA. In Figure G we show the parameter study of À and a in LAZA- Note that while the proportion
A T-a may vary, 1 1-a a should remain relatively large to ensure that the constraint in Eq. i.e.
Cn. is satisfied before optimizing the other objectives.
Number of sampled views. We further show the time cost and the performance in more sampled views (2 2) in Appendix[E44 The results validate our choice that using two samples strikes excellent balance, capturing the vast majority of the potential performance gain while maintaining the high efficiency that is critical for a practical lest-time adaptation method.
4.6 Robustness under Structure Adversarial Attack
We evaluate the robustness of GCN, GraphSAGE, and GPR on the Cora dataset (Yang et al. 2016 under structural attacks, we perturb the graph by both adding and deleting edges. For a given perturbation budget ratio, we randomly delete a number of edges corresponding to ratio/2 of the original edge count, and randomly add an equal number of new edges. For the GCN model, our mothod achieves an average improvement of 2% under 25% structural altack noise, while for GPR, it delivers stable gain of 0.5% across all noise levels. Without modifying hyperparameters (e.g.. learning rate, sampling ratio), our method consistently improves backbone model performance by 0.5% 2.5% across varying attack intensities. Although our test- time paradigm consistently enhances adversarial robustness, its performance remains dependent on the base backbone's 's inherent generalizat ion capability and adversarial robustness. Notably, this gain is achieved with just two samples with any subgraph sampling strategy, demonstrating computational efficiency alongside robustness.
4.7 Further Analysis
Universal Bias vs. Local-global Bias. We compare the average improvement of various non-customized additional parameter methods using our proposed LAZA on graph classification datasets HIV (a small-scale real-world molecular dataset adapted from MoleculeNet. (Wu et al. 2018)) under the OOD setting in GOOD (Gui et al. )022) during test time. Average results of different designs basis are shown in Table 3
We show its improvement of ROC-AUC(%) based on ERM with GCN. For node classification, is evident that UPF's universal prompts Fang et. al. 2024),
AX E Rixd, across all nodes, are less customizable
12

---

Published in Transactions on Machine Learning Research (11/2025)
for classifying each node in an OOD environment, and might even learn controversial knowledge, therefore
showing diminishing performance.
Moreover, using a selection dictionary Sun et al. 2023 AX E Rkxd (k < N), also presents difficulties during test-time training. In contrast, subgraph-focused methods (Sun et al., (022) can simultancously capture the optimal bias more effectively, yielding relatively higher results, espe- cially when it extends to node-wise bias, i.c. each nodc's learnable bias is different, AX E RN These demonstrate that, at least in OOD node classification, bias design that focuses on local-global context can better capture the relationships of nodes within the OOD enviromment. The node- wise bias method is particularly well-suited to our designed strategy, as it can better adapt to different distribution shift scenarios. This further validates the rationality of our adapter's design. Furthermore, based on the graph prompt mothods (Liu et al., 2023; Yu et al. 2023), we experimented with incorporating learnable scaling parameter that multiplies the weights of each GNN layer or node embeddings during test time. However, we found this approach difficult to apply effectively in our context.
Table 6: Bias Comparison
Method
Avg. Impr
Universal
+0.01%
Prompt diet +0.01%
Sub-graph
+1.52%
Node-wise
+2.35%
5 Further Discussion
It is worth noting that the distribution shift issuc in graph models can lead to significant risks and negative consequences in real-world applications. For instance, when GNNs are applied in financial risk control systems, distribution shifts in the input data may cause a large number of misjudgments, leading to severe economic losses or compliance issues. Similar risks exist in other high-stakes domains such as healthcare and cyber-criminal justice, where the reliability and robustness of graph- based decision-making systems under dist iibbutional changes are critical. Therefore, it. is crucial to develop effective methods to detect and adapt to distribution shift scenarios in graph learning and to carefully analyze and mitigate the potential negative socictal impacts. Our work aims to contribute to this important rescarch direction.
Another illustrative example of the potential negative impact of the distribution shift issue in graph models is in the context of social network analysis for misinformation detection. GNNs have been widely adopted to identify fake news and rumors based on the propagation patterns and content features in social networks. However, the characteristics of misinformation can evolve rapidly over time, leading to distribution shifts between the training and test data. If the GNN-based misinformation detectors fail adapt to such changes, they may miss emerging misinformation or cause false alarms, which can have severe societal consequences such as public panie, political manipulation, and erosion of trust in media. This urges the development of graph OOD detection and adaptation methods that robustly handle the dynamic and adversarial nature of online misinformation. Our GOAT framework takes step towards this goal by enabling test-time adaptation of GNNs to evolved data distributions.
6 Conclusion
In this work, we proposed GOAT a data-centric and self-supervised test-time adaptation framework for graph noural networks that avoided the complexity and instability of structuro-based transformations. By focusing exclusively on node feature optimization, GOAT enabled efficient and interpretable adaptation to distribuution-shifted test. graphs without requiring access to training data or test labels. To further enhance robustness, we introduced a low-rank adapter that generated diverse, graph-specific transformations and mitigated the risk of transformation collapse. Extensive experiments six real-world datascts demonstrated that GOAT consistently improved performance across different GNN backbones, outperforming existing data-centric and model-centric test-time adaptation baselines. These results highlighted the potential of feature-only, data-centric approaches for test-time graph adaptation. In future work, we will explore a more relaxed optimization objective and inspire more discussion and novel propositions.
13

---

Published in Transactions on Machine Learning Research (11/2025)
Acknowledgments
This work supported by the National Science Foundation (award number 2451605). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views the NSF. We thank Professor Tianhao Wang for critical feedback on the proofs and Tiange Lyu for valuable input during the carly stages of this work.
References
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. ar preprint wrXic:1907.02893 2019.
Guanzi Chen, Jiying Zhang, Xi Xiao, and Yang Li. Graphtta: Test time adaptation on graph neural networks. arXiv preprint Xrx::22008.00 09126, 2022a.
Yongqiang Chen, Yonggang Zhang, Yatao Bian, Han Yang, MA Kaili, Binghui Xie, Tongliang Liu, Bo Han, and James Cheng. Learning causally invariant representations for out-of-distribution generalization on graphs. Advances in Neural Information Processing Systems, 35:22131-22148, 2022b.
Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph neural network. In International Conference on Learning Representations, 2021.
Taoran Fang, Yunchao Zhang, Yang Yang, Chumping Wang, and Lei Chen. Universal prompt tuning for graph neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Bahare Fatemi, Layla El Asri, and Seyed Mehran Kazemi. Slaps: Self-supervision improves structure learning for graph neural networks. Advances in Neural Information Processing Systems, 34:22667-2268) 2021.
Shurui Gui, Xiner Li, Limei Wang, and Shuiwang Ji. Good: graph -attof--iittibuttion benchmark. Advances in Neural Information Processing Systems, 35:2059 -0700 2073, 2022.
Yuxin Guo, Cheng Yang, Yuluo Chen, Jixi Liu, Chuan Shi, and Junping Du. A data-centric framework to endow graph neural networks with out-of-distributton detection ability. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 638-648, 2023.
Will Hamilton, Zhitao Ying, and Jure Leskovee, Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Dan Hendryeks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint NKk::1903.122:11 2019.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In International Conference on Learning Representa-
tions.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118-222 122113,, 2020.
Kexin Huang, Ying Jin, Emmanuel Candes, and Jure Leskovec. Uncertainty quantification over graph with conformalized graph neural networks. Advances in Neural Information Processing Systems, 36, 2024.
Wei Jin, Tong Zhao, Jiayuan Ding, Yozen Liu, Jiliang Tang, and Neil Shah. Empowering graph representation learning with test-time graph transformation. In The Eleventh International Conference on Learning Representations, 2023.
Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations, 2022.
14

---

Published in Transactions on Machine Learning Research (11/2025)
Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. Ood-gnn: Out-of-distribution generalized graph neural network. IEEE Transactions on Knowledge and Dala Engineering, 35(7):7328-7755 2022a.
Jinfeng Li, Weifeng Liu, Yicong Zhou, Jun Yu, Dapeng Tao, and Changsheng Xu. Domain-invarian graph for adaptive semi-supervised domain adaptation. ACM Transactions on Multimedia Computing, Commu- nications, and Applications (TOMM), 18(3):1-18, 2022b.
Shuhan Liu and Kaize Ding. Beyond generalization: survey of out-of-distribution adaptation on graphs arXiv preprint 2rX::2202.. 11153, 2024.
Yuejiang Liu, Parth Kothari, Bastien Van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. Tu-+: When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems, 34:21808- 21820, 2021.
Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. Graphprompt: Unilying pre-training and down- stream tasks for graph neural networks. In Proceedings of the ACM Web Conference 2023, pp. 417-428, 2023.
Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In International conference machine learning, pp. 10-18. PMLR, 2013.
Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao Schardl, and Charles Leiserson. Evolvegen: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 5363-5370 2020.
Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. Dropedge: Towards deep graph convolutional networks on node classification. In International Conference on Learning Representations, 2020.
Benedek Rozemberezki, Carl Allen, and Rik Sarkar. Multi-scale attributed node embedding. Journal of Complex Networks, 922:cnaab014, 2021.
Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05885 2018.
Yu Song and Donglin Wang. Learning on graphs with out-of-distributiin nodes. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1635-1645, 2022.
Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang, and Xin Wang. Gppt: Graph pre-training and prompt tuning to generalize graph neural networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1717-1727 2022.
Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, and Jihong Guan. All in one: Multi-task prompting for graph neural networks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2120-2131, 2023.
Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In International conference machine learning, pp. 9228. PMLR, 2020.
Amanda I, Traud, Peter J Mucha, and Mason A Porter. Social structure of facebook networks. Physica A: Statistical Mechanics and its Applications, 391(16):4165 4180, 2012.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2021.
Yiqi Wang, Chaozhno Li, Wei Jin, Rui Li, Jianan Zhao, Jiliang Tang, and Xing Xic. Test-time training for graph neural networks. Nix preprint ar (tn:220000881. 2022.
15

---

Published in Transactions on Machine Learning Research (11/2025)
Man Wu, Shirui Pan, Chuan Zhou, Xiaojun Chang, and Xingquan Zhu. Unsupervised domain adaptive graph convolutional networks. In Proceedings of the web conference 2020, pp. 1457-1467, 2020.
Man Wu, Xin Zheng, Qin Zhang, Xiao Shen, Xiong Luo, Xingquan Zhu, and Shirui Pan. Graph learning un- der distribution shifts: A comprehensive survey domain adaptation, out-of-distributtinn, and continual learning. arXiv preprint arXiv:2402.1633 2024.
Qitian Wu, Hengrui Zhang, Junchi Yan, and David Wipf. Handling distribution shifts on graphs: An invariance perspective. In International Conference on Learning Representations, 2023.
Yingxin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua. Discovering invariant rationales for graph neural networks. In International Conference on Learning Represenlations, 2022.
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Ancesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learning. Chemical science, 9(2):513 530, 2018.
Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In International conference on machine learning, pp. 40-48. PMLR, 2016.
Gilad Yehudai, Ethan Fetaya, Eli Meirom, Gal Chechik, and Haggai Maron. From local structures to size generalization in graph neural networks. In International Conference on Machine Learning, pp. 11975 11986. PMLR, 2021.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph rastivevvve learning with augmentations. Advances in neural information processing systems, 33:5812 33:5812-5823, 2020.
Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Multigprompt for multi-task pre-training and prompting graphs. arXio preprint or 2r1:::3112.037311 2023.
Seongjun Yun, Minbyul Jeong, Raehyun Kim, Jaewoo Kang, and Hyunwoo Kim. Graph transformer networks. Advances in neural information processing systems, 32, 2019.
Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, and Susanne Saminger-Platz. Central moment discrepancy (emd) for domain-invariant representation learning. In International Con- ference on Learning Representations, 2022.
Zhen Zhang, Meihan Liu, Anhui Wang, Hongvang Chen, Zhao Li, Jiajun Bu, and Bingsheng He. Collaborate to adapt: Source-free graph domain adaptation via bi-directional adaptation. In Proceedings of the ACM on Web Conference 2024. pp. 664 675, 2024.
Tong Zhao, Yozen Liu, Leonardo Neves, Oliver Woodford, Meng Jiang, and Neil Shah. Data augmentation for graph neural networks. In Proceedings of the aaai conference on artificial intelligence, volume 35, pp. 11015 11023, 2021.
Qi Zhu, Natalia Ponomareva, Jiawei Han, and Bryan Perozzi. Shift-robust gnns: Overcoming the limitations of localized graph training data. Advances in Neural Information Processing Systems, 2021.
16

---

Published in Transactions on Machine Learning Research (11/2025)
A Proof of Propositions
Theorem 1. (Proved in GPF Fang et al., 2024) Given a pre-trained GNN model f, an input graph G, for any graph-level transformation g ((A,2) - (' there exists an additional extra feature vector AX that satisfies:
f(A,X ( f(g(A, x()
A.1 Population-Level Optimum of LAZA
We first characterize the theoretical optimum of the A2A loss in the population setting (infinite augmented views).
Proposition A.1. (Population Optimum) Suppose S3 = (()) is the adapted view representation and fu = f(9g) is the unadapted view representation, where Op and Gg are independently and identically distributed (i..d.) samples from a stochastic augmentation process Aug(-).
The population LAZA defined as:
L. = Epgr [1ls fall²)
The optimal adapted reprosentation that minimizes Lg is:
f Eug [fa _ paag
where Haug constant vector representing the expectation of all possible augmentedd view representations. Proof of Proposition A.1:
L. can be reformed as:
C. = Ep [E [Eg fall
[p]
For any fixed the inner expectation E,, fa 2 pp] is a standard mean squared error problem. By the property of expectation, arg miny E[[[V x = E[X], the optimal solution is:
f = 1fg [p]
Since Gp and Gn are i.idd. from Aug(.), the distribution of Sq is independent of p:
f = Eggfa] Haug
This shows that the population optimum is constant mapping for all inputs.
Remark 1. Proposition A.1 reveals an important theoretical property: the unconstrained A2A loss drives all adapted representations toward the same constant, Paag- While this may appear similar to the transformatio collapse we observe in practice, there crucial distinction: theoretical collapse converges to the correct constant Paug: whereas practical collapse lypically converges to a suboptimal constant that fails to capture the true augmentation distribution. Furthermore, while this is the theoretical optimum, our finite-sample algorithm with regularization operates in different regime.
A.2 Finite-Sample Behavior and Empirical Optimum
In practice, work with finite number of augmented views. This scction analyzes the behavior of the empirical LARA-
Proposition A.2. (Empirical Optimum) Given augmented views (G.) at test time, let f = f(e be the adapted representations and f, = =GG, be the fixed unadapted representations. The empirical LAZA
Cemp
Σ
11%
fz11
n(n 1)
3#1
17

---

Published in Transactions on Machine Learning Research (11/2025)
The empirical optimal solution that minimizes Dg (treating {fz} as fixed) is the leave-one-out (LOO) mean:
S
Kemp
E (I....,
Jf
Proof of Proposition A.2:
The empirical loss Cgmp is convex function that is separable in each f. Taking the gradient with respect to f and setting it to zero:
Σ
11
f,1l
jfi
Σ 2(1 f,)= =0
iti
(n 1),f Σn, f1
jti
1
femp
Eb
S3
n2
3+1
A.3 Mean Equivalence of Empirical Optimum and Samples
Proposition A.3. The average of all empirical optimal adapted representations equals the sample mean of unadapted representations:
plemp Jn
1
temp
Σ ビ5 = fr
n
i=1
Proof of Proposition A.3:
Computing the average of empirical optima:
plemp 1
plmp
E,
i=1 i=1
sti
For any fixed representation fg. it appears in the inner for all i # j, i.e,, it appears exactly (n times:
prmpp
Σ (n
1)53
Σ f$ = Jn
n(n 1)
j=1
Remark 2. (Key Insight Avoiding Collapse)
Propositions A.2 and A.3 reveal why transformation collapse does not occur in practice despite the theoretical prediction from Proposition A.1:
1. Input-Dependent Diversity: Each pemmp is different for different i. Specifically:
plemp
demp
1 (ʃa fi) + 0
The adapted representations remain input-dependent and diverse.
2. Fimite-Sample Effect: Only when n + oo do all flemp converge the same constant Poug- For finite n used in practice (e.g., 2 4), substantial diversity is preserved.
18

---

Published in Transactions on Machine Learning Research (11/2025)
3. Regularization via Architecture: Our low-rank adapter parameterization ge further constrains the solution space, preventing the algorithm from reaching even the finite-sample optimum {s-mp).
Therefore, the practical algorithm operates in a regularized, finite-sample regime that naturally avoids the constant mapping degeneracy predicted by population-level analysis.
A.4 Convergence Analysis
We now analyze how the average adapted representation converges to the population mean increases. Thcorem 2. (Convergence Rate via Bernstein's Incquality)
Let (G:)=1 be n ii.d. augmented views with representations (fi = f(g)) ft Nxd Define: Paug [[ (population mean) and fn = Ei= fo (sample mean)
Assume: 1. Representations bounded: || s M almost surely 2. Variance bound: Elil, - Maugl || o²
Then, applying the Vector Bernstein Inequality, for any å (0, 1), with probability at least 1 - ō:
||5n Mangl - 20 log (2Nd//)/)) 2M logg((NN/))
3n
where Nd is the node number times the dimension of representation fi-
By Proposition A.3, since fremp Jny this bound also applies to the average of empirical adapted represen- tations.
Proof of Theorem 2:
Define zero-mean random variables:
Z: = Mangs
i= 1.....
By definition, [[22] 0. We verify the conditions for vector Bernstein's inequality:
1. Zero mean: [[ = 0 by construction.
2. Bounded variance:
B|| B||Z|| B[Il Maugl | < o²
3. Bounded range: Since ||fil| < M and ||aug|] || = ||[[i||| S M (by Jensen's 's inequality):
Z = |Ife - Mangl || S [|aill + | S 2M A
Consider the sum S, = Em Zg By the vector Bernstein inequality, for any > 0:
f2/2
P P[ 2 < 2Nd exp
(Tnoz "na2 + 11/3
Setting the right-hand side equal to 8 and solving for t:
f2/2 -
2Nd exp
(=no² no² 2 + Bt/33
12/2
no² + Bt/3 log(5/2No)
f222
= log (2Nd/0)
n + Bt/3
19

---

Published in Transactions on Machine Learning Research (11/2025)
This is quadratic inequality in t. Solving yields:
B log(2Nd/6)
t t log(2N//)))
Since Jn MMagg a Sn, dividing both sides by n.
20 log log(2Nd//)
B log (2N(22)/))
[I - Hauoll S
7
3n
Substituting = 2M gives the stated bound.
Corollary 1. (Convergence Rate)
The bound in Theorem 2 consists of two teriss: Variance leri:: O(1/vT)) (dominant for large n) and Bounded term: (L)/n) (faster decay)
Therefore, as n + 00:
1
Jn. Maug O
Vn
This establishes that the average adapted representation converges to the population mean Paug at the standard parametric rate.
Remark 3. (Practical Implications)
The convergence analysis provides several insights:
1. Sample Efficiency: The O(1/VR) rate means that doubling the number of views reduces the error by a factor of /2 ss 1.41. This suggests moderate values of n (e.g., n = 2 to 4) already provide substantial benefits.
2. Diminishing Returns: Beyond a certain n, the computational cost of generating and processing addi- tional views outweighs the marginal improvement in convergence, explaining our empirical observation that performance saturates.
3. Connection to Collapsc: While fn >> Pang (collapse in the mean), individual femp maintain diversity through their leave-one-out structure. The regularization from our low-rank adapter further prevents collapse at the individual level.
20

---

Published in Transactions on Machine Learning Research (11/2025)
B Algorithm
For detailed pseudocode under the augmented graphs number of |r] = 2, please refer to the right-hand side of the page.
Algorithm GOAT for Full Test-Time Graph Adaptation
1: Input: Pre-trained GNN fe- (0* is fixed, without the last layer which is the classifier head) and test graph It. = (A,X), Sample (DropEdgd Aug(.)
Output: Model prediction y
Initialize LRAg, a, A, learning rate 1)
Lhest 00, patience = k, palience
0
5: for == to T" do
G' DEE()ie) (A', 9" DE(G(e) {A", x"
AX' = (A', X"), = X²)
7:
AX' cmb for (A', AX), AX" for ,A",AX"""" AX")
9.
Pemb = (A', X' +'XAX)) Hemo = for (A", X" *²*
10:
Lemb fe. (A- (A)) 2 fo. fo" X")
11:
L. = [E zmmb) + (ZHemo zembblll
12:
L. = 2 -'emb A8' emb) + (= =mmb AX" (mmm)ml
LR EAAX' emb AX" embl
13:
14:
L = aALL, + Le) + +1 (1- A)RR
15:
Update: v6 + v - 7
16:
if LR S Lbest then
17:
Lbeat = LR
18:
patiencc
19:
else
patience patiencenow -1 21: end if
20:
22:
patience now 2 patience then Stop end if
23:
24:
25: end for
26: AX = L
27: Ý fo. (A, X AX)
28: return Y
C Datasets and Hyper-Parametenn
In this section, we reveal the details of reproducing the results in the experiments. We will release the source code upon acceptance.
C.1 Hyper-Paree Parame-arrreeer Setting
For the setup of backbone GNNs, majorly followed FERM Wu et al. 2023
21

---

Published in Transactions on Machine Learning Research (11/2025)
1.0
1.00
Accuracy
0.9
Accuracy
0.99
0.98
0.8
GOAT
0.97
GOAT
T2
T3
T4
T5
T6
T8
T1
T2
T4
T5
T6
T8
(a) Amazon-Phoue
(b) Cora
0.65
ourr
Accuracy
0.40
Accuracy
0.55
Accuracy
0.60
0.35
0.50
0.55
T1
T2
T3
T1
T2
T3
11
T5
(c) ArXiv
(d) FR- 100
(e)
Twitch-F.
Figure Classification performance on individual test graphs within each dataset for OOD setting,
(a) GCN:
the architecture setup is layers with 32 hidden units for Elliptic and OGB-ArXiv, and 2 layers with 32 hidden units for other datasets, with batch normalization for all datasets. The pre-train learning rate is set to 0.001 for Cora and A Amz-Photo, 0.01 for other datasets; the weight decay is set to 0 for Elliptic and OGB-ArXiv, and 0.001 for other datasets.
(b) GraphSAGE: the architocture sctup is 5 layers with 32 hidden units for Elliptic and OGB-ArXiv and layers with 32 hidden units for other datasets, and with batch normalization for all datasets. The pre-train learning rate is set 0.01 for all datasets; the weight decay set to 0 for Elliptic and OGB-ArXiv, and 0.001 for other datasets.
(c) GAT:
the architecture setup is layers for Elliptic and OGB-ArXiv, and 2 layers for other datasets, with batch normalization for all datasets. Each layer contains 4 altention heads and each head is associated with 8 hidden units. The pre-train learning rate is set to 0.01 for all datasets; the weight decay set to for Elliptic and OGB-ArXiv, and 0.001 for other datasets.
(d) GPR:
We use 10 propagation layers and transformation layers with 32 hidden units. The pre-train learning rate is set to 0.01 for all datasets; the weight decay is set 0 for Elliptic and OGB-ArXiv, and 0.001 for other datasets. Note that GPR does not contain batch normalization layers.
For the baseline methods, we tuned their hyper-parameter based on the validation performance, For Tent, we search the learning rate in the range of [1e-2, 1e-3, le-4, le-5] and the running epochs in [1, 10, 20, 30). For EERM Wu et al. 2023) and GTRANS(Jin et al. 2023). we followed the instructions provided by the original paper. For GraphCTA(GCTA) (Zhang et al.] 2024), we tune the feature adaptation tn [5e-3, 1e-3, le-4, 1e-5, 1e-6], learning rate of structure adaptation 7/2 in [0.5, 0.1, 0.01], and alternatively optimize node features epochs in [1, 2, 3] and optimize graph structure epochs in 2], other parameters followed the instruction provided by the original paper. For GOAT, we adopt Droe as the augmentation function DE(+) and set the drop ratio to 0.05, K-layer aggregation in LRA set to 1 due to some GNN only has two layers in some datasets while the last GNN layer performs as a classifier head. We use Adam Optimizer for LRA module tuning. We further search the learning rate 7 in [le-2, 5e-3, 1e-3, 5e-4, 1e-4, 5e-5, 1e-5, 1e-6] for different backbones, the virtual nodes number [n| [1x, 2x, 5x, 10x, 20x] of the class number C, the attention dim datn LRA in [2, 4, 8, 16, 32], total epochs in [50, 100], and the patience in [1, 0.5, 0.1, 5e-2, le-2, 1e-3). Tn the optimization target, we search the À in [1, 3, 5, 10) and the in (0.999, 0.9, 0.75.
22

---

Published in Transactions on Machine Learning Research (11/2025)
Method
Amz-Photo
Cora
Elliptic
FB-100
OGB-ArXiv
Twitch-E
ERM
93.79:00.97
91.59-1..44
50.50
54.04+00.94
38.59-1.33
59.89++0.50
UDA-GCN
91.70=10.35
92.65-0046
51.57=1.31
54.11=00.54
39.43:1.771
32.
SRGNN
94.64:0.17
94.08+0.28
51.94+0.81
54.08=1.10
38.92:+.655
59.21=0.51
GOAT
9..32
94.79+1.36
55.83:13.81
54.19:12.04
39.44:2.02
60.15+1.300
Table 7: Performance comparison between GOAT with GCN and graph domain adaptation methods.
Method
Amz-Photo
Cora
Elliptic
FB-100
OGB-ArXiv
Twitch-E
GTN
44.99122.91
09.88
68.51 3.88
53.57+00.75 54.9410.61
43.08
62.30+00.16
GTN + GOAT
94.75
99.85#0.12
70.08+2.50
44.11-0.84
63.79:10.27
Table 8: Performance comparison between GOAT with other backbones.
0.5, 0.25, 0.1, 5e-2, 1c-2, 5c-3]. We note that the process of tuning hyper- ppper-parameters is quick due to the high efficiency of test-time adaptation learned over whole epochs set due to the patience of dissatisfaction of constraint in Eq 12
demonstrated in Section 4.1. Furthermore, not every test graph is
Evaluation Protocol. For ERM (standard pre-training), we pre-train all the GNN backbones using the common cross-entropy loss. For EERM, it optimizes a bi-level problem to obtain a trained classifier. Note that the aforementioned two methods do not perform any test-time adaptation and their model paramoters are fixed during the test. For the four test-time adaptation methods, Tent, GCTA, GTRANS, and GOAT We first obtain the GNN backbones pre-trained from ERM and adapt the model parameters or graph data at test time, respectively. Furthermore, Tent minimizes the entropy loss and GTRANS and GCTA both minimize the contrastive surrogate loss, while GOAT minimizes the Target CA2A-
C.2 Hardware and Software Configurations
We perform experiments NVIDIA GeForce RTX 3090 GPUs. The GPU memory and running time reported in Table 3 measured one single RTX 3090 GPU. Additionally, eight CPUs, with the model name Intel(R) Xcon(R) Silver 4210R CPU @ 2.40GHz. The operating system utilized in our experiments was Ubuntu 22.04.3 LTS (codename jammy).
D More Experimental Results
D.1 Overall Comparison
To show the performance on individual test graphs, we choose SAGE as the backbone model and include the box plot on all test graphs within each dataset in Figure g We observe that GOAT generally improves over each lest graph within each dataset, which validates the effectiveness our proposed method.
D.2 Comparison to More Baseline and Backbones
Wu et al. 2020). SR-GNN regularizes the model's performance on the source and target domains. Note that SR-GNN was originally developed under the transductive setting where the training graph and test graph are the same. To apply SR-GNN in our OOD setting, assume the test graph is available during the training stage of SR-GNN, as typically done in domain adaptation methods. UDA-GCN is another work that tackles graph data domain adaptation, which exploits local and global information for different domains. We followed the authors' suggestions in their paper to lune the lypae-eassmeters and the results are shown in Table 7 On the one hand, we can observe that these graph domain adaptation methods generally improve the performance of GCN under distribution shift and SRGNN the best-performing bascline. On the other hand, GOAT performs the best on all datasets except Amz-Photo. On SR-GNN, which indicates that joint optimization over source and target is necessary for this dataset.
To compare their empirical performance, we include two GraphDA methods (Zhu et al.) 2021
GOAT does not improve as much
23

---

Published in Transactions on Machine Learning Research (11/2025)
1100
100
2000
Graph
Graph 4
Graph
Graph
Graph
A00
Sraph
Grapn
Srap1
Grapn D
200
1306
196
Srazzon-Photoo
(b) Twitch-F
1800
1400
250000
1200
Densty
1000
Graph
150000
Geaph
Graph
600
50000
-0.5
0.0
0.5
1.0
1.5
(c) FB-100
(d) OGB-ArXiv
Figure 10: AX Distribution after training each dataset.
GraphID
Go
91
G2
93
94
95
G6
9T
Gs
Amz-Photo
6.4
5.1
5.5
3.7
2.8
3.7
3.9
6.6
Cora
5.4
4.2
4.8
6.3
5.5
4.8
4.6
5.4
Elliptic
80.2
90.8
114.3
86.5
789.3
781.6
99.4
100.4
150.6
OGB-ArXiv
14.7
20.6
10.4
FB-100
29.7
16.9
32.9
Twitch-E
8.6
6.1
9.0
8.4
9.7
Table 9: CMD values on each individual graph based on the pre-trained GCN.
However, recall that domain adaptation methods are less efficient due to the joint optimization on source and target. Overall, the test-time graph adaptation with our adapter could better fit the specific distribution shifts that deviate from the source target. As shown in Table S GOAT could also adapt to more popular backbones.
D.3 Quantifying distribution shift through LRA
In this section, further show more distribution of AX generated by GOAT that indicates the OOD degree of cach test graph. We can see an extreme shift in Figure () that as the snapshots flow from the validation, the mode and the mean value AX shift away from the initialized value, which shows further deviation of the later test graphs from the train source distribution. Furthermore, following SR-GNN (Zhu ct al.) 2021), we adopt central moment discrepancy (CMD) (Zeliinger al. 2022) as the measurement quantify the distribution shifts in different graphs, we present them in Table as comparison with AX in GOAT
24

---

Published in Transactions on Machine Learning Research (11/2025)
Siguuer Velue
Singuler Velue
109
10"4
10
10-17-
8 9
Index nodess
Index
(a) 91
(b) 92
Figure 11: (a)(b) SVD of E in I.A after training on test graph 91, 92 in OGB-ArXiv with distribution shifts.
Table 10: Adaptation efficiency under different numbers of virtual nodes Twitch-E (C 2).
Virtual Nodes
Value
Time (s)
GPU Memory (GB)
Default Sctting
2C' = 1.50 + 0.38
0.9
4C' =
1.44 + 0.40
0.9
20C
1.54 + 0.41
1.0
200C 400
1.62 + 0.42
1.1
Full Graph (No Compression)
N R 4000
6.72 +: 2.77
3.4
D.4 Low-rank of Node-level Representation on Large Graph
In Figure we show other A.X generated from LRA on two large test graphs in OGB-arXiv with 69499 and 120740 nodes.
D.5 Efficiency of LRA design: Impact of the Number of Virtual Nodes
The number of virtual nodes, denoted as is crucial hyperparameter in our low-rank cross-attention adapter, controlling the rank and expressive capacity of the transformation. In our main experiments, we set r = 2C (where C is the number of classes) as the default, based on a balance between efficiency and effectiveness. To evaluate the sensitivity of our method to this choice, we conduct an ablation study on the Twitch-E dataset, varying from 20 200C, and compare against full-graph baseline where adaptation is performed over all N z 4000 original nodes (denoted as "N (Full Graph)").
The results are summarized in Table 10 reporting adaptation time per graph (in seconds) and overall GPU memory consumption during inference. All configurations achieve nearly identical performance, with final accuracy stabilized at 0.601, indicating that the model's predictive quality is robust to the number of virtual nodes.
As shown the table, increasing the number of virtual nodes has minimal impact on adaptation time until 200C, while memory usage grows only slightly due to the lightweight attention mechanism. In contrast, the full-graph baseline suffers from 4.5x slower adaptation speed and consumes 3.8x more GPU memory, highlighting the effectiveness of our low-rank design.
Notably, even the smallest configuration (r = 2C) achieves the same performance as the full-graph model, demonstrating that our virtual nodes efficiently capture the essential transformation signals without requiring high-rank or full-node computation. This validates our design choice: using 2C provides sufficient expressiveness while maintaining high efficiency, making it a scalable and practical default across datasets with varying sizes and label complexities.
25

---

Published in Transactions on Machine Learning Research (11/2025)
E Ablation study
E.1
Optimization Object
In Figure we show the parameter study of and in CAZA- Noted that there could be a different proportion of nーa.. while it still should have a rather value of in that the constraint in Eq. (11 (CR) should be satisfied first then the other objective could work.
To evaluate the individual and combined contributions of each component of CA2A, we conduct an abla- tion study across all six datasets using GraphSAGE the backbone. We report the average performance improvement (in percentage points) over the no-adaptation baseline. The results are summarized in Table I1
Table 11: Ablation study on the components of the adaptation loss. All configurations include the base adapter (Caza)- Performance is measured as average improvement (%) over the no-adaplation baseline across six datasets with GraphSAGE.
Loss Components
Average Improvement (%)
Method
C,
Le
LR
Two-Sample
One-Sample
Baseline
+0.00
+0.00
+ L.
+2.08
+2.08
+ Le
+2.15
+2.09
+ LR
+2.14
+2.14
Ls + Le
+2.31
+2.20
+ L, + LR
V
+2.14
+2.14
+ L. + Cn
+2.14
+2.10
Full Caza
+2.32
+2.16
Key observations from the results include: Each individual loss term provides consistent gain over the baseline, with Le yielding the highest ntgleecomponent improvement (+2.15%), followed closely by CR (+2.14%) and L. (+2.08%). The combination of Lg and Le achieves the best overall performance (+2.31%), indicating a synergistic elfect: the consistency constraint helps align the adapter with the GNN's inductive bias, while symmetry enforces view invariance. Adding Ln to other combinations does not further improve performance and can slightly degrade results when combined with L., suggesting that excessive regularization may suppress mcaningful adaptation signals. The full model (all three terms) achieves the highest two-sample improvement. (+2.32%)) validating the design of our decomposed objective.
These findings reveal that the components are not simply additive but exhibit hierarchical and interde- pendent relationship. In particular, Le plays a central role in ensuring structural compatibility, while CR stabilizes training without dominating the update direction.
It is worth noting that the relative effectiveness of L appears somewhat weaker in this ablation (with SAGE) compared to the results on GCN backbones presented in the main paper. While the trend of L. being the most impactful component remains consistent, the performance gain from LR is less pronounced here. This discrepancy may stem from differences in the architectural sensitivity of GCN and SAGE to regularization, from variations in message-pasaas passing dynamics under different aggregation schemes.
Nevertheless, the core insight that L. and LR act as complementary mechanisms to stabilize and enable L. rather than merely augment ho-dsls across architcctures. This ablation confirms that our decomposition effectively balances expressiveness and optimizability, enabling robust. and ellective Lest-time adaptation, even if the exact contribution magnitudes vary across model families.
E.2 Different Augmentation Methods Used in Optimization
In Target 4, we used DropEdge as the augmentation function Aug(-) to obtain the augmented view and enough inductive graphs. In practice, the choice of augmentation be flexible, and here we explore two
26

---

Published in Transactions on Machine Learning Research (11/2025)
other choices: node dropping and EdipEdge (You et al. 2020). Specifically, adopt a ratio of 0.05 for node dropping, a ratio of 0.05 and 0.5 for FlipEdge, and ratios of 0.05 and 0.5 for DropEdge. We observe that (1) GOAT with any of the three augmentations can greatly improve the performance of GCN under distribution shift, and (2) different augmentations lead to slightly different performances on different datasets
E.3
Parameters in LRA
After luning over datasets, the hyperparameter almost shows a slight difference. Therefore, dattn is set to 8, |n] is almost 10 C(C the class number of nodes in test graph). Furthermore, we explore that, unlike the Transformers structure in NLP, the multi-head attention and the residual connection cannot improve the performance in our LRA module, which indicates the graph structure data information learned with GNNs has different representation from that learned as in NLP as sequences.
E.4 Choice of Number of Augmented Views (K)
The number of augmented views, denoted as K, is a key hyperparameter in our test-time adaptation frame- work. In the main paper, we adopt K = 2 as the default setting for all experiments. This choice is motivated by both robustness and efficiency considerations. Below, we provide a detailed ablation study to justify this design decision.
Using two augmented views provides simple and stable baseline that ensures consistent hyperparameter settings across diverse dalasets and backbone architectures. More importanlly our extended experiments demonstrate that while increasing K > can yield marginal performance improvements, the gains quickly diminish relative to the significantly increased computational cost.
To illustrate this trade-off, we conduct ablation experiments on three representative datasets Cora-GAT. Elliptic-GCN, and Twitch-F-GCN- with K ranging from to The results are summarized Table We report both adaptation time per graph (in seconds) and final model performance.
Table 12: Performance and adaptation time under different numbers of augmented views (K).
Cora-GAT
Elliptic-GNN
Twitch-E-GCN
(# Views)
Time (s)
Acc.
Time (s)
F1.
Time (s)
ROC-AUC.
1.3689
0.9250
1.9075
0.5222
0.8734
0.6210
1.9199
0.9249
2.7447
0.5300
1.2444
0.6210
2.7233
0.9245
3.7554
0.5344
1.5541
0.6209
3.3211
0.9266
4.0833
0.5378
2.0369
0.6209
10
7.4531
0.9281
7.5256
0.5578
4.1890
0.6209
20
14.0208
0.9308
16.0332
0.5678
8.7940
0.6209
As shown in the table, the adaptation time increases substantially with K. This is due to two factors: (1) linear increase in the number of GNN forward passes (one per view), and (2) a combinatorial increase in the number of pairwise similarity comparisons in the symmetry loss Cs, which scales as (K) = K(K - 1)/2.
The performance gains from using more than two views are marginal compared to the rising computational cost: on Cora- GAT, increasing K from to 10 yields only +0.31% improvement with a 5.4x time increase; Elliptic- GCni going from K = 2 to K = 5 improves performance by +1.56%, far less than the +5.28% gain from adding the second view, yet nearly doubles the cost; and on Twitch-E, no gain is observed beyond K = 2, indicating view diversity saturation.
These results confirm that the performance benefit of using more than two views is limited, especially when considering the super-lincar growth in computational overhead. In contrast, K 2 captures the vast majority of the potential gain from contrastive adaptation while maintaining high efficiency- mrrceencymmkkingg it an ideal choice for practical test-time adaptation. Therefore, conclude that K 2 offers excellent balance between
27

---

Published in Transactions on Machine Learning Research (11/2025)
effectiveness and efficiency, and is well-suited as general-purpose setting across various graph structures and model backbones.
28


Reviews:
[{"review_id": "YLqEmP0bKx", "reviewer": "Reviewer_5E6D", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths \n- The paper tackles a very relevant problem in graph representation learning, whose advances may have a significant impact on real-world  applications.\n- The proposed method is simple.\n- The authors made efforts to validate the low-rank property of the node-feature update (e.g., Fig. 4).\n\n\n---\n\n\n### Weaknesses \n\n**Rationale/motivation**. In test-time adaptation based on features, I would expect a design that minimizes the discrepancy in feature distributions between source and target graphs. It is not clear why the proposed loss in Eqs (5) and (6) would lead to better predictive models under distribution shifts. In other words, what is the rationale of using proxy representations that are perturbations of test-time graphs?\n\n**Empirical results.** I found the empirical gains of GOAT to be rather marginal. Overall, the gains are within one standard deviation of the second-best approach. This is particularly relevant when we compare against GTrans, which is a method that exploits structural adaptation --- a design principle this paper aims to avoid. Thus, from an empirical perspective, the paper fails to demonstrate that focusing on feature adaptation is beneficial over feature+structure adaptation. \n\n**Lack of ablation studies**. Given the empirical nature of this work and the unclear motivation of the proposed design (loss), the ablation study in Table 5 is rather insufficient. I suggest that the authors consider more datasets/GNNs and a larger number of samples.  \n\n**Clarity and organization**. Although the proposed idea seems relatively simple, the manuscript has several issues that significantly hinder its readability, including typos, notational abuse, unsupported claims, and poor organization. For instance: \n- In section 3.1, \\mathcal{A} is used to denote a matrix and also a set. \n- Eq. (5) says $L_s = \\arg\\min E[...] $ -- it should be $L_s(\\psi) = E[...]$\n- The paper uses $|\\psi|$ without specifying what $|\\cdot|$ means --- I assumed it means the number of parameters, but it should be mentioned. Also, why $|\\psi|$ should be significantly smaller than $|\\theta^\\star|$ (Pag 4)?\n- The paper says \"This consistency facilitates the rapid optimization and performance consistency of the transformation g across different designs of f\". To state this, the authors should either show it empirically or theoretically. \n- The paper introduces vanilla GNNs in the middle of the method section (Pag. 5) --- this often goes as Background.\n- Why use a parametric hypothesis test in Table 2? Have the authors verified normality assumptions?\n- What does the baseline \"ERM\" mean? Minimizing only L_s? Also, I wouldn't call it ERM since there is no guarantee of convergence to the ERM hypothesis (global minimizer) due to the nonlinear nature of the loss.\n- What does 1 < p < |\\tau| mean as a subscript of expectation in Eq. (12)?\n- Right after Eq. (6), the paper says \"we apply a subgraph sampling strategy as in Hamilton et al. [GraphSAGE, which uses neighborhood sampling] to generate sufficient graph augmentations\". And then it says \"note that the augmentations here are applied to node features\". I found it very confusing --- mentioning GraphSAGE and saying it only operates on node features.\n- The paper says \"... in Eq. (5), collapse mapping occurs when each augmented graph receives the same transformation solution\". From this, I understood that the collapse arises due to the non-injective nature of the mapping. However, it is not clear if this solution corresponds to a (local) minimizer of Eq. (5). Could you show that? Later, in Section 3.3, the paper says \"The contraint in Eq. (5) [actually Eq. (6)] requires the same input graph for both ... , which can easily result in the mapping collapse\". From this, the collapse refers to $g$ being the identity map. Could the authors precisely  describe what is the expected collapse issue in the initial formulation in Eqs. (5) and (6)?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a novel approach for test-time adaptation in node-level tasks where the node labels in test graphs are not available. The proposed method is called GOAT and builds upon adapting node representations --- bypassing the need to adapt graph structural information. To avoid the so-called transformation collapse in the proposed loss functions, the paper leverages a low-rank adapter. Experiments on six real-world datasets and three shift mechanisms aim to assess the effectiveness of the proposal. The paper also reports a small ablation study and preliminary experiments on robustness to adversarial attacks.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I believe the paper would benefit from:\n\n- A more extensive set of ablation studies to assess the role of each loss terms --- extend Table 5 with more datasets/baselines. \n- Additional results using full-rank updates to demonstrate the collapse issue.\n- A discussion about the rationale of Eqs (5) and (6). Could you provide concrete examples of why solving this optimization problem leads to better models under distribution shifts? \n- Clarification regarding the transformation collapse issue.\n- Datasets where the empirical gains of GOAT are statistically significant.\n- Fix the notation issues and typos I previously mentioned, including clarifications when needed.", "broader_impact_concerns": "I don't have concerns regarding the ethical implications of this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "F1XDW9duIr", "reviewer": "Reviewer_b8W9", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1) This paper is well-written and easy to follow.\n\n2) The topic of test-time adaptation for distribution shift is significant for the community, and the proposed self-supervised strategy and low-rank adapter are novel.\n\n** Weaknesses**\n\n1) The motivation for introducing a low-rank adapter needs further explanation. Unlike LLMs, the parameters in GNNs are much fewer, and I do not think we need to use a low-rank adapter due to the limitations of training resources.\n\n2) I suggest that the authors add a preliminary section to facilitate the understanding of this paper, especially on background knowledge about OOD or TTA.\n\n3) Efficiency analysis should also be included in the experiment to show the superiority of the low-rank adapter.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose Graph Optimization via Augmented Transformations (GOAT) to mitigate the distribution shift issues in GNNs. The merits of GOAT are two-fold:\n\n1) GOAT is self-supervised and focuses exclusively on node feature transformations.\n\n2) A parameter-efficient low-rank adapter is proposed in GOAT for the transformation collapse issue.\n\nOverall, I think the topic of test-time adaptation for distribution shift is significant for the community, and the proposed methods are novel. The experiments verify the effectiveness of GOAT.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the weakness", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "TZXB0sCh4s", "reviewer": "Reviewer_WHEa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths And Weaknesses\n\n## Strengths\n* A major strength is the method's exclusive focus on node feature transformations rather than graph structure changes, which simplifies the adaptation process and avoids the instability typically associated with structural modifications.\n\n* The proposed self-supervised learning strategy is well-designed, leveraging embedding discrepancies between augmented graphs to learn distribution-aware feature transformations, which is both elegant and effective.\n\n* The introduction of a parameter-efficient low-rank adapter to prevent transformation collapse is a valuable contribution. It ensures that each input graph receives a unique, meaningful transformation while preserving the knowledge from the pre-trained model.\n\n* By avoiding graph structure changes and instead focusing on node-level adaptations, the method also enhances interpretability—making it easier to understand the effect of transformations.\n\n* The experimental results are robust and compelling, showing consistent improvements across diverse datasets and backbone architectures, which highlights the generality and practical relevance of the approach.\n\n- The attempt to provide a thorough emprical analysis of various aspects of the propsed method such as the efficieny, interpretability, robustness is very important and attentive.\n\n## Weakness\n\n- The major weakness is the quantitative results presented in table 3 where there are significant number of cases the performance of GOAT falls behind the SoTA. A lack of analysis and explanantion is obvious. Insights and narration of such cases is important for the reader to observe the pros and cons of the latest approaches in an unbiased manner.\n\n- what is the transformation function g  in eq. 5 since the low-rank adaptation is not mentioned yet. It seems g(G) is simply adding the low-rank features to the augmented views. Please elaborate on this aspect and clarify. At the point the audience reaches eq.5 there is no clear view on what g is.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The summary of the contributions of the paper is as follows:\nThis paper introduces GOAT (Graph Optimization via Augmented Transformations), a novel self-supervised test-time tuning paradigm designed to adapt pre-trained graph neural networks (GNNs) to distribution-shifted test data by exclusively leveraging node feature transformations, thereby avoiding the limitations and instability of structure-based methods. To mitigate the issue of transformation collapse, the authors propose a parameter-efficient low-rank adapter that generates diverse, graph-specific transformations, enhancing both adaptation performance and interpretability without altering the underlying graph structure. Extensive experiments on six real-world datasets demonstrate that GOAT consistently improves the performance of various pre-trained GNN backbones, achieving state-of-the-art results under diverse distribution shifts.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Based on the aforementioned strengths and weaknesses, the requested changes are provided in an orderly manner from the highest to the lowest importance as follows:\n\n- There are unreferenced terms such as mapping collapse or transformation collapse. If new to the audience, they must be clearly defined. If known to the community, references and elaborations to the concepts behind these terms are insightful to let the audience get connected to the related literature. \n\n- the details on Fig. 1 right side for the test-time inference phase is missing.\n\n- Elaboration on the effect of the 2 view choice rather than K view choice is insightful. \n\n- Some empirical analysis on the choice of the layer k at which the features for the low-rank adaptor is used is insightful.", "broader_impact_concerns": "There are no \"Broader Impact Concerns\" for this paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)