Original: E MPOWERING G RAPH R EPRESENTATION L EARNING WITH T EST -T IME G RAPH T RANSFORMATION
Revised: E MPOWERING G RAPH R EPRESENTATION L EARNING WITH T EST -T IME G RAPH T RANSFORMATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: As powerful tools for representation learning on graphs, graph neural networks (GNNs) have facilitated various applications from drug discovery to recommender systems.
Revised: As powerful tools for representation learning on graphs, graph neural networks (GNNs) have facilitated various applications from drug discovery to recommender systems.

Original: Nevertheless, the effectiveness of GNNs is immensely challenged by issues related to data quality, such as distribution shift, abnormal features and adversarial attacks.
Revised: Nevertheless, the effectiveness of GNNs is immensely challenged by issues related to data quality, such as distribution shift, abnormal features and adversarial attacks.

Original: Recent efforts have been made on tackling these issues from a modeling perspective which requires additional cost of changing model architectures or re-training model parameters.
Revised: Recent efforts have been made on tackling these issues from a modeling perspective which requires additional cost of changing model architectures or re-training model parameters.

Original: In this work, we provide a data-centric view to tackle these issues and propose a graph transformation framework named
Revised: In this work, we provide a data-centric view to tackle these issues and propose a graph transformation framework named

Original: GT RANS which adapts and refines graph data at test time to achieve better performance.
Revised: GT RANS which adapts and refines graph data at test time to achieve better performance.

Original: We provide theoretical analysis on the design of the framework and discuss why adapting graph data works better than adapting the model.
Revised: We provide theoretical analysis on the design of the framework and discuss why adapting graph data works better than adapting the model.

Original: Extensive experiments have demonstrated the effectiveness of GT RANS on three distinct scenarios for eight benchmark datasets where suboptimal data is presented.
Revised: Extensive experiments have demonstrated the effectiveness of GT RANS on three distinct scenarios for eight benchmark datasets where suboptimal data is presented.

Original: Remarkably, GT RANS performs the best in most cases with improvements up to 2.8%, 8.2% and 3.8% over the best baselines on three experimental settings.
Revised: Remarkably, GT RANS performs the best in most cases with improvements up to 2.8%, 8.2% and 3.8% over the best baselines on three experimental settings.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Graph representation learning has been at the center of various real-world applications, such as drug discovery (Duvenaud et al., 2015),
Revised: Graph representation learning has been at the center of various real-world applications, such as drug discovery (Duvenaud et al., 2015),

Original: recommender systems
Revised: recommender systems

Original: Ying et al., 2018; Sankar et al., 2021), forecasting (Tang et al., 2020; Derrow-Pinion et al., 2021) and outlier detection (Zhao et al., 2021a; Deng & Hooi, 2021).
Revised: Ying et al., 2018; Sankar et al., 2021), forecasting (Tang et al., 2020; Derrow-Pinion et al., 2021) and outlier detection (Zhao et al., 2021a; Deng & Hooi, 2021).

Original: In recent years, there has been a surge of interest in developing graph neural networks (GNNs) as powerful tools for graph representation learning (Kipf & Welling, 2016a;
Revised: In recent years, there has been a surge of interest in developing graph neural networks (GNNs) as powerful tools for graph representation learning (Kipf & Welling, 2016a;

Original: Veli ˇ ckovi ´ c et al., 2018; Hamilton et al., 2017; Wu et al., 2019).
Revised: Veli ˇ ckovi ´ c et al., 2018; Hamilton et al., 2017; Wu et al., 2019).

Original: Remarkably, GNNs have achieved state-of-the-art performance on numerous graph-related tasks including node classification, graph classification and link prediction (Chien et al., 2021;
Revised: Remarkably, GNNs have achieved state-of-the-art performance on numerous graph-related tasks including node classification, graph classification and link prediction (Chien et al., 2021;

Original: You et al., 2021; Kipf & Welling, 2016b)
Revised: You et al., 2021; Kipf & Welling, 2016b).

Original: Despite the enormous success of GNNs, recent studies have revealed that their generalization and robustness are immensely challenged by the data quality (Jin et al., 2021b;
Revised: Despite the enormous success of GNNs, recent studies have revealed that their generalization and robustness are immensely challenged by the data quality (Jin et al., 2021b;

Original: Li et al., 2022).
Revised: Li et al., 2022).

Original: In particular, GNNs can behave unreliably in scenarios where sub-optimal data is presented:
Revised: In particular, GNNs can behave unreliably in scenarios where sub-optimal data is presented:

Original: Distribution shift (Wu et al., 2022a; Zhu et al., 2021a).
Revised: Distribution shift (Wu et al., 2022a; Zhu et al., 2021a).

Original: GNNs tend to yield inferior performance when the distributions of training and test data are not aligned (due to corruption or inconsistent collection procedure of test data).
Revised: GNNs tend to yield inferior performance when the distributions of training and test data are not aligned (due to corruption or inconsistent collection procedure of test data).

Original: Abnormal features (Liu et al., 2021a).
Revised: Abnormal features (Liu et al., 2021a).

Original: GNNs suffer from high classification errors when data contains abnormal features, e.g., incorrect user profile information in social networks.
Revised: GNNs suffer from high classification errors when data contains abnormal features, e.g., incorrect user profile information in social networks.

Original: Adversarial structure attack (Z¨ugner et al., 2018; Li et al., 2021).
Revised: Adversarial structure attack (Z¨ugner et al., 2018; Li et al., 2021).

Original: GNNs are vulnerable to imperceptible perturbations on the graph structure which can lead to severe performance degradation.
Revised: GNNs are vulnerable to imperceptible perturbations on the graph structure which can lead to severe performance degradation.

Original: To tackle these problems, significant efforts have been made on developing new techniques from the modeling perspective , e.g., designing new architectures and employing adversarial training strategies (Xu et al., 2019; Wu et al., 2022a).
Revised: To tackle these problems, significant efforts have been made on developing new techniques from the modeling perspective , e.g., designing new architectures and employing adversarial training strategies (Xu et al., 2019; Wu et al., 2022a).

Original: However, employing these methods in practice may be infeasible, as they require additional cost of changing model architectures or re-training model parameters, especially for well-trained large-scale models.
Revised: However, employing these methods in practice may be infeasible, as they require additional cost of changing model architectures or re-training model parameters, especially for well-trained large-scale models.

Original: The problem is further exacerbated when adopting these techniques for multiple architectures.
Revised: The problem is further exacerbated when adopting these techniques for multiple architectures.

Original: By contrast, this paper seeks to investigate approaches that can be readily used with a wide variety of pre-trained models and test settings for improving model generalization and robustness.
Revised: By contrast, this paper seeks to investigate approaches that can be readily used with a wide variety of pre-trained models and test settings for improving model generalization and robustness.

Original: Essentially, we provide a data-centric perspective
Revised: Essentially, we provide a data-centric perspective

Original: Test Accuracy GCN: 67.4% GAT: 58.6%
Revised: Test Accuracy GCN: 67.4% GAT: 58.6%

Original: APPNP:
Revised: APPNP:

Original: 70.7% AirGNN: 70.8%
Revised: 70.7% AirGNN: 70.8%

Original: Transform [ ⋅ ] [ ⋅ ] [ ⋅ ] [ ⋅ ] [ ⋅ ] [ ⋅ ] [ ⋅ ] [x] [ ⋅ ] [x] [ ⋅ ] [ ⋅ ] [x] [ ⋅ ] [x]
Revised: Transform [ ⋅ ] [ ⋅ ] [ ⋅ ] [ ⋅ ] [ ⋅ ] [ ⋅ ] [ ⋅ ] [x] [ ⋅ ] [x] [ ⋅ ] [ ⋅ ] [x] [ ⋅ ] [x]

Original: Node feature [ ⋅ ] [ ⋅ ]
Revised: Node feature [ ⋅ ] [ ⋅ ]

Original: Test Accuracy GCN: 44.3% GAT: 21.2%
Revised: Test Accuracy GCN: 44.3% GAT: 21.2%

Original: APPNP:
Revised: APPNP:

Original: 48.3% AirGNN: 58.5%
Revised: 48.3% AirGNN: 58.5%

Original: [x]Node connection
Revised: [x]Node connection

Original: Original Graph [ ⋅ ] [ ⋅ ] [x] [ ⋅ ] [x] [ ⋅ ] [ ⋅ ] [x] [ ⋅ ] [ ⋅ ] [ ⋅ ] [x] [ ⋅ ] [x] [ ⋅ ] [ ⋅ ]
Revised: Original Graph [ ⋅ ] [ ⋅ ] [x] [ ⋅ ] [x] [ ⋅ ] [ ⋅ ] [x] [ ⋅ ] [ ⋅ ] [ ⋅ ] [x] [ ⋅ ] [x] [ ⋅ ] [ ⋅ ]

Original: Refined Graph the to address the aforementioned issues by modifying the graph data presented at test-time.
Revised: Refined Graph the to address the aforementioned issues by modifying the graph data presented at test-time.

Original: Such modification aims to bridge the gap between training data and test data, and thus enable GNNs to achieve better generalization and robust performance on the new graph.
Revised: Such modification aims to bridge the gap between training data and test data, and thus enable GNNs to achieve better generalization and robust performance on the new graph.

Original: Figure 1 visually describes this idea: we are originally given with a test graph with abnormal features where multiple GNN architectures yield poor performance; however, by transforming the graph prior to inference (at test-time), we enable these GNNs to achieve much higher accuracy.
Revised: Figure 1 visually describes this idea: we are originally given with a test graph with abnormal features where multiple GNN architectures yield poor performance; however, by transforming the graph prior to inference (at test-time), we enable these GNNs to achieve much higher accuracy.

Original: In this work, we aim to develop a data-centric framework that transforms the test graph to enhance model generalization and robustness, without altering the pre-trained model.
Revised: In this work, we aim to develop a data-centric framework that transforms the test graph to enhance model generalization and robustness, without altering the pre-trained model.

Original: In essence, we are faced with two challenges: (1) how to model and optimize the transformed graph data, and (2) how to formulate an objective that can guide the transformation process.
Revised: In essence, we are faced with two challenges: (1) how to model and optimize the transformed graph data, and (2) how to formulate an objective that can guide the transformation process.

Original: First, we model the graph transformation as injecting perturbation on the node features and graph structure, and optimize them alternatively via gradient descent.
Revised: First, we model the graph transformation as injecting perturbation on the node features and graph structure, and optimize them alternatively via gradient descent.

Original: Second, inspired by the recent progress of contrastive learning, we propose a parameter-free surrogate loss which does not affect the pre-training process while effectively guiding the graph adaptation.
Revised: Second, inspired by the recent progress of contrastive learning, we propose a parameter-free surrogate loss which does not affect the pre-training process while effectively guiding the graph adaptation.

Original: Our contributions can be summarized as follows:
Revised: Our contributions can be summarized as follows:

Original: For the first time, we provide a data-centric perspective to improve the generalization and robustness of GNNs with test-time graph transformation.
Revised: For the first time, we provide a data-centric perspective to improve the generalization and robustness of GNNs with test-time graph transformation.

Original: We establish a novel framework GT RANS for test-time graph transformation by jointly learning the features and adjacency structure to minimize a proposed surrogate loss.
Revised: We establish a novel framework GT RANS for test-time graph transformation by jointly learning the features and adjacency structure to minimize a proposed surrogate loss.

Original: Our theoretical analysis provides insights on what surrogate losses we should use during test-time graph transformation and sheds light on the power of data-adaptation over model-adaptation.
Revised: Our theoretical analysis provides insights on what surrogate losses we should use during test-time graph transformation and sheds light on the power of data-adaptation over model-adaptation.

Original: Extensive experimental results on three settings (distribution shift, abnormal features and adversarial structure attacks) have demonstrated the superiority of test-time graph transformation.
Revised: Extensive experimental results on three settings (distribution shift, abnormal features and adversarial structure attacks) have demonstrated the superiority of test-time graph transformation.

Original: Particularly, GT RANS performs the best in most cases with improvements up to 2.8%, 8.2% and 3.8% over the best baselines on three experimental settings.
Revised: Particularly, GT RANS performs the best in most cases with improvements up to 2.8%, 8.2% and 3.8% over the best baselines on three experimental settings.

Original: Moreover, we note: (1) GT RANS is flexible and versatile.
Revised: Moreover, we note: (1) GT RANS is flexible and versatile.

Original: It can be equipped with any pre-trained
Revised: It can be equipped with any pre-trained

Original: GNNs and the outcome (the refined graph data) can be deployed with any model given its favorable transferability.
Revised: GNNs and the outcome (the refined graph data) can be deployed with any model given its favorable transferability.

Original: (2) GT RANS provides a degree of interpretability, as it can show which kinds of graph modifications can help improve performance by visualizing the data.
Revised: (2) GT RANS provides a degree of interpretability, as it can show which kinds of graph modifications can help improve performance by visualizing the data.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Distribution shift in GNNs.
Revised: Distribution shift in GNNs.

Original: GNNs have revolutionized graph representation learning and achieved state-of-the-art results on diverse graph-related tasks (Kipf & Welling, 2016a; Veliˇckovi´c et al., 2018; Hamilton et al., 2017; Chien et al., 2021; Klicpera et al., 2018; Wu et al., 2019).
Revised: GNNs have revolutionized graph representation learning and achieved state-of-the-art results on diverse graph-related tasks (Kipf & Welling, 2016a; Veliˇckovi´c et al., 2018; Hamilton et al., 2017; Chien et al., 2021; Klicpera et al., 2018; Wu et al., 2019).

Original: However, recent studies have demonstrated that GNNs yield sub-optimal performance on out-of-distribution data for node classification (Zhu et al., 2021a;
Revised: However, recent studies have demonstrated that GNNs yield sub-optimal performance on out-of-distribution data for node classification (Zhu et al., 2021a;

Original: Wu et al., 2022a; Liu et al., 2022a) and graph classification (Chen et al., 2022; Buffelli et al., 2022; Gui et al., 2022; Wu et al., 2022b).
Revised: Wu et al., 2022a; Liu et al., 2022a) and graph classification (Chen et al., 2022; Buffelli et al., 2022; Gui et al., 2022; Wu et al., 2022b).

Original: These studies have introduced solutions to tackle distribution shifts.
Revised: These studies have introduced solutions to tackle distribution shifts.

Original: For example, EERM (Wu et al., 2022a) attributes the cause of distribution shifts to an unknown environmental variable, and trains GNNs under multiple environments produced by a generator.
Revised: For example, EERM (Wu et al., 2022a) attributes the cause of distribution shifts to an unknown environmental variable, and trains GNNs under multiple environments produced by a generator.

Original: For a thorough review, we refer the readers to a recent survey (Li et al., 2022).
Revised: For a thorough review, we refer the readers to a recent survey (Li et al., 2022).

Original: Unlike existing works, we target modifying the inputs via test-time adaption.
Revised: Unlike existing works, we target modifying the inputs via test-time adaption.

Original: Robustness of GNNs.
Revised: Robustness of GNNs.

Original: Recent studies have demonstrated the vulnerability of GNNs to graph adversarial attacks (Z¨ugner et al., 2018; Z¨ugner & G¨unnemann, 2019;
Revised: Recent studies have demonstrated the vulnerability of GNNs to graph adversarial attacks (Z¨ugner et al., 2018; Z¨ugner & G¨unnemann, 2019;

Original: Xu et al., 2019; Geisler et al., 2021), i.e., small perturbations on the input graph can mislead GNNs into making wrong predictions.
Revised: Xu et al., 2019; Geisler et al., 2021), i.e., small perturbations on the input graph can mislead GNNs into making wrong predictions.

Original: Several works make efforts towards developing new GNNs or adversarial training strategies to defend against attacks (Xu et al., 2019; Zhu et al., 2019; Jin et al., 2021a;b).
Revised: Several works make efforts towards developing new GNNs or adversarial training strategies to defend against attacks (Xu et al., 2019; Zhu et al., 2019; Jin et al., 2021a;b).

Original: Instead of altering model training behavior, our work aims to modify the test graph to correct adversarial patterns.
Revised: Instead of altering model training behavior, our work aims to modify the test graph to correct adversarial patterns.

Original: Graph Structure Learning & Graph Data Augmentation.
Revised: Graph Structure Learning & Graph Data Augmentation.

Original: Graph structure learning and graph data augmentation both aim to improve GNNs’ generalization performance by augmenting the (training) graph data, either learning the graph from scratch (Franceschi et al., 2019; 
Revised: Graph structure learning and graph data augmentation both aim to improve GNNs’ generalization performance by augmenting the (training) graph data, either learning the graph from scratch (Franceschi et al., 2019; Jin et al., 2020;

Original: Jin et al., 2020; Chen et al., 2020;
Revised: Chen et al., 2020; Zhao et al., 2021b;

Original: Zhao et al., 2021b) or perturbing the graph in a rule-based way (Rong et al., 2020;
Revised: Rozemberczki et al., 2021b; Halcrow et al., 2020; Fatemi et al., 2021) or perturbing the graph in a rule-based way (Rong et al., 2020;

Original: Feng et al., 2020; Ding et al., 2022).
Revised: Feng et al., 2020; Ding et al., 2022).

Original: While our work also modifies the graph data, we focus on modifying the test data and not impacting the model training process.
Revised: While our work also modifies the graph data, we focus on modifying the test data and not impacting the model training process.

Original: Test-time Training.
Revised: Test-time Training.

Original: Our work is also related to test-time training (Sun et al., 2020; Wang et al., 2021; Liu et al., 2021b; Zhang et al., 2021; 2022), which has raised a surge of interest in computer vision recently.
Revised: Our work is also related to test-time training (Sun et al., 2020; Wang et al., 2021; Liu et al., 2021b; Zhang et al., 2021; 2022), which has raised a surge of interest in computer vision recently.

Original: To handle out-of-distribution data, Sun et al.
Revised: To handle out-of-distribution data, Sun et al.

Original: (2020) propose the pioneer work of test-time training (TTT) by optimizing feature extractor via an auxiliary task loss.
Revised: (2020) propose the pioneer work of test-time training (TTT) by optimizing feature extractor via an auxiliary task loss.

Original: However, TTT alters training to jointly optimize the supervised loss and auxiliary task loss.
Revised: However, TTT alters training to jointly optimize the supervised loss and auxiliary task loss.

Original: To remove the need for training an auxiliary task, Tent (Wang et al., 2021) proposes to minimize the prediction entropy at test-time.
Revised: To remove the need for training an auxiliary task, Tent (Wang et al., 2021) proposes to minimize the prediction entropy at test-time.

Original: It is worth noting that Tent works by adapting the statistics and parameters in batch normalization (BN) layers, which may not always be employed by modern GNNs.
Revised: It is worth noting that Tent works by adapting the statistics and parameters in batch normalization (BN) layers, which may not always be employed by modern GNNs.

Original: In this work, we focus on a novel perspective of adapting the test graph data instead of the model, which makes no assumptions about the particular training procedure or architecture.
Revised: In this work, we focus on a novel perspective of adapting the test graph data instead of the model, which makes no assumptions about the particular training procedure or architecture.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: We start by introducing the general problem of test-time graph transformation (TTGT).
Revised: We start by introducing the general problem of test-time graph transformation (TTGT).

Original: While our discussion mainly focuses on the node classification task where the goal is to predict the labels of nodes in the graph, it can be easily extended to other tasks.
Revised: While our discussion mainly focuses on the node classification task where the goal is to predict the labels of nodes in the graph, it can be easily extended to other tasks.

Original: Consider that we have a training graph
Revised: Consider that we have a training graph

Original: G Tr and a test graph G Te , and the corresponding set of node labels are denoted as Y Tr and Y Te , respectively.
Revised: G Tr and a test graph G Te , and the corresponding set of node labels are denoted as Y Tr and Y Te , respectively.

Original: Note that the node sets in G Tr and G Te can either be disjoint or overlapping, and they are not necessarily drawn from the same distribution.
Revised: Note that the node sets in G Tr and G Te can either be disjoint or overlapping, and they are not necessarily drawn from the same distribution.

Original: Further, let f θ ( · ) denote the mapping function of a GNN model parameterized by θ , which maps a graph into the space of node labels.
Revised: Further, let f θ ( · ) denote the mapping function of a GNN model parameterized by θ , which maps a graph into the space of node labels.

Original: Definition 1 (Test-Time Graph Transformation (TTGT)) .
Revised: Definition 1 (Test-Time Graph Transformation (TTGT)) .

Original: TTGT requires to learn a graph transformation function g ( · ) which refines the test graph G Te such that the pre-trained f θ can yield better test performance on g ( G Te ) than that on G Te :
Revised: TTGT requires to learn a graph transformation function g ( · ) which refines the test graph G Te such that the pre-trained f θ can yield better test performance on g ( G Te ) than that on G Te :

Original: [Equation]
Revised: [Equation]

Original: where L denotes the loss function measuring downstream performance; P ( G Te ) is the space of the modified graph, e.g., we may constrain the change on the graph to be small.
Revised: where L denotes the loss function measuring downstream performance; P ( G Te ) is the space of the modified graph, e.g., we may constrain the change on the graph to be small.

Original: To optimize the TTGT problem, we are faced with two critical challenges: (1) how to parameterize and optimize the graph transformation function g ( · ) ; and (2) how to formulate a surrogate loss to guide the learning process, since we do not have access to the ground-truth labels of test nodes.
Revised: To optimize the TTGT problem, we are faced with two critical challenges: (1) how to parameterize and optimize the graph transformation function g ( · ) ; and (2) how to formulate a surrogate loss to guide the learning process, since we do not have access to the ground-truth labels of test nodes.

Original: Therefore, we propose GT RANS and elaborate on how it addresses these challenges as follows.
Revised: Therefore, we propose GT RANS and elaborate on how it addresses these challenges as follows.

Original: 3. C ONSTRUCTING G RAPH T RANSFORMATION
Revised: 3. C ONSTRUCTING G RAPH T RANSFORMATION

Original: In this subsection, we introduce how to construct the graph transformation function and detail its optimization process.
Revised: In this subsection, we introduce how to construct the graph transformation function and detail its optimization process.

Original: Let G Te =
Revised: Let G Te =

Original: { A , X } denote the test graph, where A ∈ { 0 , 1 } N × N is the adjacency matrix, N is the number of nodes, and X ∈ R N ×
Revised: { A , X } denote the test graph, where A ∈ { 0 , 1 } N × N is the adjacency matrix, N is the number of nodes, and X ∈ R N ×

Original: d is the d -dimensional node feature matrix.
Revised: d is the d -dimensional node feature matrix.

Original: Since the pre-trained GNN parameters are fixed at test time and we only care about the test graph, we drop the subscript in G Te and Y Te to simplify notations in the rest of the paper.
Revised: Since the pre-trained GNN parameters are fixed at test time and we only care about the test graph, we drop the subscript in G Te and Y Te to simplify notations in the rest of the paper.

Original: Construction.
Revised: Construction.

Original: We are interested in obtaining the transformed test graph G ′
Revised: We are interested in obtaining the transformed test graph G ′

Original: Te = g
Revised: Te = g

Original: ( A , X ) =
Revised: ( A , X ) =

Original: ( A ′ , X ′ ) .
Revised: ( A ′ , X ′ ) .

Original: Specifically, we model feature modification as an additive function which injects perturbation to node features, i.e., X ′ =
Revised: Specifically, we model feature modification as an additive function which injects perturbation to node features, i.e., X ′ =

Original: X +∆ X ; we model the structure modification as A ′ =
Revised: X +∆ X ; we model the structure modification as A ′ =

Original: A ⊕ ∆ A 1 , where ⊕ stands for an element-wise exclusive OR operation and ∆ A ∈ { 0 , 1 } N × N is a binary matrix.
Revised: A ⊕ ∆ A 1 , where ⊕ stands for an element-wise exclusive OR operation and ∆ A ∈ { 0 , 1 } N × N is a binary matrix.

Original: In other words, ∆ A  ij = 1 indicates an edge flip.
Revised: In other words, (∆ A ) ij = 1 indicates an edge flip.

Original: Formally, we seek to find ∆ A and ∆ X that can minimize the objective function: 
Revised: Formally, we seek to find ∆ A and ∆ X that can minimize the objective function: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where ∆ A ∈ { 0 , 1 } N × N and ∆ X ∈ R N × d are treated as free parameters.
Revised: where ∆ A ∈ { 0 , 1 } N × N and ∆ X ∈ R N × d are treated as free parameters.

Original: Further, to ensure we do not heavily violate the original graph structure, we constrain the number of changed entries in the adjacency matrix to be smaller than a budget B on the graph structure, i.e., ∥ ∆
Revised: Further, to ensure we do not heavily violate the original graph structure, we constrain the number of changed entries in the adjacency matrix to be smaller than a budget B on the graph structure, i.e., ∥ ∆

Original: A ∥ 2 ≤ B .
Revised: A ∥ 2 ≤ B .

Original: We do not impose constraints on the node features to ease optimization.
Revised: We do not impose constraints on the node features to ease optimization.

Original: In this context, P can be viewed as constraining ∆ A to a binary space as well as restricting the number of changes.
Revised: In this context, P can be viewed as constraining ∆ A to a binary space as well as restricting the number of changes.

Original: Optimization.
Revised: Optimization.

Original: Jointly optimizing ∆ X and ∆ A is often challenging as they depend on each other.
Revised: Jointly optimizing ∆ X and ∆ A is often challenging as they depend on each other.

Original: In practice, we alternatively optimize ∆ X and ∆ A .
Revised: In practice, we alternatively optimize ∆ X and ∆ A .

Original: Notably, the optimization for ∆ X is easy since the node features are continuous.
Revised: Notably, the optimization for ∆ X is easy since the node features are continuous.

Original: The optimization for ∆ A is particularly difficult in that (1) ∆ A is binary and constrained; and (2) the search space of N 2 entries is too large especially when we are learning on large-scale graphs.
Revised: The optimization for ∆ A is particularly difficult in that (1) ∆ A is binary and constrained; and (2) the search space of N 2 entries is too large especially when we are learning on large-scale graphs.

Original: To cope with the first challenge, we relax the binary space to [0 , 1] N × N
Revised: To cope with the first challenge, we relax the binary space to [0 , 1] N × N

Original: and then we can employ projected gradient descent (PGD)
Revised: and then we can employ projected gradient descent (PGD)

Original: (Xu et al., 2019; Geisler et al., 2021) to update ∆ A :
Revised: (Xu et al., 2019; Geisler et al., 2021) to update ∆ A :

Original: [Equation]
Revised: [Equation]

Original: where we first perform gradient descent with step size η and call a projection Π P to project the variable to the space P .
Revised: where we first perform gradient descent with step size η and call a projection Π P to project the variable to the space P .

Original: Specifically, given an input vector p , Π P [Equation] · ) is expressed as:
Revised: Specifically, given an input vector p , Π P ( · ) is expressed as:

Original: [Equation]
Revised: [Equation]

Original: If
Revised: If

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Π[
Revised: Π

Original: [Equation]
Revised: [Equation]

Original: otherwise
Revised: otherwise

Original: [Equation]
Revised: [Equation]

Original: where Π [0 .
Revised: where Π [0 .

Original: 1] ( · ) clamps the input values to [0 , 1] , 1 stands for a vector of all ones, and γ is obtained by solving the equation 1 ⊤ Π [0 , 1] ( p − γ 1 )
Revised: 1] ( · ) clamps the input values to [0 , 1] , 1 stands for a vector of all ones, and γ is obtained by solving the equation 1 ⊤ Π [0 , 1] ( p − γ 1 )

Original: =
Revised: =

Original: B with the bisection method (Liu et al., 2015).
Revised: B with the bisection method (Liu et al., 2015).

Original: To keep the adjacency structure discrete and sparse, we view each entry in A ⊕ ∆
Revised: To keep the adjacency structure discrete and sparse, we view each entry in A ⊕ ∆

Original: A as a Bernoulli distribution and sample the learned graph as A ′ ∼ Bernoulli
Revised: A as a Bernoulli distribution and sample the learned graph as A ′ ∼ Bernoulli

Original: ( A ⊕ ∆ A ) .
Revised: ( A ⊕ ∆ A ) .

Original: Furthermore, to enable efficient graph structure learning, it is desired to reduce the search space of updated adjacency matrix.
Revised: Furthermore, to enable efficient graph structure learning, it is desired to reduce the search space of updated adjacency matrix.

Original: One recent approach of graph adversarial attack (Geisler et al., 2021) proposes to sample a small block of possible entries from the adjacency matrix and update them at each iteration.
Revised: One recent approach of graph adversarial attack (Geisler et al., 2021) proposes to sample a small block of possible entries from the adjacency matrix and update them at each iteration.

Original: This solution is still computationally intensive as it requires hundreds of steps to achieve a good performance.
Revised: This solution is still computationally intensive as it requires hundreds of steps to achieve a good performance.

Original: Instead, we constrain the search space to only the existing edges of the graph, which is typically sparse.
Revised: Instead, we constrain the search space to only the existing edges of the graph, which is typically sparse.

Original: Empirically, we observe that this simpler strategy still learns useful structure information when combined with feature modification.
Revised: Empirically, we observe that this simpler strategy still learns useful structure information when combined with feature modification.

Original: 3. P ARAMETER -F REE S URROGATE L OSS
Revised: 3. P ARAMETER -F REE S URROGATE L OSS

Original: As discussed earlier, the proposed framework GT RANS aims to improve the model generalization and robustness by learning to transform the test graph.
Revised: As discussed earlier, the proposed framework GT RANS aims to improve the model generalization and robustness by learning to transform the test graph.

Original: Ideally, when we have test ground-truth labels, the problem can be readily solved by adapting the graph to result in the minimum cross entropy loss on test samples.
Revised: Ideally, when we have test ground-truth labels, the problem can be readily solved by adapting the graph to result in the minimum cross entropy loss on test samples.

Original: However, as we do not have such information at test-time, it motivates us to investigate feasible surrogate losses to guide the graph transformation process.
Revised: However, as we do not have such information at test-time, it motivates us to investigate feasible surrogate losses to guide the graph transformation process.

Original: In the absence of labeled data, recently emerging self-supervised learning techniques (Xie et al., 2021; Liu et al., 2022b) have paved the way for providing self-supervision for TTGT.
Revised: In the absence of labeled data, recently emerging self-supervised learning techniques (Xie et al., 2021; Liu et al., 2022b) have paved the way for providing self-supervision for TTGT.

Original: However, not every surrogate self-supervised task and loss is suitable for our transformation process, as some tasks are more powerful and some are weaker.
Revised: However, not every surrogate self-supervised task and loss is suitable for our transformation process, as some tasks are more powerful and some are weaker.

Original: To choose a suitable surrogate loss, we provide the following theorem.
Revised: To choose a suitable surrogate loss, we provide the following theorem.

Original: Theorem 1.
Revised: Theorem 1.

Original: Let L c denote the classification loss and L s denote the surrogate loss, respectively.
Revised: Let L c denote the classification loss and L s denote the surrogate loss, respectively.

Original: Let ρ ( G ) denote the correlation between ∇ G L c ( G, Y ) and ∇ G L s ( G ) , and let ϵ denote the learning rate for gradient descent.
Revised: Let ρ ( G ) denote the correlation between ∇ G L c ( G, Y ) and ∇ G L s ( G ) , and let ϵ denote the learning rate for gradient descent.

Original: Assume that L c is twice-differentiable and its Hessian matrix satisfies ∥ H ( G, Y )
Revised: Assume that L c is twice-differentiable and its Hessian matrix satisfies ∥ H ( G, Y )

Original: ∥ 2 ≤ M for all G .
Revised: ∥ 2 ≤ M for all G .

Original: When ρ ( G ) > 0 and ϵ < 2 ρ (
Revised: When ρ ( G ) > 0 and ϵ < 2 ρ (

Original: [Equation]
Revised: [Equation]

Original: , we have
Revised: , we have

Original: [Equation]
Revised: [Equation]

Original:  The proof can be found in Appendix A.1.
Revised: [Figure] The proof can be found in Appendix A.1.

Original: Theorem 1 suggests that when the gradients from classification loss and surrogate loss have a positive correlation, i.e., ρ ( G ) > 0 , we can update the test graph by performing gradient descent with a sufficiently small learning rate such that the classification loss on the test samples is reduced.
Revised: Theorem 1 suggests that when the gradients from classification loss and surrogate loss have a positive correlation, i.e., ρ ( G ) > 0 , we can update the test graph by performing gradient descent with a sufficiently small learning rate such that the classification loss on the test samples is reduced.

Original: Hence, it is imperative to find a surrogate task that shares relevant information with the classification task. 
Revised: Hence, it is imperative to find a surrogate task that shares relevant information with the classification task. To empirically verify the effectiveness of

Revised: Theorem 1, we adopt the surrogate loss in Equation (6) as L s and plot the values of ρ ( G ) and L c on one test graph in Cora in Figure 2.

Revised: We can observe that a positive ρ ( G ) generally reduces the test classifica- tion loss.

Revised: More results on different surrogate losses can be found in Figure 9 in Appendix D.9 and similar patterns are exhibited.

Original: Parameter-Free Surrogate Loss.
Revised: Parameter-Free Surrogate Loss.

Original: As one popular self-supervised paradigm, graph contrastive learning has achieved promising performance in various graph-related tasks (Hassani & Khasahmadi, 2020; You et al., 2021; Zhu et al., 2021b), which indicates that graph contrastive learning tasks are often highly correlated with downstream tasks.
Revised: As one popular self-supervised paradigm, graph contrastive learning has achieved promising performance in various graph-related tasks (Hassani & Khasahmadi, 2020; You et al., 2021; Zhu et al., 2021c), which indicates that graph contrastive learning tasks are often highly correlated with downstream tasks.

Original: This property is desirable for guiding TTGT as suggested by Theorem 1.
Revised: This property is desirable for guiding TTGT as suggested by Theorem 1.

Original: At its core lies the contrasting scheme, where the similarity between two augmented views from the same sample is maximized, while the similarity between views from two different samples is minimized.
Revised: At its core lies the contrasting scheme, where the similarity between two augmented views from the same sample is maximized, while the similarity between views from two different samples is minimized.

Original: However,  existing graph contrastive learning methods cannot be directly applied to our scenario, as they requires a parameterized projection head to map augmented representations to another latent space, which inevitably alters the model architecture.
Revised: However, the majority of existing graph contrastive learning methods cannot be directly applied to our scenario, as they often require a parameterized projection head to map augmented representations to another latent space, which inevitably alters the model architecture.

Original: To address this issue, we propose a parameterfree surrogate loss.
Revised: To address this issue, we propose a parameter-free surrogate loss.

Original: Specifically, we apply an augmentation function A ( · ) on input graph G and obtain an augmented graph A ( G ) .
Revised: Specifically, we apply an augmentation function A ( · ) on input graph G and obtain an augmented graph A ( G ) .

Original: The node representations obtained from the two graphs are denoted as Z and ˆ Z , respectively; z i and ˆ z i stand for the i -th node representation taken from Z and ˆ Z , respectively.
Revised: The node representations obtained from the two graphs are denoted as Z and ˆ Z , respectively; z i and ˆ z i stand for the i -th node representation taken from Z and ˆ Z , respectively.

Original: We adopt DropEdge (Rong et al., 2020) as the augmentation function A ( · ) , and the hidden presentations z i , ˆ z i, ˜ z i are taken from the output of the second last layer of the pre-trained model.
Revised: We adopt DropEdge (Rong et al., 2020) as the augmentation function A ( · ) , and the hidden presentations z i and ˆ z i are taken from the output of the second last layer of the pre-trained model.

Original: Essentially, we maximize the cosine similarity between original nodes and their augmented view while penalizing the similarity between the nodes and their negative samples:
Revised: Essentially, we maximize the cosine similarity between original nodes and their augmented view while penalizing the similarity between the nodes and their negative samples:

Original: [Equation]
Revised: [Equation]

Original: where { ˜ z i | i = 1 , . .
Revised: where { ˜ z i | i = 1 , . .

Original: , N } are the negative samples for corresponding nodes, which are generated by shuffling node features (Velickovic et al., 2019).
Revised: , N } are the negative samples for corresponding nodes, which are generated by shuffling node features (Velickovic et al., 2019).

Original: In Eq.
Revised: In Eq.

Original: (6), the first term encourages each node to be close while  second term pushes each node away from the corresponding negative sample.
Revised: (6), the first term encourages each node to be close while the second term pushes each node away from the corresponding negative sample.

Original: Note that (1) L s is parameter-free and does not require modification of the model architecture, or affect the pre-training process; (2) there could be other self-supervised signals for guiding the graph transformation, and we empirically compare them with the proposed surrogate loss in Appendix D.6.
Revised: Note that (1) L s is parameter-free and does not require modification of the model architecture, or affect the pre-training process; (2) there could be other self-supervised signals for guiding the graph transformation, and we empirically compare them with the contrastive loss in Appendix D.9. We also highlight that our unique contribution is not the loss in Eq.

Revised: (6) but the proposed TTGT framework and the theoretical and empirical insights on how to choose a suitable surrogate loss.

Original: Furthermore, the algorithm of GT RANS is provided in Appendix B.
Revised: Furthermore, the algorithm of GT RANS is provided in Appendix B.

Original: 3. F URTHER A NALYSIS
Revised: 3. F URTHER A NALYSIS

Original: In this subsection, we study the theoretical property of the proposed surrogate loss and compare the strategy of adapting data versus that of adapting model.
Revised: In this subsection, we study the theoretical property of the proposed surrogate loss and compare the strategy of adapting data versus that of adapting model.

Original: We first demonstrate the rationality of the proposed surrogate loss through the following theorem.
Revised: We first demonstrate the rationality of the proposed surrogate loss through the following theorem.

Original: Theorem 2.
Revised: Theorem 2.

Original: Assume that the augmentation function A ( · ) generates a data view of the same class for the test nodes and the node classes are balanced.
Revised: Assume that the augmentation function A ( · ) generates a data view of the same class for the test nodes and the node classes are balanced.

Original: Assume for each class, the mean of the representations obtained from Z and ˆ Z are the same.
Revised: Assume for each class, the mean of the representations obtained from Z and ˆ Z are the same.

Original: Minimizing the first term in Eq.
Revised: Minimizing the first term in Eq.

Original: (6) is approximately minimizing the class-conditional entropy H ( Z | Y ) between features Z and labels Y .
Revised: (6) is approximately minimizing the class-conditional entropy H ( Z | Y ) between features Z and labels Y .

Original: The proof can be found in Appendix A.2.
Revised: The proof can be found in Appendix A.2.

Original: Theorem 2 indicates that minimizing the first term in
Revised: Theorem 2 indicates that minimizing the first term in

Original: Eq.
Revised: Eq.

Original: (6) will  minimize H ( Z | Y ) , which encourages high intra-class compactness, i.e., learning a lowentropy cluster in the embedded space for each class. 
Revised: (6) will approximately minimize H ( Z | Y ) , which encourages high intra-class compactness, i.e., learning a low-entropy cluster in the embedded space for each class. Notably, H ( Z | Y ) can be rewritten as H ( Z | Y ) = H ( Z ) − I ( Z, Y ) .

Original:  However, we note that only optimizing this term can result in collapse (mapping all data points to a single point in the embedded space), which stresses the necessity of the second term in Eq.
Revised: It indicates that minimizing Eq. (6) can also help promote I ( Z, Y ) , the mutual information between the hidden representation and downstream class. However, we note that only optimizing this term can result in collapse (mapping all data points to a single point in the embedded space), which stresses the necessity of the second term in Eq.

Original: Next, we use an illustrative example to show that adapting data at test-time can be more useful than adapting model in some cases.
Revised: Next, we use an illustrative example to show that adapting data at test-time can be more useful than adapting model in some cases.

Original: Given test samples { x i | i = 1 , . . .
Revised: Given test samples { x i | i = 1 , . . .

Original: , K } , we consider a linearized GNN f θ which first performs aggregation through a function Agg ( · , · ) and then transforms the aggregated features via a function Trans ( · ) .
Revised: , K } , we consider a linearized GNN f θ which first performs aggregation through a function Agg ( · , · ) and then transforms the aggregated features via a function Trans ( · ) .

Original: Hence, only the function Trans ( · ) is parameterized by θ .
Revised: Hence, only the function Trans ( · ) is parameterized by θ .

Original: Example.
Revised: Example.

Original: Let N i denote the neighbors for node x i .
Revised: Let N i denote the neighbors for node x i .

Original: If there exist two nodes with the same aggregated features but different labels, i.e., Agg ( x 1 , { x i | i ∈ N 1 } )
Revised: If there exist two nodes with the same aggregated features but different labels, i.e., Agg ( x 1 , { x i | i ∈ N 1 } )

Original: =
Revised: =

Original: Agg ( x 2 , { x j | j ∈ N 2 } ) , y 1 ̸ = y 2 , adapting the data { x i | i = 1 , . .
Revised: Agg ( x 2 , { x j | j ∈ N 2 } ) , y 1 ̸ = y 2 , adapting the data { x i | i = 1 , . .

Original: , K } can achieve lower classification error than adapting the model f
Revised: , K } can achieve lower classification error than adapting the model f

Original: θ at test stage.
Revised: θ at test stage.

Original: Illustration.
Revised: Illustration.

Original: Let ¯
Revised: Let ¯

Original: x
Revised: x

Original: =
Revised: =

Original: Agg ( x 1 , { x i | i ∈ N 1 } ) and ¯ x
Revised: Agg ( x 1 , { x i | i ∈ N 1 } ) and ¯ x

Original: 2 = Agg ( x 2 , { x j | j ∈ N 2 } ) .
Revised: 2 = Agg ( x 2 , { x j | j ∈ N 2 } ) .

Original: For simplicity, we consider the following mean square loss as the classification error:
Revised: For simplicity, we consider the following mean square loss as the classification error:

Original: [Equation]
Revised: [Equation]

Original: It is easy to see that ℓ reaches its minimum when Trans (¯ x 1 ) =
Revised: It is easy to see that ℓ reaches its minimum when Trans (¯ x 1 ) =

Original: y 1 and Trans (¯ x 2 )
Revised: y 1 and Trans (¯ x 2 )

Original: =
Revised: =

Original: y 2 .
Revised: y 2 .

Original: In this context, it is impossible to find θ such that Trans ( · ) can map x 1 , x 2 to different labels since it is not a one-to-many function.
Revised: In this context, it is impossible to find θ such that Trans ( · ) can map x 1 , x 2 to different labels since it is not a one-to-many function.

Original: However, since y 1 and y 2 are in the label space of training data, we can always modify the test graph to obtain newly aggregated features ¯ x ′ 1 , ¯ x ′ 2 such that Trans (¯ x ′ 1 )
Revised: However, since y 1 and y 2 are in the label space of training data, we can always modify the test graph to obtain newly aggregated features ¯ x ′ 1 , ¯ x ′ 2 such that Trans (¯ x ′ 1 )

Original: = yand
Revised: = yand

Original: Trans ( ¯ x ′ 2 )
Revised: Trans (¯ x ′ 2 )

Original: =
Revised: =

Original: y 2 , which minimizes ℓ .
Revised: y 2 , which minimizes ℓ .

Original: In the extreme case, we may drop all node connections for the two nodes, and let x 1 ← ¯ x ′ 1 and x 2 ← ¯ x ′ 2 where ¯ x ′ 1 and ¯ x ′ 2 are the aggregated features taken from the training set.
Revised: In the extreme case, we may drop all node connections for the two nodes, and let x 1 ← ¯ x ′ 1 and x 2 ← ¯ x ′ 2 where ¯ x ′ 1 and ¯ x ′ 2 are the aggregated features taken from the training set.

Original: Hence, adapting data can achieve lower classification loss.
Revised: Hence, adapting data can achieve lower classification loss.

Original: Remark 1.
Revised: Remark 1.

Original: Note that the existence of two nodes with the same aggregated features but different labels is not rare when considering adversarial attack or abnormal features.
Revised: Note that the existence of two nodes with the same aggregated features but different labels is not rare when considering adversarial attack or abnormal features.

Original: We provide a figurative example in Figure 5 in Appendix A.3: the attacker injects one adversarial edge into the graph and changes the aggregated features ¯ x 1 and ¯ x 2 to be the same.
Revised: We provide a figurative example in Figure 6 in Appendix A.3: the attacker injects one adversarial edge into the graph and changes the aggregated features ¯ x 1 and ¯ x 2 to be the same.

Original: Remark 2.
Revised: Remark 2.

Original: When we consider ¯ x 1 ̸ = ¯ x 2 , y 1 ̸
Revised: When we consider ¯ x 1 ̸ = ¯ x 2 , y 1 ̸

Original: = y 2 , whether we can find θ satisfying Trans (¯ x 1 ) =
Revised: = y 2 , whether we can find θ satisfying Trans (¯ x 1 ) =

Original: yand
Revised: yand

Original: Trans (¯ x 2 ) =
Revised: Trans (¯ x 2 ) =

Original: y 2 depends on the expressiveness of the the transformation function.
Revised: y 2 depends on the expressiveness of the the transformation function.

Original: If it is not powerful enough (e.g., an under-parameterized neural network), it could fail to map different data points to different labels.
Revised: If it is not powerful enough (e.g., an under-parameterized neural network), it could fail to map different data points to different labels.

Original: On the contrary, adapting the data does not suffer this problem as we can always modify the test graph to satisfy Trans (¯ x ′ 1 ) =
Revised: On the contrary, adapting the data does not suffer this problem as we can always modify the test graph to satisfy Trans (¯ x ′ 1 ) =

Original: y 1 and Trans (¯ x ′ 2 )
Revised: y 1 and Trans (¯ x ′ 2 )

Original: =
Revised: =

Original: y 2 .
Revised: y 2 .

Original: Remark 3.
Revised: Remark 3.

Original: The above discussion can be easily extended to nonlinear GNN by considering ¯ x 1 , ¯ xas the output before the last linear layer of GNN.
Revised: The above discussion can be easily extended to nonlinear GNN by considering ¯ x 1 , ¯ xas the output before the last linear layer of GNN.

Original:  E XPERIMENT 4. G ENERALIZATION ON O UT - OF - DISTRIBUTION D ATA
Revised:  E XPERIMENT 4. G ENERALIZATION ON O UT - OF - DISTRIBUTION D ATA

Original: Setup.
Revised: Setup.

Original: Following the settings in EERM (Wu et al., 2022a), which is designed for node-level tasks on
Revised: Following the settings in EERM (Wu et al., 2022a), which is designed for node-level tasks on

Original: OOD data, we validate GT RANS on three types of distribution shifts with six benchmark datasets: (1) artificial transformation for Cora (Yang et al., 2016) and
Revised: OOD data, we validate GT RANS on three types of distribution shifts with six benchmark datasets: (1) artificial transformation for Cora (Yang et al., 2016) and

Original: Amazon-Photo
Revised: Amazon-Photo

Original: (Shchur et al., 2018), (2) cross-domain transfers for Twitch-E and FB-

Revised: (Shchur et al., 2018), (2) cross-domain transfers for Twitch-E and FB-

Original: (Rozemberczki et al., 2021)
Revised: (Rozemberczki et al., 2021a)

Original: (Lim et al., 2021), and (3) temporal evolution for Elliptic (Pareja et al., 2020) and OGB-Arxiv
Revised: (Lim et al., 2021), and (3) temporal evolution for Elliptic (Pareja et al., 2020) and OGB-Arxiv

Original: (Hu et al., 2020).
Revised: (Hu et al., 2020).

Original: Moreoever,
Revised: Moreoever,

Original: Cora and Amazon-Photo have 1/1/8 graphs for training/validation/test sets.
Revised: Cora and Amazon-Photo have 1/1/8 graphs for training/validation/test sets.

Original: The splits are 1/1/5 on
Revised: The splits are 1/1/5 on

Original: Twitch-E , 3/2/3 on FB-100 , 5/5/33 on Elliptic , and 1/1/3 on OGB-Arxiv .
Revised: Twitch-E , 3/2/3 on FB-100 , 5/5/33 on Elliptic , and 1/1/3 on OGB-Arxiv .

Original: More details on the datasets are provided in Appendix C.
Revised: More details on the datasets are provided in Appendix C.

Original: We compare GT RANS with four baselines: empirical risk minimization (ERM, i.e., standard training), data augmentation technique DropEdge (Rong et al., 2020), test-timetraining method Tent (Wang et al., 2021), and the recent SOTA method EERM (Wu et al., 2022a) which is exclusively developed for graph OOD issue.
Revised: We compare GT RANS with four baselines: empirical risk minimization (ERM, i.e., standard training), data augmentation technique DropEdge (Rong et al., 2020), test-timetraining method Tent (Wang et al., 2021), and the recent SOTA method EERM (Wu et al., 2022a) which is exclusively developed for graph OOD issue.

Original: Note that SR-GNN (Zhu et al., 2021a) is not included as a baseline because it specifically targets distribution shifts between the selection of training and testing nodes instead of the general OOD issue.
Revised: Note that SR-GNN (Zhu et al., 2021a) is not included as a baseline because it specifically targets distribution shifts between the selection of training and testing nodes instead of the general OOD issue.

Original: By contrast, both Tent and EERM are designed to handle general distribution shifts.
Revised: By contrast, both Tent and EERM are designed to handle general distribution shifts.

Original: Further, we evaluate all the methods with four popular GNN backbones including GCN (Kipf & Welling, 2016a), GraphSAGE (Hamilton et al., 2017), GAT (Veli ˇ ckovi ´ c et al., 2018), and GPR (Chien et al., 2021).
Revised: Further, we evaluate all the methods with four popular GNN backbones including GCN (Kipf & Welling, 2016a), GraphSAGE (Hamilton et al., 2017), GAT (Veliˇckovi´c et al., 2018), and GPR (Chien et al., 2021).

Original: Their default setup follows that in EERM 2 .
Revised: Their default setup follows that in EERM 2 .

Original: We refer the readers to Appendix C.1 for more implementation details of baselines and
Revised: We refer the readers to Appendix C.1 for more implementation details of baselines and

Original: GT RANS .
Revised: GT RANS .

Original: Notably, all experiments in this paper are repeated 10 times with different random seeds.
Revised: Notably, all experiments in this paper are repeated 10 times with different random seeds.

Original: Results.
Revised: Results.

Original: Table 1 reports the averaged performance over the test graphs for each dataset as well as the averaged rank of each algorithm.
Revised: Table 1 reports the averaged performance over the test graphs for each dataset as well as the averaged rank of each algorithm.

Original: From the table, we make the following observations:
Revised: From the table, we make the following observations:

Original: (a) Overall Performance.
Revised: (a) Overall Performance.

Original: The proposed framework consistently achieves strong performance across the datasets: GT RANS achieves average ranks of 1.0, 1.7, 2.0 and 1.7 with GCN, SAGE, GAT and GPR, respectively, while the corresponding ranks for the best baseline EERM are 2.9, 3.4, 3.0 and 2.0.
Revised: The proposed framework consistently achieves strong performance across the datasets GT RANS achieves average ranks of 1.0, 1.7, 2.0 and 1.7 with GCN, SAGE, GAT and GPR, respectively, while the corresponding ranks for the best baseline EERM are 2.9, 3.4, 3.0 and 2.0.

Original: Furthermore, in most of the cases, GT RANS significantly improves the vanilla baseline (ERM) by a
Revised: Furthermore, in most of the cases, GT RANS significantly improves the vanilla baseline (ERM [Table] [Figure] GT RANS improves GCN on most test graphs

Revised: [Figure]

Revised: [Table]

Original:  large margin.
Revised: by a large margin.

Original: Particularly, when using GCN as backbone, GT RANS outperforms ERM by 3.1%, 5.0% and 2.0% on Cora , Elliptic and OGB-Arxiv , respectively.
Revised: Particularly, when using GCN as backbone, GT RANS outperforms ERM by 3.1%, 5.0% and 2.0% on Cora , Elliptic and OGB-Arxiv , respectively.

Original: These results demonstrate the effectiveness of GT RANS in tackling diverse types of distribution shifts.
Revised: These results demonstrate the effectiveness of GT RANS in tackling diverse types of distribution shifts.

Original: (b) Comparison to other baselines.
Revised: (b) Comparison to other baselines.

Original: Both DropEdge and EERM modify the training process to improve model generalization.
Revised: Both DropEdge and EERM modify the training process to improve model generalization.

Original: Nonetheless, they are less effective than GT RANS , as GT RANS takes advantage of the information from test graphs.
Revised: Nonetheless, they are less effective than GT RANS , as GT RANS takes advantage of the information from test graphs.

Original: As a test-time training method, Tent also performs well in some cases, but Tent only adapts the parameters in batch normalization layers and cannot be applied to models without batch normalization.
Revised: As a test-time training method, Tent also performs well in some cases, but Tent only adapts the parameters in batch normalization layers and cannot be applied to models without batch normalization.

Original: We further show the performance on each test graph on Cora with GCN in Figure 2 and the results for other datasets are provided in Appendix D.1.
Revised: We further show the performance on each test graph on Cora with GCN in Figure 3 and the results for other datasets are provided in Appendix D.4.

Original: We observe that GT RANS generally improves over individual test graphs within each dataset, which validates the effectiveness of GT RANS .[Table] [Table]
Revised: We observe that GT RANS generally improves over individual test graphs within each dataset, which validates the effectiveness of GT RANS .

Original: [Figure]

Original: GT RANS improves GCN on most test graphs.

Original: [Table]

Original: Efficiency Comparison.
Revised: Efficiency Comparison.

Original: Since EERM performs the best among baselines, Table 2 showcases the efficiency comparison between our proposed GT RANS and EERM on the largest test graph in each dataset.
Revised: Since EERM performs the best among baselines, Table 2 showcases the efficiency comparison between our proposed GT RANS and EERM on the largest test graph in each dataset.

Original: The additional running time of GT RANS majorly depends on the number of gradient descent steps.
Revised: The additional running time of GT RANS majorly depends on the number of gradient descent steps.

Original: As we only use a small number (5 or 10) throughout all the experiments, the time overhead brought by GT RANS is negligible.
Revised: As we only use a small number (5 or 10) throughout all the experiments, the time overhead brought by GT RANS is negligible.

Original: Compared with the re-training method EERM, GT RANS avoids the complex bilevel optimization and thus is significantly more efficient.
Revised: Compared with the re-training method EERM, GT RANS avoids the complex bilevel optimization and thus is significantly more efficient.

Original: Furthermore, EERM imposes a considerably heavier memory burden.
Revised: Furthermore, EERM imposes a considerably heavier memory burden.

Original: 4. R OBUSTNESS TO A BNORMAL F EATURES
Revised: 4. R OBUSTNESS TO A BNORMAL F EATURES

Original: Setup.
Revised: Setup.

Original: Following the setup in AirGNN (Liu et al., 2021a), we evaluate the robustness in the case of abnormal features.
Revised: Following the setup in AirGNN (Liu et al., 2021a), we evaluate the robustness in the case of abnormal features.

Original: Specifically, we simulate abnormal features by assigning random features taken from a multivariate standard Gaussian distribution to a portion of randomly selected test nodes.
Revised: Specifically, we simulate abnormal features by assigning random features taken from a multivariate standard Gaussian distribution to a portion of randomly selected test nodes.

Original: Note that the abnormal features are injected after model training (at test time) and we vary the ratio of noisy nodes from 0.1 to 0.4 with a step size of 0.05.
Revised: Note that the abnormal features are injected after model training (at test time) and we vary the ratio of noisy nodes from 0.1 to 0.4 with a step size of 0.05.

Original: This process is performed for four  datasets: the original version of Cora , Citeseer , Pubmed , and OGB-Arxiv .
Revised: This process is performed for four [Figure] [Figure] datasets: the original version of Cora , Citeseer , Pubmed , and OGB-Arxiv .

Original: In these four datasets, the training graph and the test graph have the same graph structure but the node features are different.
Revised: In these four datasets, the training graph and the test graph have the same graph structure but the node features are different.

Original: Hence, we use the training classification loss combined with the proposed contrastive loss to optimize GT RANS .
Revised: Hence, we use the training classification loss combined with the proposed contrastive loss to optimize GT RANS .

Original: We use GCN as the backbone model and adopt four GNNs as the baselines including GAT (Veliˇckovi´c et al., 2018), APPNP (Klicpera et al., 2018), AirGNN and AirGNN-t.
Revised: We use GCN as the backbone model and adopt four GNNs as the baselines including GAT (Veli ˇ ckovi ´ c et al., 2018), APPNP (Klicpera et al., 2018), AirGNN and AirGNN-t.

Original: Note that AirGNN-t tunes the message-passing hyper-parameter in AirGNN at test time.
Revised: Note that AirGNN-t tunes the message-passing hyper-parameter in AirGNN at test time.

Original: For a fair comparison, we tune AirGNN-t based on the performance on both training and validation nodes.
Revised: For a fair comparison, we tune AirGNN-t based on the performance on both training and validation nodes.

Original: Results.
Revised: Results.

Original: For each model, we present the node classification accuracy on both abnormal nodes and all test nodes (i.e., both normal and abnormal ones) in Figure 3 and Figure 7 (See Appendix D.2), respectively.
Revised: For each model, we present the node classification accuracy on both abnormal nodes and all test nodes (i.e., both normal and abnormal ones) in Figure 4 and Figure 8 (See Appendix D.5), respectively.

Original: From these figures, we have two observations.
Revised: From these figures, we have two observations.

Original: First, GT RANS significantly improves
Revised: First, GT RANS significantly improves

Original: GCN in terms of the performance on abnormal nodes and all test nodes for all datasets across all noise ratios.
Revised: GCN in terms of the performance on abnormal nodes and all test nodes for all datasets across all noise ratios.

Original: For example, on Cora with 30% noisy nodes, GT RANS improves GCN by 48.2%[Figure] [Figure] on abnormal nodes and 31.0% on overall test accuracy.
Revised: For example, on Cora with 30% noisy nodes, GT RANS improves GCN by 48.2% on abnormal nodes and 31.0% on overall test accuracy.

Original: This demonstrates the effectiveness of the  transformation process in GT RANS in alleviating the effect of abnormal features.
Revised: This demonstrates the effectiveness of the graph transformation process in GT RANS in alleviating the effect of abnormal features.

Original: Second,
Revised: Second,

Original: GT RANS shows comparable or better performance with AirGNNs, which are the SOTA defense methods for tackling abnormal features.
Revised: GT RANS shows comparable or better performance with AirGNNs, which are the SOTA defense methods for tackling abnormal features.

Original: It is worth mentioning that AirGNN-t improves AirGNN by tuning its hyper-parameter at test time, which aligns with our motivation that test-time adaptation can enhance model test performance.
Revised: It is worth mentioning that AirGNN-t improves AirGNN by tuning its hyper-parameter at test time, which aligns with our motivation that test-time adaptation can enhance model test performance.

Original: To further understand the effect of graph transformation, we provide the visualization of the test node embeddings obtained from abnormal graph (0.3 noise ratio) and transformed graph for Cora in Figures 4a and 4b, respectively.
Revised: To further understand the effect of graph transformation, we provide the visualization of the test node embeddings obtained from abnormal graph (0.3 noise ratio) and transformed graph for Cora in Figures 5a and 5b, respectively.

Original: We observe that the transformed graph results in well-clustered node representations, which indicates that GT RANS can promote intra-class compactness and counteract the effect of abnormal patterns.
Revised: We observe that the transformed graph results in well-clustered node representations, which indicates that GT RANS can promote intra-class compactness and counteract the effect of abnormal patterns.

Original: 4. R OBUSTNESS TO A DVERSARIAL A TTACK
Revised: 4. R OBUSTNESS TO A DVERSARIAL A TTACK

Original: Setup.
Revised: Setup.

Original: We further evaluate GT RANS under the setting of adversarial attack where we perturb the test graph, i.e., evasion attack.
Revised: We further evaluate GT RANS under the setting of adversarial attack where we perturb the test graph, i.e., evasion attack.

Original: Specifically, we use PR-BCD (Geisler et al., 2021), a scalable attack method, to attack the test graph in OGB-Arxiv .
Revised: Specifically, we use PR-BCD (Geisler et al., 2021), a scalable attack method, to attack the test graph in OGB-Arxiv .

Original: We focus on structural attacks, and vary the perturbation rate, i.e., the ratio of changed edges, from 5% to 25% with a step of 5%.
Revised: We focus on structural attacks, and vary the perturbation rate, i.e., the ratio of changed edges, from 5% to 25% with a step of 5%.

Original: Similar to
Revised: Similar to

Original: Section 4.2, we adopt the training classification loss together with the proposed contrastive loss to optimize GT RANS .
Revised: Section 4.2, we adopt the training classification loss together with the proposed contrastive loss to optimize GT RANS .

Original: We use GCN as the backbone and employ four robust baselines including
Revised: We use GCN as the backbone and employ four robust baselines including

Original: GAT (Veli ˇ ckovi ´ c et al., 2018), RobustGCN (Zhu et al., 2019), SimPGCN (Jin et al., 2021a) and GCNJaccard (Xu et al., 2019) as comparisons.
Revised: GAT (Veli ˇ ckovi ´ c et al., 2018), RobustGCN (Zhu et al., 2019), SimPGCN (Jin et al., 2021a) and GCNJaccard (Xu et al., 2019) as comparisons.

Original: Among them, GCNJaccard pre-processes the attacked graph by removing edges where the similarities of connected nodes are less than a threshold; we tune this threshold at test time based on the performance on both training and validation nodes.
Revised: Among them, GCNJaccard pre-processes the attacked graph by removing edges where the similarities of connected nodes are less than a threshold; we tune this threshold at test time based on the performance on both training and validation nodes.

Original: Results.
Revised: Results.

Original: Table 3 reports the performances under structural evasion attack.
Revised: Table 3 reports the performances under structural evasion attack.

Original: We observe that GT RANS consistently improves the performance of GCN under different perturbation rates of adversarial attack.
Revised: We observe that GT RANS consistently improves the performance of GCN under different perturbation rates of adversarial at- [Table]

Original:  Particularly, GT RANS improves GCN by a larger margin when the perturbation rate is higher.
Revised: tack. Particularly, GT RANS improves GCN by a larger margin when the perturbation rate is higher.

Original: For example, GT RANS outperforms GCN by over 40% under the 25% perturbation rate.
Revised: For example, GT RANS outperforms GCN by over 40% under the 25% perturbation rate.

Original: Such observation suggests that GT RANS can counteract the devastating effect of adversarial attacks.
Revised: Such observation suggests that GT RANS can counteract the devastating effect of adversarial attacks.

Original: In addition, the best performing baseline GCNJaccard also modifies the graph at test time, which demonstrates the importance of test-time graph adaptation.
Revised: In addition, the best performing baseline GCNJaccard also modifies the graph at test time, which demonstrates the importance of test-time graph adaptation.

Original: Nonetheless, it consistently underperforms our proposed GT RANS , indicating that a learnable transformation function is needed to achieve better robustness under adversarial attacks, which GCNJaccard does not employ.
Revised: Nonetheless, it consistently underperforms our proposed GT RANS , indicating that a learnable transformation function is needed to achieve better robustness under adversarial attacks, which GCNJaccard does not employ.

Original: Interpretation.
Revised: Interpretation.

Original: To understand the modifications made on the graph, we compare several properties among clean graph, attacked graph (20% perturbation rate), graph obtained by GCNJaccard, and graph obtained by GT RANS in Table 7 in Appendix D.3.
Revised: To understand the modifications made on the graph [Table] we compare several properties among clean graph, attacked graph (20% perturbation rate), graph obtained by GCNJaccard, and graph obtained by GT RANS in Table 13 in Appendix D.6.

Original: First, adversarial attack decreases homophily and feature similarity, but GT RANS and GCNJaccard promote such information to alleivate the adversarial patterns.
Revised: First, adversarial attack decreases homophily and feature similarity, but GT RANS and GCNJaccard promote such information to alleivate the adversarial patterns.

Original: Our experiment also shows that GT RANS removes 77% adversarial edges[Table] while removing 30% existing edges from the attacked graph.
Revised: Our experiment also shows that GT RANS removes 77% adversarial edges while removing 30% existing edges from the attacked graph.

Original: Second, both GT RANS and GCNJaccard focus on deleting edges from the attacked graph [Table] but GCNJaccard removes a substantially larger amount of edges, which may destroy clean graph structure and lead to sub-optimal performance.
Revised: Second, both GT RANS and GCNJaccard focus on deleting edges from the attacked graph, but GCNJaccard removes a substantially larger amount of edges, which may destroy clean graph structure and lead to sub-optimal performance.

Original: 4. F URTHER A NALYSIS
Revised: 4. F URTHER A NALYSIS

Original: [Table]
Revised: [Table]

Original: Cross-Architecture Transferability.
Revised: Cross-Architecture Transferability.

Original: Since the outcome of GT RANS is a refined graph, it can conceptually be employed by any GNN model.
Revised: Since the outcome of GT RANS is a refined graph, it can conceptually be employed by any GNN model.

Original: In other words, we can transform the graph based on one pre-trained GNN and test the transformed graph on another pre-trained GNN.
Revised: In other words, we can transform the graph based on one pre-trained GNN and test the transformed graph on another pre-trained GNN.

Original: To examine such transferability, we perform experiments on four GNNs including GCN, APPNP, AirGNN and GAT under the abnormal feature setting with 30% noisy nodes on Cora .
Revised: To examine such transferability, we perform experiments on four GNNs including GCN, APPNP, AirGNN and GAT under the abnormal feature setting with 30% noisy nodes on Cora .

Original: The results on all test nodes in Table 4.
Revised: The results on all test nodes in Table 4.

Original: Note that “Tr” stands for GNNs used in TTGT while “Te” denotes GNNs used for obtaining final predictions on the transformed graph; “Noisy” indicates the performance on the noisy graph.
Revised: Note that “Tr” stands for GNNs used in TTGT while “Te” denotes GNNs used for obtaining final predictions on the transformed graph; “Noisy” indicates the performance on the noisy graph.

Original: We observe that the transformed graph yields good performance even outside the scope it was optimized for.
Revised: We observe that the transformed graph yields good performance even outside the scope it was optimized for.

Original: We anticipate that such transferability can alleviate the need for costly re-training on new GNNs.
Revised: We anticipate that such transferability can alleviate the need for costly re-training on new GNNs.

Original: Adapting Model vs. Adapting Data.
Revised: Adapting Model vs. Adapting Data.

Original: We empirically compare the performance between adapting data and adapting model and consider the OOD and abnormal feature settings.
Revised: We empirically compare the performance between adapting data and adapting model and consider the OOD and abnormal feature settings.

Original: Specifically, we use
Revised: Specifically, we use

Original: GCN as the backbone and adapt the model parameters by optimizing the same loss function as used in GT RANS .
Revised: GCN as the backbone and adapt the model parameters by optimizing the same loss function as used in GT RANS .

Original: The results are shown in Figure 4c and 4d.
Revised: The results are shown in Figures 5c and 5d.

Original: In OOD setting, both adapting model and adapting data can generally improve GCN’s performance.
Revised: In OOD setting, both adapting model and adapting data can generally improve GCN’s performance.

Original: Since their performances are still close, it is hard to give a definite answer on which strategy is better.
Revised: Since their performances are still close, it is hard to give a definite answer on which strategy is better.

Original: However, we can observe significant performance differences when the graph contains abnormal features: adapting data outperforms adapting model on 3 out of 4 datasets.
Revised: However, we can observe significant performance differences when the graph contains abnormal features: adapting data outperforms adapting model on 3 out of 4 datasets.

Original: This suggests that adapting data can be more powerful when the data is perturbed, which aligns with our analysis in Section 3.3.
Revised: This suggests that adapting data can be more powerful when the data is perturbed, which aligns with our analysis in Section 3.3.

Original: Effect of Surrogate Loss.
Revised: Effect of Surrogate Loss.

Original: Recall that we used a combined loss of contrastive loss and training loss in the settings of abnormal features and adversarial attack.
Revised: Recall that we used a combined loss of contrastive loss and training loss in the settings of abnormal features and adversarial attack.

Original: We now examine the effect of using them alone and show the results in Tables 8 and 9 in Appendix D.4.
Revised: We now examine the effect of using them alone and show the results in Tables 14 and 15 in Appendix D.7.

Original: We observe that (1) both of them bring improvement over vanilla GCN, while optimizing the training loss alone improves more than optimizing the other; and (2) combining them always yields a better or comparable performance.
Revised: We observe that (1) both of them bring improvement over vanilla GCN, while optimizing the training loss alone improves more than optimizing the other; and (2) combining them always yields a better or comparable performance.

Original: Learning Features v.s. Learning Structure.
Revised: Learning Features v.s. Learning Structure.

Original: Since our framework learns both node features and graph structure, we investigate when one component plays a more important role than the other.
Revised: Since our framework learns both node features and graph structure, we investigate when one component plays a more important role than the other.

Original: Our results are shown in Tables 10 and 11 in Appendix D.5.
Revised: Our results are shown in Tables 16 and 17 in Appendix D.8.

Original: From the tables, we observe that (1) while each component can improve the vanilla performance, feature learning is more crucial for counteracting feature corruption and structure learning is more important for defending structure corruption; and (2) combining them generally yields a better or comparable performance.
Revised: From the tables, we observe that (1) while each component can improve the vanilla performance, feature learning is more crucial for counteracting feature corruption and structure learning is more important for defending structure corruption; and (2) combining them generally yields a better or comparable performance.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: GNNs tend to yield unsatisfying performance when the presented data is sub-optimal.
Revised: GNNs tend to yield unsatisfying performance when the presented data is sub-optimal.

Original: To tackle this issue, we seek to enhance GNNs from a data-centric perspective by transforming the graph data at test time.
Revised: To tackle this issue, we seek to enhance GNNs from a data-centric perspective by transforming the graph data at test time.

Original: We propose GT RANS which optimizes a contrastive surrogate loss to transform graph structure and node features, and provide theoretical analysis with deeper discussion to understand this framework.
Revised: We propose GT RANS which optimizes a contrastive surrogate loss to transform graph structure and node features and provide theoretical analysis with deeper discussion to understand this framework.

Original: Experimental results on distribution shift, abnormal features and adversarial attack have demonstrated the effectiveness of our method.
Revised: Experimental results on distribution shift, abnormal features and adversarial attack have demonstrated the effectiveness of our method.

Original: In the future, we plan to explore more applications of our framework such as mitigating degree bias and long-range dependency.
Revised: In the future, we plan to explore more applications of our framework such as mitigating degree bias and long-range dependency

Original:  T HEREOM
Revised: A. T HEREOM

Original: Theorem 2.
Revised: Theorem 2.

Original: Assume that the augmentation function A ( · ) generates a data view of the same class for the test nodes and the node classes are balanced.
Revised: Assume that the augmentation function A ( · ) generates a data view of the same class for the test nodes and the node classes are balanced.

Original: Assume for each class, the mean of the representations obtained from Z and ˆ Z are the same.
Revised: Assume for each class, the mean of the repre- sentations obtained from Z and ˆ Z are the same.

Original: Minimizing the first term in Eq.
Revised: Minimizing the first term in Eq.

Original: (6) is approximately minimizing the class-conditional entropy H ( Z | Y ) between features Z and labels Y .
Revised: (6) is approximately minimizing the class-conditional entropy H ( Z | Y ) between features Z and labels Y .

Original: Proof.
Revised: Proof.

Original: For convenience, we slightly abuse the notations to replace z i ∥
Revised: For convenience, we slightly abuse the notations to replace z i ∥

Original: z i ∥ and
Revised: z i ∥ and

Original: ˆ z i ∥
Revised: ˆ z i ∥

Original: ˆ z i ∥ with z
Revised: ˆ z i ∥ with z

Original: i
Revised: i

Original: and ˆ z i , respectively.
Revised: and ˆ z i , respectively.

Original: Then we have ∥ z i ∥ =
Revised: Then we have ∥ z i ∥ =

Original: ∥
Revised: ∥

Original: ˆ z i ∥ = 1 .
Revised: ˆ z i ∥ = 1 .

Original: Let Z k denote the set of test samples from class k ; thus | Z k | =
Revised: Let Z k denote the set of test samples from class k ; thus | Z k | =

Original: NK .
Revised: NK .

Original: Let c
Revised: Let c

Original: = denote equality up to a multiplicative and/or additive constant.
Revised: = denote equality up to a multiplicative and/or additive constant.

Original: Then the first term in Eq.
Revised: Then the first term in Eq.

Original: (6) can be rewritten as:
Revised: (6) can be rewritten as:

Original: [Equation]
Revised: [Equation]

Original: Let c k be the mean of hidden representations from class k ; then we have c k = 1 | Z k | (cid:80) z i ∈
Revised: Let c k be the mean of hidden representations from class k ; then we have c k = [Equation] | [Equation]

Revised: z i ∈

Original: Z k z
Revised: Z k z

Original: i = 1 | Z k | (cid:80)
Revised: i = 1 | Z k | [Equation]

Original: ˆ z i ∈
Revised: ˆ z i ∈

Original: ˆ Z k
Revised: ˆ Z k

Original: ˆ z i .
Revised: ˆ z i .

Original: Now we build the connection between Eq.
Revised: Now we build the connection between Eq.

Original: (17) and (cid:80) [-K i =
-] (cid:80) z i ∈
Revised: (17) and (cid:80) [-K i =
-]Ki =1 (cid:80) z i ∈

Original: Z k ∥
Revised: Z k ∥

Original: z i − c k ∥ 2 : 
Revised: z i − c k ∥ 2 : K

Original: [Equation]
Revised: [Equation]

Original: i =

Original: [Equation]
Revised: [Equation]

Original: (cid:88) z i ∈ Z k
Revised: k

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: By comparing Eq.
Revised: By comparing Eq.

Original: (17) and Eq.
Revised: (17) and Eq.

Original: (19), the only difference is that Eq.
Revised: [Equation] the only difference is that Eq.

Original: (19) includes more positive pairs for loss calculation.
Revised: (19) includes more positive pairs for loss calculation.

Original: Hence, minimizing Eq.
Revised: Hence [Equation] minimizing Eq.

Original: (17) can be viewed as approximately minimizing
Revised: (17) can be viewed as approximately minimizing

Original: Eq.
Revised: Eq.

Original: (19) or Eq.
Revised: (19) or Eq.

Original: (18) through sampling positive pairs.
Revised: (18) through sampling positive pairs.

Original: As demonstrated in the work (Boudiaf et al., 2020), Eq.
Revised: As demonstrated in the work (Boudiaf et al., 2020), Eq.

Original: (18) can be interpreted as a conditional cross-entropy between Z and another random variable ¯ Z , whose conditional distribution given Y is a standard Gaussian centered around c Y :
Revised: (18) can be interpreted as a conditional cross-entropy between Z and another random variable ¯ Z , whose conditional distribution given Y is a standard Gaussian centered around c Y :

Original: Z |
Revised: Z |

Original: Y ∼ N ( c Y , I ) :
Revised: Y ∼ N ( c Y , I ) :

Original: [Equation]
Revised: [Equation]

Original: Hence, minimizing the first term in Eq.
Revised: Hence, minimizing the first term in Eq.

Original: (6) is approximately minimizing H ( Z | Y ) .
Revised: (6) is approximately minimizing H ( Z | Y ) .

Revised: Discussion: We note that the assumption “the mean of the representations obtained from Z and ˆ Z are the same” can be inferred by the first assumption about data augmentation.

Revised: Let p k ( X ) denote the distribution of samples with class k and let x ∼ p

Revised: k

Original: A. A F IGURATIVE E XAMPLE
Revised: ( X ) denote the sample with class k . Recall that we assume the data augmentation function A ( · ) is strong enough to generate a data view that can simulate the test data from the same class. In this regard, the new data view can be regarded as an independent sample from the same class, i.e., A ( x ) ∼ p k

Revised: ( X ) .

Revised: Hence, the expectation of Z and ˆ Z is the same and we would approximately have that “the mean of Z and ˆ Z is the same for each class”.

Revised: Particularly, when the number of samples is relatively large, the mean of Z ( ˆ Z ) would be close to the true distribution mean.

Revised: For example, on one graph of Cora, the mean absolute difference between the two mean representations of Z and ˆ Z are [0.018, 0.009, 0.021, 0.024, 0.016, 0.014, 0.0, 0.016, 0.023] for each class, which are actually very small.

Original: In Figure 5, we show an example of adversarial attack which causes the aggregated features for two nodes to be the same.
Revised: In Figure 6, we show an example of adversarial attack which causes the aggregated features for two nodes to be the same.

Original: Given two nodes x 1 and x 2 and their connections, we are interested in predicting their labels.
Revised: Given two nodes x 1 and x 2 and their connections, we are interested in predicting their labels.

Original: Assume a mean aggregator is used for aggregating features from the neighbors.
Revised: Assume a mean aggregator is used for aggregating features from the neighbors.

Original: Before attack, the aggregated features for them are ¯ x 1 = [0 . 45] and ¯ x 2 = [0 .
Revised: Before attack, the aggregated features for them are ¯ x 1 = [0 . 45] and ¯ x 2 = [0 .

Original: 53] while after attack the aggregated features become the same ¯ x
Revised: 53] while after attack the aggregated features become the same ¯ x

Original: 1 = ¯ x 2 = [0 . 45] .
Revised: 1 = ¯ x 2 = [0 . 45] .

Original: In this context, it is impossible to learn a classifier that can distinguish the two nodes. 
Revised: In this context, it is impossible to learn a classifier that can distinguish the two nodes. [Figure] [Figure]

Original: Attack [0.3] [0.3] [0.8] [0.2] [0.7] [0.7] [0] [0.2] 𝒙𝒙[0.3] [0.3] [0.8] [0.2] [0.7] [0.7] [0.2] 𝒙𝒙𝒙# 𝟏 = 0.45 ,𝒙# 𝟐 = 0.
Revised: [Figure] [Figure] [Figure] where the red edge indicates the adversarial edge injected by the attacker.

Original: 𝒙# 𝟏 = 0.45 ,𝒙# 𝟐 = 0.
Revised: The aggregated features for the two center nodes become ¯ x 1 = [0 . 45] and ¯ x 2 = [0 . 45] .

Original: B A LGORITHM
Revised: B A LGORITHM

Original: We show the detailed algorithm of GT RANS in Algorithm 1.
Revised: We show the detailed algorithm of GT RANS in Algorithm 1.

Original: In detail, we first initialize ∆ A and ∆ X ′ as zero matrices and calculate L s based on Eq.
Revised: In detail, we first initialize ∆ A and ∆ X ′ as zero matrices and calculate L s based on Eq.

Original: Since we alternatively optimize ∆ A and ∆ X ′ , we update ∆ X ′ every τ 1 epochs and update ∆ A every τ 2 epochs.
Revised: Since we alternatively optimize ∆ A and ∆ X ′ , we update ∆ X ′ every τ 1 epochs and update ∆ A every τ 2 epochs.

Original: When the optimization is done, we sample the discrete graph structure for K times and select the one that results in the smallest L s as the final adjacency matrix.
Revised: When the optimization is done, we sample the discrete graph structure for K times and select the one that results in the smallest L s as the final adjacency matrix.

Original: C D ATASETS AND H YPER -P ARAMETERS
Revised: C D ATASETS AND H YPER -P ARAMETERS

Original: In this section, we reveal the details of reproducing the results in the experiments.
Revised: In this section, we reveal the details of reproducing the results in the experiments.

Original: We will release the source code upon acceptance.
Revised: We will release the source code upon acceptance.

Original: Y C. O UT - OF -D ISTRIBUTION (OOD) S ETTING
Revised: C. O UT - OF -D ISTRIBUTION (OOD) S ETTING

Original: Dataset Statistics.
Revised: Dataset Statistics.

Original: For the evaluation on OOD data, we use the datasets provided by Wu et al.
Revised: For the evaluation on OOD data, we use the datasets provided by Wu et al.

Original: (2022a).
Revised: (2022a).

Original: The dataset statistics are shown in Table 5, which includes three distinct type of distribution shifts: (1) artificial transformation which indicates the node features are replaced by synthetic spurious features; (2) cross-domain transfers which means that graphs in the dataset are from different domains and (3) temporal evolution where the dataset is a dynamic one with evolving nature.
Revised: The dataset statistics are shown in Table 5, which includes three distinct type of distribution shifts: (1) artificial transformation which indicates the node features are replaced by synthetic spurious features; (2) cross-domain transfers which means that graphs in the dataset are from different domains and (3) temporal evolution where the dataset is a dynamic one with evolving nature.

Original: Notably, we use the datasets provided by Wu et al.
Revised: Notably, we use the datasets provided by Wu et al.

Original: (2022a), which were adopted from the aforementioned references with manually created distribution shifts.
Revised: (2022a), which were adopted from the aforementioned references with manually created distribution shifts.

Original: Note that there can be multiple training/validaiton/test graphs.
Revised: Note that there can be multiple training/validaiton/test graphs.

Original: Specifically, Cora and Amazon-Photo have 1/1/8 graphs for training/validation/test sets.
Revised: Specifically, Cora and Amazon-Photo have 1/1/8 graphs for training/validation/test sets.

Original: Similarly, the splits are 1/1/5 on Twitch-E , 3/2/3 on FB-100 , 5/5/33 on
Revised: Similarly, the splits are 1/1/5 on Twitch-E , 3/2/3 on FB-100 , 5/5/33 on

Original: Elliptic , and 1/1/3 on OGB-Arxiv .
Revised: Elliptic , and 1/1/3 on OGB-Arxiv .

Original: [Table]

Original: Hyper-Parameter Setting.
Revised: Hyper-Parameter Setting.

Original: For the setup of backbone GNNs, we majorly followed Wu et al.
Revised: For the setup of backbone GNNs, we majorly followed Wu et al.

Original: (2022a):
Revised: (2022a):

Original: (a) GCN: the architecture setup is 5 layers with 32 hidden units for Elliptic and OGB-Arxiv , andlayers with 32 hidden units for other datasets, and with batch normalization for all datasets.
Revised: 1 Input: Pre-trained f θ and test graph dataset G Te = ( A , X ′ ) . 2 Output: prediction and transformed graph G ′ = X ′′ ) .

Revised: 3 Initialize ∆ A and ∆ X as zero matrices 4 for t = 0 , . .

Revised: , T − 1 do

Revised: Compute A ′ =

Revised: A ⊕ ∆ A and X ′ =

Revised: X + ∆ X

Revised: Compute L s (∆ A , ∆ X ) as shown in Eq.

Revised: (6) if t %( τ 1 + τ 2 ) <

Revised: τ 1 then

Revised: Update ∆ X ← ∆ X − η 1 ∇ ∆ X L s else

Revised: Update ∆ A ← Π P (∆ A − η ∇ ∆

Revised: A L s ) 11 ℓ best = ∞ # store the best loss 12 for k = 0 , . .

Revised: , K − 1 do

Revised: Sample A ′ 0 ∼ Bernoulli

Revised: ( A ⊕ ∆ A )

Revised: Calculate L s with A ′ 0 as input if L s <

Revised: ℓ best then ℓ best =

Revised: L s

Revised: A ′ =

Revised: A ′

Original: The learning rate is set to 0.001 for Cora and Amz-Photo , 0.01 for other datasets; the weight decay is set to 0 for Elliptic and OGB-Arxiv , and 0.001 for other datasets.
Revised: [Table] X ′ = X + ∆ X [Table]

Revised: ˆ Y =

Original: (b) GraphSAGE: the architecture setup is 5 layers with 32 hidden units for Elliptic and OGB-Arxiv ,  and 
Revised: f θ ( A ′ , X ′ ) 20 Return: ˆ Y , ( A ′ , X ′ ) [Table] (a) GCN: the architecture setup is 5 layers with 32 hidden units for Elliptic and OGB-Arxiv , andlayers with 32 hidden units for other datasets, and with batch normalization for all datasets.

Revised: The learning rate is set to 0.001 for Cora and Amz-Photo , 0.01 for other datasets; the weight decay is set to 0 for Elliptic and OGB-Arxiv , and 0.001 for other datasets.

Original:  layers with 32 hidden units for other datasets, and with batch normalization for all datasets.
Revised: (b) GraphSAGE: the architecture setup is 5 layers with 32 hidden units for Elliptic and OGB-Arxiv , and 2 layers with 32 hidden units for other datasets, and with batch normalization for all datasets.

Original: The learning rate is set to 0.01 for all datasets; the weight decay is set to 0 for Elliptic and OGBArxiv , and 0.001 for other datasets.
Revised: The learning rate is set to 0.01 for all datasets; the weight decay is set to 0 for Elliptic and OGBArxiv , and 0.001 for other datasets.

Original: (c) GAT: the architecture setup is 5 layers for Elliptic and OGB-Arxiv , and 2 layers for other datasets, and with batch normalization for all datasets.
Revised: (c) GAT: the architecture setup is 5 layers for Elliptic and OGB-Arxiv , and 2 layers for other datasets, and with batch normalization for all datasets.

Original: Each layer contains 4 attention heads and each head is associated with 32 hidden units.
Revised: Each layer contains 4 attention heads and each head is associated with 32 hidden units.

Original: The learning rate is set to 0.01 for all datasets; the weight decay is set to 0 for Elliptic and OGB-Arxiv , and 0.001 for other datasets.
Revised: The learning rate is set to 0.01 for all datasets; the weight decay is set to 0 for Elliptic and OGB-Arxiv , and 0.001 for other datasets.

Original: (d) GPR: We use 10 propagation layers and 2 transformation layers with 32 hidden units.
Revised: (d) GPR: We use 10 propagation layers and 2 transformation layers with 32 hidden units.

Original: The learning rate is set to 0.01 for all datasets; the weight decay is set to 0 for Elliptic and OGB-Arxiv , and 0.001 for other datasets.
Revised: The learning rate is set to 0.01 for all datasets; the weight decay is set to 0 for Elliptic and OGB-Arxiv , and 0.001 for other datasets.

Original: Note that GPR does not contain batch normalization layers.
Revised: Note that GPR does not contain batch normalization layers.

Original: For the baseline methods, we tuned their hyperparameters based on the validation performance.

Original: For
Revised: For

Revised: the baseline

Revised: methods,

Revised: we

Original:  DropEdge, we search the drop ratio in the range of[0, 0.05, 0.1, 0.15, 0.2, 0.3, 0.5, 0.7].
Revised: tuned their hyper-parameters based on the validation performance For DropEdge, we search the drop ratio in the range of

Original: For Tent, we search the learning rate in the range of [1e-2, 1e-3, 1e-4, 1e-5, 1e-6] and the running epochs in [1, 10, 20, 30].
Revised: For Tent, we search the learning rate in the range of [1e-2, 1e-3, 1e-4, 1e-5, 1e-6] and the running epochs in [1, 10, 20, 30].

Original: For EERM, we followed the instruction provided by the original paper.
Revised: For EERM, we followed the instruction provided by the original paper.

Original: For GT RANS , we alternatively optimize node features for τ 1 = 4 epochs and optimize graph structure τ 2 =epoch.
Revised: For GT RANS , we alternatively optimize node features for τ 1 = 4 epochs and optimize graph structure τ 2 =epoch.

Original: We adopt DropEdge as the augmentation function A ( · ) and set the drop ratio to 0.5.
Revised: We adopt DropEdge as the augmentation function A ( · ) and set the drop ratio to 0.5.

Original: We use
Revised: We use

Original: Adam optimizer for both feature learning and structure learning.
Revised: Adam optimizer for both feature learning and structure learning.

Original: We further search the learning rate of feature adaptation η 1 in [5e-3, 1e-3, 1e-4, 1e-5, 1e-6], learning rate of structure adaptation η 2 in [0.5, 0.1, 0.01], the modification budget B in [0.5%, 1%, 5%] of the original edges, total epochs T in [5, 10].
Revised: We further search the learning rate of feature adaptation η 1 in [5e-3, 1e-3, 1e-4, 1e-5, 1e-6], learning rate of structure adaptation η 2 in [0.5, 0.1, 0.01], the modification budget B in [0.5%, 1%, 5%] of the original edges, total epochs T in [5, 10].

Original: We note that the process of tuning hyper-parameters is quick due to the high efficiency of test-time adaptation as we demonstrated in Section 4.1.
Revised: We note that the process of tuning hyper-parameters is quick due to the high efficiency of test-time adaptation as we demonstrated in Section 4.1.

Original: Evaluation Protocol.
Revised: Evaluation Protocol.

Original: For ERM (standard training), we train all the GNN backbones using the common cross entropy loss.
Revised: For ERM (standard training), we train all the GNN backbones using the common cross entropy loss.

Original: For DropEdge, we drop a certain amount of edges at each training epoch.
Revised: For DropEdge, we drop a certain amount of edges at each training epoch.

Original: For EERM, it optimizes a bi-level problem to obtain a trained classifier.
Revised: For EERM, it optimizes a bi-level problem to obtain a trained classifier.

Original: Note that the aforementioned three methods do not perform any test-time adaptation and their model parameters are fixed during test.
Revised: Note that the aforementioned three methods do not perform any test-time adaptation and their model parameters are fixed during test.

Original: For the two test-time adaptation methods, Tent and GT RANS , we first obtain the GNN backbones pre-trained from ERM and adapt the model parameters or graph data at test time, respectively.
Revised: For the two test-time adaptation methods, Tent and GT RANS , we first obtain the GNN backbones pre-trained from ERM and adapt the model parameters or graph data at test time, respectively.

Original: Furthermore, Tent minimizes the entropy loss while GT RANS minimizes the contrastive surrogate loss. 
Revised: Furthermore, Tent minimizes the entropy loss while GT RANS minimizes the contrastive surrogate loss. Quantifying Distribution Shifts.

Original: C. A BNORMAL F EATURES
Revised: Following SR-GNN (Zhu et al., 2021a), we adopt central moment discrepancy (CMD) (Zellinger et al., 2017) as the measurement to quantify the distribution shifts in different graphs. Specifically, given a pre-trained model, we obtain its hidden representation on the training graph and test graphs, denoted as Z tr and Z te . Then we calculate their distance by the CMD metric, i.e., CMD ( Z tr ) , Z te . We show the results in Table 6 and we can observe certain distribution shifts as these values are not small.

Revised: Let’s take the OGB-Arxiv dataset as an example, where we select papers published before 2011 for training, 2011-2014 for validation, and within 2014-2016/20162018/2018-2020 for test.

Revised: In this context, the distribution shift is from the temporal change.

Revised: In

Revised: Table 7, we show the CMD values, ERM performance and GT RANS performance.

Revised: From the table, we can find that (1) the CMD value on the validation graph is essentially smaller than those on test graphs; and (2) GCN performances on test graphs (with larger shifts) are lower than that on

Revised: [Table]

Revised: [Table]

Revised: A BNORMAL F EATURES

Original: Dataset Statistics.
Revised: Dataset Statistics.

Original: In these two settings, we choose the original version of popular benchmark datasets Cora , Citeseer , Pubmed and OGB-Arxiv .
Revised: In these two settings, we choose the original version of popular benchmark datasets Cora , Citeseer , Pubmed and OGB-Arxiv .

Original: The statistics for these datasets are shown in
Revised: The statistics for these datasets are shown in

Original: Table 6.
Revised: Table 8.

Original: Note that we only have one test graph, and the injection of abnormal features or adversarial attack happens after the training process of backbone model, which can be considered as evasion attack.[Table]
Revised: Note that we only have one test graph [Table] and the injection of abnormal features or adversarial attack happens after the training process of backbone model, which can be viewed as evasion attack.

Original: Hyper-Parameter Settings.
Revised: Hyper-Parameter Settings.

Original: We closely followed AirGNN (Liu et al., 2021a) to set up the hyperparameters for the baselines:
Revised: We closely followed AirGNN (Liu et al., 2021a) to set up the hyperparameters for the baselines:

Original: (a) GCN: the architecture setup is 2 layers with 64 hidden units without batch normalization for Cora , Citeseer and Pubmed , and 3 layers with 256 hidden units with batch normalization for OGB-Arxiv .
Revised: (a) GCN: the architecture setup is 2 layers with 64 hidden units without batch normalization for Cora , Citeseer and Pubmed , and 3 layers with 256 hidden units with batch normalization for OGB-Arxiv .

Original: The learning rate is set to 0.01.
Revised: The learning rate is set to 0.01.

Original:  (b) GAT: the architecture setup is 2 layers with 8 hidden units in each of the 8 heads without batch normalization for Cora , Citeseer and Pubmed , and 3 layers with 32 hidden units in each of the 8 heads with batch normalization for OGB-Arxiv .
Revised: [Table] (b) GAT: the architecture setup is 2 layers with 8 hidden units in each of the 8 heads without batch normalization for Cora , Citeseer and Pubmed , and 3 layers with 32 hidden units in each of the 8 heads with batch normalization for OGB-Arxiv .

Original: The learning rate is set to 0.005.
Revised: The learning rate is set to 0.005.

Original: (c) APPNP:
Revised: (c) APPNP:

Original: the architecture setup is 2-layer transformation with 64 hidden units and 10-layer propagation without batch normalization for Cora , Citeseer and Pubmed ; the architecture is set to 3-layer transformation with 256 hidden units and 10-layer propagation with batch normalization for OGB-Arxiv .
Revised: the architecture setup is 2-layer transformation with 64 hidden units and 10-layer propagation without batch normalization for Cora , Citeseer and Pubmed ; the architecture is set to 3-layer transformation with 256 hidden units and 10-layer propagation with batch normalization for OGB-Arxiv .

Original: The learning rate is set to 0.01.
Revised: The learning rate is set to 0.01.

Original: (d) AirGNN: The architecture setup is the same as APPNP and the hyper-parameter λ is set to 0.
Revised: (d) AirGNN: The architecture setup is the same as APPNP and the hyper-parameter λ is set to 0.

Original: for OGB-Arxiv and 0.5 for other datasets.
Revised: for OGB-Arxiv and 0.5 for other datasets.

Original: (e) AirGNN-t: The architecture setup is the same as AirGNN but we tune the hyper-parameter λ in AirGNN based on performance on the combination of training and validation nodes at test stage
Revised: (e) AirGNN-t: The architecture setup is the same as AirGNN but we tune the hyper-parameter λ in AirGNN based on performance on the combination of training and validation nodes at test stage.

Original: This is because the test graph has the same graph structure as the training graph; thus we can take advantage of the label information of training nodes (as well as validation nodes) to tune the hyper-parameters
Revised: This is because the test graph has the same graph structure as the training graph; thus we can take advantage of the label information of training nodes (as well as validation nodes) to tune the hyper-parameters.

Original: Specifically, we search λ in the range of [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9] for each noise ratio
Revised: Specifically, we search λ in the range of [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9] for each noise ratio.

Original: For the setup of GT RANS , we alternatively optimize node features for τ 1 = 4 epochs and optimize graph structure τ 2 = 1 epoch.
Revised: For the setup of GT RANS [Table] we alternatively optimize node features for τ 1 = 4 epochs and optimize graph structure τ 2 = 1 epoch.

Original: We adopt DropEdge as the augmentation function A ( · ) and set the drop ratio to 0.5.
Revised: We adopt DropEdge as the augmentation function A ( · ) and set the drop ratio to 0.5.

Original: We use Adam optimizer for both feature learning and structure learning.
Revised: We use Adam optimizer for both feature learning and structure learning.

Original: We further search the learning rate of feature adaptation η 1 in [1, 1e-1, 1e-2], total epochs T in [10, 20].
Revised: We further search the learning rate of feature adaptation η 1 in [1, 1e-1, 1e-2], total epochs T in [10, 20].

Original: The modification budget B to 5% of the original edges and the learning rate of structure adaptation ηis set to 0.1.
Revised: The modification budget B to 5% of the original edges and the learning rate of structure adaptation ηis set to 0.1.

Original: It is worth noting that we use a weighted combination of contrastive loss and training classification loss , i.e., L train + λ L s , instead of optimizing the contrastive loss alone.
Revised: It is worth noting that we use a weighted combination of contrastive loss and training classification loss , i.e., L train + λ L s , instead of optimizing the contrastive loss alone.

Original: We adaopt this strategy because that the training graph and the test graph were the same graph before the injection of abnormal features.
Revised: We adaopt this strategy because that the training graph and the test graph were the same graph before the injection of abnormal features.

Original: Here the λ is tuned in the range of [1e-2, 1e-3, 1e-4].
Revised: Here the λ is tuned in the range of [1e-2, 1e-3, 1e-4].

Original: We study the effects of contrastive loss and training classification loss in Appendix D.4.
Revised: We study the effects of contrastive loss and training classification loss in Appendix D.7.

Original: C. A DVERSARIAL A TTACK
Revised: C. A DVERSARIAL A TTACK

Original: Dataset Statistics.
Revised: Dataset Statistics.

Original: We used OGB-Arxiv for the adversarial attack experiments and the dataset statistics can be found in Table 6.
Revised: We used OGB-Arxiv for the adversarial attack experiments and the dataset statistics can be found in Table 8.

Original: Again, we only have one test graph for this dataset.
Revised: Again, we only have one test graph for this dataset.

Original: Hyper-Parameter Settings.
Revised: Hyper-Parameter Settings.

Original: The setup of GCN and GAT is the same as that in the setting of abnormal features.
Revised: The setup of GCN and GAT is the same as that in the setting of abnormal features.

Original: For the defense methods including SimPGCN, RobustGCN and GCNJaccard, we use the DeepRobust 4 library to implement them.
Revised: For the defense methods including SimPGCN, RobustGCN and GCNJaccard, we use the DeepRobust (Li et al., 2020) library to implement them.

Original: For GCNJaccard, we tune its threshold hyperparameter in the range of [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9].
Revised: For GCNJaccard, we tune its threshold hyper-parameter in the range of [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9].

Original: The hyper-parameter is also tuned based on the performance of training and validation nodes (same as Appendix C.2).
Revised: The hyper-parameter is also tuned based on the performance of training and validation nodes (same as Appendix C.2).

Original: Note that the popular defenses ProGNN (Jin et al., 2020) and GCNSVD (Entezari et al., 2020) were not included because they throw OOM error due to the expensive eigen-decomposition operation.
Revised: Note that the popular defenses ProGNN (Jin et al., 2020) and GCNSVD (Entezari et al., 2020) were not included because they throw OOM error due to the expensive eigen-decomposition operation.

Original: We use the official implementation of the scalable attack PR-BCD (Geisler et al., 2021) to attack the test graph.
Revised: We use the official implementation of the scalable attack PR-BCD (Geisler et al., 2021) to attack the test graph.

Original: We note that when performing adversarial attack, the setting is more like transductive setting where the training graph and test graph are the same.
Revised: We note that when performing adversarial attacks, the setting is more like transductive setting where the training graph and test graph are the same.

Original: However, the test graph becomes different from the training graph after the attack.
Revised: However, the test graph becomes different from the training graph after the attack.

Original: Since the training graph and test graph were originally the same graph, we use a weighted combination of contrastive loss and training classification loss , i.e., L train + λ L s , instead of optimizing the contrastive loss alone.
Revised: Since the training graph and test graph were originally the same graph, we use a weighted combination of contrastive loss and training classification loss , i.e., L train + λ L s , instead of optimizing the contrastive loss alone.

Original: For the setup of GT RANS , we alternatively optimize node features for τ 1 = 1 epoch and optimize graph structure τ 2 = 4 epochs.
Revised: For the setup of GT RANS , we alternatively optimize node features for τ 1 = 1 epoch and optimize graph structure τ 2 = 4 epochs.

Original: We fix the learning rate of feature adaptation η 1 to 1e-3, learning rate of structure adaptation η 2 to 0.1, λ to 1, total epochs T to 50 and modification budget B to 30% of the original edges.
Revised: We fix the learning rate of feature adaptation η 1 to 1e-3, learning rate of structure adaptation η 2 to 0.1, λ to 1, total epochs T to 50 and modification budget B to 30% of the original edges.

Original: C. H ARDWARE AND S OFTWARE C ONFIGURATIONS .
Revised: C. H ARDWARE AND S OFTWARE C ONFIGURATIONS .

Original: We perform experiments on NVIDIA Tesla V100 GPUs.
Revised: We perform experiments on NVIDIA Tesla V100 GPUs.

Original: The GPU memory and running time reported in Table 2 are measured on one single V100 GPU.
Revised: The GPU memory and running time reported in Table 2 are measured on one single V100 GPU.

Original: Additionally, we use eight CPUs, with the model name as Intel(R)
Revised: Additionally, we use eight CPUs, with the model name as Intel(R)

Original: Xeon(R)
Revised: Xeon(R)

Original: Platinum 8260 CPU @ 2.40GHz.
Revised: Platinum 8260 CPU @ 2.40GHz.

Original: The operating system we use is
Revised: The operating system we use is

Original: CentOS Linux 7 (Core).
Revised: CentOS Linux 7 (Core).

Original: D M ORE E XPERIMENTAL R ESULTS D. O UT - OF -D ISTRIBUTION (OOD) S ETTING
Revised: D M ORE E XPERIMENTAL R ESULTS D. C OMPARISON TO G RAPH D OMAIN A DAPTATION

Revised: Our work is related to graph domain adaptation (GraphDA) (Shen et al., 2020; Wu et al., 2020; Zhu et al., 2021a), but they are also highly different .

Revised: In Table 9, we summarize the differences between

Revised: GraphDA and GT RANS .

Revised: In detail, there are the following differences:

Revised: (a) Data and losses : GraphDA methods optimize the loss function based on both labeled source data (training data) and unlabeled target data (test data), while GT RANS only requires target data during inference.

Revised: Hence, GraphDA methods are infeasible when access to the source data is prohibited such as online service.

Revised: (b) Parameter :

Revised: To our best knowledge, existing GraphDA methods are model-centric approaches while GT RANS is a data-centric approach.

Revised: GT RANS adapts the data instead of the model, which can be more useful in some settings as we showed in the Example of Section 3.3.

Revised: (c) Efficiency : GraphDA is indeed a training-time adaptation and for each given test graph, it would require training the model on the source and target data.

Revised: Thus, it is much less efficient than GT RANS , especially when we have multiple test graphs (e.g., 33 test graphs for Elliptic ).

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: To compare their empirical performance, we include two GraphDA methods (SR-GNN (Zhu et al., 2021a) and UDA-GCN (Wu et al., 2020)) and one general domain adaptation method (DANN (Ganin et al., 2016)).

Revised: SR-GNN regularizes the model’s performance on the source and target domains.

Revised: Note that SR-GNN is originally developed under the transductive setting where the training graph and test graph are the same.

Revised: To apply SR-GNN in our OOD setting, we assume the test graph is available during the training stage of SR-GNN, as typically done in domain adaptation methods.

Revised: UDA-GCN is another work that tackles graph data domain adaptation, which exploits local and global information for different domains.

Revised: In addition, we also include DANN, which adopts an adversarial domain classifier to promote the similarity of feature distributions between different domains.

Revised: We followed the authors’ suggestions in their paper to tune the hyper-parameters and the results are shown in

Revised: Table 10.

Revised: On the one hand, we can observe that GraphDA methods generally improve the performance of GCN under distribution shift and SRGNN is the best performing baseline.

Revised: On the other hand, GT RANS performs the best on all datasets except Amz-Photo .

Revised: On Amz-Photo , GT RANS does not improve as much as SR-GNN, which indicates that joint optimization over source and target is necessary for this dataset.

Revised: However, recall that domain adaptation methods are less efficient due to the joint optimization on source and target: the adaptation time of SR-GNN on the 8 graphs of

Revised: Amz-Photo is 83.5s while that of GT RANS is 4.9s (plus pre-training time 10.1s).

Revised: Overall, test-time graph transformation exhibits strong advantages of effectiveness and efficiency.

Revised: D. C OMPARISON TO G RAPH S TRUCTURE L EARNING

Revised: Our work is also relevant to graph structure learning (GSL) (Franceschi et al., 2019; Jin et al., 2020;

Revised: Chen et al., 2020; Zhao et al., 2021b;

Revised: Rozemberczki et al., 2021b; Halcrow et al., 2020; Fatemi et al., 2021) which learns the graph structure during the training time while not adapting the graph structure at test stage.

Revised: Our proposed test-time graph transform is essentially different from these works as we do not modify the training data but only the test data.

Revised: It can be of interest to adopt GSL

Revised: [Table]

Revised: method at test time by also adapting the test graph structure.

Revised: However, most existing GSL methods optimize the cross entropy loss defined on the labels to update graph structure, thus not applicable in the absence of test labels.

Revised: One exception is SLAPS (Fatemi et al., 2021) which utilizes a selfsupervised loss together with the cross entropy loss to optimize the graph structure.

Revised: However, the default setting in SLAPS is generating structure for raw data points (with no given graph structure).

Revised: Hence, using SLAPS for our settings requires considerable changes.

Revised: Furthermore, we highlight two more weaknesses of SLAPS compared to GT RANS .

Revised: (a) Introducing additional parameters .

Revised: SLAPS uses a denoising loss as self-supervision.

Revised: In detail, it first injects noise into node features and trains a denoising autoencoder to denoise the noisy features.

Revised: This introduces additional parameters from

Revised: the denoising autoencoder and inevitably changes the model architecture.

Revised: (b) Not learning features .

Revised: As other GSL methods, SLAPS does not learn node features.

Revised: We argue that feature learning is highly important under the abnormal feature setting as shown in Table 16.

Revised: For example, structure learning only improves GCN by 2% on OGB-Arxiv while feature learning can improve GCN by 20%.

Revised: Thus, without the feature learning component, the performance will significantly drop when encountering noisy features.

Revised: Since GT RANS is highly versatile and we can use any self-supervised loss as the surrogate loss, we can simply replace the contrastive loss in Eq.

Revised: (6) with the denoising loss of SLAPS instead of paying considerable efforts in adjusting SLAPS.

Revised: We refer to the loss used for denoising as SLAPS loss and adopt it for TTGT.

Revised: Note that we first train the parameters of the DAE used for denoising while keeping the pre-trained model fixed.

Revised: Then we fix both DAE and the pre-trained model and optimize the test graph for TTGT.

Revised: The results are shown in Table 11.

Revised: From the table, we can observe that the SLAPS loss (or feature denoising loss) does not work as well as the contrastive loss.

Revised: [Table]

Revised: [Table]

Revised: D. C OMPARISON TO AD-GCL

Revised: Next, we compare our method with a graph contrastive learning method with learnable augmentation

Revised: AD-GCL

Revised: (Suresh et al., 2021).

Revised: Since AD-GCL is originally designed for graph classification as a pre-training strategy, the direct empirical comparison between AD-GCL and GT RANS is not easy.

Revised: However, due to the flexibility of GT RANS , we can integrate AD-GCL into our TTGT framework, denoted as TTGT+AD-GCL.

Revised: We present the empirical results in Table 12.

Revised: We can observe that

Revised: TTGT+AD-GCL generally performs worse than GT RANS except on Amz-Photo , which indicates that GT RANS is a stronger realization of TTGT.

Revised: Furthermore, we highlight some key differences between it and GT RANS .

Revised: (a) AD-GCL requires optimization of a min-max problem which involves parameters of graph structure and model.

Revised: Thus, adopting it for TTGT would change the pre-trained model architecture.

Revised: (b) AD-GCL only augments the graph structure while not learning the features.

Revised: We argue that feature learning is highly important under the abnormal feature setting as shown in Table

Original: To show the performance on individual test graphs, we choose GCN as the backbone model and include the box plot on all test graphs within each dataset in Figure 6.
Revised: For example, structure learning only improves GCN by 2% on OGB-Arxiv while feature learning can improve GCN by 20%. Thus, without the feature learning component, the performance will significantly drop when encountering noisy features (c) According to Eq.

Original: We observe that GT RANS generally improves over each test graph within each dataset, which validates the effectiveness of test-time graph transformation.
Revised: (9) in the AD-GCL paper, it calculates the similarities between all samples within each mini-batch. When we increase the batch size, we would easily get the out-ofmemory issue while a small mini-batch will slow down the learning process. As a consequence, TTGT+AD-GCL is less efficient than GT RANS : the adaptation time of TTGT+AD-GCL on OGB-Arxiv is 12.7s while that of GT RANS is 2.6s.

Original: [Figure]
Revised: [Table]

Original: D. A BNORMAL F EATURES
Revised: D. O UT - OF -D ISTRIBUTION (OOD) S ETTING

Original: For each model [Figure] we present the node classification accuracy on all test nodes (i
Revised: To show the performance on individual test graphs, we choose GCN as the backbone model and include the box plot on all test graphs within each dataset in Figure 7.

Original: [Figure]
Revised: We observe that GT RANS generally improves over each test graph within each dataset, which validates the effectiveness of test-time graph transformation.

Original: e
Revised: T

Original: [Figure]
Revised: [Figure]

Revised: D. A BNORMAL F EATURES

Original: , both normal and abnormal ones) in Figure 7.
Revised: For each model, we present the node classification accuracy on all test nodes (i.e., both normal and abnormal ones) in Figure 8.

Original: GT RANS significantly improves GCN in terms of the performance on all test nodes for all datasets across all noise ratios.
Revised: GT RANS significantly improves GCN in terms of the performance on all test nodes for all datasets across all noise ratios.

Original: For example, on Cora with 30% noisy nodes,
Revised: For example, on Cora with 30% noisy nodes,

Original: GT RANS improves GCN by 31.0% on overall test accuracy.
Revised: GT RANS improves GCN by 31.0% on overall test accuracy.

Original: These results further validate that the proposed GT RANS can produce expressive and generalizable representations [Figure]
Revised: These results further validate that the proposed GT RANS can produce expressive and generalizable representations.

Original: D. I NTERPRETATION ON THE R EFINED G RAPH FOR A DVERSARIAL A TTACK S ETTING
Revised: D. I NTERPRETATION ON THE R EFINED G RAPH FOR A DVERSARIAL A TTACK S ETTING

Original: To understand the modifications made on the graph [Figure] we compare several properties among clean graph, attacked graph (20% perturbation rate), graph obtained by GCNJaccard
Revised: To understand the modifications made on the graph, we compare several properties among clean graph, attacked graph (20% perturbation rate), graph obtained by GCNJaccard, and graph obtained

Original: [Figure]
Revised: [Figure]

Original: and graph obtained by GT RANS in Table 7.
Revised: by GT RANS in Table 13.

Original: We follow the definition in (Zhu et al., 2020) to measure homophily; “Pairwise Feature Similarity” is the averaged feature similarity among all pairs of connected nodes; “#Edge+/-” indicates the number of edges that the modified graph adds/deletes compared to the clean graph.
Revised: We follow the definition in (Zhu et al., 2020) to measure homophily; “Pairwise Feature Similarity” is the averaged feature similarity among all pairs of connected nodes; “#Edge+/-” indicates the number of edges that the modified graph adds/deletes compared to the clean graph.

Original: From Table 7, we observe that first, adversarial attack decreases homophily and feature similarity, but GT RANS and GCNJaccard promote such information to defend against adversarial patterns.
Revised: From Table 13, we observe that first, adversarial attack decreases homophily and feature similarity, but GT RANS and GCNJaccard promote such information to defend against adversarial patterns.

Original: Second, both GT RANS and GCNJaccard focus on deleting edges from the attacked graph, but GCNJaccard removes a substantially larger amount of edges, which may destroy clean graph structure and lead to suboptimal performance. 
Revised: Second, both GT RANS and GCNJaccard focus on deleting edges from the attacked graph, but GCNJaccard removes a substantially larger amount of edges, which may destroy clean graph structure and lead to suboptimal performance. [Table]

Original: D. A BLATION S TUDY ON S URROGATE L OSS
Revised: D. A BLATION S TUDY ON S URROGATE L OSS

Original: Since we optimized a combined loss in the settings of abnormal features and adversarial attack, we now perform ablation study to examine the effect of each component.
Revised: Since we optimized a combined loss in the settings of abnormal features and adversarial attack, we now perform ablation study to examine the effect of each component.

Original: We choose GCN as the[Table] backbone model and choose 0.3 noise ratio for abnormal features.
Revised: We choose GCN as the backbone model and choose 0.3 noise ratio for abnormal features.

Original: The results for abnormal features and adversarial attack are shown in Tables 8 and 9, respectively.
Revised: The results for abnormal features and adversarial attack are shown in Tables 14 and 15, respectively.

Original: “None” indicates the vanilla GCN without any test-time adaptation and “Combined” indicates jointly optimizing a combination of the two losses.
Revised: “None” indicates the vanilla GCN without any test-time adaptation and “Combined” indicates jointly optimizing a combination of the two losses.

Original: From the two tables, we can conclude that (1) both L s and L train help counteract abnormal features and adversarial attack; and (2) optimizing the combined loss generally outperforms optimizing L s or L train alone.
Revised: From the two tables, we can conclude that (1) both L s and L train help counteract abnormal features and adversarial attack; and (2) optimizing the combined loss generally outperforms optimizing L s or L train alone.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: D. A BLATION S TUDY ON F EATURE L EARNING AND S TRUCTURE L EARNING
Revised: D. A BLATION S TUDY ON F EATURE L EARNING AND S TRUCTURE L EARNING

Original: In this subsection
Revised: In this subsection

Original: [Table]
Revised: [Table]

Original: we investigate the effects of the feature learning component and structure learning component.
Revised: we investigate the effects of the feature learning component and structure learning component.

Original: We show results for abnormal features and adversarial attack in Tables 10 and 11, respectively.
Revised: We show results for abnormal features and adversarial attack in Tables 16 and 17, respectively.

Original: Note that “None” indicates the vanilla GCN without any test-time adaptation; “ A ′ ” or “ X ′ ” is the variants of GT RANS which solely learns structure or node features; “Both” indicates the method GT RANS that learn both structure and node features.
Revised: Note that “None” indicates the vanilla GCN without any test-time adaptation; “ A ′ ” or “ X ′ ” is the variants of GT RANS which solely learns structure or node features; “Both” indicates the method GT RANS that learn both structure and node features.

Original: From Table 10, we observe that (1) while both feature learning and structure learning can improve the vanilla performance, feature learning is more powerful than structure learning; (2) combining them does not seem to further improve the performance but it achieves a comparable performance to sole feature learning.
Revised: From Table [Table] we observe that (1) while both feature learning and structure learning can improve the vanilla performance, feature learning is more powerful than structure learning; (2) combining them does not seem to further improve the performance but it achieves a comparable performance to sole feature learning.

Original: From
Revised: From

Original: Table 11, we observe that (1) while both feature learning and structure learning can improve the vanilla performance, structure learning is more powerful than feature learning; and (2) combining them can further improve the performance.
Revised: Table 17, we observe that (1) while both feature learning and structure learning can improve the vanilla performance, structure learning is more powerful than feature learning; and (2) combining them can further improve the performance.

Original: From these observations, we conclude that (1) feature learning is more crucial for counteracting feature corruption and structure learning is more important for defending structure corruption; and (2) combining them always yields a better or comparable performance.
Revised: From these observations, we conclude that (1) feature learning is more crucial for counteracting feature corruption and structure learning is more important for defending structure corruption; and (2) combining them always yields a better or comparable performance.

Original: While both feature learning and structure learning can improve the vanilla performance, feature learning is more powerful than structure learning.
Revised: While both feature learning and structure learning can improve the vanilla performance, feature learning is more powerful than structure learning.

Original: Combining them does not seem to further improve the performance but it achieves a comparable performance to sole feature learning.
Revised: Combining them does not seem to further improve the performance but it achieves a comparable performance to sole feature learning.

Original: [Table]
Revised: [Table]

Original: While both feature learning and structure learning can improve the vanilla performance, structure learning is more powerful than feature learning.
Revised: Param r =5% r =10% r =15% r =20% r =25% None 57.47±0. 47.97±0.

Original: Combining them can further improve the performance. [Table] A [Table]
Revised: 38.04±1.

Original: [Table] [Table] [Table] [Table]
Revised: 29.05±0.

Original: [Table] Both
Revised: 19.58±2.

Original: [Table] [Table] [Table]
Revised: X ′

Original: [Table]
Revised: [Table]

Original: D. C OMPARING D IFFERENT S ELF -S UPERVISED S IGNALS
Revised: D. C OMPARING D IFFERENT S ELF -S UPERVISED S IGNALS

Original: As there can be other choices to guide our test-time graph transformation process, we examine the effects of other self-supervised signals.
Revised: As there can be other choices to guide our test-time graph transformation process, we examine the effects of other self-supervised signals.

Original: We choose the OOD setting to perform experiments and consider the following two parameter-free self-supervised loss:
Revised: We choose the OOD setting to perform experiments and consider the following two parameter-free self-supervised loss:

Original: (a) Reconstruction Loss.
Revised: (a) Reconstruction Loss.

Original: Data reconstruction is considered as a good self-supervised signal and we can adopt link reconstruction (Kipf & Welling, 2016b) as the guidance.
Revised: Data reconstruction is considered as a good self-supervised signal and we can adopt link reconstruction (Kipf & Welling, 2016b) as the guidance.

Original: Minimizing the reconstruction loss is equivalent to maximizing the similarity for connected nodes, which encourages the connected nodes to have similar representations.
Revised: Minimizing the reconstruction loss is equivalent to maximizing the similarity for connected nodes, which encourages the connected nodes to have similar representations.

Original: (b) Entropy Loss.
Revised: (b) Entropy Loss.

Original: Entropy loss calculates the entropy of the model prediction.
Revised: Entropy loss calculates the entropy of the model prediction.

Original: Minimizing the entropy can force the model to be certain about the prediction.
Revised: Minimizing the entropy can force the model to be certain about the prediction.

Original: It has been demonstrated effective in Tent (Wang et al., 2021) when adapting batch normalization parameters.
Revised: It has been demonstrated effective in Tent (Wang et al., 2021) when adapting batch normalization parameters.

Original:  We summarize the results in Table 12.
Revised: (c) SLAPS Loss. SLAPS (Fatemi et al., 2021) utilizes self-supervision to guide the graph structure learning process. Specifically, it injects random noise into node features and employs a denoising autoencoder (DAE) to denoise the node features. We refer to the loss used for denoising as SLAPS loss and adopt it for TTGT.

Revised: Note that we first train the parameters of the DAE used for denoising while keeping the pre-trained model fixed.

Revised: Then we fix both DAE and the pre-trained model and optimize the test graph for TTGT.

Original:  From the table, we observe that in most of the cases, the above two losses underperforms our proposed surrogate loss and even degrade the vanilla performance.
Revised: We summarize the results in Table 18. From the table, we observe that in most of the cases, the above three losses underperform our proposed surrogate loss and even degrade the vanilla performance.

Original: It validates the effectiveness of our proposed loss in guiding the test-time graph transformation.
Revised: It validates the effectiveness of our contrastive loss in guiding the test-time graph transformation.

Original: [Table]
Revised: [Table]

Original: None 93.79±0. 91.59±1. 50.90±1.
Revised: 91.54±1. 49.93±1.

Original: 54.04±0.
Revised: 54.29±0.

Original: 38.59±1.
Revised: 41.11±2.

Original: 59.89±0.
Revised: 59.48±0.

Original: Recon 93.77±1.
Revised: SLAPS 93.97±1.

Original: 91.37±1.
Revised: 91.41±1.

Original: 49.33±1.
Revised: 50.54±1. 54.08±0. 41.38±1. 59.85±0.

Original: 53.94±1.
Revised: L s in Eq. (6) 94.13±0. 94.66±0. 55.88±3.

Original: 44.93±4.
Revised: 54.32±0. 41.59±1. 60.42±0. Gradient Correlation.

Original: [Table]
Revised: In Figure 2, we have empirically verified the effectiveness of Theoremwhen adopting the surrogate loss in Eq. (6) as L s . We further plot the values of ρ ( G ) with different surrogate losses (i.e., entropy, reconstruction and SLAPS) and L c on one test graph in Cora in Figure 9.

Revised: We can observe that a positive ρ ( G ) generally reduces the test classification loss.

Revised: For example, when using entropy loss, the test loss generally reduces when ρ ( G ) is positive and starts to increase after ρ ( G ) becomes negative.

Revised: [Figure]

Revised: when adopting different surrogate losses.

Revised: D. S ENSITIVITY TO H YPER -P ARAMETER B

Revised: In this subsection, we examine the sensitivity of GT RANS ’ performance with respect to the perturbation budget, i.e., hyper-parameter B .

Revised: Specifically, we vary the value of B in the range of { 0 . 5% , 1% , 5% , 10% , 20% , 30% } and perform experiments on the OGB-Arxiv dataset for the three settings in Table 19.

Revised: Specifically, “Abn.

Revised: Feat” stands for abnormal feature setting with 30% noisy feature while “Adv.

Revised: Attack” stands for the adversarial attack setting with 20% perturbation rate.

Revised: From the table, we can observe budget B has a smaller effect on OOD and abnormal feature settings while highly impacting the performance under structural adversarial attack.

Revised: This is because most of the changes made by adversarial attack are edge injections as shown in Table 13, and we need to use a large budget B to remove adversarial patterns.

Revised: By contrast, GT RANS is much less sensitive to the value of B in the other two settings.

Revised: [Table]

Revised: In Eq.

Revised: [Table]

Revised: we used DropEdge as the augmentation function A ( · ) to obtain the augmented view.

Revised: In practice

Revised: [Table]

Revised: the choice of augmentation can be flexible and here we explore two other choices: node dropping (You et al., 2020) and subgraph sampling (Zhu et al., 2021c).

Revised: We perform experiments on OOD setting with GCN as the backbone model and report the results in Table 20.

Revised: Specifically, we adopt a ratio of 0.05 for node dropping, and ratios of 0.05 and 0.5 for DropEdge.

Revised: From the table

Revised: [Table]

Revised: we can observe that (1) GT RANS with any of the three augmentations can greatly improve the performance of GCN under distribution shift

Revised: [Table]

Revised: and (2) different augmentations lead to slightly different performance on different datasets.

Revised: Augmentation

Revised: [Table]

Revised: E F URTHER D ISCUSSION E. D ISCUSSION ON C ONTRASTIVE L OSS

Revised: The majority of graph contrastive learning methods require a projection head to transform the hidden representations (Velickovic et al., 2019; You et al., 2021; Zhu et al., 2021b).

Revised: It is necessary to remove the projection head in test-time graph transformation so as to keep the pre-trained architecture unchanged.

Revised: Indeed, we can replace Eq.

Revised: (6) with JSD loss (without using the parameterized transformation)

Revised: and we can obtain the results in Table 21.

Revised: Notably, our core contribution in the surrogate loss section does not lie in the parameter-free surrogate loss.

Revised: Instead, we aim to provide theoretical insights on how to choose a suitable loss for test-time graph transformation (TTGT).

Revised: Our analysis shows that a good surrogate loss should share relevant information with the downstream task (their gradients should have a positive correlation).

Revised: Such analysis inspires us to investigate the potential of contrastive learning in the TTGT problem, since contrastive learning has shown enormous success in graph representation learning.

Revised: Moreover, the proposed GT RANS is highly composable in that we can try any self-supervised losses but remove the additional parameters for projection.

Revised: For example, in Appendix D.

Revised: [Table]

Revised: we examined two more self-supervised losses and found that they are less useful than the contrastive loss for the TTGT problem.

Original: [Table]
Revised: [Table]