TFGDA: Exploring Topology and Feature Alignment in Semi-supervised Graph Domain Adaptation through Robust Clustering

Authors: jun  dan
weiming  liu
chunfeng  xie
hua  yu
shunjie  dong
yanchao  tan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-24 08:52:44

Abstract: Semi-supervised graph domain adaptation, as a branch of graph transfer learning, aims to annotate unlabeled target graph nodes by utilizing transferable knowledge learned from a label-scarce source graph. However, most existing studies primarily concentrate on aligning feature distributions directly to extract domain-invariant features, while ignoring the utilization of the intrinsic structure information in graphs. Inspired by the significance of data structure information in enhancing models' generalization performance, this paper aims to investigate how to leverage the structure information to assist graph transfer learning. To this end, we propose an innovative framework called TFGDA. Specially, TFGDA employs a structure alignment strategy named STSA to encode graphs' topological structure information into the latent space, greatly facilitating the learning of transferable features. To achieve a stable alignment of feature distributions, we also introduce a SDA strategy to mitigate domain discrepancy on the sphere. Moreover, to address the overfitting issue caused by label scarcity, a simple but effective RNC strategy is devised to guide the discriminative clustering of unlabeled nodes. Experiments on various benchmarks demonstrate the superiority of TFGDA over SOTA methods.

Main paper: 
 introduction: 
 With the rise of deep learning, node classification techniques have made significant progress in diverse fields. However, due to the distribution shift issue, well-trained models often suffer severe performance degradation when applied directly to a new domain. Graph transfer learning (GTL) [1] has been proposed to tackle this issue by transferring domain-invariant features from a labeled source graph to an unlabeled target graph, effectively boosting the model's performance on the target graph.Although current studies on GTL have made significant strides, they often rely on the assumption that all nodes in the source graph are labeled. However, this ideal assumption does not hold true in many scenarios, as annotating the entire source graph is time-consuming, especially for large-scale networks. Therefore, in this paper, we focus on a more realistic application scenario known as semi-supervised graph domain adaptation (SGDA) [2] , where the source graph only has a few labeled nodes. Utilizing the transferable knowledge acquired from the label-scarce source graph to enhance the model's adaptation performance on the target graph is the most crucial challenge for SGDA. However, most existing studies [3, 4, 1] tend to focus on directly aligning feature distributions across domains to extract domain-invariant node features, while overlooking the utilization of the intrinsic structure information in graphs. Notably, the recent advancements in unsupervised learning [5, 6, 7, 8] have showcased the significance of data structure information in enhancing models' generalization. Considering the complex topological structure information presented in the graph, this paper seek to leverage these critical structure information to assist the transfer of shared node knowledge. To this end, we propose an innovative SGDA framework named TFGDA that employs a Subgraph Topological Structure Alignment (STSA) strategy to encode structure information into latent space. Specially, STSA utilizes persistent homology (PH) [9] to extract the topological structure information of the input and latent spaces, and then align the topological structures of two space, significantly improving the model's transfer performance. Furthermore, current works [2, 10, 4] primarily utilize adversarial training to reduce domain discrepancy. However, adversarial training is an unstable process that may destroy the discriminative details hidden in features [11] , thereby affecting the transfer of shared knowledge. To remedy this issue, we propose a Sphere-guided Domain Alignment (SDA) strategy, aiming to achieve more stable domain alignment. Concretely, SDA initially maps node features to the spherical space. Then, geodesic projection [12] is utilized to project spherical features onto multiple great circles, where the spherical sliced-Wasserstein (SSW) distance [13] is employed to quantify the feature distributions discrepancy across domains.More importantly, in the SGDA scenarios, due to the label scarcity of source graph, well-trained model on only a few labeled source nodes is likely to encounter overfitting. Consequently, it may make ambiguous or even incorrect predictions for certain target graph nodes located near the decision boundaries or far from their corresponding class centers. To address this overfitting issue, we devise a Robustness-guided Node Clustering (RNC) strategy to effectively enhance the model's robustness. RNC aims to guide the discriminative clustering of unlabeled nodes by maximizing the mutual information between the soft cluster assignment of the original node and its perturbed version, significantly improving the model's generalization performance on the target graph.In summary, the main contributions are listed as follows:(1) To the best of our knowledge, this is first attempt to utilize the intrinsic topological structure information hidden in graphs to assist GTL. A novel STSA strategy is proposed to preserve the topological structure information in latent space.(2) A strategy named SDA is introduced to stably align the node feature distributions across domains.(3) To address the overfitting issue, a simple but effective RNC strategy is devised to guide the discriminative clustering of unlabeled nodes.(4) Experimental results show that our TFGDA outperforms SOTA methods on various benchmarks. 
related works: 
 Graph Transfer Learning. GTL [14, 15] has gained widespread attention for relieving the burden of collecting labeled data for new tasks. Early studies usually use source nodes to pre-train expressive models for related tasks in target domain [16, 17, 18, 19, 20] . To enhance model's generalization, recent studies have shifted their emphasis to domain adaptation [10, 21] . There are two main ways to extract domain-invariant node features: (1) Using adversarial training to enforce domain confusion [15, 4, 2, 22] ; (2) Minimizing the statistical distance between two domains [23, 24, 25, 26, 27] . However, existing works tend to focus on domain alignment while overlooking the utilization of structure information in graphs.Semi-supervised Learning on Graphs. Semi-supervised learning on graphs addresses node classification with a small fraction of labeled nodes. Previous works [28, 29, 30] commonly adopt the message passing paradigm to extract discriminative features. Recent studies have explored various techniques, including adversarial training [31, 32] , data augmentation [33, 34] , continuous graph [35] , contrastive learning [25] and meta learning [36, 37] to further enhance model's generalization.PH is an essential method in topology for extracting structure information from point clouds [38, 39] . Recently, PH has shown significant advantages in various areas, including signal processing [40] , shape matching [41] , and design of network [42] . Some studies have investigated its differentiability [43, 44] . Recent works have also explored its potential in image segmentation [45] , action/image recognition [46, 47, 48] , and evaluation of GANs [49] . 
preliminaries: persistent homology (ph): 
 PH is a method used to capture the topological structure of complex point clouds as a scale parameter ρ is varied. In this section, we briefly introduce some key concepts. Further details on PH can be found in [9, 38] .Notation. X := {x i } m i=1 denotes a point cloud and ω is a distance metric over X . Vietoris-Rips (VR) Complex. The VR complex [50] is a unique simplicial complex constructed from a set of points, providing an approximation of the underlying space's topology. The VR complex of X at scale ρ, denoted as V ρ (X ), contains all simplices of X . Each component of X satisfies the distance constraint: ω(x i , x j ) ≤ ρ for any i, j. Additionally, the VR complex exhibits a nesting property: V ρi ⊆ V ρj for any ρ i ≤ ρ j , which enables us to track the evolution progress of simplical complex as ρ increases.The PD dgm is a multi-set of points (g 1 , g 2 ) in the Cartesian plane R 2 , encoding lifespan information of topological features. Concretely, it summarizes the birth time g 1 and death time g 2 information of each topological feature with a homology group. The birth time g 1 indicates the scale of feature creation and death time g 2 refers to the scale of feature destruction. 
problem definition: 
 Source Domain Graph: The source graph is defined as G s = (V s,l , V s,u , Y s,l , A s , X s ), where V s,l is the labeled node set, and V s,u is the remaining unlabeled node set in G s . Y s,l ∈ R |V s,l |×K denotes the label matrix of V s,l , where K is the number of node classes. If a node n s i ∈ V s,l belongs to the k-th class,y s i,k = 1; otherwise, y s i,k = 0. A s ∈ R N s ×N s is an adjacency matrix, where N s = |V s,l | + |V s,u | is the number of nodes in G s .If there is an edge between nodes n i and n j , the value of A s ij is set to 1; otherwise, it is set to 0. X s ∈ R N s ×e indicates an attribute matrix, where e is the dimension of node attributes. Notably, |V s,l | is much smaller than |V s,u | in the SGDA setting.Target Domain Graph: Similarly, the target graph is represented as G t = (V t , A t , X t ), which is a completely unlabeled graph with an unlabeled node set V t . A t ∈ R N t ×N t is an adjacency matrix, and X t ∈ R N t ×e is a node attribute matrix, where N t = |V t | denotes the number of nodes in G t .Semi-Supervised Graph Domain Adaptation (SGDA): Given a partially labeled source graph G s and an unlabeled target graph G t , the goal of SGDA is to precisely annotate target graph nodes by utilizing transferable knowledge learned from the limited labeled source nodes [2, 15] .The architecture of our TFGDA model is depicted in Figure 1 . It consists of two components: a graph convolutional network (GCN)-based feature extractor F and a node classifier C. Mathematically, given an input graph G = (V, A, X), the node features extracted by F is denoted as H = F(G) ∈ R |V|×d , and it is further normalized to map onto a spherical space S d-1 r to obtain spherical features Z ∈ R |V|×d , where d is the feature dimension, r is the radius and |V| denotes the number of nodes in G. The classification probability predicted by C is denoted asΨ = C(Z) ∈ R |V|×K .To capture more precise adjacency relationships of graph G, we compute the positive point-wise mutual information (PPIM) between nodes following [2] . Specially, for a given graph G = (V, A, X), we utilize random walk to sample a collection of paths on A and generate a frequency matrix R.Based on R, we can compute the PPIM matrix P as follows: where P ij denotes the positive mutual information between nodes n i and n j , which quantifies the topological proximity between nodes. A higher value of P ij indicates a strong connection between n i and n j . Then, the output of the l-th GCN layer Conv (l) (•) is defined as:EQUATIONEQUATIONwhere D is the diagonal degree matrix of P , and P = P + I (I is a identity matrix). W (l) refer to the trainable parameters of the l-th layer, σ(•) is an activation function, and H (0) = X. Thus, the feature extractor F can be constructed by sequentially stacking L layers of GCN Conv (l) (l = 1, 2, • • • , L).Given the source labeled node set V s,l , the classification loss on the source graph G s is defined as:EQUATIONwhere L ce represents the standard cross-entropy loss.As mentioned in Section 1, the graph consists of numerous nodes (e.g., ACMv9 has over 9000 nodes), thereby containing rich structure information. Inspired by the significance of data structure information in enhancing models' generalization [51, 48, 6] , we seek to leverage such critical structure information to facilitate the learning of domain-invariant features.To achieve this goal, we treat the graph as a point cloud and attempt to directly capture its underlying topological structure using PH. However, we inevitably encounter huge computational burden due to the complex attributes and adjacency relationships of graph. Fortunately, Refs. [52, 53] have indicated that the properties of graph can be well preserved in its multiple local subgraphs. To this end, we propose a Subgraph Topological Structure Alignment (STSA) strategy to encode structure information of input space into the latent space. For a given graph G = (V, A, X), STSA first sample a subgraphs Ĝ1 , Ĝ2 , • • • , Ĝa using random walk, and then employ PH to capture the intrinsic topological structure information of each subgraph Ĝi .Specially, for each subgraph Ĝi , we represent its spherical features extracted by F as Ẑi ∈ R | Vi|×d , which are obtained by indexing the local nodes features from the complete nodes features Z ∈ R |V|×d .Here | Vi | denotes the number of nodes in each subgraph. Then we construct the VR complexes V ρ ( Ĝi ) and V ρ ( Ẑi ) for point clouds Ĝi and Ẑi , employ PH to extract their topological structures, and obtain their corresponding PDs dgm( Ĝi ) and dgm( Ẑi ) respectively. To align the topological structures of the input and latent spherical spaces, we adopt the 1-Wasserstein distance W 1 to measure the discrepancy between two PDs (i.e., dgm( Ĝi ) for input space and dgm( Ẑi ) for spherical space), which aims to seek the optimal transport plan γ * between two PDs:EQUATIONwhere α ∈ dgm( Ĝi ), β ∈ dgm( Ẑi ), and ∥•∥ ∞ is the l ∞ distance. After obtaining γ * , the local structure discrepancy L sub str ( Ĝi ) between two spaces of Ĝi can be calculated as:EQUATIONHence, we can estimate the global structure discrepancy between the input and spherical spaces of graph G by aggregating the local structure discrepancy of all subgraphs:EQUATIONIn the SGDA scenario, the STSA strategy will be applied to both the G s and G t , and its loss function is defined as:EQUATIONNotably, preserving topological structure information has the potential to guide unlabeled nodes towards achieving discriminative clustering, thereby promoting the learning of transferable node features, as verified in Section 5.3. Although many GCNs-based methods have already been proposed to exploit graph structure information to promote the learning of features, these methods are not effective in addressing the SGDA task. Specifically, these GCNs-based methods [54, 55, 56, 57] typically mine the graph structure information in the deep feature space by designing well-crafted GCN architectures or introducing some complex modules. However, recent studies [25, 58, 59, 15] have pointed out that GCNs are insufficient in capturing the sophisticated structure information in graph, which means that the graph structure information may be lost or destroyed after passing through the GCNs-based feature extractors. Thus, directly mining graph structure information from the deep feature space is a suboptimal way, which affects the learning of transferable node features in our SGDA setting.The proposed STSA strategy aims to extract the graph structure information directly from the input space and encode these powerful information into the latent spherical space by aligning the topological structures of the two spaces. This method does not lose or destroy the graph structure information during training. Furthermore, our STSA strategy does not introduce any changes to the network architecture, effectively avoiding an increase in model's complexity and ensuring its adaptability to integration with other methods.As mentioned in Section 1, adversarial training has been widely adopted by existing GTL models to reduce domain discrepancy. However, it is an unstable process that may destroy the discriminative information hidden in node features, thereby impacting the learning of shared features.To tackle this issue, we propose a Sphere-guided Domain Alignment (SDA) strategy that achieves stable alignment of cross-domain node features distributions in the spherical space. Our SDA strategy mainly comprises three steps: (1) Map node features onto the sphere space S d-1 r .(2) Use geodesic projection [12] to project node features from S d-1 r to multiple great circles. (3) Compute the feature distributions discrepancy on great circles and minimize it during training.Step (1): Motivated by the effectiveness of spherical features in improving model's transfer performance [60] , we first normalize node features H ∈ R |V|×d extracted by F with z i = r hi ∥hi∥ to obtain the spherical features Z ∈ R |V|×d in the sphere spaceS d-1 r = z i ∈ R d : ∥z i ∥ 2 = r, where h i and z i is the i-th row of H and Z, respectively. Notably, Ref. [61] has proved that a proper radius r is lower bounded by parameters τ and K:EQUATIONwhere τ denotes the expected minimal classification probability of class center and K is the number of classes. In this work, τ is set to 0.999, and radius r is set to the lower bound.Step (2): Previous studies [62, 63, 64, 65] have shown the superiority of optimal transport in aligning feature distributions. Let Ω be a probability space and µ, ν be two probability measures in P(Ω). For any q ≥ 1, the q-Wasserstein distance between µ and ν is defined as:EQUATIONwhereΠ(µ, ν) = {γ ∈ P(Ω × Ω)|π 1# γ = µ, π 2# γ = ν}is the set of couplings, π 1 and π 2 denote the two marginal projections of Ω × Ω to Ω, # denotes the push-forward operator, and M : Ω × Ω → R + is a geodesic metric. However, we find that directly using the classical Wasserstein distance to compute features distributions discrepancy on the sphere S d-1 r is computationally expensive, due to numerous nodes in the graph.For more efficient calculations, we utilize geodesic projection I U to project node features lying onS d-1 r to b great circles {C 1 , C 2 , • • • , C b }.On the hypersphere, great circles [66] are circles whose diameter is equal to that of the sphere, and they correspond to the geodesics. Specially, the geodesic projection I U is determined by U :EQUATION) where M S d-1 r (z, g) = arccos(⟨z, g⟩), and V d,2 = U ∈ R d×d , U ⊤ U = I 2 is the Stiefel manifold [67].Step (3): Next, we utilize the spherical sliced-Wasserstein (SSW) distance [13] to measure the feature distributions discrepancy of two domain on multiple great circles, which can be formulated as:EQUATION)where σ is the uniform distribution over V d,2 . In SSW, the geodesic metric M q (z, g) in Wasserstein distance W q q (µ, ν) is defined as the geodesic distance [66] M S 1 r (z, g) = min(|z -g|, r -|z -g|). In practice, it's common to approximate the source and target feature distributions using samples (z s i ) N s i=1 and (z t j ) N t j=1 (i.e., through the empirical approximations μ = 1N s N s i=1 δ z s i and ν = 1 N t N t j=1 δ z t j ), where δ is the Dirac function. As a result, the node features distributions discrepancy between two domains can be measured as:EQUATION)where b is the number of projections. As training progresses, SDA strategy gradually reduces domain discrepancy, making the learning of domain-invariant features easier.Due to the label scarcity in G s , the model is prone to overfitting when solely relying on L cls for optimization, severely degrading the model's generalization performance on G t . To alleviate this overfitting issue, we devise a novel Robustness-guided Node Clustering (RNC) strategy to enhance the model's robustness by guiding the discriminative clustering of unlabeled nodes. Specially, RNC first introduces trainable shift parameters ξ s = ξEQUATIONEQUATIONt , • • • , ξ (L) tto perturbs the source and target node features respectively, at each layer of F:EQUATIONwhere H s,ξs,(l) and H t,ξt,(l) denotes the perturbed source and target node features encoded by Conv (l) respectively, and each ξ(i)s/t is specific to the output of Conv (l) . The shift parameters ξ s and ξ t are defined as randomly initialized multi-layer parameter matrices. After perturbation, based on H s,ξs,(L) and H t,ξt,(L) , we can obtain the perturbed source node spherical features Z s,ξs and the perturbed target node spherical features Z t,ξt respectively.For brevity, we will omit domain-specific notations in the following text. Ideally, regardless of how the node feature is perturbed, the model's prediction for it should remain unchanged because its class label has not changed. To achieve this goal, RNC aims to maximize the mutual information between the soft cluster assignment (i.e., classification prediction of classifier C) of the spherical features Z and its perturbed version Z ξ , capturing their intrinsic invariant information between Z and Z ξ .Concretely, given a node spherical feature z i , the classification probability predicted by C is denoted as Ψ(z i ) ∈ R K , that can be viewed as the distribution of a discrete random variable ϕ over K classes:Γ (ϕ = k|z) = Ψ k (z i ).Let ϕ and ϕ ξ denote the cluster assignment variables of z i and z ξ i , respectively. Then, their conditional joint distribution is defined as:Γ (ϕ = k, ϕ ξ = k ξ |z i , z ξ i ) = Ψ k (z i ) • Ψ k ξ (z ξ i ).After marginalization, the joint probability distribution can be formulated as a matrix Q ∈ R K×K :EQUATIONwhereQ kk ξ = Γ (ϕ = k, ϕ ξ = k ξ ), Q k = Γ (ϕ = k), and Q k ξ = Γ (ϕ ξ = k ξ ).To preserve the equivalence between pairs (z i , z ξ i ) and (z ξ i , z i ), matrix Q is typically symmetrized using (Q + Q ⊤ )/2. In this way, the mutual information [68, 69] between the soft cluster assignment of Z and Z ξ can be computed as:EQUATION)where ϖ is a coefficient that controls the scale of feature perturbation.In the SGDA scenario, all target domain nodes V t and source domain nodes V s,l ∪ V s,u are used to calculate matrices Q t and Q s , respectively. Therefore, the objective function L rnc of RNC can be expressed as:EQUATIONNotably, we leverage source labeled nodes V s,l in RNC as they can significantly guide the discriminative clustering of unlabeled nodes in the right direction. With the help of RNC, more and more intrinsic invariant features of nodes are extracted, which greatly promotes the learning of transferable features. Moreover, unlike previous studies that employ pseudo-labels strategy [2] or conditional entropy term [70] to guide the learning of unlabeled nodes, our RNC strategy does not involve any pseudo-labels and naturally avoids degenerate clustering solutions (see Figure 3 for further analysis).In summary, the total objective of TFGDA can be expressed as follows:EQUATION)where hyper-parameters η, ε and λ are used to balance the contributions of the corresponding term.The theoretical analysis of our method is based on the theory of domain adaptation (DA) [71, 72] .Formally, let H be the hypothesis space. Given two domains S and T , the probabilistic bound of error of hypothesis h on the target domain is defined as: ψ T (h) ≤ ψ S (h) + * = min h ′ ∈H ψ S (h ′ ) + ψ T (h ′ ).The goal of DA is to lower the upper bound of the expected target domain error ψ T (h). Note that in unsupervised domain adaptation (UDA), minimizing ψ S (h) can be easily achieved with source label information, as source domain samples are completely annotated. However, in our SGDA setting, due to the label scarcity of source domain, the model is prone to overfitting when solely relying on the source domain classification loss L cls for optimization. Therefore, we introduce the RNC strategy (L rnc ) to address the overfitting issue, with the aim of guiding ψ S (h) towards further minimization.Most DA methods mainly focus on reducing the domain discrepancy d H∆H (S, T ), such as utilizing techniques like adversarial learning, MMD, optimal transport [73, 74, 75, 76] , and CORAL. In comparison to these methods, our SDA strategy (L sda ) effectively eliminates the feature norm discrepancy in spherical space S d-1 r and guide a more stable alignment of feature distributions. Furthermore, considering that graph data contains rich structure information that encodes complex relationships among nodes and edges, and existing GTL methods usually adopt GCNs-based feature extractors to learn domain-invariant node features. However, recent studies [25, 58, 59, 15] have pointed out that GCNs are insufficient in capturing the sophisticated structure information in graph, which seriously affects the transfer of domain-invariant knowledge and consequently limits the model's generalization ability. To solve this problem, we thus propose the STSA strategy (L stsa ) to align the topological structures of the input space and the spherical space, in order to facilitate the GCNs-based feature extractors to capture more domain-invariant node features. Consequently, the combination of SDA (L sda ) and STSA (L stsa ) strategies further promotes the minimization of the domain discrepancy d H∆H (S, T ).Notably, µ * is expected to be extremely small, and therefore it is often neglected by previous methods. However, it is possible that µ * tends to be large when the cross-domain category distributions are not well aligned [77, 78] . In this paper, we leverage the RNC strategy (L rnc ) to guide both labeled nodes and unlabeled nodes toward achieving robust clustering, effectively promoting the fine-grained alignment of category distributions and ensuring that µ * remains at a relatively small value.In summary, our proposed method not only minimizes the source expected error ψ S (h) and domain discrepancy d H∆H (S, T ), but also keeps µ * at a small value, thereby ensuring a low upper bound. 
setup: 
 Datasets. Our experiments involve three real-world graphs: ACMv9 (A), Citationv1 (C), and DBLPv7 (D), obtained from ArnetMiner [79] . Since these graphs have varying sets of node attributes, we union their attribute sets and adjust the attribute dimension to 6775 following [2] . Each node is assigned a five-class label, determined by its relevant research areas. Six typical transfer tasks are considered in our experiments: A→C, A→D, C→A, C→D, D→A and D→C. Due to the page size limitation, more settings and implementation details are placed on Appendix.Compared Methods. We compare TFGDA with several SOTA (1) graph semi-supervised learning methods and (2) graph domain adaptation methods as Ref. [2] : (1) GCN [28] , GSAGE [29] , GAT [30] , GIN [80] , (2) DANN [81] , CDAN [11] , UDA-GCN [70] , AdaGCN [1] , CoCo [25] , StruRW [21] and SGDA [2] . DANN GCN and CDAN GCN are two variants that adopt GCN-based feature extractor.Following [2] , to showcase the superiority of our TFGDA, we report its performance on the challenging scenario, where only 5% of the nodes in the source graph are labeled. Micro-F1 and Macro-F1 are employed as evaluation metrics, and the classification results on the target graph are gathered in Table 1 . For all transfer tasks, we run each experiment 5 times and record the average accuracy with standard deviation. And we sample different label sets each time to mitigate the randomness. As can be seen, our model obtains the overall best results on all transfer tasks. Specially, TFGDA greatly surpasses the SOTA method SGDA [2] by +7.3% and +10.0% on "Micro-F1" and "Macro-F1" respectively for the C→A task, implying the superiority in extracting domain-invariant features. Notably, TFGDA enhances performance substantially on two hard transfer tasks, C→A and D→A, and achieves outstanding results in some easy transfer scenarios, such as A→C and D→C, implying that TFGDA can successfully minimize domain discrepancy. More importantly, the results with a smaller fluctuation range indicate not only the stability of our framework, but also its robustness in the face of different scenarios. Furthermore, we find that most competitors, such as CDAN GCN , UDA-GCN, and AdaGCN, exhibit poor performance due to their limited ability to align domains and ineffective utilization of unlabeled nodes. In contrast, TFGDA effectively solves these challenges.Due to the limitation of page size, more experiments and analysis are placed on the Appendix.Table 2 : Variants of TFGDA.Variant L cls L stsa L sda L rnc TFGDA-S ✓ TFGDA-T ✓ ✓ TFGDA-D ✓ ✓ TFGDA-R ✓ ✓ TFGDA-TD ✓ ✓ ✓ TFGDA-TR ✓ ✓ ✓ TFGDA-DR ✓ ✓ ✓ TFGDA ✓ ✓ ✓ ✓To investigate the contribution of each component in TFGDA, we compare TFGDA and its 7 variants on various tasks. The variants of TFGDA are shown in Table 2 , and the detailed ablation study results are gathered in Table 1 .The results in Table 1 reflect the following observations: (1) TFGDA-S (baseline) performs poorly on all tasks because it encounters overfitting problem.(2) Compared to TFGDA-S, variants TFGDA-T, TFGDA-D and TFGDA-R achieve significant performance gains, indicating that preserving topological structure information, reducing domain discrepancy on the sphere, and guiding discriminative clustering of unlabeled nodes all facilitate the learning of domain-invariant node features.As shown in Table 1 , the combination of different strategies improves the model's transfer performance, implying a clear complementary relationship among the STSA, SDA, and RNC strategies.To show the superior transfer ability of our model, we employ t-SNE to visualize node features on task A→C under the same 5% label rate setting, as depicted in Figure 2 . Although the SOTA method SGDA reduces domain discrepancy to a certain, there are some overlaps between different clusters, causing some hard-to-transfer nodes to be easily misclassfied. In comparison, TFGDA achieves exactly 5 clusters with clean decision boundaries, indicating that our model can capture more fine-grained transferable features as well as align more complex distributions.3) Effect of STSA: To showcase the effectiveness of preserving topological structure information in assisting GTL, we conduct in-depth experiments from both quantitative and visual aspects:(1)As depicted in Table 1 , TFGDA-TD and TFGDA-TR greatly outperform TFGDA-D and TFGDA-R respectively, indicating that aligning the topological structures of the input and latent spaces can effectively boost the model's generalization. (2) As shown in Figure 2 , compared to TFGDA-R, variant TFGDA-TR exhibits better intra-class compactness and inter-class separability in feature space, implying that the incorporation of topological structure information helps guide the discriminative clustering of unlabeled nodes, thereby promoting the learning of domain-invariant nodes features.To show the effectiveness of our RNC strategy, we compare it with existing node clustering strategies, including conditional entropy minimization strategy (CEM) [70] and the recently proposed re-weighted pseudo-labeling strategy (RPL) [2] . We investigate the trend of Micro-F1 score during training on tasks A→C and A→D. The curves in Figure 3 reflect the following observations:(1) TFGDA-R converges more smoothly and quickly, achieving higher transfer performance, which suggests that RNC strategy can effectively accelerate the learning of domain-invariant features.(2) Compared to TFGDA-S (baseline), TFGDA-S + CEM suffers from severe performance degradation, as CEM may enforce over-confident probability on some misclassified unlabeled nodes. (3) RPL strategy fails to achieve satisfactory performance as it's sensitive to pseudo-label noise.To verify the model's robustness under different label scarcity settings, we evaluate the performance of different methods on tasks A→C and A→D, using the following label rates for the source graph: 1%, 5%, 7%, 9%, and 10% respectively, as shown in Figure 4 . It can be observed that our TFGDA significantly outperforms other competitors, even in the most challenging environment of 1% label rate, indicating the superiority of TFGDA in capturing transferable features. 
conclusion: 
 In this paper, we develop a novel model named TFGDA for SGDA. Specially, we propose a STSA strategy to encode critical structure information into latent space, significantly improving model's transfer performance. Moreover, to stably reduce domain discrepancy, the SDA strategy is introduced to align features distributions on sphere. We also devise the RNC strategy to guide the clustering of unlabeled nodes to address the overfitting issue, greatly enhancing the model's robustness.Comprehensive experiments and analysis verify the superiority of our TFGDA. 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: The research conducted in the paper fully adheres to the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: All the datasets involved in this paper are publicly available.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper poses no such risks. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: The data and code used in this paper have obtained legal permissions. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 
 
 Appendix: We run experiments on three real-world graphs: ACMv9 (A), Citationv1 (C), and DBLPv7 (D). In these graphs, every node refers to a paper, and the attribute of each paper is represented as a sparse bag-of-words vector derived from its title. The edges in these graphs depict citation relationships among the papers. Each node is assigned a five-class label, determined by its relevant research areas, including Artificial Intelligence, Computer Vision, Database, Information Security, and Networking.Table 3 displays various statistical information of three graphs, including graph scale, attributes, average degree, and label proportion. We can observe substantial intrinsic discrepancy among these graphs. In this paper, we adopt an alternating approach where we select one of these graphs as the source domain, while considering the remaining two as the target domains. Our experiments are implemented using Pytorch library. Following previous work [2] , we choose a two-layer GCN as the feature extractor F of TFGDA model. We randomly initialize the shift parameter ξ s and ξ t using uniform distributions U s (-ϵ, ϵ) and U t (-ϵ, ϵ), respectively. In all experiments, we set the value of ϵ to 0.5. For all transfer tasks, we perform each random experiment 5 times and record the average accuracy with standard deviation. And we sample different label sets for each experiment to mitigate the randomness.To optimize the network, we employ Adam optimizer with a weight decay of 0.001 for better convergence. The base learning rate is set to 0.002 for all tasks. In terms of the balanced coefficients ε and η, we choose ε = 0.3 and η = 1 for all transfer tasks. Furthermore, instead of fixing the trade-off parameter λ of RNC strategy, we adopt a progressive schedule as [2] to dynamically adjust λ from 0 to 1 by multiplying by θ Θ to more stably guide the discriminative clustering of unlabeled nodes, where θ is the current epoch and Θ is the maximum epoch. Parameter τ in Eq.( 8) is set to 0.999, and the spherical radius r is set to the lower bound.In our STSA strategy, we choose to retain 0-dimensional topological information in PDs. This is because some preliminary experiments have shown that using higher-dimension topological information does not lead to clear accuracy improvements but noticeably increases the model's training time. For the subgraph sampling operation, we set the number of subgraphs to 10 and the number of nodes in each subgraph to 800 to strike a balance between model performance and training efficiency. During the inference process, we disable the shift parameters branch in our RNC strategy. Notably, for all methods (including the compared methods), the dimension of node features is set to 512.A.3.1 More Ablation Study 6) Effectiveness of SDA: To demonstrate the effectiveness of SDA strategy, we compare it with existing domain alignment strategies, including adversarial training alignment strategy (AT) [1] , sliced Wasserstein distance-based alignment strategy (SWD) [82] , class-conditional MMD strategy (CMMD) [3] , and the recently proposed shifting-guided adversarial training alignment strategy (SAT) [2] . We employ variant TFGDA-S as the baseline and evaluate the performance gains brought by these strategies on two typical transfer tasks: A→C and A→D.Macro-F1 (%) The results shown in Figure 5 reflect the following observations: (1) Compared to adversarial trainingbased strategies such as AT and SAT, our SDA strategy can bring greater performance improvements to the model, especially in the hard transfer task A→D. This is because SDA implements a more stable distributions alignment process, which promotes the model to capture more transferable features.(2) The proposed SDA strategy works better than the SWD strategy, as SDA takes into account the manifold structure of the data in the spherical space, while SWD primarily focuses on the distribution discrepancy in the Euclidean space.(3) TFGDA-D significantly outperforms all compared strategies, which indicates the superiority of our SDA strategy in mitigating domain discrepancy and extracting domain-invariant features.The key parameters and notations in our TFGDA model are summarized in Table 4 . (3) When the number a or size | Vi | of the subgraphs is set too small, the model's performance becomes less robust, exhibiting a higher standard deviation. This is because, in such cases, the subgraphs struggle to capture the structure information present in the original graph sufficiently.The constraint coefficient ϖ of shift parameters ξ s and ξ t is responsible for controlling the scale of node feature perturbation. To investigate the effect of coefficient ϖ in our framework, we train our TFGDA models with different ϖ and evaluate their performance on two tasks: A→C and A→D.The results are illustrated in Figure 7 As ϖ increase, the model's accuracy first rises and then falls, which implies that properly perturbing the node feature can effectively enhance the model's robustness.samples are assigned to the same cluster. In such case, maximizing MI naturally achieves a balance between reinforcing predictions and equalizing the cluster masses, thereby avoiding the occurrence of degenerate clustering solutions. The t-SNE node features visualization results in the main text also validate this viewpoint.Our model might has the following two limitations:(i) As depicted in Figure 6 of the Appendix, the effectiveness of our STSA strategy depends on the quality of the subgraphs (i.e., subgraph size | Vi | and the number of subgraphs a). As a result, the model's transfer performance may decrease if the subgraph size is small or the number of subgraphs is small, as these local subgraphs are difficult to sufficiently capture the properties of the original graph.(ii) As depicted in Figure 7 of the Appendix, the transfer ability of the model is affected by the feature perturbation scale ϖ in the RNC strategy. Specially, when the feature perturbation scale ϖ is set to a large value, it is difficult for the RNC strategy to accurately capture the intrinsic invariant features of nodes, which affects the discriminative clustering of unlabeled nodes and leads to a decline in model's generalization performance. Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 


Decision: **Summary**

This paper presents TFGDA, which addresses the challenges in semi-supervised graph domain adaptation. The reviewers provided a generally positive assessment of the paper, noting its novelty, clear motivation, and thorough experimental validation. The proposed methods were recognized for their ability to effectively utilize graph structure information to improve transfer performance, which is a contribution to the field of graph transfer learning. The extensive experiments demonstrated the superiority of TFGDA over existing state-of-the-art methods.

**Decision**

The decision to accept is based on the novelty of leveraging the topological structures of graphs of graph structure information for GDA. 
In the final version, the authors should incorporate the additional experiments conducted during the rebuttal period, particularly those involving the AliCVR dataset, and clarify the distinctions between your approach and existing works. Additionally, ensure that the practical applications of your method, as well as a summary of the SDA strategy in an algorithm table, are clearly presented. These enhancements will further strengthen the paper's contribution and clarity.

Reviews: review: paper_summary: paper_summary: This paper introduces a graph transfer learning framework called TFGDA, which leverages the structure information to enhance model’s generalization performance. The TFGDA framework includes the structure alignment strategy STSA, the feature distribution alignment strategy SDA, and the RNC strategy to address the overfitting caused by label scarcity. Experiments are conducted various benchmarks.
strength_weakness: strength_weakness: 1. This paper proposes to extract spherical features of subgraphs in Subgraph Topological Structure Alignment strategy. Please further explain the reasons for using spherical features and analyze the necessity.

2. The interplay between loss functions could affect the extracted features and model performance. It would be beneficial if the authors could provide a thorough analysis for the combination of loss functions and theoretically explanation of the success.
questions: questions: See Weaknesses
limitations: limitations: The authors should further clarify the motivation behind using spherical features of their proposed method.
paper_summary: paper_summary: This paper presents a framework called TFGDA for semi-supervised graph domain adaptation (SGDA). It addresses the challenge of annotating unlabeled target graph nodes by utilizing knowledge from a source graph with limited labels. The framework incorporates three key strategies: Subgraph Topological Structure Alignment (STSA), Sphere-guided Domain Alignment (SDA), and Robustness-guided Node Clustering (RNC). These strategies collectively aim to encode topological structure information, stably align feature distributions, and guide the clustering of unlabeled nodes, thereby enhancing the model's transfer performance.
strength_weakness: strength_weakness: 1. Limited Innovation: The innovation in the paper is limited. While the authors claim to be the first to consider graph structure, similar approaches have been explored in previous work that also utilize graph structural information.
2. Overly Optimistic Results: The experimental results seem excessively positive, with improvements of 5-10 points on certain datasets, which is highly challenging. The absence of open-source code makes it difficult to verify these results.
3. Lack of Code Availability: The paper does not provide open-source code, which hinders reproducibility and further validation of the proposed methods by the research community.
questions: questions: please refer to Weaknesses
limitations: limitations: please refer to Weaknesses
paper_summary: paper_summary: This paper focus on the semi-supervised graph domain adaptation, and introduces a new framework called TFGDA. Graph usually contains complex structure information, while existing GTL studies often overlooks the importance of structure information when extracting transferable node features. TFGDA thus proposes a novel STSA strategy to utilize the topological structures information between input and latent spaces to assist GTL. To solve the instability caused by adversarial training-based domain adaptation methods, this paper also presents an SDA strategy to reduce cross-domain node feature distributions discrepancy in the spherical space. Furthermore, an innovative mutual information-based RNC strategy is proposed to address the overfitting issue by guiding the robust clustering of unlabeled target graph nodes. Extensive experimental results show that TFGDA outperforms existing state-of-the-art methods across various transfer learning tasks, indicating its superiority and stability.
strength_weakness: strength_weakness: To make this paper more comprehensive, there are some concerns that I would like the authors to address.
(1)	This paper mainly utilizes multiple real-world academic graphs as datasets. Further exploration can be conducted on other types of graph datasets (such as real-world graph datasets), which can more effectively validate the generalizability of the proposed method.
(2)	Furthermore, I recommend the authors to include a discussion and comparison of the model’s inference efficiency in the paper. 
(3)	[Minor comment:] While the t-SNE visualization results in Figure 2 clearly show the advantage of TFGDA in reducing feature distribution discrepancy, I recommend the authors to include additional quantitative metrics to better demonstrate the transfer ability of the method, such as the {A}-distance [1].
(4)	[Minor comment:] Since the SDA strategy contains some complex mathematical details, it is recommended to add a high-level algorithm table to summarize it.

Reference:
[1] Analysis of representations for domain adaptation. (Neurips 2006).
questions: questions: I have some doubts and hope the authors to clarify them.
(1)	When the model performs inference on the target domain graph, is the shift parameter perturbation branch $\xi$ of RNC activated? Does the graph data flow through the regular branch or the perturbation branch before being fed into the classifier \mathcal{C} for inference?

(2)	What is the intrinsic reason for the need to introduce structure information in graph transfer learning frameworks? Is it because the structure information of the data tends to be lost as the network layers go deeper?

(3)	Some previous images based-transfer learning works [2-4] have utilized the wasserstein or the sliced wasserstein as distance metrics to minimize domain discrepancy. In the SDA strategy, why is the SSW distance better than these two distance in measuring domain differences? 

Reference:
[2] Sliced wasserstein discrepancy for unsupervised domain adaptation. (CVPR 2019).
[3] Reliable weighted optimal transport for unsupervised domain adaptation. (CVPR 2020).
[4] Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. (ECCV 2018).
limitations: limitations: The authors have provided clear explanations of the limitations that the proposed method may encounter in the supplementary materials. This article does not have any potential negative societal impact.
paper_summary: paper_summary: This paper proposes a novel framework named TFGDA for the semi-supervised graph domain adaptation. Considering that existing graph domain adaptation works overlook the utilization of graph structure information, this paper innovatively proposes a STSA strategy that fully leverages graph structure information to promote the learning of domain-invariant node features. Moreover, TFGDA also devises a SDA strategy to achieve a fine-grained feature distributions alignment, which maps node features onto multiple great circles and utilizes the spherical sliced-Wasserstein distance to quantify the domain discrepancy. To remedy the overfitting phenomenon caused by limited source labeled nodes, a novel RNC strategy is proposed to guide the discriminative clustering of unlabeled nodes. Comprehensive experiments on various benchmarks demonstrate that the proposed TFGDA outperforms state-of-the-art methods in semi-supervised graph domain adaptation tasks.
strength_weakness: strength_weakness: I have several concerns and questions as below:

[About writing]
1) It is suggested to include some discussions on practical applications of graph transfer learning in the Sec 1, which may better emphasize the significance of the research.
 
[About the proposed method]
2) The proposed STSA strategy employ persistent homology to extract the topological structures of both graphs. In topological theory, topological features of different dimensions capture the structure of the underling space from different perspectives. What dimensions of topological features did the authors use in STSA strategy to align the topological structures of the input space and latent space? How was this determined?

3) What distance metric was used when constructing the Vietoris-Rips simplicial complex in the STSA strategy?

4) Previous domain adaptation works [1] have used the K-means algorithm to guide unlabeled nodes clustering. What advantages does the RNC strategy have compared to K-means? 

[About the experiments]
5) Although the authors conducted detailed experiments on mainstream graph domain adaptation datasets (i.e., ACMv9, Citationv1 and DBLPv7), I hope they discuss the scalability of TFGDA on larger graph datasets, such as the recommender systems dataset AliCVR [2]. 

6) It is suggested to compare and discuss the proposed TFGDA with the recently introduced GIFI method [3].

Ref [1] Deng W, Liao Q, Zhao L, et al. Joint clustering and discriminative feature alignment for unsupervised domain adaptation[J]. IEEE Transactions on Image Processing, 2021, 30: 7842-7855.
Ref [2] Guo G, Wang C, Yan B, et al. Learning adaptive node embeddings across graphs[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 35(6): 6028-6042.
Ref [3] Qiao Z, Xiao M, Guo W, et al. Information filtering and interpolating for semi-supervised graph domain adaptation[J]. Pattern Recognition, 2024, 153: 110498.
questions: questions: See Weakness.
limitations: limitations: See Weakness. If the authors address all my concerns, I am pleased to improve the final score.

Comments: title: Response to Reviewer byDA
Response to Reviewer 9hdx
Looking forward to the reply
Looking forward to the reply
Looking forward to the reply
Response to Reviewer R5oT
my concerns have been well addressed
Discussion period starts
Anonymous link to the source code
comment: Dear Reviewer byDA:

Thank you for your recognition of our work. We appreciate your efforts in reviewing our work and rebuttal. We will incorporate your valuable suggestions into the revised version to ensure that it becomes a high-quality paper.

Best Regards, Authors of 504.
Thank you to the authors for their rebuttal. I have reviewed all comments and the authors’ responses, which effectively addressed my concerns through clear explanations and additional experiments. I recommend incorporating these details into the manuscript or supplementary materials and support the acceptance of this paper.
Dear Reviewer 9hdx:

Thank you for the positive feedback. We appreciate your efforts in reviewing our work. We will reflect your suggestions in the revised version to enable it to be a high-quality paper.

Best Regards, Authors of 504.
Thanks for the authors' responses. I have adjusted my score accordingly, and vote for acceptance of this paper.
Dear Reviewer byDA:

Thanks so much again for the time and effort in our work. According to the comments and concerns, we conduct the corresponding experiments and further discuss the related points. As the discussion period is nearing its end, please feel free to let us know if there are any other concerns. We would be happy to provide more details and clarifications.

Best Regards, Authors of 504.
Dear Reviewer aMJG:

Thanks so much again for the time and effort in reviewing our work. According to your valuable comments and concerns, we provide the source code and further discuss the related points. As the discussion period is nearing its end, we would like to kindly ask the reviewer if our rebuttal has addressed some of the reviewer's questions or concers and if any of them remain unaddressed. We would be happy to provide more details and clarifications.

Best Regards, Authors of 504.
Dear Reviewer 9hdx:

Thanks so much again for the time and effort in reviewing our work. As we are getting closer to the end of the author-reviewer discussion phase, we would like to kindly ask the reviewer if our rebuttal has addressed some of the reviewer's questions or concers and if any of them remain unaddressed. We would be happy to provide more details and clarifications.

Best Regards, Authors of 504.
Dear Reviewer R5oT:

We thank your response and appreciation of our work and rebuttal. We will make sure to incorporate the new results and discussions into our revision to enable it to be a high-quality paper.

Best Regards, Authors of 504.
Thank you to the authors for their detailed response to my questions. After reviewing all the comments and the rebuttal materials, my concerns have been well addressed. This process has further deepened my appreciation for the design and reproducibility of the proposed approach. I have no further questions. I am happy to adjust my rating to reflect my positive feedback.
Dear reviewers,

Thank you for your valuable contributions to the NeurIPS review process! The author-reviewer discussion period has now begun. I’ve noticed that the ratings for this paper are quite dispersed, and opinions among the reviewers are not fully aligned. This makes our discussion even more crucial to reach a consensus and ensure a fair and thorough evaluation. Please engage actively with the authors during this period. If you have any questions or need further clarification on any points, this is the best time to address them directly with the authors. 


best,

AC
Dear AC and all Reviewers:

Based on the valuable suggestion of Reviewer **aMJG**, we provide the source code and pre-trained models of our method to demonstrate the authority of our work. **Code and pre-trained models are available at:** https://anonymous.4open.science/r/TFGDA-003D

We appreciate your time and efforts in reviewing our work.

Best Regards, Authors of 504.