HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual Adapters
Authors: Yujie Mo
Runpeng Yu
Xiaofeng Zhu
Xinchao Wang
Venue: ICLR 2025 Poster (2025)
Keywords: Heterogeneous graph
Pre-trained models
Adapter-tuning
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: This work makes the first attempt to design a unified "pre-train, adapter-tuning'' paradigm to improve the generalization of different pre-trained HGNN models based on a theoretical understanding.

Abstract: The "pre-train, prompt-tuning'' paradigm has demonstrated impressive  performance for tuning pre-trained heterogeneous graph neural networks (HGNNs) by mitigating the gap between pre-trained models and downstream tasks. However, most prompt-tuning-based works may face at least two limitations: (i) the model may be insufficient to fit the graph structures well as they are generally ignored in the prompt-tuning stage, increasing the training error to decrease the generalization ability; and (ii) the model may suffer from the limited labeled data during the prompt-tuning stage, leading to a large generalization gap between the training error and the test error to further affect the model generalization. To alleviate the above limitations, we first derive the generalization error bound for existing prompt-tuning-based methods, and then propose a unified framework that combines two new adapters with potential labeled data extension to improve the generalization of pre-trained HGNN models. Specifically, we design dual structure-aware adapters to adaptively fit task-related homogeneous and heterogeneous structural information. We further design a label-propagated contrastive loss and two self-supervised losses to optimize dual adapters and incorporate unlabeled nodes as potential labeled data. Theoretical analysis indicates that the proposed method achieves a lower generalization error bound than existing methods, thus obtaining superior generalization ability. Comprehensive experiments demonstrate the effectiveness and generalization of the proposed method on different downstream tasks.

Published as a conference paper at ICLR 2025
HG-ADAPTER: IMPROVING PRE-TRAINED HETERO- GENEOUS GRAPH NEURAL NETWORKS WITH DUAL ADAPTERS
Yujie Mo
Runpeng Yu? Xiaofeng Zhu
Xinchao Wang?*
School of Computer Science and Engineering. University of Electronic Science and Technology of China
National University of Singapore
ABSTRACT
The "pre-train, prompt-tuning" paradigm has demonstrated impressive performance for tuning pre-trained heterogeneous graph neural networks (HGNNs) by mitigat- ing the gap between pre-trained models and downstream tasks. However, most prompt-tuning-ban
works may face at least two limitations: (i) the model may be insufficient to fit the graph structures well they are generally ignored in the prompt-tuning stage, increasing the training error to decrease the generalization ability; and (ii) the model may suffer from the limited labeled data during the prompt-tuning stage, leading to large generalization gap between the training enror and the test error to further affect the model generalization. To alleviate the above limitations, we first derive the generalization error bound for existing prompt- tuning-based based methods, and then propose unified framework that combines new adapters with potential labeled data extension to improve the generalization pre-trained HGNN models. Specifically, design dual structure-awan adapters adaptively fit task-related homogeneous and heterogencous structural information. We further design laoel-propagated contrastive loss and self-supervised losses to optimize dual adapters and incorporate unlabeled nodes as potential labeled data. Theoretical analysis indicates that the proposed method achieves lower generalization error bound than existing methods, thus obtaining superior gener- alization ability. Comprehensive experiments demonstrate the effectiveness and generalization of the proposed method on different downstream tasks.
INTRODUCTION
Pre-trained heterogeneous graph neural networks (HGNNs) are designed to pre-train models on the heterogeneous graph data and then effectively generalize to diverse tasks (Fan et al., 2019; Jiang al., 2021). To achieve this, current pre- trainee HGNNs typically utilize unsupervised techniques during pre-training to learn fundamental properties, thereby enhancing the generalization ability of models (Yang al., 2022; Fan et al., 2024). Consequently, pre-trained HGNNs have demonstrated promising potential in real applications such as recommendation systems, social network analysis, and molecular design (Shi et al., 2016; Tian et al., 2023; Wu et al., 2024).
Existing pre-trained HGNNs generally follow two paradigms, i.e., "pre-train, fine-tuning" and "pre- train, prompt-tuning" The "pre-train, fine-tuning" paradigm typically first trains the model with unlabeled data in the pre-training stage, and then updates the pre-trained model with task-related labels the fine-tuni tne-ttning stage to adapt downstream tasks (Wang al., 2021: Tian al., 2023). However, the two stages in the "pre-train, fine-tuning" paradigm optimize different objectives, resulting in the gap between pre-trained models and downstream tasks that weakens the model generalization (Liu al., 2023a; Yu et al., 2024c). To solve this issue, recent works propose the "pre-train, prompt-tuning" paradigm to connect pre-trained models with downstream tasks by designing a learnable prompt that directly appends to (or modifies) the model input (Yu et al., 2024b; Liu et al., 2023b). For
"Corresponding authors

---

Published as a conference paper at ICLR 2025
Latee Pecpptne *
Message -Pasing *
MaMisig
E
Honsogeneous Adapter
TEATED OCT EINER
Z
Heteregrnoos Adapter
M
Mirhial
Laye-
Mensnge
HARD SONTIII
Paceperce
.Procing
Up
*
*
Sum
2
Down
Viigla
Figure I: The flowchart of the proposed HG-Adapter. Given the frozen representations H from the pre-trained model, the homogeneous adapter is designed to generate the adapted representations F by tuning the homogeneous graph structure A (details in the top right corner). After that, the homogeneous representations Z obtained by summing the frozen representations E after message- passing and F. Similarly, the heterogeneous representations Ž are obtained by summing the frozen representations É after message- passsegg and the adapted representations M from the heterogeneous adapter (details the bottom right corner). Furthermore, Z and Ž are concatenated to generate Z, which is then mapped to the prediction matrix P. Finally, HG-Adapter designs a label-propagan contrastive loss (i.e., Lcon) and two self-supervised losses (i.e., Cree and Lmar) to optimize dual adapters and extend potential labeled data to improve the model generalization.
instance, HGPrompt (Yu et al., 2024a) proposes dual-template and dual-prompt to unify pre-training and downstream tasks for both homogeneous and heterogeneous graphs. Similarly, HctGPT (Ma al., 2024) introduces virtual class and heterogeneous feature prompts, and then reformulates the downstream objective function to align it with pre- pre-training tasks.
methods have least two limitations. First, the prompt- tuning in existing methods only focuses on the node features while ignoring the graph structures in the heterogeneous graph. As a result, the model may not sufficiently fit the task-related information in graph structures during the prompt-tuning stage, thereby increasing the training error and decreasing the generalization ability on downstream tasks (Bousquet & Elisseeff, 2002). Second, the pre-trained models generally trained in unsupervised manner, thus their ability to generalize to unlabeled data may be constrained by the limited labeled data during the prompt-tuning stage. Consequently, this may result in large generalization gap between the training error and the test error, leading to sub-optimal generalization performance (Arora et al., 2018).
However, existing "pre-train, prompt-tuning
To alleviate the above limitations and improve the model generalization, three challenges remain be addressed. (i) Existing prompt-tuning- based methods usually rely on heuristically designed prompts improve generalization but lack a unified theoretical framework, highlighting the necessity to formally understand and derive the key factors that affect the generalization of existing methods. (ii) Based such theoretical foundation, a new framework needs to be designed effectively capture the task-related structural information in the heterogeneous graph, thus decreasing the training error and improving the model generalization. (iii) Given the difficulty of directly increasing the labeled data, new alternative solutions are required to mitigate the issue of limited task-related labels in the tuning stage, thereby decreasing the generalization gap and further improving the model generalization.
In this paper, we address the above challenges by first deriving generalization error bound for existing prompt-tuning-bant methods and then proposing a novel adapter-tuning- based framework (termed HG-Adapter) with dual structure-aware adapters and potential labeled data extension, thus improving the generalization of pre-trained HGNNs, as shown in Figure Specifically, we first theoretically analyze the generalization generalization gap, thus building the theoretical foundation for our method and exploring challenge (i). After that, we design homogeneous and heterogeneous adapters capture the relkeeelated
bound for existing methods based on the training error and the
2

---

Published as a conference paper at ICLR 2025
structural information by adaptively tuning both homogeneous and heterogeneous graph structures, thus decreasing the training error and improving the model generalization to explore challenge (ii). Moreover, design a lroelapropagated contrastive loss and two self-supervised losses unlabeled nodes, thus potentially extending the labeled data and decreasing the generalization gap to further improve the model generalization and explore challenge (iii). Finally, theoretically demonstrate that the proposed method achieves a lower generalization error bound than existing prompt-tuning-bant methods, leading superior performance across different downstream tasks.
Compared to existing works, our main contributions are summarized as follows:
To our best knowledge, this is the first dedicated attempt to design unified "pre-train, adapter-tuning" paradigm to improve different pre-trained HGNN models.
We design dual structure-aware adapters to capture task-relater homogeneous and heteroge- neous structural information. Moreover, design a label-prop aroopaaad contrastive loss and two self-supervised losses to achieve the potential labeled data extension.
We thcoretically derive a unified generalization error bound for existing methods based on the training error and the generalization gap. Moreover, demonstrate that the proposed method achieves a lower generalization error bound than existing prompt-tuning www-tttnnnnneeeee based methods to improve the generalization ability of pre-trained HGNN models.
We experimentally validate the superior effectiveness and generalization of the proposed HG- Adapter compared to state-of-the-ar fine-tuning-bant bbased and prompt-tune commpt-uuning-aatt methods, and demonstrating its adaptability to different pre-trained HGNN models.
METHOD
2.1 PRELIMINARIES
Definition 2.1. Heterogeneous graph (Sun & Han, 2012) is defined as G (V, sp), where V and & indicate the set of nodes and the set of edges, respectively. A and R. indicate the set of node rypes and the set of edge types, respectively, where |AUR| > and the node type relevant to downstream tasks is referred to as the target node. Moreover, A and R are associated with the node rype mapping function o V A and the edge rype mapping function & E R, respectively.
Definition 2.2. Pre-trained HGNNs (Wang et al., 2021; Mo al., 2024) generally include homoge- neous and heterogeneous branches, which can be divided into two steps: 1) mapping the original node features X with the Multi-Layer Percepiron (MLP) to obtain low-dimensional representations, and 2) conducting the message-passineg among nodes from the same type and nodes from different rypes based on homogeneous and heterogeneous graph structures, respectively, iee,
H MLP(X), E = Clessage-Passinge [, Grom),
H (X), É MessaagePaassing H, Ghet),
where Ghom indicates the homogeneous graph structure that connects nodes within the same type, and Gnet indicates the heterogeneous graph structure that connects nodes from different types. The homogeneous and heterogeneous graph structures generally come from pre-defined meta pata-ppths and the given set of edges & in the heterogeneous graph, respectively.
2.2 GENERALIZATION BOUND OF PROMPT-TUNING--ASTE METHODS
Given pre-trained HGNN models, existing "pre-train, prompt-tuning" methods typically freeze the model parameters and design tunable prompts to modify the input and capture task-related information (Yu et al., 2024a). Despite effectiveness, existing methods generally rely on heuristically designed prompts and lack a unified theoretical framework to further improve the generalization ability. As a result, during the prompt-tuning stage, most existing methods ignore graph structures and may suffer from limited labeled data, leading to inferior model generalization. To address this issue, we derive the generalization error bound for existing prompt-tunin toppttttninn--ee based methods (i.e., HGPrompt (Yu et al., 2024a) and HetGPT (Ma et al., 2024)) based on the classical regime the generalization bound theory (Arora et al., 2018; Aghajanyan al., 2021) (proofs provided in Appendix C.
3

---

Published as a conference paper at ICLR 2025
Theorem 2.3. (Generalization evror bound for prompt-tuning -basid methods.) Statistically, the upper bound U(EM) ) of the test ervor Ess of a pre-trained HGNN model with prompt-rume -uning can be determined as follows:
u(EM)= = Ex PMr) +
/NM).
(2)
where training data Dus and prompt-luning purameters P are variables of the training error Ens of the model in prompt-tuning stage. The number of training samples nas and the size of parameter space (Pas) are variables of the generalization gap bound between training error and test error. Moreover, when RM is fixed, there exist an optimal [PM] to achieve the lowest upper bound for prompt-tuning-uuse methods, i.e.,
min(U((Ex)) = Em PM) - O PM //Mx).
Theorem 2.3 indicates that the lowest generalization error bound of existing prompt-tuning- ten-tt-bnnndddd based methods exists and consists of two parts, i.e., the training error EMs of the model the prompt-tuning
stage, and the generalization gap bound O(V PM /mms). As a result, we theoretically derive the generalization error bound of existing prompt-tunie connptt-uning--eee based methods based on the training error and the generalization gap, thus solving the challenge (i).
2.3 DUAL STRUCTURE-AWAT ADAPTERS
Intuitively, based on Theorem 2.3, when 71 is fixed, encouraging the parameters PM to approach the optimal parameters Pas can enable the model to approach the lowest generalization error bound. However, most existing coompt-tuning-eeee based methods focus on designing prompts for node features while ignoring the graph structures that contain task-related information in the heterogeneous graph (Ma et al., 2024; Yu et al., 2024a). As result, the parameters in existing prompt-tuning-Ne -aased methods may be insufficient to model the input data effectively, leading to the increased training error ÊM. Therefore, although we cannot directly find the optimal parameters PM, we can better fit the input data with few parameters to decrease the training error, thus decreasing the upper bound of the test eтror and approaching the optimal parameters. To do this, in this paper, we propose to design dual structure-award adapters model both node features as well homogeneous and heterogeneous graph structures, thereby making the parameters Pa closer to the optimal parameters Pa to improve the generalization ability.
To do this, for the homogeneous branch in pre-trained HGNNs, we design homogeneous adapter to capture the task-related homogeneous structural information (i.e., the connections among nodes within the same type). Specifically, the homogeneous adapter includes two steps (i.e., mapping and message- paaagee-assig)),, which tune node features and the homogeneous graph structure, respectively. In the mapping step. first employ the MLP fa to obtain the mapped representations F for the frozen representations H before message- sastaggeppssing.... i.e.,
F = ReLU(HWss).
(4)
where Ws E d indicates the trainable parameters of for and ReLU(.)) indicates the ReLU activation function. We further follow the lightweight principle (Hu al., 2021; Chen al., 2022) to decrease the number of parameters by decomposing Ws into two low-rank matrices, i.e. W downn W up, where W Wdown E dd W
up E B'xd", and t < d,d'
In the mnessage-psssing step, we focus on tuning the homogeneous graph structure to capture the task-related homogeneous structural information, i.e., assign large weights for node pairs within the same class while assigning small weights for node pairs from different classes. However, the homogeneous graph structure is typically not provided in the heterogeneous graph, and previous methods generally employ meta-paths to build it, which requires expert knowledge and incurs large computation costs (Jing et al., 2021; Wang et al., 2023). To alleviate this issue, we propose to tune the homogeneous graph structure adaptively by calculating the similarity weight ä between representations of nodes v and by from the same node type, i.e.
(h, W.) . (h, W
ā.j
(5)
W 1b, W. Wolll
4

---

Published as a conference paper at ICLR 2025
where h, and h indicate the frozen representations of nodes 8; and by before message- passing, and message-passinn,, Wa E Rddx indicates the trainable parameters that also can be decomposed into low-rank matrices. To improve the performance and reduce computation costs, we only calculate à., between node U; and its k nearest neighbors to obtain a sparse graph structure A. After that, can further derive the positive and symmetric graph structure A E R=*a by
A RcLU(A) ReLU(A)"
(6)
where n indicates the number of target nodes from the same node type.
In Eq. (6), a larger value of the element aij indicates a stronger correlation between nodes uy and Vj, suggesting that they are more likely to belong to the same class. Based on the mapped representations F and the adaptively learned homogeneous graph structure A, we conduct the message-passing among nodes that are likely to belong to the same class and obtain the adapted representations F, i.e., F AF. As a result, the adapted representations are expected to capture the adaptive homogeneous structural information and aggregate information from nodes within the same class. Subsequently, we can obtain the homogeneous representations Z by summing the adapted representations F from the homogeneous adapter and the frozen representations E after message- baasaagepassin, i.e.,
Z = E aF
where a is non-negative parameter.
In addition to the homogeneous adapter, we further propose to design a heterogencous adapter for the heterogeneous branch to capture the task-related hetcrogeneous structural information (l.e., the connections among nodes from different types). Similarly, the heterogeneous adapter also consists of mapping and bassage-passsnng steps. Specifically, in the mapping step, we first employ the MLP fo to obtain the mapped representations M for the frozen representations H before message-passing, i.e.
= (eHU
down Oup),
(8)
where Odown E Rdxt' and Oup E R xd indicate the trainable parameters of fa. and t' < d,
In the message- message-passing step, we focus tuning the heterogeneous graph structure to capture the task-related structural information, i.e., assign large weights for important neighbors while assigning small weights for unimportant neighbors. To do this, we propose calculate the weight Si, for the r-th type of neighbor of each target node vs with the score function, thereby obtaining the heterogeneous graph structure S E [R| i.e.
Siyr exp(Tanh(b,,- W.))
exp(Tanh(hh; (h,," W:))
where frr indicates the representation of the r-th type of neighbor for node 8, We E Rdx1 indicates the trainable parameters of the score function, Tanh(-) denotes the Tanh activation function, and [R] denotes the number of edges types.
In Eq. (9), larger value of the element s;, indicates that the r-th type of neighbor is more important for node v;. With the mapped representations M and the adaptively learned heterogeneous graph structure S, conduct the saassage-passingg among nodes from different types and obtain the adapted representations M, i.e,, m, = | mi where m,, indicates the mapped representation the r-th type of neighbor for node e;. As a result, the adapted representations M are expected to capture the heterogeneous structural information and aggregate information from important heterogeneous neighbors with large weights. Subsequently, we can obtain the heterogeneous representations Ž by summing the adapted representations M from the heterogeneous adapter and the frozen representations E after message- saassagg-psssing... i.e.,
(10)
where 3 is non-negative parameter.
As a result, the dual structure-an aware adapters tune node features as well as homogeneous and hetero- geneous graph structures simultaneously, thereby capturing more task-related structural information
5

---

Published as a conference paper at ICLR 2025
than existing prompt-tuning- -unii---eed methods. We then concatenate homogeneous representations Z and heterogeneous representations Z obtain the final node representations Z. Consequently, the final node representations aggregate information from nodes within the same class as well as important neighbors from other node types. This enables the model to better fit the input data and get closer to the optimal parameters PM, thus reducing the training error and improving the model's generalization ability to solve the challenge (ii) (verified in Appendix E.3).
2.4 POTENTIAL LABELED DATA EXTENSION
Designing the dual structure-aware adapters encourages the parameters PM to approach the optimal parameters PA Mr thereby decreasing the training error EN and approaching the lowest generaliza- tion error bound. Actually, based on Theorem 2.3, there is another way to further decrease the generalization error bound, i.e., increasing the number of labeled data (i.e. ntM) in training set to
decrease the generalization gap bound O(y Pas /nas). However, obtaining large number of labeled data is challenging and costly in real scenarios. To solve this issue, in this paper, we design a label-propagated contrastive loss and two self-supervised losses, extending all unlabeled nodes as the potential labeled data to further improve the model's generalization ability.
Specifically, we first propose to conduct the label propagation for the unlabeled data based on the adaptively tuned homogeneous graph structure A and the label matrix Y, i.e.,
Y = AY,
(11)
where Y consists of one-hot label indicator vectors for labeled nodes and zero vectors for unlabeled nodes. As a result, Eq. (11) encourages the label propagation from the labeled nodes and tends to assign the same label for unlabeled nodes within the same class. Note that we utilize the homogeneous graph structure for the label propagation instead of the heterogeneous graph structure, as the label propagation occurs only among target nodes of the same type.
Given the propagated labels Y, a common approach is train classifier that maps node repre- sentations Z to a probability matrix and then computes the cross-entropy loss between the label matrix and the probability matrix. However, the cross-entropy loss typically differs from the ob- jective function used in the pre-training stage, leading to the gap between pre-trained models and downstream tasks (Jing et al., 2021; Yu al., 2024c). Fortunately, existing literature indicates that any standard contrastive pre-training task on graphs can be reformulated as the objective function based on subgraph similarity (Liu al., 2023b; Yu al., 2024b). Therefore, we propose to bridge the gap between different pre-trained models and downstream tasks by reformulating the downstream objective function according to the class-subgraph similarity.
To do this, we first employ a projection 9p to map node representations Z, resulting in the prediction matrix P of all nodes, i.e., P ZW where Wp E Rdd xe is the trainable parameters of 9p. and denotes the number of classes. We then consider nodes with the same propagated label to be in the same subgraph, and we can further obtain the class- subgrahh prediction cy, by averaging the prediction vectors of nodes whose original labels equal to ý;- After that, we propose contrastive loss based on the subgraph similarity to incorporate supervision signals by enforcing the node prediction close to its class-subgran prediction while far away from different class- subsrauu predictions, iee.,
Yz
(si C/T)
YUL
exp(sim (p:, Cy.) CT)) 28.75, exp(sim(ps, (p:, C3,)/T)²
Loon = - Σ Ly:ty exp(sim(p:, C3,)/T)
(12)
where X is a non-negative parameter, Y, and Yor indicate the sets of node indices with and without original labels, respectively, sim(-) denotes a similarity function between two vectors, and T is a temperature parameter. Consequently, Eq. (12) simulates objective functions in different pre-trained HGNN models based on the subgraph similarity, thereby bridging the gap between pre-trained models and downstream tasks. This facilitates the application of the proposed method to different pre-trained HGNN models. Moreover, Eq. (12) encourages the prediction vectors of both labeled and unlabeled nodes to effectively capture the class information, potentially increasing the number of labeled data to further decrease the model's generalization error bound.
Although the propagaae contrastive loss increases the number of labeled data potentially, the confidence of propagated labels of unlabeled nodes in Y may be insufficient during the early stages

---

Published as a conference paper at ICLR 2025
of training due to the gradual optimization of the graph structure A. This may hinder the optimization of the dual adapters, i.e,, the homogeneous and heterogeneous graph structures of labeled nodes may be better tuned than that of unlabeled nodes. To address this issue, start with the essence of homogeneous and heterogeneous graph structures, and then design two self-supervised losses for the dual adapters and treat both labeled and unlabeled nodes equal supervision signals.
Intuitively, for nodes within the same type, a good homogeneous graph structure A should connect nodes from the same class while disconnecting nodes from different classes. Moreover, existing literature suggests that nodes with similar node features generally come from the same class (Liu al., 2022: Wu al., 2023). For instance, in an academic heterogeneous graph, if the features of two "paper" nodes share many keywords, they are likely to belong to the same class. Therefore, there is an intuitive way to optimize the homogeneous graph structure by connecting nodes that share similar node features while disconnecting nodes with dissimilar node features. To do this, we design feature reconstruction loss to align node features before and after the passsng. thus optimizing the graph structure A to assign appropriate weights to node pairs within the same class, i.e.,
Lree
Xi.j In In(A))
(13)
where m and I denote the number of sampled nodes and the feature dimension, respectively. Eq. (13) enforces the reconstructed node features after message-passing to be aligned with the original node features, which requires that pressage-passing occurs only among nodes with similar node features. Therefore, Eq. (13) encourages the graph structure A connect nodes within the same class while disconnecting nodes from different classes as much as possible to optimize it.
Different from tuning the homogeneous graph structure that connects nodes within the same type, the heterogeneous graph structure connects nodes from different types. Therefore, the feature reconstruction may not be suitable for optimizing the heterogencous graph structure S as the connected nodes come from different feature distributions. To alleviate this, we first analyze the significance of different heterogeneous neighbors and then optimize the weights assigned to them.
Intuitively, if a certain type of heterogeneous neighbor of node v; provides more relevant information for identifying the node's label than other neighbors, then such neighbors are more important the node it and should be assigned larger weights. That is, although heterogeneous neighbors come from different feature distributions, their representations may still partially overlap with the Hlass-subgraph representation Cy, of the node U; to provide class-related information. Furthermore, the degree of overlap is greater for important neighbors than unimportant neighbors. Therefore. this paper designs margin loss to relatively narrow the distance between the class-subgraph representation cy, and the adapted representation m, thus optimizing the graph structure S to assign large weights for important heterogeneous neighbors, i.e.,
Lmar
Σ
(d(cy,) -dcc, my)² + +
(14)
1J=18,,9,
where f- maxf. f 0}. d(.) indicates distance measurement between two vectors, and y is non-negative parameter. Eq. (14) aims to decrease the distance d(cy,, ) while increasing the distance d(cy,, )2 thus ensuring the "safe" margin " between them. Therefore, the adapted representation m, EI m, is encouraged to preserve its class-related information that shared with Cy.- This optimizes the heterogeneous graph structure S to assign large weights to important neighbors that contain more class-related information. Note that the margin loss only requires the relative distance to exceed 7, unlike traditional contrastive losses (e.g., InfoNCE (Oord al., 2018)) that enforce the alignment between the class-subgraph representation and the adapted representation. Such direct alignment may be inappropriate, as these representations come from different feature distributions (verified in Appendix E.8).
Finally, we integrate the contrastive loss in Eq. (12), the feature reconstruction loss in Eq. (13), and the margin loss in Eq. (14) to obtain the objective function of the proposed method, i.e.,
J Lrom + yLoa + phmmrr
(15)
where 77 and μ are non-negative negatigte parameters. With the objective function in Eq. (15), the proposed method optimizes the dual adapters and extends the potential labeled data by incorporating both
7

---

Published as a conference paper at ICLR 2025
Table 1:
Classification performance (i.e. Macro-Fl and Micro-FI) all heterogeneous graph datasets, where the best results are highlighted in bold, while improved results with the proposed HG-Adapter are underlined. The "+" symbol indicates the integration of HG-Adapter and HetGPT with original pre-trained HGNN models.
ACM
Yelp
DBLP
Aminer
Method
Macro-F1
Micro-FI
Macro-F1
Micro-F1
Macro-F1
Micro-F1
Macro-F1
Micro-FI
HAN
89.4+0.2
89.2++0.2
90.5+1.2
90.7+1.44
91.2+t0.4
92.0+0.5
65.3+0.7
72.8+0.4
HGT
91.5++0.7
91.6+0.6
89.9+0.5
90.24+..6
90.9+0.6
91.7+0.8
64.5+00.5
71.0=0.7
DMGI
89.8+0.1
89.8+0.1
82.9=0.8
85.8+0.9
92.1+0.2
92.9+0.3
63.8+0.4
67.6+0.5
HGCML
90.6110.7
90.7.10.5
90.71...8
91.011077
91.910.8
93.210.7
70.510.4
76.3
HGMAE
90.5.10.5
90.6.10.7
90.5
90.7110.5
92.910.5
93.410.6
72.3.109
80.3
HGPrompt
92.1110.7
92.0.10.6
92.5
92.310.5
93.5
94.0.10.7
74.8110.8
83.9-10.6
HDMI
90.1110.3
90.1.10.3
80.7110.6
84.0.10.9
91.3.0.22
92.2:0.5
65.910.4
71.7-10.6
HetGPP
91.0:10.7
90.9-10.6
81.4
84.81.0.5
91.940..8
92.940.6
67.140.3
72.9-0.4
HG-Adppteer
91.4:10.6
91.5+10.7
82.0-10.8
85.5+:1.1
92.4
23.340077
67.9
73.6-10.4
HeCo
88.3+-0.3
88.2+00.2
8.7
87.9+0.66
91.0+H..3
91.6+0.2
71.8+-0.9
78.6+0.77
*HctGPT
88.5+-0.4
88.4+-0.6
85.9+
88.64-0.9
91.5++0.6
92.2+0.5
72.1+0.7
79.0++0.4
+HG-Adpater
89.0110.5
89.01104
86.410.6
89.210.77
92.310.8
92.610.6
72.810.5
79.8+0.3
HERO
92.24-0.5
92.1+007
92.4+0.7
92.3+00.
93.8+0.66
94.4+0.4
75.1+0077
84.5+009
+HetGPT
92.4+-0.4
92.2++033
92.6+0.5
92.4+00.7
93.8++0.4
94.5+0.3
75.7++0.6
85.2+008
+IGG-Adapter
92.7-10.4
92.7
9..1
92.7.10.5
94.0.10.7
94.7100.8
78.3.0..5
87.1
labeled and unlabeled nodes supervision signals. As result, this decreases the generalization gap and thus further decreases the generalization error bound of existing methods to solve the challenge derive that the proposed method with dual adapters (iii) (verified in Appendix E.4). Actually, generalization error bound than existing methods by and labeled data extension achieves a lower decreasing the training error and the generalization gap, which can be found in Appendix C.2. Consequently, the proposed method is expected to obtain better generalization and effectiveness on downstream tasks. (verified in Section 3.2.1 and Section 3.2.2).
3 EXPERIMENTS
In this section, we conduct experiments on four public heterogeneous graph datasets to evaluate the proposed HG-Adapter in terms of different downstream tasks (i.e., node classification and node clustering), compared to both fine-tuning-bart methods and prompt-tuning tronp--tuning-based based methods. The code of the proposed method released at https: / //g. thub. com/Yu eeMo HHGH-GAo Adapter.
3.1
EXPERIMENTAL SETUP
3.1.1 DATASETS
The used datasets include three academic datasets (i.e., ACM (Wang al., 2019), DBLP (Wang et al., 2019), and Aminer (Hu et al., 2019)), and one business dataset (i.e., Yelp (Lu et al., 2019)).
3.1.2 COMPARISON METHODS
The comparison methods include two traditional semi-supervised methods (i.e., HAN (Wang et al., 2019) and HGT (Hu et al., 2020b)), six fine-tuning- HDMI (Jing et al., 2021), 1), HeCo (Wang al., 2021), HGCML (Wang et al., 2023), HGMAE (Tian et al., 2023), and HERO (Mo et al., 2024)), and two prompt-tuning based methods (i.e., HGPrompt (Yu et al., 2024a) and HetGPT (Ma al., 2024)), where the pre-training and the prompt-tuning of HGPrompt are both specifically designed, while HetGPT only designs the prompt-tuning and thus can be used for different pre-trained HGNN models.
methods (i.c., DMGI (Park et al., 2020),
8

---

Published as a conference paper at ICLR 2025
Vi vigi
90-
vit veggee
wiight
0.5-
Homppely Rato
%
0.00
D0LP 60-
0.9
50
ACM Yolp
Aminer
Epocta
(a)
(b)
(c)
Figure 2: (a) Homophily ratios of the homogeneous graph structure A learned by HERO+HG-Adapee on four datasets. (b) Node classification results without maximal/minimal weights neighbors in the heterogeneous graph structure S learned by HERO+HG--dapter on three datasets (excluding the DBLP dataset, as its target node has only one type of neighbors). (c) Test errors of HERO with different tuning methods (i.e., traditional fine-tuning, prompt-tuning-basee HetGPT, and the proposed HG-Adapter) on the ACM dataset.
3.2 RESULTS ANALYSIS
3.2.1
EFFECTIVENESS ON DOWNSTREAM TASKS
To evaluate the effectiveness of the proposed HG-Adapter, we follow previous works (Fu et al., 2020; Ma et al., 2024; Mo et al., 2024) to employ node classification and node clustering downstream tasks and report their results in Table and Appendix E, respectively. Obviously, the proposed method consistently demonstrates superior performance on both node classification task and node clustering task than other comparison methods.
Specifically, first, for the node classification task, the proposed method always outperforms fine- tuning- uuing-based based and prompt-tuning-base comparison methods by large margins. For instance, the proposed method on average, improves by 0.81%, %, compared to the best prompt-tuning- -basd method (i.e., HetGPT), on different pre-trained HGNN models (i.e., HDMI, HeCo, and HERO). This improve- ment can be attributed the dual structure-awand adapters and the potential labeled data extension, which decrease the training еттоr and the generalization gap. Consequently, the proposed method enhances the model's s generalization ability on downstream tasks to decrease the generalization error on test data, thus improving the performance of different pre-trained HGNN models.
Second, for the node clustering task, the proposed method also obtains promising improvements. For instance, the proposed method on average, improves by 2.69%, compared to the best prompt- tuning-based method (i.e., HetGPT), across different pre-trained HGNN models. This demonstrates the superiority of the proposed method, which designs a contrastive loss based the class-subgraph similarity to ensure that nodes within the same class are close to each other, thereby enhancing the clustering performance. As result, the effectiveness of the proposed method is verified both node classification and node clustering downstream tasks.
3.2.2 EFFECTIVENESS OF DUAL-ADAPTERS
To verify the effectiveness of the proposed dual structure-award aware adapters, we calculate the homophily ratio (i.e., the ratio of edges that connect nodes within the same class) of A, collect the performance degradation without minimal and maximal weights neighbors in S, evaluate the test errors of HERO with different tuning methods, and report the results in Figure 2.
First, from Figure 2(a), the homophily ratio of the learned homogeneous graph structure increases rapidly at the beginning of training and tends to stabilize after reaching a high value. This suggests that the homogeneous adapter effectively learns to build edges among nodes within the same class. As result, the homogeneous adapter captures the task-related homogeneous structural information and promotes message- saassagg-pssinggggg within the same class. Second, from Figure 2(b), the proposed HG-Adapter without minimal weights neighbors obtains significantly less performance degradation than without

---

Published as a conference paper at ICLR 2025
Table 2: Classification performance (i.e., Macro-Fl and Micro-FI) of each component in the objective function J on all heterogeneous graph datasets.
ACM
Yelp
DBLP
Aminer
Loon Lree Linar
Macro-F1
Micro-Fl
Macro-FI
Micro-FI
Macro-Fl
Micro-Fl
Macro-F1
Micro-FI
30.2.11.1
1.3
50.3.10.7
$6.7100.
11.90.55
31.4.0.7
35.710.8
67.7110
V
32.310.9
40.5.10.8
41.7110.6
52.2.10.5
10.910.8
30.8.0.9
35.6.0077
67.5.0.6
87.910.5
87.8.0.77
89.110044
89.0.10.3
80.7.0.8
82.610.7
73.2:10.5
79.8.1.33
V
32.3+0.5
40.5=0.6
40.6+0.8
48.2+0.7
11.9
31,4#0.4
35.7#006
67.7=0.7
89.6+0.5
89.5=0.7
90.010.6
89.610.4
91.6=099
92.5+1.1
75.1
82.2=0.7
90.1
90.0=0.5
92.0+0.5
91.810.6
92.7=0.8
93.5+0.7
7..310.5
84.6=0.4
92.7+0.4
92.7+0.7
93.14-0.6
92.74.0.5
94.0
94.7410.8
0.53+
0..1--0.6
maximal weights neighbors. This demonstrates that the heterogeneous adapter effectively learns the heterogeneous graph structure S to assign large weights for important neighbors while assigning small weights for unimportant neighbors. Therefore, the heterogeneous adapter enables nodes to aggregate information from important neighbors, thus capturing more class-related information to benefit downstream tasks. Third, from Figure 2(c), the proposed HG- HG-Adapter achieves the lowest test error than tine-tuning-bee based and prompt-tuning bassgd--bddddaa methods the test data. This indicates that the proposed method indeed achieves lower generalization errors than existing methods on unlabeled data, obtaining better generalization ability on downstream tasks.
3.2.3 A ABLATION STUDY
The proposed method investigates the objective function J to optimize the dual adapters and extend the labeled data potentially. To verify the effectiveness each component of J (i.e., the contrastive loss Leon, the feature reconstruction loss Lree, and the margin loss fmaar,, we investigate the performance of all variants on the node classification task and report the results in Table
From Table 2, we have the observations as follows. First, the variant without Lcon performs significantly worse to the other two variants (i.e., without Eree and without Lmer, respectively). The reason can be attributed to the fact that the label information is necessary for the proposed HG- Adapter because provides the ground truth for optimizing adapters and node predictions. Second, the proposed method with the complete objective function obtains the best performance. For example, the proposed method on average improves by 2.0%, compared to the best variant (i.e., without Lmar), indicating that all components in the objective function necessary for the proposed method. This reasonable because the feature reconstruction loss Eree and the margin loss Lmar optimize graph structures and incorporate all unlabeled nodes equal supervision signals, thus improving the model generalization. These observations are consistent with our claims, i.e., capturing the task-related structural information with dual adapters and extending the potential labeled data is essential for improving the effectiveness and generalization ability of pre-trained HGNN models.
CONCLUSION
In this paper, we derived the generalization error bound for existing prompt-tuning-base -based methods and then proposed novel framework with dual structure-aware adapters and potential labeled data extension to address existing issues. Specifically, we first established the theoretical foundation by deriving the generalization error bound for existing methods based on the training error and the generalization gap. We then designed homogeneous and heterogeneous adapters to adaptively tune homogeneous and heterogeneous graph structures, thus capturing the reskerelated structural informa- tion to decrease the training error. Moreover, we designed a aabl-ppropagated contrastive loss and two self-supervised losses on unlabeled nodes, thus optimizing dual adapters and potentially increasing the labeled data to decrease the generalization gap. Theoretical analysis indicates that the proposed method is expected to obtain a lower generalization eтror bound and better generalization ability than existing prompt-tuning-batt based methods. Comprehensive experiments verify the effectiveness and generalization of the proposed method on various heterogeneous graph datasets and different downstream tasks. We discuss potential limitations and future directions in Appendix F.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This project is supported by the National Key Research and Development Program of China under Grant No. 2022YFA 1004100, the Natural Science Foundation of Guangdong Province of China under Grant No. 2024A 1224A151500 1515011881 and National Research Foundation, Singapore, under its Al Singapore Programme (AISG Award No: AISG2-RP-2021-022
REFERENCES
Armen Aghajanyan, Sonal Gupta, and Luke Zettlemoyer. Intrinsic dimensionality explains the effectiveness language model fine-tuning. In ACL, pp. 7319-7328, 2021.
Sanjeev Arora, Rong Ge, Behnam Neyshabur, and Yi Zhang. Stronger generalization bounds for deep nets via compression approach. In ICML, pp. 254 253-263, 2018.
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. In ICLR, 2021.
Olivier Bousquet and André Elisseeff. Algorithmic stability and generalization performance. In NeurlPS, volume 13, 2000.
Olivier Bousquet and André Elisseeff. Stability and generalization. The Journal of Machine Learning Research, 2:499-525 -2699995622222 2002.
Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. Adaptformer: Adapting vision transformers for scalable visual recognition. In NeurlPS, volume 35, pp. 16664-16678, 2022.
Li Dong, Nan Yang. Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao- Wuen Hon. Unified language model pre- pre-training for natural language understanding and generation. In NewrlPS, volume 32, 2019.
Shaohua Fan, Junxiong Zhu, Xiaotian Han, Chuan Shi, Linmei Hu, Biyu Ma, and Yongliang Li. Metapath-guided heterogeneous graph neural network for intent recommendation. In SIGKDD, pp. 2478-2486, 2019.
Tianyu Fan, Lirong Wu, Yufei Huang, Haitao Lin, Cheng Tan, Zhangyang Gao, and Stan Z Li. Decoupling weighing and selecting for integrating multiple graph pre-training tasks. In ICLR, 2024.
Taoran Fang, Yunchao Zhang, Yang Yang, Chunping Wang, and Lei Chen. Universal prompt tuning for graph neural networks. In NewrlPS, volume 36, 2023.
Xinyu Fu, Jiani Zhang, Ziqiao Meng. and Irwin King. Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. In WWW, pp. 2331-2341, 2020.
Anchun Gui, Jinqiang Ye, and Han Xiao. G-adapter: Towards structure- aware parameter-efficient transfer learning for graph transformer networks. In AAAI, volume 38, pp. 12226-12234, 2024.
Binbin Hu, Yuan Fang, and Chuan Shi. Adversarial learning heterogeneous information networks. In SIGKDD, pp. 120-129, 2019.
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR, 2021.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In NeurIPS, volume 33, pp. 221 12-2232,,3, 2020a.
Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. Heterogeneous graph transformer. In WwW, pp. 2710, 2020b.
Dayu Huang and Sean Meyn. Generalized error exponents for small sample universal hypothesis testing. IEEE transactions information theory, 59(12):815--8181, 2013.
11

---

Published as a conference paper at ICLR 2025
Xunqiang Jiang, Tianrui Jia, Yuan Fang. Chuan Shi, Zhe Lin, and Hui Wang. Pre-training large-scale heterogeneous graph. In SIGKDD, pp. 756-766, 2021.
Baoyu Jing, Chanyoung Park, and Hanghang Tong. Hdmi: High-order deep multiplex infomax. In WWW, pp. 2414-2424, 2021.
Diederik Kingma and Lei Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015,
Mei Li, Xiangrui Cai, Linyu Li, Sihan Xu, and Hua Ji. Heterogeneous graph attention network for drug-targe interaction prediction. In CIKM, pp. 1166-1176, 2022.
Shengrui Li, Xueting Han, and Jing Bai. Adaptergnn: inrameter--fficient fine-tuning improves generalization in gnns. In AAAI, volume 38, pp. 13600-13608, 2024.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1-35, 2023a.
Yixin Liu, Yu Zheng, Daokun Zhang, Hongxu Chen, Hao Peng, and Shirui Pan. Towards unsupervised deep graph structure learning. In WWW, pp. 3393-1403, 2022.
Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. In WWW. pp. 417-428, 2023b.
Yuanfu Lu, Chuan Shi, Linmei Hu, and Zhiyuan Liu. Relation structure- heterogeneous information network embedding. AAAL, pp. 4456 4463, 2019.
Yihong Ma, Ning Yan, Jiayu Li, Masood Mortazavi, and Nitesh Chawla. Hetgpt: power of prompt tuning in pre-trained heterogeneous graph neural networks. In WWW, 2024.
Harnessing the
Yujic Mo, Zongqian Wu, Yuhuan Chen, Xiaoshuang Shi, Heng Tao Shen, and Xiaofeng Zhu. Multiplex graph representation learning via common and private information mining. In AAAI, pp. 9217-9225, 2023.
Yujie Mo, Feiping Nie, Zheng Zhang, Ping Hu, Heng Tao Shen, Xinchao Wang, and Xiaofeng Zhu. Self-supervised heterogeneous graph learning: a homophily and heterogeneity view. In ICLR, 2024.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
Erlin Pan and Zhao Kang. Multi-view contrastive graph clustering. In NeurlPS, volume 34, pp. 2148-2159, 2021.
Chanyoung Park, Donghyun Kim, Jiawei Han, and Hwanjo Yu. Unsupervised attributed multiplex network embedding. In AAAI, pp. 5371 5371-5378, 2020.
Chuan Shi, Yitong Li, Jiawei Zhang, Yizhou Sun, and S Yu Philip. A survey of heterogencous information network analysis. IEEE Transactions on Knowledge and Data Engineering, pp. 17-37, 2016.
Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang, and Xin Wang. Gppt: Graph pre-training and prompt tuning to generalize graph neural networks. In SIGKDD, pp. 1717-1727,209
Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, and Jihong Guan. All in one: Multi-task task prompting for graph neural networks. In SIGKDD, 2023.
Yizhou Sun and Jiawei Han. Mining heterogeneous information networks: principles and methodolo- gies. Morgan & Claypool Publishers, 2012.
Yijun Tian, Kaiwen Dong, Chunhui Zhang, Chuxu Zhang, and Nitesh V. Chawla. Heterogencous graph masked autoencoders. In AAAI, pp. 9997-10005, 2023.
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang. Yanfang Ye, Peng Cui, and Philip S. Yu. Heterogeneous graph attention network. In WWW. pp. 2022-2033 2019.
12

---

Published as a conference paper at ICLR 2025
Xiao Wang, Nian Liu, Hui Han, and Chuan Shi. Self-supervised heterogeneous graph neural network 2021.
with co-contrastive learning. In SIGKDD, pp.
Zehong Wang, Qi Li, Donghua Yu, Xiaolong Han, Xiao-Zhi Gao, and Shigen Shen. Heterogeneous graph contrastive multi-view learning. In SDM, pp. 136 136-144, 2023.
Lirong Wu, Haitao Lin, Zihan Liu, Zicheng Liu, Yufei Huang, and Stan Li. Homophily-enhance self-supervision for graph structure learning: Insights and directions. IEEE Transactions Neural Networks and Learning Systems, 2023.
Lirong Wu, Yijun Tian, Haitao Lin, Yufei Huang, Siyuan Li, Nitesh V Chawla, and Stan Z Li. Learning predict mutation effects of protein-protein interactions by microenvironmmntt-waar hierarchical prompt learning. In ICML, 2024.
Yaming Yang, Ziyu Guan, Zhe Wang, Wei Zhao, Cai Xu, Weigang Lu, and Jianbin Huang. Self- supervised heterogeneous graph pre-training based on structural clustering. In NeurlPS, volume 35, pp. 16962-16974, 2022.
Xingtong Yu, Yuan Fang, Zemin Liu, and Xinming Zhang. Hgprompt: Bridging homogeneous and heterogeneous graphs for few-shot prompt learning. In AAAI, pp. 16578-16586, 2024a.
Xingtong Yu, Zhenghao Liu, Yuan Fang, Zemin Liu, Sihong Chen, and Xinming Zhang. Generalized graph prompt: Toward a unification of pre-training and downstream tasks on graphs IEEE Transactions Knowledge and Data Engineering, 2024b.
Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Multigprompt for multi-task pre-training and prompting on graphs. In WWW, pp. 515-526, 2024c.
Rui Zhang, Arthur Zimek, and Peter Schneider Kamp. A simple meta meta-path path-free framework for heterogeneous network embedding. In CIKM, pp. 2600-2609, 2022.
Jianan Zhao, Xiao Wang, Chuan Shi, Binbin Hu, Guojie Song, and Yanfang Ye. Heterogeneous graph structure learning for graph neural networks. In AAAI, pp. 4697-4705. 2021.
Sheng Zhou, Kang Yu, Defang Chen, Bolang Li, Yan Feng, and Chun Chen. Collaborative knowledge distillation for heterogeneous information network embedding. In WWW, pp. 1631-1639, 2022.
Yanqiao Zhu, Yichen Xu, Hejie Cui, Carl Yang. Qiang Liu, and Shu Wu. Structure-enhanced heterogeneous graph contrastive learning. In SDM, pp. 82-90, 2022.
A
RELATED WORK
This section briefly reviews topics related to this work, including fine-tuning- pre-trained heterogeneous graph neural networks in Section A.l, and graph prompt-tuning and adapter-tuning in Section A.2.
A.1 FINE- THEETUNNING-BAR BASED PRE-TRAINED HETEROGENEOUS GRAPH NEURAL NETWORKS
In recent years, inspired by the prosperity of pre-trained models in the fields of computer vision (Bao et al., 2021) and natural language processing (Dong et al., 2019), many pre- rre-trinee heterogeneous graph neural networks (HGNNs) have been introduced for the heterogeneous graph data (Wang et al., 2019; Jiang et al., 2021). Generally, these pre-trained HGNN models are trained in self-supervised fashion, facilitating the transfer of knowledge to downstream tasks through a fine-tuning step (Jing al., 2021; Zhu al., 2022).
Existing fine-tuning-bart based pre-trained HGNN methods can be broadly classified into two groups, i.e. mnta--att-bbaed methods and meta-path-free methods. In meta-path-basee base- methods, several graphs are usually constructed based on different pre-defined meta-paths to examine diverse rela- tionships among nodes that share similar labels (Jing al., 2021; Zhu et al., 2022). For example, STENCIL (Zhu et al., 2022) and HDMI (Jing et al., 2021) construct meth-path sath-path-bassed based graphs and then conduct node-level consistency constraints (e.g., contrastive loss) between node representations
13

---

Published as a conference paper at ICLR 2025
in different graphs. In addition, HGCML (Wang al., 2023) and CPIM (Mo et al., 2023) propose to maximize the mutual information between node representations from different meta-path based graphs. However, pre-defined meta meta-paths in these methods generally require expert knowledge and prohibitive computation costs (Zhang et al., 2022). Therefore, meta-path-free methods are proposed to capture the relationships among nodes without meta-paths. For example, SR-RSC (Zhang et al., 2022) designs a multi-hop contrast to optimize the regional structural information by utilizing the strong correlation between nodes and their neighbor graphs. In addition, recently, HERO (Mo et al., 2024) made the first attempt to learn an adaptive self-expressive matrix to capture the homophily in the heterogeneous graph, thus avoiding meta--pths..
A.2 GRAPH PROMPT-TUNING AND ADAPTER-TUNING
The "pre-train, fine-tuning" paradigm has become prevalent in graph learning, particularly for tasks with limited labeled data (Ma et al., 2024). However, this approach often suffers from a mismatch between the objectives of pre-training and downstream tasks, resulting in a "negative transfer" problem. Consequently, the knowledge acquired during the pre-training stage can negatively impact the performance of downstream tasks.
To solve this issue, recent research suggests the "pre-train, prompt-tuning" diagram to establish a connection between downstream tasks and pre-trained models by designing learnable prompt that modifies the model input (Fang et al., 2023; Yu et al., 2024b). For example. for the homogeneous graph, GPPT (Sun et al., 2022) introduces prompt templates to align the link prediction pre-training task with the downstream node classification task. ProG (Sun et al., 2023) proposes a new multi-task prompting method for graph models by unifying the format of graph prompts and language prompts with the prompt token, token structure, and inserting pattern. GraphPrompt (Liu et al., 2023b) employs subgraph similarity as its template and designs learnable prompt to unify pre-training with multiple downstream tasks. For the heterogeneous graph, HGPrompt (Yu et al., 2024a) proposes to unify pre-training and downstream tasks as well as homogeneous and heterogeneous graphs via dual- template and dual-prompt design. HetGPT (Ma et al., 2024) designs virtual class and heterogeneous feature prompts, and reformulates downstream tasks to mirror pretext tasks. Despite effectiveness, both HGPrompt and HetGPT ignore the graph structures during the prompt-tuning stage. Although part homogeneous graph prompt-tuning methods (e.g., ProG) investigate to design structure prompts, they may not easily be transferred to the heterogeneous graph, as the more complex graph structures in the heterogeneous graph. As result, when tuning the pre-trained HGNNs, the model may not sufficiently model the input data to increase the training error and decrease the generalization ability.
Different from the "pre-train, prompt-tuning" diagram, the "pre-train, adapter-tuning" diagram aims to insert adapter modules with lightweight neural network architecture to bridge the gap between pre-trained models and downstream tasks. Recent works propose adapter-tuning for homogeneous graph pre-trained models by inserting different adapter modules (Li et al., 2024; Gui et al., 2024). For example, G-Adapter (Gui et al., 2024) leverages the graph structure and Bregman proximal point optimization strategy to mitigate the feature distribution shift issue. In addition, AdapterGNN (Li et al., 2024) proposes preserve the knowledge of the large pre-trained model and leverage highly expressive adapters for graph neural networks, adapting to downstream tasks effectively with only a few parameters. However, they designed for the homogeneous graph and cannot easily transfer to the heterogeneous graph. Moreover, these adapter-tuning-aase methods also cannot tune the graph structures, thus increasing the training error and decreasing the model generalization.
As result, when tuning pre-trained HGNN models, despite the effectiveness of existing prompt- tuning-based methods, they still have some limitations to address. That is, homogeneous and heterogeneous graph structures are generally ignored in the tuning stage, leading to increased training error and decreased generalization ability. Moreover, existing pompt-tuning based methods may suffer from the issue of limited labeled data in the tuning stage, leading to large generalization gap between training and test errors and sub-optimal generalization on downstream tasks.
B ALGORITHM
This section provides the pseudo-code of the proposed method in Section B.1 and the complexity analysis in Section B.2.
14

---

Published as a conference paper at ICLR 2025
B.1
ALGORITHM
Algorithm 1 The pseudo-code of the proposed method.
Input: Heterogeneous graph G (V,E,A R, Φ, P)), pre-trained HGNN model, maximum training
steps E;
Output: Homogeneous and heterogeneous adapters, projection 9pi
1: Initialize parameters and upload pre-trained parameters;
2: while not reaching E do
3: Obtain homogeneous graph structure A by Eq. (7);
4: Obtain the homogeneous representations Z by Eq. (7); obtain heterogeneous graph structure by Eq. (9);
5:
6:
Obtain the heterogeneous representations 2 by Eq. (10);
7:
Conduct the label propagation by Eq. (11); Conduct the label-propagated contrastive loss by Eq. (12);
8:
9:
Conduct the feature reconstruction loss by Eq. (13);
10:
Conduct the margin loss by Eq. (14);
Compute the objective function J by Eq. (15);
12:
Back-propage propagate J to update model weights;
13:
end while
B.2 COMPLEXITY ANALYSIS
The proposed method consists of two prats, i.c., dual structure-aware adapters and potential labeled data extension. We analyze the time complexity of each part as follows. First, the time complexity of the dual structure-award adapters Onkkd + n[Rl), where n, k, d, and [R] indicate the number of nodes, the number of neighbors of each node, the number representation dimension, and the number of edge types, respectively. Second, the time complexity of the potential labeled data extension is O(nke ne² + nkf) where and f indicate the number of classes and dimensions of node features, respectively. Therefore, the overall time complexity of the proposed HG- HG-Adapter is O'n(kd + |R] ke c² + k)). As result, The proposed HG-Adapter is scaled linearly with the sample size and has the potential to be implemented with limited resources.
C PROOFS OF THEOREMS
C. PROOF OF THEOREM 2.3
To prove Theorem 2.3, we first introduce the generalization error bound for finite hypothesis space in classical regime (Bousquet & Elisseeff, 2000; Huang & Meyn, 2013) by the following Lemma.
Lemma C.1. (Generalization error bound for finite hypothesis space in the classical regime.) Statistically, the upper bound U(E) of the test error of model the finite hypothesis space is determined as follows:
E {(()) & +O(VIPI/))
(16)
where training data D, and parameters P are variables of the training error E. The number of training samples n and the size of parameter space [P] are variables of the generalization gap bound O(/(Pl/m) between the training error and the test error.
Proof. Let H. represent a finite hypothesis space, where each h E H corresponds to a set of trained parameters within this parameter space H. εp, represents training error over sampled training data D,,, and represents test error. the number of training data.
Then, the probability that the difference between the test error and the training error of the hypothesis space is greater than e can be written as
P(3h E H. ED, (h) 8(h) e).
(17)
15

---

Published as a conference paper at ICLR 2025
Based on the union bound of probability, further have:
P(3h H,,8,, (h) {(h) > =) = P((66, (mn) - -((h)) v (1)p) -(hh al)
(18)
Σ P(LED, (h) - E(h) > E).
NEW
addition, based on the Hoeffding's Inequality, we have
P((ED, (h) E(h) > s) s 2 exx((- -2ne
(19)
WEH
We then let exp( 2n 8/|H1, where < 1 and have
P(EED, (h) E(h)) > $) - Σ
8/|H| < [H] 8/ = 8.
(20)
KEH
NCH
That is, when eexp( -2ne)) = 8/1711, we have
Σ
P(LED, (1h) E(h)) 8) S.
(21)
hER
According 2 (xxp(-2ns)) 8/1H), we can obtain E Eq. (21) as
In (11+))(2288)
Moreover, we can rewrite
2n
P(vh H. H. ( E(h)) 5 2 2-
(22)
Replace with
V
In (1(+1((288)
further have
2n
In H7 + In(2/5)
P vh E H. ED, (h) E(h) S
2 11-5.
(23)
2n
Therefore, with probability at least
5, we have
In |HL| + ln(2/0)
E(h) &D. (h) +
(24)
2n
For simplicity, omit the probability notation and use U represent the upper bound. We also omit the In term and use P to represent the parameter space. Therefore, |P] quantifies the size of the parameter space, sampled data D, and the trained parameters P are variables of the training error E and we have
8 E O(VIPl/n)
(25)
Thus we complete the proof.
Based on Lemma C. we can further derive the lowest generalization error bound for pre-trained HGNN models during the prompt-tuning stage, which benefit from the parameter optimization by the pre-training stage.
Theorem C.2. (Generalization error bound for prompt-tuning-bet methods.) Statistically, the upper bound U(EM) of the test error Exs of a pre-trained HGNN model with prompt-tuning can be determined as follows:
u(EM) = (P PM) +
(26)
where training data DrN and prompt-tuning parameters PM are variables of the training error Esd of the model in prompt-tuning stage. The number of training samples nas and the size of parameter space (Pasl are variables of the generalization gap bound between training error and test error. Moreover, when n& fixed, there exist optimal [Pa] to achieve the lowest upper bound for prompt-tuning-bunt methods, i.e.,
min(U((EM)) Exr(Dan PM) 0 PM 1/nMs)
(27)
16

---

Published as a conference paper at ICLR 2025
Proof. We first derive the generalization bound for existing coompt-uuning-basee methods. Based on Lemma C.I, I, if we ignore the pre-trained model, we can obtain the generalization bound of existing prompt-tuning methods, i...
=
= E(D Pas) + O [PM /nas):
(28)
However, training error of the model prompt-tuning stage will benefit from the better initial- ization with the pre-trale pre-trained stage. Actually, such benefits B on the training error be written as follows.
B == & (D PM) EN (Dasr PM).
(29)
Therefore, we can obtain the generalization error bound for existing prompt-tuning-asee methods as follows.
u(EM)==((DA)) = & Pas) + O(ViPall////
E((DuM: PM)) + ((v /Pa /MM) E(DrM, PM) Ese Pas) (D PM) + (Pal//mm)
(30)
Based on the generalization bound, to examine whether the lowest upper generalization bound exists, we derive the partial derivative of 21 (Eas) with respect to [PM] i.e.
M(U(EM)) 0(E. (DaM: PM)))
a0(
PP /nw)
0 1PPrr
1PPMsl
i |Pas
(31)
olEs (P PM))
+0L1/ (PM/ nm).
0 1PPas
In addition, according to the generalization bound, we have the observations as follows. First, with the increase of parameters PM, a larger parameter size confers stronger optimization ability, leading the decrease of the training error (i.e., Ens (Duns Pas)) in Eq. (30). In addition, the generalization gap bound (i.e., O O(viPM) /n increases. Therefore, M (DaM PM))
while 0(1/ nMr) > As result, there exist a optimal size of Pas (i.e. PM which enables the derivative of L (Exr with respect to |PMr equals to O. Therefore, we can achieve the lowest upper bound for prompt-tuning based methods, i.e.
JPPMl
min (((E))))) = Em (D PM) - +000 PM /nM)).
(32)
Thus we complete the proof.
C.2 COMPARISON OF THE GENERALIZATION ERROR BOUND
Theorem C.3. (Generalization error bound for the proposed HG-Adapter.) With dual structure- aware adapters and potential labeled data extension, the proposed HG-Adapter decreases both the training error and the generalization gap achieve a lower generalization error bound (EA)) than that of existing prompt- toato-ttnnnn--uen methods (i.e., ((as))osch ie..
U(EA)mA < <U(EA)mas < ((EM))nss
(33)
where nA indicates the number of training data for the proposed HG-Adapter.
Proof. According to the Theorem 2.3, we have the generalization error bound of cxisting prompt- tuning-based methods, i.e.,
(E Emr E (Dau. PM) + OMVPPPM//n /nM)
(34)
Moreover, given the fixed size of the training data n. we can obtain the lowest generalization enror bound when the model achieves the optimal parameters P M. i.e.,
min(U(EEm) = EM (Drar PM) O / PM //MM).
(35)
Therefore, when the training data is fixed as n. we see that the generalization error bound existing prompt-tuning-basee baase methods follows the U-shaped behavior. That is, the size of parameters P
17

---

Published as a conference paper at ICLR 2025
Table Statistics all datasets.
Datasets
#Nodes
#Node Types
#Edges
#Edge Types Target Node/Edge #Training
#Test
#Class
ACM
8,994
25,922
Paper
600
2,125
3
Yelp
3,913
72,132
Bussiness
300
2,014
3
DBLP 18,405
67,946
Author
800
2,857
Aminer 55,783
153,676
Paper
1,000
HBN-B
15,322
181
10
Drug-Target
7,875
875
2
Ogbn-mag 1,939,743
36,805,743
Paper
625,930
66,275
349
increases, the generalization error of the model first decreases until the generalization error is lowest at the optimal parameters PM. and then the size of parameters PM increases, the generalization error of the model increases. As we mentioned above, previous methods generally ignore the graph structures and thus may not be sufficient to model the input data and increase the training error EM-- As result, structure-awar adapters to tune the heterogeneous and homogeneous graph structures by increasing few parameters. Then the parameters PA are expected to closer the optimal parameters P than the parameters MM, i.e.
obtain that [Par] < 1- To solve this issue, the proposed method designs dual
|Pa| PPal < 1|PM| PP
(36)
We then have
U (EA) min((((Ess) u (EM) min()((M))) Then, when the training data is fixed n,,, we can obtain
(37)
U(E))am < <u(EM))
(38)
In addition to the dual adapters, further conduct the potential labeled data extension by designing the aabel-propagated contrastive loss and two self-supervised losses, thus incorporating unlabeled nodes supervision signals. Therefore, the size of training data after extension (i.e., nA is supposed to be larger than the original size of training data n Therefore, we have
O(ViPAl/nA) O(viPAl /nM)-
(39)
We then have
u(EA)mA <u(EA)mN < <u(EM)
(40)
Thus we complete the proof.
D EXPERIMENTAL SETTINGS
This section provides detailed experimental settings in Section Experiments, including the description of all datasets in Section D.I, summarization of comparison methods in Section D.2, evaluation protocol in Section D.3, model architectures and settings in Section D.4, and computing resource details in Section D.5.
D.1 DATASETS
We use four public heterogeneous graph datasets from various domains including three academic datasets (i.e., ACM (Wang et al., 2019), DBLP (Wang et al., 2019), and Aminer (Hu et al., 2019)). and one business dataset (i.e., Yelp (Zhao et al., 2021)). Table 3 summarizes the data statistics. We list the details of the datasets as follows.
ACM is academic heterogeneous graph dataset. It contains three types of nodes (paper (P), author (A), subject (S)), four types of edges (PA, AP, PS, SP), and treats categories of papers labels.
18

---

Published as a conference paper at ICLR 2025
Yelp is business heterogeneous graph dataset. It contains three types of nodes (business (B), user (U), service (S), level (L)), six types of edges (BU, UB, BS, SB, BL, LB), and treats categories of businesses as labels.
DBLP an academic heterogeneous graph dataset. contains three types of nodes (paper (P), authors (A), conference (C)), four types of edges (PA, AP, PC, CP), and treats research areas of authors as labels.
Aminer is academic heterogeneous graph dataset. It contains three types of nodes (paper (P), author (A), reference (R)), four types of edges (PA, AP, PR, RP), and treats categories of papers as labels.
HBN-B is a biomedical heterogeneous graph dataset. contains four types of nodes (drug (Dr), target (T), disease (Di)), side-effect (S), ten types of edges (DrT, DrDr, DrDi, DrS, TT, TDi, TDr, DiDr, SDr, DiT), and treats the existence of drug-targer interaction as labels.
Ogbn-mag is a large-scale academic heterogeneous graph dataset. It contains four types of nodes (paper (P), Author (A), Institution (I), Field (F)), seven types of edges (PA, PP, PF, AI, AP, FP, IA), and treats categories of papers as labels.
Table 4: The characteristics of all comparison methods.
Methods
Supi-supe
Fine-tuning
Prompt- tuning
Adapter-tuning
HAN (2019)
HGT (2020)
DMGI (2020)
HDMI (2021)
HeCo (2021)
HGCML (2023)
HGMAE (2023)
HERO (2024)
HGPrompt (2024)
HctGPT (2024)
HG-Adapter (ours)
D.2 COMPARISON METHODS
The comparison methods include two traditional semi-supervised methods (i.e., HAN (Wang et al., 2019) and HGT (Hu et al., 2020b)), six fine-tuning- based methods (i.e., DMGI (Park al., 2020), HDMI (Jing et al., 2021), HeCo (Wang et al., 2021), I), HGCML (Wang et al., 2023), HGMAE (Tian al., 2023), and HERO (Mo et al., 2024)), and two prompt-tuning-basee methods (i.e., HGPrompt (Yu et al., 2024a) and HetGPT (Ma et al., 2024)), where the pre-training and prompt-tuning of HGPrompt are specifically designed, while HetGPT only designs the prompt-tuning and thus can be used for different pre-trained models. The characteristics of all methods are listed in Table 4, where "Semm--upprrvieed"" "Fine-tuning", "Prompt-tuning" and "Adapter-tuning" indicate that the method conducts semi- supesviee learning, fine-tuning prompt-tuning and adapter-tunin respectively.
D.3 EVALUATION PROTOCOL
We follow the evaluation protocol in previous works (Jing al., 2021; Pan & Kang, 2021; Zhou et al., 2022) to conduct node classification and node clustering semi-supervised and unsupervised downstream tasks, respectively.
For fine-tuning-bar methods, we first train models with unlabeled data in self-supervised manner and output learned node representations. After that, the resulting representations can be used for different downstream tasks. For the node classification task, we train a simple logistic regression classifier with fixed iteration number, and then evaluate the effectiveness with Micro-FI and Macro- Fl scores. For the node clustering task. we conduct clustering and split the learned representations into e clusters with the K-means algorithm, then calculate the normalized mutual information (NMI) and average rand index (ARI) to evaluate the performance of node clustering.
19

---

Published as a conference paper at ICLR 2025
10
10
12
4
0.33
Structure-tuning
01.01
7
0.74
Label propagation
8 0.17
8
13
Figure 3: Toy example sampled from the ACM dataset, where blue nodes belong to class and yellow nodes belong to class 2. The nodes with dashed lines unlabeled nodes in the test set, and the nodes without dashed lines are labeled nodes in the training set. The left side is the original graph with edge weights of and the right side the graph after adapter-tuning, where labeled nodes are extended and edge weights are tuned.
with ailic
Munu Jun
Todd M Mmm
M
(a) ACM
(h) Yelp
(c) DBLP
(d) Aminer
Figure 4: The training error of the proposed method with and without the structure tuning on all heterogeneous graph datasets.
For prompt-tuning-bast methods and the proposed adapter-tuning-ban based method, the models are also pre-trained in unsupervised manner and output frozen representations. In the tuning stage, for the node classification task, we first obtain the predicted probability of each node by calculating the similarity between the prediction vector and each class-subgraph representation, and then using the softmax function to obtain class probabilities. Then, the class with the maximum likelihood for each node designated the predicted class. We also evaluate the effectiveness of models with Micro-F1 and Macro-Fl scores. For the node clustering task, we input the class probabilities to the K-means algorithm, then calculate the normalized mutual information (NMI) and average rand index (ARI) to evaluate the performance of node clustering.
D.4 MODEL ARCHITECTURES AND SETTINGS
As described in Section 2, the proposed method employs homogeneous and heterogeneous adapters to tune the homogeneous and heterogeneous graph structures (i.e., and S) and capture task-related structural information. Moreover, the proposed method employs the projection (i.e., go) obtain the prediction matrix P. In the proposed method, homogeneous and heterogeneous adapters are simply implemented by two linear layers, followed by the ReLU activation. In addition, the projection go is also simply implemented by the linear layer. Finally, In the proposed method, all parameters were optimized by the Adam optimizer (Kingma & Ba, 2015) with initial learning rate. In all experiments, we repeat the experiments five times for all methods and report the average results.
D.5 COMPUTING RESOURCE DETAILS
All experiments were implemented in PyTorch and conducted server with NVIDIA GeForce 3090 (24GB memory each). Almost every experiment can be done on an individual 3090, and the training time of all comparison methods well as method, is less than 1 hour.
20

---

Published as a conference paper at ICLR 2025
Table 5: Performance on biomedical and large-scale heterogeneous graph datasets, where the best results are highlighted in bold, while improved results with the proposed HG-Adapter are underlined. The "+" symbol indicates the integration of HG-Adapter and HetGPT with original pre-trained HGNN models. "OOM" indicates out-of-memoot "-" indicates that the method is not suitable for the task, indicates the mini-batch version method on Ognb-mag dataset.
HBN-B
Ogbn-mag
Method
AUC
AUPR
Accuracy
HAN
OOM
HGT
84.9+0.6
81.3+0.5
49.8+-0.9
DMGI
0OM
HGCML
OOM
HGMAE
86.310.6
87.610.2
OOM
IGGPrompt
86.6.0..5
87.810.4
0OM
HDMI
46.6=0.77
+HetGPT
47.1
+HG-Adapter
49.5
HcCo
86.1
87.5=0.7
48.7=005
+HetGPT
86.7++0.6
88.1
49.3+-0.8
+HG-Adapter
87.7++0.5
88.9--0.6
50.5+-0.7
HERO
85.6:t0.7
86.8+
49.5+-0.4
+HetGPT
86.0++0.6
87.1+-0.8
50.6+-0.3
+HG-Adapter
87.3+00.5
88.5++0.7
52.1 0.5
64k-
toik gooet
(a) ACM
(b) Yelp
(c) DBLP
(d) Aminer
Figure 5: The generalization gap (the difference between test error and training error) of the proposed method with and without the labeled data extension on all heterogeneous graph datasets.
E ADDITIONAL EXPERIMENTS
This section provides additional experimental results to support the proposed method, including the motivation illustration in Section 1. the experimental results on biomedical and large-scale datasets in Section E.2, the ablation study on the structure tuning in Section E.3, the ablation study on the potential labeled data extension in Section E.4, the ablation study on the parameters in the proposed method in Section E.5, the ablation study on the dimensions of dual adapters Section E.6, the ablation study on dual adapters in Section E.7, the ablation study of the margin loss in Section E.8, visualization of the learned representations in Section E.9, parameter analysis in Section 10, experimental results on the node clustering task in Table 8.
E.1 MOTIVATION ILLUSTRATION
In this paper, we point out that existing methods only focus on node features while ignoring graph structures, thereby increasing the training error. Moreover, existing methods may be constrained by the limited labeled data during the prompt-tuning stage, leading to large generalization gap. To further explain above motivation, we sample a few nodes from the ACM dataset and construct a toy example and implement adapter-tuning and label propagation where different colors indicate nodes from different classes, nodes with dashed lines are unlabeled nodes in the test set, and the nodes without dashed lines are labeled nodes in the training set.
it, and visualize them in Figure
21

---

Published as a conference paper at ICLR 2025
Table 6: Classification performance (i.e. Macro-FI and Miero-F1) of the variant methods without homogeneous and heterogeneous adapters on all heterogeneous graph datasets.
ACM
Yelp
DBLP
Aminer
Method
Macro-F1
Micro-FI
Macro-Fl Micro-F1
Macro-F1
Micro-F1
Macro-F1
Micro-FI
w/o hom-adapter
92.1110.6
92.0.0.55
91.110077
90.7110.8
83.7.1.2
84.5.11.0
73.2.00.8
82.3.0.99
w/o het-adapter
86.0.10.7
85.3.0.6
92.2.10.8
91.410.7
81.410.7
82.5.10.6
71.9.0.5
80.110.3
Proposed
92.7110.4
92.7.10.7 93.1 92.710.55
94.0.10.7
94.7.10.8
78.3.0..5 87.1
Table 7: Classification performance (i.e., Macro-F1 and Micro-F1) of the variant methods with the proposed margin loss and the InfoNCE loss on all heterogeneous graph datasets.
ACM
Yelp
DBLP
Aminer
Method
Macro-F1
Micro-FI Fl
Macro-F1
Micro-F1
Macro-F1
Micro-F1
Macro-F1
Micro-F1
InfoNCE loss
89.6+00.9
89.5+00.7
91.1+0055
91.4++0.6
92.1+088
93.0+0.7
75.8+00.4
02.6+0.5
Margin loss
92.7++0.4
92.7++0.7
93.1+0066
92.7++0.5
94.0+0.77
94.7++0.8
78.3+0.5
87.1+-0.6
From Figure 3, we have the observations as follows. First, if we only tune the node features and ignore tuning the graph structures, some nodes the training set may be misclassified, thereby increasing the training error. For example, for node 11, it will aggregate much information from the nodes (nodes 10, 12, 13) of another class after the pesssage-passingg with the original graph structures. Therefore, this may cause nodes to confuse their own class information, thus increasing the training error. In contrast, we tune both the node features and ignore tuning the graph structures, the misclassified node 11 may be corrected by re-weighting the edge weight. As result, the proposed method decreases the training error thus improve the model generalization. Second, compared with unlabeled nodes, the ratio of labeled nodes very small, result in a large generalization gap between the training еггоr and the test error. However, after the label propagation, the number of labeled nodes increases greatly. As a result, the proposed method decreases the generalization gap thus improve the model generalization.
E.2 EFFECTIVENESS ON DATASET FROM OTHER DOMAINS AND LCAGE-SCALE DATASET
To further verify the model's generalization ability across different domains, we evaluate the proposed method on the biomedical heterogeneous graph dataset HBN-B (Li et al., 2022) and the large-scale heterogeneous graph dataset (i.e., Ogbn-mag -ma (Hu et al., 2020a)), and report the results in Table
Obviously, on the biomedical heterogencous graph dataset HBN-B, the proposed method consistently obtains improvements on the pre-trained HGNNs (i.c., HeCo and HERO). For instance, the proposed method on average, improves by 1.3%, compared to the baseline method HeCo in terms of AUC and AUPR. In addition, the proposed HG-Adapter also obtains significant improvements to the prompt-tuning method. For instance, the proposed method on average, improves by 1.3%, compared to the best prompt-tuning method (i.e., HGPrompt) in terms of AUC and AUPR. Therefore, the effectiveness and generalization ability of the proposed method is further verified datasets from different domains. In addition, on the large-scale heterogeneous graph dataset Ogbn-mag, the proposed method always obtains promising results, compared to the original baselines (HDMI, HeCo, and HERO) as well as the mumpt--uuning-bsee method (i.e., HetGPT). For example, the proposed method improves by 5.3% and 3.0%, compared to the baseline method HERO and prompt-tuning based method HetGPT, respectively, in terms of Accuracy. Therefore, the effectiveness and scalability of the proposed method are further verified.
E.3 EFFECTIVENESS OF THE STRUCTURE TUNING
The proposed method designs the dual adapters to capture more task-related structural information than prompt-tuning-batt based methods. To verify the effectiveness of the structure tuning with dual adapters, we investigate the training eπror of the proposed method with and without the structure tuning, and report the results in Figure 4. Obviously, as the number of epochs increases, the training
22

---

Published as a conference paper at ICLR 2025
1801
18000 Muper Dir 6.0022
(a) ACM
(b) Yelp
(c) DBLP
(d) Aminer
Figure 6: The test errors of the HERO with prompt-tuning on features, adapter-tuning on features, and adapter-tuning on both features and structures on all heterogeneous graph datasets.
error of the proposed method with structure tuning continues to decrease and finally tends to 0. This indicates that the proposed method fits the input data well. Moreover, the proposed method with structure tuning obtains consistently lower training error than the method without structure tuning. The reason can be attributed to the fact that the structure tuning enables the model to fit the input data better and get closer to the optimal parameters P thus reducing the training error and improving the model's generalization ability.
E.4 EFFECTIVENESS OF THE POTENTIAL LABELED DATA EXTENSION
The proposed method designs the potential labeled data extension to incorporate both labeled and unlabeled nodes supervision signals, thus alleviating the limited labeled data in the tuning stage. To verify the effectiveness of the potential labeled data extension, we investigate the generalization gap of the proposed method with and without the label extension and report the results in Figure 5. From Figure 5, we can find that the proposed method with the potential labeled data extension consistently achieves smaller generalization gap than the method without label extension. This is reasonable because the label extension increases the number of training samples potentially, thus
decreasing the generalization gap bound O(y PM /nM) and further decreasing the generalization error bound of existing methods.
E.5 EFFECTIVENESS OF THE PARAMETERS IN THE PROPOSED METHOD
The proposed method designs dual adapters with parameters Pal to fit better the input data to decrease the training error, thus decreasing the upper bound of the test error and approaching the optimal parameters [P To verify that |P^] is indeed closer the optimal parameters |Par] than existing wrompt-uuning-baeee methods, we remove the label extension module and fix the number of training samples, and then implement several variant methods (i.e., adapter-tuning on both node features and graph structures, adapter-tuning on only node features, and prompt-tuning on only node features), and report the results in Figure 6. Obviously, when the number of training samples is fixed, the proposed adapter-tuning test error than the prompt-tuning and adapter-tuning on only node features. a result, we can obtain that the parameters |PAl of the proposed adapter-tuning is indeed closer to PM than existing prompt-tuning-be methods.
node features and graph structures always obtains lower
F.6 ABLATION STUDY OF THE DIMENSION OF DUAL--DAPTERS
Theorem 2.3 indicates that the upper bound of the test error exhibits a U-shaped pattern, where initially decreases and then increases as the number of parameters grows. Therefore, we cannot further improve the performance by simply increasing the size of dual adapters. To verify it, we conduct the ablation study by varying the size of each adapter, and report the results in Figure 7. From Figure 7, we can find that that as the adapter size increases, the performance of the model may first improve, and then decrease when the size is too large. This is consistent with results above, i.c., as the parameter size increases, the upper bound of the test error decreases first and then increases. Correspondingly, the performance of the model may increase first and then decrease.
theoretical
23

---

Published as a conference paper at ICLR 2025
160
55
116
Macro-FI (50)
85
Macro-FF
40
ACM
ACM
Yel
DBLUP
DRLP
Aztsa
Antze
125
256
1804
3005
256
512
1124
240
Dimersion
Dimension
(a) Homogencous adapter
Heterogencous adapter
Figure 7: The performance of the proposed method under different adapter dimensions and corre- sponding parameter sizes.
(a) HERO+Fine- Finn--Tinnnn (SIL: 0.31) (b) HERO+HeIGPT (SIL: 0.38) (c) HERO+HG-Adapter (SIL: 0.44)
Figure Visualization plotted by t-SNE and the corresponding silhouette scores (SII.) of the representations learned by HERO with fine-tuning, prompt-tuning (i.e., HetGPT), and the proposed adapter-tunin tuning on the DBLP dataset, respectively.
E.7 ABLATION STUDY OF DUAL ADAPTERS
The proposed method designs dual adapters to adaptively tune the homogeneous and heterogeneous graph structures (i.e., and S) to capture the task-related information. We demonstrate the effec- tiveness of dual adapters in Section 3.2.2. To further verify it. investigate the node classification performance of the variants methods without the homogeneous and heterogeneous adapters, respec- tively, and report the results in Table 6. Obviously, the proposed method with dual adapters obtains superior performance than the variant methods without homogeneous and heterogeneous adapters. The reason can be attributed to the fact that dual adapters tunes homogeneous and heterogeneous graph structures simultaneously, thus can better fit the input data than variant method without adapters achieve lower generalization enror bound and better performance on different downstream tasks. As result, the effectiveness of dual adapters is verified again.
E.8 ABLATION STUDY OF THE MARGIN Loss
The proposed method designs the margin loss to optimize the heterogencous graph structure S and treat both labeled and unlabeled nodes as equal supervision signals. The margin loss aims to decrease the distance d(cy,, 2, while increasing the distance d(cs,, m,)² thus satisfying the "safe" distance between them. To verify the effectiveness the margin loss, we investigate the node classification performance of the variants method with the InfoNCE loss instead of the proposed margin loss and report the results in Table From Table 7, we can find that the proposed method with the margin loss obtains better performance than the variant method with the InfoNCE loss. This can be attributed to the fact that directly aligning subgraph representation and adapted representation unreasonable since they come from different feature distributions.
24

---

Published as a conference paper at ICLR 2025
(a) ACM
(b) Yelp
(c) DBLP
(d) Aminer
Figure 9: The classification performance of the proposed method at different parameter settings (i.e., a and 3) on all heterogeneous graph datasets.
(a) ACM
(b) Yelp
(c) DBLP
(d) Aminer
Figure 10: The classification performance of proposed method at different parameter settings (i.e. y and (1) on all heterogeneous graph datasets.
E.9 VISUALIZATION OF LEARNED REPRESENTATIONS
To further verify the effectiveness of the learned representations, visualize node representations learned by the combinations of pre-trained HERO and different tuning methods (fine-tuning, prompt- tuning, and proposed adapter-tuning) on the ACM dataset and report the results and corresponding silhouette scores (SIL) in Figure 8. Obviously, in the visualization, the node representations learned by the proposed HG-Adapter exhibit better clustering status, i.e., nodes with different class labels are more widely separated. Moreover, the representations learned by the proposed method obtain the best silhouette score, compared to other methods (i.e., fine-tuning, and HetGPT), The reason can be attributed to the fact that the proposed method designs a contrastive loss based on class subgraph similarity to ensure that nodes within the same class are close to each other, thereby enhancing clustering performance.
E. PARAMETER ANALYSIS
In the proposed method, we employ non-negative parameters a and 3 to achieve trade-off between the adapted representations and the frozen representations. Moreover, we employ non-negative parameters 17 and μ between different terms of the final objective function J. To investigate the impact of and 3 as well as 77 and μ with different settings, we conduct the node classification on all heterogeneous graph datasets by varying the value of parameters in the range of [10-3 3,10² and reporting the results in Figure and Figure 10.
From Figure 9, we can find that if the values of parameters (i.e., and (3) are too large (e.g.. 103), the proposed method cannot achieve satisfactory performance. This verifies that both the adapted representations and the frozen representations are important for downstream tasks. In addition, from Figure 10, we can also find that if the values of parameters (i.e., r) and μ)) are too large (e.g., 102), the proposed method obtains inferior performance. The reason can be attributed to the fact that when and μ are large, the effect of the contrastive loss Loon may be affected, thus failing to provide sufficient label guidance for the model.
25

---

Published as a conference paper at ICLR 2025
Table 8: Clustering performance (i.e., NMI and ARI) on all heterogeneous graph datasets, where the best results are highlighted in bold, while improved results with the proposed HG-Adapter are underlined. The "+" symbol indicates the integration of HG-Adapter and HetGPT with original pre-trained HGNN models.
ACM
Yelp
DBLP
Aminer
Method
NMT
ART
NMI
ARI
NMI
ART
HAN
6..6
6..4+
37.8-0.9
40.1+1.1
77.5+0.7
83.0+
35.5+0.6
31.6++..5
HGT
68.9+0.9
69.9+.0.8
39.1=0.6
41.2++0.7
78.6++0.4
83.9+0.6
36.1+0.6
32.3+0088
DMGI
67.8+0.9
70.2+1.0
36.8=0.6
34.4+0.7
72.240.8
72.8+0.9
27.3+0.9
23.1+0.8
HGGML
69.1100.7
71.610.8
37.4_0.6
39.51008
74.510.9
75.111.1
35,910.6
31.110..
HGMAE
69.7.10.8
72.610.66
40.3-0.9
42.41008
76.9110.6
82.310.7
41.1108
38.310.9
HGPrompt
69.2.10.4
72.010.5
37.
39.7110.7
76.110.6
81,2.0088
37.2.1099
33.811.1
HDMI
69.5.10.5
72.3.0.77
38.9-0.6
40.7.10.8
73.1
74.4:00.4
33.5.0.4
28.9.10.5
*HeGPP
70.1+:0.6
72.8+0.8
39.2-0.6
41.5+00.7
74.81009
75.
34.4
29.310.6
+HG-Adpater
70.4-10.7
73.14:0.6
39.7-0.5
42.24:04
75.9410.6
80.7
35.1+0.9
30.9+110
HeCo 67.8+40.8
70.5+-0.7
3.63-0.6
42.1+0.8
74.5+-0.8
80.1+0..9
32.2+1.1
28.6++1.0
*HctGPT
68.24:0.6
70.8+0.5
40.1-0.7
42.5++0.9
75.1+005
80.5+
32.7+0044
29.24-0.6
+HG-Adpater
69.0:10.4
72.910.3
0.5
43.1
76.310.9
81.91.0
34.1
30.3100.8
HERO
68.8++0.6
71.8+0.66
38.6-0.8
40.6+00.9
74.1+0.7
79.
36.8+0.7
35.3++0.9
+HetGPT
69.7+0055
72.3+0044
39.3-0.7
41.3++0.9
74.6+0.66
80.4+0.5
40.2+006
37.1+0.7
+IG--Adppter
7..5.10.8
73.3.0..9
0.6
42.811..5
76.2.0.88
81.4.0.7
42.
39.4
POTENTIAL LIMITATIONS AND FUTURE DIRECTIONS
In this paper, we design dual adapters to capture task-related structural information, thus approaching the optimal parameters and benefiting the model's 's generalization ability. Moreover, we design contrastive loss, feature reconstruction loss, and margin loss to optimize dual adapters as well as extend potential labeled data. Actually, the feature reconstruction loss relies the feature-label consistency assumption. That is, two nodes have the same label, their node features will be similar. However, there few cases in which nodes have the same label but totally different node features. In that case, the learned graph structure may suffer from noisy connections. addition, the quality of the propagated labels of unlabeled data will be affected correspondingly because it relies the learned graph structure. To solve these issues, we may conduct class feature reconstruction instead of node feature reconstruction to address this issue. We consider these aspects potential directions for future research.
26


Reviews:
[{"review_id": "GJGH209uY8", "reviewer": "Reviewer_hjNU", "summary": "A dual-adapter approach has been introduced to graph prompt-tuning methods. The key insight of this work is to leverage dual adapters to capture both node features and graph structures to realize prompt tuning for pre-trained HGNN models. Experimental results on four benchmark datasets were provided in terms of two validation metrics.", "questions": "- Can the authors more clearly articulate the novel aspects of their approach, particularly in combining dual adapters with prompt tuning for heterogeneous graphs? \n- Are there any specific insights or hypotheses about prompt-tuning for HGNNs that motivated the proposed approach? It would also be helpful to explain how to find or approximate the optimal parameter size $|\\bar{P}_M|$ and provide empirical evidence or further justification to demonstrate that $|P_A|$ is closer to $|\\bar{P}_M|$. \n- Can the performance be further improved by varying the size of the dual adapters? An ablation study showing the impact of each adapter individually is also expected.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- A new dual-adapter approach, applied to different pre-trained HGNN models, has been developed to enrich the current graph prompt tuning methods. \n- Two self-supervised graph learning loss functions have been designed to realize label augmentation, showing a clear improvement in the ablation study given in `Table 2`. \n- The paper provides two orthogonal solutions to improve the generalization error bound, i.e., 1) improving prompt flexibility/complexity by dual adapters and 2) improving data efficiency by designing self-supervised label augmentation.", "weaknesses": "- **Limited Technical Novelty**: The design of dual adapters and self-supervised losses builds on top of several well-established technologies. Thus, the technical contribution of this work is relatively limited since no new loss functions or adapter architectures have been developed. \n- **Unclear Insights for HGNN**: It remains unclear what new insights or hypotheses have been introduced to HGNNs by this work. It seems common to improve the generalization bound by increasing parameters and labels. Are there any specific designs or findings for prompt-tuning of HGNNs? The theoretical results in `Theorem 2.3` and `Theorem 2.4` are relatively weak, where the former only proves the existence of $|\\bar{P}_M|$ without giving guidance on how to tune or find this optimal size, while the latter builds on *a very strong assumption* that $|P_A|$ is expected to be closer to $|\\bar{P}_M|$ without showing evidence.\n- **Marginal Improvement**: As shown in `Table 1`, the improvement of the proposed HG-Adapter over existing graph prompt tuning methods, e.g., HGPrompt and HetGPT, is quite marginal and generally less than 1% in three of four datasets. Per the provided theoretical results, can we further improve the performance by increasing the size of dual adapters? It would also be helpful to provide an abolition study on each adapter and give a parameter analysis in terms of adapter size.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ti6jM184DF", "reviewer": "Reviewer_NAje", "summary": "In this paper, a unified pre-trained and prompt tuning framework is proposed by combining two new adapters with potential labeled data extension to improve the generalization of pre-trained heterogeneous graph neural networks. In the proposed method, dual structure-aware adapters are adopted to fit task-related homogeneous and heterogeneous structural information. Meanwhile, three losses are designed, including a label-propagated contrastive loss and two self-supervised losses. The ablation studies show that each component is very essential in the proposed method.", "questions": "1)\tIn the main contributions, the authors has highlighted that “it is the first dedicated attempt to design a unified “pre-train, adapter-tuning” paradigm to improve different pre-trained HGNN models”. While HGPrompt (Yu et al., 2024a) is designed with both pre-training and the prompt-tuning. Hence, I wonder whether the first effort is suitable.\n2)\tIn the experiments, for the method HERO, the improvements are not satisfactory except on Aminer. Hence, I wonder the complexity between HG-Adapter and HetGPT.\n3)\tAlthough the proposed method uses the self-supervised technique to improve the confidence of propagated labels, I do not think it can guarantee the accuracy of the propagated labels. Hence, I wonder if the propagated labels are too noisy, whether the proposed method can obtain satisfactory performance.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "In the proposed method, dual structure-aware adapters are designed to capture task-related homogeneous and heterogeneous structural information. A label-propagated contrastive loss and two self-supervised losses are proposed to achieve the potential labeled data extension. Meanwhile, a theoretical analysis also has been given to verify the effectiveness of the proposed method.", "weaknesses": "The motivation should be further explained with some toy examples. In the analysis part of challenges, although three limitations are presented, how to demonstrate that their drawbacks really exist in the real-world applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7ZojeL2KMS", "reviewer": "Reviewer_J7pK", "summary": "Summary of the paper \n\nThe paper pointed out the major limitations remaining currently in the \"pre-train, prompt-tuning\" paradigm for heterogeneous graph neural networks (HGNNs), which are :the insufficient adaptation of graph structures during prompt-tuning and the challenges posed by limited labeled data. To mitigate these issues, the authors derived the generalization error bound for existing prompt tuning-based methods, and then proposed a unified framework that combines two new adapters with potential labeled data extension to improve the generalization of pre-trained HGNN models.This novel approach aims to improve the generalization capabilities of pre-trained HGNNs across various downstream tasks.\n\nContributions \n+ Designing dual structure-aware adapters to capture task-related homogeneous and heterogeneous structural information. Moreover, we design a label-propagated contrastive loss and two self-supervised losses to achieve the potential labeled data extension.\n+ Deriving a unified generalization error bound for existing methods based on the training error and the generalization gap. Moreover, we demonstrate that the proposed method achieves a lower generalization error bound than existing prompt-tuning-based methods to improve the generalization ability of pre-trained HGNN models.\n+ Validating the superior effectiveness and generalization of the proposed HG Adapter compared to state-of-the-art fine-tuning-based and prompt-tuning-based methods, demonstrating its adaptability to different pre-trained HGNN models by experiments.\n\nMerits\n+ Significant shortcomings in existing prompt-tuning approaches were pointed clearly and accurately in this paper , such as the neglect of graph structures and the constraints posed by limited labeled data.\n+ Solid theoretical foundation for the proposed framework was provided for the proposed framework in the generalization error bound. And methods are proven and discussed mathematically.  \n+  The introduction of dual structure-aware adapters is a noteworthy contribution, as it allows for the adaptive integration of task-related structural information from both homogeneous and heterogeneous graphs.\n+ Experiments are comprehensive enough to demonstrate the effectiveness and generalization of the proposed method across different tasks add credibility to the claims made.\n\nWeaknesses:\n+ The paper could benefit from clearer explanations of key concepts, particularly around the implementation of the dual structure-aware adapters and the label-propagated contrastive loss.\n+ While the empirical results show improvements over existing methods, a more detailed comparative analysis with specific baselines would enhance the reader's understanding of the method's relative performance.\n+ The paper would be stronger with a more explicit discussion of the limitations of the proposed method, particularly in relation to scenarios with highly variable graph structures or the quality of unlabeled data.\n+  It might be recommendable to talk about future work considerations to make background and trend of the overall research clear.", "questions": "See Summary", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "See Summary", "weaknesses": "See Summary", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YTsKB3DO37", "reviewer": "Reviewer_PQvi", "summary": "The paper \"HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual Adapters\" introduces a new framework to enhance the generalization of pre-trained heterogeneous graph neural networks (HGNNs). It addresses two key challenges: insufficient focus on graph structures during tuning and a lack of labeled data, leading to a generalization gap. The proposed HG-Adapter employs dual adapters to capture both homogeneous and heterogeneous graph patterns, improving task-specific performance. It also incorporates a label-propagated contrastive loss and self-supervised learning to utilize unlabeled data, effectively expanding the labeled dataset. This helps reduce the generalization error between training and testing phases.", "questions": "please see the weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper introduces a novel approach by using dual adapters to capture both homogeneous and heterogeneous graph structures. This allows for better adaptation to specific graph patterns, improving the model’s generalization capabilities across diverse downstream tasks.\n\n2. By incorporating a label-propagated contrastive loss and self-supervised learning, the paper effectively leverages unlabeled data to extend the training dataset. This approach helps overcome the limitation of scarce labeled data, which is a common challenge in heterogeneous graph neural network applications.\n\n3. The paper provides a solid theoretical foundation by deriving generalization error bounds for prompt-tuning methods. Additionally, it validates the proposed HG-Adapter through extensive experiments, demonstrating superior performance compared to state-of-the-art fine-tuning and prompt-tuning techniques across multiple datasets.", "weaknesses": "1. While the proposed HG-Adapter framework is innovative, its dual adapter system and the integration of multiple losses (label-propagated contrastive loss and self-supervised learning) might make the model computationally expensive and challenging to implement in real-world settings with limited resources.\n\n2. The experiments in the paper, while comprehensive, may be limited in the diversity of graph types evaluated. A more extensive validation across different types of heterogeneous graphs, particularly in domains beyond those tested (e.g., biological networks, industrial applications), would provide stronger evidence of the model's generalization ability.\n\n3. Although the framework shows improved performance, its scalability to very large datasets or extremely large graphs is not thoroughly addressed. Given that graph neural networks are often used in scenarios involving large-scale data, the paper could benefit from further analysis on how the model performs with increasing graph size and complexity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)