Edge Prompt Tuning for Graph Neural Networks
Authors: Xingbo Fu
Yinhan He
Jundong Li
Venue: ICLR 2025 Poster (2025)
Keywords: Graph Neural Networks
Prompt Learning
Primary area: learning on graphs and other geometries & topologies
TL;DR: A graph prompt tuning method from the perspective of edges.

Abstract: Pre-training powerful Graph Neural Networks (GNNs) with unlabeled graph data in a self-supervised manner has emerged as a prominent technique in recent years. However, inevitable objective gaps often exist between pre-training and downstream tasks. To bridge this gap, graph prompt tuning techniques design and learn graph prompts by manipulating input graphs or reframing downstream tasks as pre-training tasks without fine-tuning the pre-trained GNN models. While recent graph prompt tuning methods have proven effective in adapting pre-trained GNN models for downstream tasks, they overlook the crucial role of edges in graph prompt design, which can significantly affect the quality of graph representations for downstream tasks.
In this study, we propose EdgePrompt, a simple yet effective graph prompt tuning method from the perspective of edges. Unlike previous studies that design prompt vectors on node features, EdgePrompt manipulates input graphs by learning additional prompt vectors for edges and incorporates the edge prompts through message passing in the pre-trained GNN models to better embed graph structural information for downstream tasks. 
Our method is compatible with prevalent GNN architectures pre-trained under various pre-training strategies and is universal for different downstream tasks.
We provide comprehensive theoretical analyses of our method regarding its capability of handling node classification and graph classification as downstream tasks.
Extensive experiments on ten graph datasets under four pre-training strategies demonstrate the superiority of our proposed method against six baselines. Our code is available at https://github.com/xbfu/EdgePrompt.

Published as a conference paper at ICLR 2025
EDGE PROMPT TUNING FOR GRAPH NEURAL NETWORKS
Xingbo Fu University of Virginia x rgavvvirginia....
Yinhan He University of Virginia nee ine@virgint edu
Jundong Li University of Virginia
jundongevirginfe edu
ABSTRACT
Pre-training powerful Graph Neural Networks (GNNs) with unlabeled graph data in a supervvee manner has emerged as a prominent technique recent years. However, inevitable objective gaps often exist between pre-training and down- stream tasks. To bridge this gap, graph prompt tuning techniques design and learn graph prompts by manipulating input graphs reframing downstream tasks pre-training tasks without fine-tuning the pre-trained GNN models. While recent graph prompt tuning methods have proven effective in adapting pre-trained GNN models for downstream tasks, they overlook the crucial role of edges in graph prompt design, which can significantly affect the quality of graph representations for downstream tasks. In this study, we propose EdgePrompt, simple yet effec- tive graph prompt tuning method from the perspective of edges. Unlike previous studies that design prompt vectors on node features, EdgePrompt manipulates in- put graphs by learning additional prompt vectors for edges and incorporates the edge prompts through message passing in the pre-trained GNN models to better embed graph structural information for downstream tasks. Our method is com- patible with prevalent GNN architectures pre-trained under various pre-training strategies and is universal for different downstream tasks. We provide compre- hensive theoretical analyses of our method regarding its capability of handling node classification and graph classification as downstream tasks. Extensive ex- periments on ten graph datasets under four pre-training strategies demonstrate the superiority of our proposed method against six baselines. Our code is available at Ittpss:/ thub. com xbfu http://bf////eeeeeeeeee Edwgerompp.........e
INTRODUCTION
Recent years have witnessed the remarkable success of Graph Neural Networks (GNNs) (Kipf &
Welling 2017: Hamilton et al. 2017 Veličković et al. 2018 Xu al. 2019 Wu et al. 2019] Chen
al. 2020 Wang al.) 2023b) for modeling ubiquitous graph-structured data in various real world scenarios, including social networks (Wei et al.) 2023] Zhou et al. (2023), point cloud analysis (Wang
ett al. 2019 Zhou et al., 2021), and healthcare systems (Fu al. 2023) Wan et al. 2024a Liu al.; 2024 Such success is mainly attributed to their impressive capability to incorporate node features and graph structures into the representations of graph data. Generally, GNN models are trained for specific downstream tasks in an end-to-end manner. Nevertheless, the end-to-end manner for train- ing powerful GNN models usually encounters significant challenges in practical deployments (Hu et al. 2020b Sun et al. 2022a Liu et al. 2023) Fang et al., 2023). First, annotating a sufficient number of labels for graph data is typically consumnnu and resource-intensie
in the real world. Second, well-trained GNN models cannot be well generalized to other tasks, even on the same graph data (Wang et al.) (024b). To grapple with these critical challenges, applying pre-training techniques on graph data has become increasingly prevalent.
Numerous recent studies have focused on designing effective pre-training strategies for training powerful GNN models without using any label information from downstream tasks (Veličkovid et al. 2019 Hu et al. 2020b; You et al.) 2020; Hou et al. 2022 Xia et al. [022,] Wang et al., 2024a Wan et al. 2024b). The philosophy behind these pre- pre-training strategies is to first train GNN model on pre-training tasks via supe-supe learning and subsequently transfer the pre-trained GNN model to specific downstream tasks. Generally, there exists inevitable objective gaps between

---

Published as a conference paper at ICLR 2025
Table 1: A brief comparison of graph prompt tuning methods in the existing studies. (PT=Pre-
training, DT=Downstream task)
Method
PT Compatibility
DT Universality
Prompt Insertion
GPPT (Sun et al. 2022a
Task Embedding
GraphPrompt [Lu
2023
Readout
GraphPrompt+ Yu al.
2024b
Hidden Representation
ALL-in-one (Sun el al.
2023
Node Feature
GPF-plus Fang et al. (223)
Node Feature
MultiGPrompt Yu et 2024d)
x
Hidden Representation
EdgePrompt+ (Ours)
Edge Aggregation
pre-training and the downstream tasks. For example, the GNN model can be pre-trained for link prediction via self-supervised learning, while the downstream task may be node classification. To bridge the objective gap between pre-training and downstream tasks, we typically need to adapt the pre-trained GNN model for downstream tasks by either fine-tuning or graph prompt tuning. During fine-tuning, the parameters of the pre- pre-trained GNN model are updated for downstream tasks (Huang al. 2024, Zhili et al. 2024 (un et al., 2024). Unlike fine-tuning, graph prompt tuning usually keeps the pre-trained GNN model frozen and instead trains graph prompts for downstream tasks (Sun iet al. 2022a Liu et al. 2023 Fang et al.
2023
Sun et al. 2023
Tan et al. 2023 Yu et al. 2024b
Ma et al. 2024 Yu et al. 2024a; Li et al.
2025
While recent graph prompt tuning methods show great prowess in adapting pre-trained GNN models for various downstream tasks, the existing methods still have several fundamental limitations. First, a few studies (Sun et al. 2022al Yu et al. 2024d) design graph prompt tuning methods based on specific pre-traiming strategies, which hinders their application to off-the-shelf pre-trained GNN models. Second, the important dependency information carried by graph structures is ignored in the existing studies (Fang et al.) [003] Liu al. [023) [un et al. 2023). As illustrated in Table T these methods focus on designing and learning graph prompts primarily by applying them to node features or node representations. In this scenario, graph prompts are unable enhance pre-trained
GNN models in capturing complex graph structural information for downstream tasks.
Although the significant role of edges in graph learning has been amplified by a comuucopia of studies (Schlichtkrull et al. [2018) Gong & Cheng) [2119] Vashishth et al.) 2020 Yang & Li 2020), unfortunately, none of the existing studies have exploited edges for graph prompt tuning. Naturally, we may ask a question: how can we devise an edge-level graph prompt tuning method to effectively enhance the performance of a pre-trained GNN model for downstream tasks? In this study, we aim to answer this question through pioneering investigation into designing edge prompts for downstream tasks. In our investigation, we need to overcome two key challenges. First, edge prompt design needs to be universal, capable of handling graphs of varying sizes and different downstream tasks, such as node classification and graph classification. Second, edge prompt design must be compatible with prevalent GNN models pre-trained by various strategies, especially with those that cannot accommodate edge attributes. These two challenges make the edge prompt design nontrivial, requiring an ingenious approach to graph prompt tuning.
To address the above issues, we propose a novel graph prompt tuning method named EdgePrompt purely from the perspective of edges, fundamentally differing from node-level prompt designs in the existing studies (Sun et al. 2023 Fang et al., 2023). The intuition of EdgePrompt is to manipulate the input graph by adding extra learnable prompt vectors to edges and thereby enhance the capa- bility of pre-trained GNN models for downstream tasks. In EdgePrompt. all the edges in the input graph learn a shared prompt vector at each layer of the pre-trained GNN model. The edge prompts will be aggregated along with node representations during the forward pass of the message-passinn mechanism. To further enhance the capacity of edge prompts, we propose an advanced version EdgePrompt+ that enables each edge to learn its customized prompt vectors. We provide theoretical analyses to support that our proposed method has the capability of enhancing the pre-trained GNN models for downstream tasks. We conduct extensive experiments over ten graph datasets under four pre- training strategies. The results validate the superiority of our proposed method compared with six baselines. Our contributions this study can be summarized as follows:
2

---

Published as a conference paper at ICLR 2025
We devise simple yet effective graph prompt tuning method, EdgePrompt and its variant EdgePrompt+, from the perspective of edges to narrow the objective gap between pre- training and downstream tasks.
We provide comprehensive analyses of our method regarding its capability of handling various downstream tasks, including node classification and graph classification.
We conduct extensive experiments over ten datasets under four pre-training strategies to evaluate the effectiveness of our proposed method. Experimental results demonstrate the superiority of our method compared with six baselines for both node classification and graph classification tasks.
RELATED WORK
Graph Pre-training. Numerous studies have proposed to train powerful GNN models via self-
supervised learning (Velčkkoiić et al. 2019 Sun et al., 2020 Hu et al. 2019 You et al. 2020 Jin et al. 2020; Rossi et al.) 2020: Xia et al. 2022 Hou et al. 2022 Wang et al. 2023a). These
studies can be roughly categorized into two genres: contrastive methods and generative methods. Contrastive methods typically aim to maximize the agreement between augmented instances of the same object. For instance, DGI (Veličković et al. 2019) and InfoGraph (Sun et al. 2020) adopt the mutual information maximization between the local augmented instances and the global repre- sentation. GraphCI. (You et al., 2020) maximizes the agreement between two views of the same graph by different augmentation strategies. SimGRACE (Xia al. 2022)) uses GNN models with perturbed parameters to obtain contrastive views without data augmentation. In the meantime, gen- erative methods attempt to pre-train GNN models by reconstructing specific information in the input graph. For example, GraphMAE (Hou et al., (022) pre-trains GNNs by reconstructing masked node features. In addition, edge prediction is also employed as the pre-training technique by a cornucopia of studies (Rossi et al., 2020, Jin et al., 2020; Sun et al. [022a Liu et al., 2023).
Graph Prompt Tuning. To bridge the gap between pre-training and downstream tasks, graph prompt tuning methods modify the input graph with learnable prompt vectors for downstream tasks, while keeping the pre-trained GNN model frozen. For example, GPF-plus (Fang et al. 2023) trans- forms the input graph prompted one by adding extra learnable prompt vectors to node features for downstream tasks. All-in-one (Sun et al., 2023) unifies various downstream tasks as graph-level tasks and similarly learns prompt vectors that are added to node features. GPPT (Sun et al. 2022a) mainly focuses on node classification as the downstream task and adopts link prediction as the pre- training strategy. It narrows the gap between pre-training and downstream tasks by converting node classification to link prediction. GraphPrompt (Liu et al. (023) designs graph prompts as a feature representations. MultiGPrompt (Yu al.)
weighting vector to obtain task-specific (sub)graph- 2024d) chooses to insert prompt vectors into node representations at each hidden layer. However, all the aforementioned studies ignore the role of edges when designing graph prompts, which are widely regarded as fundamental properties in graph data.
3 PRELIMINARIES
is the set of N nodes, and E is the edge set. X E NxD denotes the node feature matrix where the i-th row æ; represents D-dimensional feature vector of node v; E V. The edges in g can also be represented by an adjacency matrix A E {0, 1JNNN where each entry dg = if (1i, Uz) E E, otherwise aij 0. Generally, GNN models aim to learn expressive node representations through the message-pasee passing mechanism (Kipi) & Welling 2017: Veličković et al., 2018) Hamilton et al.) 2017] Xu et al. 2019) where the repre- sentation of a target node is iteratively updated by aggregating the representations of its neighboring nodes. Specifically, a GNN model has two fundamental operators: AGG(.) extracting the neighbor- ing information of the node, and COMB(.) integrating the previous representation the node and its neighboring information. Mathematically, the 1-th layer of an L-layer GNN model f updates the representation of node v; E V by
Let -V.E) denote a graph where == {U1, U2,
uy) uu))
h ( COMB (N((-1) AGG@((()) ((h)) (1-1) uy E (N:)))))
3

---

Published as a conference paper at ICLR 2025
P1
P1
Es1
F21
12
613
(a) L earning prompt vectors on nodes
(b) L Learning prompt vectors on edges
Learning prompt vectors node may uniformly pass them to its neighboring nodes while learning prompt vectors on edges can result customized prompt aggregation.
Figure 1:
where h!" RD: denotes the Di-dimensional representation of node v; at the I-th layer, and (uss) denotes the neighbors of node h (0) E IR D is initialized with node v feature Z;. The final node representation h:4)) after the I-thh layer of the GNN model can be subsequently used for various downstream tasks (e.g., node classification and graph classification) with trainable classifier g.
4 METHODOLOGY
In this section, we present proposed method EdgePrompt and its variant EdgePrompt++ Figure illustrates the difference between node prompt-based methods (Sun et al. 2023: Fang et al. 2023) and our edge prompt-basee based method. We first formulate the research problem studied in this paper. Then we introduce our design on edge prompts in EdgePrompt and EdgePrompt+ in detail. Fur- thermore, provide comprehensive analyses to demonstrate that our method has the capability of benefiting trained GNN models for node classification tasks. At last, we extend our method to graph classification the downstream task.
4.1 PROBLEM SETTING
This study focuses the standard problem of graph prompt tuning following previous studies (Fang et al. 2023) Sun et al. 2023). We consider GNN model pre-trained by pre-training task. We aim adapt the pre-trained GNN model to downstream task graph dataset through graph prompt tuning while keeping its parameters frozen. Specifically, given pre-trained GNN model f. the goal is to transform the input graph
prompted graph G' = T()) with learnable prompts and obtain expressive node representations on G' by f for a specific downstream task. Here, T is graph transformation obtain G' by adding prompts to G. The key problem in graph prompt tuning is to design and learn suitable graph prompts to benefit downstream tasks.
4.2 EDGE PROMPT DESIGN
Inspired by pixel-level visual prompts [Bahng et al. 2022 Wu et al. 2022) in Computer Vision, the existing studies (Sun al. 2023 Fang al. 2023) design graph prompts the data level by adding extra Icarnable prompt vectors to node features. Nevertheless, this strategy does not take account of the dependencies between nodes in graph data, which can significantly impact the final node representations via the message- message-passing passing mechanism in GNN models (Fatemi et al., 2021 Sun al. 2022b Liu al., 2022a Motivated by this, we propose design our graph prompt tuning method from the perspective of edges in this study.
EdgePrompt. Considering the dependencies between nodes graph data, we design learnable prompt vectors on edges and manipulate the input graph to prompted one with the edge prompts; therefore, the pre-trained GNN model can generate expressive node representations on the prompted graph for the downstream task. More concretely, for each edge (v,, E E, we aim to learn
prompt vector e E RP on it at the I-th layer of the pre-trained GNN model. Typically, this prompt vector can be regarded as the learnable properties of edges. As discussed previously, one critical challenge arises here: many popular GNN models, such as GCN (Kipf & Welling (017). do not accommodate edge attributes during the message-passing mechanism. Therefore. they are unable to absorb e into node representations. To overcome this issue, we propose to aggregate the prompt vector along with node representations through the message-passing mechanism during
4

---

Published as a conference paper at ICLR 2025
the forward pass at each layer of the pre-trained GNN model. Specifically, compute h of each node vy at the (-th layer, the GNN model will aggregate not only h 1 from its neighboring node ei E (ex) but also eij associated with edge (vi, Uj). Mathematically, we can reformulate Equation ① with the edge prompt vector at the I-th layer of the pre-trained GNN model by
(1) COMB ( (h. (""-"".AGG"" (GG (1")) : vy E N((i)) fel] vy ((e))))).
h
To obtain the prompt vector, simple yet effective way is to learn global prompt vector shared by all the edges. Let p E Dt-1 denote the global prompt vector at the I-th layer of the pre-trained GNN model. The prompt vector for each edge (2t, U2y at the (-th layer can be written as
v(u;; Bj) E. = p
The above design with global prompt vectors on edges is termed EdgePrompt in our method.
EdgePrompt+. Although EdgePrompt designs graph prompts from the perspective of edges, a single shared prompt vector for all the edges is insufficient to model different complex dependencies between nodes. Motivated by this, we conceive an advanced version of the above EdgePrompt, called EdgePrompt+, to learn customized prompt vectors on edges. Specifically, instead of using a shared prompt vector (h)) for all the edges the I-th layer, each edge (U,, U2) & will learn its own
customized prompt vector eij- Nevertheless, learning |3| independent prompt vectors is infeasible in practice. When we optimize prompt vectors for downstream tasks (e.g.. node classification), we may have only limited number of labeled nodes. Therefore, most edges cannot receive supervision information (Fatemi et al. 202T) for optimizing their prompt vectors, especially in a few-shot setting.
(1)
(1) In this case, it will be hard to directly learn e; for edge (u;, v2) E & it is not involved in computing the representations of any labeled nodes. To overcome this issue, we propose learn the prompt vectors as a weighted average of multiple anchor prompts. To achieve this, we first construct a set of Mt anchor prompts p()) {pp() P2 _ .PM at the 1-th layer of the pre-trained GNN model, where each vector pin DP learnable anchor prompt. For each edge (U,, 22) E, its customized prompt vector e/ at the I-th layer is computed as the weighted average of the anchor
prompts in pp() with the score vector b/ E RRM, Mathematically, we can obtain e at the 1-th layer by
M.
( :j
10 ijm
Pm
(4)
m=11
where (2) igm denotes the m-th entry in b.. Since all the edges share the same anchor prompts p(!) at the (-th layer, the score vector b directly determines how e!) differs from those of the other edges. Therefore, our next goal to conceive an effective strategy to obtain the desired (()) According to Equation - ei of edge (Ua, affects message passing between nodes v and Uy so we may
naturally consider to depend on both nodes e; and Bj. Motivated by this, we propose to compute b1)) at the 1-th layer using a score function (1)) followed by the softmax operation. Formally, we compute b by
ij
(5) where Softmax(.) represents the softmax operation. Here, (f()) takes each pair of nodes e; and uy as the input and generates the score vector. Basically, it describes the relationship of two nodes at the 1-th layer and embeds them into a single vector. Many typical formulations (Veličković et al. [0118] Brody al. 2022 Yang al. (2021) can be used achieve this goal. In this study, we adopt the classic attention mechanism (Veličković et al. 2018) ((() by
D (oft vy)),
(v., = LeaLy (t- (--) (--")) w()).
(6)
where (') E 2 x M; is the weight matrix of (0)) at the 1-th layer, and [-I1-] denotes the vector concatenation. In-depth investigations into different variants of the score function & will be reserved
5

---

Published as a conference paper at ICLR 2025
for our future work. is worth noting that GPF (Fang et al. (023) can be regarded as a special case of EdgePromptt with the score function as a linear mapping of x,.
With the learnable edge prompts, we can obtain more suitable node representations h2 (L) for node vs E V by the pre-trained GNN model for node classification. Given the labeled node set Vr. E V. we optimize our edge prompts and classifier g by
1
Σ (o(o(f(())))))
min
9. (PG)
W(y [Vl
1,EV2
where yi is the ground truth label of node o, E V2, and lD is the downstream task loss, i.c. the cross-entropy loss for classification tasks.
4.3 ANALYSIS OF EDGE PROMPT TUNING FOR NODE CLASSIFICATION
In this subsection, we provide comprehensive analysis to investigate why our proposed Edge- Prompt+ is more effective for node classification than existing approaches, particularly those that focus on learning additional prompt vectors on node features.
We first provide our insights regarding the issue of uniform message passing on prompt vectors. As introduced previously, GPF-plus (Fang et al., 2023) and All-in-one (Sun et al. 2023) design learnable prompt vectors on the node level and manipulate the input graph by adding the prompt vectors to node features. For each node v,, its learned prompt vector p; completely depends on its node feature z; In many prevalent GNN models, such GCNs, the prompt vector will be uniformly aggregated by neighboring nodes through the message-passingg mechanism (Yang et al. 2021). Taking node v1 in Figure aa))
example, its two neighboring nodes 02 and vi will always receive the same prompt vector p1 from node 81 in pre-trained GCN models. Unfortunately, such propagation of prompt vectors may not benefit node classification. Instead, the prompt vector aggregated by a node can retain adverse information from different classes. In contrast, our proposed EdgePrompt+ designs prompt vectors on edges. Unlike one shared prompt vector of node for all its neighboring nodes, EdgePrompt+ enables these neighboring nodes to receive different learned prompt vectors (e.g., E21 and 631 in Figure [b)) from the node. In this way, the issue of uniform passing on prompt vectors can be mitigated.
Furthermore, we would like to provide theoretical analysis of how edge prompts in our proposed EdgePromptt can benefit node classification. Our analysis is based on random graphs generated by the contextual stochastic block model (CSBM) (Tsitsulin et al. 2022 Ma al. 2022). Specifically, we consider a random graph G generated by the CSBM consisting of two node classes and 62- For each node U; its node feature x; follows a Gaussian distribution 2; ~ N( I) if it is from class C1, otherwise x; ~ NNu,, I). Generally, we assume 141 * 142 In the graph G, edges are generated following an intra-class probability p and an inter-class probability q. More concretely, pair of nodes will be connected by an edge with probability if they are from the same class; otherwise, the probability is q. In this section, we use G ~ CSBM(H) M2. P: q) to denote a random graph generated by the CSBM.
Our analysis aims to investigate the improvement of linear separability under pre-trained GCN mod- els caused by edge prompts in EdgePrompt+. Specifically, we focus the linear classifiers with the largest margin based on node representations after GCN operations with and without edge prompts. Typically, if the expected distance between the two class centroids is larger, the node representations will have higher linear separability by the linear classifier.
Theorem 1. Given random graph G ~ CSBM USBM(u,, M4, 9) and a pre-trained GCN model f. there always exist set of M 2 2 anchor prompts P = [P1.P2." .Pa} and the score vectors big for each edge (Vi, 0z) that improve the expected distance after GCN operation between clusses
en and 02 to T times without using edge prompts, where T E 1 1
A detailed proof can be found in Appendix A According to Theorem I we will have a larger expected distance between the two class centroids after GCN operation with edge prompts in Edge- Prompt+. In this case, the node representations from the two classes will have lower probability of being misclassified. Therefore, we can conclude that our proposed EdgePrompt+ benefits pre- trained GNN models for node classification.

---

Published as a conference paper at ICLR 2025
4.4 EXTENSION TO GRAPH CLASSIFICATION
In the last subsection, we present our edge prompt design in EdgePrompt and EdgePrompt+ for node classification. As discussed previously, edge prompts should be capable of handling various downstream tasks, including graph classification. In this subsection, we would like to introduce how EdgePrompt and EdgePrompt+ tackle graph classification.
In graph classification, we have a set of labeled graphs 9g,, ... [ with their label set {y1 92. 3K}. To obtain the representation of the entire graph g. we typically integrate the final representations of all nodes in G via 2019), such as sum and mean, as the entire graph's representation hg (EADOUT((h:))) VJ). Therefore, we can optimize our edge prompts and a classifier g by
cormmutaiionnnvaatti
READOUT function (Xu et al.
K Σ l (o(o(f(())..)) v.).
min
(8)
9. (P)) PPL) W W K
Now we analyze the capability of EdgePrompt for graph classifications. The goal of our analysis is to investigate whether learning edge prompts in EdgePrompt can result in consistent graph repre- sentations with those using any prompt strategies. To this end, we propose the following theorem.
Theorem 2. Given an input graph
(X, A) and its transformation G' AX), by an arbitrary transformation function T, there exists a set of edge prompi vectors (p(") ,02) , p (1) in Edge comppt that can satisfy
f(X, A, 1 ,p = f(X',A'
(9)
for any pre-trained GNN model f.
The complete proof of Theorem 2 is provided in Appendix B According to Theorem we can conclude that our edge prompts have the capability to get graph G's representation which is equal to those of its variants by transformations with any prompt strategies. According to Theorem by Fang ct al. (2023), our EdgePrompt has comparable universal capability with GPF. Since EdgePrompt+ provides finer edge prompts than EdgePrompt, it will have stronger universality than EdgePrompt.
5
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Datasets and downstream tasks. We evaluate the effectiveness of our proposed method on node classification over five public graph datasets, including Cora (Yang al.) 2016), CiteSeer (Yang) et al. 2016), Pubmed (Yang et af.] 2016)., ogbn-arxiv (Hu et al. 2020a). and Flickr (Zeng et al,, 2020 In addition, we adopt five graph datasets from TUDataset (Morris al., 2020). including ENZYMES, DD, NCII, NCI109, and Mutagenicity, to conduct experiments for graph classification. Basic information and statistics about these datasets can be found in Appendix C.
Pre-training strategies. To evaluate the compatibility of our proposed method with various pre- training strategies, we consider four pre-training strategies in our experiments. For contrastive methods, we use GraphCL (You et al. 2020) and SimGRACE (Xia et al.) 2022). For generative methods, we use two edge prediction-base methods, i.e., EP-GPPT and GrarrrrppPropp,, adopted by GPPT (Sun et al.] 2022a) and GraphPrompt (Liu et al. 2023). respectively. We provide detailed descriptions of these pre-training strategies in Appendix C.2
We evaluate our proposed method against five state-of-the-ard graph prompt tuning meth- ods in our experiments, including GPPT (Sun et al., 2022a), GraphPrompt (Liu al. 2023) All-in- one (Sun et al.) 2023), GPF (Fang et al.) 2023), and GPF-plus (Fang et al.) (023). Since GPPT is specifically designed for node classification, we only report its performance for node classification tasks. In addition, also report the performance of solely training classifiers without any prompts (named as Classifier Only) in our experiments.
Baselines.
Implementation details. In experiments, We use 2-layer GCN (Kipf & Welling 2017) the backbone for node classification tasks and a 5-layer GIN (Xu et al. 2019) as the backbone for graph classification tasks. The size of hidden layers is set 128. The classifier adopted for downstream

---

Published as a conference paper at ICLR 2025
Table 2: Accuracy 5-shot node classification tasks five datasets. method is bolded and the runner-up is underlined.
The best-performing
Pre-training Tuning Methods
Cora
CiteSeer
Pubmed
ogbn-arxiv
Flickr
Strategies
Classifier Only
53.0544.76
38.62+3.48
64.28
21.15+1.64
21.32_2.0x
GPPT
50.9646.67
39.50+1.07
60.47 +4.75
17.99
24.35=1.84
GraphPrompt
55. 4..612
40.81
63.47
21.03+1.92
26.08_3.44
ALL-in-one
38.00
40.27
58,61
16,42::2.98
25.08
GraphCI.
GPF
58.52
43.55
67.67
21.73
23.98
GPF-plus
52.24414.59
38.47 1:327
64.30
21.03
25.32
EdgePrompt
58.60
43.31 1:323
67.76
21.90
24.83=2.78
EdgePrompt+
62.88
46.20
67.41
23.18
25.57
Classifier Only
52.27
40.45
56. 1:3.2
20. 20.75:12.22
25.53 3.98
GPPT
52.07
40.25+3.29
58.65
17.76+1.80
23.37 =4.66
GraphPrompt
51.4242.80
41.7442222
55.98 $2.94
20.48
25.88
ALL-in-one
34.6444..06
38.95
54.18
16.72
27. 27.68
SimGRACE
GPF
58.2314.19
44.87
61.55 13.79
21.86=2.91
26.51_4.66
GPF-plus
32.27743.34
41.02
56.95 4:3.86
21.44
28.35 55.50
EdgePrommp
58.37 44..51
43.94
61.10
21.85
30. 30.12 12.12
EdgePrompt+
62.40
46.62
64.91
22.74
28.50 28.50=4.08
Classifier Only
28.6544.82
26.77 #:203
40. 40.14
11.57 #1.91
28.39 14.39-7444
GPPT
41.28
35.32 +1.27
53.41 4:3.99
13.73
29.83
GraphPrompt
31.6543.33
26.98
44.18
11.31 1:.89
26.02
ALL-in-one
31.5742.16
28.87
46.0224.28
15.9440.75
31.89_1.14
EP-GPPT
GPF
37.5643.81
29.7441.78
48.8647.32
16.95
29. 29.68_6..73
GPF-plus
28.8743.18
26.65
40.32
11.78+1.55
29. 29.41_6.78
EdgePrompt
37.2644.55
29.83+1.01
47.20+7.06
17.22++1.91
31.17
EdgePrompt+
56.41+3.66 6..2
43.49 #2.62
61.51
17.78
32.70
Classifier Only
59.0045.74
44.5424.44
22.0925.70
11.28
27. 27.83_4.77
GPPT
54.2947.9u
45.81 43.54
66.5644.00
25.34+1.85
28. =3.68
GraphPrompt
60.22 60.22+4.04
47.07 43.00
73.13
32.40
28. 28.10=3.22
EP-GraphPrompt
ALL-in-one
42.55+2..9
44.364+2.02
67. 67.66 +6.38
15.22++3.04
31.79
GPF
62.6216.40
49.0244.53
73.6226.42
31.88
28.98_5.30
GPF-plus
58.2345.68
44.6044.47
72.15
31.58
28.96_4.68
EdgePrompt
62. 62.7446.77
48. 48.6924.38
73.60
32.67 =
29. 29.81 _3.50
EdgePromptt+
64.47+7.04
49.71+2.25
73.72+5.10
31.41
32.09 9..09944.99
tasks is linear probes for all the methods. We Adam optimizer (Kingma & Ba 2015) with
learning rates 0.001 for all the methods. The batch size is set as 32. The number epochs is set to 200 for graph prompt tuning. The default number of anchor prompts each GNN layer is 10 for node classification tasks and for graph classification tasks. We use the 5-shot setting for node classification tasks and the 50-shot setting for graph classification tasks. We conduct experiments five times with different random seeds and report the average results in our experiments.
5.2
MAIN RESULTS
We first compare the overall performance of our proposed methods and other baselines. Table reports the results of our method and six baselines on 5-shot node classification tasks over five datasets under four pre-training strategies. According to the table, we observe that our method consistently achieve the best most competitive performance among graph prompt tuning methods across different pre-training strategies. Generally, EdgePrompt+ has better performance than EdgePrompt, which is consistent with our analyses in Section 4.3 and validates the necessity of our design on customized edge prompts.
In addition, conduct experiments 50-shot graph classification tasks over five datasets under four pre-training strategies and report the results in Table 3 According to the table, we observe that EdgePrompt+ always get the best place runner-up for every experimental setting, espe-
8

---

Published as a conference paper at ICLR 2025
Table 3: Accuracy 50-shot graph classification tasks five datasets. The best-performing method is bolded and the runner-up underlined.
Pre-training Tuning Methods
ENZYMES
DD
NCT1
NCI109
Mutagenicity
Strategies
Classifier Only
30.50
62.89
62.49
61.68
66.62+1.87
GraphPrompt
27.83=1.61
64.33
63.19
08.18
60.66
ALL-in-one
20.92=0.65
66.5441.82
57.52+2.61
62.7440.78
63, 63.4342..5
GPF
30.08
64.54122.2
62.6611.83
62.29
62.2911..0
66.5411.85
GraphCI.
GPF-plus 31.00 1.50
67.26
64.56
62.84
66.82
EdgePrompt
29.50
64.1612.13
63.05
62.59 1..99
66.8711.88
EdgePrompt+
34.00
67.98
66.3012.54
66.5210.91
67.47
Classifier Only
27.07
61.77
61 61.27
62.12
67.36
GraphPrompt
26.87 1.447
62.58
62.45
62.41 1.069
68.03
ALL-in-onee
25.73
65. 15.16
58.52+1.50
62.01 +0.66
64.43+ 1.00
GPF
28.53
65. 65.6440.70
61.45+3.13
61.90 1.26
67. 07.19
SimGRACE
GPF-plus
27.33=2.01
67.20+1.56
61.61
62.8440.22
67. (T.6990..6
EdgePrompt
29.33
63.97 1:214
62.02
62.02 1.05
67. 67.55
EdgePrompt+
1:2.53 32.67
67.72
67 (0.
66.53 1.330
68.31 1.36
Classifier Only
29.08
62.12
56.8544.35
62.27
66,30 1.78
GraphPrompt
26,67 1.66
61.61+1.91
58.77+0.97
62. 62.1644..5
66.37
ALL-in-onc
24.92
63.61
59.1442.12
59.70+1.33 1.37
64.86
GPF 28.33
63.48 +2200
48.14
62.52+1..3
66.10+0.99
EP-GPPT
GPF-plus 29.25
66.92 66.92+2.34
62.98
64.1341.42
67.57+1..4
EdgePrompt
28.33
64.03
59.85
62.98+1.44
66.36.1
EdgePrompt+
32.75
66.1641.60
63.5812.00
65.15
68.35= 1.57
Classifier Only
31.33
62.58
62.09
60.19
65. 63.13:0.88
GraphPrompt
30.20 1.93
64.72+1.08
62.57711.45
62.32
65.85
ALL-in-one
29.07 1.16
65.60+2.38
58.6712.42
57.69+1.08
641. 64166440.76
GPF 30.93
66.2141.66
61.80
62.27
65.61
EP-GrapPPrompt
GPF-plus
30.67 43.00
67.50+2.45
62.59
61.98
65.51+1.10
EdgePrompt
30.80 #2.09
65.87
61.75+22.9
62.33
65.77
EdgePrompt+
33. 33.27 42.71
67.47
65.0641.84
64.6441.67
66.42 1.91
cially over ENZYMES, NCII, and NCI109. In addition, observe that GPF and EdgePrompt have relatively small performance gaps (always < 1.8%) in the table (we also observe this in node clas- sification tasks). As indicated in Theorem [ our proposed EdgePrompt has comparable universal capability with GPF to achieve graph representations equivalent to any graph transformation. These observations effectively support our theoretical claim in this study.
5.3 CONVERGENCE ANALYSIS
In this subsection, would like to investigate the convergence speeds of our method compared with basclines. Figure? 2 illustrates the accuracy curves of our method and the basclines under two pre-training strategies. According to Figure can observe that EdgePrompt+ can generally converge faster than other methods.
5.4 INFLUENCE OF PROMPT NUMBERS
We conduct experiments to investigate the impact of different numbers anchor prompts on model utility. Figure 3 and Figure / illustrate the performance of EdgePrompt+ with 1, 5, 10, 20, and 50 anchor prompts at each layer for node classification and graph classification tasks, respectively. Note that EdgePrompt+ will be degraded to EdgePrompt when we have only one anchor prompt at each GNN layer. From the two figures, we can conclude only one anchor prompt vector (i.e., EdgePrompt) is insufficient in most cases where each edge will leam global prompt vector. In the meantime, EdgePrompt+ with too many anchor prompts (e.g., 50) may not further improve the performance. We recommend 10 the initial number of anchor prompts.

---

Published as a conference paper at ICLR 2025
Only
GPPT
Graphirompt
Allib-one
dPrplus
EdguPrompt
EdgePt
35
55
20
40
10
25
50
100122110015
10
100125 130
30
90 75 1000251501752000
0
25 5e
75 10110225110172200
Spochs
Epochs
Epechs
Epechs
(a) Cora with SimGRACE
CiteSeer with SimGRACE
(c) PubMed with SimGRACE
Flickr with SiMGRACE
70
50
70
20.0
15.0
10.0
7.5
15
36
2.5
1001
10
100
30
0.0
160125150
(e) Cora with EP-GPPT
(f) CiteSear with EP-GPPT
(g) PubMad with EP-GPPT
(h) og agbnnarddv with EP-GPPT
Figure 2: Convergence speeds of different methods.
Prampts)=1
Prompts)o5
Prompts):10
Prompts]=20
Prompts)
Accuracy
70
50
Recurre
80
Asccung
30
26
50
El-GraphPrompt
30
Ep-GraphPrompt (b) CiteSeer
Pr-GraphPrompt PubMad
10
(a)
(d) ogbn-arxiv
Figure 3: Results of EdgePrompt+ with varying numbers of anchor prompts on node classification.
(Anchor Promptsi-1
#lAnchor Promptx)-5
#AAnchor Prompts)-10
#AAnchor 2romptai-20
#'Anchor Prompta)-50
40
75
75
75
Accuracy
35
70
70
70
GraphCL
EP-GPPT
GraphEL
GraphCL
GraphCL
EP-GPPT
(a) ENZYMES
(b) DD
(c) NCI1
(d) Mutagenicity
Figure 4: Results of EdgePrompt+ with varying numbers of anchor prompts graph classification.
CONCLUSION
Graph prompt tuning is an emerging technique to bridge the objective gap between pre-training and downstream tasks. Unlike previous studies focusing on designing prompts on nodes, we propose a simple yet effective method, EdgePrompt and its variant EdgePrompt+, that manipulates the input graph by adding extra learnable prompt vectors to edges and thereby obtaining a prompted graph suitable for downstream tasks. We provide comprehensive theoretical analyses of our method garding its capability of handling node classification and graph classification. We conduct extensive experiments over ten graph datasets under four pre-training strategies. Experiment results demon- strate the superiority of our method compared with six baselines.
ACKNOWLEDGMENTS
This work supported in part by the National Science Foundation under grants IIS-2006844, IIS-2144209, IIS-2223769, IIS 1IS-2331315, CNS-2154962, BCS-2228534, and CMMI-241124 0122412248. 1248, the Commonwealth Cyber Initiative Awards under grants VV-1Q24-011, VV-1Q25-004, and the iPRIME Fellowship Awards.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Exploring visual prompts for adapting large-scale models. arXiv preprint arXiv:2203.17274, 2022.
Shaked Brody, Uni Alon, and Eran Yahav. How attentive are graph attention networks? In Interna- tional Conference on Learning Representations, 2022.
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In International conference on machine learning, 2020.
Taoran Fang, Yunchao Zhang, Yang Yang. Chunping Wang. and Lei Chen. Universal prompt tuning for graph neural networks. Advances in Neural Information Processing Systems, 2023.
Bahare Fatemi, Layla El Asri, and Seyed Mehran Kazemi, Slaps: Self-supervision improves struc- ture learning for graph neural networks. Advances in Neural Information Processing Systems, 2021.
Xingbo Fu, Chen Chen, Yushun Dong, Anil Vullikanti, Eili Klein, Gregory Madden, and Jundong Li. Spatial-tempon networks for antibiogram pattern prediction. In 2023 IEEE 11th International Conference Healthcare Informatics (ICHI), 2023.
Johannes Gasteiger, Stefan Weißennerger, and Stephan Günnemann. Diffusion improves graph learning. Advances in neural information processing systems, 2019.
Liyu Gong and Qiang Cheng. Exploiting edge features for graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019.
Will Hamilton, Zhitao Ying. and Jure Leskovec. Inductive representation learning on large graphs. In Advances in neural information processing systems, 2017.
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong. Hongxia Yang, Chunjie Wang, and Jie Tang. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022.
Graphmae: Self-supervised masked graph autoencoders.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 2020a.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In International Conference on Learning Representations, 2020b.
Ziniu Hu, Changjun Fan, Ting Chen, Kai-Wei Chang, and Yizhou Sun. Pre-training graph neural networks for generic structural feature extraction. arXiv preprint arXiv:1905.1372 13728, 2019.
Renhong Huang, Jiarong Xu, Xin Jiang, Chenglu Pan, Zhiming Yang, Chunping Wang, and Yang Yang. Mcasuring task similarity and its implication in fine-tuning graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, 2024.
Wei Jin, Tyler DeTr, Haochen Liu, Yigi Wang, Suhang Wang. Zitao Liu, and Jiliang Tang. Self-supervised learning on graphs: Deep insights and new direction. arXiv preprint arXiv:2006. 10141, 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representutions, 2015.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In International Conference on Learning Representations, 2017.
Jiazheng Li, Jundong Li, and Chuxu Zhang. Instance-aware graph prompt learning. Transactions
on Machine Learning Research, 2025.
Xin Liu, Jiayang Cheng, Yangqiu Song, and Xin Jiang. Boosting graph structure learning with dummy nodes. In International Conference Machine Learning, 2022a.

---

Published as a conference paper at ICLR 2025
Yixin Liu, Yu Zheng, Daokun Zhang, Hongxu Chen, Hao Peng. and Shirui Pan. Towards unsuper- vised deep graph structure learning. In Proceedings of the ACM Web Conference 2022, 2022b.
Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. In Proceedings of the ACM Web Conference 2023,
2023.
Zewen Liu, Guancheng Wan, B Aditya Prakash, Max SY Lau, and Wei Jin. A review of graph neural networks in epidemic modeling. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6577-6587, 2024.
Yao Ma, Xiaorui Liu, Neil Shah, and Jiliang Tang. Is homophily necessity for graph neural networks? International Conference on Learning Representations, 2022.
Yihong Ma, Ning Yan, Jiayu Li, Masood Mortazavi, and Nitesh V Chawla. Hetgpt: Hamesssing the power of prompt tuning in pre-trained heterogeneous graph neural networks. In Proceedings of the ACM on Web Conference 2024, 2024.
Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663, 2020.
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning dynamic graphs. arXiv preprint arXiv:2006. 10637, 2020.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In 15th International Conference on Extended Semantic Web Conference, 2018.
Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi- supervised graph-level representation learning via mutual information maximization. In Interna- tional Conference on Learning Representations, 2020.
Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang, and Xin Wang. Gppt: Graph pre-training and prompt tuning to generalize graph neural networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022a.
Qingyun Sun, Jianxin Li, Hao Peng, Jia Wu, Xingcheng Fu, Cheng Ji, and S Yu Philip. Graph structure learning with variational information bottleneck. In Proceedings of the AAAI Conference on Artificial Intelligence, 2022b.
Xiangguo Sun, Hong Cheng, Jia Li. Bo Liu, and Jihong Guan. All in one: Multi-task prompting for graph neural networks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023.
Yifei Sun, Qi Zhu, Yang Yang, Chunping Wang, Tianyu Fan, Jiajun Zhu, and Lei Chen. Fine- tuning graph neural networks by preserving graph generative patterns. In Proceedings of the AAAI Conference on Artificial Intelligence, 2024.
Zhen Tan, Ruocheng Guo, Kaize Ding, and Huan Liu. Virtual node tuning for few-shot node clas- sification. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2023.
Anton Tsitsulin, Benedek Rozemberezki, John Palowitch, and Bryan Perozzi. Synthetic graph gen- eration to benchmark graph learning. arXiv preprint arXiv:2204.01376, 2022.
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. Composition- multi- relational graph convolutional networks. In International Conference on Learning Representa- tions, 2020.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
12

---

Published as a conference paper at ICLR 2025
Petar Veličkovié, William Fedus, William L Hamilton, Pietro Liô, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In International Conference on Learning Representations, 2019.
Guancheng Wan, Zewen Liu, Max SY Lau, B Aditya Prakash, and Wei Jin. Epidemiology-awee aware neural ode with continuous disease transmission graph. arXiv preprint XXi::24100000499 2024a.
Guancheng Wan, Yijun Tian, Wenke Huang, Nitesh V Chawla, and Mang Ye. S3gel: Spectral, swift, spatial graph contrastive learning. In Forty-first International Conference on Machine Learning, 2024b.
Yue Wang, Yonghin Sun, Ziwei Liu, Sanjay E. Sarma, Michael Bronstein, and Justin M Solomon. Dynamic graph enn for learning point clouds. ACM Transactions on Graphics (tog), 2019.
Zehong Wang. Qi Li. Donghua Yu, Xiaolong Han, Xiao-Zhi Gao, and Shigen Shen. Heterogeneous graph contrastive multi-view learning. In Proceedings of the 2023 SIAM International Conference on Data Mining (SDM), pp- 136-144. STAM, 2023a.
Zehong Wang, Donghua Yu, Qi Li, Shigen Shen, and Shuang Yao. Sr-hgn: Semantic-and relation- aware heterogeneous graph neural network. Expert Systems with Applications, 224: 19992, 2023b.
Zehong Wang, Donghua Yu, Shigen Shen, Shichao Zhang, Huawen Liu, Shuang Yao, and Maozu Guo. Select your own counterparts: Self-supervised graph contrastive learning with positive sampling. IEEE Trunsactions on Neural Networks and Learning Systems, 2024a.
Zehong Wang. Zheyuan Zhang, Nitesh Chawla, Chuxu Zhang, and Yanfang Ye. Gft: Graph foundation model with transferable tree vocabulary. Advances in neural information processing systems, 2024b.
Xuemei Wei, Yezheng Liu, Jianshan Sun, Yuanchun Jiang, Qifeng Tang, and Kun Yuan. Dual subgraph-based graph neural network for friendship prediction in location-based social networks. ACM Transactions on Knowledge Discovery from Data, 2023.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simpli- fying graph convolutional networks. In International conference on machine learning, 2019.
Junyang Wu, Xianhang Li, Chen Wei, Huiyu Wang, Alan Yuille, Yuyin Zhou, and Cihang Xic. Unleashing the power of visual prompting at the pixel level. arXiv preprint arXiv:2212.1055: 2022.
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learn- ing. Chemical science, 2018.
Jun Xia, Lirong Wu, Jintao Chen, Bozhen Hu, and Stan Z Li. Simgrace: A simple framework for graph contrastive learning without data augmentation. In Proceedings of the ACM Web Confer- ence 2022, 2022.
Keyulu Xu, Weihua Hu, Jure Leskovee, and Stefanie Jegelka. How powerful graph neural networks? In International Conference on Learning Representations, 2019.
Liang Yang, Mengzhe Li, Liyang Liu, Chuan Wang, Xiaochun Cao, Yuanfang Guo, et al. Diverse message passing for attribute with heterophily. Advances in Neural Information Processing Sys- tems, 2021.
Yulei Yang and Dongsheng Li. Nenn: Incorporate node and edge features in graph neural networks. In Asian conference on machine learning, 2020.
Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In International conference on machine learning, 2016.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in neural information processing systems, 2020.
13

---

Published as a conference paper at ICLR 2025
Xingtong Yu, Yuan Fang, Zemin Liu, and Xinming Zhang. Hgprompt: Bridging homogeneous and heterogeneous graphs for few-shot prompt learning. In Proceedings of the AAAI Conference on Artificial Intelligence, 2024a.
Xingtong Yu, Zhenghao Liu, Yuan Fang, Zemin Liu, Sihong Chen, and Xinming Zhang. General- ized graph prompt: Toward a unification of pre-training and downstream tasks on graphs. IEEE Transactions Knowledge and Data Engineering, 2024b.
Xingtong Yu, Jie Zhang, Yuan Fang, and Renhe Jiang. Non-homophilic graph pre-training and prompt learning. arXiv preprint arXiv:2408. 12594, 2024c.
Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Multigprompt for multi-task pre- training and prompting on graphs. In Proceedings of the ACM on Web Conference 2024, 2024d.
Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, and Viktor Prasanna. Graph- saint: Graph sampling based inductive learning method. In International Conference on Learning Representations, 2020.
WANG Zhili, DI Shimin, CHEN Lei, and ZHOU Xiaofang. Search fine-tune pre-trained graph neural networks for graph-level tasks. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), 2024.
Haoran Zhou, Yidan Feng, Mingsheng Fang, Minggiang Wei, Jing Qin, and Tong Lu. Adaptive graph convolution for point cloud analysis. In Proceedings of the IEEE/CVF international con- ference on computer vision, 2021.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for Lasounlanguuae models. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, 2022.
Zhilun Zhou, Yu Liu, Jingtao Ding, Depeng. Jin, and Yong Li. Hierarchical knowledge graph learning enabled socioeconomic indicator prediction in location- based social network. In Proceedings of the ACM Web Conference, 2023.
A PROOF OF THEOREM
Theorem 1. Given random graph G CSBM P. 9) and a pre-trained GCN model J. there always exist set of M 2 anchor prompts P = PPi.P2.-". Pas) and the score vectors b.. for each edge (v,, U3) that improve the expected distance after GCN operation between classes on and 02 T times without using edge prompts, where T (1,1+ 1
Proof. For each node v in graph 9, we can approximately regard that the labels of its neighboring
nodes independently sampled from neighborhood distribution De, - 9_ if node u; is
p+g.
class e₁ or D.2- = ' p if node v; is class 02 (Ma et al. 2022). When we do not consider edge prompts, the expected feature obtained from the GCN operation will be
E/hi]
141 +
q
P22
(10)
P+
p q
for nodes class ea and
E[hg]
9
14
P
122
(11)
p+g
p p+4
for nodes in class 22 Here, we ignore the linear transformation in the GCN operation since can be absorbed by the linear classifier. To evaluate the linear separability of linear classifiers, calculate the expected distance d between the two classes e₁ and C2 by
p q
d =
|h
E h
=
||
12
(12)
p
14

---

Published as a conference paper at ICLR 2025
When we consider edge prompts in EdgePromptt, need involve them into the aggregation in the GCN operation. Without loss of generality, we can fix bijm = 0 for m E [3, M.. Therefore, for each edge (u,, uy), its prompt vector will be
Mr
Cij-
= Σ bijm m111
Pm = big1 + biz2 P2-
(13)
Obviously, bij2
1 - big1. 1-- In addition,
set the prompt vectors ba and μ12, i.e.,
eij = biji M1 + biy2 μ42
(14)
Then the new expected feature obtained from the GCN operation with edge prompts will be Blhi] p (1l + bn Ai + (1 - b1 p2)) + q. (n2 b12 pi (1 b12 (22))
(15)
p+
nodes class e₁ and
E[hy] -- (u1 + b21 1e + (1 bu 122) p- (W2 byg p41 (1 b22 p22)
(16)
p+
for nodes in class c2. Here, bn E [0, 1] represents the expected score between nodes from class 1, b22 [0, 1] represents the expected score between nodes from class 2, b12 [0, 1] and b21 [0, 1] represents the expected score between nodes across classes. Different from the original design in our method, we can و12 b21 for simplicity. Therefore, the new expected distance with edge prompts will be
d'
[ - E[h2ll
(p + bn1 ba2 p)l1 - (- bn) p- - (1 b22) p1j2
p+q
(17)
(p + (b11 - b22) p)l |1 M2ll
p +
To improve the linear separability of the two classes, hope get d' d. In this case, may assume
d' d'=--d= T (p 4)1
|| - Hell
(18)
p+g
with 7' > 1. Therefore, we need
(T 1) (p 9)
b b22
(19)
P
Since bi [0, 1] and b22 E [0, 1J. we need
TT 1) (p 9)
-1
(20)
P
Then have
P
T T51+ p-9 q
(21)
Therefore, We can conclude that we can always find a set M 2 2 anchor prompts (M1, M2, Ps,... PPs) and the above score values for each edge (v,, 02) that improve the expected distance after GCN operation between classes Cj and 02 to times without using edge prompts, where T (1, p
B PROOF THEOREM 2
Before we prove Theorem
2 we would like to prove the following lemma.
Lemma 1. Given an input graph G (X, A) and an extra feature prompt P in GPF, there exists set of edge prompt vectors (,(), (2), ... (T)) in EdgePrompt that can satisfy
f(x, A, (1) ()... ......p.. (1))) (1)))= f(X A.
(22)
for any pre-trained GNN model f.
15

---

Published as a conference paper at ICLR 2025
Proof. Following GPF (Fang et al. 2023), first consider single-layer GIN (Xu al. 2019)
with a linear transformation. Mathematically, we can compute the node representation matrix in GIN layer by
= (A (1+6) x W = + (1+e)) W.
(23)
In GPF, the feature prompt P is added to the feature vector for each node. Then the new node representation matrix with p can be written as
Нр = (A + (1+ I) (X 11 P) W
= (A + (1++) X- W + (A+ (1 (1+c) I) [1] w
(24)
HH + (A+ (1+6) [1]x p. W
+ [Dg, +1+ - W
where [1] E Nx1 represents an N-dimensional column vector with values of 1, (Deg: + +c] E Nx represents an N-dimensional column vector with the value of i-th row is Deg, + 1 and Deg; represents the degree of node V.
In EdgePrompt, the prompt vector will be associated with each edge. Therefore, can write the node representation matrix with edge prompt by
Hp = A (X + [1] p) W (1+ X W
A W + A [1]
p W (1 e) X W
(25)
=H -H+A A []] p W
= H. + [Degil N p W
To obtain the same graph representation, we have
Sum Hp) = Sum Hp))
(26)
where Sum H) computes the sum vector for each row vector matrix. We can simplify the above equation by
Sum[ Hp) = Sum Hp))
Sum H - [Deg + 1 + <) D W) = Sum H + [Degi] W)
(27)
Sum ((eg:
c P W) Sum((Deg))*
W)
= (Deg + N + ce) P W = Deg p W
where Deg = Evegy Deg, To obtain the above equation, we only need
p Deg + NN .
P.
(28)
Deg
Therefore, for any feature prompt p. can always find edge prompt p in Equation 28 that
satisfies Lemma
Extension to other GNN backbones. Various GNN backbones can be expressed as H S WWW where is the diffusion matrix (Gasteiger et al. 2019). Different S only impact the coefficient before in Equation 28
Extension to multi-layer GNN models. For multi-layer linear GNN models, the diffusion matrix (()) at each layer can integrated as one overall
Theorem Given an input graph G (X, A) and transformation G' (X', by an
arbitrary transformation function T. there exists a set of edge prompt vectors (p(1) (()) ...).... p 11) in EdgePromes that can satisfy
f(x, (p,
1(X',
(29)
for any pre-trained GNN model f.
16

---

Published as a conference paper at ICLR 2025
Table 4: Basic information and statistics of graph datasets adopted in experiments.
Dataset
#(Graphs)
#(Nodes)
#(Edges)
#(Features)
# (Classes)
Task Level
Cora
2,708
10,556
1,433
Node
CiteSeer
3,327
9,104
3,703
Node
Pubmed
19,717
88,648
500
3
Node
Flickr
1
89,250
899,756
500
Node
ogbn-arxiv
169,343
1,166,243
Node
Dataset
#(Graphs)
# #(Avg. Nodes)
#(Avg. Edges)
# #(Features)
# (Classes)
Task Level
ENZYMES
600
32.63
124.27
3
Graph
DD
1,178
284.32
1,431.32
89
2
Graph
NCII
4,110
29.87
64.60
37
2
Graph
NC1109
4,127
29.68
64.26
Graph
Mutagenicity
4,337
30.32
61.54
2
Graph
Proof. Given any feature prompts, Lemma indicates that we can always find edge prompts that lead to the same representation of graph for any pre-trained GNN models. Given Theorem by (Fang et al. 2023), the input graph with learnable feature prompt can always obtain the same representation as those of any transformed graphs. Therefore, we can conclude that our edge prompts in EdgePrommt have the capacity to obtain the representation equal to those of any transformed graphs for any pre-trained GNN models.
C MORE DETAILS ABOUT EXPERIMENTAL SETUP
C.1 DATASETS
Table 4 shows the basic information and statistics of graph datasets adopted in our experiments.
C.2 PRE-TRAINING STRATEGIES
We provide more details about the four pre-training strategies adopted in our experiments.
GraphCL You et al., 2020) is a contrastive method for pre-training GNN models. The intuition of GraphCL is to maximize the agreement between two views of a graph perturbed by different data augmentations. We adopt node dropping and edge perturbation generate two graph views. A GNN model generates two graph representations of the same graph. A nonlinear projection head will map the two graph representations to another latent space. The contrastive loss will be used to optimize the GNN model and the projection head.
SimGRACE (Xia et al. 2022) is an augmentation-free contrastive method for GNN pre- training. We first construct perturbed version of the GNN model by adding noise sampled from the Gaussian distribution. Given an input graph, the perturbed GNN model will gen- erate its representation that forms positive pair with that generated by the original GNN model.
EP-GPPT (Sun et al. 2022a) pre-trains a GNN model using edge prediction. A set of edges in the original graph randomly masked. The pre- pre-training task is to predict whether node pair is connected. Unconnected node pairs are randomly selected to form the negative samples in pre-training.
CCP-GrapPPrompl (Liu et al. 2023) similarly uses edge prediction for GNn pre-training. Given node in the input graph, we randomly sample one positive node from its neigh- bors and negative node that does not link to it. The pre-training task is to maximize the similarity between the connected nodes while minimizing the similarity between the unconnected nodes.
17

---

Published as a conference paper at ICLR 2025
Table 5: Average running time (seconds per epoch) 5-shot node classification tasks five datasets.
Tuning Methods
Cora
CiteSeer
Pubmed
ogbn-arxiv
Flickr
Classifier Only
0.116
0.136
0,663
1.186
5.156
GPPT
0.141
0.151
0.713
1.381
5.828
GraphPrompt
0.126
0.136
0,673
1.377
4.362
All-in-one
0.477
0.578
3,090
6.085
7.357
GPF
0.121
0.131
0.678
1.070
3,482
GPF-plus
0.116
0.131
0,668
1.075
3.427
EdgePrrompt
0.121
0.136
0.693
1.106
3.824
EdgePrompt+
0.146
0.156
0.804
1.377
5.894
Table 6: Average running time (seconds per epoch) on 50-shot graph classification tasks over five datasets.
Tuning Methods
ENZYMES
DD
NCT1
NCT109
Mutagenicity
Classifier Only
0,216
0.176
0.291
0.332
0.302
GraphPrompt
0.276
0.211
0.347
0.357
0.322
All-in-one
0.457
0.643
1.337
1.397
1,206
GPF
0.221
0.191
0.342
0.322
0.307
GPF-plus
0.231
0.191
0.347
0.296
0.312
EdgePrompt
0.226
0.196
0.347
0.296
0.317
EdgePrompt+
0.332
0.302
0.442
0,382
0,402
D MORE EXPERIMENTAL RESULTS
D.1 RESULTS ON MODEL. EFFICIENCY
Table and Table 6 provide the average running time (seconds per epoch) for node classification and graph classification, respectively. From the two tables, tnning method has similar computing time except All-in-one. All-in-one needs more time per epoch since it uses alternating strategies. EdgePrompt has almost the same efficiency as Classifier only without any prompts. In addition, EdgePrompt+ does not introduce significant computational cost.
observe that most graph prompt
D.2 RESULTS ON GRAPH DATA WITH EDGE FEATURES
In our experiments, we conduct experiments over graph data without edge features. However, the real world, many graphs may inherently have edge features. Our method is still compatible with this case. We report the performance of our method and other baselines over BACE and BBBP from the MoleculeNet dataset (Wu al. 2018) in Table From the table, we can observe that our method can outperform other baselines over the two datasets under two pre- pre-training strategies.
D.3 RESULTS WITH EDGE PROMPTS THE FIRST LAYER
Unlike previous studies, we learn prompt vectors at each layer of the pre-trained GNN model. This strategy can consistently avoid adverse information aggregated from different classes. For example, node E3 in Figure [ may receive adverse information from node U when node U3 and node i are from different classes. If we learn edge prompts only at the first layer, node 1/3 will still receive adverse information from node at the following layers. In contrast, our method in EdgePromptt instead learns layer-wise edge prompts, which can consistently avoid the above issue each layer. We conduct experiments Table show the performance for node classification and graph classification, respectively. From the tables, observe performance degradation in most cases, especially for EdgePrompt+. This observation validates our design learning edge prompts at each layer the pre-trained GNN model.
methods with edge prompts only at the first layer. Table g and
18

---

Published as a conference paper at ICLR 2025
Table 7: Accuracy 50-shot graph classification tasks datasets with edge features. The best-performing method is bolded and the runner-up underlined.
Pre-training Strategies
Tuning Methods
BACE
BBBP
Classifier Only
57 57.6211.92
63.5611.08
GraphPrompt
59.37
63.39
All-in-one
56.73 56.73+1.33
65.72
SimGRACE
GPF
57.36+1.32
63. 63.8941.66
GPF-plus
57.16422.21
EdgePrompt
58.1211.01
63.89:11.26
EdgePrompt+
60.4612.65
70.5011.92
Classifier Only
60.40111.08
66.17
GraphPrompt
61 16.99
66.86 0.
All-in-one
56. 56.1711.54
61.7216.9
GPF
60.89+0.71
66.72+0..4
EP-GraphPrompt
GPF-plus
61.3940.22
67 67.58
EdgePrompt
61.09+1.22
66.94
EdgePrompt+
64. 64.66
72.75+2.11
Table 8: Accuracy on 5-shot node classification tasks over three datasets. method is bolded.
The best best-performing
Pre-training Strategies
Tuning Methods
Cora
CiteSeer
Pubmed
EdgePrompt (first layer)
57.7414.42
42.41 13.21
67.33
EdgePrompt
58.6014.46
43.31 23.23
67. 67.76.3.0r
GraphCL
EdgePrompt+ (first layer)
61.66+6.81
44.96
67.5443.00
EdgePrompt+
62.88
46.20
67. 67.41
EdgePrompt (first layer)
36.7444.79
29.47 .3.47
47.98
EdgePrompt
37.26+4.55
29.83
47. 47.20+7.00
EP-GPPT
EdgePrompt+ (first layer)
56.1046.39
42.10=1.41
60.61
EdgePrompt+
56.41.3.62
43.4942.62
61.5114.91
Table 9: Accuracy 50-shot graph classification tasks three datasets. The best-performing method is bolded.
Pre-training Strategies
Tuning Methods
ENZYMES
NCT1
NCT109
EdgePrompt (first layer)
28.83
61.58=2.71
61.82++1.15
EdgePrompt
29.33
62.02=3.0
62.02
SimGRACE
EdgePrompt+ (first layer)
28.58 42.45
61.81 =3.03
62.3644..9
EdgePrompt+
32.67:2.53
67.07 1..9
66.5% 66.53+1.30
EdgePrompt (first layer)
30.75+1.08
61.81 22.07
62.07+1.42
EdgePrompt
30.80+22.09
61.7542.44
62.33
PrrGraphPrompl
EdgePrompt+ (first layer)
31.92+1.41
62.07 22.66
61.66:1.61
EdgePrompt+
33.27:2.77
65.0611.84
64.6411.57
D.4 MORE RESULTS ON CONVERGENCE PERFORMANCE
Figure 5 illustrates the accuracy curves our method and the baselines under two pre-training strategies for graph classification.
D.5 RESULTS WITH DIFFERENT SHOTS
We conduct experiments with different shots. Table 10 shows the performance for 10-shot node classification tasks. In addition, also conduct experiments for 100-shot graph classification tasks
19

---

Published as a conference paper at ICLR 2025
Classifier Only
GraphPrompt
Allin-nns
GPPplux
EdgePrompt
Edg EdgwPrompt+
Accurscc
Accuracy
65
50
200
Epochs
(a) ENZYMES with SimGRACE
(b) NCI1 with SimGRACE
(c) NC1109 with SimGRACE
40
70
70
30
Accurwcy
60
Accuracy
80
20
50
30
Epochs
Epochs
(d) ENZYMES with Ep-GraphPrompt (e) NCI with EP-GraphPrompt NC1109 with GP-GraphPrompt
Figure Convergence speeds of different methods.
and report the results in Table 11 Since ENZYMES uses up all graphs as the training samples in 100-shot setting, we run experiments on the remaining four datasets.
E FUTURE WORKS
In the future, we will investigate the performance of our method under more pre-training strategies, such DGI (Veličković et al.) 2019). InfoGraph (Sun al. 2020), GraphMAE (Hou et al., 2022) In addition, we will explore other designs for edge prompts, such as conditional prompting Yu ett al.) (2024c); Zhou et al. (2022). Furthermore, will also study how to adapt method for heterogeneous graphs.
20

---

Published as a conference paper at ICLR 2025
Table 10: Accuracy 10-shot node classification tasks over five datasets. The best-performing method is bolded and the runner-up is underlined.
Pre-training Tuning Methods
Cora
CiteSeer
Pubmed
ogbn-arxiv
Flickr
Strategies
Classifier Only
65.43
43.97
68.23
26.7841.66
30.34_2.34
GPPT
58.3843.48
44.65+8.47
67.71
26.5443.69
28, 28.80
GraphPrompt
63.5542.40
46.17+3.07
67.73+1.83
25.5141.00
26. 26.74_1.90
GraphCL
ALL-in-one
51.5747.11
43.71
61.20+2.88
21.8442.45
24.63_3.75
GPF 70.06+1..8
47.34
70.70
27.43
27.59 -2.21
GPF-plus
65.32+1.93
43.97
68.32
26.75.11.22
29.81 11.43
EdgePrompt
70.20
47.85
70.54
27.52
28.58 2.55
EdgePrompt+
74.27
52.93
72.70+2.50
28.79
30.74
Classifier Only
62.18
45.62
60.60
47.09
30.35
GPPT
60.00
40.27 47.11
62.16
27.26
30.31 6.39
GraphPrompt
59.2642.06
47. 47.2213.37
62.53
25.6620.89
30. 30.16_1.222
ALL-in-one
49.83
43.9442.63
59.99+1.09
20.03
29. 29.64_3.77
SimGRACE
GPF
67.7344.06
49.08
63.58
27.92
32.96 92.96_3..4
GPF-plus
62.224+3.36
45.44444.15
60.67
27.09#0.88
33 #3.31
EdgePrompt
68, 68.2844.05
49.29
63.67
27. 27.88
33. 33.56-3.68
EdgePrompt+
72.57 73.5743..0
52.78 43.2
59.56+2..04
28.70
32.17 22.77
Classifier Only
34.12+3.26
28.42 #3.32
45.05
15.94
31.96
GPPT
48.4346.16
35.94
56.50+9.44
23.55 133.8... +1.84
29.58-6.81
GraphPrompt
35.08+1.43
28.12+1.56
48.71 +0.28
13.38
29.08
EP-GPPT
ALL-in-one
35.1241.62
27.1942.63
47.11
16.5772.37
32.30
GPF 19.61
35.19+2.46
50.52
22.48+2221
31.60
GPF-plus
33.60+2.34
28.1813.31
45. 15.13
16.07
30.81
EdgePrompt
50.43
31.5643.04
50.90
22.61
30.80
EdgePrompt+
69.65 44
50.74
60.83
21.66422.08
30.78_a.75
Classifier Only
68.1719.25
47.94
75.49
36.6940.66
31.38
GPPT
68.9344.50
48.8348.45
74.78+6.81
25, 25.65+3.55
32.85_3.08
GraphPrompt
68.9542.57
50.26+2.21
75.73+1.40
36. 36.8640.84
30.39_a.a1
ALL-in-one
37.7443.10
46.1446.72
74.24+3.04
22.84 22.84+2.66
30.61-a.25
EP-GrrahPrompt
GPF
72.24
51.07 13.76
77.7712.42
36.91 1..09
29. 7..74.8.9
GPF-plus
68.3219.75
48.33
75.57
36.63
29.40
EdgePrompt
72.2012.47
51.4013..0
77.35
37. 37.16+1.118
32.01
EdgePrompt+
75.0848.11
56.09
76.66+22.0
37.28
34.49
21

---

Published as a conference paper at ICLR 2025
Table 11: Accuracy 100-shot graph classification tasks over four datasets. The best-performing method is bolded and runner-up underlined.
Pre-training Strategies
Tuning Methods
DD
NCT1
NCT109
Mutagenicity
Classifier Only
63.23+1.42
1..03
62. 18
68 1..29
GraphPrompt
62.80+1.15
62.17
61.79
18.14.0000
All-in-one
66.33
60.69 +1.15
62.00
64.39 4..74
GPF
66.75
62.48 1..65
61.98
68.41 1:.60
GraphCL
GPF-plus
68.49
65.39
64.85
68.78 68.78+1.2
EdgePrompt
66.96
63.8441.75
62.422.0.1
68. 68.6941.550
EdgePrompt+
67.81
67.54+1.40
67.94
70.52
Classifier Only
63.7440.96
63.27 1:168
63.20
67 07.65:41.22
GraphPrompt
63.82220.5
63.58
61.5221.10
67. 67.9720.97
All-in-one
68.92=0.61
59.9442.12
62.7920.48
64.47
GPF
65.90-2202
64.32+1..5
63.48
67. 67.44411.01
SimGRACE
GPF-plus 67.04+1.6a
65.28+:2.05
64.7221.64
67.95
EdgePrompt
65.99
65.09+1.06
63. 63.65
68. 68.23+4.81
EdgePrompt+
68.03+
67.24
67.59 11.63
69.50 o..00=0..4
Classifier Only
62.68
58.47 +1.07
63. 63.24.20.77
66.57 1.22
GraphPrompt
1..55++1.o
39.1140.660
62.7640.85
67. 67.12+11.2
All-in-one
62.51+1.25
59.06+1.47
62. 20.96
65. 65.04+0.84
GPF
63.82
59.31
63.75.20.6
66.64 66.64+1134
EP-GPPT
GPF-plus
68.87=2.66
64.48
65.10.2081
69.00..1.10
EdgePrompt
641.8143.27
60.57 1.557
63.60
67. 67.15
EdgePrompt+
68.28
66. 66.28411.16
66.72+1.88
71.52:11.68
Classifier Only
65.95
62.88
62.02
67. 67.39
GraphPrompt
66.21+1.70
62.93
62.27
67.67-10.78
All-in-one
66.45+1.24
60.73+1.46
58.5640.70
66.53+1.110
GPF
68.3742.66
62.68+1.46
63, 1..767
67 07.97
HrrGrrapPrroppt
GPF-plus
68.89
63.91
63.55
67 07..64
EdgePrompt
67.81
63.33
64.00
68 68.04:1107
EdgePrompt+
69.04=2.96
66. 86.8010.55
65.94:1.15
71.48 71.48.1.89
22


Reviews:
[{"review_id": "cs37OmNQtG", "reviewer": "Reviewer_D2XH", "summary": "Recent graph prompt tuning methods have proven effective in adapting pre-trained GNNs to downstream tasks. However, they often overlook the crucial role of edges in graph prompt design. To address this research gap, this submission introduces a new graph prompt tuning method focused on edges, called EdgePrompt. Nevertheless, despite emphasizing the importance of edges in graphs, the authors make an overly strong assumption by considering only a single type of edge. Additionally, the paper does not address edge-related tasks, which significantly undermines the overall contribution and impact of the work.", "questions": "Please focus on answering concerns C1-C3.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "S1. Clear motivation and presentation.\n\nS2. The proposed method can be integrated with existing pre-trained GNNs.", "weaknesses": "**Weakness**\n\nW1. The unclear statements regarding the edge-level aspect weaken the paper’s contributions.\n\nW2. The authors need to further elaborate on the technical contributions.\n\nW3. More experiments are needed to better support the superiority of the proposed method.\n\n**Concerns**\n\nC1. As a study focused on edge-level prompt tuning, the assumption that there is only one type of edge could significantly undermine the contributions and claims of this paper. In line 154, the modeling of the adjacency matrix, $\\mathbf{A} \\in \\{0,1\\}^{N \\times N}$, implies that the paper does not target multi-relational graphs. However, compared to other node-level graph prompting systems, the proposed edge-level graph prompting method could be more suitable for graphs with multiple edge types. The authors may need to clarify this in the submission.\n\nC2. Since this work emphasizes edge-level prompt tuning, it would be beneficial for the authors to explore edge-related tasks, such as edge classification and link prediction, to further expand the scope of the paper.\n\nC2-1. In many real-world scenarios, studying edge-level tasks is highly relevant because the space of edge types can evolve over time. For example, in a social network, a newly introduced user interaction feature might require predicting new edge types using a trained GNN.\n\nC2-2. If the research on edge-level tasks is beyond the scope of current pre-trained GNNs (i.e., no existing pre-trained GNNs focus on edge-level tasks), the authors should clarify this limitation in the submission.\n\nC3. The core Equation (4) in EdgePrompt+ appears overly similar to existing work, which may diminish the paper’s technical contribution. In CompGCN [1], the operation of weighting relation embeddings based on relation base embeddings has already been shown to be simple and parameter-efficient. Therefore, the authors should elaborate on the unique technical contributions of their method.\n\nMinor Concerns:\n\nC4. More classic and promising pre-trained GNNs, such as Infomax, EdgePred, AttrMasking, MGSSL, GraphMAE, and Mole-BERT, could be included in the experimental section. At the very least, the authors should discuss these models and explain why they are excluded from comparison.\n\nC5. Figure 2 presents convergence speeds in terms of the number of epochs. The authors should also analyze the efficiency of the proposed method using learning curves or running time comparisons.\n\n\n**Reference**\n\n[1] COMPOSITION-BASED MULTI-RELATIONAL GRAPH CONVOLUTIONAL NETWORKS, ICLR 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8qOH77P0iu", "reviewer": "Reviewer_WRHJ", "summary": "This paper introduces EdgePrompt, a new graph prompt tuning method that improves graph representation for downstream tasks by learning edge-specific prompts, enhancing the performance of pre-trained GNNs. Extensive experiments show EdgePrompt’s effectiveness across various datasets and pre-training strategies, outperforming several baseline methods.", "questions": "How can the performance of EdgePrompt be improved in scenarios with limited labeled data to enhance its effectiveness in node classification tasks?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. EdgePrompt improves the adaptation of pre-trained GNN models for downstream tasks by introducing edge-level prompts, which helps bridge the objective gap between pre-training and downstream tasks..\n2. Extensive experiments on multiple datasets and pre-training strategies demonstrate the method’s effectiveness, showing better performance compared to existing graph prompt tuning approaches.", "weaknesses": "1. EdgePrompt uses shared prompt vectors, which may not capture the different relationships between edges well. This can limit the model’s ability to use all the information in the graph.\n2. EdgePrompt+ adds multiple anchor prompts and score calculations, which can make the model more complex. This can lead to higher computational costs, making it harder to use in larger graphs.\n3. The method struggles with few-shot learning because most edges lack supervision. This can reduce the model’s performance in real-world tasks where labeled data is limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8VlUFZOSzv", "reviewer": "Reviewer_7GLy", "summary": "The paper proposes EdgePrompt, a graph prompt tuning method that enhances GNNs by learning prompt vectors for edges, improving graph representations. EdgePrompt integrates these edge prompts through message passing, outperforming existing methods across ten datasets under four pre-training strategies.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-motivated. It's important to integrate structural knowledge in prompt learning.\n2. The authors conducted extensive experiments, demonstrating the effectiveness of the proposed methods.\n3. The authors provide theoretical analysis, further proving the effectiveness of the proposed methods.\n4. The paper is well written and easy to follow.", "weaknesses": "1. **Inaccurate statement**: GraphPrompt [1] is not based on a specific pre-training strategy. As shown in GraphPrompt+ [2], all contrastive learning pre-training methods can be unified as subgraph similarity calculations. The link prediction used in [1] can be replaced by other methods.\n2. **Missing related work**: GraphPrompt+ [1] also adds prompt vectors to each layer of the pre-trained graph encoder, which should be discussed and compared.\n3. **Unclear explanation of anchor prompts in EdgePrompt+**: It is unclear what the anchor prompts in EdgePrompt+ represent. In my opinion, anchor prompts are introduced to address the overfitting problem caused by directly learning edge-specific prompts for different edges, but there lacks a explanation for the meaning of the anchor prompts. A more reasonable and effective solution could be conditional prompting [3,4], which I highly recommend the authors explore in future work.\n\n\n[1] Liu et al. \"Graphprompt: Unifying pre-training and downstream tasks for graph neural networks.\" Proceedings of the ACM Web Conference 2023. 2023.\\\n[2] Yu et al. \"Generalized graph prompt: Toward a unification of pre-training and downstream tasks on graphs.\" IEEE Transactions on Knowledge and Data Engineering (2024).\\\n[3] Zhou et al. \"Conditional prompt learning for vision-language models.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.\\\n[4] Yu  et al. \"Non-Homophilic Graph Pre-Training and Prompt Learning.\" arXiv preprint arXiv:2408.12594 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uJCNle2UY7", "reviewer": "Reviewer_t54u", "summary": "The paper presents EdgePrompt, a method that enhances pre-trained GNNs for downstream tasks by using learnable prompt vectors on edges. EdgePrompt+ further customizes these vectors for individual edges. This approach improves graph structural representation and is compatible with various GNN architectures. Experiments on multiple datasets show its effectiveness over existing methods for node and graph classification tasks.", "questions": "Please refer to the points I mentioned in the weakness part.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-organized, with clear points, and is easy to follow.\n2. The effectiveness of EdgePrompt is theoretically guaranteed, and it performs excellently in downstream tasks.", "weaknesses": "1. The motivation for constructing EdgePrompt is insufficient. Why is it necessary to design EdgePrompt under graph prompt tuning? What core problem does EdgePrompt address compared to existing graph prompt tuning methods? What are its advantages?\n2. Compared to ALL-in-one and GPF, EdgePrompt and EdgePrompt+ set different prompt vectors $p^{(l)}$ for each layer. What are the benefits of this design? Both All-in-one and GPF only add prompt vectors in the first layer to reduce dependency on the specific structure of the model. EdgePrompt lacks such advantages, and the paper does not explore the reasoning behind this design. Furthermore, the experimental section does not include relevant comparisons to demonstrate the necessity of setting different prompt vectors for each layer.\n3. The datasets included in the experimental section do not contain initial edge features, which raises doubts about the effectiveness of EdgePrompt on graphs that inherently have edge features. If the original graph already contains edge features, how should EdgePrompt be integrated with these edge features? What would its performance be like in that case?\n4. The downstream tasks involved in the experiments are limited to node classification and graph classification, with other graph tasks such as link prediction and node regression not being included.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)