GPromptShield: Elevating Resilience in Graph Prompt Tuning Against Adversarial Attacks
Authors: Shuhan Song
Ping Li
Ming Dun
Maolei Huang
Huawei Cao
Xiaochun Ye
Venue: ICLR 2025 Poster (2025)
Keywords: pre-training; prompt tuning; robustness; adversarial attacks.
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: The paradigm of ``pre-training and prompt-tuning", with its effectiveness and lightweight characteristics, has rapidly spread from the language field to the graph field. Several pioneering studies have designed specialized prompt functions for diverse downstream graph tasks based on various graph pre-training strategies. These prompts concentrate on the compatibility between the pre-training pretext and downstream graph tasks, aiming to bridge the gap between them. However, designing prompts blindly to adapt to downstream tasks based on this concept neglects crucial security issues. By conducting covert attacks on downstream graph data, we find that even when the downstream task data closely matches that of the pre-training tasks, it is still feasible to generate highly misleading prompts using simple deceptive techniques. In this paper, we shift the primary focus of graph prompts from compatibility to vulnerability issues in adversarial attack scenarios. We design a highly extensible shield defense system for the prompts, which enhances their robustness from two perspectives:Direct Handling and Indirect Amplification. When downstream graph data contains unreliable biases, the former directly combats invalid information by incorporating hybrid multi-defense prompts to the input graph's feature space, while the latter adopts a training strategy to bypass the invalid components and amplifies valid part. We provide a theoretical derivation that proves their feasibility, indicating that unbiased prompts exist under certain conditions on unreliable data. Extensive experiments across various scenarios of adversarial attacks (including adaptive and non-adaptive attacks) indicate that the prompts within our  defense system exhibit enhanced resilience and superiority. This paper explores a new perspective in graph prompt learning, offering a novel option for robust prompt tuning in downstream tasks.

Published as a conference paper at ICLR 2025
GPROMPTSHIELD: ELEVATING RESILIENCE IN GRAPH PROMPT TUNING AGAINST ADVERSARIAL ATTACKS
Shuhan Song!.? Ping Ming Dun't Maoleii Huang
Huawei Cao ; Xiaochun Ye"
State Key Lab of Processors, Institute of Computing Technology,
Chinese Academy of Sciences, Beijing, China
2 University of Chinese Academy of Sciences, Beijing, China
Zhhogggunneun Laboratory, Beijing. China
FFiigyy, Alibaba Group
[songshuhan1 9s, liping20b, dunming, caohuawei, yexiacchun/@ict... acc.on (hhangmaolei. hm1 3al aalalbaba-iinn com
ABSTRACT
The paradigm of "pre-training and prompt-tuning", with its effectiveness and lightweight characteristics, has rapidly spread from the language field to the graph field. Several pioneering studies have designed specialized prompt functions for diverse downstream graph tasks based on various graph pre-training strategies. These prompts concentrate the compatibility between the pre- training pretext and downstream graph tasks, aiming to bridge the gap between them. However, designing prompts blindly adapt downstream tasks based on this concept glects crucial security issues. By conducting covert attacks on downstream graph data, we find that even when the downstream task data closely matches that of the pre-training tasks, it is still feasible to generate highly misleading prompts using simple deceptive techniques. In this paper, we shift the primary focus of graph prompts from compatibility to vulnerability issues in adversarial attack scenar- ios. We design a highly extensible shield defense system for the prompts, which enhances their robustness from two perspectives: Direct Handling and Indirect Amplification. When downstream graph data contains unreliable biases, the for- mer directly combats invalid information by incorporating hybrid multi-defense prompts to the input graph's 's feature space, while the latter adopts a training strat- egy to bypass the invalid components and amplifies valid part. We provide the- oretical derivation that proves their feasibility, indicating that unbiased prompts exist under certain conditions unreliable data. Extensive experiments across various scenarios of adversarial attacks (including adaptive and non-adaptive at- tacks) indicate that the prompts within our defense system exhibit enhanced re- silience and superiority. This paper explores a new perspective in graph prompt learning, offering a novel option for robust prompt tuning in downstream tasks.
INTRODUCTION
Graph neural networks (GNNs) have demonstrated impressive performance across various appli- cations due to their unique ability to handle complex and irregular data, such as recommendation systems (Liu al. 2024), traffic prediction (Shao al.] (2022), and social computing (Sun al.) 2023b). With the advancement of society, real-world scenarios typically align with three character- isties: abundant data, sparse labels, and diverse task domains. Therefore, the algorithm paradigm has shifted from designing specific models for particular problems to training general models that can be fine-tuned for downstream
known as "pre- ppre-training and fine-tuning". This paradigm effectively maximizes the benefits of the large volume of data and has led to the emergence of many outstanding works (Zhu et al. 2021 2020 Velickovic et al. 2019 Hou et al. 2022 2023).
"Corresoonding author.

---

Published as a conference paper at ICLR 2025
+ Pronen ( Tire
Atrck Yerrm! FEEME
SerssTess:: muthemarics instrucior, calenlate the snower the
Valid Prompe Tuning
Pourrt Featere
following problem
Deep's Wrellig As mathematics iestructor, calcclate the answex to the
true (]:
following problam related to
CheckList: mothemarics instructor, calculate
following problema related to KjP112a7K35 ():
Prompt for nodes 3, 6.
Mislead
Invalid
Prompt Tuning
Prongt Sincture
Pretrained GNN
Support Dowastream Tasks
Misicad Dewastream Tasks
adont unified prompts.
Prompl modes
The prompts can be easily misled by simple deceptive tactics. The red edges represent the edges changed by structural perturbations.
Figure 1:
The adversarial language prompts generated by different attacks inspire to design graph prompts tai- lored to various node scenarios.
Figure 2:
As pre-trained models in the language field have become increasingly powerful, prompt- comptt-bsedd ing has gradually emerged as a new research focus. Unlike fine-tuning which adjusts the parameters of pre-trained models, prompt-based methods freeze the parameters of the pre-trained model and focus on adjusting the data space through input transformation. These transformation operations are termed prompts, whose objective is narrow the gap between the pretext of pre- pre-training and the ob- jectives of various downstream tasks, while avoiding the computational cost of retraining the model from seratch. Therefore, "Pre-training, prompting, and fine-tuning" has become the new paradigm.
based tun-
This new paradigm has rapidly spread to the graph field due to its popularity. Although apply- ing prompt- based tuning strategies GNN models poses greater challenges compared to language prompts, many pioneering studies (Sun et al., 2023a Liu et al. 2023b Fang et al.) [224 Yu et al.) 2024) have attempted to propose viable prompts from different perspectives. These studies have made outstanding contributions and they share a commonality in their concentrate on the com- patibility between the pre-training pretext and downstream graph tasks, aiming to bridge the gap between them. However, designing prompts blindly to adapt to downstream tasks based on this compatibility neglects crucial security issues. As shown Figure I effective prompts often pro- vide strong support for downstream tasks, but making certain interventions during the prompt tuning phase, such as simple perturbations to the graph structure, can render the prompts ineffective and lead to catastrophic consequences.
To illustrate this phenomenon more intuitively, we conduct an interesting experiment. For node classification tasks, we employ the well-known MetaAttack (Zügner & Günnemann 2019) to attack the downstream graph data, and we present the classification results of existing mainstream prompt functions on clean and attacked graphs in Figure
GraphPrompt
GPF-plus
GraphPrompt
GPF-plus
Clom
Clan
Mata
GPPT
GPF
GPPT
GPF
All-in-one
MusiGPrompt
All-in-one
MutiGPrompt
(a) Cora
(b) Citeseer
Figure 3: The performance of existing prompt functions on clean graphs and graphs attacked by MetaAttack with 20% attack ratio. The scale of the radar chart represents classification accuracy.
find that when the downstream data closely matches that of the pre-training tasks, it is still feasible to generate highly misleading prompts using simple deceptive techniques. Therefore, this inspires to consider the safety issues of graph prompts.
2

---

Published as a conference paper at ICLR 2025
A recent study Zhu al. 2023) has proposed PromptBench for language prompts, as shown in Figure 2 For instance, there is a clean prompt "As mathematics instructor, calculate the answer to the following problem related to {}", The pre-trained model can adapt downstream tasks based on its key terms. However, adversarial prompts can cause the pre-trained model to focus on both the target text and the adversarial content, amplifying its sensitivity to adversarial perturbations. In word-level attacks like DeepWordBus introducing typos or altering specific words diverts the model's attention from key terms. In sentence-level attacks like StressTest and CheckList, attackers attempt to distract the pre-trained model by adding irrelevant or unrelated sentences to the input text, which may cause to lose focus on the primary context. The robustness of language prompts has led us to examine the robustness of graph prompts, and we aim to equip graph prompts with certain adversarial perturbation capabilities.
Challenges. However, designing robust graph prompt is significant challenge, graph prompts are inherently more fragile than language prompts for two main reasons. First, the cost of perturbing graph is very low and the implementation is straightforward. Just simply adding a few irrelevant edges to the graph structure can lead to catastrophic changes due to the message-passing mechanism, reminiscent of the saying "a single spark can start prairie fire". Second, the design of graph prompts is complex and specialized. Prompts that are suited for specific graph property or structure can quickly become ineffective due to even slight alterations.
Contributions. To smoothly tackle these two challenges, we design a highly extensible shield defense system for the prompts, which enhances their robustness from two perspectives: Direct Handling and Indirect Amplification. When downstream graph data contains unreliable biases, the former directly combats invalid information by incorporating hybrid multi-defense prompts to the input graph's 's feature space. We do not adopt a universal learnable prompt for all node features like GPF (Fang et [ 2024), as we analyzed above, prompt that works for one node may not be suitable for all nodes. Instead, our hybrid multi-defense prompts will customize a individual prompt for each node according to the diverse sensitive information associated with different perturbations. When attacks cause biased node distributions, customized prompts can provide context-specific guidance tailored to nodes in different situations. While the latter adopts a training strategy to bypass the in- valid components and amplifies valid part. We do not need design prompts. Instead, we can directly build upon existing prompts by providing a supplementary tool to address their overlooked safety issues. To ensure the effectiveness of proposed strategies, we provide theoretical analy- sis demonstrating that when the downstream graph data is unreliable, there exists at least one viable prompt capable of achieving results comparable those clean graph under certain conditions.
Overall, the contributions of this paper can be summarized as follows:
We have opened up a new perspective on graph prompt tuning focused on robustness, we shift the primary focus of graph prompts from compatibility vulnerability issues in ad- versarial attack scenarios.
We propose a highly extensible shield defense system for the prompts. This system in- cludes hybrid multi-defense prompt and robust prompt tuning strategy, and we provide theoretical proof of its feasibility.
We conduct extensive experiments in few-shot scenarios under various adversarial attacks (including adaptive and non-adaptive attacks). The results indicate that our hybrid multi- defense prompt and robust prompt tuning strategy significantly enhance the resilience of prompt tuning in downstream biased tasks.
2 RELATED WORK
This paper primarily focuses on advancements in the field of prompt tuning. Additionally, We also provide a systematic description of graph pre- pre-training strategies Appendix A.I
Graph Prompt Tuning
Prompt- based methods freeze the parameters of the pre-trained model and focus on adjusting the data space through input transformation. Due to the parameter efficiency of prompts, they have been widely used in the language domain (Liu et al. 2023a (Sivarajkumar et al.) [2024] Greshake et al.
2023
Mizrahi al. 2024), but they are still in the early stages in the graph domain. However, some
3

---

Published as a conference paper at ICLR 2025
pioneering studies have already been conducted explore this area. GPPT (Sun et al. 2022 applies the pairwise token template (the task token for downstream problem and the structure token contain- ing the node information) to modify nodes. All-in-one (Sun et al., 2023a reformulates node-level and edge-level tasks as graph-level tasks, introducing meta-learning technique to learn prompt. GPF/GPF-plus (Fang et al., 2024) proposes universal prompt tuning method by introducing addi- tional learnable parameter as prompt in the feature space of the imput graph. To achieve effective knowledge transfer from pre-training to a downstream task, GraphPrompt (Liu et al. 2023b) pro- poses unified framework based on subgraph similarity, aiming to retain graph properties that are compatible with the given task during pre-training. MultiGPrompt (Yu et al. 2024) argues that single pretext task is not sufficient. Therefore, it designs a series of pretext tokens to collaboratively address different pretext tasks during pre-training. It also introduces a dual-prompt mechanism that uses both composed prompt and an open prompt to leverage task-specific and global pre-training knowledge. However, all of these prompts focus on the compatibility between pre-training and downstream tasks, neglecting potential security issues.
PRELIMINARIES
In this paper, we primarily discuss the robustness of graph prompt tuning in downstream node clas- sification tasks, as most mainstream attacks are centered around this scenario. We do not focus on the compatibility between upstream and downstream tasks, as existing studies have already provided excellent discussions on this aspect. Instead, we place greater emphasis on the resilience of prompts when faced with biased data.
Define undirected, unweighted graph G = (V,8) with N nodes, N |/]. V {U1, U2 ... .N and S V V represent the set of nodes and edges, respectively. Its feature matrix X' [x1, #2, ,.n] R NNxdi,, where 2,, is din dimeensiioi feature vector of the n- th node.
Notations.
10. 11NNNN is the symmetric adjacency matrix where Ai = if (v;, 0x) E E. Moreover, the labels of all nodes are denoted y. Each node is associated with a label E C, where C {cr, ... (.
Fine-Tuning Define a pre-trained GNN model f. learnable projection head B. Next, we re- formulate the node task as a subgraph classification task and define a downstream task dataset D = w1), (S Mm)) J. where Sz, = (S,, Xns, is the multi-hop neighbor subgraph of node i extracted from G. S, is the structure of the node subgraph. n.s is the set of nodes con- tained in S,. Xns, represents the features of the contained nodes. We adjust the parameters of the pre-trained model and the projection head 0 to maximize the likelihood of predicting the correct labels y, of the downstream local subgraph Sa.- Fine-Tuning aims to maximize the classification likelihoods for nodes in the graph, which can be expressed as follows:
max F.B
Pr.olvi St)
Prompt-Tuning. In prompt-tuning for downstream node classification tasks, the parameters of the pre-trained model ff frozen, and instead introduces lightweight graph prompt function t.. s can be attached in the form of structure or features, transforming the input subgraph into prompt subgraph for pre-trained model f's input. The prompt subgraph be expressed as follows:
Sr XAs (S2.)
(2)
Therefore, the process of prompt-tuning can be described as:
max 6.8
(3)
In the evaluation phase after prompt-tuning, by adding a prompt & to the subgraph of a test node, the frozen model f can process it directly.
4

---

Published as a conference paper at ICLR 2025
Effectiveness Analysis. This study explores prompt effectiveness from the perspective of data dis- tribution. Following (Li et al. 2022b), define St, as prompt subgraph embedding output. As- sume subgraph embeddings follow p($;ly) and are sampled from the joint distribution p(S;. y). Existing prompts seek general templates. Their effectiveness relies on unbiased data in down- stream tasks (Ptrain (S 1) = Preae(S5 y) = Ptrue y)), as analyzed in (Li et al. 2022b). With consistent distributions, prompts work well even with few samples. But attacks cause distribu- tion deviations (Ptrain ($ y) # (Se y) & Perue (S;, y)), leading to overfitting to erroneous data. Despite fixed pre-trained model parameters, misleading prompt harm is underestimated. As p(S;, y) p(y)p(())))) (St1y) and p(y) is consistent, inconsistent p p($²ly) causes differences. Therefore, when the prompts obtained through prompt tuning with few-shot samples the training data are added to the test data based on the principle of distribution consistency, it will result in catastrophic performance. This phenomenon is well reflected in Figure
Attacks. In this paper, we mainly adopt two attack scenarios: the commonly used gray-box global poisoning attack and the white-box adaptive attack. We focus attacks against the graph struc- ture. In the setting of the former scenario, attackers have visibility into the graph data and labels but lacks visibility into the details of the model. MetaAttack (Zügner & Günnemann, (019)) classic gray-box poisoning attack, utilizes surrogate model for the attack, which can be formulated mathematically bilevel optimization problem:
min Lock(fo (G)) s.t. =- =argmmn Ctrain (fo(G) GET(G)
(4)
(G') represents of graphs that satisfies the disturbance budget constraint A. A indicates limit on the number of changes ||A All A. Latk the attack loss function, could be -errai or -Leelf-
Adaptive attacks (Gosch al. 2024 are type of white-box attack with stronger capabilities, where the attacker possesses complete information, including the defender's ' model features, graph struc- ture. labels, and all details. (Gosch et al., (2024) categorizes defenses into seven types and designs adaptive attacks for the most representative method in each category.
Problem Statement. This paper explores the robustness of prompt strategies in node classification tasks. To mimic real-world situations, we pretrain a GNN model on clean graphs and freeze its parameters. Our target is to design a robust prompt and optimization strategy. When the downstream graph data is biased by attacks, our prompt strategy leverages few-shot samples for robust tuning, improving the classification accuracy of unlabelled nodes.
METHODS
In this section, we will introduce highly extensible shield defense system designed for the prompts, which enhances their robustness from two perspectives: Direct Handling and Indirect Amplifica- tion. They provide viable solutions for enhancing the resilience of prompts at two different stages: robust prompt design and robust optimization strategy.
4.1 DIRECT HANDLING
We have previously analyzed that when the data distribution biased, prompts that are effective for certain nodes may mislead others. Therefore, when the downstream graph is attacked, we do not use a universal prompt like GPF. Instead, we designed a hybrid multi-defense prompt. Since we cannot know in advance the tactics employed by attackers, we customize specific prompt for each node's situation by organizing the commonalities and analyzing the focal points of different attacks.
Downstream unbiased subgraphs that share consistent properties with pre-training data often do not require complex prompts. A typical example is subgraphs with high homophily similar to the pre- training data, which inherently exhibit strong compatibility. The nodes that truly require focused prompting are the biased ones resulting from attacks. For these biased nodes, adding prompts iden- tical to those of unbiased nodes is obviously irrational.
5

---

Published as a conference paper at ICLR 2025
We aim for the prompts on these biased nodes capable of mitigating the impact of biased infor- mation. Therefore, we propose several different node filtering strategies based on various sensi- tive points that the attacks may target. These strategies identify potential biased nodes with dif- ferent properties and apply attribute-specific prompts to each of them. For a perturbed subgraph Sp. (S,, Xng, in a few-shot training set, we will next demonstrate how to add hybrid multi- defense prompt it.
Filtering Tip 1: Degree Some indiscriminate attacks (such as DICE (Waniek et al.) 2018) and Random attack) apply the same attack probability to all nodes. In this context, nodes with higher degrees tend to exhibit more stable community features and maintain good consistency with the pre-training data. Under the same attack cost, low-degree nodes are more likely to generate biased community features. Therefore, we filter out the set of nodes with low degree, denoted as Rid = {ul Wul Tdegree where N₂ is u's neighbors and Tdegree is the degree threshold. We provide an auxiliary degree defense prompt pa E idd applied to their feature space, where dın is the dimensionality of the node features. this process can be expressed as:
X [nia] = Xng. [mal + Pa
(5)
Filtering Tip 2: Node Centrality Similarity. Most mainstream attacks aim to disrupt the ho- mophily assumption in graphs, where nodes with the same label and similar features are often con- nected. They utilize low-cost, high-reward tactics increase the graph's 's heterophily as much as possible. This change in heterophily is typically reflected in the central similarity of nodes, where the attacked biased nodes connect with nodes that have different labels and features, thus confusing the assessment of their own community characteristics. Therefore, we filter out the set of nodes with
low central similarity, denoted as v4s, = { |Vo1 EVEN, sim X.) < Tsim }. where Tsim is the
similarity threshold. We provide an auxiliary similarity defense prompt E R2d applied to their feature space. This process can be expressed as:
X
[mss] Xag, [mss] + Ps ng.
Filtering Tip 3: Out-of-distribution nodes. Li ct al.) proposes novel adversarial training paradigm that generates perturbations through adversarial attacks during training, using adversarial edges as out-of -iitr---iistiibuioon samples and initial edges as in-distribution samples to train multiple de- tectors fn = fā. }. Using this detector, we can identify out-of-distribttion edges generated by attacks when faced with a biased graph. More detailed description is provided in Appendix A..2 By detecting these edges, the nodes at both ends the edges become the focal points of the attack method. We add prompts to these nodes and denote them as nod {ullfo(e) = 1, È, e). We provide an auxiliary -ffoofrrbitiitiion defense prompt po E IRd applied to their feature space. This process can be expressed as:
X ng. [Mad] = Xng, [Mad] Po
In this paper, we initially propose mentionet three feasible solutions. As attacks continue to evolve, node sets can be selected based taining different sets of vulnerable nodes, the prompt selection for a node is actually included in set (p,(pa (pa. Pa. Po), (pa, pa), (po).... ..... J. In the experiments, we also design an enhanced prompt Pe E Iddin for nodes that do not have any sensitive prompts added.
wider range of sensitive properties. After ob-
Since each node has different number of prompts, integrating all the prompts that belong each node on graph is a challenge. Defining a node's prompt set as PSS,, we propose two fusion strategies: one is the weighted average Mean, and the other the self-attention mechanism SA (Vaswani 2017) used in language models. Therefore, a node's 's hybrid muti-defense prompt can be expressed as:
PS.1
Σ
PSS. n=1 di Pa E PS
Xng.
WaPo or
(8)
n=1

---

Published as a conference paper at ICLR 2025
Dillierent Sensitivity Prompix
Photti Pool
Date 1
3
Disse
Dis
1
Dete
1
2
0
Dio
DEL
6
Overlap Matrix E
Pruning Based
Hybrid Muti Defense Prompt
Figure 4: Robust Prompt Optimization Workflow for Hybrid Salti-Defense Prompt.
We present the fusion strategy of SA in Figure Finally, the process of adding hybrid muti-defense prompt to the perturbed subgraph Sz, is described as follows:
" = (S Xng, + Xigg)
(9)
Our prompt is applied in the feature space. We do not choose structure-based prompts like All- in-one (Sun et al., 2023a). Instead, feature-based prompts allow for real-time adjustments to the structure of the disturbed subgraphs based on the current state of prompt tuning.
Prompt Tuning, To optimize the learnable prompts, we propose three robust auxiliary con- straints. Consider a biased node classification task with a few-shot labeled training set T. Under the few-shot setting, there are only k labeled cases for each class C, i.e., T We define 52, with prompt added S After inputting S
(3 (2),,,1),... V.),,.., into the pre-trained model f, we define the output node embeddings as Hng.. and the graph em-
Mix(ci) ( (Sexxie
bedding as Gj. To enhance the denoising capability of our prompt, we hope that the output node embeddings better satisfy the first-order proximity. Therefore, we use hidden feature smoothness as a regularizer, as shown below:
C, =
Σ Ex|C
i=1 Σ
[1angy [m]]
Hngy [n]lle
(10)
In each filtering scenario t, we hope reduce the distance between the output distribution bi- ased nodes and unbiased nodes. To elaborate clearly, we define the nodes with prompt added in filtering scenario as n, and the of remaining nodes as nhyp i.e., the complement nodes Cu n
(nf, p(pp) E NT, where NT = (mma. Cmge nta). (Tiss Ong. m.); (Rod Cngy Rod); ... }. Therefore, utilize distribution loss to minimize the distance between the distributions of biased nodes and unbiased nodes in each perturbed subgraph across all filtering scenarios, enabling the prompt to guide and correct the biased distributions. This can be specifically expressed as follows:
IcI Σ
CkI
Σ
DKI(p(Hngg IndDllp (Hng (m(p)))
(II)
NT
(nt (nt JEN
Additionally, to impose constraint on the correlation between different defense prompts, we propose node overlap matrix E for each S Assuming there are T filtering scenarios corresponding to T defense prompts, E is an T x T symmetric matrix. Eg represents the degree of overlap between the biased node sets of the filtering scenarios. Take nua and nts for example, this implies:
n4a 0 11t
Eig = Eÿg =
(12)
Tia U Mita

---

Published as a conference paper at ICLR 2025
The matrix E indicates that defense prompts with higher node overlap should be more similar, while those with lower overlap should exhibit greater differences. Defining the defense prompt matrix of S as P. E Rd'xddn, we obtain the following multi-prompl constraint loss.
kx IC)
C.
Σ
P.P.
E
(13)
With these three auxiliary robust constraints, perform robust prompt tuning by optimizing the following expression:
Ax|C
min
Σ L() aafs 3Lk + 7L.
(14)
P.paPaa Po
C is the downstream loss function (e.g., cross entropy). During the tuning process, prune each S in training set using a tuning similarity threshold Ttanes based on the intermediate state of the current Hag. denoted as
5; (1.)) siim [1) Hngg Lil) > Trune
(15)
After the tuning process, during the evaluation phase, add the tuned prompt to different biased nodes of a biased test graph. Subsequently, we obtain the classification result using the frozen pre- trained model and the projection head 0. We clearly illustrate our hybrid multi-defense prompt tuning workflow in Figure
4.2 INDIRECT AMPLIFICATION
Inspired by (Li et al. 2022b), given the contaminated local structure but consistent global feature distribution the downstream task, we commence by training multi-layer perceptron (MLP) de- void of structural information with few-shot training samples. Subsequently, the trained MLP is employed generate pseudo-lahels for subgraphs. Through comparison of softmax scores, we identify the most confident predictions for each class and incorporate them into new label set V Compute cross-entropy loss with Vysw to tune prompts while freezing the pre-trained model. This method applies to any prompt template. By reducing bias focus and improving unbiased understand- ing, it effectively alleviates the interference of irrelevant information during prompt tuning.
4.3 THEORETICAL ANALYSIS
We are inspired by the theory of GPF (Fang et al, 2024) to prove the existence the robust prompt proposed under specific conditions.
Theorem 1. Given a pre- pre-trained GNN model f and an unbiased input graph 9 = (A, An attack
yields biased graph structure Ā, where = A C o (C = 11T - I 24, and Si₃ = Sj means the edge between nodes u and modified). Taking f(A, X) as the output of the model on unbiased data, there exists hybrid multi-defense prompt P N x M in the feature space (N E R |VXT is the prompt selection matrix, M E Txdin is the prompt matrix, T is the number of defense prompts) satisfying:
X P)
(16)
The proof is in Appendix A.3 Thus, for biased downstream tasks, under certain assumptions, a hybrid multi prompt help the model produce unbiased outputs biased data.
8

---

Published as a conference paper at ICLR 2025
5 EXPERIMENTS
5.1 EXPERIMENT SETUP
This paper primarily investigates the robust tuning of prompts on biased data in downstream node tasks across various attack scenarios. Thus, we use several datasets that are the focus of most attacks, including (MMeallu al. 22000), citation graph (Cora, Citeseer) (Sen et al. 2008). We use a wide range of attacks for experiments. For non-adaptive attacks, we select four representative attacks of different types, including gradient-based attack (MetaAttack (Zügner Günnemann attack (Heuristic attack eLi al.) 2022b)), heuristic- attack (DICE (Waniek al., 2018)). and Random attack. We attack downstream graph data used for prompt luning with relatively large perturbation ratios, where MetaAttack is 25% while the others are 50%. For convenience, it is abbreviated as M-0.25 in the Tables. For adaptive attacks, (Mujkanovic al,) 2022) proposes unit test that includes 2,700 testable graphs. We select perturbed graphs targeting four defenses, including GCNSVD (Entezari et al. 2020 GRAND (Feng et al. 2020). GNNGuard Zhang & Zitnik 2020), and GCNJaccard (Wu et al. 2019), which basically cover most of the de- sign principles of defense models. We have also conducted a general exploration on link prediction and graph classification tasks, with only the robustness abbuttess-rellatd results presented in the main text. All detailed experimental data can be found in Appendix A.4
2019)), Distribution-basee
5.2 PRE-TRAINING STRATEGIES AND PROMPT TUNING
To better showcase the applicability of the prompt function, we select the most representative gen- erative method GraphMAE (Hou et al. 2022) and contrastive method GraphCL Youu et al. 2020) in the field of graph self-supervised learning as pre-training strategies. Additionally, we select the following widely-used prompts to explore their robustness, including GPF/GPF-plus (Fang et al., 2024). All-in-one (Sun et al. 2023a). GPPT (Sun al.) 2022), GraphPrompt (Liu et al., 2023b) and MultiGPrompt (Yu et al. 2024). GraphPrompt and MultiGPrompt require specific pre-training methods, so we follow their templates. The implementations of all prompts are available in Ap- pendix A.5
Table I: Performance of Different Prompts in 5-shot scenario under Non-Adaptive Attacks.
Pre-Training Strsegies
Prompex
Cora
Cieeseer D3-0.
CocaMI
Clean
M-0.25
R-0.5
H-0.5
Clean
M-025
R-0.5
H-0.5
Clean
M-0.25
D-.05
R-0.5
H-0.5
Graphirompe MuliiGrroppt
57.68
34.06
36.01
42.68
35.19
67.20
40 22
34.24
34.20
31.68
67.17
41.96
56.75
44.68
44.32
39.37
43.71
42.31
39
46.68
All-in-on
5053
31.02
17.91
34.29
28.34
43.96
57.18
37.29
27.72
21.31
37.43
28.38
26.50
35.35
27,46
GPF
54.06
33.83
29.43
15.92
38.75
14:37
22.12
30.26
25.58
39.31
24.46
GPF-plas
66.39
36.83
30.52
42.22
26.53
66.19
45. 30
38.68
51.68
14.37
71.30
35.45
58.59
30.74
26.30
GraphCt
GPT
45:24
20.86
24.04
19.50
21.31
41.41
30.18
33.35
23:38
24.57
41.75
34.67
38.51
30.90
32.15
MM-PT
44.94
53.69
54.38
55.24
"LA-PT
58.82
53.65
62.17
52.08
50.50
47.92
52.24
60.53
47.50
60.25
62.77
All-in-aee
44:67
29.75
30.35
35.85
S5. 36.56
44.71
45:20
32.43
24.50
28.82
GPF
66.71
38.82
33.15
48.53
39.27
73.24
53.31
41.45
41.61
29.38
61.01
15.73
20.42
10.57
16.65
GPF-plus
63.40
39.46
33.18
49.80
36.16
75.16
51
40.33
36.81
28.26
70.26
22.62
40.95
45.36
21.70
GrphMAF
GPPTT
67.79
43.17
47.71
42,40
35.46
61.70
46.5 90
49.11
43.56
50.05
71.62
41.55
4% 08
54,84
41.59
*MD-PT
62.49
51.61
58.81
50.11
68.37
55.24
50.72
*[A-PT
68.93
68.30
61.90
67.30
66.62
39.72
60.63
57.26
59.40
61.22
75.90
$7.85
54.32
66.41
64.65
5.3 PROMPT TUNING UNDER NON-ADAPTIVE ATTACKS
We conduct experiments in both 5-shot and 10-shot scenarios. We demonstrate the robustness per- formance under the 5-shot setting here. 10-shot results can be found in Appendix A.6 We denote our hybrid multi-defense prompt as MD-P7 and the indirect amplification prompt as IA- In Table 1 we can see that our prompts offer viable solutions to the potential security issues of prompts from two different perspectives. When the data biased due to attacks, our tuning process can effectively avoid these invalid pieces of information to prevent the generation of misleading prompts. Aside from individual cases, the data in the Table also shows that the robustness of our prompts generally remains within range of + +10% unlike other prompts, which vary by more than 40%. Notably, on clean unbiased tuning data, our prompts still maintain a state-of the art performance in many cases, without sacrificing excessive accuracy for the sake of robustness.
9

---

Published as a conference paper at ICLR 2025
5.4 PROMPT TUNING UNDER ADAPTIVE ATTACKS
We conduct experiments on adaptive poisoning attack graphs targeting classical defenses to demon- strate the versatility of our prompt tuning process. We still present the 5-shot results, while the 10-shot results can be found in Appendix A.7 In Table [
it can be seen that the defense capability of MD-PT appears slightly inferior that of IA-PT when facing more targeted attacks. This is understandable, "as directly ignoring a problem is often easier than dealing with more challenging issues". However, our two strategies still remain highly competitive compared to other prompts.
Table 2: Performance of Different Prompts in 5-shot scenario under Adaptive Attacks.
Pre-Tra Pro-Training
Prompts
CoraML
Citeseer
Stratogles
GCNSVD
GCNJaccard
GRAND
GNNGuard
GruphiPrompt
47.66
34.99
MulticPraampe
24.22
30.20
15.56
14.97
25.16
25.96
31.41
33,55
ProGi
37.32
31.02
52.78
32,38
23.93
30.56
33.92
31.14
GPF
30.88
1844
25.51
GPF-plux
52.29
28.93
41.41
38.37
24.95
21.85
24.95
18.38
GaybCT
GPPT
35.69
41.96
"MD-PT
47.85
45.54
49.67
41.35
51,60
$1.60
$1.01
53.90
1A-PT
61.27
57.05
62.61
ProG
17.05
23.22
27.62
23.45
34.35
27.14
39.05
44.34
GPF
48.56
25.12
50.84
41.00
45.83
31.46
50.37
44.96
GPF-plus
47:21
GraphMAL
GPPT
29.52
20.32
32.06
70. 80
44.76
33.01
46
36.41
"MD-PT
49.17
"LY-FE
66.17
63.54
63.17
63.49
65.54
64.21
64,42
65.60
5.5 ABLATION STUDY
Since IA-PT is nested within any prompt, we primarily discusses MD-PT. We compare MD-PT with six prompt variants: "w/o pa" is variant that does not degree prompt. "w/o ps" is variant that does not use similarity prompt. "w/o ps" is a variant that does not use prompt for out- of-distribution nodes. "w/o Lxi" indicates that distribution alignment is used during the tuning process. "w/o L," signifies that multi-prompt constraints are not employed. "w/o C," indicates that feature smoother is used. Table illustrates the key modules of the robust prompt tuning phase across different biased environments. Additionally, we have implemented a transfer attack experiment in Appendix to validate the scalability and transferability of GPromptShield.
Table 3: Performance of Hybrid Multi-Defense Prompt by Ablating Different Modules.
Cora S-shot
Citeseer S-shot
Cora 10-sbot
Citeseer 10-shot
Ablation Variants
Meta 0.25
DICE 0.5
Meta 0.25
DICE 0.5
Meta 01.55
DICE 0.5
Meta 0.25
DICE
MD-PT
60.12%
54.55
64.20%
52.39
71.02%
62.30%
67.04%
55.35
MD-PT w/o Pu
62.31 %
53.05%
63.68%
51.76%
67.8 17.877%
61.36%
66.67%
56.31 %
MD-PT w/o pa
58.46%
$2.02%
58.97%
53.21 %
69. 69.92%
59.48%
61.57%
54.39%
MD PT w/o Plo
59.32%
55.37%
60.42%
51.98%
69.09%
60.99%
65.69%%
55.18%
MD-PT w/o Cat
59.86%
55.28%
60.42%
52.14%
69.83%
60.81%
64.42%
54.23%
MD-PT w/o C.
59.59%
51.20%
59,9 39.94%
50.00%
69.23%
58.33%
65.64%
55.01%
MD.PT C. 58.32%
56.46%
60.84%
51.50%
69.87%
60.72%
66.34 16.34%
54.58
6 CONCLUSION
In this paper, we shift the primary focus of graph prompts from compatibility to vulnerability issues in adversarial attack scenarios. We design highly extensible shield defense system for the prompts. Specifically, we design hybrid multi-defense prompt based on the principle of direct handling and an adaptable tuning strategy based on the principle of indirect amplification. We theoretically prove the feasibility of our strategies and achieve outstanding robustness performance in both adaptive and non-adaptive attack scenarios.
10

---

Published as a conference paper at ICLR 2025
7 REPRODUCIBILITY STATEMENT
The code for this paper is provided at http://iibb.ooontloyyyCCn////oo tailed experimental data can be found in Appendix [A.4] The implementations of all baselines are available in Appendix A.5 The explanations of the theoretical part in the paper can be found in Appendix A.3
All de-
ETHIC STATEMENT
The robustness of graph prompts in the face of adversarial attacks is key research focus. In this work, we have opened up a new perspective on graph prompt tuning focused on robustness, we shift the primary focus of graph prompts from compatibility to vulnerability issues in adversarial attack scenarios. We design shield defense system for prompts, including direct handling with hybrid multi-defense prompt and indirect amplification through a robust tuning strategy. Our work is aimed at improving the security of graph-based applications. Overall, we believe our research can positively contribute to the field without introducing new security risks.
ACKNOWLEDGEEMEN
The research work supported by National Key Research and Development Program (Grant No.2023YFB45003305) the Beijing Natural Science Foundation (4232036).
REFERENCES
Heng Chang, Yu Rong, Tingyang Xu, Yatao Bian, Shiji Zhou, Xin Wang, Junzhou Huang, and Wenwu Zhu. Not all low-pass filters are robust in graph convolutional networks. Advances in Neural Information Processing Systems, 34:25058-2500 2021.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018.
Negin Entezari, Saba A Al-Sayouri, Amirali Darvishzadeh, and Evangelos E Papalexakis. All you need is low (rank) defending against adversarial attacks on graphs. In Proceedings of the 13th International Conference Web Search and Data Mining, pp. 169-177, 2020.
Taoran Fang, Yunchao Zhang, Yang Yang, Chunping Wang, and Lei Chen. Universal prompt tuning for graph neural networks. Advances Neural Information Processing Systems, 36, 2024.
Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, and Jie Tang. Graph random neural networks for semi-supervised learning on graphs. Advances in neural information processing systems, 33:22092-22111 2020.
Lukas Gosch, Simon Geisler, Daniel Sturm, Bertrand Charpentier, Daniel Zügner, and Stephan Günneeannn. Adversarial training for graph neural networks: Pitfalls, solutions, and new direc- tions. Advances in Neural Information Processing Systems, 36. 2024.
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you've signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, pp. 79-90, 2023.
Kaveh Hassani and Amir Hosein Khasahmadi. Contrastive multi-view representation learning on graphs. In International conference on machine learning, pp. 416-4446. PMLR, 2020.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000-16009, 2022.

---

Published as a conference paper at ICLR 2025
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. Graphmae: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 594-604, 2022.
Zhenyu Hou, Yufei He, Yukuo Cen, Xiao Liu, Yuxiao Dong, Evgeny Kharlamov, and Jie Tang. Graphmae2: A decoding-enhancee masked self-supervised graph learner. arXiv preprint AXXv::2304.04779, 2023.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 18-221188.2.3. 2020.
Nils Kriege and Petra Mutzel. Subgraph matching kernels for attributed graphs. arXiv preprint arXiv: 1206.6483, 2012.
Jintang Li, Ruofan Wu, Wangbin Sun, Liang Chen, Sheng Tian, Liang Zhu, Changhua Meng, Zibin Zheng, and Weiqiang Wang. Maskgae: masked graph modeling meets graph autoencoders. arXiv preprint arxiv:2205. 10053, 2022a.
Kuan Li, YiWen Chen, Yang Liu, Jin Wang, Qing He, Minhao Cheng, and Xiang Ao. Boosting the adversarial robustness of graph neural networks: An ood perspective. In The Twelfth International Conference on Learning Representations.
Kuan Li. Yang Liu, Xiang Ao, and Qing He. Revisiting graph adversarial attack and defense from data distribution perspective. In The Eleventh International Conference on Learning Representa- tions, 2022b.
Ralph Linsker. Self-organization in perceptual network. Computer, 21(3):105-117, 1988.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre- train, prompt, and predict: systematic survey of prompting methods in natural language pro- cessing. ACM Computing Surveys, 55(9):1-33)) 2023a.
Yuxi Liu, Lianghao Xia, and Chao Huang. Selfgnn: Self-supervised graph neural networks for sequential recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1609-1618, 2024.
Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. Proceedings of the ACM Web Conference 2023, pp. 417-428, 2023b.
Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3:127-163, 2000.
Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky, State of what art? a call for multi-prompt Ilm evaluation. Transactions of the Association for Compu- tational Linguistics, 12:933-949, 2024.
Felix Mujkanovic, Simon Geisler, Stephan Günnemann, and Aleksandar Bojchevski. Are defenses for graph neural networks robust? Advances in Neural Information Processing Systems, 35: 9954-8968, 2022.
Galileo Namata, Ben London, Lise Getoor, Bert Huang, and U Edu. Query-driven active surveying for collective classification. In 10th international workshop on mining and learning with graphs, volume 8, pp. 1, 2012.
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-gen: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287 2020.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. Al magazine, 29(3):93-93 2008
Zezhi Shao, Zhao Zhang, Wei Wei, Fei Wang, Yongjun Xu. Xin Cao, and Christian S Jensen. De- coupled dynamic spatial-temporey graph neural network for traffic forecasting. arXiv preprint arXiv:2206.098 /9/ 12, 2022.
12

---

Published as a conference paper at ICLR 2025
Sonish Sivarajkumar, Mark Kelley, Alyssa Samolyk-Mazzanti, Shyam Visweswaran, and Yanshan Wang. An empirical evaluation of prompting strategies for large language models in zero-shot clinical natural language processing: algorithm development and validation study. JMIR Medical Informatics, 12:555318, 2024.
Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang, and Xin Wang. Gppt: Graph pre-training and prompt tuning generalize graph neural networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1717-1727, 2022.
Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, and Jihong Guan. All in one: Multi-task prompting for graph neural (2toorks.(2023)) 2023a.
Xiangguo Sun, Hong Cheng, Bo Liu, Jia Li, Hongyang Chen, Guandong Xu, and Hongzhi Yin. Self-supervised hypergraph representation learning for sociological analysis. IEEE Transactions on Knowledge and Data Engineering. 35(11):1 ((11)::1160-111 18600-1877. 2023b.
Qiaoyu Tan, Ninghao Liu, Xiao Huang, Soo-Hyun Choi, Li Li, Rui Chen, and Xia Hu. S2gae: Self- supervised graph autoencoders are generalizable learners with graph masking. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pp. 787-795, 2023.
Vaswani. Attention all you need. Advances Neural Information Processing Systems, 2017.
Petar Velickovic, William Fedus, William Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. ICLR (Poster), 2(3):4, 2019.
Song Wang, Yushun Dong, Xiao Huang, Chen Chen, and Jundong Li. Faith: Few-shot graph classi- fication with hierarchical task graphs. arXiv preprint 220K:::200502433. 2022.
Marcin Waniek, Tomasz P Michalak, Michael J Wooldridge, and Talal Rahwan. Hiding individuals and communities in social network. Nature Human Behaviour, 2(2):139-147, 2018.
Huijun Wu, Chen Wang, Yuriy Tyshetskiy, Andrew Docherty, Kai Lu, and Liming Zhu. Adversarial examples on graph data: Deep insights into attack and defense. arXiv preprint arXiv: 19Xvv11903.01110, 2019.
Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. Proceedings of the 21th ACM SIGKDD international conference knowledge discovery and data mining, pp. 1365-1374, 2015a.
Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1365-1374, 2015b.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in neural information processing systems, 33: 5812-5823, 2020.
Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Multigprompt for multi-task pre- training and prompting on graphs. In Proceedings of the ACM on Web Conference 2024, pp- 515-526, 2024.
Xiang Zhang and Marinka Zitnik. Gnnguard: Defending graph neural networks against adversarial attacks. Advances in neural information processing systems, 33:9263-9275. 2020.
Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Yue Zhang, Neil Zhenqiang Gong, al. Prompthench: Towards evaluating the robustness of large language models on adversarial prompts. arXiv preprint arXiv:2306.0455 2023.
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.0 04131, 2020.
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021, pp. 2069-2080 2021.
13

---

Published as a conference paper at ICLR 2025
Chenyi Zi, Haihong Zhao, Xiangguo Sun, Yiqing Lin, Hong Cheng, and Jia Li. Prog: A graph prompt learning benchmark. arXiv preprint arXiv:2406.05346, 2024.
Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. Adversarial attacks on neural networks for graph data. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2847-2856, 2018.
Daniel Zügner and Stephan Günnemann. Adversarial attacks on graph neural networks via meta learning. In International Conference on Learning Representations, 2019. URL https://
openreview wttprrvvvwww.et/foorroeeee www.ooorrniiiyyyooe Bdd-By1x20999Xx
14

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 GRAPH PRE-TRAINING STRATEGIES
The pre-training strategy aims to extract knowledge from the vast amount of data in the real world to learn a general model, thereby reducing the costs associated with task-specific training. Most graph pre-training strategies exist in self-supervised form and can be broadly divided into contrastive strategies and generative strategies. Contrastive strategies train an encoder by maximizing the mu-
tual information between different graph views. The representative GraphCL (You al., 2020)
minimizes the differences in graph representations from different views of the same graph. DGI (Velickovic et al., 2019) follows the InfoMax principle (Linsker 1988) by learning node represen- tations through local-global mutual information maximization. GRACE (Zhu et al. (220)) proposes edge removal and feature masking augmentation, GCA (Zhu et al.) 2021) adopts probabilistic adap- tive augmentation. while MVGRL (Hassani & Khasahmadi (2200) employs diffusion graph aug- mentation view. Generative strategies use the reconstruction of input data as a pretext, significantly lowering training costs. Currently, popular paradigm is based on masking strategies inspired by BERT (Devlin et al. 2018) in the language domain and MAE (He et al. 2022) in the image domain. GraphMAE (Hou et al. 2022). as representative example, reconstructs features using masking strategies and designs scaled cosine error. GraphMAE2 (Hou et al., 2023) further extends this idea by designing the strategies of multi-view random re-mask decoding and latent representation pre- diction for feature reconstruction, aiming to reduce excessive reliance on feature discriminability. S2GAE (Tan al.] (023) proposes direction-aware graph masking and cross-correlation decoder. MaskGAE (Li et al. (222a)) adopts edge-wise and path-wise random masking, also introduces a degree decoder to alleviate the problem of structural information overfitting. Pre-training strategies have been studied in great depth, with wide range of pretext tasks extracting knowledge from multiple aspects.
A.2 ADAPTIVE ROBUSTNESS FROM THE PERSPECTIVE OF OUT-OF DISTRIIDDSTTIIIUIION (OOD) GENERALIZATION.
Adaptive attacks are a proposed new robustness evaluation standard and are a type of white-box attack. They indicate that, with thorough understanding of the defense model, attackers can bypass defenses using corresponding strategies. Thus, existing defenses are not as robust as they were evaluated in their papers. These defenses are vulnerable adaptive attacks because most of them designed based on some specific properties that can be used to differentiate adversarial edges from original edges. Adversarial modifications graphs often violate some intrinsic properties shared by the real-world graphs (c.g.. increasing heterophily (Zügner& & Günnemann 2019) and focusing on the high-frequency component (Chang et al.) 2021)). The adversary can easily defeat the defenses by imposing constraints on the same properties during the attack. Therefore, the key to enhancing adaptive robustness is not relying on artificially defined properties. To overcome this dependence, Li et al.) proposed a new approach from the nut-of-distribution perspective, arguing that adversarial edges generated through attacks are inherently ouitof-distributiin compared to the original edges. Thus, by modeling the entire out-of- aatoof-distribution problem, an integrated out-of -i--rri-ittiittionn detector is trained using adversarially generated edges to detect and remove perturbations.
A.3 PROOF FOR THEOREM 1
Proof. Consider pre-trained graph neural network (GNN) model f and unbiased input graph X), An adversarial attack can induce a biased graph structure A. Specifically, A is formulated as Ā = A + C o S, where C = 11 - 2A. When S.j = S, = 1, the edge linking nodes and is either added or removed. Based on this formulation, we can simplify Á as Ā = A + AA Taking [(A, X) as the output of the model on unbiased data, there exists prompt P that can be incorporated into the feature space, ensuring the satisfaction of the following equation:
f(,, + = f
(17)
We adopt the assumptions from GPF (Fang et al. 2024), the pre-trained GNN model f can be represented as:
15

---

Published as a conference paper at ICLR 2025
=
(18)
We substitute f(A, X P) to obtain:
J(A,X + (A++))))))
=AA [[++AA+(L) (1+ + (A AA + P
(19)
=1A+(1+ + AAXW + (A+ (1++)) P)))) + AAPW
-H+AAXI + (A+ (1++)) AAPW
Therefore, to satisfy f(A, X + P) (), need solve the equation:
H AAXW + + + (1 (1) AAPW H
(20)
In our hybrid multi-defense prompt, assuming there are T defense prompts, then P can be repre- sented as N x M, where N E |V|x' is the prompt selection matrix and M E RTx' is the prompt matrix. Therefore, by replacing P with N and M the equation, the problem is transformed into
solving for M. To solve the equation
H + AAXW + (A+ (1+ N)))NWW + AANMW H
(21)
Subtracting H from both sides of this equation, we get
AAXW + (A+ (1 e)4)NMW + AANMW =
(22)
By factoring M from the terms involving M, we have
AAXW ((A+ (1 e))+ + MA)NMW -0
(23)
Let B = (A + AA, then the equation becomes
AAXW + BNMW =0
(24)
Isolating the term with M by subtracting AAXW from both sides, we obtain
BNMW = =AAWW
(25)
Assuming that BN invertible (if BN is not invertible, the solution may not exist in the gen- eral sense may have infinitely many solutions depending the properties of the matrices), we multiply both sides the equation on the left by 1BNN)"1rresllting resulting in
(BN) -1
(BN))'BBMMN BNMW
(BN) (ax))-((-AA)) (-AAXW)
(26)
Since (BN)-1B 7 (the identity matrix), the equation simplifies to
MW -BBN)-
(27)
Case 1: W is invertible
we multiply both sides of the equation on the right by W-1, so
MWW- (BN) - AAXWW-1
(28)
Since WW-1 7, get the solution for M as
16

---

Published as a conference paper at ICLR 2025
M = -BBN
(29)
Substitute == (1++ + AA, the solution for the matrix M is
M - ( ((++ A
(30)
Case 2: W is non-invertible
is non-invertible, we can solve the problem by calculating the pseudoinverse W. In this case, cannot multiply by W-1 directly. We can use Singular Value Decomposition (SVD). Let W = UEVT, where is an m x r orthogonal matrix (m the number of rows of W, is the rank of W), satisfying UTU L (I, is the r-order identity matrix). Σ is matrix, and its diagonal elements σj (i = 1.... r) are the non-zero singular values W, that is, Σ diag( satisfying VVV= ,r)). V is
diagonal
orthogonal matrix (n is the number of columns of W),
The pseudoinverse of W, W+, can be expressed as W VE+UT where Σ+ is the pseudoinverse 1 o, (d=1,-. 1,..,,,
of Σ, also an x r diagonal matrix, and its diagonal elements are
Multiply both sides of the equation MW (BN))-AAXW by W+ the right-hand side, and we can get:
MWW+ -BNN - -BB)) M(WN))--(BN)
(31)
WW+ is a projection matrix onto the column space of W. If W is a full column-rant matrix, WW+ is the identity matrix in the column space of W. Even if W does not have full column-rank, WW+ can still project vectors onto the column space of W.
Discussion on the solution of M
If W is a full column-rank matrix, at this time WW+ an invertible matrix (it is the identity matrix in the dimension of the column space of W). Multiply both sides of the equation on the left by (WW+)-1, and we can get M =
If W is not a full column-rank matrix, let C = - -BN) Although we cannot directly obtain unique solution for M in the multiple values in the complementary space of the column space of W, that is, the solution of M is not unique. However, if there are other conditions, such restrictions the norm M, constraints on some elements of M, etc., we can further determine M by combining these conditions.
of full column-rank, M can still take
Therefore, after finding the inverse-like matrix of W (pseudoinverse in the general case). under certain conditions (such as having full column- -rank,, we can obtain a unique solution for M;; in general cases, we obtain the constraints of M in the column space of W. and further solve for M by combining other conditions. This work does not conduct a detailed analysis of some non-ideal situations. Of course, hope that future work can provide more rigorous proofs and derivations for robust prompts in different biased environments.
A.4 DATASETS AND ATTACKS
A.4.1 DATASETS
For the attacked data, following the settings in Nettack (Zügner et al. 2018), we only consider the largest connected component (LCC). The details of the datasets are presented in Table
For node classification and link prediction tasks, we select homophilic datasets (Cora, Citeseer, Pubmed) (Sen et al. 2008 Namata et al.) 2012). heterophilic datasets (Wisconsin) (Pei et al. 2020) and explore the prompts on large graph (ogbn-arxiv) (Hu et al.] (020). We conduct experiments under 5-shot and 10-shot settings, where the link prediction task includes 5,000 positive edges and
17

---

Published as a conference paper at ICLR 2025
Table Details of the largest connected component (LCC) for each dataset.
Datasets
Nlce
ELeC
Features
Classes
Citeseer
2110
3668
3703
6
Cora
2485
S069
1433
7
Cora-ML
2810
7981
2879
5,000 negative edges, The details of the datasets are presented in Table 5 We present the perfor- mance of the prompt functions on node classification and link prediction tasks for these datasets in Appendix A.8
Table 5: Statisties of all datasets for node classification and link prediction tasks.
Datasets
Graphs
Nodes
Edges
Features
Classes
Task
Catagory
Cora
2708
5429
1433
N
Homophilic
Pubmed
19717
88648
2879
3
N/L
Homophilic
Citeseer
3327
9104
3703
N/L
Homophilic
Wisconsin
251
515
1703
N/L
Heterophilic
ogbn-arxiv
169343
1166243
128
40
N/I.
Homophilic Large scale
For the graph classification task, in order to be more comprehensive, chose the molecular dataset MUTAG (Kriege & Mutzel, 2012), the social network dataset COLLAB (Yanardag Vishwanathan 2015a), the protein dataset PROTEINS (Wang al. 2022). and the social network dataset IMDB- BINARY (Yanardag & Vishwanathan 2015b). We also conduct experiments under 5-shot and 10- shot settings. The details of the datasets are presented in Table We present the performance of the prompt functions on graph classification tasks for these datasets Appendix A.9
Table 6: Statistics of all datasets for graph classification tasks.
Datasets
Graphs
Avg. Nodes
Avg. Edges
Features Classes
Task
Catagory
MUTAG
188
17.9
G
small molecule
COLLAB
5000
74.5
2457.8
0
3
G
social network
PROTEINS
1113
39.1
72.8
3
2
G
proteins
IMDB-BINE BINARY
1000
19.8
96.53
0
2
G
social network
The data recording format for all node classification, link prediction, and graph tasks datasets follows
the approach used in (Zi et al. 2024).
Through generalization experiments on these datasets, we find that our auxiliary robust system does make excessive sacrifice of accuracy favor of robustness. In fact, maintains strong competitiveness on many task datasets, which is a pleasant surprise during the expansion of the experiments. We speculate that the original datasets might already contain some inherent noise, and our robustness enhancement tool further purifies the data during the training process.
A.4.2 UNIT TEST AND ATTACKS
To provide a more comprehensive evaluation of robustness, (Mujkanovic et al. 2022) presents an interesting research point: almost all defenses are evaluated against non-adaptive attacks, leading to overly optimistic robustness estimates. Therefore, they categorize 49 defense methods and select the most representative method from each category to design targeted adaptive attack methods. The adversarial graphs generated by these attacks can be bundled together to test other defenses and can be considered a minimal standard for evaluating the adaptive robustness defense models. In unit test, the datasets are centered around Citeseer and Cora-ML. for each representative model, there are 5 random data splits, each containing poisoning and evasion attacks. The attack budget ranges from 0% 15%, resulting in approximately 2700 testable graphs in total. The unit test module used in this paper is adapted from the following repository: https://github..om// hotie/ubb.ooomooooonggyyyttttttr gon-defensee--obust
18

---

Published as a conference paper at ICLR 2025
The code implementation for attacking the graph using different attacks can be found at the following links.
Metattack: http://ghhhhl..ooommmmuuuggngnnn
Heuristic attack: htt.ah/wwwwyyygiioooooooooooulln
DICE: httpl//sw/hlooooSSS/////o/oo//os
dice.py
Random: http://hhhllloottollooooooooooot rannaa/mmmmmrpppppc.eraaaaaoooo. random.py
A.5 BASELINES FOR PROMPT
The code implementation for all prompts be found at the following links. Additionally, ProG is a library built upon PyTorch to easily conduct single or multi-task prompting for pre-trained Graph Neural Networks (GNNs). You can easily use this library conduct various graph workflows.
GPPT:
GPF/GPFFplus/ https//ss//ghhhuu.ooo///////////
All-in-one: http://tttil.ooo//hdddddsssssssr
GraphPrompt: GraphPronntt/ntts//gggnnnnn http://giihbbboomSSaaaaaiiiiiipp
MutiGPrompt: MutiiPrompttttttss/// http://:/h/hgg/bboo////////////a
ProG: http:///tttbbb.ooomssccdoorrrrrr
A.6
RESULTS OF FEW-SHOT NODE CLASSIFICATION UNDER NON-ADAPTIVE ATTACKS
The results for 10-shot scenarios under non-adaptive attacks. Table 7 shows the robustness performance of 10-shot graph prompt tuning under different non-adaptive attacks.
Table 7: Performance of Different Prompts in 10-shot scenario under Non- Adaativee Attacks.
Strutogles
Training
Prompes
M-8.25
Cora
R-0.5
H-0.5
M.025
Cheseer
R-0.5
M-0.25
CoraML
R-0.5
GraphPrrmpi
72.15
21
39.01
MulliGPionge
52.53
46.88
50.24
37.31
46.76
Alkin-one
69.78
45.26
17.40
27.09
32,08
GPV
60.99
35.65
35.05
39.37
29.85
73.66
41.03
42.17
$7.40
21.90
44.48
15.98
30.45
34,71
24.49
GPF Gli-plus
70.61
41.63
37.21
51.01
36.89
69.16
42.38
42,60
45.58
26.02
65.09
34,83
36.01
24.70
31.63
GraphCt.
GPPT
28.08
"MD-PT
47.01
44.11
56.42
56.12
52.76
*LA-F"
63.96
63.52
55.34
$7.96
62.77
62.66
02.83
$2.20
$7.45
56.31
79.05
70.52
64.64
68.13
74.74
All-in-one
50.55
35.46
30.04
36. 89
32.24
69.97
59.73
43.41
50.73
42.28
46.19
20.48
28.75
21.57
21.98
GPF
71.62
43.56
44.43
55.80 84
30
77.07
59 24
45.80
45:20
27.26
67.17
34.06
29
22.95
29.24
GPF-plos
60.09
44.48
41.49
55.60
40.05
76.59
56 48
40.49
36.
79.76
71.21
21,82
86.86
11.68
24.66
GrahMMEE
GPPT
*MD-PT
72.22
69.41
59.34
64.40
58.23
63.96
64.99
55.45
58. 16
61.90
78.36
67.72
62.49
65.17
50.16
*LA-PT
73.05
73.32
65.23
71.16
71.11
67.43
64.66
59.84
62.49
39.30
72.42
71.37
67.40
72.87
69.95
A.7 RESULTS OF FEW-SHOT NODE CLASSIFICATION UNDER ADAPTIVE ATTACKS
The results for 10-shot scenarios under adaptive attacks. Table shows the robustness perfor- mance of 10-shot graph prompt tuning under different adaptive attacks. On the CoraML dataset, the performance of most prompts vary widely. Similarly, on the Citeseer dataset, there are significant differences across various prompts. As can be seen from the Table, MD-PT and IA-PT strategies per- form more prominently than other prompt functions, demonstrating their effectiveness in enhancing the robustness of prompt tuning against adaptive attacks.
A.8 PERFORMANCE OF HYBRID MULTI-DEFENSE PROMPT ON NODE CLASSIFICATION AND LINK PREDICTION TASKS
The results for 5-shot scenarios under Node Classification and Link Prediction Tasks. Table 9 shows the performance of 5-shot graph prompt tuning under different node classification and link prediction tasks.
19

---

Published as a conference paper at ICLR 2025
Table 8: Performance of Different Prompts in 10-shot scenario under Adaptive Attacks.
Pre-Trainne Pre-Training
Strategies
Prompes
GUNSVD
GRAND
CoeaML
GNINGoard
GCNJsccard
GCNSVD
GRAND
Citeseer
GINNGuard
UCNJaccard
GruphiPrompt
$1.77
38.78
$1.01
35,40
55.99
32.25
45.64
$3.77
MolliGPrompx
29.99
32.38
19.09
13.25
31.49
25.04
34.20
33.39
All-in-one
40.48
25.44
36.71
21.90
41.25
33.05
37.15
42.33
GPP
40,80
43.79
16.65
26.91
38.59
37.78
18.92
26.29
GPF-plus
GraghCL
GPPT
36.11
30.88
35.33
36.52
45.83
41.03
44.25
39.89
*M PT
41.45
50.15
44,82
45.72
58.37
$7.72
63.14
57.72
"IA-PT
63.25
66.61
All-in-nee
20.47
24.34
30.73
23,92
43.20
28.40
44.61
45.15
GPF
GPF-plus
$3.96
53.77
33.55
51.33
53.93
GaphMAE
GPPT
50.74
31.51
34.82
36.57
63.69
31.49
56.42
47.4
"MD-PT
44.20
43.88
45.77
48.34
60.38
60.25
60.76
63.52
65.91
Table 9: Performance of Hybrid Multi-Defense Prompt and Mainstream Prompts on 5-shot Node Classification and Link Prediction Tasks.
Datasets
Cora
Citescor
Pubmed
Wisconsin
ogbo-arxiv
Methods
Node
Link
Node
Link
Node
Link
Node
Link
Node
Link
Pre-train & Fine-tune
42.73
68.79
52.92
55.14
51.07
54.04
20.95
45.15 15
10.92
53.34
GPPT 35.96
65.13
49.79
56.56
55.42
43.88
16.99
77.65
6.52
73.52
All-in-one
50.06
74.68
51.11
60.15
44.76
64.99
26.21
78.85
80.34
Gprompt
68.03
67.25
75.25
58.83
59.93
62.57
44,92
61.13
21.27
53.31
GPF
49.23
GPF GPF-plus 70.55
62.45
74.51
58.47
40.94
82.20
39.45
45.85
22.40
72.43
*MD-PT
71.76
69.95
75.38
78.24
64.00
64.75
50.49
52.49
18.92
65.42
The results for 10-shot scenarios under Node Classification and Link Prediction Tasks. Table 10 shows the performance of 10-shot graph prompt tuning under different node classification and link prediction tasks.
Table 10: Performance of Hybrid Multi-Defense Prompt and Mainstream Prompts 10-shot Node Classification and Link Prediction Tasks.
Datasets
Cora
Citeseer
Pubmed
Wisconsin
ogbo-arxiv
Methods
Nade
Link
Nade
Link
Node
Link
Node
Link
Node
Link
Pre-train Fine-tune
52.23
58.82
60.57
50.35
65.49
55.41
GPPT
47.77
65.47
54.09
74.48
48.91
55.35
22.43
59.13
7.74
$2.86
All-in-one
80.50
Gprompt
70.81
66.45
76.46
50.46
60.10
61.58
43.96
50.43
24.31
75.55
GPF
69.93
73.35
53.82
60.37
39.26
50.08
31.72
57.75
10.36
63.03
GPF-plus
72.79
88.24
78.46
73.02
42.45
88.75
32.26
59.57
14.12
81.40
*MD-PT
73.45
74.55
75.85
77.49
62.34
62.98
35.65
54.42
20.45
68.45
A.9 PERFORMANCE OF HYBRID BULTI-DEFENSE PROMPT ON GRAPH CLASSIFICATION TASKS
The results for 5-shot and 10-shot scenarios under Graph Classification Tasks. Table II shows the performance of graph prompt tuning under different graph classification tasks.
A.10 PERFORMANCE OF PROPOSED PROMPTS ON TRANSFER TASKS
The results for Proposed Prompts and GPF/GPF-plus on Different Attack Transfer Scenarios.
Table 12 shows the performance of graph prompt tuning under different attack transfer scenarios. As seen, IA-PT still demonstrates excellent robustness, while MD-PT is slightly less effective, but still consistently improves the robustness of GPF/GPF-plus. This aligns well with what describe:
20

---

Published as a conference paper at ICLR 2025
Table 11: Performance of Hybrid Multi-Defense Prompt and Mainstream Prompts on Graph Classi- fication Tasks.
Datasets
MUTAG
COLLAB
PROTEINS
IMDB-B
Methods
5-shot
10-shot
5-shot
10-shot
5-shot
10-shot
5-shot
10-shot
Pre-train Fine-tune
68.00
68.00
61.28
62.77
53.37
63.51
63.00
67.00
GPPT
64.00
68.67
50.48
51.97
59.55
60.33
51.25
54.48
All-in-one
70.00
66.67
63.60
54.70
62.92
67.22
61.12
66.12
Gprompt
72.67
70.67
64,53
64.25
61.24
58.65
60.50
69.50
GPF
64.67
65.33
61.80
62.45
64.61
66.29
60.17
66.75
GPF-plus
68.00
72.00
62.20
63.75
65.17
63.26
60.75
67.12
*MD-PT
74.67
74.35
65.07
65.63
67.39
66.52
63.13
62.57
avoiding difficulties is often much easier than facing them head-on. This transferability experiment has been instrumental in helping us validate the scalability and transferability of the designed system.
Table 12: Performance Proposed Prompts and GPF/GPF-plus on Different Attack Transfer Sce- narios.
Cora
Citeseer
CoraMI
Prompts
Attacks
M-0.25
D-0.5
R-0.5
H-0.5
M-0.25
D-0.5
R-0.5
H-0.5
M-0.25
D-0.5
R-0.5 H-0.5
M-0.25
31.44
32.47
20.23
26.26
20.26
20.85
32.80
35.79
28.99
32.07
33.07
30.42
D-0.5
33.97
23.38
28.21
32.70
36.22
25.37
25.37
20.75
36.75
33.79
34.79
31.71
GPF
R-0.5
36.92
36.37
36.24
34.37
25.37
21.96
21.96
21.96
34.63
34.47
39.31
39.95
H-0.5
29.16
35.85
29.30
29.02
31.26
25.37
20.15
19.86
31.39
21.74
34.19
32.54
M-0.25
35.56
35.51
34.92
36.33
25.37
24.46
38.76
40.48
33.15
35.67
35.15
36.75
D-0.5
37.19
36.87
18.19
37.10
26.98
28.77
29.49
27.40
34.35
33.99
32.31
32.03
GPF-plus
R-0.5
42.26
40.35
40.26
44.35
34.29
35.71
35.34
35.68
38.11
37.67
36.03
37.83
H-0.5
33.56
30.79
35.92
32.79
20.20
33.12
34.78
33.07
40.03
40.45
37.11
39.95
M-0.25
47.06
42.40
43.11
47.89
42.81
39.62
40.46
44.85
53.82
43.57
42.17
47.51
D-0.5
47.08
40.15
42.81
47.13
42.18
36.72
36.32
38.88
43.44
44.55
40.02
51.25
*MD-PT
R-0.5
48.04
45.82
50.34
53.52
40.46
37.82
37.77
38.75
52.76
41.57
45.24
50.80
H-0.5
50.15
41.45
44.67
47.48
41.82
38.39
35.55
41.08
41.35
41.34
39.45
52.36
M-0.25
65.22
49.50
60.16
62.53
55.66
48.76
55.04
58.35
61.61
54.89
59.96
65.57
D-0.5
64.54
52.24
57.60
61.57
57.41
55.34
51.24
53.61
58.19
52.52
53.02
57.56
*IA-PT
R-0.5
57.85
48.04
54.69
60.46
56.52
54.03
48.96
54.21
62,19
49.13
54.00
61.92
H1-055
60.41
53.62
55.13
62.55
57.94
52.73
50.47
58.55
62.10
$2.71
59.48
58.77
21


Reviews:
[{"review_id": "abotdXDOVR", "reviewer": "Reviewer_iA8J", "summary": "This paper introduces a novel approach to graph prompt tuning focused on robustness in adversarial attack scenarios. It proposes a highly extensible shield defense system with a hybrid multi-defense prompt and robust prompt tuning strategy, demonstrating theoretical feasibility. Extensive experiments in few-shot scenarios under various adversarial attacks show that their strategies significantly enhance prompt tuning resilience in downstream biased tasks. The use of feature-based prompts allows real-time adjustments, and the paper highlights the effectiveness of their methods in both adaptive and non-adaptive attack scenarios.", "questions": "Refer to the weaknesses above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The proposed mixed multi-layer defense strategy in this paper demonstrates technical innovation by combining various defense mechanisms, such as feature-based prompting and real-time adjustments. This approach significantly enhances the system’s tolerance to adversarial attacks, offering a fresh perspective on the research of graphic prompting adjustment systems.", "weaknesses": "One weakness of the paper is the absence of a comparative analysis with existing systems or defense strategies against adversarial attacks. \n\nThe paper primarily focuses on theoretical analysis and experimental validation in controlled settings. However, the lack of real-world implementation or case studies hinders the practical applicability assessment of the proposed method. To enhance the relevance and impact of the research, future work could involve implementing the system in real-world scenarios and evaluating its performance in practical applications.\n\nThe evaluation section of the paper might benefit from a more comprehensive selection of evaluation metrics to assess the proposed method’s performance accurately. Specifically, incorporating additional metrics such as computational efficiency, scalability, or user experience aspects could provide a more holistic evaluation of the system’s capabilities and limitations. Including a diverse set of evaluation criteria would offer a more thorough understanding of the method’s strengths and weaknesses.\n\nThe paper could further strengthen its impact by discussing the generalizability of the proposed method to different types of adversarial attacks or diverse datasets. Providing insights into how the approach could be adapted or extended to address a broader range of security challenges would enhance the paper’s contribution to the field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wFrSII0MJb", "reviewer": "Reviewer_1kqh", "summary": "This paper explores enhancing the robustness of graph prompt-tuning methods against adversarial attacks in graph neural networks (GNNs). It identifies a vulnerability in current prompt-based methods, which are highly susceptible to adversarial alterations. The proposed solution is a \"shield defense system\" that enhances robustness through two strategies: Direct Handling and Indirect Amplification. Direct Handling customizes multi-defense prompts based on node-specific attributes, targeting biased nodes introduced by attacks, while Indirect Amplification leverages few-shot learning and selectively focuses on untainted information, thereby circumventing the misleading data introduced by adversaries. Theoretical validation demonstrates that this approach can maintain unbiased outputs despite adversarial perturbations. Experimental results show that the defense system enhances resilience under various attack scenarios, outperforming existing prompts in maintaining accuracy and robustness.", "questions": "1. Why other methods' results are getting close to the proposed methods in 10-shot experiments? Is it possible that other methods will outperform the proposed methods with even more shots?\n2. Adding the figures to represent prompt design and robust optimization strategies stages could be better.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea and structure is easy to follow.", "weaknesses": "1. Some of the tables are confusing, for example, the table 1. Pre-Training Strategies & Prompts appear simultaneously in the top left corner. Should we design the table in a better way or use other charts to represent these results?\n2. Section 2 Related work, if there is no more subsection 2.1, is not necessary.\n3. The whole framework should have a figure to clearly show the stages of prompt design and robust optimization strategies", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6ZGR7MdsJ2", "reviewer": "Reviewer_zGFc", "summary": "This paper introduces a robust graph prompt learning method based on adversarial training to defend Graph Neural Networks (GNNs) against adversarial attacks. The method begins by identifying nodes typically targeted in these attacks, such as low-degree nodes, nodes with low central similarity within multi-hop subgraphs, and nodes linked to adversarially perturbed edges. Next, the approach fine-tunes three distinct types of graph prompts, each corresponding to one of the targeted node types. The objective of the prompt tuning process is to ensure: (1) smoothness in node embeddings within multi-hop subgraphs across the three types of graph prompts, (2) consistency between prompt-enhanced and unprompted node embeddings, and (3) smoothness across embeddings from different types of prompts. Finally, the study applies this method to multiple graph prompt learning models, evaluating its defense effectiveness against non-adaptive attacks and its performance in combination with other defense strategies for adaptive and graph poisoning attacks.", "questions": "Please check the question raised in the comments concerning the weakness of this study.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "First of all, this is an interesting work in investigating robust graph prompt tuning to increase the stability of GPL against potential graph poisoning attacks. Especially, if the test graph structure is intentionally manipulated, this method offers a solution to suppress the impact of the poisoning efforts over the decision of GPL.", "weaknesses": "There are several points that need to be strengthened: \n\n1/ This work only focuses on node classification-oriented GPL and does not discuss graph poisoning attacks targeting at manipulating node attributes. It is not clear if the proposed robust prompt tuning method can be applied to edge or graph level learning tasks and can be adapted to both node- and edge-manipulation based attacks. I am asking this as GPL is a multi-task system that can transfer knowledge between different types of GNN learning tasks (node, edge and graph classification). From this perspective, the evaluation in this study is not sufficient. Extending the coverage over more graph datasets and GNN learning tasks should be considered. I'd recommend the author refer to the dataset choice and learning task configurations in [1]. \n\n[1] All in One: Multi-Task Prompting for Graph Neural Networks, https://arxiv.org/pdf/2307.01504. \n\n2/ The theoretical proof in Section 4.3 is not associated with the main claim (robust prompt tuning) in this work. This theoretical analysis should be dedicated to explain how the robustness is improved via the proposed tuning approach. However, the link between Section 4.3 and the algorithmic design is unclear in the current submission. \n\n3/ The ablation study may also need to consider one situation: what if every node belonging to the three types (low degree, low central similarity and out-of-distribution nodes) has the same graph prompting function ? Besides, which types of the nodes should be given more weights in the robust tuning process, or they are handled equally in adversarial attacks ? \n\n4/ What are out-of-distribution nodes in a graph? Citing the paper (Li et al) is not enough. Further discussion regarding what these nodes are and why they are concerned in adversarial attacks should be involved. \n\n5/ How do you choose the threshold values $\\tau_{degree}$ and $\\tau_{sim}$ ? Are they set empirically, or is there any protocol to choose adaptively in different GNN adversarial attack scenarios ? Similarly, how do you choose $\\tau_{tune}$ in Eq.15 ? Does it mean you completely exclude the subgraphs if the embeddings of nodes in these subgraphs meet the condition in Eq.15? Will it also bring harm to the utility of the trained GPL as it excludes information from the training process. \n\n6/ As an adversarial training method, it is not surprised to see the proposed defensive graph prompts can mitigate one attack method when they are trained to be resilient to this attack. I am wondering if the proposed method has the transferability. In particular, if it is trained with the perturbed graph using one of the attack methods (MetaAttack, Heuristic attack, Random attack, DICE) can be also resilient to the other three attacks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)