Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Level
Authors: Runlin Lei
Yuwei Hu
Yuchen Ren
Zhewei Wei
Venue: NeurIPS 2024 poster (2024)
Keywords: Graph Neural Networks
Graph Adversarial Attack
Graph Injection Attack
Primary area: graph_neural_networks
TL;DR: We study text-level Graph Injection Attacks beyond the embedding level, revealing new challenges and insights about Graph Injection Attack designs.

Abstract: Graph Neural Networks (GNNs) excel across various applications but remain vulnerable to adversarial attacks, particularly Graph Injection Attacks (GIAs), which inject malicious nodes into the original graph and pose realistic threats.
Text-attributed graphs (TAGs), where nodes are associated with textual features, are crucial due to their prevalence in real-world applications and are commonly used to evaluate these vulnerabilities.
However, existing research only focuses on embedding-level GIAs, which inject node embeddings rather than actual textual content, limiting their applicability and simplifying detection.
In this paper, we pioneer the exploration of GIAs at the text level, presenting three novel attack designs that inject textual content into the graph.
Through theoretical and empirical analysis, we demonstrate that text interpretability, a factor previously overlooked at the embedding level, plays a crucial role in attack strength. 
Among the designs we investigate, the Word-frequency-based Text-level GIA (WTGIA) is particularly notable for its balance between performance and interpretability. 
Despite the success of WTGIA, we discover that defenders can easily enhance their defenses with customized text embedding methods or large language model (LLM)--based predictors. 
These insights underscore the necessity for further research into the potential and practical significance of text-level GIAs.

Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Level
Runlin Lei
Yuwei Hu Renmin University of China huyuweiyisui @ruc. edu. cn
Yuchen Ren Renmin University of China siriusrenOruc. edu. cn
Renmin University of China runlin in leideruuu edu. en
Zhewei Wei Renmin University of China zhewei@ruc. edu. cn
Abstract
Graph Neural Networks (GNNs) excel across various applications but remain vulnerable adversarial attacks, particularly Graph Injection Attacks (GIAs), which inject malicious nodes into the original graph and pose realistic threats. Text-attributed graphs (TAGs), where nodes are associated with textual features, are crucial due to their prevalence in real-world applications and are commonly used to evaluate these vulnerabilities. However, existing research only focuses on embedding-lewee GIAs, which inject node embeddings rather than actual textual content, limiting their applicability and simplifying detection. In this paper, we pioneer the exploration of GIAs at the text level, presenting three novel attack designs that inject textual content into the graph. Through theoretical and empirical analysis, we demonstrate that text interpretability, a factor previously overlooked at the embedding level, plays a crucial role in attack strength. Among the designs investigate, the Word-frequency- based Text-level GIA (WTGIA) particularly notable for its balance between performance and interpretability. Despite the success of WTGIA, with customized text embedding methods or large language model (LLM))based predictors. These insights underscore the necessity for further research into the potential and practical significance text-level GIAs. The code is available at www/Leerrnnin//eeeee...... wew--leeellleee Caae--rraphaatte Attack
discover that defenders can easily enhance their defenses
Introduction
Graph Neural Networks (GNNs) have exhibited exceptional performance in various tasks 30 34
However, their vulnerability graph adversarial attacks has been increasingly exposed. Research shows that Graph Modification Attacks (GMAs), which perturb a small portion of edges and node features of the original graph, can significantly degrade the performance of GNNs [37 38). Besides GMAs, Graph Injection Attacks (GIAs) generate harmful nodes and connect them to the original nodes, thereby reducing the performance GNNs. Since adding new content to the public dataset more practical than modifying existing content, GIAs are considered one of the most realistic graph adversarial attacks
28
"Zhewei Wei is the corresponding author. The work was partially done at Gaoling School of Artificial Intelligence, Beijing Key Lahoratory of Big Data Management and Analysis Methods, MOE Key Lab of Data Engineering and Knowledge Engineering, and Pazhou Laboratory (Huangpu), Guangzhou, Guangdong 510555, China.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Text-Attributed Graphs (TAGs), characterized by nodes with text-based features, form a crucial category among various graph types. In the early stages of GNNs, textual node feature processing is decoupled from the network design. Researchers transform text into fixed embeddings to investigate GNN designs on downstream tasks [13 g 27 However, recent advancements have shifted toward non-decoupled framework that integrates raw text into the design, significantly improving the ability to capture textual characteristics and enhance performance [9] [0]. Despite this progress in GNN design, current GIAs still focus fixed embedding, aiming at only the injection of node embeddings rather than raw text on TAGs. This leads to several practical issues: 1) The attack setting is unrealistic. In real-world scenarios like social or citation networks, it is more practical to inject nodes with raw text rather than node embeddings. For example, to attack a citation network dataset, it would be more natural to upload fake papers rather than batch of embeddings. 2) The injected embeddings may be uninterpretable. As minor perturbations at the embedding level can lead to unpredictable semantic shifts, traditional GIAs fail to ensure that injected nodes convey understandable semantic information. 3) Increased Detectability of Attacks. Attackers usually have access only to raw text, not the processed embeddings defenders use. This discrepancy makes it challenging for attackers to mimic the embedding methods of defenders, leading to structurally abnormal embeddings. For example, embeddings based on word frequency are sparse, while those from Pre-trained Language Models (PLMs) are typically output continuous and dense. If an attacker injects dense embeddings while defender uses a word-frequent www.uercyegeeaeeee based model, the difference in the embedding structure makes the injected embeddings easy to detect.
Embelding based GITA
Cietinaaza
Invervias Methoch
ITGIA
Evabuation and veleetion of bines is machine fearning In herdaadins we aiflanity sema Nus as in machine inporbeiz of earewate
Vanila
Conreet Generntion
Task: Prisonisg
VTGIA
Attacker
Peloming Fronpt
D
Bitayy
Tisk: Words to Tox
WIGIA
Emieedding based CITA
(a) The task Texe-Level GIAs
(b) Framework of ITGIA, VTGIA and WTGIA
Figure 1: Illustration of the Text-Level GIA setup and the three designs explored.
To address the above limitations, in this paper, we innovatively explore text-level GIAs, com- prehensively examining their implementation, performance, and challenges. We explore three text-level GIAs: Vanilla Text-level GIA (VTGIA), Inversion- Text-level GIA (ITGIA), and Word-frequency-boee Text-level GIA (WTGIA), all of which successfully inject raw text into graphs and degrade the performance of GNNs, as shown in Figure [b] Experiment results indicate that interpretability presents significant trade-off against attack performance. ITGIA struggles to invert embeddings to interpretable text because the interpretable regions of the injected embeddings are ill-defined. VTGIA, which solely relies on Large I Language Models (LLMs) for text generation, compromises attack effectiveness for its pursuit of text interpretability. WTGIA, as a balanced approach, manages to produce harmful and coherent text more effectively. However, the trade-off' between performance and interpretability, coupled with poor transferability to different embedding methods, limits the performance of text-level GIAs in practice. Moreover, LLM- b-sedddd predictors shown to significantly enhance defenses against text-level GLAs, highlighting the substantial progress still needed to refine these attacks. In summary, contributions
follows:
To the best of our knowledge, are the first to unveil the dynamics of text-level graph adversarial attacks on TAGs, especially for GIAs. We discuss the impracticality of embedding- based GIA in real-world applications and propose a more realistic attack setting.
We propose three effective text-level GIAs and demonstrate the trade-off between attack perfor- mance and text interpretability from both theoretical and empirical perspectives, providing insights for further refining text-level GLAs.
We reflect on the challenges of graph adversarial attacks at the text level. We discover that simple strategies the text level can enhance defense performance against text- GIAs, which reemphasizes the importance of exploring GLAs at the text level.
2

---

2 Background and Preliminaries
Graph (TAG) is represented as = (V.&, {si)), where V is the node set and &, the edge set. We denote N as the number of nodes and A E {0, 13 as the adjacency matrix. For each node V; E V. a sequential text feature 8; is associated. We focus on semi-supervised node classification tasks, where each node is categorized into one of C' classes. The node labels are denoted The task is to predict test labels test based on & and training labels Y train
by y {O....,CC CO 1)
Graph Neural Networks. We use Jo to denote a GNN parameterized by 0. Generally, raw text {sc} first embedded into feature matrix X E IR- using word embedding techniques, where F is the output dimension. In the rest of our paper, when referring to vanilla GNNs that take text embeddings as imputs, we represent the (ext-attributed graph as = (V, & X) and express f(X; A). The update process of the (-th layer of f can be formally described as:
h = f (h. AGGR ({h) :j N.))),
where hi X,, .G the neighborhood set of node us, AGGR is the aggregation function, and f is a wressage-passingg layer that takes the features of by and its neighbors as inputs.
Graph Injection Attacks and Related Works. Graph injection attacks aim create perturbed graph g' = (V",'', ,') to disrupt the accuracy of a GNN on target nodes. Formally, denote the number of injected nodes Ninj- and the number of nodes in the poisoned graph G' as N Then, the node set of G' can be formulated as V = =VU Vinj- and:
X
A
Aijj
X
Xin
A''
AT
O
(2)
inj
where Xijj E NNP.P Anj E {0, 1.1...x.... and O E {0, 1,,13NxpxNa)
For the design of injected node features, one line of research focuses statistical features to ensure unnoticeability, sacrificing attack strength by excluding learnable processes for features (25] 4 In
parallel, other works target learnable injected embeddings. Wang al. (29] pioneer injecting fake nodes with binary features to degrade GNN performance. AFGSM [2]] introduces an approximation strategy linearize the surrogate model, efficiently solving the objective function.
Starting from KDDCUP 2020, research shifts towards injecting continuous node embeddings under the constraints in Equation and minimum elements in the original feature matrix X as Tmax and Tmin, and the maximum degree allowed for each injected node as b. The objective of these GIAs can be formulated as:
[33 36 2 Denote the budget of injected nodes as A. the maximum
Kf (G'), = y; Us E VTH
min G'
Vrl
s.t. Ninj < S. dinj S b, Tmin Se Xinj Se Xmmax 1,
where Vr is the target node set, d,, the degree for node u, and X1 Se X2 indicates each element of X₁ is less than or equal to the corresponding element in X₂. Among continuous strategies, TDGIA 36) generates node features by optimizing a feature smoothness objective function. Chen et al. 2 enhance homophily unnoticeability by adding a regularization term to the objective function. These approaches show better performance while maintaining unnoticeability the embedding level.
It is worth noting that the evaluation datasets for the aforementioned GIAs are primarily TAGs 33 e.g., such Cora and CiteSeer. However, all the studies remain at the embedding level. In experiments, we adhere to the structural budget detailed in Equation omitting embedding-level constraints we pioneer GIAs the text level. More related works are provided in Appendix D
Evaluation Protocol. In this paper, we explore GIAs in an inductive, evasion setting, following the framework established by 333 to ensure closeness to practical scenarios. Attackers have access to the entire graph and associated labels, except for the ground-trute labels of the test set and the victim models of the defenders. Defenders train their models clean, unperturbed training data and then use trained models to classify unseen test data.
General Experiments Set-up. We conduct experiments on three TAGs, Cora 17 CiteSeer 6
and PubMed [23], which are commonly used in evaluating GIAs. The dataset statistics and attack budgets are provided in Table 5nd and Table 7 in the Appendix. We adopt full splits in (3 for each
3

---

dataset, which uses 60% nodes for training, 10% nodes for validation, and 30% for testing. For word embeddings techniques, we include Bag-of-Words (BoW) and GTR [T]] a T5-based pre- trained transformer that generates continuous embeddings of unit norm. The default BoW method constructs vocabulary using raw text from the original dataset based on word frequency, with common stop- -ords excluded. The embedding dimension of BoW is set to 500. In the experiments, we use a 3-layer vanilla GCN or a 3-layer homophily-basee defender EGNNGuard following [2]. EGNNGuard defends nomophily-noticeabll attacks by cutting off message-passing between nodes below the specified similarity threshold. The hyperparameters and training details are provided in Appendix H. We repeat each experiment 10 times and report the average performance and the standard deviation. Unless otherwise specified, we bold the best (the lowest) results.
3 Text-Level GIAs: Interpretability Matters
Although GIAs are considered realistic graph attacks, traditional methods focus mainly on the embedding level, which can be overly idealized. Typically, attackers only have access to public raw data, such as paper content and citation networks, on platforms like arXiv, and defenders process this data more personally. Since attackers struggle to access embeddings trained by defenders. injecting nodes with suitable embeddings is challenging. Conversely, injecting new raw data into the public dataset is more realistic for attackers. This leads to an important question: Can we generate coherent text for injection that ensures the effectiveness of attacks while maintaining interpretabilltyy
3.1 Inversion-based Text-Level GIAs: Effective yet Uninterpretable
Embedding- GIAs have demonstrated strong attack performance against GNNs 36 2
To
achieve text-level attacks, a straightforward idea is to use inversion methods to convert embeddings back into text. Recent advancements in text inversion methods, such as Vec2Text (18) have enabled text recovery from PLM embeddings with a 66% success rate for examples averaging 16 tokens. This breakthrough opens new possibilities for achieving text-level GIA that is comparable in performance to those at the embedding level. which we term ineesion-basee Text-level GIA (ITGIA).
Implementation of ITGIA. ITGIA consists of two steps: generating poisoning and invertible embeddings, and then converting them into text. Since the effectiveness of current inversion methods has only been validated normal text embeddings, it is crucial ensure that generated embeddings lie within an interpretable and feasible region. However, defining these interpretable regions precisely is challenging. The constraint specified in Equation (3) limits generated embeddings to cubic space, which may be overly broad for feasible embeddings. For example, PLMs typically produce embeddings located on spherical surface with an L.2-norm of 1, representing a significantly smaller region. To meet this fundamental requirement, we employ Projected Gradient Descent (PGD) [6). projecting each injected embedding onto the unit sphere after every feature update. To further address the challenge of defining interpretable regions, we incorporate Harmonious Adversarial Objective (HAO) [2) into the GIA objective function. Optimizing over HAO increases the similarity between injected embeddings and those of normal text from original datasets, thereby bringing the injected embeddings closer to interpretable regions. Details of ITGIA and HAO are provided in Appendix [.] and Once the embeddings are obtained, we utilize the GTR inversion model in [18] with 20-step correction process to revert embeddings back into text.
Analysis of Performance and Interpretability. Following (2,. we use sequential injection frame- works, including the sequential variants of SeqGiA (Random injection). TDGIA, ATDGIA, MetaGIA, and AGIA as the embedding-levee backbones. The results of ITGIA are displayed in Table We can see the embeddings after inversion deviate significantly from the injected embeddings, as indicated by the low cosine similarity. This suggests substantial information loss during inversion, leading to poor attack performance. Figure illustrats the underlying reason: although injected embeddings meet basic norm constraints, they struggle to fall within the much smaller interpretable embedding region, hindering accurate inversion. Although incorporating HAO enhances the inversion accuracy and improves attack performance, the improvement comes at the expense of embedding-level attack performance (2). As a result, increasing HAO weights does not consistently yield better results, as shown in Figure Furthermore, the generated text remains incoherent with high perplexity, as demonstrated Appendix ! This renders real-world applications impractical and highlights the challenges faced by continuous embedding-level GLAs.

---

Table 1: Performance of GCN graphs under ITGIA. Raw text is embedded by GTR before being fed to GCN for evaluation. "Avg. cos" represents the average cosine similarity between the embeddings of the inverted text and their corresponding original embeddings across five ITGIAs. "Best Emb." represents the best attack performance across the five variants at the embedding level.
Dataset
Clean
HAO
Avg. cos
SeaGLA
MetaGIA
TDGIA
ATDGIA
AGIA
Best Emb.
0.14
1.76
76.52 + 1.45
76.73 74,63 2:48
Cora
87.19 0.62
0.67
65.67 + 1.48
65.77 = 1.15
71.49 1.71
68.59 2 1.51
31.14 + 0.05
CiteSoer
75.93 + 0.41
0.11
68.17 a 0.94
69.39 = 0.89
68.24 = 1.30
69.72
66.18 : 1.19
21.45 + 0.58
0.56
64.79 + 1,30
65.11 = 1.01
67.43:0..8
71.89 0.550
64.79
0.06
65.13 a: 1.67
58.96 = 1,25
59.49 =: 1.08
69.81 a: 1.90
66.16 a: 0.97
PubMed
87.91 0.26
0.59
66.40 2.33
58.56 + 1.22
60,26 + 1.32
76.23 2.08
65.77 0.91
38.32 0.00
Cora
CiteSeer
PubMed
1.0
80.0
Cosine Simllrity
75.0
71
0.5
19/63
70.0
40
56,8
65.0
0.0
5
0
3
60.0
11A0 Weight
11A0 Weight
HAA Weight
Cosine Similarity
Attaok Performncce
Figure 2: The change of attack performance as the weight of HAO increases. Lower performance stands for better attack results. Details of the setting is given in Appendix H.1
3.2 LLM-based Text-Levee GIAs: Interpretable yet Ineffective
Although ITGIA manages to generate harmful text, interpretability remains challenging to address. To ensure that the generated text is both interpretable and deceptive, a direct approach is to utilize an LLM for text generation and guide it to produce poisoning content through carefully designed graph adversarial attacks, when the features of
prompts. Based on experiences with embedding- injected nodes exhibit heterophily, confusion, or irrelevance, the classification performance of their neighboring nodes is negatively affected [35]. Thus, we attempt to incorporate prior knowledge used in traditional GIAs and create prompts from three different angles:
Heterophily Prompt. This prompt involves sampling the text of nodes from the original dataset generate heterophilic (dissimilar) content. Specifically, we first conduct embedding-leved GIAs to obtain a perturbed graph. Next, we sample the content from the neighbors of injected nodes. Finally, we design a prompt that requests the generated content to be dissimilar to their neighboring nodes, exploiting the weakness of GNNs in handling heterophily.
This prompt generates node text that is entirely unrelated to the original categories of the graph. For example, in the Cora dataset, the original categories about machine learning topics. In this prompt, we generate papers in categories like sports, art, and history, which are dissimilar to all nodes in the original graph.
Random Prompt.
Mixing Prompt
This prompt generates node text that may be classified into multiple or even all categories. Using Cora an example, a potential paper title could be: "Unified Approach for Solving Complex Problems using Integrated Rule Learning, Neural Networks, and Probabilistic Methods".
After defining the prompts, we propose a simple Vanilla Text-level GIA (VTGIA). This method first generates text for the injected nodes based on the prompt via LLMs and subsequently optimizes the graph structure. We use G 106 [20] as the LLM backbone for text generation, and the five sequential injection backbones same as ITGIA. The pseudo-code of VTGIA, prompts, and generated examples are provided in Appendix I.2 H.2 [ The results are shown in Table Although the generated content exhibits adversarial effects, directly producing harmful text via prompts is not sufficiently effective and is far from the optimal results achievable by attacks at the embedding level.
5

---

Table 2: Performance of GCN against VTGIA. Raw text is embedded by GTR before being fed to GCN for evaluation. "Best Emb." refers to the best-perforning embedding-level GIAs that directly update embeddings across various injection strategies.
Datasct
Clean
Prompt
SoqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
Best Emb.
Heterophily
83.35 # 0.49
80.81 0.37
84.23 + 0.80
82.05 = 0.88
83.88 0.83
Cora
87.19 = 0.62
Random
84.65 4 1.11
82.32 = 0.66
85.51 0.81
84.73 0.82
8
31.14 1 0.05
Mixing
83.10 0.80
80.78 a 0.66
83.89 a: 1.32
83.91 a 1.73
84.19 4: 1.21
Heterophily
74.91
73.32 0.39
75.50 0.44
73.73 1.04
74.622
CiteSeer
0.443
Random
73.84
73.28
72.61 + 1.30
0,996
70.81 1.27
21.45 + 0.58
Mixing
75.29 0.67
74.16 0.51
74.61 + 0.71
74.74 1.15
74.87 1.03
Heterophily
80.80 0.83
0.55
75.41 #: 1.22
75.78
82.36
PubMed
87.91 a 0.26
Random
81.99 4: 2.34
78.34 2,08
800.39 4: 2.87
82.26 a: 4.46
86.23 à 0.87
38.32 à 0.00
Mixing
1.91
1.59
78.62 +
80.37 1.99
0.80
4 Word-FFreuueny--aased Text-Level GIAs
Based on previous analysis, to achieve an interpretable and effective GIA, the following conditions should be met: 1) Guidance from Embedding- GIA to ensure effectiveness. Utilization of LLMS to guarantee interpretability. 3) A well-defined LLM task formulation to minimize information loss during the
conversion process.
We address these conditions by employing word-frequency-baate based embeddings, specifically binary Bag-of-Words (BoW) embeddings, which offer several advantages: 1) Clear physical meaning. In binary BoW embeddings, a I' indicates the presence of a word, while a 'O" indicates its absence. 2) Clear task formulation. The task involves generating text containing specified words while excluding prohibited words, which can be effectively done via LLMs. 3) Controllable embedding- embedding process. As long as the text includes specified words and excludes prohibited words, the embeddings can be exactly inverted. Building on these advantages, we design the Word-frequency- based Text-level GIA (WTGIA), leveraging BoW embeddings and the generative capabilities of LLMs. Its implementation consists of three steps: 1) Obtain binary injected embeddings. 2) Formulate embedding to text tasks for LLMs. 3) Perform multi-round corrections for LLMs.
4.1 Row-wise Constrained FGSM: Unnoticeable and Effective at the Embedding Level
Implementaion of Row-wise Constrainted FGSM. To generate binary embeddings, we adopt row-wise constrained Fast Gradient Sign Method (FGSM) algorithm based on [28]. Specifically, we first obtain vocabulary based on BoW on the original dataset. Next, we set the injected embedding Xinj to all zeros. We then set sparsity constraint S on injected embeddings, that each embedding can retain at most F' = [S * FJ non-zero entries, where F is the dimension of features. In each epoch, we define the flippable set as the entries in Xinj that satisfy the sparsity constraint. Based on the element- gradient of the loss concerning Xinj- flip the most significant B entries from the flippable set, where B is the batch size. The process stops when all rows run out of word budgets.
Note that the sparsity budget S a hyper-parameter that limits the use of most F" words from the predefined vocabulary in a single text. Intuitively, higher number of used words intensifies the embedding- attack but can compromise text-level interpretabilityy as becomes harder to integrate more specified words naturally in the generated text. Based on the row-wise constraints, we can formally establish the relationship between embedding- performance and unnoticeability and text- interpretability.
Definition (Single node GIAs towards one-hot embedding). For target node vi with embedding
It 1,0, O] E {0, k < F, the embedding indicates that the node uses k words words are not used. We define the set containing the
from a predefined vocabulary, while specified k words set W., and the set containing the F- k prohibited words set W.. We assume words in W. are related to their corresponding class, whereas words in Wn, are associated with other classes. To attack node Ut, adversarial node Us with embedding F; E {0, 1) F is injected.
We do not consider the co- o--ocuureence constraint in 28 because we find the co- occurrence matrix is dense and ineffective in practice (see Appendix G.3

---

Cora
CiteSecr
PubMed
Porfoemazce
50
Sparsity Budget (%)
Speriity Budget (%)
Sparsity Budget (76)
Best GCN
Aug GCN
Besl Gloed
Aug Guard
Mon Spunity
Spanityy
Figure 3: The Best and Average performance of FGSM against GCN and EGNNGuard among the five injection methods w.r.t increasing sparsity budgets. The results for EGNNGuard reveal that as the budget increases, FGSM attacks can satisfy the similarity constraint while significantly enhancing the attack performance at the embedding level.
Performance: An attack is more effective against node vs if it uses more words wn E Wn- The intuition is that mixing words from other classes makes 24, harder to be correctly classified.
Unnoticeability: An attack is less detectable if the similarity between It and x; is higher.
Interpretability:
An attack is more interpretable fewer words are required to use in I4, which means 34 1 is smaller. The intuition is that generating content with fewer specified words is easier, which fits in practical scenarios involving limited length.
Theorem 1, In the setting outlined in Definition [ assume we apply a cosine similarity constraint with threshold e. E (0, 1) for unnoticeability. Specifically, this constraint requires that the cosine similarity between I4 and z; satisfies Ze-Z;
| > c. Let a denotes the number of words used by Iı
from the set Wu, and b denotes the number of words used by x; from Wn. If the budget is m words at - cv
most to ensure interpretability, then the maximum value of b is max(b) = -max (1
The proof and detailed illustration for intuitions are provided in Appendix Theorem reveals that as long as m 2 k. we can find a positive value for b (disregarding the flooring operation for simplicity), effectively introducing harmful information to the target node. Practically, given the typically low value of c, a destructive v, can be easily identified with relatively large m at the embedding level. In Figure 3 present the average and best performance of Row-wise Constrained FGSM coupled with five sequential injection methods used in ITGIA and VTGIA against GCN and EGNNGuard. As the sparsity budget increases, the performance of EGNNGuard declines sharply, confirming that enhancements in performance are possible under the unnoticeability constraint. However, achieving significant performance degradation requires sacrificing interpretability. For example, to achieve 10% performance drop in Cora and CiteSeer, FGSM attacks need a budget close to or exceed the maximum sparsity level of all texts belonging to the original datasets. The subsequent subsection will explore the consequences of sacrificing text-level interpretability while completing WTGIA.
4.2 Trade-off for Interpretability at the Text Level
Task Formulation of LLMs. After obtaining the binary injected embeddings, we can obtain the Specified words included and the Prohibited words excluded in each text based on the BoW vocabulary. Subsequently, we formulate the task for LLMs as a "word-to-tews generation problem. We request LLMs to generate text under a specified length limit using specified words, optionally within a given topic based on the origin dataset. [ Handling prohibited words is more complex. Given the sparsity of BoW embeddings, the number of prohibited words relatively large. Directly requesting the absence of prohibited words through the prompt would be challenging for LLMs to understand. So, for closed-source LLMs like GPT, we only constrain the specified words in the prompt. For open-source LLMs, we mask their corresponding entries during the output process.
Masking Prohibited Words. Denote f++1 as the next token time i. The standard generative process for LLM can be formulated as: f++1 argmax (SoftMax (logits(t:41 it, (t,t4-1...)))
'The length requirement is necessary because the task would become trivial if the text length is unlimited. However, we usually cannot arbitrarily inject long text into real-worle wollworld applications.
7

---

where logits(f;41 t4.........) are the unnormalized log probabilities of the potential next token given the previous tokens. To enforce constraints on avoiding prohibited words, we introduce masking vector M such that: M[j] = O if token j is prohibited, and otherwise. The genera- tive process then modified by applying this mask to the logits before the SoftMax operation: f++1 argmax MSoftMax (logits (f,+1 4,, .. @ M)), where O represents the element-wise multiplication of the logits by the mask. This mask effectively removes words from outputs by making their probabilities negligible.
Multi-Round Correction. After obtaining the output from the LLM, we calculate the use rate Number of Specified Words used in the generated text and correct any omissions in their usage. Through Number of Specified words
multiple rounds of dialogue, we select the text with the highest use rate as the final output for LLMs.
Main Results. We utilized GPT3.5-turbo 100..--turb--111 1106 106 (20) as the close- source LLM and Llama3-8b 26 as the open-source LLM to implement the WTGIA. Based on Row-wise FGSM in Figure we combine WTGIA with the five injection strategies and report the average results. The specific prompt details are in Appendix H.2 The length limit is specified in Table 6 The variants that use prompts that specify the generated content must belong to the original class set are marked as "-T." The results are displayed in Figure can be seen that
Desirable Performance: WTGIA achieves comparable attack performance to Row-wise FGSM, demonstrating strong embedding-to- capabilities.
Trade-off: Variants with topic specification perform worse than those without, sacrificing attack strength for better text-level unnoticeability.
Masking Helps: Llama-Wa M and GPT that do not mask prohibited words generally perform worse than Llama.
Complete experiments and full results are given in Appendix K and Appendix L Overall, WTGIA effectively replicates the performance FGSM at the embedding level, achieving both effectiveness and interoperability.
Cora
CiteSeer
PubMed
52
Attack Performance (%)
:00
18,66
54-
$2.27
48.48
48-
47.70
47.11
47.09
09 47.35 47,58
49.99 92
50.44
46.67
46.56
45.70
46.78
46
50-
49.00
49.00
44
48-
44
42-
46
10
40
GPT
GPT-T
Lhama
Llama-T
Llamo-Wh
Llama-WM-T
Avg Emb
Figure 4: Performance of WTGIA against GCN. Sparsity budget is the average sparsity of the original dataset. Methods with -T include topic requirements in the prompt. Methods with -WM exclude masks for prohibited words in Llama. Avg Emb. represents the average FGSM attack performance at the embedding level. Lower values indicate better attack performance.
New Trade-offs. However, we fail to further enhance performance the text level by increasing the sparsity budget, similar to FGSM at the embedding level. As shown in Figure with the increasing sparsity budget, the use rate keeps decreasing, representing a trade-off to maintain the interpretability of the generated content. In Cora and CiteSeer, the text-level attacks perform best when sparsity is around 10%. PubMed, GPT-Topic even fails to generate meaningful text with a sparsity budget S > 15%. These observations confirm that in text-level attacks, interpretability represents an additional trade-off that is overlooked in embedding-level attacks. Enhancing or evaluating performance solely at the embedding level fails to provide practical understanding of GLAs.
8

---

Cora without Topic
Citeseer without Topic
Pubmed without Topic
48-
80
50
Performane (N)
46
Rone
84
70
70
46
150
40
12
12
18
12
18
Spansity Budget
Spucsit Budaet
Spuziit Budget
GPT Attack Performancc
Liuna Attack Performance
GPT Use Rate
Liama Use Rate
Figure 5: The performance of WTGIA w/ topic against GCN w.r.t sparsity budget. As the budget increases, the use rate keeps decreasing, and the attack performance increases and then decreases.
New Challenges for Text-level GIAs
5.1 Transferbility to Different Embeddings
In practical scenarios, defenders can employ any text embedding techniques they want to process the text once an attacker completes the injection. However, embedding-lewe attacks implicitly assume that defenders will use the same embeddings as the attackers during evaluation, potentially benefiting the attackers. In fact, it is crucial to fully consider the transferability of a text attack across different text embedding technologies for text-level attacks.
In T we present the results of ITGIA and WTGIA when transferred different embeddings on the Cora dataset. When transferred to GTR embeddings, WTGIA does not exhibit significant performance advantage over ITGIA. The performance of ITGIA when transferred to BoW embeddings is even poorer. Since the injected text is highly uninterpretable and deviates significantly from the original dataset, the BoW method inherently filters out uncommon terms, rendering the injected embeddings particularly sparse and ineffective. These limitations in embedding transferability underscore the research gaps between embedding- and text-level GIAs.
Table 3: Performance of ITGIA and WTGIA-Llama transferred to different embeddings on Cora.
Text-GIA
Embedding
Clean
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
BoW
86.48 = 0.41
84.85 0.76
84.04 + 0.78
85.56 = 0.61
86.49 = 0.50
84,90 = 0.73 68.81 + 1.39
ITGIA
GTR
87.19 = 0.62
66.70 0.94
+ 0.75
71.49 + 1.71
74.63 = 2.48
BoW
86.48 + 0.41
48.32 0.74
51.58 &: 0.78
$2.49 a 1.32
35.33 a 1.29
47.81 + 0.78
WTGIA
GTR
87.19 0.62
78.15 1.70
76.88 + 0.96
79.27 1.24
83.77 1 1.11
77.95 + 1.51
5.2
LLMs as Defender
Since attacks are text-level, the defenders can directly utilize LLMs as predictors for node classifi- cation tasks without necessarily employing GNNs. Taking the CoMssas-Prediccto from B as an example. The method feeds the formulation of the node classification task, texts of target nodes, and optionally, the sampled texts of their neighbors in the prompt. It then utilizes LLMs to predict the labels of target nodes directly. Remarkably, this method demonstrates superior performance on clean datasets. We now examine the performance of WTGIA against L.Ms-as-predictoe from 3 For WTGIA, we select the strongest variant in Figure 4 for each dataset as the attacker. the zero-shot setting, only the text of target nodes is fed to LLMs. In the few-shot setting, the text of labeled nodes with their labels is fed to LLMs as examples. On Cora and CiteSeer, we adopt the whole test set for evaluation. For PubMed, we sample 1000 nodes from the test set for evaluation.
The results are presented in Table Remarkably, on the PubMed dataset, the baseline without the use of neighborhood information ("Clean (w/o Nei)") achieves outstanding performance, which is consistent with observations LLMM-bssed predictors can achieve high accuracy in node classification, meaning defenders can evade the influence of injected nodes. For other datasets, Cora and CiteSeer, while incorporating neighborhood information enhances the performance of LLMs-as- -reeirictoosss the "Clean (w/o Nei.)"
Therefore, even without utilizing any neighborhood information,

---

Table The performance of WTGIA against LLMs-as -iMssas--redictor. The term "(wlo Nei.)" means the exclusion of neighborhood information in the prompt. Methods "Clean (w/o Nei.) and "WTGIA (w Nei.)" can be used as LLLM-based defenders. The best results for defenders are bold.
Zero-shot
Few-shot
Dataset
Clean Nei.)
Clean (w/o Nei.)
WTGIA Nei.)
Clean (w Nei.)
Clean (w/o Nei.)
WTGIA (w Nei.)
Cora
78.64
67.90
74.81
79.51
66.54
72.71
CiteSeer
69.18
59.53
67.71
73.90
66.67
68.44
PubMed
89.80
89,80
89.30
84.50
80.00
80.20
baseline still represents a practical upper limit for the GIAs. Even if attackers manage degrade the performance below this baseline, defenders can turn to this neighbor-frn frghhor-frree method as secure fallback. Consequently, the flexibility of defenders should be fully considered when evaluating GIAs the text level in practical scenarios.
6 Conclusion
In this paper, we explore the design of GIAs the text level. We address the limitations of embedding- level GIAs TAGs by extending them to the text level, which better aligns with real-world scenarios. We present three types of text-level GIA designs: ITGIA, VTGIA, and WTGIA. Our theoretical and empirical analysis reveals trade-off between attack performance and the interpretability of the injected text. Among these methods, WTGIA achieves the best balance between performance and interpretability. Additionally, our findings indicate that defenders effectively counter these attacks using different text embedding techniques or LLM-based predictors, highlighting the complex and challenging nature of graph adversarial attacks in practical applications.
7
Acknowledgement
This research was supported in part by National Natural Science Foundation of China (No. U2241212, No. 61932001), by National Science and Technology Major Project (20227D01 14802), by Beijing Natural Science Foundation (No. 4222028), by Beijing Outstanding Young Scientist Program By Huawei-Renmin University joint program on Information Retrieval. We also wish to acknowledge the support provided by the fund for building world- class universities (disciplines) of Renmin University of China, by Engineering Research Center of Next-Generation Intelligent Search and Recommendation, Ministry of Education, Intelligent Social Governance Interdisciplinary Platform, Major Innovation & Planning Interdisciplinary Platform for Decision-making the "Double-First Class" Initiative, Public Policy and Decision making Research Lab, and Public Computing Cloud, Renmin University of China.
No.
References
MV Balashov, BT Polyak, and AA Tremba. Gradient projection and conditional gradient methods for constrained nonconvex minimization. Numerical Functional Analysis and Optimization, 41(7):822-849, 2020.
[2] Yongqiang Chen, Han Yang. Yonggang Zhang, Kaili Ma, Tongliang Liu, Bo Han, and James Cheng. Un- derstanding and improving graph injection attack by promoting unnoticeability. The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, OpenReview.net, 2022.
[3] Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang. Dawei Yin, Wenqi Fan, Hui Liu, and Jiliang Tang. Exploring the potential of large language models (Ilms) in learning graphs. CoRR, abs/2307. 03393,, 2023.
[4] Junyuan Fang, Haixian Wen, Jiajing Wu, Qi Xuan, Zibin Zheng, and K Tse Chi. Gani: Global attacks on graph neural networks via imperceptible node injections. IEEE Trunsactions on Computational Social Systems, 2024.
[5] Simon Geisler, Tobias Schmidt, Hakan Sirin, Daniel Zügner, Aleksandar Bojchevski, and Stephan Gün- nemann. Robustness of graph neural networks at scale. In Mare" Aurelio Ranzato, Alina Beygelzimer,
10

---

Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6- 14, 2021, virtual, pages 7637-7649, 2021.
[6] C. Lee Giles, Kurt D. Bollacker, and Steve Lawrence. Citeseer: An automatic citation indexing system. In Proceedings of the 3rd ACM International Conference on Digital Libvaries, June 23-26, 1998, Pittsburgh, PA, USA, pages 89-98. ACM, 1998.
[7] Jiayan Guo, Lun Du, and Hengyu Liu. Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking. CoRR, abs/2305. 10666, 2023.
[8] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 1024-1034 2017.
[9] Xiaoxin He, Xavier Bresson, Thomas Laurent, and Bryan Hooi. Explanations as features: Lim-based features for text-attributed graphs. CoRR, 09022305.195233 2023.
[10] Weihua Hu, Matthias Fey, Marinka Zitikk Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Tien Lin, editors, Advances in Neural Information Processing Systems 33; Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
[11] Xuanwen Huang, Kaiqiao Han, Yang Yang. Dezheng Bao. Quanjin Tao, Ziwei Chai, and Qi Zhu. Can GNN be good adapter for Ilms? In Tat- Tat-Seng Chua, Chong- Wah Ngo, Ravi Kumar, Hady W. Lauw, and Roy Ka-Wei Lee, editors, WWW. pages 893-904. ACM, 2024.
[12] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
[13] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017 Conference Track Proceedings. OpenReview.net, 2017.
[14] Yuhan Li. Zhixun Li, Peisong Wang. Jia Li, Xiangguo Sun, Hong Cheng, and Jeffrey Xu Yu. A survey of graph meets large language model: Progress and future directions. CoRR, abs/2311. 1.22311..22999 2023.
[15] Jiawei Liu, Cheng Yang, Zhiyuan Lu, Junze Chen, Yibo Li, Mengmei Zhang, Ting Bai, Yuan Fang, Lichao Sun, Philip S. Yu, and Chuan Shi. Towards graph foundation models: A survey and beyond. CoRR, abs/2310.1 11829, 2023.
[16] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Repre- sentations, ICLR 2018, Vancouver; BC, Canada, April May 3. 2018, Conference Track Proceedings OpenReview.net, 2018
[17] Andrew McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Inf. Retr. 3(2):127-163, 2000.
[18] John X. Morris, Volodymyr Kuleshov, Vitaly Shmatikov, and Alexander M. Rush. Text embeddings reveal (almost) as much as text. Proceedings of the 2023 Conference on Empirical Methods Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 12448-12460. Association for Computational Linguistics, 2023.
[19] Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernández Ábrego, Ma, Vincent 7hao, Yi Luan, Keith B. Hall, Ming-Wei Wei Chang, and Yinfei Yang. Large dual encoders are generalizable retrievers. In Proceedings of the 2022 Conference on Empiricul Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 9844-9855. Association for Computational Linguistics, 2022.
[20] OpenAL. Gpt-3.5-turbo https::/oppenti http://oppai.coooo///o 2023. Accessed via OpenAl API.
[21] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models unsupervised multitask learners. OpenAl blog. 1(8):9, 2019.
11

---

[22] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Kentaro Inui, Jing Jiang, Vincent Ng. and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods Natural Language Prucessing and the 9th International Joint Conference on Natural Language Processing, SMNLP-IJCNLE 2019, Hong Kong. China, November 3-7, 2019, pages Association for Computational Linguistics, 2019.
[23] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lisc Getoor, Brian Gallagher, and Tina Eliassi- Eliassi-Rad. Collective classification in network data, Al Mag. 29(3):93-106, 2008.
[24] Xiangguo Sun, Jiawen Zhang. Xixi Wu, Hong Cheng. Yun Xiong, and Jia Li. Graph prompt learning: A comprehensive survey and beyond. CoRR, abs/2311.16534, 2023.
[25] Yiwei Sun, Suhang Wang, Xianfeng Tang. Tsung-Yu Yu Hsich, and Vasant G. Honavar. Adversarial attacks on graph neural networks via node injections: A hierarchical reinforcement learning approach. In WWW '20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, pages 673-683, ACM / IW3C2, 2020.
[26] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez. Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models.
CoRR, abs/2302.
[27] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
[28] Jihong Wang, Minnan Luo, Fnu Suya, Jundong Li, and Zijiang Yang and== Qinghua Zheng. Scalable attack on graph data by injecting vicious nodes. Data Min. Knowl. Discov., 34(5):1363-133333 2020.
[29] Xiaoyun Wang, Joe Eaton, Cho-Jui Hsich, and Shyhtsun Felix Wu. Attack graph convolutional networks by adding fake nodes. CoRR, abs/1810 10751, 2018.
[30] Lingfei Wu, Peng Cui, Jian Pei, Liang Zhao, and Xiaojie Guo. Graph neural networks: Foundation, frontiers and applications. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2023, Long Beach, CA, USA, August 6-10, 2023, pages 5831-5832. ACM, 2023.
[31] Kaidi Xu, Hongge Chen, Sijia Liu, Pin-Yu Chen, Tsui-Wei Wei Weng. Mingyi Hong, and Xue Lin. Topology attack and defense for graph neural networks: An optimization perspective. In Sarit Kraus, editor, Proceedings of the Twenty-Eight International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 3961-3967. ijeai.org, 2019.
[32] Ruosong Ye, Caiqi Zhang, Runhui Wang. Shuyuan Xu, and Yongleng Zhang, Natural language is all a graph needs. CoRR, abs/2308.07 031344 2023.
[33] Qinkai Zheng, Xu Zou, Yuxiao Dong, Yukuo Cen. Da Yin, Jiarong Xu, Yang Yang, and Jie Tang. Graph robustness benchmark: Benchmarking the adversarial robustness of graph machine learning. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks I, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual, 2021.
[34] Yanping Zheng, Lu Yi, and Zhewei Wei. survey dynamic graph neural networks. Frontiers of Computer Science, 1966)3,,323, 2025.
[35] Jiong Zhu, Junchen Jin, Donald Loveland, Michael T. Schaub, and Danai Koutra. How does heterophily impact the robustness of graph neural networks?: Theoretical connections and practical implications. In Aidong Zhang and Huzefa Rangwala, editors, KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 18, 2022, pages 2637-2647. ACM, 2022.
[36] Xu Zou, Qinkai Zheng, Yuxiao Dong, Xinyu Guan, Evgeny Kharlamov, Jialiang Lu, and Tang. TDGIA: effective injection attacks on graph neural networks. In KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021, pages 2461-2474
ACM, 2021
[37] Daniel Zügner, Amir Akharnejad, and Stephan Günnemann. Adversarial attacks on neural networks for graph data. In Yike Guo and Faisal Farooq, editors, Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018, pages 2847-2856 ACM, 2018
[38] Daniel Zügner and Stephan Günnemann. Adversarial attacks on graph neural networks via meta learning. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
12

---

A Broader Impacts
In this paper, we potentially aid in developing more advanced and sophisticated attacks through understanding text-level vulnerabilities and providing significant positive impacts by enhancing the design robust defense mechanisms. By pioneering the exploration of text-level GIAs, we enable the creation of more effective countermeasures to protect against these advanced threats, thereby improving the overall security and resilience of systems that rely on GNNs. This dual impact highlights the importance of our work in driving both offensive and defensive advancements, ultimately contributing to safer and more secure AI applications.
B Open Access to Data and Code
GPT-turbo: https://openai: com/| In this paper, the GPT-3.5-turbo model provided by OpenAl for generating textual data. The model was accessed through OpenAl's API services, and all uses were in compliance with the terms of service provided by OpenAl.
Llama3: pestps://githan
this paper, we
use Meta Llama 3, which is governed by the Meta Ilama 3 Community License Agreement (Meta
Platforms, Inc., released April 18, 2024). This tool is employed strictly in accordance with the terms set forth in the licensing agreement and Meta's Acceptable Use Policy.
Raw Data and LLMssas-Predictors: cense).
(MIT Li-
GIA- http::///ggtttuu..oo///aaaaaaaaa
(MIT License)
C Device Information
All our experiments are conducted on machine with an NVIDIA A100-SXM4 (80GB memory), Intel Xcon CPU (2.30 GHz), and 512GB of RAM.
D More Related Works
Graph Injection Attacks. Graph Injection Attacks aim decrease the performance of GNNs by injecting suspicious nodes into the original graph. The newly injected nodes are allowed to process fake features and form connections with existing nodes on the graph. Wang et al. (29 first attempt to inject fake nodes to degrade the performance of GNNs. NIPA (25) applies reinforcement learning to make graph injection decisions. AFGSM 28 uses an approximation strategy to linear the model and solve the objective function efficiently. TDGIA [36] selects edges for newly added edges based on a topological defective edge selection strategy. It then generates node features by optimizing proposed feature smooth objective function. Chen et al. [ add regularization term to the existing objective function for GIA to improve homophily unnoticeability. GANI 4 employs cattistics-based approach to calculate node features, ensuring the generated features remain unnoticeable but leading to a significant decline in performance. All the GIAs mentioned are designed the embedding level. Despite some constraints being applied to the injected features, the semantic coherence and logical consistency of generated features at the text level remain unexplored.
Graph Modification Attacks. Graph Modification Attacks aim to decrease the performance of GNNs by modifying the graph structure or node features of the original graphs. Nettack [37] adopts a greedy strategy to select edge and node feature perturbations with unnoticeability constraints. Zügger et al. 38) treat the graph structure as a hyperparameter and use meta- meta-gradients to solve the bilevel poisoning attack objective. Xu al. [3 use Projected Gradient Descent (PGD) to find the best perturbation matrix. The above algorithms significantly degrade the performance of GNNs, yet all require (N2) space complexity. To address the scalability issues, Geisler et al. [5] propose PRBCD and GRBCD using Randomized Block Coordinate Descent. The algorithms are of O(EE) time and space complexity and be scaled up datasets containing more than million nodes.
Raw Text learning on TAGs. With the development of L.LMs, works combining them with GNNs have continuously emerged, showing outstanding performance in various graph-related tasks
13

---

The budget of GIAs is summarized in TAGs. TAPE [9 uses LLMs feature enhancer to obtain high-quality node features. In Instruct- GLM [2], the authors propose scalable prompts based on natural language instructions that describe the topological structure of a graph. Guo et al. 7 analyze the ability of LLMSs to understand graph data and solve graph-related tasks. Chen al.
3 conduct a benchmark evaluating the performance of L.MMs as the feature enhancer and the predictor, respectively. More related works about L.Mss in Graph could be referred [5] [2] [4]. Although LLMs have seen numerous applications in graph tasks, works exploring the robustness of GNNs are still sparse.
Although Graph Modification Attacks are powerful attacks, recent works point out that they do align well with real-world attack scenarios [33], and they are theoretically inferior to GIAs in attack performance [2]. Moreover, while raw texts are proven be helpful improving the performance of graph tasks, works exploring the robustness of GNNs at the text level are still sparse.
E Datatset Statistics and Budgets of Attacks
Dataset information is summarized in Table 5 and Table 6
Table
Table Statistics datasets. Edges (UnD.) stands for number of edges after transforming each dataset into undirected. (Max) Sparsity stands for the average (max) percentage of non-zero elements in BoW embeddings of raw texts.
Dataset
Nodes
Edges (UnD.)
Classes
Avg. Degree
Avg. Sparsity
Max Sparsity
Cora
2,708
10,556
3.90
0.06
0.17
CiteSeer
3,186
8,450
6
2.65
0.06
0.13
PubMed
19,717
88,648
3
4.50
0.10
0.23
ogbn-arxiv 169,343
2,332,486
40
13.77
0,06
0.33
Reddit 33,434
396,896
2
11.87
0.04
0.44
Table 6: The statistics reflect the number of words used in each dataset. " "Avg." represents the average word count per dataset, while "Std." indicates the standard deviation word counts. Assuming normal distribution of word lengths, "Avg. + 2Std.. denotes the upper bound of the 95% confidence interval. This metric is employed benchmark for establishing maximum word/token limits in paper generation experiments.
Dataset
Min
Max
Avg.
Std.
Avg. + 2Std.
Cora
2
757
132.59
83.60
299.79
CiteSeer
4
397
150.85
45.58
242.01
PubMed
889
241.18
71.68
384.54
ogbn-arxiv
15
1,410
170.36
57.24
284.84
Reddit
3,708
134.34
165.61
465.56
Table 7: Attack budgets on the datasets.
Dataset
Nodes
Degree
Node Per.(%)
Edge Per. (%)
Cora
60
20
2.22
22,74
CiteSeer
90
10
2.82
21.30
PubMed
400
25
2.03
22.56
ogbn-arxiv
1500
100
0.89
12.86
Reddit
500
30
1.50
7.56
F Further Illustration and Proof to Theorem
The formulation for Theorem Drelie on the below intuitions.
14

---

Performance: An attack is more effective against node if it uses more words un Wn.
Unnoticeability: An attack is less detectable if the similarity between and is higher.
Interpretability: An attack is more interpretable if fewer words are required to use, where the Zy 1 is smaller.
In the intuition behind performance, actually believe that words beneficial the classification of the target node are located in W.. while words detrimental to the classification of the target node located in n. However, strictly speaking, all words in W,, have a negative impact on the classification of the target node. Additionally, when a certain amount of classification words is introduced, the negative impact becomes marginal and does consistently increase. Therefore, using more words W, E Wn does not linearly correspond to higher performance.
However, this does not affect the insight provided by Theorem I Since k < F, the continuous increase in the usage of Wn. E Wn within certain range is sigmificant. Since we do not consider situations where > k, we indeed focus "uses more words w,, Wa in certain range" intrinsically. Thus, under this premise, the theorem's 's description of performance remains applicable.
Proof. The problem is formulated as:
b s.t. a a+bs m,
(4)
(5)
v Vkva+1 1b 26
(6)
From inequality (6. we obtain:
a² 62 ka 2kbb 20,
which leads to:
e ck + /k(4b+
c
ck c(ck
v 1 c2t))
a
or a <
<
2
2
Since a is positive, the only feasible solution is:
c ck + 1k(4+++
2
By substituting this into equation
5 obtain:
(ck + kk(4b= k2k)
S m a S m
Solving the resulting inequality with respect to we find:
b < cYin.
Hence, the solution to the maximization problem is: max(b) |m cVmk)]
G Collections of Illustrations
G.1 Illustration of Interpretable Regions
The illustration of interpretable regions is provided in Figure 6
15

---

Feasible region (1)
Feasible region (2)
Interpretable region
Figure 6: Illustration of the interpretable regions. Left: Feasible solution under constraints (3)
Middle: The theoretically feasible set of PLM embeddings. Right: Subset of embeddings corre- sponding interpretable texts, representing narrower selection within the feasible sets.
G.2 Alternative for PGD ITGIA
In addition to the methods mentioned in the main body of the paper, we also considered using the Riemannian Gradient Descent as way obtain the embedding for ITGIA. This method performs gradient descent in the tangent space of the sphere to ensure the resulting embedding's 's normality.
Formally, solve the optimization problem
min f(z)
x
(7)
subject to || =
using Riemannian gradient descent, we follow these steps:
Initialization: Start with an initial point such that || = 1.
Gradient Calculation: At each iteration k compute the Fuclidean gradient ()(x)).
Projection onto Tangent Space: Project the Euclidean gradient onto the tangent space of the sphere at Zg:
VRf(0s) vf(xx) - (Z)(
(8)
where (RF(Fx) is the Riemannian gradient.
Update Step: Move in the direction of the Riemannian gradient and re-project onto the sphere:
Tk+1 = Projg og V (xx)),)
where 0g is the step size and Projs (v) Tyll
y ensures that the updated point lies on the sphere.
Iteration: Repeat steps 2-4 until convergence.
The PGD method adopted can be regarded as the variant of Riemannian Gradient Descent without the "Projection onto Tangent Space" step. Although the Riemannian Gradient Descent has better theoretical properties (. since we did not observe significant advantages over PGD in our experiments, we opted for the simplicity using PGD, as discussed main body of our paper.
G.3 Illustration of Co-occurrence Constraint
The co-occurrence constraint in (28) requires that columns that do not appear together in the original dataset must not appear together in the injected embeddings either.
Let X binary feature matrix with dimensions N x F, where N represents the number of samples and F represents the number of features. Each element æi in the matrix X is defined as:
{1
if word j is present in texti,
otherwise.
The co-occurrence matrix C is then an F matrix where each element Cij, representing the pair of words i and j. is computed as follows:
16

---

Table 8: The sparsity of co-occurrence matrix.
Dataset
Cora
CiteSeer
PubMed
Sparsity (%)
96.99
98.21
99.97
C (X >0
Here, X is the transpose of X, and the multiplication X results in matrix where each element bg is the sum of the products of corresponding elements of X:
Cj
Tkikkj
The value Cij represents the number of texts where both words i and j are present. The physical meaning of Cj > 1 is that features and j co-occur in at least one text, indicating a possible association dependency between them within the dataset. A co-occurence--costtriinne FGSM only allows flips that satisfy if word and word j appear the injected embeddings, they must have Cij
We examine the sparsity of in Table We can see that almost every two words have co-occurred before, making the constraints meaningless.
H Experiment Details
H.1 Setup and Hyperparameters
Attacks. For the injected embedding features, use PGD 33 2
with learning rate of 0.01, and the training epoch is 500. An early stop of 100 epochs according to the accuracy of the surrogate model on the target nodes. For TDGIA and ATDGIA, we use the method of [36] to generate injected embeddings. The weights L and k2 are set 0.9 and 0.1 by default. The sequential step set as 1.0 for MetaGIA to save computational costs, while 0.20 for methods. For Binary FGSM, the batch size is set to as B in Cora and CiteSeer, and B = 50 for PubMed. For the hyperparameters of SeqGIA, MetaGIA, and SeqAGIA, directly follow the setting in 22. A simplified description of the Sequential injection framework is provided in Appendix | and the FGSM function is provided in Appendix 2
Harmonious Adversarial Objective.
The HAO constraint can be formulated as:
C= Lpred 7L AO
Here, Lyred represents the cross-entropy loss on target nodes, calculated between the current predic- tions of the surrogate model and the original predictions made on the clean dataset by the same model. The loss term LHAO = sim (Xinj. (X' A' Jinj), where sim returns the cosine similarity between injected node features and propagated injected node features. We set y = 10 in SeqGIA, SeqAGIA and MetaGIA, a balance the attack performance and unnoticeability to escape from homophily-based defender.
1 in TDGIA, ATDGIA and set le 6 in FSGM, where we empirically find
Note that in Figure 2 y is multiplyed by the weights as the final coefficient. Denote 7' = * w,
where w is the weight listed in the x-axis of the figure, we apply L Cpred + 7 CHO as the final loss function.
Defense. For both GCN and EGNNGuard, we set the number of layers as and use hidden dimension of 64. We set the dropout rate as 0.5 and adopt Adam optimizer [12] with a learning rate of 0.01. The number of training epochs is 400. We adopt the early stop strategy with patience of 100 epochs. The threshold is set as 0.1 following (2). which we find powerful enough in detecting noticeable injections.
Others. The temperature of the GPT-turbo is set as 0.7. ITGIA, the beam-width is set as 2. In WTGIA, the max-token for gpt-turbo and Llama3 is set to 500 for Cora and CiteSeer and 550 for
17

---

PubMed. We set the the number of dialogs in multi-round corrections as 3. We set the max-token to avoid over-length generation. Based on the typical ratio of 0.75:1 between the number of words and the number of tokens, we proportionally scale this relationship using the data from Table 6 obtain the specific numbers for this limitation.
H.2 Prompt Design
Table 9 provides our prompt designs for different prompt strategies of vanilla text-based GIAs. The comprised of paper id, title, and abstract of adjacent/example nodes, and denotes fillable blanks for the model to fill in. The <LMIN> and <L.MA>> correspond to the length limit for each dataset specified in T Generally, experiments find that the current instructions allow the LLM to produce output that conforms well to the expected format without significant deviations.
<adi/example__agge catplipee__ppp_ootttettt
Table 9: Prompts of vanilla text-based GIAs. Number of positive examples are set to 5, and number of negative examples are set to as
Attack
Prompts
Task: Paper Generation.
There are <num <num_
classes>> types of paper, which are names>>>
Positive Examples of the papers are:
Content: <example Satplepppppe____ lalpppaapppteeeeee
content> Type: <cur labels:
Negative Examples of the papers are:
Vanilla Heterophily
Content: <example
Generate title and an abstract for paper <inj_ which is dissimilar to the negative examples, but belongs to least one of the types in <category names> similar to positive examples.
Length limit: Min: <LMIN> words, Max: <LMAX> words.
Title: Abstract:
Task: Paper Generation.
There are <num_ _ettssselssssseae types of paper, which are cantegory_namesse Examples of the papers are:
Content: Generate a title and an abstract for paper <inj_ sinj_node> that belong all the above paper types.
Vanilla Mixing
The generated content should be able to be classified into any type of paper.
Length limit:
Min: <LMIN> words, Max: <LMAX> words.
Title: .... Abstract:
Task: Paper Generation.
There are <num classes> types of paper, which are <category names>. Generate title and an abstract for paper <inj_ mnj_nooe>> that does not belong any of the paper types.
Vanilla Random
For example, the paper could belong to ('history', 'art', 'philosophyy 'sports', "music"] ] and types like that.
Length limit: Min: <LMIN> words, Max: <LMAX> words. Title: ... Abstract:
Table 10 provides prompts of WTGIA with different generation type.
I Collections of Pseudo-code
I.1 Pseudo-code for Sequential GIAS
The pseudo-code for sequential node injection strategy is displayed in Algorithm 1 A simple illustration of the feature update function of SeqGIA, MetaGIA, TDGIA, and SeqAGiA is given in Algorithm 3 The pseudo-code for Binary FGSM feature update functions is displayed in Algorithm 2 For FGSM binary attacks, we implement the structure updates in the sequential node injection manner, as it is shown more powerful by previous works 28 The initialization of Xinj is set as X
18

---

Table 10: Prompts of vanilla text-based GIAs.
Generation Type
Prompts
Generate title and an abstract for an academic article.
Ensure the generated content explicitly contains the following words:
<use_wordee words>.
These words should appear as specified, without using synonyms,
LIama/GPT
plural forms, other variants. Length limit: <max_ <max_words> words. Output the TITLE and ABSTRACT without explanation. TITLE:...
ABSTRACT...
There <num classes> types of paper, which <category names>. Generate a title and an abstract for paper belongs to one of the given categories.
Ensure the generated content explicitly contains the following words: juse_words>>
Llama/GPT-topic
These words should appear as specified, without using synonyms, plural forms, or other variants. Length limit: fmax_wordss words. Output the TITLE and ABSTRACT without explanation. TITLE:..
ABSTRACT...
and applying feature update function as Algorithm 2 For the details about the structure update function H. of SeqGIA, MetaGIA, TDGIA, ATDGIA, AGIA, and the feature update function of TDGIA and ATDGIA, please refer to 2 36
Note that ITGIA, if normalized, after each single epoch feature update, we project the output Xinj unit sphere. A text illustration of PGD be found in G.2 where alternative vanilla PGD is discussed.
Algorithm 1: Sequantial Attack Framework
Input
:Graph G = (V, E, X), Adjacency matrix A, Number of features F, Number of injected nodes Mijj Number of target nodes |rr. Surrogate model f. Feature update function F, Structure update function H: (Function specific arguments excluded)
Output
:Attacked Graph g' -V","' X')
Parameter: :Sqquential step st, Normalization flag Norm;
V + E' + - <
2 ning +0
Y Yorig =((X,A) A)
while Ning Nug do
NNoull (Nm s(])
6
Initialize Vinj. Ein, Xini
// Initialization of the current epoch
V x- V' J Vijji El + U
U
8
Vinji Enj t- H(X,X
Yorig)
// Update Structure
V - D' Vini: El t E' Emj
10
X +- F(X,
Xinjs V" E' Yorig, Norm)
// Update Features
X X'=X'X
12
Tinj +o Ting +
13 return Attacked Graph g' (V',''X' X')
19

---

Algorithm 2: Feature Update for Continuous Features
Input
:Model J, features X, attack features Xnnj adjacency matrix A', original labels
Y Yorig. target indices T, epochs e, learning rate
Output :Updated attack features Xnj
for epoch 0 to do
2 Xeat = Xinj
3 Xijj +- Proj [Xinj VV Leh (f(Xeat, A'), Yorig)
// Project to unit sphere
for ITGIA
return Xinj
Algorithm 3: Fast Gradient Sign Method (FGSM) Update for Binary Features
Input
:Model features X. attack features Xinj. adjacency matrix A', original labels Yorig- target indices T, sparsity budget S, batch size Output :Updated attack features Xinj
F t X.shape[1]:: F' < -S.
// Allowed flips per node.
while any row flips F" do
3
Xeat + XXUX
4
+ S(Xan, A')
VL Vx.., Yorg.
Determine flip mask M (Current flips per < F*)
7 Gealid + VC-M.
// Mask gradients by rows that can still flip Compute flip direction Gup < sign(Gvvlial (Xinj =)
Select indices of top B gradients 1 + Top-K indices from Gtipp
foreach i do
r. < Row and column of index i if Row r can still flip then
13
Flip Xinj [r. c) + 1 - Xing (r, cl Update the flip counter for row
14
return Xinj
1.2
Pseudo-code for Vanilla Text-Level GIAS
The pseudo-code for vanilla text-level GIAs is displayed in Algorithm 4 which is implemented in sequential manner following SeqGIA. Note that the feature is not updated within the "while loop" since gradient information is infeasible.
J Interpretability of Generated Texts
J.1 Perplexity and Use Rate
We calculate perplexity based on GPT2 21 following https://huggingface co/doca/ aransformmrre/ppppleiiite
Note that due to the model capacity constraint, output IT- GIA is truncated to 32 tokens. So, we truncate all texts into 32 tokens before calculating Perplexity to make them comparable to each other, We present the results of average perplexity, and use rate in WTGIA in Table 11 Table 12 Table 13 and Table 14
In the experiments, VTGIA consistently achieves the lowest perplexity scores, indicating superior performance, followed by the WTGIA model. Conversely, ITGIA exhibits the highest perplexity, rendering it the least interpretable. Notably, even the variant incorporating -HAO fails enhance interpretability corrsponding to normal texts. Furthermore, our findings suggest positive correlation between model use rate and perplexity. Specifically, lower use rate corresponds to reduced perplexity, thereby enhancing interpretability. This observation aligns with the hypothesis that as the use rate decreases, I.LMs preferentially select words from specified set that are more interpretable, rather than striving to utilize all specified words.
20

---

Algorithm 4: Text to Graph Attack
Input
:Graph G = (V. & {s,)),, Class set C, Number of injected nodes Ninj Number of target nodes [
Output :Attacked Graph g (V', 8" {s{]
: :Sequential step st. prompt type p
V t V; +
p Heterophily prompt then
/* Integrate attacked graph into prompts */
V. ". Embedding based GIAs
{sj <- Generate (Ninj G¹, S', V,)
else
{s, hinj t GenerateTex((NNg. 7 {.s: 1' = {s:} U {s; haj
X Tex(Emeedding((s))
Nootal 0
/* Graph structure refinement */ while Nona < Ning do
10
11
n 4- min (Nij - NToal: [Nini st)
V. E.E'= + Adaptiveljeccton(] , X')
13
NTaal + NToal + n
15 return Attacked Graph G' (V,E', (8:))
Table 11: Average perplexity of raw text generated by VTGIA and ITGIA. Clean refers to the average perplexity of original dataset.
Dataset
Clean
VeGIA-Het.
VTGIA-Rand.
VTGIA-Mix.
ITGIA
ITGIA-HAO
Cora
110.47
14.02
18.12
16.63
623.65
546.89
CiteSeer
66.71
14.37
16.53
21.21
705,41
379.80
PubMed
30.85
8.21
16.76
13.52
503.14
348.07
Table 12: Average perplexity (L) and use rate of raw texts generated by WTGIA w.r.t sparsity budget
on Cora dataset.
WTGIA Variant
Avg.
0.10
0.15
0.20
GPT Perplexity GPT Use Rate (%)
53.88
43.11
39.08
35.60
84.29
80.84
66.48
52.72
GPT-Topic Perplexity
30.70
26.92
26.40
25.01
GPT-Topic Use Rate (%)
81.78
73.93
58.85
44.81
Llama Perplexity
90.23
75.95
58.17
54.73
Llama Use Rate (%)
93.43
86.71
54.60
51.29
Hama-Topic Perplexity
83.21
65.97
54.60
55.69
Llama-Topie Use Rate (%)
93.08
86.03
71.56
50.67
21

---

Table 13: Average perplexity (L) and use rate of raw texts generated by WTGIA wir. sparsity budget on CiteSeer dataset.
WTGIA Variant
Avg.
0.10
0.15
0.20
GPT Perplexity
60.55
51.30
43.28
38.09
GPT Use Rate (%)
87.54
79.79
65.27
53.20
GPT-Topic Perplexity
38.34
36.39
30.59
38.34
GPT-Topic Use Rate (%)
83.11
72.91
55.98
45.57
Llama Perplexity
86.11
67.57
51.89
48.79
Llama Use Rate (%)
96.69
87.72
59.75
59.73
Llama-Topic Perplexity
83.52 96.58
65.02
44.23
51.06
Llama-Topic Use Rate (%)
88.18
73.60
54.45
Table 14: Average perplexity (4) and use rate of raw texts generated by WTGIA w.r.t sparsity budget on PubMed dataset.
WTGIA Variant
Avg
0.15
0.20
GPT Perplexity
45.57
42.65
78.08
GPT Use Rate (%)
73.06
58.90
40.83
GPT-Topic Perplexity
30.78
GPT-Topic Use Rate (%)
59.72
Llama Perplexity
55.89
43.84
40.24
Llama Use Rate (%)
82.23
72.60
61.97
Llama- Llama-Topic Perplexity
58.63
44.22
40.02
Llama-Topy Topic Use Rate (%)
82.33
72.21
62.20
J.2 Examples of VTGIA
We show the generated examples on the Cora dataset for VTGIA, one for each prompt type.
Reinforcement Learning Techniques for Rule Learning Optimization Abstract: This paper explores the application of reinforcement learning techniques optimize the process of rule learning. Traditional rule-learning algorithms often struggle with complex datasets and high-diimensiona feature spaces, leading to suboptimal rule sets. By leveraging the power of reinforcement learning, we propose a novel approach that dynamically adapts the rule learning process based on feedback from the environment. We start by formulating the rule learning problem Markov decision process, where the agent's actions are the selection and refinement of rules based on the input data. We introduce reward function that incentivizes the discovery of rules with high predictive power while maintaining simplicity and interpretability. Through series of experiments on benchmark datasets, we demonstrate the effectiveness of our approach in outperforming traditional rule learning methods. Furthermore, we investigate the convergence properties of our reinforcement learning algorithm and provide theoretical bounds on its performance. Our results show that by combining reinforcement learning with rule learning, we can achieve more accurate and concise rule sets, leading to improved generalization performance Overall, this paper contributes to the intersection of reinforcement learning and rule learning, opening up new possibilities for optimizing rule-based systems in various domains.
Heterophily Prompt Example. Title:
unseen data.
Random Prompt Example. Title: Exploring the Influence of Martial Arts on Japanese History Abstract: This paper delves into the deep-rooted connection between martial arts and the historical landscape of Japan. By tracing the evolution of various martial art forms such as Judo, Kendo, and Karate, we analyze how these disciplines have not only shaped the physical combat techniques but also contributed significantly to the cultural, philosophical, and political development Japan. Through studying the impact of samurai warriors, martial arts schools (ryu), and renowned martial artists throughout different historical periods, this paper uncovers the intricate interplay between martial arts and Japan's 's socictal norms, values, and power dynamics. Furthermore, by exploring the
22

---

ways in which martial arts have been depicted in art, literature, and popular culture over the centuries, we gain a deeper understanding of how these practices have transcended mere combat techniques to embody spiritual, ethical, and aesthetic principles. Ultimately, this paper aims to shed light on the enduring legacy of martial arts in Japanese history and the profound influence they continue to exert on various aspects of Japanese society today.
Unified Approaches in Machine Learning and Control Systems Abstract: This paper presents novel approach to machine learning and control systems by integrating various techniques such as probabilistic methods, neural networks, case-based reasoning, genetic algorithms, theory, reinforcement learning, and rule learning. The proposed framework leverages the strengths of each approach to address complex problems in a unified manner. It introduces new algorithm, contextual machine learning, which incorporates adaptative history-sensitive models for source separation, making use of the temporal structure of input data. Furthermore, the paper explores the application of non-linear systems in robust control and the use of functional programming by analogy in the domain of program synthesis. Additionally, presents a learning algorithm for Disjunctive Normal Form (DNF) under the uniform distribution, with implications for the case when certain parameters are constant. The paper also discusses the use of Gaussian noise in sigmoidal belief networks and the training of such networks using slice sampling for inference and learning. Finally, it showcases the application of radial basis function networks in predicting power system security margin. Overall, this paper provides comprehensive overview of the diverse applications and advancements in machine learning and control systems, demonstrating the potential for creating unified approaches that transcend traditional boundaries.
Mixing Prompt Example. Title:
J.3 Examples of ITGIA
We show the generated examples on the Cora dataset for ITGIA in this subsection. The output is highly uninterpretable due to ill-defined interpretable regions for injected embeddings.
ITGIA-TDGIA without HAO, Cos: 0.136
of the following: MC Dallas' NSA sign, and some of the relays of the past few years: Trumpet Jam's sinus
the liner notes of The MC6's "Desirty Pigs": the relayed that some of the plaque
liner notes of The MC's "The Lion's Pie" album: some relayed that the 'bumpy' fluid
the following: One of the croons of the "Petsy on the MC12" relayed: Demitab
ITGIA-TDGIA with HAO, Cos: 0.742
"Sympteric" Alignment. This is one of the best examples of how to achieve high level information retrieval algorithms using proportion
"Asymptotically" by Grant. This is a key method for constructing local information repre- sentation systems that support multicluster connections
Emilia. The "Proplective information with no associativity all" NNNN algorithms are based on Monte simple scenarios
J.4 WTGIA
We show the generated examples on the Cora dataset with sparsity budget set "average" for WTGIA in the subsection. We choose the FGSM-TDGIA for embedding generation. We display one example for each generation type.
GPT. Title:
Hicrarchical Recurrent Neural Networks for Visual Sequence Identification and Dimen- sional Improvement
this article, we demonstrate the idea of using hierarchical recurrent neural networks for visual sequence identification, resulting in dimensional improvement. We show that this approach requires the underlying mechanisms of backpropagation through time to obtain a more powerful representation of visual sequences. Additionally, we identify the nonlinear nature of the sequence identification problem and how the use of recurrent neural networks leads to nonlinear improvement in identifying visual sequences. Furthermore, we present a novel approach to dealing with hierarchical
Abstract:
23

---

structures in visual sequences, utilizing the power of recurrent neural networks to identify objects in a hierarchical and sample-efficient manner. This work contributes the science of visual sequence processing and demonstrates the capability of recurrent neural networks to handle complex visual sequences, such as trees, lines, and vector objects.
GPT-Topic. Title: Hierarchical Neural Networks for Visual Sequence Identification: Exploring the Power of Recurrent Mechanisms in Computer Science
Abstract: This paper presents an investigation into the use of hierarchical neural networks for visual sequence identification in computer science. The underlying idea is to demonstrate the power of recurrent mechanisms in identifying dimensional visual sequences. The resulting improvement in identification accuracy leads to the direct application of backpropagation mechanisms for obtaining nonlinear vector representations of visual sequences. In addition, the paper explores the application of hierarchical neural networks in identifying hierarchical tree structures within visual sequences. The research also requires the exploration of probabilistic methods for identifying sample sequences and tree structures within visual sequences. This work contributes the advancement of science in computer vision and object recognition by exploring the hierarchical structure of sequences.
Llama. Title: Harnessing the Power of Hierarchical Neural Trees for Dimensionally- Transsforrdddeee Sequence Identification and Improvement through Addition and Recurrent Back- Propagation
Abstract: This article proposes groundbreaking idea that combines the strengths of hierarchical neu- ral trees and recurrent prok-propagation to identify and classify nonlinear sequences. By leveraging the underlying mechanisms of neural trees and incorporating the addition of recurrent feedback, we demonstrated remarkable improvement over conventional sequence identification methodologies through the power of prokapropagttton. Our methodology requires the creation of a rector-based object that represents the sequence, which is then fed into a hierarchical neural line that utilizes recurrent back-propagation to identify the underlying dimensional transformations. This leads to resulting sample of sequences that can be analyzed and visualized to obtain a deeper comprehension of the object's underlying mechanisms. To validate our idea, conducted comprehensive se- quence identification experiment, where we utilized a sample of sequences to train and fine-tune our hierarchical neural trees. Our resulting sequences demonstrated remarkable capability to identify and classify nonlinear sequences, out-performing other leading methodologies, and providing visual insight into the underlying mechanisms. This breakthrough has the power revolutionize the science of sequence identification and has far-reaching implications for fields such as signal-processin image-recogaition, and predictive analytics.
Llama-Topic Topic. Title: Hierarchical Recurrent Neural Trees for Dimensionally Reduced Sequence Identification and Power System Monitoring
Abstract: This article demonstrates the effectiveness of hierarchical recurrent neural trees (HRNT) for identifying sequences of nonlinear phenomena occurring within power grids. By leveraging the power of poeckpropagation through trees, HRNT is capable of capturing underlying mechanisms governing the system's 's nonlinear interactions. Our HRNTT-based sequence identification idea requires a hierarchical arrangement of recurrent neural trees, which are then utilized to identify sequences of interest along a dimensional line of power transmission. Notably, our methodology leads to a resulting improvement of up to [X]% over conventional sequence identification schemes, which has been demonstrated through sample dataset of power system measurements. Furthermore, our HRNT-bbsed system can effectively identify nonlinear phenomena, such as addition of harmonics, and nonlinear interactions between power system objects. leveraging the power of vector calculus to obtain a comprehensive picture. Our visualizations of the resulting sequences and underlying mechanisms have been found to be particularly insightful for power system scientists, contributing to the advancement of the science and providing a valuable addition to the scientific community.
K More Experiments about WTGIA and ITGIA
K.1 Transfer WTGIA to More Datasets
In Tables 15 and Table 16 we included the ogbn-arxiv dataset 110 containing over 160,000 nodes, and a social network dataset, Reddit []] with more than 30,000 nodes, to evaluate the scalability and generalization of text-level GLAs. We injected 1,500 nodes into ogbn-arxiv, following the budget
24

---

specified in (2], and proportionally injected 500 nodes into Reddit. On both datasets. WTGIA maintains use rates above 90% and shows good attack performance when transferred GTR.
Table 15: Performance of WTGIA with the average sparsity on the ogbn-arxiv dataset. Clean accuracy for BoW embedding is 71.87 0.11, and for GTR embedding is 72.36 0.13.
LLM Type
Emb.
Random
SeqGIA
TDGIA
ATDGIA
AGIA
Emb.
BoW
70.25 = 0.25
63.00 + 3.44
60.56 + 2.69
64.33 # 3.18
53.34 = 1.75
BoW
70.19 = 0.24
63.65 + 2.99
60.96 # 2.33
64.53 + 2.78
55,38 = 1.59
GPT
GTR
68.44 = 0.56
67.47 0.90
64.95 1.18
67.24 = 0.99
65.90 0.82
BoW
70.03 = 0.29
65.11 2.53
62.47 2.10
65.89 2.39
56.69 1.45
GPT-Topic
GTR
68.25 + 0.56
67.50 + 0.88
64.97 # 1.19
67.35 + 0.96
65.70 + 0.80
BoW
70.25 + 0.24
63.61 + 3.33
61.09 = 2.70
64.67 = 3.15
53,95 + 1.74
Llama
GTR
68.64 0.59
67.38 0.91
64.72 1.22
67.28 + 1.06
65.12 0.97
BoW
70.25 0.24
63.67 3.32
61.09 2.69
64.65 3.14
53.98 1.73
Llama-Topic
GTR
68.65 = 0.58
67.24 0.89
64.42 = 1.23
67.18 + 1.05
65.45 = 0.94
Table 16: Performance of WTGIA with the average sparsity on the Reddit dataset. Clean accuracy for BoW embedding is 62.84 0.89, and for GTR embedding is 68.28 + 0.24.
LLM Type
Emb.
Random
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
Emb.
BoW
59.48 # 1.36
55.34 # 1,25
54.48 + 0.96
50.72 # 0.78
54.65 * 1,38
55.80 # 1.12
BoW
59.84 1.34
56.48 1.47
55.49 1.21
52.03 1.29
55.92 1.75
56.66 1.29
GPT
GTR
66.94 0.47
66.97 0.47
67.13 0.52
67.77 0.52
67.01 0.52
66.87 0.47
BoW
59.51 + 1.35
55.48 + 1.28
54.69 + 1.06
50.86 + 0.87
54.71 + 1.41
55.89 = 1.16
Llama
GTR
67.39 = 0.56
66.77 = 0.49
66.34 + 0.66
67.02 = 0.81
66.86 + 0.61
66.44 + 0.51
K.2
Against More Defense Models
Table 19 Table 20 Table 21 and Table 22] we include classic methods like GAT (27] and GraphSAGE 8 as well as the EGNNGuard and Layernorm (LN) methods, which
In Table 17 Table 18
have been proven highly effective in
discover interesting new results, such as LN's effectiveness on text-level GIAs not being as strong as previously reported for embedding-level attacks, especially for WTGIA. This is because embedding-lever GLAs often exhibit abnormal norms, which LN exploits for defense. However, text-level GLAs derive embeddings from real text, avoiding structural anomalies and bypassing LN's defenses. This suggests that some traditional defense methods that were particularly effective may be limited to the embedding level. The EGNNGuard method generally effective and performs well overall. As mentioned in the main paper, attackers can use techniques like HAO to bypass EGNNGuard, but this often involves trade-offs.
Table 17: ITGIA with default HAO weight against more defense models on Cora, GTR embedding.
Model
Clean
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
GCN
87.19 + 0.62
65.67 # 1.48
65.77 1.15
71.49 = 1.71
74.63 2.48
68.59 + 1.51
GAT
87.47 + 0.49
70.52 # 5.90
70.77 + 4.54
72.62
74.25
0525533 3.64
73.89 + 4.27
SAGE
85.51 + 0.66
67.38 1.92
71.04 + 1.76
7.903
76.31
70.78 + 2.01
Guard
87.81 = 0.36
68.83 1.82
68.17
1.64
73.62
78.33 2.74
70.79 + 1.96
GCN-LNi
87.07 + 0.21
75.00 + 0.98
73.85 0.69
7.99
81.63 2 1.09
76.74 + 1.32
K.3
Transfer to More Embedding Methods
In Table 23 and Table 24 we included SentenceBert all-Minil.M-L.6-V 22 as a new embedding
backbone. We find that WTGIA shows some transferability between different embeddings, but the
25

---

Table 18: WTGIA with average sparsity against more defense models on Cora, BoW embedding.
Model
Clean
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
GCN
86.48 + 0,41
48.32 = 0.74
51.58 0.78
52.49 = 1.32
35,33 1.29
47.81 = 0.78
GAT
86.67 + 0.41
51.46 5.08
57.65 4.98
54.72 + 7.81
33.28 8.31
50.17 : 4.35
SAGE
85.74 0.57
54.86 1.97
58.35
1.64
2.11
56.01 2.71
53.21 # 1.32
Guard
84.62 + 0.38
80.58 0.44
79.44 - 0.33
81.94
79.33
0.65
79.90 + 0.35
GCN-LNi
85.70 + 0.65
68.12 1.37
69.91 + 1.49
73.97 1.9711.5
66.09 * 2.05
67.15 + 1.79
Table 19: ITGIA with default HAO weight against more defense models CiteSeer, GTR embed-
ding.
Model
Clean
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
GCN
75.93
64.79 1.30
65.11 1.01
67.43 # 0.89
71.89
0.50
64.79 : 1.30
GAT
75.67 0.53
54.96 2.11
63.90 2.65
61.71 + 1.43
68.80 1.01
57.20 1.93
SAGE
7..84
64.48 # 1.22
66.98 1.17
65.50 1.12
69.05
2.66
65.33 # 1.32
Guard
76.84
65.97 1.06
67.28 + 1.62
68.77 = 0.72
72.72
66.37 # 1.04
GCN-LNi
75.22
65.29 0.81
66.97 1.22
69.17 1.23
71,66 1.03
65.71 = 0.62
Table 20: WTGIA with average sparsity against more defense models on CiteSeer, BoW embedding.
Model
Clean
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
GCN
75.92 + 0.55
47.52 + 0.65
51.25 + 0.95
49.17 = 0.94
49.50 0.74
47.63 + 0.81
GAT
76.18
49.38 2.74
55.18 4.13
48.37 = 3.96
50.12
50. : 2.69
SAGE
73.54
1.13 51.67 1.13
55.56 1.19
50.29 = 1.40
54.16 1.29
51.75 + 1.07
Guard
74.71 + 0.58
0.61
72.04
70.40
70.17
71.84 + 0.47 0.47
GCN-LNi
74.91 + 0.70
52.30 # 0.81
56.46 1.52
55.98 + 1.50
58.33
56.48 + 0.89
Table 21: ITGIA with default HAO weight against more defense models PubMed, GTR embedding.
Model
Clean
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
GCN
87.91 + 0.26
66.40 2.33
58.56
60.26 + 1.32
76.23
65.77 + 0.91
GAT
86.85 + 0.27
62.44 3.96
60.40 + 3.38
59.00 + 2.25
58.83 4.65
64.27 # 1.92
SAGE
88.79 + 0.23
82.42 # 1.71
78.66 + 2.02
82.56 + 2.03
84,94 # 1.29
79.49 + 1.86
Guard
87.59 0.20
67.83
1.38
59.93 0.87
62.31 + 1.00
77.43 1.26
66.77 + 0.62
GCN-LNi
87.67 4 0.20
74.09
1.71
65.45 1.29
69.39 +
81.96 1.19
70.18 4 0.82
Table 22: WTGIA with average sparsity against more defense models on PubMed, BoW embedding.
Model
Clean
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
GCN
86.42
43.87 0.38
48.62 0.26
48.33 = 0.33
41.73
0.37
45.24 + 0.43
GAT
85.93
45.62 + 2.15
50.72 1.77
50.50 + 1.22
42.50
2.26
47.19 1.83
SAGE
87.00
58.19
60.67 1.67
60,56
57.88
2.30
59.28 4 1.87
Guard
86.54
62.08 # 0.60
64.64 1..440043
65.81
62.54 0.66
63.73 + 0.65
GCN-LNi
51.15 1.32
54.28 + 1.01
53.83 1.06
50.05 1.45
52.29 4 1.35
26

---

results are still unsatisfactory. Although GTR and SBERT are both PLM-based embeddings, the adversarial text by ITGIA performs even worse than WTGIA when transferred SBERT. We believe the transferability of text-level GIAs remains significant challenge.
Table 23: Transferring ITGIA with default HAO weight to SBERT embedding on Cora.
Model
Clean
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
BoW GTR SBERT
86.48 0.41
84.85 0.76
84.04 + 0.78
85.56 + 0.61
86.49 + 0.50
84.90 + 0.73
87.19 # 0.62
65.67 # 1,48
65.77 # 1.15
71.49 # 1.71
74.63 # 2.48
68.59 = 1.51
89.12 # 0.44
85.07 + 0.85
82.97 = 0.58
83.78 + 0.76
86.69 + 0.93
86.06 + 0.65
Table 24: Transferring WTGIA with average sparsity to SBERT embedding on Cora.
Model
Clean
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
BoW GTR SBERT
86.48 e:2 0.41
84.85 + 0.76
84.04 0.78
85.56 0.61
86.49 + 0.50
84,90 + 0.73
87.19 0.62
65.67 # 1.48
65.77 = 1.15
71.49 + 1.71
74.63 + 2.48
68.59 = 1.51
89.12 0.44
80.39 # 1.43
78.01 = 0.92
82.17 # 0.89
85.85 # 0.71
79.41 # 1.16
L Full Experiment Results
L.1
Figure for WTGIA with Topic
The results of WTGIA variants with topic constrained in the prompt are shown in Figure 7
Cora with Topic
Citesoer with Topic
Pubmed with Topic
80
50
48
48
60 46
46-
50 44
12
12
14
15
12
16
1.5
Spensty Buddet (15)
Spersiy Utdget (76)
Sporiity Brdggrt (3)
GPT Allack Performanc:
LlbmnAAllcck Allack Performance
GPT Use Rate
Llama Use Rate
Figure 7: The performance of WTGIA(W topic) against GCN w.r. sparsity budget. As the budget increases, the use rate keeps decreasing, and the attack performance increases and then decreases.
L.2 Full Results for ITGIA
Full results for ITGIA, as shown in Figure are displayed in Table Table 26
Table 27 and Table 28
Table 25; Performance of ITGIA without HAO. Raw texts are embedded by BoW, and GTR before being fed to GCN for evaluation.
Dataset
Model
Clean
Random
SegGIA
MetaGiAA
TDGIA
ATDGIA
AGIA
BoW
86.48 0.41
84.89 0.46
86.42 1 0.56
84.86 + 0,56
86.93 0.65
86.10 1 0.42
85.83 =: 0.54 72.25
Cora
GTR
87.19 + 0.62
82.73 0.59
74.16 + 1.76
71.35 # 1.14
76.52 + 1.45
76.73 1.46
1.32
BoW
75.92 a 0.55
75.50 0.49
75.21 1: 0.67
75.48 a 0.55
75.35
0.52
74.67
0.55
75.25 : 0.39
CiteSeer
GTR
0.44
74.79 1.66 0.66
68.17 0.94
69.39 0.89
68.24 1.30
69.72 1.34
66.18
1.19
PubMed
BoW
86.42 0.24
85.38
0.26
85.75 0.21
85.41 0.23
85.67 0.18
86.36
0.24
85.10 0.30
GTR
87.91
0.26
82.36 1.91
65.13 1.67
58.96 + 1.25
59.47 1.08
69.81 1.90
66.16 0.97
L.3
Full Results for Row-wise FGSM
Full results for Row-wise FGSM, as shown Figure are displayed in Table Table 32 and Table 33
Table 30
Table 31
27

---

Table 26: Performance of ITGIA with default HAO weight. Raw texts are embedded by BoW, and GTR before being fed to GCN for evaluation.
Dataset
Model
Clean
Random
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
BoW
86.48 a 0.41
84.89 0.46
85.23 1 0.65
84.77 = 0.55
8
86.49 =: 0.50
86.05 0.53
Cora
GTR
87.19
82,73 = 0.59
65.67 = 1.48
65.77 = 1.15
1.71
74.63 2.48
68.59 1,51
BoW
75.92 0.55
75.50 0.49
75.02 # 0,41
75.61 + 0.59
74.89 0.56
74.45 0.68
73.77 = 0.50
CiteSeer
GTR
75.93 0.41
0.66
64.79 1.30
65.11
67.43
0.50
64.79 1.30
BoW
86.42 0.24
85.38
0,26
85.76 0.19
84.53
85.35 0.25
86.39
84.41 0.40
PuhMed
GTR
87.91 # 0.26
82.36
1.91
66.40 = 2.33
58.56 + 1.22
60.26 = 1.32
= 2.08
65.77 0.91
Table 27: Performance of ITGIA with 3x HAO weight. Raw texts are embedded by BoW, and GTR before being fed GCN for evaluation.
Dataset
Model
Clean
Random
SegGIA
MetaGiA
TDGIA
ATDGIA
AGIA
BoW
86.48 0.41
84.89 0.46
84.85 1 0.76
84.04 = 0.78
86.01 0.76
85.58 0.85
84,90 =: 0.73
Cora
GTR
8..19
82.73 0.59
66.70 0.94
67.83 0.75
71.90 1.92
2..69
68.81 1.39
BoW GTR
75.92 0.55
75.50 73.39 0.49 0.69
75.17 0.50
13.72 0.57
0.776
74.11 = 0.60
CiteSeer
7
74.79 0.66
64.12 + 1.14
60.77 = 0.71
68.99 = 1.24
71.75 0.95
61.55 = 1.39
BoW
46.42
85,38 = 0.26
85.72 # 0.25
85.17 # 0.35
84,98 = 0.24
86.33 0.11
85.51 = 0.20 70.03 1.56
PubMed
GTR
87.91 0.26
82.36 1,91
63.78 2.41
59.74 #
62.53 1.92
78.31 2.08
Table 28: Performance of ITGIA with 5x HAO weight. Raw texts are embedded by BoW and GTR before being fed to GCN for evaluation.
Dataset
Model
Clean
Random
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
BoW
86.48 = 0.4]
84.89 0.46
84.49 = 0.86
83,06 = 0.63
85.83 1.02
15.6220.80
85.36 = 0.85 72.25 = 1.28
Cora
GTR
87.19 + 0.62
82.73 + 0.59
70.22 * 0.89
77.28 + 0.62
73.86 * 1.93
65.19 = 1.89
BoW GTR
75.92
75.50 0.49
73,47 = 0.74
75.58 = 0.55
74.44 0.55
74.57 0.45
73.99 = 0.91 63.32 1.31
CiteSeer
7
74.79 0.66
65.40 1.01
72.97 0.57
70.93 1.14
72.26 0.83
PubMed
BoW
86.42 a 0.24
85.38
0.26
85.62 + 0.19
84.91 + 0.30
84.69
0,28
86.50 0.15
85.68 0.26 71.71 = 1.78
GTR
87.91 + 0.26
82.36 1.91
63.63 + 2.42
60.12 # 1.47
62.69 2.08
77.93 = 2.09
Table 29; Performance GCN and Guard under FGSM attack with budget being average sparsity of original dataset. Models evaluated using BoW embeddings of raw texts.
Datuset
Model
Clean
Random
SegGIA
MetaGIA
TDGIA
ATDGIA
AGIA
Corn
GCN
0.55
0.73
51.12 +
0.96
0.68
Guard
84.74 a 0.35
83.78 0.49
81.20 1 0.43
81.41 a 0.26
82.83 1 0.40
79.16 0.53
81.30 0.37
CiteScer
GCN
47.35 = 0.62
51.20 + 0.91
0.70 0.99
47.30 = 0,79
Guard
74.85 0.49
74.26 0.34
72.69 a 0.44
72.48 a 0.42
72.31 = 0.38
71.69 0.43
73.31 : 0.42
GCN
86.42.0..2
83.72
0.36
43.44 = 0.25
48.11 0.22
47.80 0.36
41.30 0.19
44.71 0.38
PubMed
Guard
86.51
84.73
81.31 0.39
0.44
81.94 0.33
80.89 0.47
82.26 0.36
Table 30: Performance of GCN and Guard under FGSM attack with budget of 0.10 sparsity in the attacked embeddings. Models are evaluated using BoW embeddings of raw texts.
Dataset
Model
Clean
Random
SeqGIA
MetaGIA
TDGIA
ATDGIA
AGIA
GCN
86.48 a 0.41
85.07 + 0.56
43.75 : 0.31
44.54 + 0.39
43.70 0.92
27.26 0.71
42.23 = 0.44 77.81 = 0.43
Cora
Guard
84.74
83.58 + 0.39
77.07 = 0.54
76.86 = 0.52
99.15
76.41 0.70
CiteSeer
GCN
75.92 + 0.55
74.33 0.62
42.66 + 0.53
45.59 + 0.68
41.06 1.23
43.33 0.69
39.02 = 0.43
Guard
74.85 0.49
73.92 0.67
70.41 0.46
69.73 a 0.63
69.33 0.68
69.19 0.41
69.99 0.62
Table 31: Performance of GCN and Guard under FGSM attack with budget of 0.15 sparsity in the attacked embeddings. Models are evaluated using BoW embeddings of raw texts.
Dataset
Model
Clean
Random
SegGIA
MetaGIA
TDGIA
ATDGIA
AGIA
GCN
86.48 0.41
84.36 a 0.63
41.37 a 0.32
41.98 : 0.49
40.80 # 0.38
23.75 0.55
39.25 : 0.26
Cora
Guard
84.74
83.06 0.59
74.88
75.77 0.47
71,06 0.48
72.89 0.28
GCN
0.55
74.57 44.54 0.72
42.68 + 0.40
0.97 36.72
37.42
42.96 0.26
CiteSeer
Guard
74,85 a 0..9
73.59 0.55
66.72 a 0.45
66.65 + 0.44
65.66 + 0.47
64.15 + 0.53
66.35 a 0.31
GCN
4..24
82.53 1 0.48
42.07 a 0.22
46.41 + 0.38
46.52 0.45
40.61 = 0.10
44,25 = 0.34
PubMed
Guard
86.51 0.17
83.47 0.40
71.29 = 0.86
72.15 = 0.82
2.55
70.09 0.59
71.65 = 0.84
28

---

Table 32: Performance of GCN and Guard under FGSM attack with budget of 0.20 sparsity in the attacked embeddings. Models are evaluated using BoW embeddings of raw texts.
Dataset
Model
Clean
Random
SegGlA
MetaGiA
TDGIA
ATDGIA
AGLA
GCN
86.48 a 0.41
84.15 a 0,70
39.80 a: 0.10
40.63 +: 0.44
40.23 a 0.51
21.21 = 0.26
41.15 = 0.23 70.90 0.55
Cora
Guard
84.74
82.21 0.36
71.63 = 0.37
70.85 = 0.38
72.65 = 0.38
64.62 0.66
CiteSeer
GCN
75.92
0.72 74.07
39.69 = 0.29
43.21 = 0.43
33.88 = 0.71
37,80 0.32
41.57 0.22
Guard
74.85 + 0.49
72.43 0.69
60.62 0.61
62.39 0.56
60.29 0.59
60.21 0.38
64.42 0.39
GCN
86.42 a 0.24
82.02
0.45
41.97 1: 0.25
45.76 a: 0.41
45.86 0.51
40.36 0.10
43.71 = 0.39
PuhMed
Guard
86.51 0.17
82.96 0.96
62.26 * 0.94
63.43 + 0.80
64.21 0.71
59.98 0.76
63.90 0.86
Table 33: Performance of GCN and Guard under FGSM attack with budget of 0.25 sparsity in the attacked embeddings. Models are evaluated using BoW embeddings of raw texts.
Dataset
Model
Clean
Random
SeqGIA
MetaGiA
TDGIA
ATDGIA
AGLA
GCN
86.48 i 0.41
84.21 # 0.75
38.96 + 0.16
39.88 + 0.39
39.36 : 0.41
21.89 = 0.29
40.14 = 0.13
Cora
Guard
84.74 a 0.35
81.28 0.83
68.56 a: 0.51
65.72 a: 0.47
69.19 a 0.52
57.58 = 0.91
66.28 0.44
GCN
0.55
0.96 74.15
38.43
40.59 + 0.39
33.44 0.86
40.08 = 0.28
CiteSeer
Guard
71.61 0.49
0.46
55.91 0.58
52.11 0.51
0.54
56.70
PubMed
GCN
86.42 0.24
81.76 0.57
41,80 0.31
45.05 0.45
45.51 0.60
40,08 0.09
43.21 0,46
Guard
86.51 + 0.17
83.01 a 1.21
55.48 a 0.74
56.47 + 0.79
58.84 a 0.67
53.62 # 0,55
55.85 0.92
L.4
Full Results for WTGIA
Full results for WTGIA, shown in Figure 5
are displayed in the below tables. For average sparsity
budget:
Table 34 Table 35
Table 36
For sparsity budget 0.10: Table 37
Table 38
For sparsity budget 0.15:
Table 39
Table
40
Table
For sparsity budget 0.20: Table
42
Table
43
Table
Table 34: Performance of WTGIA on Cora. The sparsity budget is set as the average sparsity of the original dataset. The injected raw texts are embedded with BoW and GTR, and then evaluated using& GCN. The last column reports the average result of GIAs in the last five columns. The (swes(((tronges)) result in the column is bold.
Generation Type
Embedding
Clean
Random
SegGIA
MenaGA
TDGIA
ATDGIA
AGIA
Avg Last Five
Fibhdding
86 0141
85.00
48.00
0.73
0..12
$1,70
34.11
0.96
GPT
GTR
-648-0.4 0.41
$7.19.0.62
82.97 1 0.66
49
78.01 1 1.49
0.68
1.06 75,95
52.14 0.79
79.00
$2.90 1.12
1.30
83.68 = 1.06
36,02 1.38
48.40
77.25
1.50
47.70 0.93
78.78 1.28
GPT-Topic
BaW
0.41
85.09 0.55
49.58 1.00
$2.06
0.36
53.75 1.19
38,96 1.94
48.95
18.66 1: 1.18
GTR
17.19 0.62
83.07 1.77
77.59 1.62
0.97 76.41
78.85 à 1.49
83.57 & 1.01
1.54 76.89
78.66 1.33
Elama
BaW
-6.48 0.41
84.99 0.56
4
51.58
0.78
$2,49
35.33 1.29
47.81+
0.98
GTR
$7.19
0.78
78.15 1,70
76.88
79.27
$3.77
77.95
79.20
Llpma-Topic
GTR
BoW
87 $7.19 0.62
84,90 0.62
82.84 0.79
48.57 a 0.82
77.80
1.74
51.58 0.73
76.42 1.10
78.96 + 1.26
$2.30 a 1.26
83.52 = 1.02
35.32 a 1.13
47.69 1 0.76
77.69 + 1.46
47,09 0.94
78.88 1.32
Table 35: Performance of WTGIA on CiteSeer. The sparsity budget is set as the average sparsity of the original dataset. The injected raw texts are embedded with BoW and GTR, and then evaluated using GCN. The last column reports the average result of GIAs in the last five columns. The (owest()trongest) result in the column is bold.
Generation Type
Enbedding
Clean
Rundom
SogRIA
MeuGlA
TEGLA
ATDGIA
AGIA
Aug Last Pive
Embedding
BoW
2.922..55
74.73 1
47.35 a 0.62
51.20 0.91
48.90 0.93
29.01
47.30 1 0.79
48.76 0.79
GPT
BoW
0.55
0.788
51.23 a 0.68
$1.97 1.02
1.422
25.8
48.63 a 0.84
52.27 0.98
GTR
15493
75.03 0.566
1.23 68.50
65.04
0.85
69:43 1
73.53 2
71.24 0.89
64.55 0.84
GPT-Topic
BaW
74.73 7..92+0.55
52.74 1.66
54,88 1.19
-6.15-0.79
49,06 0.61
$3.12
GTR
0..9
0..72
1.25 69.71
64.17 0.86
69.45
03.26=
Llnma
GTR
BoW
05.5
75:93 0.41
74.58 0.57
75.14
0.58
47.50 a 0.65
65.21 a 0.98
51.25 0.95
66.07 0.77
70.15 0.76
19.17...99
49:50
72. 0.71
69.82 0.94
47.63 1 0.81
68.71 + 0.83
49.01 0.82
Llama- Topic
GTR
BoW
75.51 1.49
2..92
74.65 0.64
0.488
47.60 : 0.66
65.18 1.10
51.34 0.98
65.84 0.72
49.09 0.88
70.33 0.700
49:41 = 0.79
72.72
69.71 0.83
47.866 + 0.91
68.76 0.84
49.02 a: 0.84
29

---

Table 36: Performance of WTGIA on PubMed. The sparsity budget is set the average sparsity of the original dataset. The injected raw texts are embedded with BoW and GTR and then evaluated using& GCN. The last column reports the average result of GIAs in the last five columns. The (owest(strongest) result in the column is bold.
Generation Type
Enbeedding
Clean
Random
SoyGIA
MeuGGA
TDGIA
ATDGIA
AGIA
Avg Last Five
Embooding
BoW
86-42 a 0.24
83.72 0.36
43.44
48 0.22
47.80 0.36
41.30 a 0.19
44.71 a 0.38
45.07 a 0.28
GPT
BaW
0.24
83.49 0.33
0.6
03.3
+9.22200.00
43.46 0.86
46.29 + 0.59
46.67 0.56
GIR
$7.11
83.53
67.442243
199 61.23
68.59 2.16
67.28 2.26
66.88 2.35
GPI-Topic
GTR
BoW
57.1
82.65
82.02
46.75 1.00
64.99 2,50
50.71
1.67 64.01
0.337
50.38 0.48
61.23 1.72
6661
46303 1.21
2.96
64.65
0.85
64.30
Lluma
BoW GTR
87.91
83.15 a 0.38
82.82 a 2.03
43.87 4
(8.13
65.92 1.76
48.62
0.26
64.08 a 2.05
48.33 a
=
69.38 2.78
67.06
45.24 0.43
66.91 2:2.2
45.56 0.35
Llama/Topic
BoW GTR
82.11 2..10
83.01 1 0.38
66.66 2.42
44.67 0.59
66.58 1.76
08.51 0.25
48.32 a 0,38
2.12
70.16 2.82
41.76 a 0.39
67.20 2.184
45.20 0.11
15.70
Table 37: Performance of WTGIA on Cora. The sparsity budget of the embedding is set as 0.10. The injected raw texts are embedded with BoW and GTR, and then evaluated using a GCN. The last column reports the average result of GIAs in the last five columns. The lowest((trongest) result in the column is bold.
Generation Type
Embeddiing
Clean
Random
SugGIA
MetaGIA
TDGIA
ATDGIA
AGIA
Arg Last Five
Embedding
BaW
86.48 e 0.41
85.07 0.36
43.75 + 0.31
44.54 0.39
4 0.92
27.26 = 0.71
42.25 0.44
40.30 0.55
GPT
BaW GTR
19 a 0.62
86.48 + 0141
85.36 0.49
82,45
0.86
45.04 + 0.48
76.11 1.48
76.04 1.18
46.27
0.70
4743 1.36
79.54
81.79 a 1.22
33.49 1.61
74.75 1.64
44.00 + 0.55
77.65 1.35
43.29 + 0.94
GPT-Topic
96-18
46.52
51.57
39.43 2.19
GTR
+7.9900.0
82.85
75.79 a 1.38
75.58
79.47 1.12
82.88 1.24
Llnma
BoW
85.22 26.48
44.33 = 0.44
0.65
45.56 1.00
29.44 1.05
43.06 0.54
41.58 +: 0.74
GTR
$7.19:0.62
83.11 0.77
76.96 1.48
76.74 1.40
80.23 à 1.29
82.23 2 1.09
75.43 2: 1.53
78.32 à: 1.26
Lhma Topic
BoW GTR
+
85.39 + 0.57
44 0.45
4..64
46.17 1.10
29.35 1.05
43.01 + 0.45
41.72 + 0.74
=7.19
83.43 0.99
1.34
75.84
80.22 1,24
1.18
74.73
78.02 1.28
Table 38: Performance of WTGIA on CiteSeer. The sparsity budget of the embedding is set as 0.10.
The injected raw texts are embedded with BoW and GTR, and then evaluated using a GCN. The last column reports the average result of GIAs in the last five columns. The lowest(strongget) result in the column is bold.
Generation Type
Enbeddinn
Clean
Rendoen
SegFiAA
MeuGlA
TDGLA
ATDGIA
AGIA
Aug Last Five
Embodding
BaW
75.92 0.55
0.55
74.33 0.62
42.68 + 0.53
45.59 0.68
41.06 1,23
43.33 = 0.69
39,02 + 0.43
42.34 0.71
GPT
GTR
BoW
75493
0.55
74,58 0.63
75.16 1.69
44.22 0.73
1
67.66 0.94
46.43 2.84
44,97 1.14
0.55
71.55
43.49 1.30
11.552097
68.17 1.08
40.23 a 0.65
69.65 1 0.83
44.67 0.89
GPI-Topic
BaW
75.92+0.55
+0.74
4.56
47,91
46.84
51.41 1.09
41.31
0.88 70.39
GTR
0..93
71.29 0:.49
0.57
68.43 0.99
70.81
71.86= 1.12
69.56
Llnma
GTR
BaW
75.92=0.55
74.31 0065
75.35 0.54
43.20 0.57
70.51 a 0.74
46, 10:10== 0.77
68.17 1.01
41.80 1.24
70.77 a 0.76
44 1.:00-0:
1.09 71.47
68.89 1.35
39.67 + 0.45
69.96 e 0.99
42.90 0.78
Llama: Topic
BoW GTR
25.93
0.55
75.41 0.229
74.27 0.65
43.42 a 0.57
21.04.0.82
69.34 0.77
45.95 0.78
7..65
41,84 1.20
71.95 1.14
43.91 0.72
39,42 0.46
70.61
42,91 0.75
Table 39: Performance of WTGIA on Cora. The sparsity budget of the embedding is set as 15.
The injected raw texts are embedded with BoW and GTR, and then evaluated using a GCN. The last column reports the average result of GIAs in the last five columns. The lowest(strongest) result in the column is bold.
Generation Type
Embedding
Clean
Randoen
SogGIA
MeuGlA
TDGIA
ATDGIA
AGIA
Avg Last Five
Embedding
BoW
86.48 0.41
84.36 +
41.37 2 0.33
41.98 0.49
40.80 0.38
23.75 0.55
39.25 : 0.26
37.23 + 0.40
GPT
GTR
BoW
87 19 0.62
86.48 01.1
82,80 e 0.71
85.31 0.67
44335 a 0.54
78.04 + 1.28
45.04 0.78
77.19 0.97
46.31
80.90 1.26
41.28 3.10
81.72 1.10
77.84 1.06
45.59 a 0.90
79.14 1.13
44,51 + 1.16
GPT-Topic
BaW
86.48 041
85.56 0.63
46.32 0187
47.14 0.86
1.55
48 18 3.06
1.46 79.27
GTR
87.19
83.32
77.54 1.19
77.27
80.66 1.29
82.09 0.93
78.11
Llama
BoW
36.48
84.68 1 0.64
43.58 1 0.54
44.05 0.56
44.25 4 1.23
30.07 = 1.44
42.53 1 0.83
40.90 à: 0.92
GTR
$7.19
82.75 1 0.84
76.52 1.34
1.01 75.84
60.16 4 1.17
$2.09 :: 0.96
77.26 1.19
78.38 1.1.
I Lloma-Topic
BoW
0.44
84.89 + 0.52
43. 19 0.54
43.78 0.51
43.85 + 0.98
33 = 1.53
42.94 + 0.77
41.29 + 0.87
GTR
83.14 87.19
76.64 1.17
75.81
1.10
80.00 1.22
01.5
77.41
78.27
30

---

Table 40: Performance of WTGIA on CiteSeer. The sparsity budget of the embedding is set as 0.15. The injected raw texts are embedded with BoW and GTR, and then evaluated using GCN. The last column reports the average result of GIAs in the last five columns. The lowest(strongest) result in the column is bold.
Ginnenaioon Type
Enibedding
Clean
Rendoen
SegGiAA
MeuGiA
TDGIA
ATDGIA
AGIA
Aug Last Five
Embedding
=5.52
0.72
44.54
42.68 0.40
36.72
37.42 0.60
0:53
GPT
BoW
25.52
74,86
0.90
4
45.53
44.71 +
44.93 = 1.07
46.09 1
45.80 a 0.79
GTR
75.93
75.22
0.73
70.86.10.77
71.43 0.73
70.75 a 0,76
70.85 a 1.21
0.776
71.24 a 0.86
GPT-Toopcc
BoW
75.92=0.55
74.92 0.59
49.66 0.66
46,97 0.52
48.43 1.20
50.29 = 1.18
48.21 0..99
48.71 + 0.81
GTR
=
71.70 0.72
71.17 0.59
71.96 1.07
72.58 0..71
71.57 0.76
Llama
GTR
=3.93
75.39
a 0.61
45
70.42 a 0.60
70.83 0.76
43.73 0.46
70.94 0.57
71.47 = 1.08
40.26
72.43 a 0.96
71.22 0.79
0.66
Llama-Topic
GTR
BaW
05.5
23.93
74.60
75.15
0.45
0.74
46.56 = 0.58
70.29 a 0.77
44.28 0.49
70.78 0.72
40.82 1.03
71.19 0.67
49.21 = 0.78
71.31 1.14
44.40 =
71.75 0.55
43.25 = 0.66
21.10000.78
Table 41: Performance of WTGIA on PubMed. The sparsity budget of the embedding is set 0.15,
The injected raw texts are embedded with BoW and GTR, and then evaluated using GCN. The last column reports the average result of GIAs in the last five columns. The lowest(strongest) result in the column is bold. Note that GoT-Topic fails to give a meaningful response.
Generation Type
Embeddiig
Clean
Random
SoyGIA
MetaGIA
TDGIA
ATDGIA
AGIA
Avg Last Five
Embedding
BaW
82.53 0.48
42.07 + 0.22
46.41
0.38
46.52 + 0.45
49.61 = 0.10
44.25 + 0.34
43.97 +
GPT
BoW GTR
0.22
82.69 e
44.19 0.55
48.65 0.25
49.04 0.39
43.61 = 0.70
47.11 + 0.58
46,52 + 0.49
83.06
66.37
68.05 1.77
63.19
67.83 281
66.08
65.82 2.18
Llama
BoW GTR
a
87.01 0.26
82.39 0.51
82.35 2.05
42.62
66.19 2.07
47.24
64.49 1.70
62.78
1.84
41,02 0.28
68.12 2.60
66.66
0.442
Llame-oppc
GTR
BoW
87.91
82.28 0.32
81.72.2.000
66.66
42.58 a 0.30
47.18 0.26
65.21
1.77
47.71 + 0.38
63.65 + 1.94
68.30
40,99 = 0.32
67.61 a 1.95
0.445
66.29 2.09
44.72 + 0.34
Table 42: Performance of WTGIA on Cora. The sparsity budget of the embedding is set as 0.20.
The injected raw texts are embedded with BoW and GTR, and then evaluated using a GCN. The last column reports the average result of GIAs in the last five columns. The lowest(strongest) result in the column is bold.
Generation Type
Emieddding
Clean
Rendoen
SogFiAA
MeuClA
TDGIA
ATDGIA
AGIA
Avg Last Five
Embedding
BoW
36.4820.44
84.15 0,70
2.110
40.63 0.44
40.23 a 0.51
21.31:0.22
41.15 a 0.23
36.68 a 0.31
GPT
BoW
$6.48 0.41
85.16 0.53
44,41 0.87
0
50.58 1.36
41.83
2.69
46.22 a 0.85
47.10 1.30
GTR
87.19
82.80
1.83
79.72
79.27 1.1
81.53
$1,46 1.27
78.36 1.29
80.07 1.21
GPT-Topic
BaW
46448041
85:37
48,80 1.68
49.00
56.06
16.16-22.0
50.23
GTR
87.19
82.37
79.68 1.47
79.12
81.43 1.24
82.27 1.05
78.27
80.15
Llama
BoW
26.1
81,70 0.65
42.84 1 0.70
15.14
49.35 1.32
17.19
44.20 0.58
45.80 1.19
GTR
87.19
82.62 0.64
77.93 1.30
77.78
1.12
80.84
1.02
2..9
76.74 1.47
78.96 1.18
Llama:
GTR
BoW
8..2
=6.48
82.79 0.78
85.09 e 0.49
7 1.28
43.83 + 0.81
71.58 1.03
45.89 0.77
48.99 1.20
80.22 1.00
81.81 1.07
43.11 2:48
44.96 + 0.71
76.51 1.28
78.68
45.36 1.19
Table 43: Performance of WTGIA on CiteSeer. The sparsity budget of the embedding is set as 0.20.
The injected raw texts are embedded with BoW and GTR, and then evaluated using GCN. The last column reports the average result of GIAs in the last five columns. The lowest(strongest) result the column is bold.
Generation Type
Embodding
Clean
Random
SeyGIA
MetGGA
TDGIA
ATDGIA
AGIA
Avg Last Five
Emhedding
BoW
75.92 (.55
74.07 0.72
39.69 0.29
43.21 0.43
33.88 +
37.80 0.32
41.57 +
39.23 + 0.39
BaW
-.55
74,72 0.83
45,07 a f162
43.45 0.58
0.99
49.16 0.85
46.25 0.62
46.70 0
GTR
#..93
74.79
0.5.8
70,04 0.56
70.55
72.46
71.96= 1.04
72.42
GPT-Topic
BoW
0.55
74.65 0.733
47.41 a 0.74
49.06 0.52
49.61 a 1.01
54:43 1.17
47.85
49.68 1 0.89
GTR
1:9330044
74881 0.71
71.14 a 0.78
71.53 + 0.73
73.34
0.57
72.44 1.17
72.28 0.52
72.15 0.75
Llama
GTR
BoW
1.49
01.5
74.54 0.81
75.01 0.63
70.16 a 0.71
43.00 0.59
45
70.79 0.75
0.54
72.64 a
41.00 1.27
0.94
4
43.84 a
0.62
71.64 0.73
43.45 0.69
Lioma-Topic
0.55
74.81 0.89
32 0.80
46
42,95
46.14-0.7
44.41 0.44
GTR
75.93=0.41
75.26 0.48
69.80 + 0.74
70.75
72.50 + 0.82
71.95 1.30
0.417
71.45 0.82
31

---

Table 44: Performance of WTGIA PubMed. The sparsity budget of the embedding is set 0.20. The injected raw texts are embedded with BoW and GTR, and then evaluated using GCN. The last column reports the average result of GIAs in the last five columns. The lowest(strongest) result in the column is bold. Note that GPT-Topic fails to give meaningful response.
Genenatio Type
Fnieedding
Clean
Rendoen
SogrlAA
MeuCiA
TEGIA
ATDGIA
AGIA
Avg Last Five
Embolding
BoW
86.42 0.24
82.02
0.45
41.97 + 0.25
45.76 0.4
45.86 0.51
40.36 = 0.10
43.71 0.39
43.53 0.33
GPT
BaW GIR
47.91
83.53
63.74 1.27
0.29
47.89 0.81
1.85 66.43 2.221
49.82 0.35
0155
2.33
48.06 1.10
69.88 2.49
66.56
4..53
67.25 2.20
49.20 a: 0.66
02442:024
48.53
GTR
2,02
63.06
61.86
66.88 = 2,45
Uloma-Topic
BoW GTR
87.91 2
0.22
81.52
79,53
0.40
2.18
46,80 +
6..13
63.04 a 1.79
49.22 0.31
50.67 + 0.41
62.21 a 1.87
68.25 2.52
46.88 = 1:26
64.78 1.90
48.69 0.59
48.45 0.67
64.74 2.00
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The claims we made accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of text-level attacks in Section 5
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
32

---

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification:
We provide proof to our theorem in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Thcorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification:
We provide the pseudo-code in the Appendix and the code URL in the abstract.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
33

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they (c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described supplemental material?
Answer: [Yes]
Justification:
We report open access information in Appendix B
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. ec/
public/guide Btaaa//ggiiis//ooouuubnissssss "CodeSummissionn
ceiiionoolcyl for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
/ /pips.cc/pptt ptsiss.cc/publli/uuudeeee wetggidddeee/eeeebieemaaaaea /wtp///ooeeSeseeeeeeeeeeee
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
We provide experimental settings in Appendix H.I
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We repeat experiments 10 times and report the standard deviation in the tables.
34

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the eπror bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
We have included resources information in Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httppp://heurrppp..cpppbll/////t
Answer: [Yes]
Justification:
We conform in every respect with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
35

---

Justification:
We have included Broader Impacts section in the Appendix A
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We have included the license clarification in Appendix B
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
36

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thoode com comddatasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper included anonymized code in the supplementary file. The training, license, and limitations have been discussed in Appendix B Appendix H.1
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
37

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
38


Reviews:
[{"review_id": "g9OoAst6DW", "reviewer": "Reviewer_YfqF", "summary": "This paper explores the vulnerability of GNNs to Graph injection Attacks (GIAs), which involve injecting malicious nodes into a graph. This paper explores GIAs at the text level, presenting three types of GIA designs that inject textual content into the graphs. The significance of text interpretability in attack effectiveness and the defense against text-level GIAs are also explored.", "questions": "Refer to weaknesses.", "limitations": "A section to discuss the limitations is missing.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The idea is straightforward and easy to follow.\n2. The study of text-level graph injection attacks is interesting and practical.", "weaknesses": "1. In line 126–131, the paper claims that traditional methods focus mainly on the embedding level. However, some existing works (e.g., [1]) focus on the raw feature level. The authors fail to discuss why these works cannot be applied to this task.\n2. Limited technical contribution. In this paper, three kinds of text-level GIA are proposed. However, I think all of them are simply combining both existing GIA techniques and language models, which are kind of incremental contributions. \n3. I think the proposed text-level GIA is similar to the traditional GIA, especially when the nodes' texts are encoded into embeddings by LMs. Therefore, I think some existing defense methods against GIA can be also applied to this task. The authors need to discuss that. \n4. The experiments only focus on small-scale datasets. More large-scale datasets need to be included.\n5. Only vanilla GCN is used in the experiments, which may induce to biases to the conclusions. More advanced GNNs need to be included.\n\n\n\n[1] Understanding and Improving Graph Injection Attack by Promoting Unnoticeability. ICLR 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful feedback on our paper. We have addressed your concerns and provided clarifications below.\n\n---\n\n**W1:** *Why previous works cannot be applied to this task.*\n\nWe would like to clarify that by “Raw Feature,” we are referring to raw text, which represents the unembedded original features of TAGs. \nPrevious GIAs, including the work referenced in [1], require the use of text embeddings during the generation process and are limited to injecting embeddings rather than raw text. \nConsequently, these methods cannot be directly applied to our task.\n\nTo apply the method from [1] to this task, the embedding-text step must be completed, which is achieved by ITGIA in our paper.\nIt advances embedding-level GIAs by transforming the embeddings generated in [1] back into raw text. \nHowever, we discovered several practical issues with the generated content, such as poor interpretability, loss of accuracy, and transferability challenges. \nThese issues have not been addressed in previous embedding-level studies of GIAs and are found to be a significant challenge first identified by us.\n\n---\n\n**W2:** *Limited technical contribution.*\n\nFirstly, the three proposed text-level GIAs require the introduction of previously unconsidered designs.\nWhile ITGIA builds on traditional GIAs, we additionally introduce normalization to align with the structure of normal embedding and discover the importance of interpretable regions.\nIn VTGIA and WTGIA, we first introduce LLMs to Graph Adversarial Attacks, especially GIAs, ensuring interpretability during poisoning content generation. \nIn VTGIA, we explore three different strategies for direct generation.\nIn WTGIA, we transform FGSM into a sparsity-constrained version suited for raw text scenarios, unlike traditional FGSM, which samples features based on average sparsity. \nAdditionally, WTGIA constructs the “Words to Text” task and uses masking to enhance attack performance.\n\nSecond, our research pioneers the exploration of text-level GIAs, aiming to fill the gaps left by previous studies. \nWe identify incomplete evaluations in earlier GIAs and propose three new lines for development, highlighting the inherent challenges in this domain.\nAs an initial attempt, it is essential to explore techniques that appear to be straightforward.\nOur work focuses on understanding the landscape of text-level GIAs and identifying critical issues that need future attention, which sets the stage for more sophisticated advancements in subsequent research.\n\nThus, the perceived technical limitations should not detract from the significance of our work.\n\n---\n\n**W3:** *Limited Defense GNNs.*\n\nIn the original paper, we discussed the EGNNGuard, which demonstrated significant defensive capabilities as described in [1]. \nWhen incorporating HAO or increasing sparsity, attackers can achieve effective improvements at the embedding level. \nIn the **author rebuttal**, we have added the performance of text-level GIAs against other defense models, including the Layernorm trick (LN) mentioned in [1] and common methods like GAT and GraphSAGE. \nInterestingly, LN’s effectiveness on text-level GIAs is not as strong as previously reported for embedding-level attacks. \nThis is because embedding-level GIAs often exhibit abnormal norms, which LN exploits for defense. However, text-level GIAs derive embeddings from real text, avoiding structural anomalies and bypassing LN’s defenses.\n\nThe above results indicate that re-evaluating defenses against GIAs at the text level is a promising research direction.\nWe acknowledge that adding more defense models could make the evaluations more comprehensive. \nHowever, considering that we have supplemented with classical backbones and methods proven effective in [1], as well as discussed new defense mechanisms at the text level, we believe this sufficiently supports the conclusions presented in our paper.\n\n---\n\n**W4:** *Large-scale Datasets.*\n\nIn the **author rebuttal**, we include results for the Reddit and ogbn-arxiv datasets, the former containing over 30,000 nodes and the latter containing over 160,000 nodes. \nNote that current text-level GIAs do not increase complexity concerning the graph structure. \nThe text generation process operates in $O(N_{{inj}})$, proportional to the number of injected nodes, which is significantly smaller than the total number of nodes in the original graphs. \nConsequently, WTGIA can complete text generation even for ogbn-arxiv within 8 hours without parallelization optimization, demonstrating high efficiency.\n\n---\n\n**Limitations**\n\nIn Section 5, we discussed the new challenges faced by text-level GIAs. \nRegarding more limitations of the paper, we believe that the current framework can be extended to better integrate graph structure and text generation to improve performance. \nWe will address more about limitations in future versions.\n\n---\n\n**More about Experimental Completeness**\n\nDue to the complexity of presenting results for **all combinations of datasets, attacks, defenses, and hyperparameters** (which would require more than 100 tables for presentation by estimation), we did not include all results in the original paper.\nWe controlled the number of datasets and defense models selected to better present our findings. \nAs **we did not emphasize the advantages of text-level GIA in terms of technical performance and efficiency**, we believe it is reasonable to choose the most commonly used datasets and victim models of traditional GIAs for evaluation.\n\nIn the **author rebuttal**, we also included the running time, more embedding models, and a dataset from a new domain, Reddit. \nWe will consider your suggestions and incorporate newly added experiments.\nIf you have any further questions, we welcome more discussion.\n\n---\n\n**Reference**\n\n[1] Chen, et al. Understanding and improving graph injection attack by promoting unnoticeability. ICLR 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eaKFM9yvv6", "reviewer": "Reviewer_ZG7Z", "summary": "The paper studies GIAs, particularly focusing on text-attributed graphs (TAGs). It introduces a method for GIAs by injecting textual content directly into the graph, as opposed to the traditional method of embedding-level attacks. The authors propose three new attack designs: Vanilla Text-level GIA (VTGIA), Inversion-based Text-level GIA (ITGIA), and Word-frequency-based Text-level GIA (WTGIA). They demonstrate that text interpretability is necessary for the effectiveness of these attacks. They also show that defenses can be enhanced using customized text embedding methods or large language model (LLM)–based predictors.", "questions": "1. Can the authors provide more details on how the sparsity budget impacts the interpretability and effectiveness of the text-level GIAs across different datasets?\n2. How would this work apply to dynamic graphs where nodes and edges evolve over time?\n3. Are there any challenges in scaling the proposed GIAs to larger graphs or more complex datasets?", "limitations": "N/A", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "I found this to be a very original paper as it shifts from embedding-level GIAs to text-level GIAs. Three new attacks are created and are tested theoretically and empirically.  Most importantly, the GIAs created ensure that the attacked nodes have semantically useful information, making the attack more realistic, using LLMs.", "weaknesses": "The paper does not explore the robustness of the proposed GIAs against more sophisticated defense mechanisms beyond customized text embedding methods and LLM-based predictors. \n\nAlthough the authors mention that LLM-based defenses enhance protection against text-level GIAs, the experiments primarily focus on a few specific methods and datasets. These are limited in scope. \n\nThe trade-offs between performance, unnoticeability, and interpretability, as discussed in Sections 4.1 and 4.2, could be examined more thoroughly. While the paper shows that increasing the sparsity budget enhances attack performance but reduces interpretability, further experiments across a wider variety of datasets could validate these findings more convincingly, especially with additional baselines. Including more traditional GIA methods or other recent advancements in graph attack techniques would also be useful.\n\nThe discussion on the practical implementation of these attacks in real-world applications could be expanded. The paper mainly provides experimental insights but lacks detailed exploration of the real-world scenarios you stake your paper around, and practical challenges like the scalability of the proposed methods to larger and more complex graphs, the computational costs associated with implementing these attacks, and potential detection mechanisms in real-world systems are not thoroughly addressed. Providing more actionable insights and guidelines for practitioners could significantly enhance the practical relevance of the research.\n\nLastly, while the authors discuss the potential for enhancing defenses using customized text embedding methods or LLM-based predictors, they do not explore the possibility of integrating multiple defense strategies. Combining different defense mechanisms could potentially create more robust protection schemes against text-level GIAs. \n\nTypos: line 296 and line 211", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed feedback on our paper. We appreciate your insights and have addressed your concerns below.\n\n---\n\n**W1 and W5:** *Exploration of Robustness Against Defense Mechanisms*\n\nIn the **author rebuttal**, we included classic methods like GAT and GraphSAGE, as well as the EGNNGuard and Layernorm (LN) methods, which have been proven highly effective in [1].\nWe indeed found interesting results, such as LN’s effectiveness on text-level GIAs not being as strong as previously reported for embedding-level attacks. \nThis is because embedding-level GIAs often exhibit abnormal norms, which LN exploits for defense. \nHowever, text-level GIAs derive embeddings from real text, avoiding structural anomalies and bypassing LN’s defenses. This also means that some traditional defense methods that were particularly effective may actually be limited to the embedding level. \nThe EGNNGuard method is generally effective and performs well overall. As we mentioned in the main paper, attackers can add tricks like HAO to bypass EGNNGuard, but this often involves trade-offs.\n\nRegarding using LLM-based methods for defense, we believe the current LLM for Graph methods still face the following challenges: 1) There is still a lack of dedicated methods combining LLM and Graph for defense. 2) The LLM-as-Predictors [2] method still faces scalability issues, and directly using it on large graphs requires significant resources. 3) LLM-as-Enhancer [2] methods still rely on GNN as the backbone, making them not much different from using defensive GNNs in terms of defense. \nConsidering these challenges, we used predictors as an initial exploration. We believe that further exploring the combination of LLM and GNN to enhance defense capabilities will be an interesting direction.\n\n---\n\n**W2, W3, Q1:** *More Datasets, More GIAs, More about the trade-off*\n\nIn terms of datasets, we selected the most common GIA test datasets in the original paper. \nIn the **author rebuttal**, we added larger datasets, such as ogbn-arxiv, and the social network dataset Reddit. \nWe demonstrated that WTGIA can be applied to larger graphs (over 160,000 nodes) and different domains. \nConsidering that the complexity of VTGIA and ITGIA is lower than that of WTGIA (see running time), we believe this sufficiently demonstrates the versatility of the text-level GIA framework. \nIn the future, we believe it is interesting to explore the differences in text-level GIA across various domains.\n\nFor GIAs, we selected representative and competitive GIA models for evaluation. \nFor instance, TDGIA is a top-ranked model on the GRB benchmark [3], while the SeqGIA family [1] effectively addresses unnoticeability with desirable efficiency. \nFor embeddings, we included PGD, FGSM, and feature-smooth (TDGIA/ATDGIA), three representative learnable embeddings. Therefore, we believe the current GIAs are sufficient to support our conclusions. \nWe are open to incorporating new GIAs as needed in the future.\n\nRegarding the trade-off, we have more discussion in Appendix J. \nFor example, we found that larger models tend to reduce the Use Rate to ensure the fluency of the output, selecting words that more easily form a coherent text. This also manifests as lower perplexity, indicating more fluent text.\n\n---\n\n**W4, Q3:** *Practical issues in Real-world Applications*\n\nWe acknowledge that the paper could benefit from a more detailed exploration of practical implementation challenges.\nWe believe one of them is similar to that of graph foundation models, namely the issue of uniformity. \nDifferent graphs have varying text length limitations and text themes, and the LLMs’ understanding of them differs. \nIn practice, it will be challenging to propose a unified text-level attack algorithm that does not require adjustments for each domain.\n\nRegarding scalability and complexity, please refer to the **author rebuttal**. \nIn summary, scalability is not a major issue in current text-level GIAs, but it may need to be considered in future designs.\n\n---\n\n**W6:** *Typos*\n\nThank you for pointing out the typos. We will correct these in the revised version.\n\n---\n\n**Q2:** *Application to Dynamic Graphs*\n\nWe believe that attack and defense on dynamic graphs is a very interesting direction. \nWhen generating attacks on dynamic graphs, the historical interaction information of nodes should also be considered as a basis for generating injected nodes. \nAt the same time, in dynamic graph scenarios, not all injected nodes are harmful. \nA more accurate characterization of this scenario will help us better align with real-world applications.\n\n---\n\nThank you once again for your valuable feedback and suggestions. We believe that incorporating these insights will significantly enhance our work’s quality and impact. We welcome any further questions or discussions.\n\n---\n\n**Reference**\n\n[1] Chen, et al. Understanding and improving graph injection attack by promoting unnoticeability.\n\n[2] Chen, et al. Exploring the potential of large language models (LLMs) in learning on graphs.\n\n[3] Zheng, et al. Graph robustness benchmark: Benchmarking the adversarial robustness of graph machine learning.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aGIaKzgeox", "reviewer": "Reviewer_nfHh", "summary": "This paper investigates an interesting topic. The authors first discuss the practicality of Graph Injection Attacks (GIA), arguing that previous approaches, which merely inject harmful embeddings, are unrealistic. It introduces three text-level GIA methods: ITGIA, VTGIA, and WTGIA, and conducts a thorough examination of their attacking performance and interpretability. The findings reveal that WTGIA excels in both attacking effectiveness and interpretability. Additionally, the paper explores the defensive capabilities of Large Language Models (LLMs) and discovers that they can effectively evade the impact of injected nodes. This underscores the importance of enhancing the performance of text-level attackers.", "questions": "1. For VTGIA, how does the LLM optimize the graph structure after generating the text for the injected nodes?\n\n2. The attacking method of WTGIA is interesting. Could this idea inspire other areas of GNN?\n\n3. In Section 4, could you clarify the embedding-text-embedding process? Specifically, in Section 3, under Analysis of Performance and Interpretability, it seems that after inversion, text is generated. What embedding results after this inversion?\n\n4. Why do you introduce FGSM to generate binary embeddings? Is it not possible to directly transform the embeddings into binary embeddings?\n\n5. In Figure 5, why does the attack performance initially decrease and then increase as the sparsity budget increases?", "limitations": "Some parts of Section 4.1 are hard to understand, I admit I didn't check the proof related to the theorem in the Appendix. I think the authors could add figures or improve their presentations to better illustrate WTGIA.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. This paper first addresses the impracticality of embedding-based GIAs and explores text-level GIAs, introducing three novel text-level GIA methods. Among these, WTGIA effectively balances attack performance and interpretability.\n\n2. The authors have conducted comprehensive experiments that thoroughly address the research questions posed in the study.\n\n3. The theorem discusses the conditions under which destructive texts can be identified, providing a clear theoretical foundation for the work.\n\n4. This paper offers valuable insights into the new challenges associated with text-level GIAs, identifying emerging directions in this field.", "weaknesses": "1. The authors should provide references for the cited papers on TDGIA, ATDGIA, MetaGIA, and AGIA.\n\n2. The authors should explain why the attack performance initially decreases and then increases as the sparsity budget increases, as shown in Figure 5.\n\n3. The authors should clarify how the LLM optimizes the graph structure after generating the text for the injected nodes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful comments and suggestions. Please find our detailed responses below:\n\n---\n\n**W1:** *Citations of TDGIA, ATDGIA, MetaGIA, and AGIA.*\n\nWe have cited TDGIA in Section 2. \nThe other methods (ATDGIA, MetaGIA, and AGIA) are introduced in [1]. \nIn our experiments, we followed [1]'s implementation using a sequential optimization pipeline, which is detailed in the Appendix. \nWe will ensure that the citations and explanations are clarified in future versions of the paper.\n\n---\n\n**W2, Q5:** *Why attack performance decreases and then increases as the sparsity budget increases.*\n\nThis phenomenon is explained in lines 287-294 of the main text. \nLLMs prioritize maintaining coherence and fluency in their outputs. \nWhen the number of required words increases within a fixed text length, the Use Rate of these words decreases, leading to reduced accuracy in the embedding-text-embedding process. \nThis results in a decline in attack performance when the sparsity budget exceeds a certain threshold. \nAppendix J provides a more detailed illustration, quantifying coherence using Perplexity.\n\n---\n\n**W3, Q1:** *How graph structure is optimized.*\n\nThe LLM is not involved in the optimization of graph structures. \nAppendix I (Algorithm 4) provides pseudocode demonstrating that we rely on embedding-level techniques for selecting graph structures. \nAfter text generation, we employ structure optimization techniques from [1] to generate the graph structure. \nWe maintain a fixed feature set once generated and do not alter the pipeline. \nUsing LLMs for graph structure optimization is an intriguing direction for future research, but current studies are limited, and scalability issues need to be addressed.\n\n---\n\n**Q2:** *Transferring WTGIA to more fields.*\n\nThe WTGIA concept transforms text generation tasks into word-to-text tasks, enabling LLMs to effectively complete generation tasks while maintaining downstream classification performance. \nGiven the importance of classification in graph representation learning, we believe this approach simplifies the generation tasks for related fields.\n\n---\n\n**Q3:** *Embedding-Text-Embedding process.*\n\nOur paper focuses on the embedding-to-text process, introducing three text-level GIAs. The text-to-embedding process is necessary because, except for the LLM-as-predictor (Section 5), GNNs require converting injected text into embeddings as input. We utilize embedding methods like BoW and GTR to facilitate GNN testing on downstream tasks. During this process, the ITGIA method experienced significant accuracy loss. This loss occurs because, even with the same embedding method, the inverted text’s embedding differs significantly from the original. Section 3 discusses the reasons in detail, emphasizing the challenges of the ill-defined interpretable region.\n\n---\n\n**Q4:** *Use of FGSM*\n\nFGSM is a classic optimization method suitable for binary states (0 and 1). \nIt uses flipping for perturbation, preserving physical meaning, such as 0-1 for using a word and 1-0 for not using a word. \nAs a result, it facilitates the later embedding-to-text process in WTGIA.\n\nConverting continuous embeddings to binary is challenging and relies on text as a bridge. \nWhat's more, the accuracy of the embedding-to-text process is crucial, posing a challenge for continuous features.\n\n---\n\nThank you again for your valuable feedback.\n\n---\n\n**Reference**\n\n[1] Chen, et al. Understanding and improving graph injection attack by promoting unnoticeability.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IG1JRV8P4h", "reviewer": "Reviewer_Da4q", "summary": "The paper studies Graph Injection Attacks on text-attributed graphs. The study presents three attack designs: Vanilla Text-level GIA (VTGIA), Inversion-based Text-level GIA (ITGIA), and Word-frequency-based Text-level GIA (WTGIA). The key contributions include demonstrating the effectiveness of text level perturbation on Graph Injection Attack and the performance of using LLMs as a defender. WTGIA shows significant attack potential while maintaining understandability. It is a timely study on the vulnerabilities of text attributes in graphs.", "questions": "1. What is the computing complexity of WTGIA? How long does it take to generate an attack in average in your experiment environment? \n\n2. What is the impact of different text embedding models on the attack performance?", "limitations": "Limitations on transferability is discussed in the paper.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. Good investigation on the role of text manipulation in graph injection attacks. The text attributes in graphs are getting increasingly important. Demonstrating their vulnerabilities to GIA and the role of LLM can play as a defender are important to applications using text-attributed graphs.\n\n2. The proposed WTGIA method achieves good attack performance with understandable text perturbation.", "weaknesses": "1. The embedding-text-embedding process is not very clear to me. Additional explanation will be helpful. It is unclear how different embedding models may affect the performance of this process. \n\n2. The term interpretability in this paper has different semantics to the commonly used one in explainable AI. It is confusing and creates unnecessary obstacles in reading. It is better replace with understandability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful feedback on our paper. We have addressed your concerns and provided clarifications below.\n\n---\n\n**W1:** *More about the Embedding-Text-Embedding Process.*\n\n**Why Embedding-to-Text:** Traditional GIAs included only the text-to-embedding process, with the pipeline as follows: 1) Preprocess the original graph text to embeddings, denoted as A. \n2) The attacker designs an attack on this embedding by injecting abnormal embeddings, denoted as B. \n3) The defender evaluates the same embeddings (A and B). \nThere are several unreasonable aspects in this process, including: embedding B may not be reasonable at the text level (ITGIA); in real scenarios, attackers cannot directly inject embeddings into the defender's system, making step 3 overly idealistic for the attacker; and the defender can choose a different process in step 1 compared to the attacker. \nThese issues arise because the attacker did not consider the embedding-to-text process. \nOur paper focuses on supplementing this step, requiring the attacker to generate text at the text level, injecting the original text.\n\n**Why Text-to-Embedding:** The embedding-text-embedding process is ultimately to align with previous GNN evaluations, as the GNN ultimately uses embeddings as input. So we finally transform the generated text into embeddings for evaluation. An exception is the LLM-as-predictor in Section 5, which only includes the embedding-to-text process. In the future, we believe that studying graph attacks and defenses involving embedding-to-text will be a fascinating direction.\n\nWe will provide a clearer illustration of the Embedding-Text-Embedding process in the revised versions.\n\n---\n\n**W2:** *Use of the term Interpretability*\n\nThank you very much for your clarification, which helps avoid potential misunderstandings we might have caused. \nWe will consider clarifying the distinction between this term and interpretability in the field of explainability when it is first mentioned, emphasizing that it is more about understandability to humans.\n\n---\n\n**Q1:** *Running Time*\n\nWe have included the running time and discussion about complexity in the **author rebuttal**. \nIn summary, scalability is not a major issue in current text-level GIAs.\n\n---\n\n**Q2:** *Impact of Embedding Models*\n\nWe added a new embedding model, SentenceBert, and related experiments in the **author rebuttal**. \nOverall, the impact of text-embedding includes:\n1) **Impact on the Original GNN:** According to [1], different embeddings can cause performance differences in GNNs on clean datasets. For example, on the Reddit dataset, GTR can have more than a 5-point performance advantage over the original dataset, which is significant in some cases.\n2) **Impact on Attacker Design:** Designing a better attack for continuous embeddings can be more challenging. WTGIA shows some transferability across both, but the results are still unsatisfactory. Although GTR and SBERT are both PLM-based embeddings, the adversarial text by ITGIA performs even worse than WTGIA when transferred to SBERT. We believe that exploring this aspect remains a challenge.\n3) **Impact on Defender Design:** LayerNorm is effective in defending against continuous features because it can capture structural anomalies caused by attacks on continuous features, but it performs mediocrely with BoW methods. While EGNNGuard can be easily bypassed by HAO, it remains competitive when defending against WTGIA. Therefore, different defense methods may be suitable for different types of embeddings.\n\nTo sum up, there are still many areas worth exploring in this regard, and we believe considering more combinations of graph and text methods will be an interesting research direction in the future.\n\n---\n\nThank you once again for your valuable feedback and suggestions. We believe that incorporating these insights will significantly enhance the quality and impact of our work. We welcome any further questions or discussions.\n\n---\n\n**Reference**\n\n[1] Chen, et al. Exploring the potential of large language models (LLMs) in learning on graphs.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**Summary of Rebuttal**

Thanks to the reviewers for their diligent efforts and thorough evaluation. 
We are glad to receive such positive feedback. 
Notably, all reviewers acknowledged that conducting text-GIA is novel and meaningful. 
Reviewers nfHh and Da4q recognized our method's technical contributions, while Reviewer ZG7Z also appreciated our method's contribution to ensuring the semantic integrity of the injection text.

The reviewers' main concerns focused on the comprehensiveness of our evaluation. 
In response, during the rebuttal period, we provided additional analyses on defense methods, efficiency, results from more embedding methods, and experiments on additional datasets. 
Specifically, In the author rebuttal and the uploaded PDF file, we have added the following content:

---

**Experiments Against More Defense Models**

In **Tables 1 to 6 in the pdf**, we included classic methods like GAT and GraphSAGE, as well as the EGNNGuard and Layernorm (LN) methods, which have been proven highly effective in [1].

We discovered interesting new results, such as LN’s effectiveness on text-level GIAs not being as strong as previously reported for embedding-level attacks, especially for WTGIA. 
This is because embedding-level GIAs often exhibit abnormal norms, which LN exploits for defense. 
However, text-level GIAs derive embeddings from real text, avoiding structural anomalies and bypassing LN’s defenses. This suggests that some traditional defense methods that were particularly effective may be limited to the embedding level. 
The EGNNGuard method is generally effective and performs well overall. 
As mentioned in the main paper, attackers can use techniques like HAO to bypass EGNNGuard, but this often involves trade-offs.

While including more baselines could offer a more comprehensive assessment, it entails a substantial workload in the domain of Graph Adversarial Attacks. 
Based on our estimates, presenting results for **all combinations of datasets, attacks (both embedding-level and text-level), defenses, and hyperparameters would require over 100 tables**. 
Therefore, we have chosen to focus on the most representative examples to provide a clearer and more concise analysis.

---

**Running Time and Complexity**

In **Tables 7 and 8 in the pdf**, we included the running time of the largest dataset, PubMed, from the original paper. 
We also included results on ogbn-arxiv, which contains more than 160,000 nodes, to verify the scalability of our method.

In terms of graph structure, the text-level GIA proposed in the paper does not introduce additional complexity regarding graph structure. 
Based on the results in [1], even the most complicated variant, MetaGIA, has a time complexity bounded by 
$O(|V_T| N_{inj}^2 \log (|V_T| N_{inj}))$ and the space complexity is bounded by $O(|V_T| N_{inj})$

Considering that the number of injected nodes and edges is usually small compared to the original graph, current Text-GIAs are intrinsically scalable. 
For example, on ogbn-arxiv, as long as the most complex MetaGIA is not used as the backbone, other experiments can be effectively completed.

In terms of text generation, our methods are linear to the number of injected nodes and can be executed efficiently.
For all methods that rely on LLM, we can further improve efficiency using techniques like LLM's inference acceleration, parallel generation, and reducing the number of correction rounds. 

Therefore, scalability is not our bottleneck at present.

---

**More Embeddings**

In **Tables 9 and 10 in the pdf**, we included SentenceBert all-MiniLM-L6-v2 [2] as a new embedding backbone. 
We find that WTGIA shows some transferability between different embeddings, but the results are still unsatisfactory. Although GTR and SBERT are both PLM-based embeddings, the adversarial text by ITGIA performs even worse than WTGIA when transferred to SBERT. 
We believe the transferability of text-level GIAs remains a significant challenge.

---

**More Datasets**

In **Tables 11 and 12 of the PDF**, we included the ogbn-arxiv dataset [3], containing over 160,000 nodes, and a social network dataset, Reddit, with more than 30,000 nodes [4], to evaluate the scalability and generalization of text-level GIAs. 
We injected 1,500 nodes into ogbn-arxiv, following the budget specified in [1], and proportionally injected 500 nodes into Reddit.
Due to space constraints, we use WTGIA with average sparsity budgets, as it has the longest runtime and best performance.
 
On ogbn-arxiv, WTGIA maintains use rates above 90% and shows good attack performance when transferred to GTR. 
On Reddit, though effective on BoW embeddings, the transferability issue is more severe, indicating domain variability in graph attacks. 
With more datasets from various domains becoming available in the future, we look forward to exploring the differences in graph attacks and defenses across different domains.

---

**Update Plan**

We addressed other weaknesses and questions in each separate rebuttal.
We will incorporate the above content in the updated version of our paper. 
In addition, we will improve explanations of interpretability and the embedding-text-embedding process, correct typos, and add necessary citations based on the reviewers’ suggestions.

We sincerely thank the reviewers for their valuable feedback and constructive comments, which have significantly contributed to improving our work. 
If there are any further questions or if you would like additional clarification, we welcome further discussion and engagement.

---

**Reference**

[1] Chen, et al. Understanding and improving graph injection attack by promoting unnoticeability.

[2] Reimers, Nils, and Iryna Gurevych. Sentence-BERT: Sentence embeddings using Siamese BERT-networks.

[3] Hu, et al. Open graph benchmark: Datasets for machine learning on graphs.

[4] Huang et al. Can GNN be a good adapter for LLMs?

Meta-review: It seems to me that all major concerns of the reviewers have been addressed during the discussion period and that the paper merits acceptance at the NeurIPS conference. 

The reviews and the rebuttal have given rise to several interesting points and results that I encourage the authors to include in the camera-ready version. Especially, the multitude of additional experimental results that the authors added in the general author rebuttal would be a valuable addition to the paper, since the evaluation on only three datasets, seems to be insufficient to me too. I furthermore want to encourage the authors to add even more defense baseline models to their camera-ready version (as several reviewers also requested). While the new LayerNorm and EGNNGuard results provided in the rebuttal are interesting, I feel that they still do not do justice to the vast literature that exists on defense methodology to adversarial attacks on graph structure and node features. The impact of this paper could be improved if the method was benchmarked against a broader range of defense methodologies.
Decision: Accept (poster)