Adversarial Attacks and Defenses on Graph-aware Large Language Models

## Abstract

Abstract Large Language Models (LLMs) are increasingly integrated with graph-structured data for tasks like node classification, a domain traditionally dominated by Graph Neural Networks (GNNs). While this integration leverages rich relational information to improve task performance, their robustness against adversarial attacks remains unexplored. We take the first step to explore the vulnerabilities of graph-aware LLMs by leveraging existing adversarial attack methods tailored for graph-based models, including those for poisoning (training-time attacks) and evasion (test-time attacks), on two representative models, LLaGA (Chen et al. 2024 ) and GraphPrompter (Liu et al. 2024 ) . Additionally, we discover a new attack surface for LLaGA where an attacker can inject malicious nodes as placeholders into the node sequence template to severely degrade its performance. Our systematic analysis reveals that certain design choices in graph encoding can enhance attack success, with specific findings that: (1) the node sequence template in LLaGA increases its vulnerability; (2) the GNN encoder used in GraphPrompter demonstrates greater robustness; and (3) both approaches remain susceptible to imperceptible feature perturbation attacks. Finally, we propose an end-to-end defense framework GaLGuard , that combines an LLM-based feature correction module to mitigate feature-level perturbations and adapted GNN defenses to protect against structural attacks 1 1 1 Our code is available.

## 1 Introduction

Graphs, characterized by nodes and edges that represent entities and relationships, are used to model complex structures in various real-world domains, including social networks (Fan et al. 2019), biology (Dong, Mücke, and Khosla 2022), finance (Wang et al. 2021), and healthcare (Ahmedt-Aristizabal et al. 2021). Graph Neural Networks (GNNs) (Hamilton, Ying, and Leskovec 2017; Kipf and Welling 2022; Veličković et al. 2018), designed specifically for graph-structured data, excel in tasks such as node classification and link prediction. Recently, the success of large language models (LLMs) such as GPT-4 (Achiam et al. 2023),
Gemini (Team et al. 2023), LLaMA (Touvron et al. 2023; Dubey et al. 2024), Vicuna (Chiang et al. 2023), and PaLM-2 (Anil et al. 2023), trained on vast amounts of textual data, has sparked interest in their potential to enhance graph-related tasks. This fusion of graph data with LLMs (called graph-aware LLMs (Xie et al. 2023)) has opened a new line of research, where LLMs are increasingly being utilized to perform tasks traditionally handled by GNNs (Ye et al. 2023; Fatemi, Halcrow, and Perozzi 2023; Kuang et al. 2023; Zhang et al. 2023; Chen et al. 2023b, 2024).
In fact, recent research has shown that graph-aware LLMs can outperform GNNs on various graph-related tasks (Wenkel, Wolf, and Knyazev 2023; Chen et al. 2023b; Zhu et al. 2024; Chen et al. 2024), positioning them as powerful alternatives.

Despite their growing popularity, the vulnerabilities of graph-aware LLMs to attacks remain largely unexplored.
Existing adversarial attacks on GNNs, including those for poisoning (training-time attacks) and evasion (test-time attacks) (Zügner, Akbarnejad, and Günnemann 2018; Zügner and Günnemann 2019; Dai et al. 2023), operate under a black-box setting where the attacker has no knowledge of the model’s internal workings. These attacks undermine model integrity by intentionally degrading model performance by subtly altering graph structure or node features. While GNN research has predominantly focused on structural attacks, the textual nature of graph-aware LLMs makes imperceptible perturbations to node features a significant concern that has been largely overlooked.

Motivated by these three key observations: the shift from GNNs to LLMs for graph tasks, the black-box nature of current adversarial attacks on GNNs, and the potentially new vulnerabilities to feature perturbations, we ask the critical question of whether adversarial attacks designed for GNNs remain effective against the emerging graph-aware LLM paradigm.
In addition, the success of graph-aware LLMs in handling graph tasks highly depends on the effective integration of graphs into the LLM. While processing the node features is straightforward given LLM’s proficiency in handling textual data, incorporating the graph structure is more complex and requires innovative techniques to capture relational information (Chen et al. 2024; Guo et al. 2023; Liu and Wu 2023; Zhang et al. 2023; Sun et al. 2023; Bi et al. 2024; Pan et al. 2024).
Therefore, to effectively answer our research question, we first categorize existing graph-aware methods into two classes based on their approach to encoding graphs into LLMs; one that represents graph data as textual descriptions, converting graph information into natural language, and another that utilizes learned projectors to bridge the inherent differences between graph and text domains. While conventional text-based adversarial attacks on LLMs should generally suffice to compromise LLMs handling textual descriptions of graphs, we focus our investigation on learned projector methods, using two computationally efficient representative models, LLaGA (Chen et al. 2024) and GraphPrompter (Liu et al. 2024), which both utilize frozen pre-trained LLM.

Our analysis reveals that certain design choices in graph encoding for LLMs can unexpectedly enhance the success of attacks. Notably, we identify a new attack surface in LLaGA that could allow an attacker to severely degrade the model’s performance by injecting malicious nodes as placeholders within the node sequence template. In addition, we propose an end-to-end graph-aware LLM defense, GaLGuard, which integrates an LLM-based feature corrector with adapted GNN-based structural defenses to provide robust protection. We are the first to investigate the vulnerability of graph-aware LLMs to adversarial attacks.
The contributions of the paper are summarized as follows:

- •
We leverage representative adversarial attacks for GNNs, specifically Nettack (Zügner, Akbarnejad, and Günnemann 2018) and MetaAttack (Zügner and Günnemann 2019), to investigate the transferability of poisoning and evasion attacks to graph-aware LLMs.
- •
We create a taxonomy for graph-aware LLMs based on their encoding approaches to understand how different methodologies integrate graph structures into LLMs.
- •
We discover a new attack surface in LLaGA, showing that injecting malicious nodes as placeholders into the node sequence template can degrade its performance.
- •
We provide a holistic assessment of the vulnerabilities of graph-aware LLMs by performing imperceptible attacks on textual node features, a perspective often overlooked in GNNs.
- •
We propose GaLGuard, a novel end-to-end defense strategy that integrates LLM-based feature correction with the adaptation of existing GNN-based structural defenses.

## 2 Graph Encoding-Based Taxonomy of LLM Adaptations

Figure: Figure 1: Graph encoding-based adaptations of LLMs. In the textual description graph representation, graphs are explained textually (top part). In the learned projector representation, graphs are encoded either by node templates that turns the graph into sequences or by GNNs (bottom part). Note that in both cases, the LLM is frozen. Only the projector that maps the graph into tokens is trained.
Refer to caption: x1.png

Graph encoding methods for LLMs can be categorized into two main approaches: textual descriptions of graphs (where graph information is encoded as natural language) and learned graph projectors (where a projector is utilized to encode graph data into embeddings or structured forms that LLMs can directly process). We briefly describe the different encoding approaches in each category. [Figure 5](https://arxiv.org/html/2508.04894v1#A1.F5) provides the illustration of both approaches. Full description in Appendix A.1 and related work on GNN attacks in Appendix A.2.

##### Graph Representation as a Textual Description.

This approach converts graph data into natural language, enabling LLMs to process them as textual descriptions, often with specific instructions to query LLMs (Guo et al. 2023; Liu and Wu 2023; Ye et al. 2023; Wenkel, Wolf, and Knyazev 2023; Wei et al. 2024b; Fatemi, Halcrow, and Perozzi 2023; Tan et al. 2023; Zhao et al. 2023; Wei et al. 2024a; Zhang 2023). In multimodal applications, Liu et al. (2023) and Su et al. (2022) linked molecule graphs with textual descriptions. Furthermore, Huang et al. (2023a) investigated incorporating graph structure into LLM prompts. Their findings revelas that LLMs tend to process graph-related prompts as contextual paragraphs, underscoring the importance of prompt design, as seen in studies like (Brannon et al. 2023; Zhang et al. 2023; Sun et al. 2023; Bi et al. 2024; Pan et al. 2024; He et al. 2024). Other research has focused on using LLMs to enhance node features, graph structures, act as predictors, or serve as feature extractors for GNNs (Chen et al. 2023a; Chandra et al. 2020; Chen et al. 2023b; Mavromatis et al. 2023). For inference, Zhu et al. (2024) proposed a fine-tuning method for textual graph that combined LLMs and GNNs, and Duan et al. (2023) introduced an effective approach that enhances textual graph learning. While these methods offer a straightforward way to use LLMs for graph tasks, they often struggle to fully capture structural intricacies (especially for large graphs) and are heavily dependent on prompt engineering.

##### Graph Representation as a Learned Projector.

To address the limitations of textual representations, learned projector methods transform graph data into embeddings or graph vectors that LLMs can process more directly. This includes methods like LLaGA, which uses node-level templates and a linear projector (Chen et al. 2024), and others that incorporate GNN layers into a pre-trained LLM (Liu et al. 2024; Qin et al. 2023). Other approaches use graph neural prompts to encode knowledge graphs (Tian et al. 2024), develop prompt-based node feature extractors with a GNN adapter (Huang et al. 2023b), or condense graph information into fixed-length prefixes using graph transformers (Chai et al. 2023; Yang et al. 2021; Peng et al. 2024).

This Work: We focus on projector methods that utilize frozen pre-trained LLMs because they are computationally efficient and avoid the substantial costs of retraining (Chen et al. 2024; Liu et al. 2024; Qin et al. 2023; Tian et al. 2024; Huang et al. 2023b). For our investigation, we use two representative models: LLaGA (Chen et al. 2024), which employs node-level templates with a linear projector, and GraphPrompter (Liu et al. 2024), which uses a GNN with a linear projector. We do not consider textual description methods, as they often struggle with scalability and do not introduce the novel, graph-specific vulnerabilities we aim to investigate.

## 3 Threat Model

We characterize our threat model based on established research on adversarial attacks in graphs, and extend its applicability to graph-aware LLMs (Zügner, Akbarnejad, and Günnemann 2018; Wu et al. 2019; Zügner and Günnemann 2019; Ma, Ding, and Mei 2020; Jin et al. 2021; Dai et al. 2023; Li et al. 2023). For example, in social networks, attackers often control only a few bot accounts; this limited access aligns with our threat model, where adversaries manipulate a small subset of nodes to evade detection or influence legitimate user classification. Another example is Wikipedia, where hoax articles with sparse connections to legitimate ones can be strategically linked to manipulate the classification of genuine content, causing misclassification of real articles’ categories. This transferability is particularly concerning as these advanced models have the potential of being deployed in critical applications such as recommendation system, social media analysis or even healthcare, potentially replacing GNNs. Thus, following existing works on adversarial attacks in graphs, we characterize the threat model along three dimensions; model access, adversarial capabilities, and attack strategies, followed by defining the attacker’s goal.

##### Model Access.

The attacker operates in a black-box setting, with no knowledge of the model’s architecture or parameters. The attacker can only influence the model indirectly by interacting with the training or test data.

##### Adversarial Capabilities.

Consistent with existing poisoning (evasion) attacks, the attacker has complete access to the training (inference) data of the target model. Within a predefined budget and under unnoticeability constraints (aiming to avoid detection), the attacker can modify or inject data.

##### Attack Strategies.

The attacker leverages two primary strategies: feature manipulation (altering node features) and structure manipulation (adding or removing edges).

##### Attacker’s Goal.

The attacker’s goal is to compromise model integrity by degrading the overall performance, specifically by increasing the misclassification rate of the classification model.

## 4 Experimental Setup

##### Dataset.

We utilize three widely used graph datasets: Cora, Citeseer, and PubMed (Yang, Cohen, and Salakhudinov 2016). These text-attributed graph datasets, which vary in size and sparsity from small to medium scales, are common benchmarks for models like those in our study (Chen et al. 2023a, 2024; Liu et al. 2024). To further validate scalability, we also include the larger ArXiv dataset (Hu et al. 2020) in an additional experiment (see Appendix A.5).
We adopt standard train/validation/test splits of 6:2:2 for the three smaller datasets and 5:2:3 for ArXiv, as in (Chen et al. 2024). Detailed statistics are provided in [Table 1](https://arxiv.org/html/2508.04894v1#S4.T1).

**Table 1: Dataset statistics.**
| Data | #Node | #Edge | Sparsity (‰) |
| --- | --- | --- | --- |
| Cora | 2708 | 5429 | 14.81 |
| CiteSeer | 3327 | 4614 | 8.34 |
| PubMed | 19717 | 44338 | 2.28 |

##### Tasks.

We use node classification as our primary task, a widely used method for evaluating graph machine learning models and the primary setting for adversarial attacks (Zügner, Akbarnejad, and Günnemann 2018; Zügner and Günnemann 2019; Liu and Wu 2023).

##### Evaluation Metrics.

We employ accuracy as the evaluation metric, , which measures the proportion of correctly predicted labels. Each experiment is repeated three times, and we report the mean and standard deviation. All experiments were conducted on a machine equipped with NVIDIA A40 GPU with 48GB of memory.

**Table 2: Performance of Nettack and MetaAttack on Cora, Citeseer, and PubMed datasets for graph-aware LLMs LLaGA and GraphPrompter.**
| Attack | Type | Cora | Citeseer | PubMed |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| LLaGA | GraphPrompter | LLaGA | GraphPrompter | LLaGA | GraphPrompter |  |  |
| Clean | — | $0.89_{\pm 0.07}$ | $0.60_{\pm 0.03}$ | $0.71_{\pm 0.04}$ | $0.70_{\pm 0.03}$ | $0.90_{\pm 0.04}$ | $0.90_{\pm 0.02}$ |
| Nettack | Poisoning | $0.87_{\pm 0.04}$ (2%) | $0.60_{\pm 0.03}$ (0%) | $0.64_{\pm 0.05}$ (10%) | $0.68_{\pm 0.02}$ (3%) | $0.89_{\pm 0.03}$ (1%) | $0.88_{\pm 0.02}$ (2%) |
| Evasion | $0.55_{\pm 0.09}$ (38%) | $0.53_{\pm 0.05}$ (12%) | $0.59_{\pm 0.04}$ (19%) | $0.61_{\pm 0.03}$ (14%) | $0.84_{\pm 0.05}$ (7%) | $0.81_{\pm 0.02}$ (10%) |  |
| MetaAttack | Poisoning | $0.79_{\pm 0.03}$ (11%) | $0.58_{\pm 0.05}$ (3%) | $0.63_{\pm 0.04}$ (12%) | $0.65_{\pm 0.06}$ (7%) | $0.89_{\pm 0.08}$ (1%) | $0.85_{\pm 0.02}$ (6%) |
| Evasion | $0.44_{\pm 0.06}$ (51%) | $0.52_{\pm 0.04}$ (13%) | $0.50_{\pm 0.03}$ (35%) | $0.56_{\pm 0.02}$ (22%) | $0.73_{\pm 0.04}$ (19%) | $0.77_{\pm 0.03}$ (14%) |  |

##### Choice of Graph-aware LLM and Adversarial Attack Methods.

We clarify our selection of graph-aware LLM methods and adversarial attack methods used to investigate the vulnerabilities of graph-aware LLMs.

##### Graph-Aware LLM Methods.

We employ two representative graph-aware LLM methods: LLaGA (Chen et al. 2024) and GraphPrompter (Liu et al. 2024). Both methods utilize a frozen pre-trained LLM, making them computationally efficient and adaptable to various graph tasks and LLM architectures. LLaGA uses node-level templates with a linear projector, while GraphPrompter employs a GNN-based encoder and a linear projector. In both cases, the projector is a 2-layer multi-layer perceptron (MLP). he foundational LLMs for LLaGA and GraphPrompter are Vicuna-7B (Chiang et al. 2023) and LLAMA2-7B (Touvron et al. 2023), respectively.

##### Adversarial Attack Methods.

We employ the poisoning and evasion attack from two representative adversarial attack methods Nettack (Zügner, Akbarnejad, and Günnemann 2018) and MetaAttack (Zügner and Günnemann 2019)
for conducting our investigation. Nettack is a targeted attack that finds minimal perturbations to misclassify specific nodes, while MetaAttack performs untargeted attacks by treating the graph structure as a hyperparameter to optimize. For Nettack, we adopt the approach of (Li et al. 2023; Zügner and Günnemann 2019) by sequentially targeting each node and aggregating the results, effectively transforming it into an untargeted evaluation. For all experiments, we perturb only 10% of the edges to maintain the unnoticeability constraint, as in (Zügner, Akbarnejad, and Günnemann 2018; Zügner and Günnemann 2019). Full details in Appendix A.2.

## 5 Results

### 5.1 Poisoning and Evasion Attacks on Graph-aware LLMs

We analyze poisoning and evasion attacks by adapting methods from Nettack (Zügner and Günnemann 2019) and MetaAttack (Zügner and Günnemann 2019). For poisoning attacks on LLaGA, we use a perturbed graph to generate the node-level sequences and then retrain the projector. For GraphPrompter, we use the perturbed graph to re-encode the graph using a GAT and retrain its projector. For evasion attacks, both models are trained on unperturbed data, and perturbations are introduced only during the inference phase.

##### Results.

Our experiments reveal several key insights, as summarized in [Table 2](https://arxiv.org/html/2508.04894v1#S4.T2). First, we observe that poisoning attacks have a limited impact on graph-aware LLMs, with a maximum performance degradation of 11% on both attack methods. This suggests that the models’ robust representations and reliance on additional context from the LLM provide resilience. In contrast, evasion attacks prove highly effective, causing significant performance degradation of up to 51%. Secondly, for the different adversarial attack methods, MetaAttack generally outperforms Nettack for both poisoning and evasion attacks across all datasets. This is attributed to MetaAttack’s ability to create small but highly effective perturbations by treating graph edges as hyperparameters. Finally, we observe that denser datasets are more vulnerable to attacks than sparse ones. For instance, LLaGA achieved a 51% performance decrease on the denser Cora during an evasion attack, but only a 19% decrease on the sparser PubMed dataset.

### 5.2 Which Graph-aware Paradigm is More Susceptible to Graph Adversarial Attacks and Why?

Our experiments ([Table 2](https://arxiv.org/html/2508.04894v1#S4.T2)) show that LLaGA is more susceptible to attacks than GraphPrompter due to its sensitivity to structural perturbations.
LLaGA integrates the graph structure in two ways: by generating a fixed-length node-level sequence for each node and by computing a graph Laplacian embedding. Each position in this sequence uniquely corresponds to a relative structural position within the original graph. Since the node embeddings are directly derived from the perturbed graph structure, these attacks significantly influence the information processed by the LLM, leading to performance degradation. In contrast, GraphPrompter’s textual node embeddings remain unchanged during structural perturbations, as only the GNN component is affected. This design allows the unperturbed textual features to augment the perturbed graph structure, enhancing robustness.

## 6 Node Sequence Template Injection Attacks

We demonstrate a new attack surface in LLaGA. Specifically, by exploiting the neighborhood detail template used to construct the node sequences. The attack is illustrated in [Figure 2](https://arxiv.org/html/2508.04894v1#S6.F2) and the algorithm is in Appendix A.4.
To obtain the node sequence for a particular node $u$, the neighborhood detail template in LLaGA first constructs a fixed-shape computational tree for each node, where the children of a node are its sampled 1-hop neighbors. This process is repeated to sample neighbors for each subsequent level of the tree. If a node has an insufficient number of neighbors, a placeholder is used to maintain the fixed size of e.g. $k=10$ children, which is consistent with LLaGA’s original design (see ”Clean” from [Figure 2](https://arxiv.org/html/2508.04894v1#S6.F2) and lines 1-14 in the algorithm). This fixed structure provides a new attack surface.

Figure: Figure 2: Illustration of the node sequence template injection attacks. The attack shown here is the Non-adjacent Injection attack (NI). The red dashed lines are the edge perturbation by the attacker. For the Supernode Injection (SI) and Multiple Supernode Injection (MSI) attack, the supernode(s) are injected in the place of the placeholders. Here, $k=3$.
Refer to caption: x2.png

We demonstrate a new attack surface in LLaGA by exploiting its neighborhood detail template algorithm. An attacker can craftily inject malicious nodes as placeholders through graph manipulation to significantly degrade model performance. We propose three strategies for this attack. The first, termed Non-Adjacent Injection (NI), involves injecting nodes that are not within the 2-hop neighborhood of a target node $u$ (see ”Attacked” from [Figure 2](https://arxiv.org/html/2508.04894v1#S6.F2) and lines 21-27). The second approach, referred to as Supernode Injection (SI), involves injecting a high-degree node as a placeholder, ensuring it is outside the target node’s 2-hop neighborhood. The third, Multiple Supernode Injection (MSI) is a variant of SI that injects multiple distinct supernodes as placeholders (lines 35-43). The rationale is that the inclusion of uninformative nodes in the node sequence corrupts the representation of the target node, leading to a poorly trained projection layer. This compromises the integrity of the model’s learning and inference process.

##### Results.

As shown in [Table 3](https://arxiv.org/html/2508.04894v1#S6.T3), LLaGA shows high resilience to poisoning attacks on the Cora and Citeseer datasets, but suffers a significant performance drop during evasion, with accuracies decreasing to $0.42$, $0.36$, and $0.30$ for Cora and to $0.36$, $0.30$, and $0.24$ for Citeseer on the NI, SI, and MSI attacks respectively.

Conversely, on the larger PubMed dataset, the model demonstrates a stronger resilience to all injection attacks, with only a modest drop in accuracy for evasion ($0.85$, $0.82$, and $0.76$). One reason is that in smaller, more tightly-connected graphs like Cora and Citeseer, a localized injection has a more concentrated impact, severely disrupting the model’s ability to reason about the local neighborhood. In contrast, on the much larger PubMed graph, the effect of the same attack is diluted, leading to a less significant performance degradation.

Compared to traditional graph adversarial methods, node injection attacks are more effective. For instance, the MSI method achieved evasion attack accuracies of $0.30$, $0.24$ and $0.76$ on Cora, CiteSeer and PubMed, respectively, while MetaAttack only reached $0.44$, $0.50$ and $0.73$ ([Table 2](https://arxiv.org/html/2508.04894v1#S4.T2)).

**Table 3: Results of the node sequence template injection attack across different datasets and methods. NI, SI, and MSI indicates the non-adjacent, single supernode, and multiple supernodes injection attacks respectively. Clean refers to the performance of the model without any attack.**
| Data | Attack | NI | SI | MSI |
| --- | --- | --- | --- | --- |
| Cora | Clean | ——– $0.89_{\pm 0.07}$ ——– |  |  |
|  | Poisoning | $0.86_{\pm 0.03}$ | $0.84_{\pm 0.06}$ | $0.82_{\pm 0.04}$ |
|  | Evasion | $0.42_{\pm 0.08}$ | $0.36_{\pm 0.05}$ | $0.30_{\pm 0.05}$ |
| Citeseer | Clean | ——– $0.71_{\pm 0.04}$ ——– |  |  |
|  | Poisoning | $0.69_{\pm 0.05}$ | $0.66_{\pm 0.03}$ | $0.62_{\pm 0.02}$ |
|  | Evasion | $0.36_{\pm 0.06}$ | $0.30_{\pm 0.04}$ | $0.24_{\pm 0.05}$ |
| PubMed | Clean | ——– $0.90_{\pm 0.04}$ ——– |  |  |
|  | Poisoning | $0.88_{\pm 0.04}$ | $0.85_{\pm 0.05}$ | $0.81_{\pm 0.03}$ |
|  | Evasion | $0.85_{\pm 0.07}$ | $0.82_{\pm 0.04}$ | $0.76_{\pm 0.03}$ |

## 7 Poisoning and Evasion Attack via Feature Perturbation

Existing graph adversarial attacks primarily focus on structural perturbations (Li et al. 2023; Wu et al. 2019; Dai et al. 2018; Sun et al. 2022; Zügner and Günnemann 2019), with the exception of Nettack (Zügner, Akbarnejad, and Günnemann 2018), which is not transferable to the textual domain. Given that structural attacks are only moderately successful on robust models like GraphPrompter, and that textual features have proven to be important for model robustness, we explore unnoticeable feature perturbation attacks to degrade model performance.

We adopt the imperceptible attack strategies proposed by (Boucher et al. 2022) but with a key modification: we use a task-agnostic translation objective (English to French) to generate perturbations. The objective is to modify the input text such that the changes are visually identical but lead to an incorrect translation, thereby corrupting the encoded semantics. The intuition is that if the translation for a given text is wrong, the encoded semantics would also be wrong. We employ two imperceptible attacks: Homoglyphs, which replaces characters with visually similar ones and Reorderings, which use direction control characters to alter the rendering sequence. Examples of Homoglyphs and Reorderngs are in Appendix A.6.

Following (Boucher et al. 2022), we use differential evolution (Storn and Price 1997) to optimize these perturbations, maximizing the Levenshtein distance between the model’s output on the perturbed and unperturbed text. The perturbation budget is set to 10% of the average length of each textual feature.

##### Results.

As shown in [Table 5](https://arxiv.org/html/2508.04894v1#S7.T5), Reordering attacks are generally more potent than Homoglyph attacks across all models and datasets. On Cora and CiteSeer, LLaGA shows vulnerability to both attack types, with poisoning attacks causing accuracy drops of $27\%$ and $23\%$ for Homoglyph and a more significant $35\%$ and $25\%$ for Reordering, respectively. In both cases, GraphPrompter proves more resilient. The PubMed dataset exhibits similar trends, though the overall impact is less pronounced than on the smaller datasets.

A comparison with structural perturbations reveals a fundamental shift in vulnerabilities, as shown in [Figure 3](https://arxiv.org/html/2508.04894v1#S7.F3). Feature perturbation attacks (specifically Reordering) outperform structural attacks (MetaAttack) by a large margin on both LLaGA and GraphPrompter. This suggests that graph-aware LLMs place a greater emphasis on input features than on graph structure, making them more vulnerable to feature perturbations. We further explore this by combining both attack types in a unified setting in the next section.

**Table 4: Performance of feature perturbation attacks, with percentage drop from clean accuracy in parentheses.**
| Data | Method | Poisoning | Evasion |  |  |
| --- | --- | --- | --- | --- | --- |
| LLaGA | GraphPrompter | LLaGA | GraphPrompter |  |  |
| Cora | Homoglyph | $0.65_{\pm 0.05}$ (27%) | $0.52_{\pm 0.04}$ (13%) | $0.41_{\pm 0.04}$ (54%) | $0.38_{\pm 0.03}$ (37%) |
| Reordering | $0.58_{\pm 0.06}$ (35%) | $0.45_{\pm 0.05}$ (25%) | $0.24_{\pm 0.05}$ (73%) | $0.20_{\pm 0.03}$ (67%) |  |
| Citeseer | Homoglyph | $0.54_{\pm 0.04}$ (23%) | $0.65_{\pm 0.04}$ (7%) | $0.32_{\pm 0.03}$ (55%) | $0.52_{\pm 0.05}$ (26%) |
| Reordering | $0.46_{\pm 0.03}$ (35%) | $0.39_{\pm 0.02}$ (44%) | $0.20_{\pm 0.05}$ (72%) | $0.27_{\pm 0.02}$ (61%) |  |
| PubMed | Homoglyph | $0.72_{\pm 0.04}$ (20%) | $0.83_{\pm 0.04}$ (8%) | $0.36_{\pm 0.06}$ (60%) | $0.55_{\pm 0.03}$ (39%) |
| Reordering | $0.67_{\pm 0.05}$ (26%) | $0.80_{\pm 0.03}$ (11%) | $0.28_{\pm 0.04}$ (69%) | $0.42_{\pm 0.04}$ (53%) |  |

Figure: (a) LLaGA
Refer to caption: x3.png

## 8 A Unified Imperceptible Attack on Graph-aware LLMs

We propose a unified attack that combines both feature and structural perturbations to create a more imperceptible and effective adversarial strategy. We hypothesize that combining these attack types will more significantly degrade model accuracy than either method alone, while maintaining imperceptible alterations to the graph structure and features.

For the structural perturbation, we use MetaAttack, which modifies graph structure by subtly adjusting edges. For the feature component, we use a combination of Homoglyph and Reordering attacks, as described in previous section. Together, these perturbations create a highly imperceptible attack that simultaneously targets the relational and textual information on which graph-aware LLMs rely.

##### Results.

As shown in [Table 5](https://arxiv.org/html/2508.04894v1#S7.T5), the unified attack approach, which jointly applies structural and feature perturbations, is significantly more effective than either perturbation method alone in degrading model performance across various datasets. This analysis offers a key insight into the architecture of graph-aware LLMs: unlike traditional GNNs, which are primarily impacted by structural perturbations, these models exhibit substantial vulnerability to feature-level attacks. The effectiveness of the unified attack indicates that these models rely on the consistency of both graph structure and node features, making them highly susceptible to such combined, inconspicuous modifications.

## 9 Ablation Study

### 9.1 Varying Feature Perturbation

To better understand the relationship between perturbation level and attack success, we study the impact of varying the percentage of perturbed features. Our analysis of the Reordering attack, which exhibits similar patterns to the Homoglyph attack, reveals a consistent trend as the perturbation percentage varies from $1\%$ to $10\%$. For all models and datasets (Cora, Citeseer, and PubMed), reducing the percentage of perturbed features leads to a less successful attack, as expected. Notably, as the perturbation level decreases, the success rates of poisoning and evasion attacks tend to converge.
This is because with fewer perturbations, the adversarial signal becomes too weak to significantly impact the model, regardless of whether it was introduced during the training (poisoning) or inference (evasion) phase. A detailed plot illustrating these trends is in the Appendix (Figure 2).

### 9.2 Does the Choice of LLM Affect Attack Performance?

We evaluate whether the foundation LLM influences adversarial robustness in graph-aware LLMs. LLaGA uses Vicuna-7B (Chiang et al. 2023) and GraphPrompter uses Llama2-7B (Touvron et al. 2023). We replace LLaGA’s backbone with Llama2-7B and compare performance under poisoning and evasion attacks on all datasets.
Both models achieve similar accuracy in clean settings (e.g. Vicuna: $0.89\pm 0.07$, Llama2: $0.90\pm 0.03$ on Cora) and exhibit comparable vulnerability under attacks. The trend holds when swapping LLMs in GraphPrompter. These results suggest that the choice of foundation LLM has minimal impact on adversarial robustness. The full results are in Table 2 of the Appendix.

Figure: Figure 4: Overview of GaLGuard defense. Perturbed node features and edges are colored in blue and red, respectively. The LLM identifies perturbations in node features extracted from an adversarial graph and generates a corrected version. The refined features, together with the perturbed edges, are then passed through existing GNN defenses, which can either be used to obtain a sanitized graph or adapted for projectors’ robustness training.
Refer to caption: x5.png

## 10 Defense: GaLGuard

Existing GNN defenses against adversarial attacks are ineffective for graph-aware LLMs due to their dual vulnerability to structural and feature perturbations. Many defenses rely on node feature similarity to prune or reweight edges (Wu et al. 2019; Dai et al. 2018; Zhang and Zitnik 2020; Jin et al. 2020; Zhang and Ma 2024), but they assume access to clean features. When features are perturbed, these methods fail (see Figure 3 in the Appendix)
Similarly, defenses that decouple structure and features (Shanthamallu, Thiagarajan, and Spanias 2021; Wu et al. 2021) and even OCR-based sanitization (Boucher et al. 2022) break down under coordinated attacks. This dual vulnerability necessitates a new, integrated defense.

To address this, we propose graph-aware LLM defense (GaLGuard), an end-to-end defense strategy combining two complementary approaches: (1) an LLM-based feature corrector and (2) adapted GNN-based structural defenses.

##### Feature Robustness Strategy.

As shown in [Figure 4](https://arxiv.org/html/2508.04894v1#S9.F4), an LLM serves as an adaptive mechanism to analyze textual node features for anomalous patterns. It provides approximate corrections to restore feature integrity, thereby mitigating the effect of feature-level perturbations. This LLM-based correction is a practical solution for feature vulnerabilities. The specific instructions and prompts are in Appendix A.6.
For our experiments, we employ GPT-4 Turbo as the LLM corrector.

##### Graph Structure Robustness Strategy.

To enhance structural robustness, we employ a two-step approach. First, we adopt a graph purification heuristic (Wu et al. 2019) as a preprocessing step. For each node $v$, we compute the cosine similarity between its embedding $\phi(x_{v})$ and its neighbors’ embeddings $\phi(x_{u})$ and remove edges with low similarity. Second, we integrate GNNGuard (Zhang and Zitnik 2020). We directly apply GNNGuard’s principles to the message-passing layers of GraphPrompter. For LLaGA, we propose a novel adaptation: we incorporate the purification step into the construction of the computational tree for its node sequence. Furthermore, we introduce a learnable global structural context embedding ($M_{global}$) as a learnable parameter in LLaGA’s projector. By concatenating $M_{global}$ with the purified node sequence embedding and jointly training them, the model learns to operate robustly on potentially perturbed graph inputs. The plug-and-play design of GaLGuard’s allows other defenses, e.g. Pro-GNN (Jin et al. 2020), to be used instead of GNNGuard.

### 10.1 Results

Table [6](https://arxiv.org/html/2508.04894v1#S10.T6) summarizes the performance, measured in accuracy, of models under MetaAttack, as well as our defense approaches: GaLGuardp, which combines a feature corrector with a graph purification module, and the full GaLGuard, which integrates the feature corrector, the graph purification module, and GNNGuard (or its adaptation).
Across all evaluated datasets (Cora, CiteSeer, and PubMed) and for both poisoning and evasion attack types, our defenses consistently mitigate the impact of MetaAttack. Specifically, GaLGuardp proves to be an effective foundational step, demonstrating that addressing feature perturbations with the LLM corrector and applying initial graph purification significantly improve robustness. Notably, the full GaLGuard consistently yields superior improvements in model robustness compared to GaLGuardp, highlighting the synergistic effect of integrating the GNN-inspired structural defense.

**Table 6: Performance of MetaAttack and the proposed defense GaLGuard under poisoning and evasion attacks. GaLGuardp combines feature correction with graph purification**
| Data | Method | LLaGA | GraphPrompter |  |  |
| --- | --- | --- | --- | --- | --- |
|  |  | Poisoning | Evasion | Poisoning | Evasion |
| Cora | MetaAttack | $0.79_{\pm 0.03}$ | $0.44_{\pm 0.06}$ | $0.58_{\pm 0.05}$ | $0.52_{\pm 0.04}$ |
| GaLGuardp | $0.82_{\pm 0.05}$ | $0.62_{\pm 0.04}$ | $0.59_{\pm 0.03}$ | $0.56_{\pm 0.04}$ |  |
|  | GaLGuard | $\mathbf{0.85_{\pm 0.04}}$ | $\mathbf{0.83_{\pm 0.04}}$ | $\mathbf{0.60_{\pm 0.05}}$ | $\mathbf{0.59_{\pm 0.03}}$ |
| CiteSeer | MetaAttack | $0.63_{\pm 0.04}$ | $0.50_{\pm 0.03}$ | $0.65_{\pm 0.06}$ | $0.56_{\pm 0.03}$ |
| GaLGuardp | $0.64_{\pm 0.03}$ | $0.56_{\pm 0.02}$ | $0.67_{\pm 0.05}$ | $0.60_{\pm 0.06}$ |  |
|  | GaLGuard | $\mathbf{0.67_{\pm 0.04}}$ | $\mathbf{0.62_{\pm 0.04}}$ | $\mathbf{0.68_{\pm 0.04}}$ | $\mathbf{0.64_{\pm 0.05}}$ |
| PubMed | MetaAttack | $0.89_{\pm 0.08}$ | $0.73_{\pm 0.04}$ | $0.85_{\pm 0.02}$ | $0.77_{\pm 0.03}$ |
| GaLGuardp | $0.90_{\pm 0.06}$ | $0.78_{\pm 0.03}$ | $0.85_{\pm 0.04}$ | $0.81_{\pm 0.04}$ |  |
|  | GaLGuard | $\mathbf{0.90_{\pm 0.01}}$ | $\mathbf{0.87_{\pm 0.02}}$ | $\mathbf{0.89_{\pm 0.02}}$ | $\mathbf{0.89_{\pm 0.01}}$ |

## 11 Conclusion

In this paper, we take a substantial first step in uncovering the vulnerabilities of graph-aware LLMs to adversarial attacks, a largely unexplored area. Our work reveals that the methods used for integrating graph data are a critical source of vulnerability. We discovered new attack surfaces, demonstrating that design choices like the node sequence template in LLaGA significantly increase a model’s susceptibility. Furthermore, our findings show that feature perturbation attacks are highly effective (unlike traditional GNNs), often outperforming traditional structural attacks, and that a unified attack combining both perturbations can render graph-aware LLMs near-useless.

To address this, we propose an end-to-end defense, GaLGuard, which integrates an LLM-based feature corrector with adapted GNN-based structural defenses to provide robust protection. As these models become more prevalent and has potentials in applications spanning social networks, healthcare, and finance, ensuring their resilience to adversarial exploitation is of the essence. We hope this work serves as a foundation for further research, inspiring deeper exploration into the vulnerabilities and defenses of graph-aware LLMs and fostering both theoretical insights and practical safeguards for their real-world deployment.

## References

- Achiam et al. (2023)
Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023.
Gpt-4 technical report.
*arXiv preprint arXiv:2303.08774*.
- Ahmedt-Aristizabal et al. (2021)
Ahmedt-Aristizabal, D.; Armin, M. A.; Denman, S.; Fookes, C.; and Petersson, L. 2021.
Graph-based deep learning for medical diagnosis and analysis: past, present and future.
*Sensors*, 21(14): 4758.
- Anil et al. (2023)
Anil, R.; Dai, A. M.; Firat, O.; Johnson, M.; Lepikhin, D.; Passos, A.; Shakeri, S.; Taropa, E.; Bailey, P.; Chen, Z.; et al. 2023.
Palm 2 technical report.
*arXiv preprint arXiv:2305.10403*.
- Bi et al. (2024)
Bi, B.; Liu, S.; Wang, Y.; Mei, L.; and Chen, X. 2024.
Scalable Link Prediction on Large-Scale Heterogeneous Graphs with Large Language Models.
*arXiv preprint arXiv:2401.13227*.
- Boucher et al. (2022)
Boucher, N.; Shumailov, I.; Anderson, R.; and Papernot, N. 2022.
Bad characters: Imperceptible nlp attacks.
In *2022 IEEE Symposium on Security and Privacy (SP)*, 1987–2004. IEEE.
- Brannon et al. (2023)
Brannon, W.; Fulay, S.; Jiang, H.; Kang, W.; Roy, B.; Kabbara, J.; and Roy, D. 2023.
Congrat: Self-supervised contrastive pretraining for joint graph and text embeddings.
*arXiv preprint arXiv:2305.14321*.
- Chai et al. (2023)
Chai, Z.; Zhang, T.; Wu, L.; Han, K.; Hu, X.; Huang, X.; and Yang, Y. 2023.
GraphLLM: Boosting Graph Reasoning Ability of Large Language Model.
*arXiv preprint arXiv:2310.05845*.
- Chandra et al. (2020)
Chandra, S.; Mishra, P.; Yannakoudakis, H.; Nimishakavi, M.; Saeidi, M.; and Shutova, E. 2020.
Graph-based Modeling of Online Communities for Fake News Detection.
*arXiv preprint arXiv:2008.06274*.
- Chen et al. (2024)
Chen, R.; Zhao, T.; Jaiswal, A.; Shah, N.; and Wang, Z. 2024.
LLaGA: Large Language and Graph Assistant.
In *Forty-first International Conference on Machine Learning*.
- Chen et al. (2023a)
Chen, Z.; Mao, H.; Li, H.; Jin, W.; Wen, H.; Wei, X.; Wang, S.; Yin, D.; Fan, W.; Liu, H.; and Tang, J. 2023a.
Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs.
*arXiv preprint arXiv:2307.03393*.
- Chen et al. (2023b)
Chen, Z.; Mao, H.; Wen, H.; Han, H.; Jin, W.; Zhang, H.; Liu, H.; and Tang, J. 2023b.
Label-free Node Classification on Graphs with Large Language Models (LLMS).
*arXiv preprint arXiv:2310.04668*.
- Chiang et al. (2023)
Chiang, W.-L.; Li, Z.; Lin, Z.; Sheng, Y.; Wu, Z.; Zhang, H.; Zheng, L.; Zhuang, S.; Zhuang, Y.; Gonzalez, J. E.; Stoica, I.; and Xing, E. P. 2023.
Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality.
- Conti et al. (2022)
Conti, M.; Li, J.; Picek, S.; and Xu, J. 2022.
Label-only membership inference attack against node-level graph neural networks.
In *Proceedings of the 15th ACM Workshop on Artificial Intelligence and Security*, 1–12.
- Dai et al. (2023)
Dai, E.; Lin, M.; Zhang, X.; and Wang, S. 2023.
Unnoticeable backdoor attacks on graph neural networks.
In *Proceedings of the ACM Web Conference 2023*, 2263–2273.
- Dai et al. (2018)
Dai, H.; Li, H.; Tian, T.; Huang, X.; Wang, L.; Zhu, J.; and Song, L. 2018.
Adversarial attack on graph structured data.
In *International conference on machine learning*, 1115–1124. PMLR.
- Dong, Mücke, and Khosla (2022)
Dong, N.; Mücke, S.; and Khosla, M. 2022.
Mucomid: A multitask graph convolutional learning framework for miRNA-disease association prediction.
*IEEE/ACM Transactions on Computational Biology and Bioinformatics*, 19(6): 3081–3092.
- Duan et al. (2023)
Duan, K.; Liu, Q.; Chua, T.-S.; Yan, S.; Ooi, W. T.; Xie, Q.; and He, J. 2023.
SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning.
*arXiv preprint arXiv:2308.02565*.
- Dubey et al. (2024)
Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; Yang, A.; Fan, A.; et al. 2024.
The llama 3 herd of models.
*arXiv preprint arXiv:2407.21783*.
- Duddu, Boutet, and Shejwalkar (2020)
Duddu, V.; Boutet, A.; and Shejwalkar, V. 2020.
Quantifying privacy leakage in graph embedding.
In *MobiQuitous 2020-17th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services*, 76–85.
- Fan et al. (2019)
Fan, W.; Ma, Y.; Li, Q.; He, Y.; Zhao, E.; Tang, J.; and Yin, D. 2019.
Graph neural networks for social recommendation.
In *The world wide web conference*, 417–426.
- Fatemi, Halcrow, and Perozzi (2023)
Fatemi, B.; Halcrow, J.; and Perozzi, B. 2023.
Talk like a Graph: Encoding Graphs for Large Language Models.
*arXiv preprint arXiv:2310.04560*.
- Gong and Liu (2018)
Gong, N. Z.; and Liu, B. 2018.
Attribute inference attacks in online social networks.
*ACM Transactions on Privacy and Security (TOPS)*, 21(1): 1–30.
- Guo et al. (2023)
Guo, J.; Du, L.; Liu, H.; Zhou, M.; He, X.; and Han, S. 2023.
GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking.
*arXiv preprint arXiv:2305.15066*.
- Hamilton, Ying, and Leskovec (2017)
Hamilton, W.; Ying, Z.; and Leskovec, J. 2017.
Inductive representation learning on large graphs.
*Advances in neural information processing systems*, 30.
- He et al. (2024)
He, X.; Bresson, X.; Laurent, T.; Perold, A.; LeCun, Y.; and Hooi, B. 2024.
Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning.
In *The Twelfth International Conference on Learning Representations*.
- He et al. (2021)
He, X.; Wen, R.; Wu, Y.; Backes, M.; Shen, Y.; and Zhang, Y. 2021.
Node-level membership inference attacks against graph neural networks.
*arXiv preprint arXiv:2102.05429*.
- Hu et al. (2020)
Hu, W.; Fey, M.; Zitnik, M.; Dong, Y.; Ren, H.; Liu, B.; Catasta, M.; and Leskovec, J. 2020.
Open graph benchmark: Datasets for machine learning on graphs.
*Advances in neural information processing systems*, 33: 22118–22133.
- Huang et al. (2023a)
Huang, J.; Zhang, X.; Mei, Q.; and Ma, J. 2023a.
Can LLMs Effectively Leverage Structural Information for Graph Learning: When and Why.
*arXiv preprint arXiv:2309.16595*.
- Huang et al. (2023b)
Huang, X.; Han, K.; Bao, D.; Tao, Q.; Zhang, Z.; Yang, Y.; and Zhu, Q. 2023b.
Prompt-based Node Feature Extractor for Few-shot Learning on Text-Attributed Graphs.
*arXiv preprint arXiv:2309.02848*.
- Jin et al. (2021)
Jin, W.; Li, Y.; Xu, H.; Wang, Y.; Ji, S.; Aggarwal, C.; and Tang, J. 2021.
Adversarial attacks and defenses on graphs.
*ACM SIGKDD Explorations Newsletter*, 22(2): 19–34.
- Jin et al. (2020)
Jin, W.; Ma, Y.; Liu, X.; Tang, X.; Wang, S.; and Tang, J. 2020.
Graph structure learning for robust graph neural networks.
In *Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining*, 66–74.
- Kipf and Welling (2022)
Kipf, T. N.; and Welling, M. 2022.
Semi-Supervised Classification with Graph Convolutional Networks.
In *International Conference on Learning Representations*.
- Kuang et al. (2023)
Kuang, H.; Xu, J.; Zhang, H.; Zhao, Z.; Zhang, Q.; Huang, X.; and Wei, Z. 2023.
Unleashing the Power of Language Models in Text-Attributed Graph.
In *Findings of the Association for Computational Linguistics: EMNLP 2023*, 8429–8441. Association for Computational Linguistics.
- Li et al. (2023)
Li, K.; Liu, Y.; Ao, X.; and He, Q. 2023.
Revisiting graph adversarial attack and defense from a data distribution perspective.
In *The Eleventh International Conference on Learning Representations*.
- Liu and Wu (2023)
Liu, C.; and Wu, B. 2023.
Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis.
*arXiv preprint arXiv:2308.11224*.
- Liu et al. (2023)
Liu, S.; Nie, W.; Wang, C.; Lu, J.; Qiao, Z.; Liu, L.; Tang, J.; Xiao, C.; and Anandkumar, A. 2023.
Multi-modal molecule structure–text model for text-based retrieval and editing.
*Nature Machine Intelligence*, 5(12): 1447–1457.
- Liu et al. (2024)
Liu, Z.; He, X.; Tian, Y.; and Chawla, N. V. 2024.
Can we soft prompt LLMs for graph learning tasks?
In *Companion Proceedings of the ACM on Web Conference 2024*, 481–484.
- Ma, Ding, and Mei (2020)
Ma, J.; Ding, S.; and Mei, Q. 2020.
Towards more practical adversarial attacks on graph neural networks.
*Advances in neural information processing systems*, 33: 4756–4766.
- Mavromatis et al. (2023)
Mavromatis, C.; Ioannidis, V. N.; Wang, S.; Zheng, D.; Adeshina, S.; Ma, J.; Zhao, H.; Faloutsos, C.; and Karypis, G. 2023.
Train your own GNN teacher: Graph-aware distillation on textual graphs.
*arXiv preprint arXiv:2304.10668*.
- Olatunji et al. (2023a)
Olatunji, I. E.; Hizber, A.; Sihlovec, O.; and Khosla, M. 2023a.
Does black-box attribute inference attacks on graph neural networks constitute privacy risk?
*arXiv preprint arXiv:2306.00578*.
- Olatunji, Nejdl, and Khosla (2021)
Olatunji, I. E.; Nejdl, W.; and Khosla, M. 2021.
Membership inference attack on graph neural networks.
In *2021 Third IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA)*, 11–20. IEEE.
- Olatunji et al. (2023b)
Olatunji, I. E.; Rathee, M.; Funke, T.; and Khosla, M. 2023b.
Private Graph Extraction via Feature Explanations.
*Proceedings on Privacy Enhancing Technologies*.
- Pan et al. (2024)
Pan, S.; Luo, L.; Wang, Y.; Chen, C.; Wang, J.; and Wu, X. 2024.
Unifying large language models and knowledge graphs: A roadmap.
*IEEE Transactions on Knowledge and Data Engineering*.
- Peng et al. (2024)
Peng, Y.; Lin, S.; Chen, Q.; Xu, L.; Ren, X.; Li, Y.; and Xu, J. 2024.
ChatGraph: Chat with Your Graphs.
*arXiv preprint arXiv:2401.12672*.
- Qin et al. (2023)
Qin, Y.; Wang, X.; Zhang, Z.; and Zhu, W. 2023.
Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs.
*arXiv preprint arXiv:2310.18152*.
- Shanthamallu, Thiagarajan, and Spanias (2021)
Shanthamallu, U. S.; Thiagarajan, J. J.; and Spanias, A. 2021.
Uncertainty-matching graph neural networks to defend against poisoning attacks.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 35, 9524–9532.
- Storn and Price (1997)
Storn, R.; and Price, K. 1997.
Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces.
*Journal of global optimization*, 11: 341–359.
- Su et al. (2022)
Su, B.; Du, D.; Yang, Z.; Zhou, Y.; Li, J.; Rao, A.; Sun, H.; Lu, Z.; and Wen, J.-R. 2022.
A molecular multimodal foundation model associating molecule graphs with natural language.
*arXiv preprint arXiv:2209.05481*.
- Sun et al. (2022)
Sun, L.; Dou, Y.; Yang, C.; Zhang, K.; Wang, J.; Philip, S. Y.; He, L.; and Li, B. 2022.
Adversarial attack and defense on graph data: A survey.
*IEEE Transactions on Knowledge and Data Engineering*, 35(8): 7693–7711.
- Sun et al. (2023)
Sun, S.; Ren, Y.; Ma, C.; and Zhang, X. 2023.
Large Language Models as Topological Structure Enhancers for Text-Attributed Graphs.
*arXiv preprint arXiv:2311.14324*.
- Tan et al. (2023)
Tan, Y.; Zhou, Z.; Lv, H.; Liu, W.; and Yang, C. 2023.
WalkLM: A Uniform Language Model Fine-tuning Framework for Attributed Graph Embedding.
In *Thirty-seventh Conference on Neural Information Processing Systems*.
- Team et al. (2023)
Team, G.; Anil, R.; Borgeaud, S.; Wu, Y.; Alayrac, J.-B.; Yu, J.; Soricut, R.; Schalkwyk, J.; Dai, A. M.; Hauth, A.; et al. 2023.
Gemini: a family of highly capable multimodal models.
*arXiv preprint arXiv:2312.11805*.
- Tian et al. (2024)
Tian, Y.; Song, H.; Wang, Z.; Wang, H.; Hu, Z.; Wang, F.; Chawla, N. V.; and Xu, P. 2024.
Graph neural prompting with large language models.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, 19080–19088.
- Touvron et al. (2023)
Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023.
Llama 2: Open foundation and fine-tuned chat models.
*arXiv preprint arXiv:2307.09288*.
- Veličković et al. (2018)
Veličković, P.; Cucurull, G.; Casanova, A.; Romero, A.; Liò, P.; and Bengio, Y. 2018.
Graph Attention Networks.
In *International Conference on Learning Representations*.
- Wang et al. (2021)
Wang, J.; Zhang, S.; Xiao, Y.; and Song, R. 2021.
A review on graph neural network methods in financial applications.
*arXiv preprint arXiv:2111.15367*.
- Wang and Wang (2022)
Wang, X.; and Wang, W. H. 2022.
Group property inference attacks against graph neural networks.
In *Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security*, 2871–2884.
- Wei et al. (2024a)
Wei, Y.; Fu, S.; Jiang, W.; Kwok, J. T.; and Zhang, Y. 2024a.
Rendering graphs for graph reasoning in multimodal large language models.
*arXiv preprint arXiv:2402.02130*.
- Wei et al. (2024b)
Wei, Y.; Huang, Q.; Kwok, J. T.; and Zhang, Y. 2024b.
Kicgpt: Large language model with knowledge in context for knowledge graph completion.
*arXiv preprint arXiv:2402.02389*.
- Wenkel, Wolf, and Knyazev (2023)
Wenkel, F.; Wolf, G.; and Knyazev, B. 2023.
Pretrained Language Models to Solve Graph Tasks in Natural Language.
In *ICML workshop on Structured Probabilistic Inference & Generative Modeling*.
- Wu et al. (2019)
Wu, H.; Wang, C.; Tyshetskiy, Y.; Docherty, A.; Lu, K.; and Zhu, L. 2019.
Adversarial examples on graph data: Deep insights into attack and defense.
*arXiv preprint arXiv:1903.01610*.
- Wu et al. (2021)
Wu, X.; Wu, H.; Zhou, X.; and Lu, K. 2021.
CoG: a Two-View Co-training Framework for Defending Adversarial Attacks on Graph.
*arXiv preprint arXiv:2109.05558*.
- Xie et al. (2023)
Xie, H.; Zheng, D.; Ma, J.; Zhang, H.; Ioannidis, V. N.; Song, X.; Ping, Q.; Wang, S.; Yang, C.; Xu, Y.; et al. 2023.
Graph-aware language model pre-training on a large graph corpus can help multiple graph applications.
In *Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining*, 5270–5281.
- Yang et al. (2021)
Yang, J.; Liu, Z.; Xiao, S.; Li, C.; Lian, D.; Agrawal, S.; Singh, A.; Sun, G.; and Xie, X. 2021.
GraphFormers: GNN-nested Transformers for Representation Learning on Textual graph.
In *Proceedings of the 35th Conference on Neural Information Processing Systems*.
- Yang, Cohen, and Salakhudinov (2016)
Yang, Z.; Cohen, W.; and Salakhudinov, R. 2016.
Revisiting semi-supervised learning with graph embeddings.
In *International conference on machine learning*, 40–48. PMLR.
- Ye et al. (2023)
Ye, R.; Zhang, C.; Wang, R.; Xu, S.; and Zhang, Y. 2023.
Natural Language is All a Graph Needs.
*arXiv preprint arXiv:2308.07134*.
- Zhang and Ma (2024)
Zhang, A.; and Ma, J. 2024.
Defensevgae: Defending against adversarial attacks on graph data via a variational graph autoencoder.
In *International Conference on Intelligent Computing*, 313–324. Springer.
- Zhang (2023)
Zhang, J. 2023.
Graph-toolformer: To empower llms with graph reasoning ability via prompt augmented by chatgpt.
*arXiv preprint arXiv:2304.11116*.
- Zhang and Zitnik (2020)
Zhang, X.; and Zitnik, M. 2020.
Gnnguard: Defending graph neural networks against adversarial attacks.
*Advances in neural information processing systems*, 33: 9263–9275.
- Zhang et al. (2022)
Zhang, Z.; Chen, M.; Backes, M.; Shen, Y.; and Zhang, Y. 2022.
Inference attacks against graph neural networks.
In *31st USENIX Security Symposium (USENIX Security 22)*, 4543–4560.
- Zhang et al. (2021)
Zhang, Z.; Liu, Q.; Huang, Z.; Wang, H.; Lu, C.; Liu, C.; and Chen, E. 2021.
Graphmi: Extracting private graph data from graph neural networks.
*arXiv preprint arXiv:2106.02820*.
- Zhang et al. (2023)
Zhang, Z.; Wang, X.; Zhang, Z.; Li, H.; Qin, Y.; and Zhu, W. 2023.
LLM4DyG: Can Large Language Models Solve Problems on Dynamic Graphs?
*arXiv preprint arXiv:2310.17110*.
- Zhao et al. (2023)
Zhao, J.; Zhuo, L.; Shen, Y.; Qu, M.; Liu, K.; Bronstein, M.; Zhu, Z.; and Tang, J. 2023.
GraphText: Graph Reasoning in Text Space.
*arXiv preprint arXiv:2310.01089*.
- Zhu et al. (2024)
Zhu, Y.; Wang, Y.; Shi, H.; and Tang, S. 2024.
Efficient tuning and inference for large language models on textual graphs.
*arXiv preprint arXiv:2401.15569*.
- Zügner, Akbarnejad, and Günnemann (2018)
Zügner, D.; Akbarnejad, A.; and Günnemann, S. 2018.
Adversarial attacks on neural networks for graph data.
In *Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining*, 2847–2856.
- Zügner and Günnemann (2019)
Zügner, D.; and Günnemann, S. 2019.
Adversarial Attacks on Graph Neural Networks via Meta Learning.
In *International Conference on Learning Representations (ICLR)*.

## Appendix A Appendix

##### Organization:

The Appendix is organized as follows. Appendix [A.1](https://arxiv.org/html/2508.04894v1#A1.SS1) provides a detailed taxonomy of graph-aware LLMs. We discuss related work on attacks against Graph Neural Networks (GNNs) in Appendix [A.2](https://arxiv.org/html/2508.04894v1#A1.SS2). Appendix [A.3](https://arxiv.org/html/2508.04894v1#A1.SS3) gives an overview of the adversarial attacks, Nettack and MetaAttack, while Appendix [A.4](https://arxiv.org/html/2508.04894v1#A1.SS4) details the algorithm for the node sequence injection attack on LLaGA. We present the results of our ablation studies in Appendix [A.5](https://arxiv.org/html/2508.04894v1#A1.SS5), including the effect of varying the amount of perturbed features and the choice of base LLM on attack performance. Finally, Appendix [A.7](https://arxiv.org/html/2508.04894v1#A1.SS7) shows examples of feature attacks and the template for the LLM corrector, and Appendix [A.8](https://arxiv.org/html/2508.04894v1#A1.SS8) discusses the implications of our research.

##### Code.

Our code is available.

### A.1 Graph Encoding-Based Taxonomy of LLM Adaptations

Figure: Figure 5: Graph encoding-based adaptations of LLMs. In the textual description graph representation, graphs are explained textually (top part). In the learned projector representation, graphs are encoded either by node templates that turns the graph into sequences or by GNNs (bottom part). Note that in both cases, the LLM is frozen. Only the projector that maps the graph into tokens is trained.
Refer to caption: x6.png

The integration of large language models (LLMs) with graph data has been an emerging area of research, with various approaches exploring how LLMs can be adapted or enhanced for graph-related tasks such as reasoning, representation, and inference. These approaches can be largely divided into two categories based on their methods for encoding graphs for LLMs: textual descriptions of graphs, where graph information is encoded as natural language, and learned graph projector, where a projector is utilized to encode graph data into embeddings or structured forms that LLMs can directly process. In this section, we describe the methods used in the different encoding approaches for integrating graph data into LLMs. [Figure 5](https://arxiv.org/html/2508.04894v1#A1.F5) provides the illustration of both approaches and [Table 7](https://arxiv.org/html/2508.04894v1#A1.T7) provides a summary of the differences between both approaches.

#### Graph Representation as a Textual Description

In this first category, graph data is represented as natural language, allowing LLMs to process graph structures as textual descriptions. This approach has been explored extensively with works such as (Guo et al. 2023; Liu and Wu 2023; Ye et al. 2023; Wenkel, Wolf, and Knyazev 2023; Wei et al. 2024b; Fatemi, Halcrow, and Perozzi 2023; Tan et al. 2023; Zhao et al. 2023; Wei et al. 2024a) converting graph data into textual formats, often paired with specific instructions to query LLMs.
For example, Tan et al. (2023) introduced a unified framework for attributed graph embedding by generating random walk-based textual sequences from graphs, which are then used to fine-tune LLMs. Similarly, Zhao et al. (2023) focused on graph reasoning within text spaces, translating graph structures into natural language to facilitate LLM interaction. Extending this idea, Wei et al. (2024a) augmented textual representations of graphs with visual representations of the graph, enhancing the reasoning capabilities of LLMs on graph data. In multimodal applications, Liu et al. (2023) and Su et al. (2022) linked molecule graphs with textual descriptions, enabling advances in drug discovery and molecular retrieval. Furthermore, Huang et al. (2023a) investigated how incorporating graph structure into LLM prompts can enhance predictive performance. Their findings revealed that LLMs tend to process graph-related prompts as contextual paragraphs, rather than explicitly recognizing graph structures. This further underscores the importance of prompt design, as seen in studies like (Brannon et al. 2023; Zhang et al. 2023; Sun et al. 2023; Bi et al. 2024; Pan et al. 2024), which extended LLMs to heterogeneous and large-scale graph tasks. Complementing these efforts, (He et al. 2024) explored using LLM-generated explanations as features to enhance both the performance and interpretability of LLMs on graphs.

While still representing graphs as textual description, another line of work integrates pretraining strategies with GNNs to enhance the LLMs’ graph reasoning capabilities. For instance, Chen et al. (2023a) proposed two LLM-GNN integration approaches: 1) using LLMs to enhance node features for node classification, and 2) employing LLMs as direct predictors for graph tasks. In line with this, Chandra et al. (2020) utilized LLMs as text encoders and GNNs as graph structure encoders for tasks such as fake news detection in online communities. Similarly, Chen et al. (2023b) explored the possibility of performing node classification without explicit labels by using LLMs to annotate nodes, which are then used in training a GNN model. Likewise, Mavromatis et al. (2023) utilized LLM as a feature extractor for GNNs.

For inference, Zhu et al. (2024) proposed a fine-tuning method for textual graph that combined LLMs and GNNs through a tunable side structure alongside each layer of the LLM, enhancing both training efficiency and inference speed. Similarly, Duan et al. (2023) introduced SimTeG, a simple yet effective approach that enhances textual graph learning by leveraging LLMs. In their method, the LLM is first fine-tuned on a downstream task, such as node classification. Afterward, node embeddings are generated by extracting the last hidden states from the fine-tuned LLM. Additionally, Zhang (2023) proposed Graph-toolformer, which leverages prompt engineering, powered by ChatGPT, to augment LLMs’ graph reasoning capabilities.

While these approaches make LLMs more accessible for graph-related tasks, they often struggle to fully capture the structural intricacies inherent in graph data. Moreover, the performance of these methods remains heavily dependent on effective prompt engineering, which can be a challenging process, requiring careful design and experimentation to obtain optimal results.

**Table 7: Comparison of graph encoding-based adaptations of LLMs.**
| Aspect | Textual Description Approach | Learned Projector Approach |
| --- | --- | --- |
| Graph Encoding | Converts graph data (nodes, edges) into natural language descriptions. | Uses a learned projector to encode graph structures into embeddings or graph tokens. |
| LLM Modifications | No modification to the LLM. | No modification to the LLM. |
| Graph Structure<br>Representation | Describes graph topology and relationships using textual narratives. | Encodes relational and structural information directly into vector representations. |
| Flexibility | Can only be used for simple graphs. | More flexible in representing complex graph structures. |
| Computational Cost | No additional training is required. Lower cost. | Lightweight projector is trained, typically a linear layer. Minimal cost. |
| Suitability for<br>Graph Tasks | May lose some relational nuance due to limitations in textual descriptions. | Captures detailed structural information, improving performance on graph-specific tasks. |

#### Graph Representation as a Learned Projector

To address the challenges posed by textual representation approaches, learned projector methods offer an alternative by transforming graph data into embeddings or graph vectors that LLMs can process more directly. For instance, Chen et al. (2024) introduced LLaGA, a large language and graph assistant that retains the general-purpose nature of LLMs while transforming graph structures into graph tokens, a format suitable for LLM processing. Similarly, (Liu et al. 2024; Qin et al. 2023) incorporated graph structure information into a pre-trained LLM through a tailored disentangled graph neural network layers.
Tian et al. (2024) proposed using graph neural prompts to encode knowledge graph, transforming them into graph embeddings that LLMs can leverage during inference. Furthermore, Huang et al. (2023b) introduced a prompt-based node feature extractor (G-Prompt) for few-shot learning on text-attributed graphs. Their method combines a graph adapter (a learnable GNN layer) with task-specific prompts to capture neighborhood information and extract informative node features.

Another promising line of work condenses graph information into fixed-length prefixes using graph transformers, which are attached to each layer of the LLM to enhance its reasoning capabilities (Chai et al. 2023). Similarly, Yang et al. (2021) developed GraphFormers which seamlessly integrates text encoding and graph aggregation in an iterative workflow by nesting GNNs alongside the transformer blocks of a language model. Additionally, Peng et al. (2024) developed ChatGraph, an interactive interface that allows users to interact with their graphs through natural language. Here, graphs are represented as sequential paths and the LLM is fine-tuned on the graph data to support graph analysis.

In this paper, we do not consider projector adaptations that require retraining the LLM, as they significantly increase computational costs without yielding substantial performance gains. Instead, we focus on adaptation methods that utilize frozen pre-trained LLMs (Chen et al. 2024; Liu et al. 2024; Qin et al. 2023; Tian et al. 2024; Huang et al. 2023b). This approach not only reduces computational costs but also offers greater flexibility in integrating with various LLM models. Additionally, we can take advantage of LLMs’ extensive knowledge and capabilities without the overhead of retraining while still adapting them for specific graph tasks.
Throughout this work, we employ two representative projector methods: LLaGA (Chen et al. 2024), which utilizes node-level templates and a linear projector, and GraphPrompter (Liu et al. 2024), which employs a GNN-based projector to encode the graph.

This work.
In this work, we do not consider textual description methods, as they often struggle to fully capture the structural intricacies inherent in graph data. These methods are generally limited to simpler tasks, such as describing basic graph properties like node degree, and become infeasible for larger graphs due to scalability issues. In particular, the token limit in prompts restricts the detailed descriptions required for larger graph structures, quickly exceeding allowable token counts.
Additionally, from a vulnerability perspective, textual description methods do not introduce any unique attack surface; since the graph is represented as simple text descriptions (e.g., ”Node A is connected to Node B”), they are susceptible to traditional text-based adversarial attacks rather than the structural or feature perturbations specific to graph-based models. Therefore, they lack the complexity that would expose them to novel, graph-specific vulnerabilities, making them less relevant for our investigation into adversarial robustness of graph-aware LLMs.

### A.2 Attacks on GNN

Several types of attacks exist on GNNs, including membership inference attacks (Olatunji, Nejdl, and Khosla 2021; He et al. 2021; Conti et al. 2022), which aim to determine whether a specific node was part of the model’s training set; attribute inference attacks (Gong and Liu 2018; Olatunji et al. 2023a; Duddu, Boutet, and Shejwalkar 2020), which exploit the GNN’s output and learned embeddings to infer sensitive or missing node attributes; property inference attacks (Zhang et al. 2022; Wang and Wang 2022), which seek to extract global properties of the entire graph; adversarial attacks (Zügner, Akbarnejad, and Günnemann 2018; Zügner and Günnemann 2019; Li et al. 2023; Dai et al. 2018; Sun et al. 2022), which attempt to degrade the model’s performance by introducing subtle perturbations to the graph or node features; and graph reconstruction attacks (Zhang et al. 2021; Olatunji et al. 2023b), which aim to reconstruct the underlying graph structure using the learned embeddings or model outputs.

Among these attack types, adversarial attacks are of particular concern due to their direct impact on the model’s performance. In adversarial attacks, small, carefully crafted perturbations are introduced into the graph, such as modifying node features or altering edges, with the goal of significantly degrading the GNN’s performance. These attacks can be categorized into two types: poisoning attacks, which occur during the training phase by manipulating the training data, and evasion attacks, which occur at test time, where the adversary introduces perturbations to the input graph to mislead the model’s predictions. For instance, Zügner, Akbarnejad, and Günnemann (2018) demonstrated that adversarial perturbations on graph structures or node features can significantly degrade the performance of GNNs. Their method, Nettack, employed a greedy optimization technique to compute the minimal perturbations that would maximize the model’s loss function during training. This approach effectively identifies the smallest changes in the features or graph structure needed to disrupt the model’s predictions. Li et al. (2023) revisited the problem of adversarial attacks on graph data from a data distribution perspective, highlighting how adversarial examples can significantly alter the underlying data distribution and proposed a novel defense mechanism to mitigate such effects. Similarly, Wu et al. (2019) provided a comprehensive analysis of adversarial examples on graph data, offering deep insights into both attacks and defenses. They focused on the vulnerabilities of GNNs to small perturbations and proposed defense techniques designed to detect and counteract adversarial modifications, thus enhancing model robustness.
MetaAttack (Zügner and Günnemann 2019) performed both poison and evasion attacks on GNNs by treating the graph structure as a hyperparameter that can be optimized. The attack directly modifies the graph structure using gradient descent, formulating the problem as a bi-level optimization task. Their method utilized meta-gradients, which capture how small perturbations in the graph structure affect the attacker’s loss after training, enabling precise updates to the graph that degrade the model’s performance.

In this paper, we leverage Nettack (Zügner, Akbarnejad, and Günnemann 2018) and MetaAttack (Zügner and Günnemann 2019) as representative adversarial attack methods, encompassing both poisoning and evasion strategies, to conduct our investigation. Specifically, poisoning attacks target model integrity during training, while evasion attacks compromise performance at inference or test time.

### A.3 How Nettack and MetaAttack Work

Nettack performs graph structure perturbation by adding and removing edges between nodes, while ensuring that the degree of the perturbed graph is preserved. It uses a greedy optimization technique to compute the minimal perturbations that will maximize the model’s loss function during training, with respect to a surrogate model. This surrogate model is a linearized 2-layer graph convolutional network (GCN), which approximates how changes in the graph structure affect the classification result. The primary objective of Nettack is to increase the model’s confidence in incorrect classifications by maximizing the distance between the logits of the perturbed graph and those of the clean graph. To achieve this, Nettack identifies candidate edges for perturbation around a target node and evaluates a score function, for each connected edge. This score indicates how significantly altering a particular edge will impact the classification result of the target node. The edge with the highest score is then selected, and the graph is updated by either adding or removing that edge. Since Nettack is a targeted attack (only cause misclassification of a specific single node), we adapt the same method in (Zügner and Günnemann 2019) to convert it into untargeted attack (compromise node classification performance of the model) which randomly select one test node and attack it using Nettack while considering all nodes in the network. We then aggregate the result over the test nodes.

MetaAttack optimizes the graph structure directly via gradient descent by treating the adjacency matrix (graph structure) as a hyperparameter. It frames the attack as a bi-level optimization problem: the inner level minimizes the training loss on the original graph, while the outer level minimizes the loss on the poisoned graph. To achieve this, MetaAttack utilizes a surrogate model, specifically a linearized two-layer GCN, to approximate how changes in the graph structure affect model performance. A score function evaluates the impact of each perturbation, and the attack flips the sign of meta-gradients (gradient wrt hyperparameters) for connected nodes, indicating which edges to remove. The algorithm then greedily selects perturbations—either adding or removing edge based on those with the highest scores, to maximize the attack’s impact.

For both methods, we use the same unoticeability constraint introduced by (Zügner, Akbarnejad, and Günnemann 2018), which ensures that the graph’s degree distribution only changes slightly. Therefore, we only perturb $10\%$ of the edges.

In our poisoning attack experiments with LLaGA, we employ the perturbed graph structure obtained from Nettack or MetaAttack to generate the node-level sequences and retrain the projector that converts the node-level sequences into a sequence of token embeddings. Similarly, for GraphPrompter, we utilize the perturbed graph structure, re-encode the graph using a graph attention network (GAT), and retrain the projector with the perturbed graph. For evasion attacks, the graph-aware LLM is trained on unperturbed data, with perturbations introduced only during the inference stage.

Figure: Algorithm 1 Injection Attacks on Neighborhood Detail Template for Obtaining Node Sequence.

### A.4 Algorithm for Node Sequence Template Injection Attacks

We present the algorithm for the node sequence template injection attack in LLaGA in Algorithm [1](https://arxiv.org/html/2508.04894v1#alg1). The model constructs a fixed-shape computational tree for each node $u$, sampling $k$ neighbors per node up to depth $d$ (lines 1–14). When a node has fewer than $k$ neighbors, placeholders are inserted, creating an attack surface. We exploit this by injecting malicious nodes into placeholder positions through graph manipulation. We propose three strategies: Non-Adjacent Injection (NI) inserts nodes outside the 2-hop neighborhood of u (lines 21–27); Supernode Injection (SI) injects a high-degree node into one placeholder (lines 29–33); and Multiple Supernode Injection (MSI) distributes distinct high-degree nodes across all placeholders (lines 35–43). These uninformative nodes corrupt the node sequence, degrading the projection layer’s performance. The inclusion of uninformative nodes in the sequence corrupts the target’s representation, impairing training of the projection layer.

### A.5 Ablation Studies

Here, we present the results for the ablation studies presented in the main paper.

#### Ablation Results: Varying Feature Perturbation

In [Figure 6](https://arxiv.org/html/2508.04894v1#A1.F6), we present the performance of two graph-aware LLMs, LLaGA and GraphPrompter, as the percentage of perturbed features is varied. We observe that increasing the proportion of perturbed features consistently degrades model accuracy for both models, indicating their sensitivity to feature-level attacks.

Figure: (a) LLaGA
Refer to caption: x7.png

#### Ablation Results: Does the Choice of LLM Affect Attack Performance?

The results in [Table 8](https://arxiv.org/html/2508.04894v1#A1.T8) show that the choice of base LLM has minimal impact on both clean performance and adversarial vulnerability in the LLaGA framework. Across Cora, Citeseer, and PubMed, models built on Vicuna-7B and Llama2-7B achieve nearly identical accuracy under clean conditions and exhibit similar degradation under both poisoning and evasion attacks. For instance, under MetaAttack evasion, Cora performance drops to $0.44\pm 0.06$ for Vicuna-7B and $0.43\pm 0.08$ for Llama2-7B. These differences are well within standard deviation. Similar patterns hold across all datasets and attack types. This indicates that attack effectiveness is largely independent of the base LLM.

**Table 8: Impact of base model changes on model’s performance and under different attacks. We show the result for LLaGA. The results demonstrate that the choice of base model does not significantly affect susceptibility to attacks.**
| Data | Base Model | Clean | Poisoning | Evasion |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  |  | Nettack | MetaAttack | MSI | Nettack | MetaAttack | MSI |
| Cora | Vicuna-7B | 0.89 $\pm$ 0.07 | 0.87 $\pm$ 0.04 | 0.79 $\pm$ 0.03 | 0.82 $\pm$ 0.04 | 0.55 $\pm$ 0.09 | 0.44 $\pm$ 0.06 | 0.30 $\pm$ 0.05 |
| Llama2-7B | 0.90 $\pm$ 0.03 | 0.86 $\pm$ 0.02 | 0.77 $\pm$ 0.05 | 0.82 $\pm$ 0.02 | 0.56 $\pm$ 0.07 | 0.43 $\pm$ 0.08 | 0.31 $\pm$ 0.01 |  |
| Citeseer | Vicuna-7B | 0.71 $\pm$ 0.04 | 0.64 $\pm$ 0.05 | 0.63 $\pm$ 0.04 | 0.62 $\pm$ 0.02 | 0.59 $\pm$ 0.04 | 0.50 $\pm$ 0.03 | 0.24 $\pm$ 0.05 |
| Llama2-7B | 0.71 $\pm$ 0.05 | 0.65 $\pm$ 0.03 | 0.60 $\pm$ 0.03 | 0.61 $\pm$ 0.04 | 0.58 $\pm$ 0.03 | 0.48 $\pm$ 0.05 | 0.26 $\pm$ 0.03 |  |
| PubMed | Vicuna-7B | 0.90 $\pm$ 0.04 | 0.89 $\pm$ 0.03 | 0.89 $\pm$ 0.08 | 0.81 $\pm$ 0.03 | 0.84 $\pm$ 0.05 | 0.73 $\pm$ 0.04 | 0.76 $\pm$ 0.03 |
| Llama2-7B | 0.91 $\pm$ 0.06 | 0.88 $\pm$ 0.04 | 0.90 $\pm$ 0.02 | 0.80 $\pm$ 0.04 | 0.85 $\pm$ 0.03 | 0.75 $\pm$ 0.02 | 0.77 $\pm$ 0.02 |  |

### A.6 Results on the ArXiv Dataset

We now evaluate our attack on the large-scale ArXiv dataset, which consists of 169,343 nodes, 1,166,243 edges, and has a sparsity of 0.81. This experiment aims to investigate the effectiveness of node sequence template attacks in large, real-world graphs. As shown in [Table 9](https://arxiv.org/html/2508.04894v1#A1.T9), we observe a significant gap between the success rates of poisoning and evasion attacks, with poisoning attacks consistently outperforming evasion across all three injection strategies, which is not the case for the other datasets, Cora, Citeseer and PubMed. This suggests that ArXiv may have inherent structural differences that make it more vulnerable to poisoning yet more robust to evasion. While the sparsity of the ArXiv dataset is very low, inserting a single supernode could significantly affect the model performance.

**Table 9: Results of the node sequence template injection attack across different datasets and methods. NI, SI, and MSI indicates the non-adjacent, single supernode, and multiple supernodes injection attacks respectively. Clean refers to the performance of the model without any attack.**
| Data | Attack | NI | SI | MSI |
| --- | --- | --- | --- | --- |
| Cora | Clean | ——– $0.89_{\pm 0.07}$ ——– |  |  |
|  | Poisoning | $0.86_{\pm 0.03}$ | $0.84_{\pm 0.06}$ | $0.82_{\pm 0.04}$ |
|  | Evasion | $0.42_{\pm 0.08}$ | $0.36_{\pm 0.05}$ | $0.30_{\pm 0.05}$ |
| Citeseer | Clean | ——– $0.71_{\pm 0.04}$ ——– |  |  |
|  | Poisoning | $0.69_{\pm 0.05}$ | $0.66_{\pm 0.03}$ | $0.62_{\pm 0.02}$ |
|  | Evasion | $0.36_{\pm 0.06}$ | $0.30_{\pm 0.04}$ | $0.24_{\pm 0.05}$ |
| PubMed | Clean | ——– $0.90_{\pm 0.04}$ ——– |  |  |
|  | Poisoning | $0.88_{\pm 0.04}$ | $0.85_{\pm 0.05}$ | $0.81_{\pm 0.03}$ |
|  | Evasion | $0.85_{\pm 0.07}$ | $0.82_{\pm 0.04}$ | $0.76_{\pm 0.03}$ |
| ArXiv | Clean | ——– $0.73_{\pm 0.02}$ ——– |  |  |
|  | Poisoning | $0.50_{\pm 0.03}$ | $0.43_{\pm 0.04}$ | $0.42_{\pm 0.03}$ |
|  | Evasion | $0.71_{\pm 0.06}$ | $0.69_{\pm 0.02}$ | $0.60_{\pm 0.03}$ |

### A.7 Feature Attack Examples and Defense Template

#### Feature Attack Examples

The examples in the prompt, as shown in Appendix [A.7](https://arxiv.org/html/2508.04894v1#A1.SS7.SSSx2), illustrate our adversarial attacks on real features from the Cora dataset. The ”input” is the text received by the LLM where the attack is being applied. It is important to note that while a human reader perceives the original text and the perturbed text as visually identical or very similar, the LLM processes a sequence of different Unicode characters or a different character order, which can alter its behavior. We show this in [Table 10](https://arxiv.org/html/2508.04894v1#A1.T10) for homoglyph and [Table 11](https://arxiv.org/html/2508.04894v1#A1.T11) for reordering attack.

**Table 10: An illustration of the imperceptible homoglyph attack. While the perturbed text is processed as distinct characters by the LLM, the visual output is nearly identical for a human, as shown in the image.**
| Original Feature | What LLM Sees | What a Human Sees |
| --- | --- | --- |
| Title: The megaprior heuristic for discovering protein sequence patterns<br>Abstract: Several computer algorithms for discovering patterns in groups of protein sequences are in use that are based on fitting the parameters of a statistical model to a group of related sequences. |  |  |

**Table 11: An illustration of the imperceptible reordering attack. While the perturbed text is processed as distinct characters by the LLM, the visual output is nearly identical for a human, as shown in the image.**
| Original Feature | What LLM Sees | What a Human Sees |
| --- | --- | --- |
| Title: Applications of machine learning: a medical follow up study.<br>Abstract: This paper describes preliminary work that aims to apply some learning strategies to a medical follow-up study. An investigation of the application of three machine learning algorithms-1R, FOIL and InductH to identify risk factors that govern the colposuspension cure rate has been made. |  |  |

#### Feature Corrector Defense Template

We designed a feature corrector module to mitigate the effects of imperceptible attacks, specifically homoglyph substitutions and character reorderings. This module is implemented as a prompt-based defense using LLMs. The prompt is carefully crafted to instruct the LLM to act as a linguistic validator, focusing on detecting and correcting subtle, human-imperceptible perturbations.

The template for our prompt is shown below. It explicitly states the task and provides a clear output format, which significantly improves the LLM’s performance.

Figure: Figure 7: The prompt template used for the LLM-based feature corrector module. It explicitly defines the role, task, and expected output format for the LLM.
Refer to caption: content/defense_prompt.png

### A.8 Detailed Discussion

The findings of our work raise several critical insights into the vulnerabilities and design considerations of graph-aware LLMs. As the integration of graph-structured data into LLMs continues to gain traction for graph tasks, it is important to explore not only the performance benefits but also the vulnerabilities of these novel architectures. Our systematic investigation of adversarial attacks on two representative graph-aware LLMs, LLaGA and GraphPrompter —reveals significant weaknesses, particularly when exposed to poisoning and evasion attacks, which are traditionally used to undermine GNNs. Our key observations are discussed below.

#### Impact of Graph Encoding on Adversarial Vulnerability

The key observation from our results is that the integration method of graph structure into LLMs can inadvertently increase their susceptibility to adversarial attacks. Specifically, the node sequence template employed in LLaGA appears to create a vulnerability that can be exploited by adversaries, leading to substantial degradation in model performance. This attack surface, unique to LLaGA, highlights a critical tradeoff in graph-aware LLM design: while the use of node-level templates provides an effective means of incorporating graph structure and achieves better performance on graph task, it also exposes the model to new attack vectors not present in traditional GNNs. On the other hand, GraphPrompter, which uses a GNN-based projector for graph encoding, demonstrates comparatively greater resilience to these attacks, underscoring the potential advantages of hybrid architectures that leverage the strengths of both LLMs and GNNs.

#### Sensitivity of Graph-aware LLMs to Feature Perturbations

In addition to the vulnerabilities in the graph encoding techniques, our results also highlight the effectiveness of feature perturbation attacks across both graph-aware LLMs. Despite differences in how graph structure is encoded, both LLaGA and GraphPrompter exhibit significant sensitivity to imperceptible perturbations in the node features. This highlights a broader trend observed in the LLM paradigm: while these models excel in handling textual data, their robustness to adversarial perturbations, especially those targeting node features, remains a critical challenge. The ability of adversaries to degrade model performance by subtly altering node attributes without being detected calls into question the integrity of graph-aware LLMs, especially in security-sensitive applications. While text sanitization techniques, such as those discussed by (Boucher et al. 2022), offer potential defenses, addressing these imperceptible attacks such as homoglyph-based manipulations remains a complex challenge. As proposed, using LLMs as a corrector is one way to mitigate such vulnerabilities, by prompting the model to detect and repair suspicious or malformed inputs before downstream processing.

#### Data-dependent Variability of Poisoning and Evasion Attacks on Graph-aware LLMs

Our findings reveal that the efficacy of poisoning and evasion attacks in the context of graph-aware LLMs is not uniform across datasets. While attacks are particularly effective on the Cora, Citeseer and PubMed datasets, we observe a more nuanced interaction between the attack strategies and the graph structure on the ArXiv dataset. This variation emphasizes the importance of dataset characteristics in evaluating the vulnerabilities of graph-aware LLMs. As these models are increasingly deployed in diverse real-world settings, their vulnerabilities must be assessed not only in terms of algorithmic design but also in relation to the specific domains and data they are applied to.

#### Implications for Graph-aware LLM Development

The vulnerabilities exposed in this study have broader implications for the future development and deployment of graph-aware LLMs. Given their increasing adoption for tasks that were traditionally the domain of GNNs, it is crucial to consider the tradeoffs between performance and vulnerabilities in the design of these paradigms. As graph-aware LLMs continue to evolve, there is a pressing need for the development of more robust encoding techniques and defenses against adversarial manipulation, particularly in the context of feature perturbations. Furthermore, understanding how adversarial attacks transfer from GNNs to graph-aware LLMs opens new avenues for research in protecting these models against sophisticated adversaries.

#### Difficulty in Applying Existing GNN defenses Against Adversarial Attacks on Graph-aware LLMs

##### Defenses on adversarial attacks based on feature perturbation .

Adversarial attacks on GNNs have predominantly focused on structural perturbations, as feature-based perturbations typically fail to influence the predicted class of the target node (Dai et al. 2018; Wu et al. 2019; Zügner and Günnemann 2019), making them ineffective attack strategies. Consequently, there are no known defenses against feature-based attacks on GNNs, given their limited effectiveness. However, as demonstrated in the main paper, attacks based on feature perturbations proves effective in the graph-aware LLM paradigm. This introduces a novel challenge within the graph-aware LLM framework, where feature-based attacks, previously ineffective against GNNs, can now successfully manipulate model behavior. This shift highlights the need for developing new defenses that address both structural and feature-based adversarial perturbations. While text sanitization techniques, such as the use of optical character recognition (OCR) as a pre-processsing step for encoding text for the LLM, offer potential defenses, addressing these imperceptible attacks such as homoglyph-based manipulations remains a complex challenge as OCR is imperfect and tends to misinterpret homoglyph at a higher rate than unperturbed text (Boucher et al. 2022). This led to our design of self-correcting mechanism for feature perturbation using LLMs.

##### Defenses on adversarial attacks based on structural perturbation .

Due to the transferability of structure-based adversarial attacks from GNNs to graph-aware LLMs, particularly in evasion attacks, it is natural to expect that defenses developed for GNNs would also be effective in this paradigm. However, this assumption does not hold. Unlike traditional GNN settings, graph-aware LLMs are also susceptible to feature perturbations, rendering these defenses ineffective.

One such defense mechanism in GNNs involves making the adjacency matrix trainable, allowing edge weights to adapt during training to mitigate adversarial perturbations.
This approach assigns lower weights to edges connecting dissimilar nodes and higher weights to edges linking nodes with greater feature similarity. Several preprocessing-based defenses have been proposed building on this approach, such as removing edges between nodes with low feature similarity (Wu et al. 2019) or introducing randomized edge dropping (Dai et al. 2018). However, these defenses rely heavily on clean, unperturbed node features to compute reliable similarity scores. When node features are also perturbed, as in the case of our attacks, such defenses become ineffective and may even exacerbate the problem by connecting incorrect edges.
We demonstrate this effect on a random node in the Cora dataset ([Figure 8](https://arxiv.org/html/2508.04894v1#A1.F8)). As observed, while the clean feature similarity graph preserves some original edges alongside introducing some wrong connections, the edges reconstructed by the perturbed features fail to preserve any original connections, leading to entirely incorrect reconstructions.

Figure: Figure 8: Edges reconstructed for a random node 1044. Original (left) is the ground truth edges from the original graph. Feature Similarity (middle) shows the edges reconstructed using clean node features, where blue nodes indicate preserved connections from the original graph. Perturbed Feature Similarity (right) shows the edges reconstructed using perturbed node features.
Refer to caption: x9.png

Another line of work focuses on decoupling reliance on the graph structure. For instance, (Shanthamallu, Thiagarajan, and Spanias 2021) proposed a defense that trains a surrogate predictor dependent solely on node features and employs an uncertainty matching strategy to extract graph information from the GNN. Similarly, Wu et al. (2021) introduced a co-training framework that trains submodels independently on feature and structural views, allowing knowledge distillation through the inclusion of confident unlabeled data into the training set. However, these approaches also assume access to unperturbed node features, an assumption that does not hold in our setting, where both structure and features are adversarially perturbed. Consequently, structural defense strategies developed for GNNs are not directly applicable to the graph-aware LLM paradigm, which calls for novel defenses tailored to this paradigm.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 2
presentation: 2
questions: 1. Why is the evasion attack more effective? Please elaborate on the explanations in 4.1 and provide more evidence.
2. How do the surrogate models perform under different attacks?? 
3. You state that you perturb “only 10% of the edges” for NETTACK/METAATTACK (unnoticeability constraint). For the node-injection attacks (NI/SI/MSI): (a) exactly how many nodes are injected per target / per graph? (b) Is there a budget expressed as a % of total nodes or edges?
4. Is the LLM corrector necessary for GraphLLMs, which uses LLM to predict the results? Please provide an ablation study.  What’s the similarity between the adversarial graph and the sanitized graph in terms of graph structure and features?
soundness: 2
strengths: 1. The paper identifies and systematically explores attack surfaces that are unique to LLM-based graph encodings (e.g., node-sequence template injection).
2. The node sequence template injection (NI/SI/MSI) is a convincing new attack tailored to LLAGA’s fixed-shape neighborhood template and shows strong evasion success on small graphs.
3. Proposed a new defensive framework that integrates LLM for feature correction.
summary: This paper claims to be the first systematic study of adversarial attacks and defenses for graph-aware LLMs. It adapts classic GNN attacks (NETTACK, METAATTACK), discovers a new node-sequence injection attack that targets LLAGA’s template, develops imperceptible text feature attacks, and proposes an end-to-end defense (GALGUARD) combining an LLM-based feature corrector with adapted GNN defenses. The experiments use standard benchmarks (Cora, Citeseer, PubMed) to evaluate the robustness of graphLLMs under structural and textual attacks.
weaknesses: 1. The experimental baseline set is narrow: more modern graph attack/defense baselines and a broader set of graph-LLM pipelines (and LLM sizes/types) would strengthen claims, e.g., “Does the Choice of LLM Affect Attack Performance?”
2. The paper states a 10% edge perturbation budget for NETTACK/METAATTACK experiments, but it does not clearly state per-experiment budgets for node-injection attacks (how many nodes can an attacker inject before detection?), or how “unnoticeability” is operationalized for injection and feature attacks.
3. The paper swaps Vicuna and LLaMA-2 in ablations and reports similar trends, but more extensive evaluation over different LLMs (and different GNN encoders / projectors) would strengthen the “choice of LLM has minimal impact” claim. Relatedly, the surrogate model used by NETTACK/METAATTACK is a linearized 2-layer GCN, and authors should justify that this surrogate is adequate to model LLM-projector interactions.

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 2
questions: For the rebuttal, I would suggest that the authors provides and answers the following important elements to further enhance their manuscript:
- Could you provide more clarification and elements regarding the performance difference between LlaGa and GraphPrompter? It would be nice to visualise how does the intermediate output of these elements is affected by the structural perturbations.
- Could you add elements regarding state-of-the-art adversarial defenses (for instance GCORN) that focuses on feature-based attacks? 
- Could you provide additional elements regarding the complexity (for instance time complexity) of the proposed GalGuard?
soundness: 3
strengths: - As the integration between graph-structured data and LLMs have recently attracted a lot of attention, understanding their adversarial robustness is clearly an important subject.
- This work seems to be (from my understanding and knowledge) the first to tackle this aspect.
- The writing and approach is very clear, specifically the taxonomy makes it easier to understand how to interact with the rest of the attack/defense in this specific context.
summary: The authors are interested in the subject of adversarial robustness in the context of approaching graph-related tasks using Large Language Models (LLMs). The authors start by exploring the application of previously available graph attacks, and then consequently propose a new attack adaptation that exploits putting placeholders into the input node sequence template. The authors also present a novel end-to-end defense strategy, denoted as GalGuard to defend against these perturbations.
weaknesses: - Some elements in the analysis are not very clear:
    - In Section 4.2, I didn’t understand your claim regarding the difference between LlaGa and GraphPrompter. Specifically, the latter is also based on message-passing framework, and therefore should also be subject to vulnerability when the structure is modified. 
- The proposed attack, while novel and clearly clever, is rather adapted for the LLaGa and therefore lacks the desired transferability for other approaches. For instance, if all the graph attack approaches have only focused on changing some specific elements in one specific model, the overall attack literature would still be limited. 
- When considering defense methodology, and specifically in the case of node feature-based attacks, you don’t compare or adapt state-of-the-art defense in that specific direction. Namely GCORN [1], which is interested in defending that specific line of attack or AirGNN [2].
- A specific discussion of the complexity of the proposed defense methodology is missing.

—

[1] Bounding the Expected Robustness of Graph Neural Networks Subject to Node Feature Attacks. - ICLR 2024.

[2] Graph Neural Networks with Adaptive Residual. - NeurIPS 2021.

---

confidence: 5
contribution: 2
initial_rating: 2
presentation: 2
questions: See weaknesses.
soundness: 3
strengths: 1. The paper identifies a previously unknown vulnerability in GraphPrompter and LLaGA.
2. The paper provides a comprehensive analysis of feature perturbation. 
3. The propsed GalGuard framework is effective in defending.
summary: This paper investigates the adversarial robustness of graph-aware LLMs, focusing on two representative models: LLAGA and GraphPrompter. The authors adapt existing GNN adversarial attacks to the graph-aware LLM paradigm and discover a novel attack surface specific to LLAGA's node sequence template design. The work introduces imperceptible feature perturbation attacks using homoglyphs and character reordering, demonstrating that graph-aware LLMs are particularly vulnerable to textual feature manipulations. To defend against these attacks, the authors propose GALGUARD, an end-to-end defense framework combining an LLM-based feature corrector with adapted GNN-based structural defenses.
weaknesses: 1. My most concern is about the comprehensiveness. 
Only two graph-aware LLM methods are evaluated (LLaGA and GraphPrompter, see Section 3), which is insufficient to draw generalizable conclusions about the entire paradigm. The paper claims to investigate "graph-aware LLMs" broadly but provides no evidence for other prominent methods in the taxonomy (Section 2, Appendix A.1). \
The attack for node sequence template seems solely effective for LLaGA, the applicability of it to other Graph-LLM is unclear.

2. The relationship between graph sparsity and vulnerability is observed (denser Cora more vulnerable than sparse PubMed, Section 4.1, page 5) but not rigorously analyzed. 
PubMed also differ from Cora and CiteSeer in paper sub-category, text length and many other aspects. 
More analysis are required to reach solid conclusions.

3. Missing comparison to existing works exploring Graph adversarial attacks with LLMs. [1-4]

4. The attack is named Mettack. And Table 2 is confusing. Why GalGuard is compared with attack method Mettack?

5. The ArXiv dataset experiment (Appendix A.6, Table 8) shows different attack patterns than smaller datasets, but the paper does not explain why poisoning significantly outperforms evasion on ArXiv (opposite of other datasets). This suggests the findings may not scale consistently.

6. GALGUARD's scalability to graphs with millions of nodes is questionable given its reliance on GPT-4 Turbo API calls for every node's features.


[1] Zhang, Qihai, et al. "TrustGLM: Evaluating the Robustness of GraphLLMs Against Prompt, Text, and Structure Attacks." Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 2025.

[2] Guo, Kai, et al. "Learning on graphs with large language models (llms): A deep dive into model robustness." arXiv preprint arXiv:2407.12068 (2024).

[3] Zhang, Zhongjian, et al. "Can large language models improve the adversarial robustness of graph neural networks?." Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1. 2025.

[4] Lei, Runlin, et al. "Intruding with words: Towards understanding graph injection attacks at the text level." Advances in Neural Information Processing Systems 37 (2024): 49214-49251.

[5] Chen, Zhikai, et al. "Exploring the potential of large language models (llms) in learning on graphs." ACM SIGKDD Explorations Newsletter 25.2 (2024): 42-61.


Decision: Reject