Learning on Graphs with Large Language Models (LLMs): A Deep Dive into Model Robustness

## Abstract

Abstract Large Language Models (LLMs) have demonstrated remarkable performance across various natural language processing tasks. Recently, several LLMs-based pipelines have been developed to enhance learning on graphs with text attributes, showcasing promising performance. However, graphs are well-known to be susceptible to adversarial attacks and it remains unclear whether LLMs exhibit robustness in learning on graphs. To address this gap, our work aims to explore the potential of LLMs in the context of adversarial attacks on graphs. Specifically, we investigate the robustness against graph structural and textual perturbations in terms of two dimensions: LLMs-as-Enhancers and LLMs-as-Predictors. Through extensive experiments, we find that, compared to shallow models, both LLMs-as-Enhancers and LLMs-as-Predictors offer superior robustness against structural and textual attacks.
Based on these findings, we carried out additional analyses to investigate the underlying causes. Furthermore, we have made our benchmark library openly available to facilitate quick and fair evaluations, and to encourage ongoing innovative research in this field.

## 1 Introduction

In recent years, significant progress has been made in the development of Large Language Models (LLMs) like Sentence-BERT [], GPT [], LLaMA [], etc. These variants showcase exceptional performance across a range of natural language processing tasks, such as sentiment analysis [, ], machine translation [, ], and text classification [, ]. While LLMs are widely employed for handling plain text, there is an increasing trend of applications where text data is linked with structured information represented as text-attributed graphs (TAGs) [, ]. Recently, solely utilizing LLMs for graph data has proven effective in various downstream graph-related tasks, and integrating LLMs with Graph Neural Networks (GNNs) [, ] can further enhance graph learning capabilities [].

Although graph machine learning methods with LLMs (Graph-LLMs) have reported promising performance [, , , , , , , , , , ], their robustness to adversarial attacks remains unknown.
Robustness has always been a crucial aspect of model performance, especially in high-risk tasks like medical diagnosis [], autonomous driving [], epidemic modeling [], where failures can have severe consequences. It is well-known that graph learning models such as GNNs are vulnerable to adversarial attacks, where adversaries manipulate the graph structure to produce inaccurate predictions [, ]. Moreover, text attributes in TAGs are also at risk of manipulation by attackers, which further raises concerns about the reliability of graph learning algorithms in safety-critical applications. In the era of LLMs embracing graphs, a critical question arises: Are Graph-LLMs robust against graph adversarial attacks?

Figure: Figure 1: An overview of our benchmark. The evaluation is divided into two perspectives: LLMs-as-Enhancers and LLMs-as-Predictors, both of which consider structural and textual attacks.
Refer to caption: x1.png

To address this question, we identify several research gaps in existing evaluations that hinder our understanding of current methods. These include: (1) Limited Structural Attacks for Graph-LLMs: Existing structural attacks are tailored for GNNs and have not been tested on Graph-LLMs, leaving it uncertain whether Graph-LLMs are sensitive to subtle structural changes within graphs. Understanding this sensitivity is crucial for developing robust Graph-LLMs.
(2) Limited Textual Attacks for TAGs: Current feature attacks [, ] particularly manipulate node attributes in the embedding of continuous space, rather than in textual space. This calls for creating an evaluation framework for assessing the robustness of Graph-LLMs to textual attacks. This area has been minimally explored, and it is essential to understand how text manipulations within graphs can impact model performance.
(3) Diverse Architectures of Graph-LLMs: Various strategies exist for utilizing LLMs for graph data, with approaches such as LLMs-as-Enhancers [, , ]and LLMs-as-Predictors [, , ] being among the most popular. This diversity necessitates the development of specialized attack pipelines to address the unique characteristics of different Graph-LLM architectures.

In response to these challenges, we aim to conduct a fair and reproducible evaluation of both structural and textual attacks under the representative pipelines of Graph-LLMs for node classification. Our contributions can be summarized as follows:

- •
New evaluation perspective. Different from existing works focusing on the predictive power of Graph-LLMs,
we stress test the robustness of Graph-LLMs against graph adversarial attacks. To the best of our knowledge, this problem has not been studied before. Our robustness assessment of each methodology includes both structural attack and textual attack aspects, while previous investigations on textual attacks on graphs are rather limited.
- •
Reproducible and comprehensive comparison.
We conduct a comprehensive comparison of various Graph-LLM pipelines across multiple datasets. Specifically, we systematically evaluate two distinct groups of Graph-LLMs: LLMs-as-Enhancers and LLMs-as-Predictors. To guarantee reproducibility and fairness in our comparison, we fine-tune all models using the same set of hyper-parameters. We employ two types of evaluation metrics: performance degradation percentage and attack success rate.
- •
Insights into the robustness of Graph-LLMs: This study reveals several interesting observations about the robustness of Graph-LLMs:
(a)
LLMs-as-Enhancers exhibit greater robustness against adversarial structural attacks compared to shallow embeddings like Bag of Words (BOW) and TF-IDF, particularly at high attack rates. Additionally, the better the distinguishability of the encoded features, the better the robustness.
(b)
LLMs-as-Predictors show better robustness in resisting structural attacks than MLP and the fine-tuned predictor holds even better robustness than vanilla GNN.
(c)
LLMs-as-Enhancers demonstrate excellent robustness against textual attacks, with GCN being significantly more robust than MLP as the victim model.
(d)
Text entropy, text length, and node centrality show certain negative correlations with the attack success rate of the textual attack for Graph-LLMs.
- •
Open-source benchmark library: To support and advance future research, we have developed an easy-to-use open-source benchmark library, now publicly available on GitHub: [https://github.com/KaiGuo20/GraphLLM_Robustness](https://github.com/KaiGuo20/GraphLLM_Robustness). This library allows researchers to quickly evaluate their own methods or datasets with minimal effort. Additionally, we have outlined potential future directions based on our benchmark findings to inspire further investigations.

## 2 Formulations and Background

We begin by providing preliminaries on graph neural networks, and then formalize the graph adversarial attacks. Finally, we briefly introduce the developments in large language models on graphs. More details are shown in Appendix A.

Notations. We define a graph as $G=(V,E)$, where $V$ denotes the node set and $E$ represents the edge set. We employ $\mathbf{X}\in\mathbb{R}^{N\times d}$ to denote the node feature matrix, where $N$ is the number of nodes and
$d$ is the dimension of the node features. Furthermore, we use the matrix $\mathbf{A}\in\mathbb{R}^{N\times N}$ to signify the adjacency matrix of $G$.
Finally, the graph data can be denoted as $G=(\mathbf{A},\mathbf{X})$.

Graph Neural Networks. GCN [] is one of the most representative models of GNNs, utilizing aggregation and transformation operations to model graph data. Unlike GCN, which treats all neighbors equally, GAT [] assigns different weights to different nodes within a neighborhood during aggregation.

Graph Adversarial Attacks.
While graph adversarial attacks can perturb node features or graph structures, most existing attacks focus on modifying the graph structure due to its complexity and effectiveness. These modifications often involve adding, deleting, or rewiring edges [, , , , , , , , ], exemplified by the PGD [] and PRBCD[] attacks. Besides evaluating structural attacks, we also adopt a more direct approach by using textual attacks instead of feature attacks targeting continuous features for evaluation, which remains a relatively unexplored direction.

Textual Attack.
Textual attacks can be performed on different levels like character level or sentence level according to the target to be perturbed. In this work, we focus on word-level attacks, applying substitutions to fool the classifier with minimal text perturbation. For example, SemAttack [] generates adversarial text by employing various semantic perturbation functions.

Large Language Models (LLMs) on Graphs.
Recent advances in Large Language Models (LLMs) like BERT [], Sentence-BERT (SBert) [] E5 [], GPT [], LLaMA [] and their variants have significantly impacted graph-related tasks. Two main paradigms are LLMs-as-Enhancers, improving node features (e.g., TAPE [], KEA [], GLEM []), and LLMs-as-Predictors, leveraging LLMs for graph predictions (e.g., InstructGLM [], GraphLLM [], GraphGPT []).

## 3 Benchmark Design

To deepen our understanding of the potential of Graph-LLMs in the context of robustness on graph learning, we need to design diverse pipelines to systematically assess the robustness of LLM approaches against adversarial attacks on graphs. Our benchmark evaluation encompasses two pivotal dimensions: LLMs-as-Enhancers and LLMs-as-Predictors. In this section, we will introduce the benchmark design. Details about the benchmark datasets are provided in Appendix B.

Figure: (a)
Refer to caption: x2.png

### 3.1 Threat Model

We describe the characteristics of the graph adversarial attacks we developed, including both structural and textual attacks, from the following aspects. (1) Adversary’s Goal: The primary objective is focused on evasion attacks, where the adversary seeks to manipulate the input graph data at inference time to cause the model to make incorrect predictions. In this scenario, the adversaries do not have the authority to change the classifier or its parameters.
(2) Victim Models:
The targets of these attacks include LLMs-as-Predictors and LLMs-as-Enhancers, of which the details will be thoroughly elaborated in Sections 3.2 and 3.3. (3) Adversary’s Knowledge: The attacks are designed under white-box and grey-box frameworks, meaning that the adversary either possesses complete knowledge of the model architecture and parameters or not respectively. White-box attacks are employed during the LLM-as-Enhancers experiments to evaluate robustness in the worst-case scenarios. However, for LLM-as-Predictors, it becomes impractical to perform white-box attacks due to the significant time costs associated with these large, complex models, and thus we adopt a grey-box setting.

### 3.2 LLMs-as-Enhancers

For LLMs-as-Enhancers, our benchmark provides a fair and comprehensive comparison of existing representative methods from two perspectives: structural attack and textual attack.

Structural attack:
We have discussed two commonly used methods for structural attacks, PGD [] and PRBCD [], in Section 2.
Currently, existing frameworks rely on shallow features such as Bag of Words (BOW) and TF-IDF. In the era of LLMs, it’s imperative to examine the impact of LLM features on structural attacks. Therefore, we designed a pipeline for structural attacks using LLMs-as-Enhancers. Specifically, we first generate diverse feature types derived from various LLMs, including SBert, E5, LLaMA, and Angle-LLaMA [], as well as LLaMA fine-tuned (LLaMA-FT) with LoRa [], etc. We then utilize these features to evaluate the performance of structural attacks. The pipeline is visualized in Figure 2(a).

Textual attack:
We conduct an evaluation on text attacks to verify whether LLMs-as-Enhancers can withstand textual attacks compared to traditional text preprocessing techniques. Specifically, in the white-box setting, we first conduct text attacks by using SemAttack [] on the texts of text-attributed graphs. Then, we encode the texts using different methods such as traditional techniques and LLMs. Finally, we assess their performance on GCN and MLP. Incidentally, to enhance efficiency on LLMs, we modified SemAttack for batch-wise operation instead of word-level processing. The pipeline is illustrated in Figure 2(b).

### 3.3 LLMs-as-Predictors

For LLMs-as-Predictors, we also perform structural and textual attacks on pre-trained and fine-tuned LLMs, respectively. Different from the attacks on LLMs-as-Enhancers, white-box attacks on the predictors can bring enormous computational costs due to the complexity of the models. Therefore, this study adopts the grey-box setting, choosing LLMs-as-Enhancers as the victim model and transferring the attacked texts or graphs to the LLMs-as-Predictors. For pre-trained models, we utilize the same pipeline in [], which describes the graph structure in text and inputs it along with text features directly into GPT-3.5 for prediction.
For fine-tuned models, we perform attacks on InstructGLM [], which uses LLaMA [] as the backbone and is fine-tuned on different benchmark datasets.

Structural attack:
Although the LLMs-as-Predictors in this study flatten graph structure into texts and only incorporate a small number of neighbors when predicting a node, it is possible that introducing irrelevant or false neighbors can influence the prediction results. During the structure attack, we use PRBCD as the attack algorithm and choose SBert and GCN as the surrogate model. The perturbed graph then serves as the input of GPT-3.5 or InstructGLM. The pipeline is depicted in Figure 2(c).

Textual attack:
The LLMs-as-Predictors directly utilize texts as inputs. Thus, perturbing the input texts is also likely to have impacts to the prediction results. In this study, we first perform SemAttack on the selected nodes with SBert and GCN as the surrogate model. After that, the perturbed texts are used to evaluate GPT-3.5 and InstructGLM. The pipeline is shown in Figure 2(d).

## 4 Experiments

In this section, we assess the robustness of LLMs against graph adversarial attacks in their two roles: LLMs-as-Enhancers and LLMs-as-Predictors. Specifically, we aim to answer the following questions: Q1: How effective are the LLMs-as-Enhancers on structural attack? Q2: What is the effectiveness of LLMs-as-Enhancers on textual attacks?
Q3: How effective are the LLMs-as-Predictors on structural attack?
Q4: How do LLMs-as-Predictors perform on textual attacks?

### 4.1 Structural Attack for LLMs-as-Enhancers

**Table 1: Performance of LLMs-as-Enhancers against 0%, 5%and 25% structural attacks. GAP represents the percentage (%) decrease in performance after an attack compared to the clean performance. N/A indicates that TAPE does not provide the explanation features. We use pink to denote the best performance, green for the second-best, and yellow for the third-best.**
|  |  | BOW | TF-IDF | SBert | E5 | LLaMA | Angle-LLaMA | Explanation | Ensemble | LLaMA-FT |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Dataset | Ptb. | ACC | GAP | ACC | GAP | ACC | GAP | ACC | GAP | ACC | GAP | ACC | GAP | ACC | GAP | ACC | GAP | ACC | GAP |
| Pudmed | 0% | 74.69 | 0% | 76.86 | 0% | 78.71 | 0% | 81.83 | 0% | 77.65 | 0% | 75.15 | 0% | 88.84 | 0% | 83.59 | 0% | 77.4 | 0% |
| 5% | 72.40 | 15.90% | 74.68 | 2.8% | 74.68 | 5.1% | 79.39 | 3.0% | 76.68 | 1.3% | 74.65 | 0.67% | 87.32 | 1.5% | 82.55 | 1.2% | 76.55 | 1.1% |  |
| 25% | 62.83 | 24.30% | 66.18 | 13.9% | 69.24 | 12.0% | 71.06 | 13.1% | 74.62 | 3.9% | 72.72 | 3.2% | 82.11 | 7.6% | 77.85 | 6.9% | 75.63 | 2.3% |  |
| Arxiv | 0% | 50.99 | 0% | 48.39 | 0% | 52.51 | 0% | 57.04 | 0% | 58.04 | 0% | 58.53 | 0% | 54.37 | 0% | 59.9 | 0% | 52.46 | 0% |
| 5% | 42.35 | 16.9% | 43.32 | 10.5% | 47.59 | 9.4% | 48.15 | 15.6% | 51.76 | 10.8% | 51.46 | 12.1% | 47.19 | 13.2% | 54.33 | 9.3% | 47.59 | 9.3% |  |
| 25% | 18.95 | 62.9% | 24.36 | 49.7% | 29.24 | 44.3% | 25.24 | 55.8% | 31.67 | 45.4% | 31.73 | 45.8% | 21.61 | 60.3% | 32.82 | 45.2% | 29.19 | 44.4% |  |
| Cora | 0% | 78.49 | 0% | 81.46 | 0% | 81.99 | 0% | 83.17 | 0% | 78.13 | 0% | 80.28 | 0% | 82.79 | 0% | 82.57 | 0% | 78.54 | 0% |
| 5% | 73.71 | 3.1% | 77.23 | 5.2% | 79.18 | 3.4% | 80.43 | 3.3% | 70.91 | 9.2% | 80.23 | 0.06% | 80.47 | 2.8% | 80.67 | 2.3% | 78.51 | 0% |  |
| 25% | 60.72 | 22.6% | 67.53 | 17.1% | 72.14 | 12.0% | 69.84 | 16.0% | 69.99 | 10.4% | 76.75 | 4.4% | 71.32 | 13.9% | 74.89 | 9.3% | 70.72 | 10.0% |  |
| WikiCS | 0% | 74.92 | 0% | 75.96 | 0% | 75.89 | 0% | 76.71 | 0% | 79.72 | 0% | 73.33 | 0% | N/A | N/A | N/A | N/A | 79.69 | 0% |
| 5% | 61.46 | 18.0% | 62.03 | 18.3% | 64.44 | 15.1% | 63.01 | 17.9% | 70.91 | 11.1% | 67.0 | 8.6% | N/A | N/A | N/A | N/A | 70.11 | 12.0% |  |
| 25% | 45.34 | 39.5% | 45.74 | 39.8% | 50.19 | 33.9% | 46.38 | 39.6% | 59.23 | 25.7% | 56.75 | 22.6% | N/A | N/A | N/A | N/A | 60.13 | 24.6% |  |
| History | 0% | 51.25 | 0% | 58.38 | 0% | 65.59 | 0% | 66.96 | 0% | 66.73 | 0% | 66.52 | 0% | N/A | N/A | N/A | N/A | 64.53 | 0% |
| 5% | 49.04 | 4.3% | 57.57 | 1.4% | 65.6 | 0% | 64.69 | 3.4% | 61.64 | 7.6% | 63.47 | 4.6% | N/A | N/A | N/A | N/A | 62.99 | 2.4% |  |
| 25% | 38.93 | 24.0% | 22.66 | 61.2% | 56.34 | 14.1% | 55.37 | 17.3% | 54.54 | 18.3% | 52.75 | 20.7% | N/A | N/A | N/A | N/A | 53.34 | 17.4% |  |
| Citeseer | 0% | 70.74 | 0% | 73.02 | 0% | 74.94 | 0% | 75.14 | 0% | 67.48 | 0% | 71.71 | 0% | N/A | N/A | N/A | N/A | 69.7 | 0% |
| 5% | 68.84 | 2.7% | 71.40 | 2.2% | 72.60 | 3.1% | 72.29 | 3.8% | 70.82 | 5.0% | 68.46 | 4.5% | N/A | N/A | N/A | N/A | 68.62 | 1.6% |  |
| 25% | 61.76 | 12.7% | 64.36 | 11.9% | 64.51 | 13.9% | 64.38 | 14.3% | 64.69 | 4.1% | 68.61 | 4.3% | N/A | N/A | N/A | N/A | 66.17 | 5.1% |  |

Experiment Design.
To tackle the research question Q1, we enhance text attributes using LLMs and generate new features. These enriched features are then used to train a GCN as the victim model. Specifically, we use PGD to conduct white-box evasion attacks on the structures of small graphs such as Cora, Citeseer, Pubmed, and Wikics, while we employ PRBCD to conduct white-box evasion attacks on the structures of large graphs Arxiv and History.
We vary the perturbation rates at 0% (clean graphs), 5%, and 25%, which represent the ratio of perturbed edges to original edges. Subsequently, we feed the perturbed graphs into different LLMs-as-Enhancers architectures and compare the LLM features with shallow features. To quantify model robustness, we report the test accuracy (ACC) and the percentage accuracy degradation after attacks compared to the original accuracy (GAP).

Results. The results are reported in Table 1. Further details, including standard deviations, can be found in Appendix D. From these results, we have the following observations.

Performance comparison on clean graphs.
The features generated by pre-trained language models exhibit better performance on most clean datasets. For instance, SBert and e5-large (E5) show improvements of 2.4% and 6.5% respectively on clean Pubmed datasets compared to TF-IDF.

Robustness against structural attacks.
For 25% evasion attacks, almost all language models exhibit greater robustness compared to traditional BOW and TF-IDF approaches. For example, in the case of a 25% evasion attack on Pubmed, while TF-IDF experiences a decrease of 13.9%, LLaMA only drops by 3.9%. Moreover, we observe that fine-tuning enhances the robustness of LLMs, as demonstrated by the fact that fine-tuned LLaMA exhibits greater robustness compared to its unfine-tuned counterpart.
Further, by comparing the results on 5% and 25% attacks, we find that LLMs-as-Enhancers are more helpful at higher perturbation rates. For example, with a 5% perturbation budget on Cora, only 4 language models show a lower GAP compared to the shallow BOW features, while this number increases to 7 at the perturbation rate of 25%.
However, by examining the accuracy on clean graphs, we note that the performance of LLM features still declines considerably on certain datasets, such as Arxiv, under high perturbation rates, indicating that graph LLMs remain vulnerable to attacks.

Robustness of explanation features.
The explanation features generated by TAPE not only exhibit higher accuracy on clear graphs but also demonstrate good robustness against evasion attacks. When ensemble with LLaMA features, these explanation features show higher accuracy and robustness compared to using LLaMA alone.

Key Takeaways 1: Most LLMs demonstrate greater robustness against structural attacks compared to shallow models.

Key Takeaways 2: The higher the attack rate, the more robust the features of LLMs compared to shallow features.

### 4.2 Textual Attack for LLMs-as-Enhancers

Experiment Design.
To answer Q2, we conduct white-box evasion attacks on LLMs-as-Enhancers, targeting textual attributes.
We first utilize diverse LLMs-as-Enhancers to transform the text into node embeddings and train a GCN and an MLP. Then we perform the evasion attack at the model inference stage by perturbing text using SemAttack. Then, LLMs are used to transform the perturbed text into node embeddings, which will then be fed into the trained GCN or MLP for inference. For all models and datasets, we randomly sample 200 nodes as target nodes and utilize the Attack Success Rate (ASR) [] as the evaluation metric.

Results. The results are reported in Table 2. Additional details, including standard deviations of performance, are provided in Appendix E. From these results, we can draw the following observations.

LLaMA performs well against textual attack when MLP is used as the victim model.
When using MLP as the victim model, E5 and LLaMA demonstrate greater resilience against SemAttack, with a noticeable downward trend in ASR for SBert, E5, and LLaMA models. For example, on the WikiCS dataset, SBert has an ASR of 62.45%, while the performance of LLaMA dropped to 22.17%. Another interesting observation is that BOW shows better robustness than SBert on Cora, Pubmed, and Arxiv. Given that BOW has a limited input of words, the robustness of BOW is likely to come from filtering the perturbed words, whose frequencies are often low.

For textual attack, GCN as the victim model is more robust compared to MLP as the victim model.
In terms of GCN as the victim model, fine-tuned LLaMA achieves the lowest ASR among all datasets, ranging from 2.98% to 6.91%. Also, compared to MLP, the ASR for GCN decreases significantly and remains below 20% across all datasets.

Key Takeaways 3:
Among all models and settings, the fine-tuned model, LLaMA-FT, generally exhibits the best robustness against the textual attack on most datasets.

Key Takeaways 4:
In the LLMs-as-Enhancers framework, GCN greatly improves the robustness against textual attacks compared to MLP as the victim model.

**Table 2: Performance of LLMs-as-Enhancers against the textual attack. Bold numbers represent the lowest Attack Success Rate (ASR), indicating superior robustness.**
|  | Cora | Pubmed | Arxiv | Wikics | History | Citeseer |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Features | MLP | GCN | MLP | GCN | MLP | GCN | MLP | GCN | MLP | GCN | MLP | GCN |
| BOW | 62.11 | 9.27 | 41.00 | 8.58 | 72.69 | 15.19 | 67.85 | 3.82 | 74.53 | 18.40 | 68.80 | 18.98 |
| SBert | 73.18 | 14.76 | 45.36 | 9.32 | 82.69 | 11.17 | 62.12 | 9.08 | 76.73 | 13.41 | 66.33 | 16.93 |
| E5 | 65.29 | 10.53 | 35.62 | 8.76 | 81.92 | 15.22 | 65.99 | 6.64 | 61.51 | 6.54 | 57.10 | 14.69 |
| LLaMA | 56.49 | 12.58 | 19.66 | 4.95 | 67.87 | 6.05 | 22.17 | 4.17 | 65.07 | 15.92 | 46.81 | 13.77 |
| LLaMA-FT | 40.10 | 4.37 | 16.69 | 3.27 | 67.71 | 6.08 | 30.00 | 2.98 | 56.98 | 6.91 | 38.46 | 6.58 |

### 4.3 Structual Attack for LLMs-as-Predictors

Experiment Design. To answer Q3, we explore the robustness of LLM-as-Predictors against graph structural attacks, using GPT-3.5 and InstructGLM as the selected predictors. Given the difficulty in directly attacking these predictors due to their large number of parameters or lack of model access, we adopt a grey-box setting. In this setting, LLMs-as-Enhancers are used as surrogate models during adversarial attacks, and the resulting attacked graph structures are then fed into the LLMs-as-Predictors.
When employing GPT-3.5 in the graph domain, we follow the approach in [] and evaluate 200 nodes in the test set. For InstructGLM, we directly attack the pre-processed datasets provided by the author.

Results.
The results of InstrucGLM are reported in Table 3. More results about GPT-3.5 are presented in Figure 5 in Appendix F.
Based on these results, we can make the following observations.

GPT-3.5 shows the strongest robustness against structural attack in the zero-shot setting.

We evaluate the model under various few-shot and zero-shot settings with the input of summarized two-hop neighbors, as illustrated in [].
The results demonstrate that GPT-3.5 maintains the highest robustness in the zero-shot setting, with minimal performance degradation even under significant perturbations. We conjecture that the neighbor sampling and summarizing processes likely mitigate noise introduced by structural alterations. Although GAT also uses the migrated attacked graph structure from GCN, its accuracy drops much faster than that of GPT-3.5.

Similarly, InstructGLM also shows stronger robustness against structural attack compared to GCN.
While GCN experiences a noticeable accuracy decrease after 5% structural perturbation, InstructGLM maintains performance close to that on the clean graph. Like GPT-3.5, InstructGLM also employs neighbor sampling and its robustness may benefit from this procedure.

Key Takeaways 5: LLMs-as-Predictors show strong robustness against the structural attack, especially in the zero-shot setting.

**Table 3: The robustness of the predictor InstructGLM against 5% structural attack. GAP refers to the percentage (%) performance decrease after an attack compared to the clean ACC. The bold font is used to highlight the smallest gap.**
| Dataset | Cora | Pubmed | Arxiv |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| model \ Perturbation Rate | Clean | Attack | GAP | Clean | Attack | GAP | Clean | Attack | GAP |
| GCN | 87.45 | 81.73 | 6.54% | 87.07 | 84.25 | 3.24% | 57.5 | 52.94 | 7.93% |
| InstructGLM (structure-aware) | 82.47 | 80.73 | 2.11% | 91.63 | 91.10 | 0.58% | 72.87 | 71.84 | 1.41% |

### 4.4 Textual Attack for LLMs-as-Predictors

Experiment Design.
To answer Q4, we explore the performance of GPT-3.5 and InstructGLM with perturbed texts as inputs. The perturbed texts are generated by SemAttack and used as adversarial inputs for GPT-3.5 and InstructGLM to evaluate the robustness of LLMs-as-Predictors against textual attacks. Following the experiment design in Q3, the experiment is conducted in a grey-box setting. We use GCN with SBert embeddings as the surrogate model, randomly sampling 200 target nodes for attack and evaluation.

Results. The results of InstructGLM are presented in Table 4. More results about GPT-3.5 are presented in Table 11 in Appendix G.
These results lead us to the following observations.

GPT-3.5 shows stronger robustness compared to MLP but failed to exceed GCN.
As shown in Table 11, our experiment reveals that GPT-3.5 has strong robustness compared to MLP but fails to exceed GCN and GAT in all few shot settings. On the other hand, GAT exhibits the strongest robustness, followed by GCN. However, on Cora and Pubmed, the Attack Success rate of GPT-3.5 is close to GCN, ranging from 0.55% to 1.70%.

InstructGLM shows the strongest robustness compared to both MLP and GCN.
Different from the above observations of GPT-3.5, the fine-tuned InstructGLM shows stronger robustness on the three datasets compared to GCN. While there is an ASR of 30.37% for GCN on Cora, InstructGLM can resist the textual perturbation and achieves an ASR of 1.21% with structural information incorporated. The 0% ASR result of MLP on the Arxiv dataset is due to the low ACC, as we can not find a sample that is both predicted correctly before the attack and predicted wrong after the attack. Also, this low accuracy is likely to be caused by the dataset used by InstructGLM, which only utilizes titles of a few words as the node attributes.

Key Takeaways 6:
LLMs-as-Predictors exhibit stronger robustness against textual attacks compared to MLP. However, GPT-3.5, which is not fine-tuned, shows poorer robustness compared to GCN.

**Table 4: The robustness of the predictor InstructGLM against textual attacks. The bold font is used to highlight the lowest Attack Success Rate (ASR).**
| Dataset | Cora | Pubmed | Arxiv |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Model | Clean ACC | ASR | Clean ACC | ASR | Clean ACC | ASR |
| MLP | 67.53 | 30.37 | 86.92 | 10.11 | 5.86 | 0 |
| GCN | 88.01 | 10.22 | 86.82 | 3.55 | 58.85 | 6.61 |
| InstructGLM (structure-aware) | 82.50 | 1.21 | 91.50 | 1.09 | 74.87 | 3.42 |

### 4.5 Analysis for Structural Attack

Since Sections 4.1 and 4.3 have shown the robustness of Graph-LLMs against the structure attack, we conduct analysis from the following perspectives to explore the reasons behind such robustness.

Figure: (a) BOW
Refer to caption: x6.png

t-SNE visualization.
First, we examine t-SNE visualizations of various initial features and find that initial features generated by language models are more distinguishable in categories compared to traditional features as shown in Fig. 3.

DBI. To further evaluate the separability of input features, we use the Davies-Bouldin Index (DBI) [], as shown in Table 5. The DBI score represents the average similarity measure between each cluster and its most similar cluster, with lower scores indicating better clustering quality. A score of zero is ideal, signifying optimal clustering. We find that initial LLM features have lower initial DBI scores. We further examine the DBI of embeddings both before and after the attack, and the differences therein. Notably, embeddings from larger models exhibit a smaller decrease in DBI scores post-attack, as indicated by the DBI Diff in the Table 5.

**Table 5: DBI and homophily of Cora. “Init DBI” indicates the DBI of initial features, “Embed DBI” represents the DBI of trained embedding, “Post-Attack DBI” refers to the DBI of embedding after an attack, “DBI Diff” donates the difference between Embed DBI and Post-Attack DBI. The homophily value of the graph before being attacked is 0.81.**
|  | GCN | MLP | Init DBI↓ | Embed DBI↓ | Post-Attack DBI↓ | DBI Diff↓ | GAP↓ | Homophily (0.81) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| BOW | 78.49 | 55.90 | 9.34 | 1.60 | 2.75 | 1.15 | 22.68% | 0.66 |
| TF-IDF | 81.46 | 65.85 | 8.85 | 1.24 | 1.89 | 0.65 | 17.10% | 0.66 |
| SBert | 81.99 | 70.80 | 4.47 | 1.28 | 1.56 | 0.28 | 12.00% | 0.70 |
| E5 | 83.17 | 69.16 | 5.92 | 1.27 | 1.66 | 0.39 | 16.00% | 0.67 |
| LLaMA | 78.13 | 67.48 | 4.88 | 1.83 | 1.90 | 0.07 | 10.41% | 0.68 |

Homophily.
Additionally, we analyze the homophily of the Cora dataset before and after attacks, as shown in Table 5. It discovers that the pre-attack Cora dataset has a homophily of 0.81, and after the attack, Cora with shallow features exhibits lower homophily.

Based on the above results, we find that robustness is strongly positively correlated with the quality of features, indicating that higher distinguishability of features leads to stronger robustness and higher homophily after attacks. This could be attributed to the richer information present in features generated by pre-trained language models, resulting in higher distinguishability in clustering. The higher the quality of the features, the less the model depends on the structure. Therefore, high-quality features of LLMs can be more robust against structural attacks.

### 4.6 Analysis for Textual Attack

From the experiments of the LLMs-as-Enhancers against textual attack in Section 4.2, we observe that LLaMA performs much better than other smaller models and the fine-tuned LLaMA exhibits the best robustness. In addition, GCN demonstrates much stronger robustness compared to MLP. To explore the reasons behind this, we perform analysis from feature and structure perspectives.

Figure: Figure 4: Centrality distributions of nodes being attacked successfully and unsuccessfully on WikiCS.
Refer to caption: x11.png

Feature Perspective. From the feature perspective, we assume that text attributes can have an impact on the attack success rate. Specifically, we first explore some basic indicators like text entropy, text length, and the number of words among successfully attacked nodes and failed attacked nodes, as shown in Table 6. Based on the observation with LLaMA and GCN as the victim model, it is obvious that the successfully attacked nodes tend to have smaller entropy (less richness of texts), shorter texts, and smaller amounts of words. By comparing the DBI of the fine-tuned and not fine-tuned model,
we also observe that the fine-tuned model always has a smaller DBI compared to the fine-tuned model.

**Table 6: Comparisons between successfully attacked nodes and failed attacked nodes from the feature perspective (success / failed), with LLaMA and GCN as the victim model.**
| Dataset | Pubmed | Citeseer | History |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Fine-tuned | w/o Fine-tuned | Fine-tuned | w/o Fine-tuned | Fine-tuned | w/o Fine-tuned |
| Entropy | 6.25/6.40 | 6.24/6.41 | 5.90/6.09 | 5.86/6.09 | 5.45/6.21 | 5.28/6.25 |
| Text Length | 206.10/237.22 | 197.94/237.64 | 125.17/148.94 | 124.07/149.38 | 114.71/250.45 | 95.51/237.71 |
| Words | 108.48/122.04 | 108.69/122.89 | 78.60/90.80 | 77.57/90.99 | 73.45/139.59 | 63.79/138.39 |
| DBI | 4.77 | 4.87 | 5.01 | 5.24 | 4.65 | 5.02 |

Structure Perspective.
From the structure perspective, we investigate the relations between the degree centrality of nodes and the attack success rate. As shown in Fig. 4, we use SBert and GCN as the victim model and visualize the distribution of degrees from successfully attacked nodes and failed attacked nodes respectively. The results clearly show that successfully attacked nodes often have smaller degrees, indicating that nodes with less structural information in the graph are more vulnerable to textual attacks. Additionally, similar patterns are observed for eigenvector centrality and PageRank values, which we detail in Appendix H due to page limit.

## 5 Conclusion and Future Directions

This work introduces a comprehensive benchmark for exploring the potential of LLMs in context of adversarial attacks on graphs. Specifically, we investigate the robustness against graph structural and textual attacks in two dimensions: LLMs-as-Enhancers and LLMs-as-Predictors.
Through extensive experiments, we find that, compared to shallow models, both LLMs-as-Enhancers and LLMs-as-Predictors offer superior robustness against structural and textual attacks.
Despite these promising results, several critical challenges and research directions remain worthy of future investigation.

Rethinking Textual Attack. Based on the observations above, we realize that textual attacks can significantly affect the prediction of individual samples. However, when GCN serves as the victim model, the incorporation of neighbor information helps mitigate these perturbations, significantly reducing the attack’s effectiveness. From an attack perspective, how the resistance capability of the GCN-based victim models can be weakened deserves attention, especially for attacking stronger Graph-LLMs.

Combining Textual and Structural Attack on Graphs.
To enhance attack capabilities, a combined framework that perturbs both text attributes and graph structure is needed. However, challenges such as integrating textual and structural attacks to improve attack efficiency remain unsolved. In this study, we provide preliminary results in the Appendix I. Our experiment shows that adding additional textual perturbations on top of structural perturbations can further degrade model performance.

Rethinking Graph-LLMs.
From the results in Table 1, we can conclude that the angle-optimized Angle-LLaMA, which is more suitable for text encoding, exhibits better robustness against adversarial attacks compared to LLaMA. This phenomenon may inspire us to design better Graph-LLMs for text attribute encoding. Finally, we can use LLMs to perform attacks on graphs by generating harmful structures and text attributes.

## Checklist

- 1.
For all authors…
(a)
Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes]
(b)
Did you describe the limitations of your work?
[Yes] See Section J
(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] See Section J
(d)
Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
- 2.
If you are including theoretical results…
(a)
Did you state the full set of assumptions of all theoretical results?
[N/A]
(b)
Did you include complete proofs of all theoretical results?
[N/A]
- 3.
If you ran experiments (e.g. for benchmarks)…
(a)
Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes]
(b)
Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes]
(c)
Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d)
Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Section C
- 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets…
(a)
If your work uses existing assets, did you cite the creators?
[Yes]
(b)
Did you mention the license of the assets?
[Yes] See Section B
(c)
Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d)
Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A]
(e)
Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A]
- 5.
If you used crowdsourcing or conducted research with human subjects…
(a)
Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b)
Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A]
(c)
Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]

## Appendix A Formulations and Background

We begin by providing preliminaries on graph neural networks, and then formalize the graph adversarial attacks. Finally, we introduce the developments in large language models on graphs.

Notations. We define a graph as $G=(V,E)$, where $V$ denotes the node set and $E$ represents the edge set. We employ $\mathbf{X}\in\mathbb{R}^{N\times d}$ to denote the node feature matrix, where $N$ is the number of nodes and
$d$ is the dimension of the node features. Furthermore, we use the matrix $\mathbf{A}\in\mathbb{R}^{N\times N}$ to signify the adjacency matrix of $G$.
Finally, the graph data can be denoted as $G=(\mathbf{A},\mathbf{X})$.

Graph Neural Networks. GCN [] is one of the most representative models of GNNs, utilizing aggregation and transformation operations to model graph data. Unlike GCN, which treats all neighbors equally, GAT [] assigns different weights to different nodes within a neighborhood during aggregation.

Graph Adversarial Attacks.
In the context of $G=(\mathbf{A},\mathbf{X})$ and a subset $V_{m}\subseteq V$ comprising victim nodes, where $y_{i}$ signifies the label for node $i$, the attacker’s objective is to discern a perturbed graph denoted as $\tilde{G}=(\tilde{\mathbf{A}},\tilde{\mathbf{X}})$. The primary goal is to minimize the attack objective $\mathcal{L}_{attack}$.

$$ $\min\mathcal{L}_{\text{attack }}\left(f_{\theta}(\tilde{G})\right)=\sum \nolimits_{i\in V_{m}}\ell_{\text{attack }}\left(f_{\theta^{*}}(\tilde{G})_{i} ,y_{i}\right)\text{ s.t., }\theta^{*}=\arg\min\nolimits_{\theta}\mathcal{L}_{ \text{train }}\left(f_{\theta}\left(\hat{G}\right)\right),$ (1) $$

where $f_{\theta}$ indicates the model function of GNN,
$\mathcal{L}_{attack}$ represents the loss function for attacks, one option is to set $\mathcal{L}_{attack}$ = $-\mathcal{L}$, and $\hat{G}$ can be either $G$ or $\tilde{G}$. Here, $\tilde{G}$ is chosen from a constrained domain $\Psi(G)$. Given a fixed perturbation budget $\mathbb{D}$, a typical constraint for $\Psi(G)$ can be expressed as
$\|\tilde{\mathbf{A}}-\mathbf{A}\|_{0}+\|\tilde{\mathbf{X}}-\mathbf{X}\|_{0}
\leq\mathbb{D}$.
This constraint implies that the perturbations introduced in the adjacency matrix $\tilde{\mathbf{A}}$ and node feature matrix $\tilde{\mathbf{X}}$ should be limited, and their combined $L_{0}$ should not exceed the specified budget $\mathbb{D}$.

While graph adversarial attacks can perturb either node features or graph structures, the complexity of structural information has led the majority of existing adversarial attacks on graph data to focus on modifying graph structure, particularly through actions such as adding, deleting, or rewiring edges [, , , , , , , , ].
For example, the PGD attack [] uses edge perturbation to overcome the challenge of attacking discrete graph structures via first-order optimization. In contrast, the PRBCD attack [] addresses the high cost of adversarial attacks on large graphs with a sparsity-aware optimization approach.

On one hand, we explore the robustness against structural attacks. On the other hand, instead of targeting continuous features as in existing feature attack works, we adopt a direct approach by employing textual attacks to evaluate robustness, which remains a relatively unexplored direction.

Textual Attack.
For tasks on TAGs, the raw inputs are in text format and it can be hard for attackers to manipulate the encoded features directly, which makes the traditional feature attacking on graphs less practical. Therefore, the textual attack is used in this study to evaluate the robustness of LLMs enhanced graph features. Textual attacks can be performed on different levels like character level or sentence level according to the target to be perturbed. In this work, we focus on word-level attacks, which can be defined as follows.

Word-level attacks. Given a classifier $f$ that predicts labels $y\in Y$, the input $X$ is defined in a categorical space and each input is a sequence of $n$ words ${x_{1},x_{2},\cdots,x_{n}}$. Each word $x_{i}$ has a limited amount of substitution candidates, denoted as $S(x)$. To keep the perturbation as unnoticeable as possible, a constraint, usually $L_{1}$ or $L_{2}$ distance is applied. Finally, to fool classifier $f$ to the largest extent, the following objective is maximized:

$$ $\begin{array}[]{ll}\arg\max_{x^{\prime}\in S(x)}L(f(x^{\prime}),y)\\ \text{ s.t.,}\quad||x^{\prime}-x||_{2}<\epsilon,\\ \end{array}$ (2) $$

where $L$ is the loss function between original prediction $y$ and prediction $f(x^{\prime})$ after perturbation. $\epsilon$ is the budget of the perturbation, which keeps the original and the perturbed sample as close as possible. As the loss function above is maximized, we will get a perturbed sample $x^{\prime}$ that makes the classifier $f$ generate a prediction far from the original output.
For example, SemAttack [] generates natural adversarial text by employing various semantic perturbation functions.

Large Language Models on Graphs.
In recent years, remarkable progress has been achieved in the field of Large Language Models (LLMs), with notable contributions from transformative architectures such as Transformers [], BERT [], Sentence-BERT (SBert) [] E5 [], GPT [], LLaMA [] and their variants. These LLMs can be applied to graph-related tasks. The collaboration between Large Language Models and Graph Neural Networks (GNNs) can be mutually advantageous, leading to improved graph learning. The two most popular paradigms for applying LLMs to graphs are LLMs-as-Enhancers and LLMs-as-Predictors.
LLM-as-Enhancers aim to enhance the quality of node features through the assistance of LLMs.
TAPE [] is a groundbreaking example of explanation-based enhancement, encouraging LLms to produce explanations and pseudo-labels for the augmentation of textual attributes. KEA [] instructs LLMs to produce a compilation of knowledge entities, complete with text descriptions, and encodes them using fine-tuned pre-trained language models (PLMs). GLEM [] considers pseudo labels generated by both PLMs and GNNs and incorporates them into a variational EM framework.
The fundamental concept of the LLMs-as-Predictors are to leverage LLMs for making predictions in graph-related tasks. InstructGLM [] formulates a set of scalable prompts grounded in the maximum hop level and fine-tunes LLMs to output predicted labels directly. GraphLLM [] derives the graph-enhanced prefix
from the graph representation. This method boosts the LLM’s capability in conducting graph reasoning tasks by graph-enhanced prefix tuning. The GraphGPT [] framework aligns LLMs with graph structural knowledge using a paradigm of graph instruction tuning.

## Appendix B Benchmark Datasets

To comprehensively and effectively assess the robustness of LLMs in graph learning, we present six text-attributed graphs that offer original textual sentences. For instance, renowned citation graphs such as Cora [], PubMed [], Citeseer [], and ogbn-arxiv (Arxiv) [] fall under the category of TAGs. These datasets extract node attributes from textual information, including titles and abstracts of papers. WikiCS [] serves as a Wikipedia-based dataset for benchmarking Graph Neural Networks. It comprises 10 classes corresponding to branches of computer science, demonstrating high connectivity. Node features are derived from the text of the respective articles. Moreover, we employ the History [] dataset sourced from Amazon, where node attributes originate from book titles and descriptions. For instance, "Description: Collection of Poetry; Title: The golden treasury of poetry". All datasets are utilized with a low labeling rate split, following the setting described in KEA []. The statistics of datasets are reported in Table 7.
The licenses of these datasets are MIT License.

**Table 7: Statistics of datasets.**
|  | Cora | Citeseer | Pubmed | WikiCS | History | ogbn-arxiv |
| --- | --- | --- | --- | --- | --- | --- |
| #Nodes | 2,708 | 3,327 | 19,717 | 11,701 | 41,551 | 169,343 |
| #Edges | 5,429 | 4,732 | 44,338 | 216,123 | 358,574 | 1,166,243 |
| #Class | 7 | 6 | 3 | 10 | 12 | 40 |
| #Domain | Academic | Academic | Academic | Wikipedia | E-commerce | Academic |

## Appendix C Experimental Settings

All algorithms in our benchmark are implemented by PyTorch []. All experiments are conducted on a Linux server with GPU (NVIDIA RTX A6000
48Gb and Tesla V100 32Gb), using PyTorch 2.0.0, PyTorch Geometric 2.4.0 [] and Python 3.10.13.
We train each model using binary cross-entropy loss, optimized with the Adam optimizer. We use PGD and PRBCD to attack the structures on graphs, which are implemented by DeepRobust []. For hyperparameter, a hidden size of 256 is used uniformly across all datasets. For the sake of reproducibility,
the seeds of random numbers are set to the same. For all the datasets and models, we tune the following hyper-parameters: learning rate: $lr\in\{0.01,0.001\}$, weight decay: $\lambda\in\{1e-4,5e-4\}$.

## Appendix D Results of LLMs-as-Enhancers against attacks on graph structures

To answer Q1, we conduct experiments to examine the robustness against structural attack for LLMs-as-Enhancers. The results are reported in Table 8 and Table 9, which include additional details on the standard deviation (std) compared to Table 1.

**Table 8: The performance of LLMs-as-Enhancers against 5% attacks on graph structures**
|  | Cora | Pudmed | Arxiv |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Feature | Clean | 5% Attack | Gap | Clean | 5% Attack | Gap | Clean | 5% Attack | Gap |
| BOW | 78.49 ± 1.13 | 73.71 ± 1.10 | 03.06% | 74.69 ± 2.07 | 72.40 ± 1.89 | 15.88% | 50.99 ± 2.15 | 42.35 ± 2.64 | 16.94% |
| TFIDF | 81.46 ± 1.21 | 77.23 ± 1.07 | 05.19% | 76.86 ± 1.34 | 74.68 ± 1.40 | 02.84% | 48.39 ± 1.15 | 43.32 ± 1.27 | 10.48% |
| SBert | 81.99 ± 0.76 | 79.18 ± 0.71 | 03.43% | 78.71 ± 1.17 | 74.68 ± 1.40 | 05.12% | 52.51 ± 0.87 | 47.59 ± 1.72 | 09.37% |
| E5 | 83.17 ± 0.73 | 80.43 ± 0.59 | 03.29% | 81.83 ± 1.16 | 79.39 ± 1.06 | 02.98% | 57.04 ± 1.77 | 48.15 ± 1.71 | 15.59% |
| Llama | 78.13 ± 1.07 | 70.91 ± 1.02 | 09.24% | 77.65 ± 0.74 | 76.68 ± 0.94 | 01.25% | 58.04 ± 1.79 | 51.76 ± 1.95 | 10.82% |
| Angle-Llama | 80.28 ± 1.42 | 80.23 ± 0.92 | 0.06% | 75.15 ± 2.50 | 74.65 ± 1.45 | 0.67% | 58.53 ± 1.56 | 51.46 ± 3.33 | 12.08% |
| Explanation | 82.79 ± 1.17 | 80.47 ± 1.28 | 02.80% | 88.84 ± 0.34 | 87.32 ± 0.38 | 01.52% | 54.37 ± 5.51 | 47.19 ± 1.13 | 13.21% |
| Ensemble | 82.57 ± 1.48 | 80.67 ± 1.39 | 02.30% | 83.59 ± 0.81 | 82.55 ± 1.41 | 01.24% | 59.90 ± 2.93 | 54.33 ± 2.43 | 09.30% |
| Llama-FT | 78.54 ± 1.53 | 78.51 ± 0.88 | 0% | 77.40 ± 0.78 | 76.55 ± 0.85 | 1.10% | 52.46 ± 0.84 | 47.59 ± 1.71 | 09.28% |
|  | Wikics | History | Citeseer |  |  |  |  |  |  |
| Feature | Clean | 5% Attack | Gap | Clean | 5% Attack | Gap | Clean | 5% Attack | Gap |
| BOW | 74.92 ± 0.05 | 61.46 ± 0.32 | 17.95% | 51.25 ± 4.80 | 49.04 ± 5.80 | 04.31% | 70.74 ± 0.72 | 68.84 ± 0.82 | 02.69% |
| TFIDF | 75.96 ± 0.14 | 62.03 ± 0.31 | 18.34% | 58.38 ± 3.25 | 57.57 ± 1.69 | 01.39% | 73.02 ± 0.66 | 71.40 ± 0.78 | 02.22% |
| SBert | 75.89 ± 1.77 | 64.44 ± 2.33 | 15.09% | 65.59 ± 1.60 | 65.60 ± 1.48 | 0% | 74.94 ± 0.88 | 72.60 ± 0.81 | 03.14% |
| E5 | 76.71 ± 1.74 | 63.01 ± 2.13 | 17.86% | 66.96 ± 2.30 | 64.69 ± 2.10 | 03.39% | 75.14 ± 0.93 | 72.29 ± 1.27 | 03.79% |
| Llama | 79.72 ± 0.35 | 70.91 ± 1.02 | 11.05% | 66.73 ± 3.53 | 61.64 ± 3.21 | 07.63% | 67.48 ± 3.40 | 70.82 ± 1.07 | 04.95% |
| Angle-Llama | 73.33 ± 1.77 | 67.00 ± 3.62 | 08.63% | 66.52 ± 1.69 | 63.47 ± 3.14 | 04.59% | 71.71 ± 1.73 | 68.46 ± 4.92 | 04.53% |
| Llama-FT | 79.69 ± 0.39 | 70.11 ± 1.03 | 12.02% | 64.53 ± 2.02 | 62.99 ± 1.44 | 02.39% | 69.70 ± 2.18 | 68.62 ± 3.15 | 01.55% |

**Table 9: The performance of LLMs-as-Enhancers against 25% attacks on graph structures**
|  | Cora | Pudmed | Arxiv |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Feature | Clean | 25% Attack | Gap | Clean | 25% Attack | Gap | Clean | 25% Attack | Gap |
| BOW | 78.49 ± 1.13 | 60.72 ± 1.92 | 22.64% | 74.69 ± 2.07 | 62.83 ± 1.61 | 15.88% | 50.99 ± 2.15 | 18.95 ± 1.63 | 62.85% |
| TFIDF | 81.46 ± 1.21 | 67.53 ± 0.82 | 17.10% | 76.86 ± 1.34 | 66.18 ± 1.44 | 13.90% | 48.39 ± 1.15 | 24.36 ± 2.10 | 49.67% |
| SBert | 81.99 ± 0.76 | 72.14 ± 2.13 | 12.01% | 78.71 ± 1.17 | 69.24 ± 1.85 | 12.03% | 52.51 ± 0.87 | 29.24 ± 3.27 | 44.32% |
| E5 | 83.17 ± 0.73 | 69.84 ± 0.55 | 16.03% | 81.83 ± 1.16 | 71.06 ± 1.12 | 13.15% | 57.04 ± 1.77 | 25.24 ± 2.77 | 55.75% |
| Llama | 78.13 ± 1.07 | 69.99 ± 1.92 | 10.42% | 77.65 ± 0.72 | 74.62 ± 1.64 | 03.89% | 58.04 ± 1.79 | 31.67 ± 3.09 | 45.43% |
| Angle-Llama | 80.28 ± 1.42 | 76.75 ± 1.33 | 04.40% | 75.15 ± 2.50 | 72.72 ± 3.00 | 03.23% | 58.53 ± 1.56 | 31.73 ± 4.80 | 45.79% |
| Explanation | 82.79 ± 1.17 | 71.32 ± 2.28 | 13.85% | 88.84 ± 0.34 | 82.11 ± 0.72 | 07.57% | 54.37 ± 5.51 | 21.61 ± 3.10 | 60.25% |
| Ensemble | 82.57 ± 1.48 | 74.89 ± 0.93 | 09.30% | 83.59 ± 0.81 | 77.85 ± 0.50 | 06.87% | 59.90 ± 2.93 | 32.82 ± 3.19 | 45.21% |
| Llama-FT | 78.54 ± 1.53 | 70.72 ± 4.15 | 09.96% | 77.40 ± 0.78 | 75.63 ± 2.15 | 2.29% | 52.46 ± 0.84 | 29.19 ± 3.19 | 44.36% |
|  | Wikics | History | Citeseer |  |  |  |  |  |  |
| Feature | Clean | 25% Attack | Gap | Clean | 25% Attack | Gap | Clean | 25% Attack | Gap |
| BOW | 74.92 ± 0.05 | 45.34 ± 0.31 | 39.45% | 51.25 ± 4.80 | 38.93 ± 5.53 | 24.02% | 70.74 ± 0.72 | 61.76 ± 1.44 | 12.69% |
| TFIDF | 75.96 ± 0.14 | 45.74 ± 0.51 | 39.76% | 58.38 ± 3.25 | 22.66 ± 1.69 | 61.15% | 73.02 ± 0.66 | 64.36 ± 0.72 | 11.86% |
| SBert | 75.89 ± 1.77 | 50.19 ± 3.20 | 33.86% | 65.59 ± 1.60 | 56.34 ± 2.41 | 14.10% | 74.94 ± 0.88 | 64.51 ± 1.50 | 13.92% |
| E5 | 76.71 ± 1.74 | 46.38 ± 4.50 | 39.55% | 66.96 ± 2.30 | 55.37 ± 1.80 | 17.30% | 75.14 ± 0.93 | 64.38 ± 1.83 | 14.31% |
| Llama | 79.72 ± 0.35 | 59.23 ± 1.47 | 25.71% | 66.73 ± 3.53 | 54.54 ± 1.74 | 18.26% | 67.48 ± 3.40 | 64.69 ± 2.99 | 04.13% |
| Angle-Llama | 73.33 ± 1.77 | 56.75 ± 4.65 | 22.61% | 66.52 ± 1.69 | 52.75 ± 3.66 | 20.71% | 71.71 ± 1.73 | 68.61 ± 3.61 | 04.32% |
| Llama-FT | 79.69 ± 0.39 | 60.13 ± 0.31 | 24.56% | 64.53 ± 2.02 | 53.34 ± 2.47 | 17.36% | 69.70 ± 2.18 | 66.17 ± 3.62 | 05.07% |

## Appendix E Results of LLMs-as-Enhancers against attacks on texts

To address Q2, we perform a white-box evasion attack on LLMs-as-Enhancers, specifically targeting textual attributes. The results are shown in Table 10, which includes additional details on the standard deviation (std) compared to Table 2.

**Table 10: Attack success rate results of LLMs-as-Enhancers against 5% textual attack.**
|  | Cora | PubMed | Arxiv |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Features | MLP | GCN | MLP | GCN | MLP | GCN |
| BOW | 62.11±4.48 | 9.27±2.97 | 41.00±7.86 | 8.58±2.45 | 72.69±6.93 | 15.19±4.86 |
| Sbert | 73.18±2.02 | 14.76±2.48 | 45.36±3.88 | 9.32±3.89 | 82.69±10.88 | 11.17±1.39 |
| E5 | 65.29±6.42 | 10.53±1.86 | 35.62±5.64 | 8.76±2.59 | 81.92±8.76 | 15.22±4.18 |
| LLaMA | 56.49±7.34 | 12.58±0.85 | 19.66±4.89 | 4.95±2.59 | 67.87±13.10 | 6.05±2.56 |
| LLaMA-FT | 40.10±12.16 | 4.37±1.94 | 16.69±5.57 | 3.27±1.65 | 67.71±9.83 | 6.08±2.00 |
|  | Wikics | History | Citeseer |  |  |  |
| Features | MLP | GCN | MLP | GCN | MLP | GCN |
| BOW | 67.85±4.88 | 3.82±1.46 | 74.53±10.13 | 18.40±6.00 | 68.80±6.99 | 18.98±4.67 |
| Sbert | 62.12±4.32 | 9.08±2.34 | 76.73±8.79 | 13.41±3.86 | 66.33±6.09 | 16.93±3.31 |
| E5 | 65.99±5.05 | 6.64±2.47 | 61.51±15.43 | 6.54±1.04 | 57.10±6.25 | 14.69±3.73 |
| LLaMA | 22.17±4.96 | 4.17±0.80 | 65.07±9.50 | 15.92±8.47 | 46.81±14.07 | 13.77±5.34 |
| LLaMA-FT | 30.00±2.07 | 2.98±1.83 | 56.98±14.55 | 6.91±3.64 | 38.46±11.19 | 6.58±2.97 |

## Appendix F Results of LLMs-as-Predictors against attacks on graph structures

To address Q3, we explore the robustness of predictors against graph structure attacks, using GPT-3.5 as the predictor. The results are presented in Figure 5.
We evaluate the model under various few-shot settings and the zero-shot setting with the summary of two-hop neighbors. The experiment shows that GPT-3.5 demonstrates the strongest robustness in the zero-shot setting, with performance barely dropping even under significant perturbations. This resilience is likely due to the neighbor sampling and summarizing processes, which mitigate the noise introduced by structural changes. Although GAT also uses the attacked graph structure migrated from GCN, its accuracies decline much faster compared to GPT-3.5.

Figure: Figure 5: The performance of LLMs-as-Predictors against structural attacks, evaluated by accuracy.
Refer to caption: x12.png

## Appendix G Results of LLMs-as-Predictors against attacks on texts

To address Q4, we explore the performance of GPT-3.5 with perturbed texts as inputs. The results are presented in Table 11, showing that GPT-3.5 demonstrates strong robustness compared to MLP but does not surpass GCN and GAT in all few-shot settings. On the other hand, GAT exhibits the strongest robustness, followed by GCN. However, on the Cora and Pubmed datasets, the Attack Success Rate of GPT-3.5 is comparable to GCN, ranging from 0.55% to 1.70%.

**Table 11: The performance of the predictor GPT-3.5 against 5% textual attacks. The bold font is used to highlight the lowest Attack Success Rate.**
| Dataset | Cora | Pubmed | Arxiv |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Model | Clean ACC | ASR | Clean ACC | ASR | Clean ACC | ASR |
| MLP | 69.25 | 35.97 | 78.00 | 14.74 | 69.50 | 30.10 |
| GCN | 81.00 | 4.30 | 80.00 | 3.20 | 69.25 | 5.42 |
| GAT | 81.75 | 2.99 | 80.75 | 2.23 | 69.00 | 3.97 |
| GPT-3.5 (2-hop zero-shot) | 74.25 | 10.03 | 90.00 | 5.53 | 55.75 | 13.33 |
| GPT-3.5 (2-hop one-shot) | 75.75 | 5.59 | 84.00 | 3.75 | 54.75 | 18.65 |
| GPT-3.5 (2-hop three-shot) | 76.50 | 5.59 | 81.00 | 5.29 | 51.00 | 19.10 |
| GPT-3.5 (2-hop five-shot) | 76.25 | 6.72 | 82.50 | 6.07 | 52.00 | 21.24 |

## Appendix H Analysis for Textual Attack

From the experiments on LLMs-as-Enhancers against textual attacks in Section 4.2, we observe that LLaMA performs significantly better than other smaller models, with fine-tuned LLaMA demonstrating the best robustness. Additionally, GCN shows much stronger robustness compared to MLP.

To explore the reasons behind these observations, we analyze the models from feature and structure perspectives. From the feature perspective, we use SBert and GCN as the victim model and use Latent Dirichlet Allocation to generate the distribution of three themes for each node. Then, we acquire the mean value for each theme. As shown in Fig. 7, the theme distributions for the successfully and failed attacked nodes are similar, indicating that there is not a strong relation between text content and robustness. From the structure perspective, we examine the relationship between the centrality of nodes and the attack success rate. As shown in Fig. 6, we use SBert and GCN as the victim models and visualize the degree distributions of successfully attacked nodes versus failed attacked nodes.

It is evident that successfully attacked nodes often have smaller degrees, indicating that nodes with less structural information in the graph are more vulnerable to textual attacks. Furthermore, similar patterns are observed with eigenvector centrality and PageRank values.

Figure: Figure 6: Centrality distributions of the node being attacked successfully and unsuccessfully. We use Sentence-Bert as the victim model and gather all attacked results from the PubMed dataset.
Refer to caption: x13.png

Figure: Figure 7: Theme distributions of the node being attacked successfully and unsuccessfully.
Refer to caption: x14.png

## Appendix I Combining Textual and Structural Attack

To enhance attack capabilities, maybe a combined framework that perturbs both text attributes and graph structure is needed. In this study, we provide some preliminary results. First of all, we design a simple strategy that combines the structural attack and textual attack directly, further improving the attack ability. The results are reported in Table 12. In addition, we design various combination strategies for modifying the textual attack. These include prioritizing the attack on text attributes of small-degree nodes, targeting large-degree nodes first, and attacking text attributes within the same cluster. Based on the results in Table 13, we find that prioritizing attacks on the text attributes of small-degree nodes is more effective. This is merely a preliminary attempt, and we hope it will inspire more in-depth research.

**Table 12: Result of combined attack using SemAttack and PRBCD. Here we perform 5% perturbation on both text features and the graph structure.**
|  | Cora | Citeseer | PubMed | WikiCS |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Feature \ Ptb. | 5% edge | 5% edge & node | 5%edge | 5% edge & node | 5%edge | 5% edge & node | 5%edge | 5% edge & node |
| BOW | 78.40±1.67 | 78.23±1.70 | 69.64±0.98 | 69.58±1.18 | 74.18±0.76 | 74.01±0.78 | 66.38±4.65 | 66.07±4.63 |
| Sbert | 79.93±0.49 | 79.75±0.54 | 74.16±1.23 | 74.15±1.31 | 75.92±1.40 | 75.87±0.90 | 66.94±2.54 | 66.73±2.53 |
| LLaMA | 80.03±0.50 | 79.14±1.17 | 71.70±1.50 | 70.68±1.70 | 75.78±1.33 | 74.95±0.98 | 66.60±5.54 | 66.41±5.64 |

**Table 13: Results of 5% edge and 15% perturbations under three different sampling strategies on Cora. Small degree first and Large degree first means we sample nodes with the smallest or largest degrees respectively. Clustering refers to sampling nodes that are in the same cluster.**
| Cora Dataset | 5% edge & 15% node |  |
| --- | --- | --- |
| sampling\perturbation | ASR | ACC |
| Small degree first | 17.20±3.33 | 79.37±0.53 |
| Large degree first | 3.27±1.15 | 79.44±0.85 |
| Clustering | 4.40±1.69 | 79.53±0.61 |

## Appendix J Broader Impact

Graph machine learning methods with Large Language Models (Graph-LLMs) have reported promising performance, covering a wide range of applications. We are initiating this benchmark to call more researchers’ attention to whether Graph-LLMs are robust against graph adversarial attacks. The proposed benchmark can significantly promote the development of robustness in Graph-LLMs. Our benchmark provides a comprehensive evaluation, including LLM-as-Enhancers and LLM-as-Predictors, and consider both structural attacks and textual attacks. However, there are still many research gaps to be filled. Firstly, we have only explored node classification tasks and have yet to explore link prediction and graph classification tasks. Secondly, we have focused only on evasion attacks and have not addressed poisoning attacks. These limitations will be addressed in our future work.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: see Weaknesses
soundness: 2
strengths: 1. It seems to be the first work discussing the robustness of Graph-LLMs against adversarial attacks.
2. Extensive experiments are conducted and open-source benchmark library is released to promote reproducibility and enable others to build upon their findings.
summary: The paper investigates the robustness of graph machine learning methods with LLMs (Graph-LLMs) against adversarial attacks. More specifically, the investigation is mainly focused on adversarial attacks of structural and textual perturbations on two kinds of GraphLLM Architectures: LLMs-as-Enhancers and LLMs-as-Predictors. The study reveals some interesting observations about the robustness of Graph-LLMs, and an open-source benchmark library is further released to facilitate further research.
weaknesses: 1. As a work aimed to provide a comprehensive benchmark of the robustness of Graph-LLMs against adversarial attacks, the attack evaluated is limited. For both structural attacks and textual attacks, the paper would benefit from a broader range of attack scenarios including: (1) Different knowledge of adversary: white-box, black-box, gray-box, (2) Variety in manipulation techniques: Including adding, deleting, or rewiring of edges, nodes, or texts. For textual attacks, the paper currently only evaluates SemAttack. This leads to the current benchmark cannot provide a thorough understanding of the Graph-LLMs' robustness.
2. The performance metrics include ACC and GAP. It's important to consider which of these better reflects robustness. For example, in Table 1, although LLaMA-FT has a good GAP, the Explanation model achieves the highest ACC under attack, suggesting that this model retains better performance under adversarial conditions, also indicating robustness.
3. I observed an inconsistency in the evaluation metrics used for different types of attacks, with Attack Success Rate being used for textual attacks and GAP for structural attacks. This disparity in metrics may compromise the ability to perform a holistic evaluation of the models' robustness. Adopting a unified metric or a consistent set of metrics across different attack types would facilitate a clearer comparative analysis of results.
4. The paper provides an insightful examination of vulnerabilities in Graph-LLMs; however, it stops short of testing the effectiveness of existing defense strategies against these identified attacks. Including such an analysis would greatly enrich the paper's practical contributions. I recommend that the authors conduct additional experiments to test various defense mechanisms against both textual and structural perturbations.

Minor issues: There is a minor ambiguity in the text, specifically in sentences 201-202 where the term "transferring" is used. It is unclear from the context what exactly is meant to be transferred.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: How does the robustness of GraphLLMs vary across different types of graphs, such as social networks and biological networks?
soundness: 2
strengths: This paper conducts comprehensive evaluation of GraphLLMs robustness under both structural and textual adversarial attacks. They highlight the improved robustness of GraphLLMs as both Enhancers and Predictors. They provide observations and analyses on different factors impacting the model robustness like node degrees.
summary: This paper mainly focuses on evaluating the robustness of GraphLLMs against graph structural and textual perturbation in the term of both LLMs-as-Enhancers and LLMs-as-Predictors.
weaknesses: 1. This paper mainly focuses on robustness evaluation and thus lacks novelty. Developing new attack methods tailored for GraphLLMs or proposing novel defense strategies based on the robustness findings might be interesting.
2. While they evaluate both structural and textual attacks, it lack the full scope of adversarial strategies, like joint structural-textual attack or node injection.
3. The types of evaluation datasets are limited. More datasets can be used like molecular dataset, bioinformatic dataset, social network dataset.
4. This paper only include results of simple architectures like Vanilla GNN and MLP. More complicated models should be evaluated.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. Providing more context for the structural attacks PGD and PRBCD could improve readability.

2. Line 296: correct ”Explanation” 

3. Could the observations/conclusions be applied to broaden the architectural choices of LLMs (e.g., Llama, Qwen, Mistral of various sizes) and graph models (e.g., GCN, GIN, GraphSAGE, GAT)?
soundness: 2
strengths: Originality: The paper develops a new benchmark for evaluating the robustness of graph-LLM models in adversarial attacks.

Quality: The paper provides comprehensive results for the benchmark with reasonable outcomes.

Clarity: The paper is easy to understand.

Significance: The paper demonstrates a promising future direction in combining LLMs with graph models.
summary: The paper benchmarks the robustness of large language model (LLM)-assisted graph learning. The authors suggest using LLMs both as enhancers and as predictors against structural and textual attacks, finding that LLM-assisted graph learning methods demonstrate better robustness. The topic is interesting and provides insights: the rich knowledge contained in LLMs may help mitigate noise in graph structures and node features, showing promise for developing multimodal graph and language models.
weaknesses: 1. The authors' comprehensive studies and experiments are much appreciated. However, a concern regarding the contributions is that these observations may not be surprising and could be somewhat trivial, as they essentially confirm that LLMs, with their rich knowledge, make graph neural networks (GNNs) more robust to both structural and feature perturbations through improving node features.

2. More actionable observations or conclusions would improve the quality of the paper. For example, in line 538: "This phenomenon may inspire us to design better Graph-LLMs for text attribute encoding." What specific inspiration can we draw from this?

3. It is unclear to what extent (e.g., types of graphs) these observations can be applied, such as to citation graphs, protein-protein interaction graphs, molecular graphs, or social networks.

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. I'm curious about how to formally define a reasonable attack setting in the era of LLMs. 
In the benchmark, multiple settings and budgets are included.
Is it possible to set up a universal attack setting? 

2. Under the setting in Question 1, both enhancers and predictors can be utilized by the defender, which facilitates further comparison between them. Is it possible to set up a complete benchmark where both enhancer and predictor can be compared together?
Currently, the benchmark includes too many detailed parts, which might hinder future studies following up.
soundness: 3
strengths: 1. This paper studied an under-explored area, the robustness of Graphs with Large Language Models. The paper presents many interesting insights that are motivating.

2. The experiments are comprehensive, covering both text and structural attacks.
summary: The paper explores the robustness of Large Language Models (LLMs) when applied to graph adversarial attacks. 
It evaluates the ability of LLMs-as-Enhancers and LLMs-as-Predictors to withstand structural and textual perturbations. 
Through extensive experiments, the study finds that LLM-based approaches generally outperform traditional shallow models in robustness against both types of attacks. 
The research introduces novel benchmarks for evaluating Graph-LLMs’ robustness, provides insights into the underlying factors contributing to LLMs’ superior performance, and offers an open-source benchmark library to promote further research.
weaknesses: 1. The paper seems more like a benchmark paper but is submitted under the primary area: *learning on graphs and other geometries & topologies*, not *dataset and benchmark*.
If not considered as a benchmark, the novelty of the paper needs further consideration, as the methodology is mostly a combination of previous works.
While still valuable, the results are mostly simple empirical discoveries.

2. It would be better if more fine-tuned models could be included rather than only InstructGLM. I think it is interesting to see more results for the predictors, given that enhancers share mechanisms similar to traditional GNNs.

3. Paper [1] is a very relevant paper that 1) studies textual-level graph adversarial attacks and 2) identifies the robustness of predictors. It would be better to include it for discussion. 

4. The presentation could be improved in some areas. For example, the \citep and \citet commands seem to be misused by the authors.


[1] Lei, Runlin, et al. "Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Level." In Neurips 2024.

Meta-review: This paper focuses on providing empirical evaluation of LLMs over graphs in terms of its impact on the robustness. The reviewers are concerned about the limited attacks studied in the experiments, the inconsistent metrics used across different experiments, as well as the limited datasets used. Since the main focus of this paper is extensive empirical evaluation, the above comments are important to address. There is no rebuttal provided. 

I urge the authors to take the above suggestions into account when preparing the next iteration of the paper.

Decision: Reject